2024-06-10T00:08:42.491 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-10T00:08:42.491 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} --name teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323 --verbose -- /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323/orig.config.yaml 2024-06-10T00:08:42.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-10T00:08:42.663 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7747323' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 178 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 130/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6ZS5R7h8TH3FBS4zJzza9VBbTYEep8kERIDvppufoyjjBqDonr7XT+8dnMDHS65oISyfgnFfsKU912DgFzrTM= smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBy/q/ke4ikd30k+rm1NL+SMnmQhlScRv6bLuHi24HK+gfMAQuVboSUNP6n83QSD/6j7jdX3CxYxrbp1CFLl8QY= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOrKDeredohu381vmNPHC8aoyudyMrqqQH8aQQ/BmEfhJl0Z5mLt0Uv+RxYLFGr6/PrLRXDsGVT2IpVRejevafU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-09_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-10T00:08:42.664 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa; will attempt to use it 2024-06-10T00:08:42.665 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks 2024-06-10T00:08:42.665 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-10T00:08:42.667 INFO:teuthology.task.internal:Checking packages... 2024-06-10T00:08:42.698 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-10T00:08:42.698 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-10T00:08:42.698 INFO:teuthology.packaging:ref: None 2024-06-10T00:08:42.699 INFO:teuthology.packaging:tag: None 2024-06-10T00:08:42.699 INFO:teuthology.packaging:branch: squid 2024-06-10T00:08:42.699 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:08:42.699 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-10T00:08:42.948 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2970.g7b7a5d37 2024-06-10T00:08:42.950 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-10T00:08:42.955 INFO:teuthology.task.internal:no buildpackages task found 2024-06-10T00:08:42.955 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-10T00:08:42.965 INFO:teuthology.task.internal:Saving configuration 2024-06-10T00:08:42.988 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-10T00:08:43.000 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-10T00:08:43.023 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-10 00:03:46.847489', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6ZS5R7h8TH3FBS4zJzza9VBbTYEep8kERIDvppufoyjjBqDonr7XT+8dnMDHS65oISyfgnFfsKU912DgFzrTM='} 2024-06-10T00:08:43.041 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-10 00:03:46.845282', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBy/q/ke4ikd30k+rm1NL+SMnmQhlScRv6bLuHi24HK+gfMAQuVboSUNP6n83QSD/6j7jdX3CxYxrbp1CFLl8QY='} 2024-06-10T00:08:43.066 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-10 00:03:46.846508', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOrKDeredohu381vmNPHC8aoyudyMrqqQH8aQQ/BmEfhJl0Z5mLt0Uv+RxYLFGr6/PrLRXDsGVT2IpVRejevafU='} 2024-06-10T00:08:43.066 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-10T00:08:43.072 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-10T00:08:43.072 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-10T00:08:43.072 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-10T00:08:43.072 INFO:teuthology.run_tasks:Running task console_log... 2024-06-10T00:08:43.160 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f77e30f1630>, signals=[15]) 2024-06-10T00:08:43.160 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-10T00:08:43.165 INFO:teuthology.task.internal:Opening connections... 2024-06-10T00:08:43.165 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-06-10T00:08:43.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:08:43.242 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-06-10T00:08:43.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:08:43.317 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-06-10T00:08:43.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:08:43.391 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-10T00:08:43.398 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-06-10T00:08:43.417 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-06-10T00:08:43.417 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-06-10T00:08:43.475 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2024-06-10T00:08:43.475 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2024-06-10T00:08:43.475 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2024-06-10T00:08:43.475 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2024-06-10T00:08:43.475 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2024-06-10T00:08:43.475 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2024-06-10T00:08:43.475 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-10T00:08:43.475 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2024-06-10T00:08:43.475 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2024-06-10T00:08:43.475 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-10T00:08:43.475 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2024-06-10T00:08:43.476 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-10T00:08:43.476 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-10T00:08:43.476 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-10T00:08:43.476 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-06-10T00:08:43.499 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-06-10T00:08:43.516 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-06-10T00:08:43.517 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-06-10T00:08:43.573 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2024-06-10T00:08:43.573 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="9" 2024-06-10T00:08:43.573 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2024-06-10T00:08:43.573 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2024-06-10T00:08:43.573 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="9" 2024-06-10T00:08:43.573 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el9" 2024-06-10T00:08:43.573 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-10T00:08:43.573 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-06-10T00:08:43.573 INFO:teuthology.orchestra.run.smithi105.stdout:LOGO="fedora-logo-icon" 2024-06-10T00:08:43.573 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-10T00:08:43.574 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2024-06-10T00:08:43.574 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-10T00:08:43.574 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-10T00:08:43.574 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-10T00:08:43.574 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-06-10T00:08:43.590 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-06-10T00:08:43.608 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-06-10T00:08:43.608 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-06-10T00:08:43.665 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-06-10T00:08:43.665 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-06-10T00:08:43.665 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-06-10T00:08:43.665 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-06-10T00:08:43.665 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-06-10T00:08:43.666 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-06-10T00:08:43.666 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-10T00:08:43.666 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-06-10T00:08:43.666 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-06-10T00:08:43.666 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-10T00:08:43.666 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-06-10T00:08:43.666 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-10T00:08:43.666 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-10T00:08:43.666 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-10T00:08:43.667 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-06-10T00:08:43.707 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-10T00:08:43.727 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-10T00:08:43.742 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-10T00:08:43.742 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-06-10T00:08:43.745 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-06-10T00:08:43.747 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-06-10T00:08:43.763 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-10T00:08:43.774 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-10T00:08:43.774 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-06-10T00:08:43.802 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-06-10T00:08:43.804 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-06-10T00:08:43.876 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-10T00:08:43.950 INFO:teuthology.run_tasks:Running task kernel... 2024-06-10T00:08:43.980 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-06-10T00:08:43.980 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-06-10T00:08:43.980 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-06-10T00:08:43.980 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-10T00:08:43.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:08:43.998 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-06-10T00:08:44.056 INFO:teuthology.orchestra.run.smithi062.stdout:5.14.0-437.el9.x86_64 2024-06-10T00:08:44.056 INFO:teuthology.task.kernel:Running kernel on smithi062: 5.14.0-437.el9.x86_64 2024-06-10T00:08:44.056 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum install -y kernel 2024-06-10T00:08:47.669 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.1 MB 00:01 2024-06-10T00:08:49.661 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-06-10T00:08:55.278 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-06-10T00:08:57.286 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 17 kB 00:00 2024-06-10T00:08:58.247 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-06-10T00:09:03.812 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-06-10T00:09:05.909 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-10T00:09:05.910 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-10T00:09:06.007 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-06-10T00:09:06.024 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-06-10T00:09:06.024 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2024-06-10T00:09:06.025 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-06-10T00:09:06.025 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-06-10T00:09:06.025 INFO:teuthology.orchestra.run.smithi062.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-10T00:09:06.025 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-06-10T00:09:06.025 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-10T00:09:06.025 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-10T00:09:06.025 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-10T00:09:06.025 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:09:06.025 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-06-10T00:09:06.025 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-06-10T00:09:06.025 INFO:teuthology.orchestra.run.smithi062.stdout:Install 4 Packages 2024-06-10T00:09:06.025 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:09:06.026 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 80 M 2024-06-10T00:09:06.026 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 124 M 2024-06-10T00:09:06.026 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-06-10T00:09:06.232 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:09:06.232 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:09:06.236 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:09:06.555 INFO:teuthology.orchestra.run.smithi062.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 2.2 MB/s | 893 kB 00:00 2024-06-10T00:09:07.263 INFO:teuthology.orchestra.run.smithi062.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-06-10T00:09:07.813 INFO:teuthology.orchestra.run.smithi062.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 23 MB/s | 29 MB 00:01 2024-06-10T00:09:08.130 INFO:teuthology.orchestra.run.smithi062.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 17 MB/s | 35 MB 00:01 2024-06-10T00:09:08.133 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:09:08.134 INFO:teuthology.orchestra.run.smithi062.stdout:Total 38 MB/s | 80 MB 00:02 2024-06-10T00:09:08.861 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-06-10T00:09:09.275 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-06-10T00:09:09.275 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-06-10T00:09:10.243 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-06-10T00:09:10.244 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-06-10T00:09:12.830 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-06-10T00:09:13.478 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-10T00:09:13.571 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-10T00:09:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-10T00:09:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-10T00:09:21.617 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-10T00:09:21.669 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:09:26.781 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:09:57.637 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:09:57.679 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:09:58.048 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:09:58.049 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-10T00:09:58.049 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-10T00:09:58.049 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-10T00:09:58.880 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:09:58.880 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:09:58.880 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-06-10T00:09:58.880 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-10T00:09:58.880 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-10T00:09:58.881 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-10T00:09:58.881 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-10T00:09:58.881 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:09:58.881 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-06-10T00:09:59.212 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel | sort -rV | head -n 1 2024-06-10T00:09:59.292 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-10T00:09:59.293 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-10T00:09:59.293 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-10T00:09:59.293 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-10T00:09:59.293 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-10T00:09:59.293 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-06-10T00:09:59.293 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi062.front.sepia.ceph.com, path=None, version=distro) 2024-06-10T00:09:59.293 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel | sort -rV | head -n 1 2024-06-10T00:09:59.348 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-10T00:09:59.348 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-10T00:09:59.348 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -qi grub2-tools 2024-06-10T00:09:59.390 INFO:teuthology.orchestra.run.smithi062.stdout:Name : grub2-tools 2024-06-10T00:09:59.390 INFO:teuthology.orchestra.run.smithi062.stdout:Epoch : 1 2024-06-10T00:09:59.390 INFO:teuthology.orchestra.run.smithi062.stdout:Version : 2.06 2024-06-10T00:09:59.391 INFO:teuthology.orchestra.run.smithi062.stdout:Release : 68.el9 2024-06-10T00:09:59.391 INFO:teuthology.orchestra.run.smithi062.stdout:Architecture: x86_64 2024-06-10T00:09:59.391 INFO:teuthology.orchestra.run.smithi062.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-10T00:09:59.391 INFO:teuthology.orchestra.run.smithi062.stdout:Group : Unspecified 2024-06-10T00:09:59.391 INFO:teuthology.orchestra.run.smithi062.stdout:Size : 8264254 2024-06-10T00:09:59.391 INFO:teuthology.orchestra.run.smithi062.stdout:License : GPLv3+ 2024-06-10T00:09:59.391 INFO:teuthology.orchestra.run.smithi062.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-10T00:09:59.392 INFO:teuthology.orchestra.run.smithi062.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-10T00:09:59.392 INFO:teuthology.orchestra.run.smithi062.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-10T00:09:59.392 INFO:teuthology.orchestra.run.smithi062.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-10T00:09:59.392 INFO:teuthology.orchestra.run.smithi062.stdout:Packager : builder@centos.org 2024-06-10T00:09:59.392 INFO:teuthology.orchestra.run.smithi062.stdout:Vendor : CentOS 2024-06-10T00:09:59.392 INFO:teuthology.orchestra.run.smithi062.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-10T00:09:59.393 INFO:teuthology.orchestra.run.smithi062.stdout:Summary : Support tools for GRUB. 2024-06-10T00:09:59.393 INFO:teuthology.orchestra.run.smithi062.stdout:Description : 2024-06-10T00:09:59.393 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:09:59.393 INFO:teuthology.orchestra.run.smithi062.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-10T00:09:59.393 INFO:teuthology.orchestra.run.smithi062.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-10T00:09:59.393 INFO:teuthology.orchestra.run.smithi062.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-10T00:09:59.394 INFO:teuthology.orchestra.run.smithi062.stdout:hardware devices. 2024-06-10T00:09:59.394 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:09:59.394 INFO:teuthology.orchestra.run.smithi062.stdout:This subpackage provides tools for support of all platforms. 2024-06-10T00:09:59.396 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-10T00:09:59.396 INFO:teuthology.task.kernel:Updating grub on smithi062 to boot 5.14.0-452.el9.x86_64 2024-06-10T00:09:59.397 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-10T00:10:00.025 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2024-06-10T00:10:01.518 INFO:teuthology.orchestra.run.smithi062.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-10T00:10:01.540 INFO:teuthology.orchestra.run.smithi062.stderr:done 2024-06-10T00:10:01.541 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-06-10T00:10:01.557 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.6zaS3Oz8jn 2024-06-10T00:10:01.558 DEBUG:teuthology.orchestra.run.smithi062:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6zaS3Oz8jn 2024-06-10T00:10:01.623 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.6zaS3Oz8jn 2024-06-10T00:10:01.765 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.6zaS3Oz8jn is 6KB 2024-06-10T00:10:01.785 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.6zaS3Oz8jn 2024-06-10T00:10:01.800 DEBUG:teuthology.orchestra.run.smithi062:> sudo /bin/ls /boot/loader/entries || true 2024-06-10T00:10:01.862 INFO:teuthology.orchestra.run.smithi062.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-10T00:10:01.862 INFO:teuthology.orchestra.run.smithi062.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-10T00:10:01.862 INFO:teuthology.orchestra.run.smithi062.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-10T00:10:01.862 INFO:teuthology.orchestra.run.smithi062.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-10T00:10:01.863 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-10T00:10:02.019 DEBUG:teuthology.orchestra.run.smithi062:> sudo shutdown -r now 2024-06-10T00:10:32.022 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-06-10T00:10:32.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-06-10T00:10:32.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:11:06.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-06-10T00:11:15.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-06-10T00:11:15.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:11:34.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-06-10T00:11:46.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-06-10T00:11:46.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:11:49.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-06-10T00:12:04.119 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-06-10T00:12:04.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:12:07.198 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-06-10T00:12:25.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-06-10T00:12:25.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:12:25.525 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-06-10T00:12:25.887 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-06-10T00:12:25.887 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-452.el9.x86_64"... 2024-06-10T00:12:25.888 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-06-10T00:12:25.946 INFO:teuthology.orchestra.run.smithi062.stdout:5.14.0-452.el9.x86_64 2024-06-10T00:12:25.946 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-10T00:12:25.946 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-10T00:12:25.949 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-10T00:12:26.950 DEBUG:teuthology.parallel:result is None 2024-06-10T00:12:26.950 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-10T00:12:26.997 INFO:teuthology.task.internal:Creating test directory... 2024-06-10T00:12:26.998 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-10T00:12:27.000 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-10T00:12:27.002 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-10T00:12:27.020 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-10T00:12:27.029 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-10T00:12:27.037 INFO:teuthology.task.internal:Creating archive directory... 2024-06-10T00:12:27.037 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-10T00:12:27.059 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-10T00:12:27.062 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-10T00:12:27.190 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-10T00:12:27.236 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-10T00:12:27.237 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-10T00:12:27.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:12:27.257 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-10T00:12:27.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:12:27.274 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-10T00:12:27.277 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-10T00:12:27.301 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-10T00:12:27.340 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:12:27.748 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:12:27.748 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:12:27.748 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:12:27.748 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:12:27.748 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:12:27.749 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-10T00:12:27.766 INFO:teuthology.task.internal:Configuring sudo... 2024-06-10T00:12:27.766 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-10T00:12:27.769 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-10T00:12:27.771 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-10T00:12:27.800 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-10T00:12:27.811 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-10T00:12:27.812 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-10T00:12:27.841 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-10T00:12:27.844 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-10T00:12:27.861 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:12:27.941 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:12:27.999 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:12:28.090 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:12:28.127 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:12:28.128 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-10T00:12:28.194 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:12:28.241 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:12:28.296 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:12:28.380 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:12:28.421 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-06-10T00:12:28.421 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-10T00:12:28.485 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:12:28.528 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:12:28.579 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:12:28.665 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:12:28.708 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-06-10T00:12:28.709 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-10T00:12:28.776 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-06-10T00:12:28.779 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-06-10T00:12:28.781 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-06-10T00:12:28.818 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T00:12:28.820 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T00:12:28.864 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T00:12:29.322 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-10T00:12:29.369 INFO:teuthology.task.internal:Starting timer... 2024-06-10T00:12:29.369 INFO:teuthology.run_tasks:Running task pcp... 2024-06-10T00:12:29.385 INFO:teuthology.run_tasks:Running task selinux... 2024-06-10T00:12:29.395 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-10T00:12:29.395 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2024-06-10T00:12:29.447 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2024-06-10T00:12:29.449 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2024-06-10T00:12:29.525 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2024-06-10T00:12:29.527 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-06-10T00:12:29.606 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-06-10T00:12:29.608 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-10T00:12:29.608 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2024-06-10T00:12:29.632 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2024-06-10T00:12:29.632 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2024-06-10T00:12:29.659 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2024-06-10T00:12:29.659 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-06-10T00:12:29.686 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-06-10T00:12:29.686 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi105.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive'} 2024-06-10T00:12:29.687 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T00:12:29.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:12:29.717 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T00:12:29.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:12:29.745 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T00:12:29.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:12:29.773 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-10T00:12:29.773 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2024-06-10T00:12:29.802 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2024-06-10T00:12:29.830 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-06-10T00:12:29.859 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-10T00:12:29.961 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-10T00:12:29.963 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-10T00:12:30.074 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-10T00:12:30.084 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-10T00:12:30.085 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi062.front.sepia.ceph.com,smithi105.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-10T00:17:19.349 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi105.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2024-06-10T00:17:19.351 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-06-10T00:17:19.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:17:19.428 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-06-10T00:17:19.505 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-06-10T00:17:19.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-06-10T00:17:19.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:17:19.580 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-06-10T00:17:19.661 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-06-10T00:17:19.661 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-06-10T00:17:19.662 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:17:19.737 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-06-10T00:17:19.776 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-06-10T00:17:19.777 INFO:teuthology.run_tasks:Running task clock... 2024-06-10T00:17:19.789 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-10T00:17:19.789 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-10T00:17:19.790 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T00:17:19.792 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-10T00:17:19.793 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T00:17:19.794 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-10T00:17:19.795 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T00:17:19.819 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-10T00:17:19.822 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-10T00:17:19.832 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-10T00:17:19.837 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-10T00:17:19.843 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-10T00:17:19.855 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2024-06-10T00:17:19.857 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-10T00:17:19.862 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2024-06-10T00:17:19.865 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2024-06-10T00:17:19.874 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2024-06-10T00:17:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-10T00:17:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-06-10T00:17:19.888 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-10T00:17:19.889 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-06-10T00:17:19.890 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-10T00:17:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-10T00:17:19.903 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-10T00:17:19.915 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-10T00:17:19.948 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-06-10T00:17:19.951 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T00:17:19.952 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-06-10T00:17:19.952 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:17:19.952 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:17:19.952 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:17:19.952 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:17:19.969 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-06-10T00:17:19.971 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-06-10T00:17:19.972 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T00:17:19.973 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-06-10T00:17:19.973 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:17:19.973 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:17:19.973 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:17:19.973 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:17:19.974 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T00:17:19.974 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-06-10T00:17:19.974 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:17:19.974 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:17:19.974 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:17:19.975 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:17:19.975 INFO:teuthology.run_tasks:Running task install... 2024-06-10T00:17:19.985 DEBUG:teuthology.task.install:project ceph 2024-06-10T00:17:19.985 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-10T00:17:19.985 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-10T00:17:19.985 INFO:teuthology.task.install:Using flavor: default 2024-06-10T00:17:19.991 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-10T00:17:19.991 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-10T00:17:19.991 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-10T00:17:19.992 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:17:19.993 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-10T00:17:19.994 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:17:19.995 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-10T00:17:19.995 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:17:20.210 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-10T00:17:20.210 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-10T00:17:20.228 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-10T00:17:20.228 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-10T00:17:20.229 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-10T00:17:20.229 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-10T00:17:20.322 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-10T00:17:20.322 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:17:20.322 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-10T00:17:20.336 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-10T00:17:20.336 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-06-10T00:17:20.336 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-10T00:17:20.349 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-10T00:17:20.350 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-10T00:17:20.360 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-10T00:17:20.361 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-10T00:17:20.396 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-10T00:17:20.396 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-06-10T00:17:20.396 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-10T00:17:20.417 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-10T00:17:20.420 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-10T00:17:20.420 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-10T00:17:20.425 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-10T00:17:20.484 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-10T00:17:20.493 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-10T00:17:20.498 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-10T00:17:20.557 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-10T00:17:20.558 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2024-06-10T00:17:20.560 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2024-06-10T00:17:20.561 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-06-10T00:17:20.563 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-06-10T00:17:20.619 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2024-06-10T00:17:20.621 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2024-06-10T00:17:20.790 INFO:teuthology.orchestra.run.smithi138.stdout:45 files removed 2024-06-10T00:17:20.792 INFO:teuthology.orchestra.run.smithi062.stdout:45 files removed 2024-06-10T00:17:20.819 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-10T00:17:20.843 INFO:teuthology.orchestra.run.smithi105.stdout:45 files removed 2024-06-10T00:17:20.870 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-10T00:17:20.898 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-10T00:17:21.528 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 210 kB/s | 83 kB 00:00 2024-06-10T00:17:21.577 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 177 kB/s | 83 kB 00:00 2024-06-10T00:17:21.628 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 177 kB/s | 83 kB 00:00 2024-06-10T00:17:21.912 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 34 kB/s | 11 kB 00:00 2024-06-10T00:17:22.010 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-06-10T00:17:22.054 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-06-10T00:17:22.346 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-06-10T00:17:22.362 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-06-10T00:17:22.436 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-06-10T00:17:23.173 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.1 MB 00:00 2024-06-10T00:17:23.264 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.1 MB 00:00 2024-06-10T00:17:25.438 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.1 MB 00:02 2024-06-10T00:17:28.092 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 5.0 MB/s | 19 MB 00:03 2024-06-10T00:17:29.233 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 19 MB 00:04 2024-06-10T00:17:29.964 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 19 MB 00:03 2024-06-10T00:17:33.386 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.2 MB 00:01 2024-06-10T00:17:33.441 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.2 MB 00:00 2024-06-10T00:17:34.966 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 17 kB 00:00 2024-06-10T00:17:35.281 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.2 MB 00:01 2024-06-10T00:17:35.750 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 17 kB 00:01 2024-06-10T00:17:36.186 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-06-10T00:17:36.978 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-06-10T00:17:36.989 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 17 kB 00:00 2024-06-10T00:17:37.942 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-06-10T00:17:41.609 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 5.1 kB/s | 1.7 kB 00:00 2024-06-10T00:17:42.125 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-06-10T00:17:43.207 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-06-10T00:17:43.885 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:17:43.886 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:17:43.896 INFO:teuthology.orchestra.run.smithi138.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-10T00:17:43.897 INFO:teuthology.orchestra.run.smithi138.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-10T00:17:43.898 INFO:teuthology.orchestra.run.smithi138.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-10T00:17:43.899 INFO:teuthology.orchestra.run.smithi138.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-10T00:17:43.900 INFO:teuthology.orchestra.run.smithi138.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-10T00:17:43.902 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-10T00:17:43.903 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-10T00:17:43.965 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-06-10T00:17:43.975 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-06-10T00:17:43.976 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-06-10T00:17:43.976 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-06-10T00:17:43.976 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-06-10T00:17:43.976 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-10T00:17:43.976 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-10T00:17:43.976 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-10T00:17:43.976 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-10T00:17:43.976 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-10T00:17:43.976 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-10T00:17:43.976 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-10T00:17:43.976 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-10T00:17:43.977 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-10T00:17:43.977 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-10T00:17:43.977 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-10T00:17:43.977 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-10T00:17:43.977 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-10T00:17:43.977 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-10T00:17:43.977 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-10T00:17:43.977 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-10T00:17:43.977 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-10T00:17:43.977 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-10T00:17:43.977 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-10T00:17:43.978 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-10T00:17:43.978 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-10T00:17:43.978 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-10T00:17:43.978 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-10T00:17:43.978 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-10T00:17:43.978 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-10T00:17:43.978 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-10T00:17:43.978 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-10T00:17:43.978 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-10T00:17:43.978 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-10T00:17:43.978 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-10T00:17:43.978 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-10T00:17:43.979 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-10T00:17:43.979 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-10T00:17:43.979 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-10T00:17:43.979 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:17:43.979 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-10T00:17:43.979 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-10T00:17:43.979 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-06-10T00:17:43.979 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-10T00:17:43.979 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-10T00:17:43.979 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-10T00:17:43.979 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-10T00:17:43.979 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-10T00:17:43.980 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-10T00:17:43.980 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-10T00:17:43.980 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-10T00:17:43.980 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:17:43.980 INFO:teuthology.orchestra.run.smithi138.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-10T00:17:43.980 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-10T00:17:43.980 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-06-10T00:17:43.980 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-10T00:17:43.980 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-10T00:17:43.980 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-10T00:17:43.980 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-10T00:17:43.980 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-10T00:17:43.981 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-10T00:17:43.981 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-10T00:17:43.981 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-10T00:17:43.981 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-10T00:17:43.981 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-10T00:17:43.981 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-10T00:17:43.981 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-10T00:17:43.981 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-10T00:17:43.981 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-10T00:17:43.981 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-10T00:17:43.981 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-10T00:17:43.981 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-10T00:17:43.981 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-10T00:17:43.982 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-10T00:17:43.982 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-10T00:17:43.982 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-10T00:17:43.982 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-10T00:17:43.982 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-10T00:17:43.982 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-10T00:17:43.982 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-10T00:17:43.982 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-10T00:17:43.982 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-10T00:17:43.982 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-10T00:17:43.982 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-10T00:17:43.983 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-10T00:17:43.983 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-10T00:17:43.983 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-10T00:17:43.983 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-10T00:17:43.983 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-10T00:17:43.983 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-10T00:17:43.983 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-10T00:17:43.983 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-10T00:17:43.983 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-10T00:17:43.983 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-10T00:17:43.983 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-10T00:17:43.984 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-10T00:17:43.984 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-10T00:17:43.984 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-10T00:17:43.984 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-10T00:17:43.984 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-10T00:17:43.984 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-10T00:17:43.984 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-10T00:17:43.984 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-10T00:17:43.984 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-10T00:17:43.984 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-10T00:17:43.984 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-10T00:17:43.984 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-10T00:17:43.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-10T00:17:43.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-10T00:17:43.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-10T00:17:43.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-10T00:17:43.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-10T00:17:43.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-10T00:17:43.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-10T00:17:43.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-10T00:17:43.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-10T00:17:43.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-10T00:17:43.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-10T00:17:43.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-10T00:17:43.985 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-10T00:17:43.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-10T00:17:43.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-10T00:17:43.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-10T00:17:43.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-10T00:17:43.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-10T00:17:43.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-10T00:17:43.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-10T00:17:43.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-10T00:17:43.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-10T00:17:43.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-10T00:17:43.986 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-10T00:17:43.987 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-10T00:17:43.987 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-10T00:17:43.987 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-10T00:17:43.987 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-10T00:17:43.987 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-10T00:17:43.987 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-10T00:17:43.987 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-10T00:17:43.987 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-10T00:17:43.987 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-10T00:17:43.987 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-10T00:17:43.987 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-10T00:17:43.987 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-10T00:17:43.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-10T00:17:43.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-10T00:17:43.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-10T00:17:43.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-10T00:17:43.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-10T00:17:43.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-10T00:17:43.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-10T00:17:43.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-10T00:17:43.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-10T00:17:43.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-10T00:17:43.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-10T00:17:43.988 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-10T00:17:43.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-10T00:17:43.989 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-10T00:17:43.989 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-10T00:17:43.989 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-10T00:17:43.989 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-10T00:17:43.989 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-10T00:17:43.989 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-10T00:17:43.989 INFO:teuthology.orchestra.run.smithi138.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-10T00:17:43.989 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-10T00:17:43.989 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-10T00:17:43.989 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-06-10T00:17:43.989 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-10T00:17:43.989 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:17:43.990 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-06-10T00:17:43.990 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-06-10T00:17:43.990 INFO:teuthology.orchestra.run.smithi138.stdout:Install 148 Packages 2024-06-10T00:17:43.990 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 11 Packages 2024-06-10T00:17:43.990 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:17:43.990 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 225 M 2024-06-10T00:17:43.990 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-06-10T00:17:44.385 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:17:44.386 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:17:44.397 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-10T00:17:44.398 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-10T00:17:44.399 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-10T00:17:44.400 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-10T00:17:44.401 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-10T00:17:44.403 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-10T00:17:44.404 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-10T00:17:44.466 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-06-10T00:17:44.485 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-06-10T00:17:44.486 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-06-10T00:17:44.486 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-06-10T00:17:44.486 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-06-10T00:17:44.486 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-10T00:17:44.486 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-10T00:17:44.486 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-10T00:17:44.486 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-10T00:17:44.486 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-10T00:17:44.486 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-10T00:17:44.486 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-10T00:17:44.486 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-10T00:17:44.487 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-10T00:17:44.487 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-10T00:17:44.487 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-10T00:17:44.487 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-10T00:17:44.487 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-10T00:17:44.487 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-10T00:17:44.487 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-10T00:17:44.487 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-10T00:17:44.487 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-10T00:17:44.487 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-10T00:17:44.487 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-10T00:17:44.487 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-10T00:17:44.488 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-10T00:17:44.488 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-10T00:17:44.488 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-10T00:17:44.488 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-10T00:17:44.488 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-10T00:17:44.488 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-10T00:17:44.488 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-10T00:17:44.488 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-10T00:17:44.488 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-10T00:17:44.488 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-10T00:17:44.488 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-10T00:17:44.488 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-10T00:17:44.489 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-10T00:17:44.489 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-10T00:17:44.489 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:17:44.489 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-10T00:17:44.489 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-10T00:17:44.489 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2024-06-10T00:17:44.489 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-10T00:17:44.489 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-10T00:17:44.489 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-10T00:17:44.489 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-10T00:17:44.489 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-10T00:17:44.489 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-10T00:17:44.489 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-10T00:17:44.490 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-10T00:17:44.490 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:17:44.490 INFO:teuthology.orchestra.run.smithi105.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-10T00:17:44.490 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-10T00:17:44.490 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-06-10T00:17:44.490 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-10T00:17:44.490 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-10T00:17:44.490 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-10T00:17:44.490 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-10T00:17:44.490 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-10T00:17:44.490 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-10T00:17:44.491 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-10T00:17:44.491 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-10T00:17:44.491 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-10T00:17:44.491 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-10T00:17:44.491 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-10T00:17:44.491 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-10T00:17:44.491 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-10T00:17:44.491 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-10T00:17:44.491 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-10T00:17:44.491 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-10T00:17:44.491 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-10T00:17:44.491 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-10T00:17:44.492 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-10T00:17:44.492 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-10T00:17:44.492 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-10T00:17:44.492 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-10T00:17:44.492 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-10T00:17:44.492 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-10T00:17:44.492 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-10T00:17:44.492 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-10T00:17:44.492 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-10T00:17:44.492 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-10T00:17:44.492 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-10T00:17:44.492 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-10T00:17:44.492 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-10T00:17:44.492 INFO:teuthology.orchestra.run.smithi105.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-10T00:17:44.493 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-10T00:17:44.493 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-10T00:17:44.493 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-10T00:17:44.493 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-10T00:17:44.493 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-10T00:17:44.493 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-10T00:17:44.493 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-10T00:17:44.493 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-10T00:17:44.493 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-10T00:17:44.493 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-10T00:17:44.493 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-10T00:17:44.493 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-10T00:17:44.494 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-10T00:17:44.494 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-10T00:17:44.494 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-10T00:17:44.494 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-10T00:17:44.494 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-10T00:17:44.494 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-10T00:17:44.494 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-10T00:17:44.494 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-10T00:17:44.494 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-10T00:17:44.494 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-10T00:17:44.494 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-10T00:17:44.494 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-10T00:17:44.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-10T00:17:44.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-10T00:17:44.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-10T00:17:44.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-10T00:17:44.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-10T00:17:44.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-10T00:17:44.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-10T00:17:44.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-10T00:17:44.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-10T00:17:44.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-10T00:17:44.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-10T00:17:44.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-10T00:17:44.495 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-10T00:17:44.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-10T00:17:44.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-10T00:17:44.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-10T00:17:44.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-10T00:17:44.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-10T00:17:44.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-10T00:17:44.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-10T00:17:44.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-10T00:17:44.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-10T00:17:44.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-10T00:17:44.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-10T00:17:44.496 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-10T00:17:44.497 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-10T00:17:44.497 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-10T00:17:44.497 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-10T00:17:44.497 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-10T00:17:44.497 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-10T00:17:44.497 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-10T00:17:44.497 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-10T00:17:44.497 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-10T00:17:44.497 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-10T00:17:44.497 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-10T00:17:44.497 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-10T00:17:44.497 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-10T00:17:44.498 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-10T00:17:44.498 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-10T00:17:44.498 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-10T00:17:44.498 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-10T00:17:44.498 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-10T00:17:44.498 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-10T00:17:44.498 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-10T00:17:44.498 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-10T00:17:44.498 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-10T00:17:44.498 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-10T00:17:44.498 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-10T00:17:44.498 INFO:teuthology.orchestra.run.smithi105.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-10T00:17:44.498 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-10T00:17:44.499 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-10T00:17:44.499 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-10T00:17:44.499 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-10T00:17:44.499 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-10T00:17:44.499 INFO:teuthology.orchestra.run.smithi105.stdout:Installing weak dependencies: 2024-06-10T00:17:44.499 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-10T00:17:44.499 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:17:44.499 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-06-10T00:17:44.499 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-06-10T00:17:44.499 INFO:teuthology.orchestra.run.smithi105.stdout:Install 148 Packages 2024-06-10T00:17:44.499 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 11 Packages 2024-06-10T00:17:44.499 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:17:44.500 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 225 M 2024-06-10T00:17:44.500 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-06-10T00:17:44.868 INFO:teuthology.orchestra.run.smithi138.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-06-10T00:17:45.085 INFO:teuthology.orchestra.run.smithi138.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 4.0 MB/s | 890 kB 00:00 2024-06-10T00:17:45.135 INFO:teuthology.orchestra.run.smithi138.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 2.8 MB/s | 145 kB 00:00 2024-06-10T00:17:45.444 INFO:teuthology.orchestra.run.smithi138.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 7.7 MB/s | 2.4 MB 00:00 2024-06-10T00:17:45.661 INFO:teuthology.orchestra.run.smithi138.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 6.9 MB/s | 1.5 MB 00:00 2024-06-10T00:17:45.693 INFO:teuthology.orchestra.run.smithi105.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-06-10T00:17:45.721 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:17:45.721 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:17:45.733 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-10T00:17:45.734 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-10T00:17:45.738 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-10T00:17:45.738 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-10T00:17:45.738 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-10T00:17:45.740 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-10T00:17:45.741 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-10T00:17:45.809 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-06-10T00:17:45.826 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-06-10T00:17:45.826 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-06-10T00:17:45.826 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-06-10T00:17:45.826 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-06-10T00:17:45.826 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-10T00:17:45.826 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-10T00:17:45.827 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-10T00:17:45.827 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-10T00:17:45.827 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-10T00:17:45.827 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-10T00:17:45.827 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-10T00:17:45.827 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-10T00:17:45.827 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-10T00:17:45.827 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-10T00:17:45.827 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-10T00:17:45.827 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-10T00:17:45.827 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-10T00:17:45.827 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-10T00:17:45.827 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-10T00:17:45.828 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-10T00:17:45.828 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-10T00:17:45.828 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-10T00:17:45.828 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-10T00:17:45.828 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-10T00:17:45.828 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-10T00:17:45.828 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-10T00:17:45.828 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-10T00:17:45.828 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-10T00:17:45.828 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-10T00:17:45.828 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-10T00:17:45.828 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-10T00:17:45.829 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-10T00:17:45.829 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-10T00:17:45.829 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-10T00:17:45.829 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-10T00:17:45.829 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-10T00:17:45.829 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-10T00:17:45.829 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-10T00:17:45.829 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:17:45.829 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-10T00:17:45.829 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-10T00:17:45.829 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-06-10T00:17:45.829 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-10T00:17:45.830 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-10T00:17:45.830 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-10T00:17:45.830 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-10T00:17:45.830 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-10T00:17:45.830 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-10T00:17:45.830 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-10T00:17:45.830 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-10T00:17:45.830 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:17:45.830 INFO:teuthology.orchestra.run.smithi062.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-10T00:17:45.830 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-10T00:17:45.830 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-06-10T00:17:45.830 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-10T00:17:45.830 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-10T00:17:45.831 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-10T00:17:45.831 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-10T00:17:45.831 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-10T00:17:45.831 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-10T00:17:45.831 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-10T00:17:45.831 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-10T00:17:45.831 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-10T00:17:45.831 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-10T00:17:45.831 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-10T00:17:45.831 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-10T00:17:45.831 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-10T00:17:45.831 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-10T00:17:45.832 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-10T00:17:45.832 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-10T00:17:45.832 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-10T00:17:45.832 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-10T00:17:45.832 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-10T00:17:45.832 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-10T00:17:45.832 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-10T00:17:45.832 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-10T00:17:45.832 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-10T00:17:45.832 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-10T00:17:45.832 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-10T00:17:45.832 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-10T00:17:45.832 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-10T00:17:45.833 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-10T00:17:45.833 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-10T00:17:45.833 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-10T00:17:45.833 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-10T00:17:45.833 INFO:teuthology.orchestra.run.smithi062.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-10T00:17:45.833 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-10T00:17:45.833 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-10T00:17:45.833 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-10T00:17:45.833 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-10T00:17:45.833 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-10T00:17:45.833 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-10T00:17:45.833 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-10T00:17:45.834 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-10T00:17:45.834 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-10T00:17:45.834 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-10T00:17:45.834 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-10T00:17:45.834 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-10T00:17:45.834 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-10T00:17:45.834 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-10T00:17:45.834 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-10T00:17:45.834 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-10T00:17:45.834 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-10T00:17:45.834 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-10T00:17:45.834 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-10T00:17:45.835 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-10T00:17:45.835 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-10T00:17:45.835 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-10T00:17:45.835 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-10T00:17:45.835 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-10T00:17:45.835 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-10T00:17:45.835 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-10T00:17:45.835 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-10T00:17:45.835 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-10T00:17:45.835 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-10T00:17:45.835 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-10T00:17:45.835 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-10T00:17:45.835 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-10T00:17:45.835 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-10T00:17:45.836 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-10T00:17:45.836 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-10T00:17:45.836 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-10T00:17:45.836 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-10T00:17:45.836 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-10T00:17:45.836 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-10T00:17:45.836 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-10T00:17:45.836 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-10T00:17:45.836 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-10T00:17:45.836 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-10T00:17:45.836 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-10T00:17:45.836 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-10T00:17:45.836 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-10T00:17:45.836 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-10T00:17:45.837 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-10T00:17:45.837 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-10T00:17:45.837 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-10T00:17:45.837 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-10T00:17:45.837 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-10T00:17:45.837 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-10T00:17:45.837 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-10T00:17:45.837 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-10T00:17:45.837 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-10T00:17:45.837 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-10T00:17:45.837 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-10T00:17:45.837 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-10T00:17:45.837 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-10T00:17:45.837 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-10T00:17:45.838 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-10T00:17:45.838 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-10T00:17:45.838 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-10T00:17:45.838 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-10T00:17:45.838 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-10T00:17:45.838 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-10T00:17:45.838 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-10T00:17:45.838 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-10T00:17:45.838 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-10T00:17:45.838 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-10T00:17:45.838 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-10T00:17:45.838 INFO:teuthology.orchestra.run.smithi062.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-10T00:17:45.838 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-10T00:17:45.839 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-10T00:17:45.839 INFO:teuthology.orchestra.run.smithi062.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-10T00:17:45.839 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-10T00:17:45.839 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-10T00:17:45.839 INFO:teuthology.orchestra.run.smithi062.stdout:Installing weak dependencies: 2024-06-10T00:17:45.839 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-10T00:17:45.839 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:17:45.839 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-06-10T00:17:45.839 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-06-10T00:17:45.839 INFO:teuthology.orchestra.run.smithi062.stdout:Install 148 Packages 2024-06-10T00:17:45.839 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 11 Packages 2024-06-10T00:17:45.840 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:17:45.840 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 225 M 2024-06-10T00:17:45.840 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-06-10T00:17:46.053 INFO:teuthology.orchestra.run.smithi138.stdout:(6/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 3.9 MB/s | 5.4 MB 00:01 2024-06-10T00:17:46.387 INFO:teuthology.orchestra.run.smithi138.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 6.5 MB/s | 4.7 MB 00:00 2024-06-10T00:17:46.460 INFO:teuthology.orchestra.run.smithi105.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 890 kB 00:00 2024-06-10T00:17:46.577 INFO:teuthology.orchestra.run.smithi105.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.2 MB/s | 145 kB 00:00 2024-06-10T00:17:46.662 INFO:teuthology.orchestra.run.smithi138.stdout:(8/159): ceph-common-19.0.0-2946.g3d89accf.el9. 11 MB/s | 21 MB 00:01 2024-06-10T00:17:46.737 INFO:teuthology.orchestra.run.smithi138.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 336 kB/s | 25 kB 00:00 2024-06-10T00:17:46.783 INFO:teuthology.orchestra.run.smithi062.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-06-10T00:17:47.388 INFO:teuthology.orchestra.run.smithi138.stdout:(10/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 10 MB/s | 10 MB 00:01 2024-06-10T00:17:47.463 INFO:teuthology.orchestra.run.smithi138.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.9 MB/s | 221 kB 00:00 2024-06-10T00:17:47.538 INFO:teuthology.orchestra.run.smithi138.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 360 kB/s | 27 kB 00:00 2024-06-10T00:17:47.596 INFO:teuthology.orchestra.run.smithi138.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 544 kB/s | 32 kB 00:00 2024-06-10T00:17:47.747 INFO:teuthology.orchestra.run.smithi138.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 4.9 MB/s | 747 kB 00:00 2024-06-10T00:17:47.775 INFO:teuthology.orchestra.run.smithi062.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 897 kB/s | 890 kB 00:00 2024-06-10T00:17:47.822 INFO:teuthology.orchestra.run.smithi138.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 2.1 MB/s | 163 kB 00:00 2024-06-10T00:17:47.897 INFO:teuthology.orchestra.run.smithi138.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.6 MB/s | 127 kB 00:00 2024-06-10T00:17:47.933 INFO:teuthology.orchestra.run.smithi062.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 914 kB/s | 145 kB 00:00 2024-06-10T00:17:47.990 INFO:teuthology.orchestra.run.smithi138.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 5.3 MB/s | 501 kB 00:00 2024-06-10T00:17:48.019 INFO:teuthology.orchestra.run.smithi105.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 1.6 MB/s | 2.4 MB 00:01 2024-06-10T00:17:48.648 INFO:teuthology.orchestra.run.smithi138.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 8.1 MB/s | 5.3 MB 00:00 2024-06-10T00:17:48.724 INFO:teuthology.orchestra.run.smithi138.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 602 kB/s | 45 kB 00:00 2024-06-10T00:17:48.807 INFO:teuthology.orchestra.run.smithi138.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.6 MB/s | 135 kB 00:00 2024-06-10T00:17:48.891 INFO:teuthology.orchestra.run.smithi138.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.9 MB/s | 162 kB 00:00 2024-06-10T00:17:48.991 INFO:teuthology.orchestra.run.smithi138.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.1 MB/s | 321 kB 00:00 2024-06-10T00:17:49.083 INFO:teuthology.orchestra.run.smithi138.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 3.2 MB/s | 300 kB 00:00 2024-06-10T00:17:49.102 INFO:teuthology.orchestra.run.smithi105.stdout:(5/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-06-10T00:17:49.144 INFO:teuthology.orchestra.run.smithi105.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 1.3 MB/s | 1.5 MB 00:01 2024-06-10T00:17:49.158 INFO:teuthology.orchestra.run.smithi138.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.3 MB/s | 99 kB 00:00 2024-06-10T00:17:49.234 INFO:teuthology.orchestra.run.smithi062.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 1.8 MB/s | 2.4 MB 00:01 2024-06-10T00:17:49.242 INFO:teuthology.orchestra.run.smithi138.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.0 MB/s | 85 kB 00:00 2024-06-10T00:17:49.443 INFO:teuthology.orchestra.run.smithi062.stdout:(5/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-06-10T00:17:49.792 INFO:teuthology.orchestra.run.smithi138.stdout:(26/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 5.6 MB/s | 3.1 MB 00:00 2024-06-10T00:17:49.801 INFO:teuthology.orchestra.run.smithi062.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-06-10T00:17:49.934 INFO:teuthology.orchestra.run.smithi138.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.2 MB/s | 171 kB 00:00 2024-06-10T00:17:50.018 INFO:teuthology.orchestra.run.smithi138.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 380 kB/s | 32 kB 00:00 2024-06-10T00:17:50.110 INFO:teuthology.orchestra.run.smithi138.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.5 MB/s | 142 kB 00:00 2024-06-10T00:17:50.476 INFO:teuthology.orchestra.run.smithi062.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 4.5 MB/s | 4.7 MB 00:01 2024-06-10T00:17:50.685 INFO:teuthology.orchestra.run.smithi138.stdout:(30/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 3.6 MB/s | 17 MB 00:04 2024-06-10T00:17:51.519 INFO:teuthology.orchestra.run.smithi138.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 2.5 MB/s | 3.5 MB 00:01 2024-06-10T00:17:51.703 INFO:teuthology.orchestra.run.smithi138.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 1.3 MB/s | 249 kB 00:00 2024-06-10T00:17:51.794 INFO:teuthology.orchestra.run.smithi138.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 538 kB/s | 49 kB 00:00 2024-06-10T00:17:51.879 INFO:teuthology.orchestra.run.smithi138.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 197 kB/s | 16 kB 00:00 2024-06-10T00:17:52.037 INFO:teuthology.orchestra.run.smithi138.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 1.6 MB/s | 264 kB 00:00 2024-06-10T00:17:52.168 INFO:teuthology.orchestra.run.smithi062.stdout:(8/159): ceph-radosgw-19.0.0-2946.g3d89accf.el9 6.1 MB/s | 10 MB 00:01 2024-06-10T00:17:52.235 INFO:teuthology.orchestra.run.smithi062.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 378 kB/s | 25 kB 00:00 2024-06-10T00:17:52.470 INFO:teuthology.orchestra.run.smithi105.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 1.4 MB/s | 4.7 MB 00:03 2024-06-10T00:17:52.521 INFO:teuthology.orchestra.run.smithi138.stdout:(36/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 8.4 MB/s | 48 MB 00:05 2024-06-10T00:17:52.563 INFO:teuthology.orchestra.run.smithi138.stdout:(37/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 1.4 MB/s | 763 kB 00:00 2024-06-10T00:17:52.613 INFO:teuthology.orchestra.run.smithi138.stdout:(38/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 199 kB/s | 18 kB 00:00 2024-06-10T00:17:52.617 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:52.654 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:52.822 INFO:teuthology.orchestra.run.smithi138.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 311 kB/s | 80 kB 00:00 2024-06-10T00:17:52.822 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:52.914 INFO:teuthology.orchestra.run.smithi138.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-06-10T00:17:52.947 INFO:teuthology.orchestra.run.smithi138.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-06-10T00:17:52.977 INFO:teuthology.orchestra.run.smithi062.stdout:(10/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 5.3 MB/s | 17 MB 00:03 2024-06-10T00:17:52.980 INFO:teuthology.orchestra.run.smithi138.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-06-10T00:17:53.039 INFO:teuthology.orchestra.run.smithi138.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-06-10T00:17:53.060 INFO:teuthology.orchestra.run.smithi062.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.6 MB/s | 221 kB 00:00 2024-06-10T00:17:53.073 INFO:teuthology.orchestra.run.smithi138.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-06-10T00:17:53.098 INFO:teuthology.orchestra.run.smithi138.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-06-10T00:17:53.128 INFO:teuthology.orchestra.run.smithi062.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 405 kB/s | 27 kB 00:00 2024-06-10T00:17:53.132 INFO:teuthology.orchestra.run.smithi138.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-06-10T00:17:53.133 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-10T00:17:53.165 INFO:teuthology.orchestra.run.smithi138.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-06-10T00:17:53.166 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:53.203 INFO:teuthology.orchestra.run.smithi062.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 423 kB/s | 32 kB 00:00 2024-06-10T00:17:53.318 INFO:teuthology.orchestra.run.smithi138.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-06-10T00:17:53.319 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:53.366 INFO:teuthology.orchestra.run.smithi138.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-06-10T00:17:53.369 INFO:teuthology.orchestra.run.smithi062.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 4.4 MB/s | 747 kB 00:00 2024-06-10T00:17:53.391 INFO:teuthology.orchestra.run.smithi138.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-06-10T00:17:53.417 INFO:teuthology.orchestra.run.smithi138.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-06-10T00:17:53.450 INFO:teuthology.orchestra.run.smithi138.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-06-10T00:17:53.470 INFO:teuthology.orchestra.run.smithi062.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.6 MB/s | 163 kB 00:00 2024-06-10T00:17:53.476 INFO:teuthology.orchestra.run.smithi138.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-06-10T00:17:53.518 INFO:teuthology.orchestra.run.smithi138.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-06-10T00:17:53.570 INFO:teuthology.orchestra.run.smithi062.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.2 MB/s | 127 kB 00:00 2024-06-10T00:17:53.576 INFO:teuthology.orchestra.run.smithi138.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-06-10T00:17:53.593 INFO:teuthology.orchestra.run.smithi138.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-06-10T00:17:53.619 INFO:teuthology.orchestra.run.smithi138.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-06-10T00:17:53.652 INFO:teuthology.orchestra.run.smithi138.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-06-10T00:17:53.694 INFO:teuthology.orchestra.run.smithi138.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-06-10T00:17:53.704 INFO:teuthology.orchestra.run.smithi062.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 3.7 MB/s | 501 kB 00:00 2024-06-10T00:17:53.720 INFO:teuthology.orchestra.run.smithi138.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-06-10T00:17:53.737 INFO:teuthology.orchestra.run.smithi138.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 606 kB/s | 25 kB 00:00 2024-06-10T00:17:53.754 INFO:teuthology.orchestra.run.smithi138.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-06-10T00:17:53.779 INFO:teuthology.orchestra.run.smithi138.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-06-10T00:17:53.804 INFO:teuthology.orchestra.run.smithi138.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-10T00:17:53.830 INFO:teuthology.orchestra.run.smithi138.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-06-10T00:17:53.931 INFO:teuthology.orchestra.run.smithi138.stdout:(66/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-06-10T00:17:53.989 INFO:teuthology.orchestra.run.smithi138.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-06-10T00:17:54.047 INFO:teuthology.orchestra.run.smithi138.stdout:(68/159): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-06-10T00:17:54.081 INFO:teuthology.orchestra.run.smithi138.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 9.7 MB/s | 911 kB 00:00 2024-06-10T00:17:54.106 INFO:teuthology.orchestra.run.smithi138.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-10T00:17:54.157 INFO:teuthology.orchestra.run.smithi138.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-06-10T00:17:54.247 INFO:teuthology.orchestra.run.smithi138.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-06-10T00:17:54.274 INFO:teuthology.orchestra.run.smithi138.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 750 kB/s | 87 kB 00:00 2024-06-10T00:17:54.316 INFO:teuthology.orchestra.run.smithi138.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-06-10T00:17:54.391 INFO:teuthology.orchestra.run.smithi138.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 42 MB/s | 6.0 MB 00:00 2024-06-10T00:17:54.416 INFO:teuthology.orchestra.run.smithi138.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-06-10T00:17:54.442 INFO:teuthology.orchestra.run.smithi138.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-06-10T00:17:54.467 INFO:teuthology.orchestra.run.smithi138.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-06-10T00:17:54.492 INFO:teuthology.orchestra.run.smithi138.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-06-10T00:17:54.517 INFO:teuthology.orchestra.run.smithi138.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-06-10T00:17:54.547 INFO:teuthology.orchestra.run.smithi138.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-06-10T00:17:54.561 INFO:teuthology.orchestra.run.smithi062.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 6.2 MB/s | 5.3 MB 00:00 2024-06-10T00:17:54.585 INFO:teuthology.orchestra.run.smithi138.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-06-10T00:17:54.638 INFO:teuthology.orchestra.run.smithi062.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 589 kB/s | 45 kB 00:00 2024-06-10T00:17:54.668 INFO:teuthology.orchestra.run.smithi138.stdout:(83/159): ceph-mgr-diskprediction-local-19.0.0- 1.9 MB/s | 7.4 MB 00:03 2024-06-10T00:17:54.694 INFO:teuthology.orchestra.run.smithi138.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-06-10T00:17:54.713 INFO:teuthology.orchestra.run.smithi062.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.8 MB/s | 135 kB 00:00 2024-06-10T00:17:54.778 INFO:teuthology.orchestra.run.smithi138.stdout:(85/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-06-10T00:17:54.797 INFO:teuthology.orchestra.run.smithi062.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.9 MB/s | 162 kB 00:00 2024-06-10T00:17:54.803 INFO:teuthology.orchestra.run.smithi138.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 388 kB/s | 42 kB 00:00 2024-06-10T00:17:54.828 INFO:teuthology.orchestra.run.smithi138.stdout:(87/159): python3-packaging-20.9-5.el9.noarch.r 484 kB/s | 77 kB 00:00 2024-06-10T00:17:54.845 INFO:teuthology.orchestra.run.smithi138.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-06-10T00:17:54.870 INFO:teuthology.orchestra.run.smithi138.stdout:(89/159): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-06-10T00:17:54.880 INFO:teuthology.orchestra.run.smithi062.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.8 MB/s | 321 kB 00:00 2024-06-10T00:17:54.896 INFO:teuthology.orchestra.run.smithi138.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-10T00:17:54.921 INFO:teuthology.orchestra.run.smithi138.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-10T00:17:54.946 INFO:teuthology.orchestra.run.smithi138.stdout:(92/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-06-10T00:17:54.964 INFO:teuthology.orchestra.run.smithi062.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 3.5 MB/s | 300 kB 00:00 2024-06-10T00:17:54.971 INFO:teuthology.orchestra.run.smithi138.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-06-10T00:17:54.997 INFO:teuthology.orchestra.run.smithi138.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 879 kB/s | 44 kB 00:00 2024-06-10T00:17:55.022 INFO:teuthology.orchestra.run.smithi138.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-06-10T00:17:55.048 INFO:teuthology.orchestra.run.smithi062.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.2 MB/s | 99 kB 00:00 2024-06-10T00:17:55.048 INFO:teuthology.orchestra.run.smithi138.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-06-10T00:17:55.073 INFO:teuthology.orchestra.run.smithi138.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-06-10T00:17:55.098 INFO:teuthology.orchestra.run.smithi138.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-06-10T00:17:55.123 INFO:teuthology.orchestra.run.smithi062.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 85 kB 00:00 2024-06-10T00:17:55.123 INFO:teuthology.orchestra.run.smithi138.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-10T00:17:55.149 INFO:teuthology.orchestra.run.smithi138.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-10T00:17:55.174 INFO:teuthology.orchestra.run.smithi138.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-10T00:17:55.199 INFO:teuthology.orchestra.run.smithi138.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-06-10T00:17:55.200 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:55.201 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:55.441 INFO:teuthology.orchestra.run.smithi138.stdout:(103/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-06-10T00:17:55.466 INFO:teuthology.orchestra.run.smithi138.stdout:(104/159): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-06-10T00:17:55.500 INFO:teuthology.orchestra.run.smithi138.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-06-10T00:17:55.525 INFO:teuthology.orchestra.run.smithi138.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-06-10T00:17:55.551 INFO:teuthology.orchestra.run.smithi138.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-06-10T00:17:55.556 INFO:teuthology.orchestra.run.smithi062.stdout:(26/159): ceph-common-19.0.0-2946.g3d89accf.el9 2.3 MB/s | 21 MB 00:09 2024-06-10T00:17:55.594 INFO:teuthology.orchestra.run.smithi138.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 610 kB/s | 26 kB 00:00 2024-06-10T00:17:55.657 INFO:teuthology.orchestra.run.smithi062.stdout:(27/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 5.8 MB/s | 3.1 MB 00:00 2024-06-10T00:17:55.682 INFO:teuthology.orchestra.run.smithi062.stdout:(28/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.3 MB/s | 171 kB 00:00 2024-06-10T00:17:55.716 INFO:teuthology.orchestra.run.smithi062.stdout:(29/159): ceph-grafana-dashboards-19.0.0-2946.g 543 kB/s | 32 kB 00:00 2024-06-10T00:17:55.741 INFO:teuthology.orchestra.run.smithi062.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 2.4 MB/s | 142 kB 00:00 2024-06-10T00:17:55.743 INFO:teuthology.orchestra.run.smithi138.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 326 kB/s | 49 kB 00:00 2024-06-10T00:17:55.768 INFO:teuthology.orchestra.run.smithi138.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-06-10T00:17:55.802 INFO:teuthology.orchestra.run.smithi138.stdout:(111/159): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-06-10T00:17:55.803 INFO:teuthology.orchestra.run.smithi105.stdout:(8/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x86 2.5 MB/s | 17 MB 00:06 2024-06-10T00:17:55.852 INFO:teuthology.orchestra.run.smithi138.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-06-10T00:17:55.895 INFO:teuthology.orchestra.run.smithi138.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 14 MB/s | 548 kB 00:00 2024-06-10T00:17:55.920 INFO:teuthology.orchestra.run.smithi138.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-06-10T00:17:55.945 INFO:teuthology.orchestra.run.smithi105.stdout:(9/159): ceph-radosgw-19.0.0-2946.g3d89accf.el9 2.9 MB/s | 10 MB 00:03 2024-06-10T00:17:55.945 INFO:teuthology.orchestra.run.smithi138.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-06-10T00:17:55.971 INFO:teuthology.orchestra.run.smithi105.stdout:(10/159): ceph-selinux-19.0.0-2946.g3d89accf.el 151 kB/s | 25 kB 00:00 2024-06-10T00:17:55.971 INFO:teuthology.orchestra.run.smithi138.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 607 kB/s | 15 kB 00:00 2024-06-10T00:17:56.004 INFO:teuthology.orchestra.run.smithi138.stdout:(117/159): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-06-10T00:17:56.055 INFO:teuthology.orchestra.run.smithi138.stdout:(118/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.2 MB/s | 839 kB 00:00 2024-06-10T00:17:56.080 INFO:teuthology.orchestra.run.smithi138.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-06-10T00:17:56.147 INFO:teuthology.orchestra.run.smithi138.stdout:(120/159): libarrow-9.0.0-11.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-06-10T00:17:56.162 INFO:teuthology.orchestra.run.smithi105.stdout:(11/159): ceph-common-19.0.0-2946.g3d89accf.el9 2.0 MB/s | 21 MB 00:10 2024-06-10T00:17:56.166 INFO:teuthology.orchestra.run.smithi062.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 7.8 MB/s | 3.5 MB 00:00 2024-06-10T00:17:56.172 INFO:teuthology.orchestra.run.smithi138.stdout:(121/159): python3-google-auth-2.29.0-2.el9.noa 2.2 MB/s | 205 kB 00:00 2024-06-10T00:17:56.187 INFO:teuthology.orchestra.run.smithi105.stdout:(12/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.0 MB/s | 221 kB 00:00 2024-06-10T00:17:56.197 INFO:teuthology.orchestra.run.smithi138.stdout:(122/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-06-10T00:17:56.221 INFO:teuthology.orchestra.run.smithi105.stdout:(13/159): cephfs-shell-19.0.0-2946.g3d89accf.el 464 kB/s | 27 kB 00:00 2024-06-10T00:17:56.223 INFO:teuthology.orchestra.run.smithi138.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-06-10T00:17:56.246 INFO:teuthology.orchestra.run.smithi105.stdout:(14/159): libcephfs-devel-19.0.0-2946.g3d89accf 542 kB/s | 32 kB 00:00 2024-06-10T00:17:56.248 INFO:teuthology.orchestra.run.smithi138.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-06-10T00:17:56.250 INFO:teuthology.orchestra.run.smithi062.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.9 MB/s | 249 kB 00:00 2024-06-10T00:17:56.273 INFO:teuthology.orchestra.run.smithi138.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-06-10T00:17:56.296 INFO:teuthology.orchestra.run.smithi105.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 3.2 MB/s | 163 kB 00:00 2024-06-10T00:17:56.298 INFO:teuthology.orchestra.run.smithi138.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-06-10T00:17:56.324 INFO:teuthology.orchestra.run.smithi138.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-06-10T00:17:56.349 INFO:teuthology.orchestra.run.smithi138.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-06-10T00:17:56.363 INFO:teuthology.orchestra.run.smithi105.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.9 MB/s | 127 kB 00:00 2024-06-10T00:17:56.375 INFO:teuthology.orchestra.run.smithi138.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-10T00:17:56.400 INFO:teuthology.orchestra.run.smithi138.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-10T00:17:56.422 INFO:teuthology.orchestra.run.smithi105.stdout:(17/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 3.6 MB/s | 747 kB 00:00 2024-06-10T00:17:56.425 INFO:teuthology.orchestra.run.smithi138.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-06-10T00:17:56.459 INFO:teuthology.orchestra.run.smithi138.stdout:(132/159): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-06-10T00:17:56.467 INFO:teuthology.orchestra.run.smithi062.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 227 kB/s | 49 kB 00:00 2024-06-10T00:17:56.484 INFO:teuthology.orchestra.run.smithi138.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-06-10T00:17:56.509 INFO:teuthology.orchestra.run.smithi138.stdout:(134/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-06-10T00:17:56.534 INFO:teuthology.orchestra.run.smithi062.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 247 kB/s | 16 kB 00:00 2024-06-10T00:17:56.539 INFO:teuthology.orchestra.run.smithi105.stdout:(18/159): libradosstriper1-19.0.0-2946.g3d89acc 2.8 MB/s | 501 kB 00:00 2024-06-10T00:17:56.543 INFO:teuthology.orchestra.run.smithi138.stdout:(135/159): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-06-10T00:17:56.568 INFO:teuthology.orchestra.run.smithi138.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-06-10T00:17:56.595 INFO:teuthology.orchestra.run.smithi138.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-06-10T00:17:56.606 INFO:teuthology.orchestra.run.smithi105.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 676 kB/s | 45 kB 00:00 2024-06-10T00:17:56.626 INFO:teuthology.orchestra.run.smithi062.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.8 MB/s | 264 kB 00:00 2024-06-10T00:17:56.656 INFO:teuthology.orchestra.run.smithi105.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 2.6 MB/s | 135 kB 00:00 2024-06-10T00:17:56.686 INFO:teuthology.orchestra.run.smithi138.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-06-10T00:17:56.711 INFO:teuthology.orchestra.run.smithi138.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-06-10T00:17:56.723 INFO:teuthology.orchestra.run.smithi105.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 2.4 MB/s | 162 kB 00:00 2024-06-10T00:17:56.762 INFO:teuthology.orchestra.run.smithi138.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-06-10T00:17:56.787 INFO:teuthology.orchestra.run.smithi138.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-06-10T00:17:56.807 INFO:teuthology.orchestra.run.smithi105.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.8 MB/s | 321 kB 00:00 2024-06-10T00:17:56.812 INFO:teuthology.orchestra.run.smithi138.stdout:(142/159): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-06-10T00:17:56.846 INFO:teuthology.orchestra.run.smithi138.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-06-10T00:17:56.882 INFO:teuthology.orchestra.run.smithi105.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 3.9 MB/s | 300 kB 00:00 2024-06-10T00:17:56.888 INFO:teuthology.orchestra.run.smithi138.stdout:(144/159): python3-grpcio-1.46.7-10.el9.x86_64. 2.8 MB/s | 2.0 MB 00:00 2024-06-10T00:17:56.913 INFO:teuthology.orchestra.run.smithi138.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-06-10T00:17:56.932 INFO:teuthology.orchestra.run.smithi105.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.9 MB/s | 99 kB 00:00 2024-06-10T00:17:56.938 INFO:teuthology.orchestra.run.smithi138.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-06-10T00:17:56.964 INFO:teuthology.orchestra.run.smithi138.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-06-10T00:17:56.983 INFO:teuthology.orchestra.run.smithi105.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.7 MB/s | 85 kB 00:00 2024-06-10T00:17:57.006 INFO:teuthology.orchestra.run.smithi138.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-06-10T00:17:57.168 INFO:teuthology.orchestra.run.smithi062.stdout:(36/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 9.8 MB/s | 48 MB 00:04 2024-06-10T00:17:57.201 INFO:teuthology.orchestra.run.smithi062.stdout:(37/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 1.3 MB/s | 763 kB 00:00 2024-06-10T00:17:57.260 INFO:teuthology.orchestra.run.smithi062.stdout:(38/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 199 kB/s | 18 kB 00:00 2024-06-10T00:17:57.263 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:57.301 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:57.314 INFO:teuthology.orchestra.run.smithi138.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 7.5 MB/s | 2.3 MB 00:00 2024-06-10T00:17:57.415 INFO:teuthology.orchestra.run.smithi138.stdout:(150/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-06-10T00:17:57.435 INFO:teuthology.orchestra.run.smithi062.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 344 kB/s | 80 kB 00:00 2024-06-10T00:17:57.436 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:57.490 INFO:teuthology.orchestra.run.smithi138.stdout:(151/159): NetworkManager-libnm-1.47.91-1.el9.x 10 MB/s | 1.8 MB 00:00 2024-06-10T00:17:57.523 INFO:teuthology.orchestra.run.smithi138.stdout:(152/159): NetworkManager-team-1.47.91-1.el9.x8 381 kB/s | 41 kB 00:00 2024-06-10T00:17:57.524 INFO:teuthology.orchestra.run.smithi105.stdout:(26/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 4.8 MB/s | 5.3 MB 00:01 2024-06-10T00:17:57.527 INFO:teuthology.orchestra.run.smithi062.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-06-10T00:17:57.549 INFO:teuthology.orchestra.run.smithi138.stdout:(153/159): NetworkManager-tui-1.47.91-1.el9.x86 4.1 MB/s | 248 kB 00:00 2024-06-10T00:17:57.566 INFO:teuthology.orchestra.run.smithi138.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 876 kB/s | 37 kB 00:00 2024-06-10T00:17:57.575 INFO:teuthology.orchestra.run.smithi105.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 3.4 MB/s | 171 kB 00:00 2024-06-10T00:17:57.608 INFO:teuthology.orchestra.run.smithi138.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 3.5 MB/s | 207 kB 00:00 2024-06-10T00:17:57.633 INFO:teuthology.orchestra.run.smithi105.stdout:(28/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 4.8 MB/s | 3.1 MB 00:00 2024-06-10T00:17:57.651 INFO:teuthology.orchestra.run.smithi105.stdout:(29/159): ceph-grafana-dashboards-19.0.0-2946.g 421 kB/s | 32 kB 00:00 2024-06-10T00:17:57.661 INFO:teuthology.orchestra.run.smithi062.stdout:(41/159): ceph-mgr-diskprediction-local-19.0.0- 3.9 MB/s | 7.4 MB 00:01 2024-06-10T00:17:57.675 INFO:teuthology.orchestra.run.smithi138.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.3 MB/s | 260 kB 00:00 2024-06-10T00:17:57.675 INFO:teuthology.orchestra.run.smithi105.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 3.3 MB/s | 142 kB 00:00 2024-06-10T00:17:57.686 INFO:teuthology.orchestra.run.smithi062.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-06-10T00:17:57.711 INFO:teuthology.orchestra.run.smithi062.stdout:(43/159): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-06-10T00:17:57.716 INFO:teuthology.orchestra.run.smithi138.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-06-10T00:17:57.745 INFO:teuthology.orchestra.run.smithi062.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-06-10T00:17:57.771 INFO:teuthology.orchestra.run.smithi062.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.2 MB/s | 135 kB 00:00 2024-06-10T00:17:57.804 INFO:teuthology.orchestra.run.smithi062.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-06-10T00:17:57.830 INFO:teuthology.orchestra.run.smithi062.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-06-10T00:17:57.830 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-10T00:17:57.831 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:57.875 INFO:teuthology.orchestra.run.smithi138.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-06-10T00:17:57.955 INFO:teuthology.orchestra.run.smithi062.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 4.5 MB/s | 689 kB 00:00 2024-06-10T00:17:57.956 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:58.013 INFO:teuthology.orchestra.run.smithi062.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-06-10T00:17:58.039 INFO:teuthology.orchestra.run.smithi062.stdout:(50/159): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-06-10T00:17:58.064 INFO:teuthology.orchestra.run.smithi062.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-06-10T00:17:58.106 INFO:teuthology.orchestra.run.smithi062.stdout:(52/159): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-06-10T00:17:58.131 INFO:teuthology.orchestra.run.smithi062.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-06-10T00:17:58.134 INFO:teuthology.orchestra.run.smithi138.stdout:(159/159): librados2-19.0.0-2946.g3d89accf.el9. 2.8 MB/s | 3.4 MB 00:01 2024-06-10T00:17:58.140 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:17:58.140 INFO:teuthology.orchestra.run.smithi138.stdout:Total 16 MB/s | 225 MB 00:14 2024-06-10T00:17:58.143 INFO:teuthology.orchestra.run.smithi105.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 7.1 MB/s | 3.5 MB 00:00 2024-06-10T00:17:58.190 INFO:teuthology.orchestra.run.smithi062.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-06-10T00:17:58.209 INFO:teuthology.orchestra.run.smithi105.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 3.7 MB/s | 249 kB 00:00 2024-06-10T00:17:58.215 INFO:teuthology.orchestra.run.smithi062.stdout:(55/159): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-06-10T00:17:58.248 INFO:teuthology.orchestra.run.smithi062.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-06-10T00:17:58.260 INFO:teuthology.orchestra.run.smithi105.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 983 kB/s | 49 kB 00:00 2024-06-10T00:17:58.307 INFO:teuthology.orchestra.run.smithi062.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 778 kB/s | 45 kB 00:00 2024-06-10T00:17:58.310 INFO:teuthology.orchestra.run.smithi105.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 330 kB/s | 16 kB 00:00 2024-06-10T00:17:58.332 INFO:teuthology.orchestra.run.smithi062.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-06-10T00:17:58.382 INFO:teuthology.orchestra.run.smithi062.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-06-10T00:17:58.386 INFO:teuthology.orchestra.run.smithi105.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 3.4 MB/s | 264 kB 00:00 2024-06-10T00:17:58.441 INFO:teuthology.orchestra.run.smithi062.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-06-10T00:17:58.508 INFO:teuthology.orchestra.run.smithi062.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 712 kB/s | 47 kB 00:00 2024-06-10T00:17:58.511 INFO:teuthology.orchestra.run.smithi105.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 6.0 MB/s | 763 kB 00:00 2024-06-10T00:17:58.561 INFO:teuthology.orchestra.run.smithi105.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 365 kB/s | 18 kB 00:00 2024-06-10T00:17:58.600 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:58.692 INFO:teuthology.orchestra.run.smithi062.stdout:(62/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-06-10T00:17:58.695 INFO:teuthology.orchestra.run.smithi105.stdout:(38/159): ceph-mgr-diskprediction-local-19.0.0- 7.3 MB/s | 7.4 MB 00:01 2024-06-10T00:17:58.696 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:58.717 INFO:teuthology.orchestra.run.smithi062.stdout:(63/159): libtool-2.4.6-45.el9.x86_64.rpm 1.5 MB/s | 582 kB 00:00 2024-06-10T00:17:58.776 INFO:teuthology.orchestra.run.smithi062.stdout:(64/159): openblas-0.3.21-2.el9.x86_64.rpm 606 kB/s | 35 kB 00:00 2024-06-10T00:17:58.809 INFO:teuthology.orchestra.run.smithi062.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-06-10T00:17:58.812 INFO:teuthology.orchestra.run.smithi105.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 321 kB/s | 80 kB 00:00 2024-06-10T00:17:58.813 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:58.904 INFO:teuthology.orchestra.run.smithi105.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-06-10T00:17:58.962 INFO:teuthology.orchestra.run.smithi105.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-06-10T00:17:58.988 INFO:teuthology.orchestra.run.smithi105.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-06-10T00:17:59.046 INFO:teuthology.orchestra.run.smithi105.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-06-10T00:17:59.080 INFO:teuthology.orchestra.run.smithi105.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-06-10T00:17:59.105 INFO:teuthology.orchestra.run.smithi105.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-06-10T00:17:59.139 INFO:teuthology.orchestra.run.smithi105.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-06-10T00:17:59.140 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-10T00:17:59.164 INFO:teuthology.orchestra.run.smithi105.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-06-10T00:17:59.165 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:59.316 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-06-10T00:17:59.398 INFO:teuthology.orchestra.run.smithi105.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 689 kB 00:00 2024-06-10T00:17:59.399 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:17:59.443 INFO:teuthology.orchestra.run.smithi062.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.2 MB/s | 3.0 MB 00:01 2024-06-10T00:17:59.448 INFO:teuthology.orchestra.run.smithi105.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-06-10T00:17:59.474 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-06-10T00:17:59.475 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-06-10T00:17:59.515 INFO:teuthology.orchestra.run.smithi105.stdout:(50/159): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-06-10T00:17:59.540 INFO:teuthology.orchestra.run.smithi105.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 236 kB/s | 22 kB 00:00 2024-06-10T00:17:59.590 INFO:teuthology.orchestra.run.smithi105.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 657 kB/s | 33 kB 00:00 2024-06-10T00:17:59.649 INFO:teuthology.orchestra.run.smithi105.stdout:(53/159): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-06-10T00:17:59.699 INFO:teuthology.orchestra.run.smithi105.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 366 kB/s | 18 kB 00:00 2024-06-10T00:17:59.749 INFO:teuthology.orchestra.run.smithi105.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-06-10T00:17:59.833 INFO:teuthology.orchestra.run.smithi105.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-06-10T00:17:59.875 INFO:teuthology.orchestra.run.smithi105.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-06-10T00:18:00.051 INFO:teuthology.orchestra.run.smithi105.stdout:(58/159): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-06-10T00:18:00.193 INFO:teuthology.orchestra.run.smithi105.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-06-10T00:18:00.243 INFO:teuthology.orchestra.run.smithi105.stdout:(60/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.6 MB/s | 3.0 MB 00:00 2024-06-10T00:18:00.268 INFO:teuthology.orchestra.run.smithi105.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 337 kB/s | 25 kB 00:00 2024-06-10T00:18:00.285 INFO:teuthology.orchestra.run.smithi062.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-06-10T00:18:00.294 INFO:teuthology.orchestra.run.smithi105.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-06-10T00:18:00.352 INFO:teuthology.orchestra.run.smithi105.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-06-10T00:18:00.377 INFO:teuthology.orchestra.run.smithi105.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-06-10T00:18:00.403 INFO:teuthology.orchestra.run.smithi105.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 699 kB/s | 35 kB 00:00 2024-06-10T00:18:00.777 INFO:teuthology.orchestra.run.smithi062.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 1.8 MB/s | 911 kB 00:00 2024-06-10T00:18:00.912 INFO:teuthology.orchestra.run.smithi062.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 987 kB/s | 133 kB 00:00 2024-06-10T00:18:01.178 INFO:teuthology.orchestra.run.smithi105.stdout:(66/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-06-10T00:18:01.210 INFO:teuthology.orchestra.run.smithi062.stdout:(70/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.8 MB/s | 4.4 MB 00:02 2024-06-10T00:18:01.271 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-06-10T00:18:01.271 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-06-10T00:18:01.361 INFO:teuthology.orchestra.run.smithi062.stdout:(71/159): openblas-openmp-0.3.21-2.el9.x86_64.r 1.8 MB/s | 4.7 MB 00:02 2024-06-10T00:18:01.428 INFO:teuthology.orchestra.run.smithi105.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 4.5 MB/s | 4.7 MB 00:01 2024-06-10T00:18:01.478 INFO:teuthology.orchestra.run.smithi062.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 749 kB/s | 87 kB 00:00 2024-06-10T00:18:01.503 INFO:teuthology.orchestra.run.smithi105.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-06-10T00:18:01.570 INFO:teuthology.orchestra.run.smithi105.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-06-10T00:18:01.721 INFO:teuthology.orchestra.run.smithi105.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 3.0 MB/s | 911 kB 00:00 2024-06-10T00:18:01.820 INFO:teuthology.orchestra.run.smithi062.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 1.7 MB/s | 1.0 MB 00:00 2024-06-10T00:18:01.878 INFO:teuthology.orchestra.run.smithi062.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-06-10T00:18:02.054 INFO:teuthology.orchestra.run.smithi062.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-06-10T00:18:02.146 INFO:teuthology.orchestra.run.smithi105.stdout:(71/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 7.8 MB/s | 48 MB 00:06 2024-06-10T00:18:02.188 INFO:teuthology.orchestra.run.smithi105.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 2.2 MB/s | 1.0 MB 00:00 2024-06-10T00:18:02.204 INFO:teuthology.orchestra.run.smithi062.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-06-10T00:18:02.271 INFO:teuthology.orchestra.run.smithi062.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 385 kB/s | 26 kB 00:00 2024-06-10T00:18:02.355 INFO:teuthology.orchestra.run.smithi062.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 230 kB/s | 19 kB 00:00 2024-06-10T00:18:02.355 INFO:teuthology.orchestra.run.smithi105.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 420 kB/s | 87 kB 00:00 2024-06-10T00:18:02.422 INFO:teuthology.orchestra.run.smithi105.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 923 kB/s | 61 kB 00:00 2024-06-10T00:18:02.430 INFO:teuthology.orchestra.run.smithi062.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-06-10T00:18:02.514 INFO:teuthology.orchestra.run.smithi062.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 417 kB/s | 35 kB 00:00 2024-06-10T00:18:02.547 INFO:teuthology.orchestra.run.smithi105.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-06-10T00:18:02.689 INFO:teuthology.orchestra.run.smithi105.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-06-10T00:18:02.773 INFO:teuthology.orchestra.run.smithi105.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 308 kB/s | 26 kB 00:00 2024-06-10T00:18:02.848 INFO:teuthology.orchestra.run.smithi105.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-06-10T00:18:02.932 INFO:teuthology.orchestra.run.smithi105.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-06-10T00:18:03.040 INFO:teuthology.orchestra.run.smithi105.stdout:(80/159): postgresql-server-13.14-1.el9.x86_64. 3.9 MB/s | 5.8 MB 00:01 2024-06-10T00:18:03.066 INFO:teuthology.orchestra.run.smithi105.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 260 kB/s | 35 kB 00:00 2024-06-10T00:18:03.149 INFO:teuthology.orchestra.run.smithi105.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-06-10T00:18:03.233 INFO:teuthology.orchestra.run.smithi105.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-06-10T00:18:03.283 INFO:teuthology.orchestra.run.smithi105.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-10T00:18:03.333 INFO:teuthology.orchestra.run.smithi105.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 850 kB/s | 42 kB 00:00 2024-06-10T00:18:03.417 INFO:teuthology.orchestra.run.smithi105.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-06-10T00:18:03.447 INFO:teuthology.orchestra.run.smithi062.stdout:(81/159): postgresql-server-13.14-1.el9.x86_64. 2.3 MB/s | 5.8 MB 00:02 2024-06-10T00:18:03.493 INFO:teuthology.orchestra.run.smithi105.stdout:(87/159): python3-babel-2.9.1-2.el9.noarch.rpm 4.6 MB/s | 6.0 MB 00:01 2024-06-10T00:18:03.510 INFO:teuthology.orchestra.run.smithi105.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-06-10T00:18:03.551 INFO:teuthology.orchestra.run.smithi105.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-06-10T00:18:03.577 INFO:teuthology.orchestra.run.smithi105.stdout:(90/159): python3-pyasn1-modules-0.4.8-6.el9.no 3.2 MB/s | 279 kB 00:00 2024-06-10T00:18:03.597 INFO:teuthology.orchestra.run.smithi062.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-06-10T00:18:03.602 INFO:teuthology.orchestra.run.smithi105.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-10T00:18:03.678 INFO:teuthology.orchestra.run.smithi105.stdout:(92/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.0 MB/s | 5.1 MB 00:00 2024-06-10T00:18:03.694 INFO:teuthology.orchestra.run.smithi105.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-06-10T00:18:03.720 INFO:teuthology.orchestra.run.smithi105.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-06-10T00:18:03.748 INFO:teuthology.orchestra.run.smithi062.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-06-10T00:18:03.762 INFO:teuthology.orchestra.run.smithi105.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 859 kB/s | 36 kB 00:00 2024-06-10T00:18:03.787 INFO:teuthology.orchestra.run.smithi105.stdout:(96/159): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-06-10T00:18:03.804 INFO:teuthology.orchestra.run.smithi105.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-06-10T00:18:03.840 INFO:teuthology.orchestra.run.smithi062.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 841 kB/s | 77 kB 00:00 2024-06-10T00:18:03.854 INFO:teuthology.orchestra.run.smithi105.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-10T00:18:03.880 INFO:teuthology.orchestra.run.smithi105.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-06-10T00:18:03.898 INFO:teuthology.orchestra.run.smithi062.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 725 kB/s | 42 kB 00:00 2024-06-10T00:18:03.905 INFO:teuthology.orchestra.run.smithi105.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-10T00:18:03.922 INFO:teuthology.orchestra.run.smithi105.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-06-10T00:18:03.923 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:18:03.955 INFO:teuthology.orchestra.run.smithi105.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-06-10T00:18:03.972 INFO:teuthology.orchestra.run.smithi105.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 443 kB/s | 22 kB 00:00 2024-06-10T00:18:03.973 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:18:04.014 INFO:teuthology.orchestra.run.smithi105.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 13 MB/s | 548 kB 00:00 2024-06-10T00:18:04.056 INFO:teuthology.orchestra.run.smithi105.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 7.2 MB/s | 309 kB 00:00 2024-06-10T00:18:04.073 INFO:teuthology.orchestra.run.smithi105.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-06-10T00:18:04.082 INFO:teuthology.orchestra.run.smithi062.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 1.4 MB/s | 269 kB 00:00 2024-06-10T00:18:04.132 INFO:teuthology.orchestra.run.smithi105.stdout:(107/159): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-06-10T00:18:04.199 INFO:teuthology.orchestra.run.smithi105.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-06-10T00:18:04.207 INFO:teuthology.orchestra.run.smithi062.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-06-10T00:18:04.224 INFO:teuthology.orchestra.run.smithi105.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-06-10T00:18:04.249 INFO:teuthology.orchestra.run.smithi105.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-06-10T00:18:04.316 INFO:teuthology.orchestra.run.smithi105.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-06-10T00:18:04.342 INFO:teuthology.orchestra.run.smithi105.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-06-10T00:18:04.384 INFO:teuthology.orchestra.run.smithi105.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-06-10T00:18:04.391 INFO:teuthology.orchestra.run.smithi062.stdout:(88/159): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 279 kB 00:00 2024-06-10T00:18:04.409 INFO:teuthology.orchestra.run.smithi105.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-06-10T00:18:04.434 INFO:teuthology.orchestra.run.smithi105.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 865 kB/s | 43 kB 00:00 2024-06-10T00:18:04.460 INFO:teuthology.orchestra.run.smithi105.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-06-10T00:18:04.474 INFO:teuthology.orchestra.run.smithi062.stdout:(89/159): python3-babel-2.9.1-2.el9.noarch.rpm 2.0 MB/s | 6.0 MB 00:02 2024-06-10T00:18:04.485 INFO:teuthology.orchestra.run.smithi105.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 296 kB/s | 15 kB 00:00 2024-06-10T00:18:04.500 INFO:teuthology.orchestra.run.smithi062.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 469 kB/s | 51 kB 00:00 2024-06-10T00:18:04.510 INFO:teuthology.orchestra.run.smithi105.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-06-10T00:18:04.533 INFO:teuthology.orchestra.run.smithi062.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 917 kB/s | 54 kB 00:00 2024-06-10T00:18:04.536 INFO:teuthology.orchestra.run.smithi105.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-06-10T00:18:04.561 INFO:teuthology.orchestra.run.smithi105.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-06-10T00:18:04.586 INFO:teuthology.orchestra.run.smithi105.stdout:(121/159): python3-google-auth-2.29.0-2.el9.noa 4.0 MB/s | 205 kB 00:00 2024-06-10T00:18:04.592 INFO:teuthology.orchestra.run.smithi062.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-06-10T00:18:04.611 INFO:teuthology.orchestra.run.smithi105.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-06-10T00:18:04.637 INFO:teuthology.orchestra.run.smithi105.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-06-10T00:18:04.650 INFO:teuthology.orchestra.run.smithi062.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 758 kB/s | 44 kB 00:00 2024-06-10T00:18:04.679 INFO:teuthology.orchestra.run.smithi105.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 424 kB/s | 18 kB 00:00 2024-06-10T00:18:04.704 INFO:teuthology.orchestra.run.smithi105.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 927 kB/s | 23 kB 00:00 2024-06-10T00:18:04.729 INFO:teuthology.orchestra.run.smithi105.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-06-10T00:18:04.754 INFO:teuthology.orchestra.run.smithi105.stdout:(127/159): python3-jaraco-text-3.2.0-6.el9.noar 796 kB/s | 20 kB 00:00 2024-06-10T00:18:04.801 INFO:teuthology.orchestra.run.smithi062.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-06-10T00:18:04.813 INFO:teuthology.orchestra.run.smithi105.stdout:(128/159): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-06-10T00:18:04.838 INFO:teuthology.orchestra.run.smithi105.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-10T00:18:04.864 INFO:teuthology.orchestra.run.smithi105.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-10T00:18:04.876 INFO:teuthology.orchestra.run.smithi062.stdout:(95/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.2 MB/s | 5.1 MB 00:02 2024-06-10T00:18:04.889 INFO:teuthology.orchestra.run.smithi105.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-06-10T00:18:04.901 INFO:teuthology.orchestra.run.smithi062.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 358 kB/s | 36 kB 00:00 2024-06-10T00:18:04.935 INFO:teuthology.orchestra.run.smithi062.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-06-10T00:18:04.943 INFO:teuthology.orchestra.run.smithi105.stdout:(132/159): python3-kubernetes-26.1.0-2.el9.noar 5.4 MB/s | 1.0 MB 00:00 2024-06-10T00:18:04.973 INFO:teuthology.orchestra.run.smithi105.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 563 kB/s | 16 kB 00:00 2024-06-10T00:18:04.993 INFO:teuthology.orchestra.run.smithi062.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 991 kB/s | 58 kB 00:00 2024-06-10T00:18:04.998 INFO:teuthology.orchestra.run.smithi105.stdout:(134/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-06-10T00:18:05.024 INFO:teuthology.orchestra.run.smithi105.stdout:(135/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-06-10T00:18:05.049 INFO:teuthology.orchestra.run.smithi105.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-06-10T00:18:05.074 INFO:teuthology.orchestra.run.smithi105.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-06-10T00:18:05.085 INFO:teuthology.orchestra.run.smithi062.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-06-10T00:18:05.099 INFO:teuthology.orchestra.run.smithi105.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-06-10T00:18:05.111 INFO:teuthology.orchestra.run.smithi062.stdout:(100/159): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 305 kB 00:00 2024-06-10T00:18:05.124 INFO:teuthology.orchestra.run.smithi105.stdout:(139/159): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-06-10T00:18:05.150 INFO:teuthology.orchestra.run.smithi105.stdout:(140/159): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-06-10T00:18:05.170 INFO:teuthology.orchestra.run.smithi062.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-06-10T00:18:05.175 INFO:teuthology.orchestra.run.smithi105.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-10T00:18:05.195 INFO:teuthology.orchestra.run.smithi062.stdout:(102/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 584 kB/s | 64 kB 00:00 2024-06-10T00:18:05.201 INFO:teuthology.orchestra.run.smithi105.stdout:(142/159): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-06-10T00:18:05.226 INFO:teuthology.orchestra.run.smithi105.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-06-10T00:18:05.243 INFO:teuthology.orchestra.run.smithi105.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-06-10T00:18:05.256 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:18:05.261 INFO:teuthology.orchestra.run.smithi062.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:18:05.268 INFO:teuthology.orchestra.run.smithi105.stdout:(145/159): re2-20211101-3.el9.x86_64.rpm 7.5 MB/s | 192 kB 00:00 2024-06-10T00:18:05.294 INFO:teuthology.orchestra.run.smithi105.stdout:(146/159): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-06-10T00:18:05.344 INFO:teuthology.orchestra.run.smithi105.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-06-10T00:18:05.345 INFO:teuthology.orchestra.run.smithi062.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 127 kB/s | 22 kB 00:00 2024-06-10T00:18:05.379 INFO:teuthology.orchestra.run.smithi062.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 16 MB/s | 548 kB 00:00 2024-06-10T00:18:05.404 INFO:teuthology.orchestra.run.smithi062.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-06-10T00:18:05.429 INFO:teuthology.orchestra.run.smithi062.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 787 kB/s | 19 kB 00:00 2024-06-10T00:18:05.479 INFO:teuthology.orchestra.run.smithi062.stdout:(107/159): protobuf-compiler-3.14.0-13.el9.x86_ 3.3 MB/s | 863 kB 00:00 2024-06-10T00:18:05.505 INFO:teuthology.orchestra.run.smithi062.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-10T00:18:05.538 INFO:teuthology.orchestra.run.smithi062.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-06-10T00:18:05.544 INFO:teuthology.orchestra.run.smithi105.stdout:(148/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.3 MB/s | 16 MB 00:01 2024-06-10T00:18:05.572 INFO:teuthology.orchestra.run.smithi062.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-06-10T00:18:05.597 INFO:teuthology.orchestra.run.smithi062.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-06-10T00:18:05.664 INFO:teuthology.orchestra.run.smithi062.stdout:(112/159): libarrow-9.0.0-11.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-06-10T00:18:05.706 INFO:teuthology.orchestra.run.smithi062.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-06-10T00:18:05.731 INFO:teuthology.orchestra.run.smithi062.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-06-10T00:18:05.748 INFO:teuthology.orchestra.run.smithi062.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-06-10T00:18:05.761 INFO:teuthology.orchestra.run.smithi105.stdout:(149/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 7.5 MB/s | 3.1 MB 00:00 2024-06-10T00:18:05.765 INFO:teuthology.orchestra.run.smithi062.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 955 kB/s | 32 kB 00:00 2024-06-10T00:18:05.782 INFO:teuthology.orchestra.run.smithi062.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 446 kB/s | 15 kB 00:00 2024-06-10T00:18:05.807 INFO:teuthology.orchestra.run.smithi062.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-06-10T00:18:05.833 INFO:teuthology.orchestra.run.smithi062.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-06-10T00:18:05.858 INFO:teuthology.orchestra.run.smithi062.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-06-10T00:18:05.875 INFO:teuthology.orchestra.run.smithi062.stdout:(121/159): python3-google-auth-2.29.0-2.el9.noa 4.8 MB/s | 205 kB 00:00 2024-06-10T00:18:05.878 INFO:teuthology.orchestra.run.smithi105.stdout:(150/159): librados2-19.0.0-2946.g3d89accf.el9. 5.8 MB/s | 3.4 MB 00:00 2024-06-10T00:18:05.900 INFO:teuthology.orchestra.run.smithi062.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-06-10T00:18:05.917 INFO:teuthology.orchestra.run.smithi062.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 644 kB/s | 11 kB 00:00 2024-06-10T00:18:05.920 INFO:teuthology.orchestra.run.smithi105.stdout:(151/159): NetworkManager-1.47.91-1.el9.x86_64. 6.2 MB/s | 2.3 MB 00:00 2024-06-10T00:18:05.934 INFO:teuthology.orchestra.run.smithi062.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 1.1 MB/s | 18 kB 00:00 2024-06-10T00:18:05.951 INFO:teuthology.orchestra.run.smithi062.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-06-10T00:18:05.977 INFO:teuthology.orchestra.run.smithi062.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 777 kB/s | 19 kB 00:00 2024-06-10T00:18:05.979 INFO:teuthology.orchestra.run.smithi105.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 4.2 MB/s | 248 kB 00:00 2024-06-10T00:18:06.002 INFO:teuthology.orchestra.run.smithi062.stdout:(127/159): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-06-10T00:18:06.037 INFO:teuthology.orchestra.run.smithi105.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 629 kB/s | 37 kB 00:00 2024-06-10T00:18:06.052 INFO:teuthology.orchestra.run.smithi062.stdout:(128/159): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-06-10T00:18:06.062 INFO:teuthology.orchestra.run.smithi105.stdout:(154/159): NetworkManager-team-1.47.91-1.el9.x8 225 kB/s | 41 kB 00:00 2024-06-10T00:18:06.077 INFO:teuthology.orchestra.run.smithi062.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-10T00:18:06.096 INFO:teuthology.orchestra.run.smithi105.stdout:(155/159): NetworkManager-libnm-1.47.91-1.el9.x 5.4 MB/s | 1.8 MB 00:00 2024-06-10T00:18:06.102 INFO:teuthology.orchestra.run.smithi062.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-10T00:18:06.121 INFO:teuthology.orchestra.run.smithi105.stdout:(156/159): elfutils-libelf-0.191-4.el9.x86_64.r 2.4 MB/s | 207 kB 00:00 2024-06-10T00:18:06.128 INFO:teuthology.orchestra.run.smithi062.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-06-10T00:18:06.162 INFO:teuthology.orchestra.run.smithi062.stdout:(132/159): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-06-10T00:18:06.187 INFO:teuthology.orchestra.run.smithi062.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-06-10T00:18:06.205 INFO:teuthology.orchestra.run.smithi105.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-06-10T00:18:06.212 INFO:teuthology.orchestra.run.smithi062.stdout:(134/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-06-10T00:18:06.230 INFO:teuthology.orchestra.run.smithi105.stdout:(158/159): elfutils-libs-0.191-4.el9.x86_64.rpm 1.5 MB/s | 260 kB 00:00 2024-06-10T00:18:06.237 INFO:teuthology.orchestra.run.smithi062.stdout:(135/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-06-10T00:18:06.263 INFO:teuthology.orchestra.run.smithi062.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-06-10T00:18:06.288 INFO:teuthology.orchestra.run.smithi062.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-06-10T00:18:06.313 INFO:teuthology.orchestra.run.smithi062.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-06-10T00:18:06.314 INFO:teuthology.orchestra.run.smithi105.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-06-10T00:18:06.319 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:18:06.319 INFO:teuthology.orchestra.run.smithi105.stdout:Total 10 MB/s | 225 MB 00:21 2024-06-10T00:18:06.339 INFO:teuthology.orchestra.run.smithi062.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-06-10T00:18:06.364 INFO:teuthology.orchestra.run.smithi062.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-06-10T00:18:06.389 INFO:teuthology.orchestra.run.smithi062.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 2.0 MB/s | 51 kB 00:00 2024-06-10T00:18:06.406 INFO:teuthology.orchestra.run.smithi062.stdout:(142/159): python3-websocket-client-1.2.3-2.el9 5.3 MB/s | 90 kB 00:00 2024-06-10T00:18:06.431 INFO:teuthology.orchestra.run.smithi062.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-06-10T00:18:06.474 INFO:teuthology.orchestra.run.smithi062.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 467 kB/s | 20 kB 00:00 2024-06-10T00:18:06.508 INFO:teuthology.orchestra.run.smithi062.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-06-10T00:18:06.690 INFO:teuthology.orchestra.run.smithi062.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 888 kB/s | 192 kB 00:00 2024-06-10T00:18:06.749 INFO:teuthology.orchestra.run.smithi062.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 6.6 MB/s | 1.6 MB 00:00 2024-06-10T00:18:07.233 INFO:teuthology.orchestra.run.smithi062.stdout:(148/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 6.5 MB/s | 3.1 MB 00:00 2024-06-10T00:18:07.283 INFO:teuthology.orchestra.run.smithi062.stdout:(149/159): librados2-19.0.0-2946.g3d89accf.el9. 5.7 MB/s | 3.4 MB 00:00 2024-06-10T00:18:07.506 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-06-10T00:18:07.511 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-06-10T00:18:07.598 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-10T00:18:07.608 INFO:teuthology.orchestra.run.smithi062.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 5.6 MB/s | 1.8 MB 00:00 2024-06-10T00:18:07.637 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-10T00:18:07.661 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-06-10T00:18:07.661 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-06-10T00:18:07.667 INFO:teuthology.orchestra.run.smithi062.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 712 kB/s | 41 kB 00:00 2024-06-10T00:18:07.690 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-10T00:18:07.724 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-10T00:18:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-10T00:18:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-10T00:18:07.817 INFO:teuthology.orchestra.run.smithi062.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 1.6 MB/s | 248 kB 00:00 2024-06-10T00:18:07.876 INFO:teuthology.orchestra.run.smithi062.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 628 kB/s | 37 kB 00:00 2024-06-10T00:18:07.959 INFO:teuthology.orchestra.run.smithi062.stdout:(154/159): NetworkManager-1.47.91-1.el9.x86_64. 3.2 MB/s | 2.3 MB 00:00 2024-06-10T00:18:08.052 INFO:teuthology.orchestra.run.smithi062.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.2 MB/s | 207 kB 00:00 2024-06-10T00:18:08.168 INFO:teuthology.orchestra.run.smithi062.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 1.2 MB/s | 260 kB 00:00 2024-06-10T00:18:08.200 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-10T00:18:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:18:08.380 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:18:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:18:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:18:08.539 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:18:08.572 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-10T00:18:08.764 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-10T00:18:08.793 INFO:teuthology.orchestra.run.smithi062.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 1.8 MB/s | 1.3 MB 00:00 2024-06-10T00:18:08.820 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-10T00:18:08.849 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:18:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:18:08.958 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:18:09.020 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:18:09.042 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:18:09.068 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:18:09.259 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:18:09.400 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:18:09.446 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-06-10T00:18:09.447 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-06-10T00:18:09.460 INFO:teuthology.orchestra.run.smithi062.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.9 MB/s | 2.4 MB 00:01 2024-06-10T00:18:09.565 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:18:09.626 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:18:09.737 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:18:09.802 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:18:10.012 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:18:10.126 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-10T00:18:10.228 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-10T00:18:10.266 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-10T00:18:10.300 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-10T00:18:10.365 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-10T00:18:10.399 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-10T00:18:10.444 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-10T00:18:10.475 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-10T00:18:10.503 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-10T00:18:10.570 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-10T00:18:10.625 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-10T00:18:10.656 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-10T00:18:10.691 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-10T00:18:10.767 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-10T00:18:10.802 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-10T00:18:10.845 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-10T00:18:10.874 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-10T00:18:10.943 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-10T00:18:10.989 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-10T00:18:11.062 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-10T00:18:11.195 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-10T00:18:11.249 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-10T00:18:11.764 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-10T00:18:11.844 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-10T00:18:11.882 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-10T00:18:11.927 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-10T00:18:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-10T00:18:12.014 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-10T00:18:12.055 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-10T00:18:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-10T00:18:12.276 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-10T00:18:12.330 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-10T00:18:12.403 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-10T00:18:12.434 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-10T00:18:12.471 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-10T00:18:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-10T00:18:12.553 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-10T00:18:12.596 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-10T00:18:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-10T00:18:12.774 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-10T00:18:12.815 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-10T00:18:12.856 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-10T00:18:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-10T00:18:13.069 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 63/170 2024-06-10T00:18:13.265 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-10T00:18:13.297 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-10T00:18:13.322 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-10T00:18:13.398 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-10T00:18:13.522 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-10T00:18:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-10T00:18:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-10T00:18:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-10T00:18:13.672 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-10T00:18:13.704 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-10T00:18:13.734 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-10T00:18:13.768 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-10T00:18:13.802 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-10T00:18:13.835 INFO:teuthology.orchestra.run.smithi062.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.7 MB/s | 16 MB 00:09 2024-06-10T00:18:13.842 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:18:13.842 INFO:teuthology.orchestra.run.smithi062.stdout:Total 8.0 MB/s | 225 MB 00:28 2024-06-10T00:18:13.843 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-10T00:18:13.878 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-10T00:18:13.963 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-10T00:18:14.016 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-10T00:18:14.048 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-10T00:18:14.094 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-10T00:18:14.192 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-10T00:18:14.240 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-10T00:18:14.949 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-10T00:18:14.996 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-10T00:18:15.029 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-10T00:18:15.110 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-06-10T00:18:15.126 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-10T00:18:15.176 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-10T00:18:15.222 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-10T00:18:15.286 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-06-10T00:18:15.286 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-06-10T00:18:15.307 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-10T00:18:15.349 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-10T00:18:15.527 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-10T00:18:15.555 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-10T00:18:15.632 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-10T00:18:15.766 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-06-10T00:18:15.876 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-10T00:18:15.914 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-10T00:18:15.968 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-10T00:18:16.002 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-10T00:18:16.028 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-10T00:18:16.070 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-10T00:18:16.270 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-10T00:18:16.403 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-10T00:18:16.403 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-10T00:18:16.512 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:18:16.633 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:18:16.663 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:18:16.754 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:18:16.784 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:18:16.817 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-10T00:18:17.011 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-10T00:18:17.057 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-10T00:18:17.093 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:18:17.198 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:18:17.228 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:18:17.290 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:18:17.312 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:18:17.334 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:18:17.520 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:18:17.621 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-06-10T00:18:17.621 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-06-10T00:18:17.622 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-10T00:18:17.645 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:18:17.700 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-10T00:18:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-10T00:18:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-10T00:18:17.826 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:18:17.897 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:18:17.987 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:18:18.065 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:18:18.071 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-10T00:18:18.125 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:18:18.215 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:18:18.246 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-10T00:18:18.264 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:18:18.282 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-10T00:18:18.383 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-10T00:18:18.476 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-10T00:18:18.513 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-10T00:18:18.548 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-10T00:18:18.622 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-10T00:18:18.656 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-10T00:18:18.700 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-10T00:18:18.733 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-10T00:18:18.752 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-10T00:18:18.778 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-10T00:18:18.786 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:18:18.847 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-10T00:18:18.865 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:18:18.887 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-10T00:18:18.901 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-10T00:18:18.931 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-10T00:18:18.975 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-10T00:18:19.068 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-10T00:18:19.102 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-10T00:18:19.146 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-10T00:18:19.175 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-10T00:18:19.246 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-10T00:18:19.290 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-10T00:18:19.365 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-10T00:18:19.499 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-10T00:18:19.551 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-10T00:18:20.084 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-10T00:18:20.163 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-10T00:18:20.191 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-10T00:18:20.228 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-10T00:18:20.277 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-10T00:18:20.306 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-10T00:18:20.348 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-10T00:18:20.499 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-10T00:18:20.559 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-10T00:18:20.614 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-10T00:18:20.687 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-10T00:18:20.718 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-10T00:18:20.763 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-10T00:18:20.810 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-10T00:18:20.854 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-10T00:18:20.897 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-10T00:18:20.940 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-10T00:18:21.076 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-10T00:18:21.087 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:18:21.131 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:18:21.167 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-10T00:18:21.208 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-10T00:18:21.304 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-10T00:18:21.341 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 63/170 2024-06-10T00:18:21.382 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-10T00:18:21.407 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-10T00:18:21.432 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-10T00:18:21.506 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-10T00:18:21.633 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-10T00:18:21.653 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:18:21.664 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-10T00:18:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:18:21.692 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-10T00:18:21.734 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-10T00:18:21.773 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-10T00:18:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:18:21.777 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-10T00:18:21.777 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-10T00:18:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:18:21.805 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-10T00:18:21.805 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:18:21.835 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-10T00:18:21.870 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-10T00:18:21.903 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-10T00:18:21.944 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-10T00:18:21.979 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-10T00:18:22.066 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-10T00:18:22.109 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-10T00:18:22.149 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-10T00:18:22.196 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-10T00:18:22.303 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-10T00:18:22.350 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-10T00:18:23.074 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-10T00:18:23.122 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-10T00:18:23.156 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-10T00:18:23.256 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-10T00:18:23.303 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-10T00:18:23.348 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-10T00:18:23.436 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-10T00:18:23.484 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-10T00:18:23.661 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-10T00:18:23.690 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-10T00:18:23.768 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-10T00:18:24.048 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-06-10T00:18:24.124 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-10T00:18:24.173 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-10T00:18:24.248 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-10T00:18:24.283 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-10T00:18:24.308 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-10T00:18:24.358 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-10T00:18:24.419 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-10T00:18:24.538 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-10T00:18:24.714 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-10T00:18:24.817 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:18:24.941 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:18:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:18:25.068 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:18:25.098 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:18:25.137 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-10T00:18:25.322 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-10T00:18:25.375 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-10T00:18:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:18:25.520 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:18:25.549 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:18:25.620 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:18:25.642 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:18:25.663 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:18:25.771 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-10T00:18:25.853 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:18:25.860 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-10T00:18:25.888 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-10T00:18:25.921 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-10T00:18:26.025 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:18:26.172 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:18:26.232 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:18:26.236 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-10T00:18:26.276 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:18:26.330 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:18:26.358 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:18:26.389 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-10T00:18:26.425 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:18:26.434 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-10T00:18:26.641 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:18:26.752 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-10T00:18:26.847 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-10T00:18:26.881 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-10T00:18:26.907 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-10T00:18:26.916 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-10T00:18:26.946 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:18:27.001 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-10T00:18:27.008 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:18:27.031 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-10T00:18:27.040 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-10T00:18:27.083 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-10T00:18:27.116 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-10T00:18:27.143 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-10T00:18:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-10T00:18:27.267 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-10T00:18:27.296 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-10T00:18:27.340 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-10T00:18:27.418 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-10T00:18:27.459 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-10T00:18:27.504 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-10T00:18:27.531 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-10T00:18:27.606 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-10T00:18:27.655 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-10T00:18:27.732 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-10T00:18:27.869 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-10T00:18:27.932 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-10T00:18:28.491 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-10T00:18:28.571 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-10T00:18:28.606 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-10T00:18:28.660 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-10T00:18:28.700 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-10T00:18:28.738 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-10T00:18:28.779 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-10T00:18:28.939 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-10T00:18:29.015 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-10T00:18:29.071 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-10T00:18:29.145 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-10T00:18:29.182 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-10T00:18:29.219 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-10T00:18:29.267 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-10T00:18:29.301 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-10T00:18:29.346 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-10T00:18:29.369 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:18:29.387 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-10T00:18:29.533 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-10T00:18:29.572 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-10T00:18:29.614 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-10T00:18:29.643 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:18:29.711 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-10T00:18:29.746 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 63/170 2024-06-10T00:18:29.787 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-10T00:18:29.820 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-10T00:18:29.845 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-10T00:18:29.941 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-10T00:18:30.080 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-10T00:18:30.113 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-10T00:18:30.141 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-10T00:18:30.175 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-10T00:18:30.188 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:18:30.223 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-10T00:18:30.231 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:18:30.263 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-10T00:18:30.309 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-10T00:18:30.321 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:18:30.321 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-10T00:18:30.322 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-10T00:18:30.322 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:18:30.344 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-10T00:18:30.348 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:18:30.386 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-10T00:18:30.428 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-10T00:18:30.462 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-10T00:18:30.552 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-10T00:18:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-10T00:18:30.649 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-10T00:18:30.696 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-10T00:18:30.808 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-10T00:18:30.868 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-10T00:18:31.637 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-10T00:18:31.697 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-10T00:18:31.731 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-10T00:18:31.835 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-10T00:18:31.887 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-10T00:18:31.932 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-10T00:18:32.022 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-10T00:18:32.058 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-10T00:18:32.243 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-10T00:18:32.281 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-10T00:18:32.363 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-10T00:18:33.060 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-10T00:18:33.184 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-10T00:18:33.485 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:18:33.485 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-06-10T00:18:33.485 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-06-10T00:18:33.485 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-06-10T00:18:33.485 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-06-10T00:18:33.485 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-06-10T00:18:33.485 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-06-10T00:18:33.485 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-06-10T00:18:33.486 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:18:33.659 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:18:33.674 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:18:33.674 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:33.675 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-10T00:18:33.675 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:18:34.509 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-10T00:18:34.603 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-10T00:18:34.630 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-10T00:18:34.663 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-10T00:18:34.991 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-10T00:18:35.035 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:18:35.126 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:18:35.156 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:18:35.156 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-10T00:18:35.181 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:18:35.181 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:35.181 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-10T00:18:35.181 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:18:35.181 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:18:35.181 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:18:35.193 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-10T00:18:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:18:35.429 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:18:35.429 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:35.429 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-10T00:18:35.429 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:18:35.429 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:18:35.430 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:18:35.690 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-10T00:18:35.712 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:18:35.792 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:18:35.823 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-10T00:18:35.865 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:18:35.887 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:18:35.887 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:35.887 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-10T00:18:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:18:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:18:35.888 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:18:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-10T00:18:36.043 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-10T00:18:36.100 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-10T00:18:36.285 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-10T00:18:36.334 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-10T00:18:36.411 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-10T00:18:36.509 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-10T00:18:36.648 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-10T00:18:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-10T00:18:36.734 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-10T00:18:36.761 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-10T00:18:37.198 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:18:37.228 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:18:37.259 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-10T00:18:37.322 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-10T00:18:37.351 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:18:38.169 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:18:38.228 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:18:38.384 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:18:38.420 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:18:38.807 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:18:38.848 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:18:38.922 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:18:38.922 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-10T00:18:38.922 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-10T00:18:38.922 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:18:38.949 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:18:39.380 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:18:39.412 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:18:39.513 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:18:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-10T00:18:39.723 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:18:39.746 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:18:39.746 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:39.746 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-10T00:18:39.746 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:18:39.746 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:18:39.747 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:18:39.791 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:18:39.802 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:18:39.900 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-10T00:18:39.947 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-10T00:18:40.840 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-10T00:18:40.869 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:18:40.891 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:18:40.892 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:40.892 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-10T00:18:40.892 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:18:40.892 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:18:40.892 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:18:40.945 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:18:40.966 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:18:40.966 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:40.966 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-10T00:18:40.966 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:18:41.020 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:18:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:18:41.043 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:41.043 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-10T00:18:41.043 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:18:41.043 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:18:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:18:41.347 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:18:41.369 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:18:41.369 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:41.370 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-10T00:18:41.370 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:18:41.370 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:18:41.370 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:18:41.818 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:18:41.818 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2024-06-10T00:18:41.818 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2024-06-10T00:18:41.818 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2024-06-10T00:18:41.818 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2024-06-10T00:18:41.819 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2024-06-10T00:18:41.819 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /root 2024-06-10T00:18:41.819 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2024-06-10T00:18:41.819 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:18:41.990 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:18:42.007 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:18:42.007 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:42.007 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-10T00:18:42.007 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:18:43.459 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:18:43.485 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:18:43.485 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:43.485 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-10T00:18:43.485 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:18:43.485 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:18:43.485 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:18:43.710 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:18:43.732 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:18:43.732 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:43.733 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-10T00:18:43.733 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:18:43.733 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:18:43.733 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:18:44.178 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:18:44.200 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:18:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-10T00:18:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:18:44.200 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:18:44.200 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:18:44.316 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-10T00:18:44.364 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-10T00:18:44.419 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-10T00:18:44.603 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-10T00:18:44.653 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-10T00:18:44.731 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-10T00:18:44.819 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-10T00:18:44.959 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-10T00:18:45.003 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-10T00:18:45.043 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-10T00:18:45.088 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-10T00:18:45.490 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:18:45.520 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:18:45.551 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-10T00:18:45.613 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-10T00:18:45.643 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:18:45.859 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-10T00:18:45.901 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-10T00:18:45.941 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-10T00:18:45.974 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-10T00:18:46.002 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-10T00:18:46.181 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-10T00:18:46.232 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-10T00:18:46.265 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-10T00:18:46.305 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-10T00:18:46.344 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-10T00:18:46.390 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:18:46.667 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:18:46.706 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:18:47.022 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:18:47.057 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:18:47.076 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:18:47.665 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:18:47.695 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:18:47.797 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:18:47.930 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-10T00:18:47.958 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-10T00:18:47.989 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:18:48.012 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:18:48.012 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:48.013 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-10T00:18:48.013 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:18:48.013 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:18:48.013 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:18:48.023 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-10T00:18:48.056 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:18:48.064 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-10T00:18:48.067 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:18:48.145 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-10T00:18:48.164 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-10T00:18:48.185 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-10T00:18:48.219 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:18:48.221 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-10T00:18:48.245 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-10T00:18:48.271 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-10T00:18:48.271 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:18:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:18:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-10T00:18:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:18:48.465 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:18:48.490 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-10T00:18:48.495 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:18:48.495 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:18:48.676 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:18:48.676 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:18:48.760 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:18:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-10T00:18:48.859 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:18:49.119 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-10T00:18:49.151 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:18:49.173 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:18:49.174 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:49.174 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-10T00:18:49.174 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:18:49.174 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:18:49.174 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:18:49.218 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:18:49.240 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:18:49.240 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:49.240 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-10T00:18:49.240 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:18:49.285 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:18:49.307 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:18:49.307 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:49.307 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-10T00:18:49.308 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:18:49.308 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:18:49.308 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:18:49.611 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:18:49.633 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:18:49.633 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:49.633 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-10T00:18:49.634 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:18:49.634 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:18:49.634 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:18:51.100 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:18:51.100 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2024-06-10T00:18:51.100 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2024-06-10T00:18:51.100 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2024-06-10T00:18:51.100 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2024-06-10T00:18:51.100 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2024-06-10T00:18:51.101 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /root 2024-06-10T00:18:51.101 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2024-06-10T00:18:51.101 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:18:51.273 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:18:51.290 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:18:51.290 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:51.290 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-10T00:18:51.290 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:18:52.826 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:18:52.855 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:18:52.855 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:52.855 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-10T00:18:52.855 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:18:52.855 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:18:52.855 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:18:53.094 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:18:53.117 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:18:53.117 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:53.118 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-10T00:18:53.118 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:18:53.118 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:18:53.118 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:18:53.586 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:18:53.610 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:18:53.611 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:53.611 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-10T00:18:53.611 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:18:53.611 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:18:53.611 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:18:53.717 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-10T00:18:53.775 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-10T00:18:53.843 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-10T00:18:54.037 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-10T00:18:54.077 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-10T00:18:54.156 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-10T00:18:54.159 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-10T00:18:54.197 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-10T00:18:54.238 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-10T00:18:54.255 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-10T00:18:54.270 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-10T00:18:54.298 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-10T00:18:54.416 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-10T00:18:54.453 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-10T00:18:54.483 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-10T00:18:54.485 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-10T00:18:54.511 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-10T00:18:54.527 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-10T00:18:54.560 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-10T00:18:54.592 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-10T00:18:54.621 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-10T00:18:54.668 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:18:54.832 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:18:54.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-10T00:18:54.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-10T00:18:54.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-10T00:18:54.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-10T00:18:54.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-10T00:18:54.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-10T00:18:54.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-10T00:18:54.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:18:54.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:18:54.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:18:54.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-10T00:18:54.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-10T00:18:54.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-10T00:18:54.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:18:54.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-10T00:18:54.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-10T00:18:54.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-10T00:18:54.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-10T00:18:54.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-10T00:18:54.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-10T00:18:54.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-10T00:18:54.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-10T00:18:54.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-10T00:18:54.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-10T00:18:54.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-10T00:18:54.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-10T00:18:54.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-10T00:18:54.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-10T00:18:54.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-10T00:18:54.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-10T00:18:54.847 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-10T00:18:54.847 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-10T00:18:54.848 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-10T00:18:54.848 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-10T00:18:54.848 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-10T00:18:54.848 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-10T00:18:54.848 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-10T00:18:54.848 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-10T00:18:54.848 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-10T00:18:54.848 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-10T00:18:54.848 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-10T00:18:54.848 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-10T00:18:54.848 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-10T00:18:54.849 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-10T00:18:54.849 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-10T00:18:54.849 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-10T00:18:54.849 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-10T00:18:54.849 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-10T00:18:54.849 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-10T00:18:54.849 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-10T00:18:54.849 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-10T00:18:54.849 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-10T00:18:54.849 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-10T00:18:54.849 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-10T00:18:54.849 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-10T00:18:54.849 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-10T00:18:54.850 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-10T00:18:54.850 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-10T00:18:54.850 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-10T00:18:54.850 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-10T00:18:54.850 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-10T00:18:54.850 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-10T00:18:54.850 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-10T00:18:54.850 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-10T00:18:54.850 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-10T00:18:54.850 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-10T00:18:54.850 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-10T00:18:54.850 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-10T00:18:54.851 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-10T00:18:54.851 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-10T00:18:54.851 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-10T00:18:54.851 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-10T00:18:54.851 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-10T00:18:54.851 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-10T00:18:54.851 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-10T00:18:54.851 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-10T00:18:54.851 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-10T00:18:54.851 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-10T00:18:54.851 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-10T00:18:54.851 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-10T00:18:54.852 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-10T00:18:54.852 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-10T00:18:54.852 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-10T00:18:54.852 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-10T00:18:54.852 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-10T00:18:54.852 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-10T00:18:54.852 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-10T00:18:54.852 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-10T00:18:54.852 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-10T00:18:54.852 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-10T00:18:54.853 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-10T00:18:54.853 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-10T00:18:54.853 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-10T00:18:54.853 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-10T00:18:54.853 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-10T00:18:54.853 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-10T00:18:54.853 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-10T00:18:54.853 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-10T00:18:54.853 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-10T00:18:54.853 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-10T00:18:54.853 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-10T00:18:54.854 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-10T00:18:54.854 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-10T00:18:54.854 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-10T00:18:54.854 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-10T00:18:54.854 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-10T00:18:54.854 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-10T00:18:54.854 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-10T00:18:54.854 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-10T00:18:54.854 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-10T00:18:54.854 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-10T00:18:54.855 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-10T00:18:54.855 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-10T00:18:54.855 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-10T00:18:54.855 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-10T00:18:54.855 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-10T00:18:54.855 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-10T00:18:54.855 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-10T00:18:54.855 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-10T00:18:54.855 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-10T00:18:54.855 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-10T00:18:54.855 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/170 2024-06-10T00:18:54.855 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-10T00:18:54.856 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-10T00:18:54.856 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-10T00:18:54.856 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-10T00:18:54.856 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-10T00:18:54.856 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-10T00:18:54.856 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-10T00:18:54.856 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-10T00:18:54.856 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-10T00:18:54.856 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-10T00:18:54.856 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-10T00:18:54.856 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-10T00:18:54.856 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-10T00:18:54.857 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-10T00:18:54.857 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-10T00:18:54.857 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-10T00:18:54.857 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-10T00:18:54.857 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-10T00:18:54.857 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-10T00:18:54.857 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-10T00:18:54.858 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-10T00:18:54.858 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-10T00:18:54.858 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-10T00:18:54.858 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-10T00:18:54.858 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-10T00:18:54.858 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-10T00:18:54.858 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-10T00:18:54.858 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-10T00:18:54.858 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-10T00:18:54.858 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-10T00:18:54.858 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:18:54.858 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-10T00:18:54.859 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-10T00:18:54.859 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-10T00:18:54.859 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-10T00:18:54.859 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-10T00:18:54.859 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-10T00:18:54.859 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:18:54.859 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-10T00:18:54.859 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-10T00:18:54.859 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-10T00:18:54.859 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-10T00:18:54.859 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-10T00:18:54.859 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-10T00:18:54.860 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-10T00:18:54.860 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-10T00:18:54.860 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-10T00:18:54.966 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:18:54.994 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:18:55.025 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-10T00:18:55.090 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-10T00:18:55.116 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:18:55.264 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:18:55.299 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:18:55.319 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:18:56.143 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-10T00:18:56.206 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:18:56.237 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-10T00:18:56.243 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:18:56.278 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-10T00:18:56.360 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-10T00:18:56.391 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-10T00:18:56.415 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:18:56.441 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-10T00:18:56.467 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-10T00:18:56.467 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:18:56.552 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:18:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-10T00:18:56.585 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:18:56.669 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:18:56.694 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-10T00:18:56.699 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:18:56.699 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:18:56.880 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:18:56.880 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:18:56.964 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:18:56.989 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-10T00:18:57.052 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:18:57.259 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:18:57.294 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:18:57.403 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:18:57.580 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-10T00:18:57.612 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:18:57.637 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:18:57.637 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:57.637 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-10T00:18:57.637 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:18:57.637 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:18:57.637 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:18:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-10T00:18:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:18:57.689 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-06-10T00:18:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-10T00:18:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-10T00:18:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-10T00:18:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-10T00:18:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-10T00:18:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-10T00:18:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-10T00:18:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-10T00:18:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-10T00:18:57.691 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-06-10T00:18:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-10T00:18:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.2-8.el9.noarch 2024-06-10T00:18:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.7.4-5.el9.x86_64 2024-06-10T00:18:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-10T00:18:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:18:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:18:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:18:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:18:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:18:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:18:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:18:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:18:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:18:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:18:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-10T00:18:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.4-9.el9.x86_64 2024-06-10T00:18:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-10T00:18:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-10T00:18:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-10T00:18:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-10T00:18:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-10T00:18:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-10T00:18:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-10T00:18:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-10T00:18:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-10T00:18:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-10T00:18:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-10T00:18:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-10T00:18:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-10T00:18:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-10T00:18:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-10T00:18:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-10T00:18:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-10T00:18:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-10T00:18:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-10T00:18:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-10T00:18:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-06-10T00:18:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-10T00:18:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-10T00:18:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-10T00:18:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-10T00:18:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-10T00:18:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-10T00:18:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-13.14-1.el9.x86_64 2024-06-10T00:18:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-10T00:18:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-10T00:18:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-10T00:18:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-10T00:18:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-10T00:18:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-10T00:18:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-10T00:18:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-10T00:18:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-10T00:18:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-10T00:18:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-10T00:18:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-10T00:18:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-10T00:18:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-10T00:18:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-10T00:18:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-10T00:18:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-10T00:18:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-10T00:18:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-10T00:18:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-10T00:18:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-10T00:18:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-10T00:18:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-10T00:18:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-10T00:18:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-10T00:18:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-10T00:18:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-10T00:18:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-10T00:18:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-10T00:18:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-10T00:18:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-10T00:18:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-10T00:18:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-10T00:18:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-10T00:18:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-10T00:18:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-10T00:18:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-10T00:18:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-10T00:18:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-10T00:18:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-10T00:18:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-06-10T00:18:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-10T00:18:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-10T00:18:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-10T00:18:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-10T00:18:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-10T00:18:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-10T00:18:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-10T00:18:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-10T00:18:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-10T00:18:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-10T00:18:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-10T00:18:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-10T00:18:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-10T00:18:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-10T00:18:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-10T00:18:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-10T00:18:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-10T00:18:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-10T00:18:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-10T00:18:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-10T00:18:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-10T00:18:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-10T00:18:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-10T00:18:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-10T00:18:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-10T00:18:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-10T00:18:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-10T00:18:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:18:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-3.el9.x86_64 2024-06-10T00:18:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-10T00:18:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-10T00:18:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-10T00:18:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-10T00:18:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-10T00:18:57.704 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:18:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-10T00:18:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:18:57.704 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-06-10T00:18:57.708 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:18:57.809 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-10T00:18:57.857 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-10T00:18:58.165 DEBUG:teuthology.parallel:result is None 2024-06-10T00:18:58.800 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-10T00:18:58.832 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:18:58.857 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:18:58.857 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:58.857 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-10T00:18:58.857 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:18:58.857 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:18:58.857 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:18:58.908 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:18:58.932 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:18:58.932 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:58.932 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-10T00:18:58.932 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:18:58.984 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:18:59.007 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:18:59.008 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:59.008 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-10T00:18:59.008 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:18:59.008 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:18:59.008 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:18:59.326 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:18:59.350 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:18:59.350 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:18:59.350 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-10T00:18:59.350 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:18:59.350 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:18:59.350 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:19:02.806 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:19:02.806 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-10T00:19:02.806 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-10T00:19:02.806 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-10T00:19:02.806 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-10T00:19:02.807 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-10T00:19:02.807 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-10T00:19:02.807 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-10T00:19:02.807 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:19:02.807 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:19:02.807 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:19:02.807 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-10T00:19:02.807 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-10T00:19:02.807 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-10T00:19:02.807 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:19:02.807 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-10T00:19:02.807 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-10T00:19:02.807 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-10T00:19:02.808 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-10T00:19:02.808 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-10T00:19:02.808 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-10T00:19:02.808 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-10T00:19:02.808 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-10T00:19:02.808 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-10T00:19:02.808 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-10T00:19:02.808 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-10T00:19:02.808 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-10T00:19:02.808 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-10T00:19:02.808 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-10T00:19:02.808 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-10T00:19:02.809 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-10T00:19:02.809 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-10T00:19:02.809 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-10T00:19:02.809 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-10T00:19:02.809 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-10T00:19:02.809 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-10T00:19:02.809 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-10T00:19:02.809 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-10T00:19:02.809 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-10T00:19:02.809 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-10T00:19:02.809 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-10T00:19:02.809 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-10T00:19:02.809 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-10T00:19:02.810 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-10T00:19:02.810 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-10T00:19:02.810 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-10T00:19:02.810 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-10T00:19:02.810 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-10T00:19:02.810 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-10T00:19:02.810 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-10T00:19:02.810 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-10T00:19:02.810 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-10T00:19:02.810 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-10T00:19:02.810 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-10T00:19:02.810 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-10T00:19:02.811 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-10T00:19:02.811 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-10T00:19:02.811 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-10T00:19:02.811 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-10T00:19:02.811 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-10T00:19:02.811 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-10T00:19:02.811 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-10T00:19:02.811 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-10T00:19:02.811 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-10T00:19:02.811 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-10T00:19:02.811 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-10T00:19:02.811 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-10T00:19:02.811 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-10T00:19:02.812 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-10T00:19:02.812 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-10T00:19:02.812 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-10T00:19:02.812 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-10T00:19:02.812 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-10T00:19:02.812 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-10T00:19:02.812 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-10T00:19:02.812 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-10T00:19:02.812 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-10T00:19:02.812 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-10T00:19:02.812 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-10T00:19:02.813 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-10T00:19:02.813 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-10T00:19:02.813 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-10T00:19:02.813 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-10T00:19:02.813 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-10T00:19:02.813 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-10T00:19:02.813 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-10T00:19:02.813 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-10T00:19:02.813 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-10T00:19:02.813 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-10T00:19:02.813 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-10T00:19:02.813 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-10T00:19:02.814 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-10T00:19:02.814 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-10T00:19:02.814 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-10T00:19:02.814 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-10T00:19:02.814 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-10T00:19:02.814 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-10T00:19:02.814 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-10T00:19:02.814 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-10T00:19:02.814 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-10T00:19:02.814 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-10T00:19:02.814 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-10T00:19:02.815 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-10T00:19:02.815 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-10T00:19:02.815 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-10T00:19:02.815 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-10T00:19:02.815 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-10T00:19:02.815 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-10T00:19:02.815 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-10T00:19:02.815 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-10T00:19:02.815 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-10T00:19:02.815 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-10T00:19:02.815 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-10T00:19:02.815 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-10T00:19:02.815 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-10T00:19:02.816 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-10T00:19:02.816 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-10T00:19:02.816 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-10T00:19:02.816 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-10T00:19:02.816 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-10T00:19:02.816 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-10T00:19:02.816 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-10T00:19:02.816 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/170 2024-06-10T00:19:02.816 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-10T00:19:02.816 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-10T00:19:02.816 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-10T00:19:02.816 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-10T00:19:02.816 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-10T00:19:02.817 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-10T00:19:02.817 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-10T00:19:02.817 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-10T00:19:02.817 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-10T00:19:02.817 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-10T00:19:02.817 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-10T00:19:02.817 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-10T00:19:02.817 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-10T00:19:02.817 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-10T00:19:02.817 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-10T00:19:02.817 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-10T00:19:02.817 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-10T00:19:02.817 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-10T00:19:02.818 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-10T00:19:02.818 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-10T00:19:02.818 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-10T00:19:02.818 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-10T00:19:02.818 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-10T00:19:02.818 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-10T00:19:02.818 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-10T00:19:02.818 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-10T00:19:02.818 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-10T00:19:02.818 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-10T00:19:02.818 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-10T00:19:02.818 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-10T00:19:02.819 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:19:02.819 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-10T00:19:02.819 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-10T00:19:02.819 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-10T00:19:02.819 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-10T00:19:02.819 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-10T00:19:02.819 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-10T00:19:02.819 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:19:02.819 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-10T00:19:02.819 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-10T00:19:02.819 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-10T00:19:02.819 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-10T00:19:02.819 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-10T00:19:02.820 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-10T00:19:02.820 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-10T00:19:02.820 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-10T00:19:02.820 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-10T00:19:04.114 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-10T00:19:04.155 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-10T00:19:04.196 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-10T00:19:04.228 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-10T00:19:04.255 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-10T00:19:04.451 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-10T00:19:04.492 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-10T00:19:04.534 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-10T00:19:04.573 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-10T00:19:04.603 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-10T00:19:04.650 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:19:05.309 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:19:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:19:05.360 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:19:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-10T00:19:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:19:05.772 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2024-06-10T00:19:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-10T00:19:05.772 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-10T00:19:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-10T00:19:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-10T00:19:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-10T00:19:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-10T00:19:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-10T00:19:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-10T00:19:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-10T00:19:05.773 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-06-10T00:19:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-10T00:19:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.2-8.el9.noarch 2024-06-10T00:19:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.7.4-5.el9.x86_64 2024-06-10T00:19:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-10T00:19:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:05.774 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.775 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-10T00:19:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.4-9.el9.x86_64 2024-06-10T00:19:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-10T00:19:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-10T00:19:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-10T00:19:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-10T00:19:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-10T00:19:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-10T00:19:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-10T00:19:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-10T00:19:05.776 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-10T00:19:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-10T00:19:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-10T00:19:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-10T00:19:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-10T00:19:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-10T00:19:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-10T00:19:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.777 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-10T00:19:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-10T00:19:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-10T00:19:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-10T00:19:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-10T00:19:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: lua-5.4.4-4.el9.x86_64 2024-06-10T00:19:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-10T00:19:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-10T00:19:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-10T00:19:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-10T00:19:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-10T00:19:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-10T00:19:05.778 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-13.14-1.el9.x86_64 2024-06-10T00:19:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-10T00:19:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-10T00:19:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-10T00:19:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-10T00:19:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-10T00:19:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-10T00:19:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-10T00:19:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-10T00:19:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-10T00:19:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-10T00:19:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.779 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-10T00:19:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-10T00:19:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-10T00:19:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-10T00:19:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-10T00:19:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-10T00:19:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-10T00:19:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-10T00:19:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-10T00:19:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-10T00:19:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-10T00:19:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-10T00:19:05.780 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-10T00:19:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-10T00:19:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-10T00:19:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-10T00:19:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-10T00:19:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-10T00:19:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-10T00:19:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-10T00:19:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-10T00:19:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-10T00:19:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-10T00:19:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-10T00:19:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-10T00:19:05.781 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-10T00:19:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-10T00:19:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-10T00:19:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-10T00:19:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-10T00:19:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply-3.11-14.el9.noarch 2024-06-10T00:19:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-10T00:19:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-10T00:19:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-10T00:19:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-10T00:19:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-10T00:19:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-10T00:19:05.782 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-10T00:19:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-10T00:19:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-10T00:19:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-10T00:19:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-10T00:19:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-10T00:19:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-10T00:19:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-10T00:19:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-10T00:19:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-10T00:19:05.783 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-10T00:19:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-10T00:19:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-10T00:19:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-10T00:19:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-10T00:19:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-10T00:19:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-10T00:19:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-10T00:19:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-10T00:19:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-10T00:19:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-10T00:19:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:05.784 INFO:teuthology.orchestra.run.smithi105.stdout: re2-1:20211101-3.el9.x86_64 2024-06-10T00:19:05.785 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-10T00:19:05.785 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-10T00:19:05.785 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-10T00:19:05.785 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-10T00:19:05.785 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-10T00:19:05.786 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-10T00:19:05.786 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-06-10T00:19:05.786 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-06-10T00:19:06.217 DEBUG:teuthology.parallel:result is None 2024-06-10T00:19:06.268 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-10T00:19:06.368 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-10T00:19:06.411 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-10T00:19:06.501 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-10T00:19:06.538 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-10T00:19:06.563 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:19:06.588 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-10T00:19:06.615 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-10T00:19:06.615 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:19:06.683 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:19:06.716 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-10T00:19:06.716 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:19:06.800 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:19:06.833 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-10T00:19:06.839 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:19:06.840 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:19:07.036 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:19:07.036 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:19:07.120 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:19:07.146 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-10T00:19:07.214 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:19:12.665 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:19:12.666 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-10T00:19:12.666 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-10T00:19:12.666 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-10T00:19:12.666 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-10T00:19:12.666 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-10T00:19:12.666 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-10T00:19:12.666 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-10T00:19:12.666 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:19:12.666 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:19:12.666 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:19:12.666 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-10T00:19:12.667 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-10T00:19:12.667 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-10T00:19:12.667 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:19:12.667 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-10T00:19:12.667 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-10T00:19:12.667 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-10T00:19:12.667 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-10T00:19:12.667 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-10T00:19:12.667 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-10T00:19:12.667 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-10T00:19:12.667 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-10T00:19:12.667 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-10T00:19:12.667 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-10T00:19:12.668 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-10T00:19:12.668 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-10T00:19:12.668 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-10T00:19:12.668 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-10T00:19:12.668 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-10T00:19:12.668 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-10T00:19:12.668 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-10T00:19:12.668 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-10T00:19:12.668 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-10T00:19:12.668 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-10T00:19:12.668 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-10T00:19:12.668 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-10T00:19:12.668 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-10T00:19:12.669 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-10T00:19:12.669 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-10T00:19:12.669 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-10T00:19:12.669 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-10T00:19:12.669 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-10T00:19:12.669 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-10T00:19:12.669 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-10T00:19:12.669 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-10T00:19:12.669 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-10T00:19:12.669 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-10T00:19:12.669 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-10T00:19:12.669 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-10T00:19:12.669 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-10T00:19:12.670 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-10T00:19:12.670 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-10T00:19:12.670 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-10T00:19:12.670 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-10T00:19:12.670 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-10T00:19:12.670 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-10T00:19:12.670 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-10T00:19:12.670 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-10T00:19:12.670 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-10T00:19:12.670 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-10T00:19:12.670 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-10T00:19:12.670 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-10T00:19:12.671 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-10T00:19:12.671 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-10T00:19:12.671 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-10T00:19:12.671 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-10T00:19:12.671 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-10T00:19:12.671 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-10T00:19:12.671 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-10T00:19:12.671 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-10T00:19:12.671 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-10T00:19:12.671 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-10T00:19:12.671 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-10T00:19:12.671 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-10T00:19:12.671 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-10T00:19:12.672 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-10T00:19:12.672 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-10T00:19:12.672 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-10T00:19:12.672 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-10T00:19:12.672 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-10T00:19:12.672 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-10T00:19:12.672 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-10T00:19:12.672 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-10T00:19:12.672 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-10T00:19:12.672 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-10T00:19:12.672 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-10T00:19:12.672 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-10T00:19:12.672 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-10T00:19:12.673 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-10T00:19:12.673 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-10T00:19:12.673 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-10T00:19:12.673 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-10T00:19:12.673 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-10T00:19:12.673 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-10T00:19:12.673 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-10T00:19:12.673 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-10T00:19:12.673 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-10T00:19:12.673 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-10T00:19:12.673 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-10T00:19:12.673 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-10T00:19:12.673 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-10T00:19:12.673 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-10T00:19:12.674 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-10T00:19:12.674 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-10T00:19:12.674 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-10T00:19:12.674 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-10T00:19:12.674 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-10T00:19:12.674 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-10T00:19:12.674 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-10T00:19:12.674 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-10T00:19:12.674 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-10T00:19:12.674 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-10T00:19:12.674 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-10T00:19:12.674 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-10T00:19:12.675 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-10T00:19:12.675 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-10T00:19:12.675 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-10T00:19:12.675 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-10T00:19:12.675 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-10T00:19:12.675 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-10T00:19:12.675 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-10T00:19:12.675 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/170 2024-06-10T00:19:12.675 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-10T00:19:12.675 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-10T00:19:12.675 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-10T00:19:12.675 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-10T00:19:12.675 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-10T00:19:12.675 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-10T00:19:12.676 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-10T00:19:12.676 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-10T00:19:12.676 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-10T00:19:12.676 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-10T00:19:12.676 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-10T00:19:12.676 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-10T00:19:12.676 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-10T00:19:12.676 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-10T00:19:12.676 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-10T00:19:12.676 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-10T00:19:12.676 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-10T00:19:12.676 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-10T00:19:12.676 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-10T00:19:12.676 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-10T00:19:12.677 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-10T00:19:12.677 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-10T00:19:12.677 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-10T00:19:12.677 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-10T00:19:12.677 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-10T00:19:12.677 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-10T00:19:12.677 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-10T00:19:12.677 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-10T00:19:12.677 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-10T00:19:12.677 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-10T00:19:12.677 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:19:12.677 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-10T00:19:12.677 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-10T00:19:12.677 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-10T00:19:12.678 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-10T00:19:12.678 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-10T00:19:12.678 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-10T00:19:12.678 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:19:12.678 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-10T00:19:12.678 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-10T00:19:12.678 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-10T00:19:12.678 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-10T00:19:12.678 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-10T00:19:12.678 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-10T00:19:12.678 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-10T00:19:12.678 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-10T00:19:12.679 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-10T00:19:15.432 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-10T00:19:15.432 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:19:15.432 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-06-10T00:19:15.432 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-10T00:19:15.432 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-10T00:19:15.432 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-10T00:19:15.432 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-10T00:19:15.432 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-10T00:19:15.432 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-10T00:19:15.432 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-10T00:19:15.433 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.433 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.433 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-10T00:19:15.433 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-10T00:19:15.433 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-06-10T00:19:15.433 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-10T00:19:15.433 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.2-8.el9.noarch 2024-06-10T00:19:15.433 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.7.4-5.el9.x86_64 2024-06-10T00:19:15.433 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-10T00:19:15.433 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.433 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.433 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.433 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.434 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:15.434 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.434 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.434 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.434 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:15.434 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:15.434 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:15.434 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:15.434 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:15.434 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.434 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.434 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:15.435 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.435 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.435 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.435 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:15.435 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:15.435 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.435 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.435 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:19:15.435 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-10T00:19:15.435 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.4-9.el9.x86_64 2024-06-10T00:19:15.435 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-10T00:19:15.435 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-10T00:19:15.436 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-10T00:19:15.436 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-10T00:19:15.436 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-10T00:19:15.436 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-10T00:19:15.436 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-10T00:19:15.436 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-10T00:19:15.436 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-10T00:19:15.436 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.436 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.436 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.436 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-10T00:19:15.436 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-10T00:19:15.437 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-10T00:19:15.437 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-10T00:19:15.437 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-10T00:19:15.437 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.437 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.437 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-10T00:19:15.437 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.437 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-10T00:19:15.437 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-10T00:19:15.437 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-10T00:19:15.437 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-10T00:19:15.437 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-10T00:19:15.437 INFO:teuthology.orchestra.run.smithi062.stdout: lua-5.4.4-4.el9.x86_64 2024-06-10T00:19:15.438 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-10T00:19:15.438 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-10T00:19:15.438 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-10T00:19:15.438 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-10T00:19:15.438 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-10T00:19:15.438 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-10T00:19:15.438 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-13.14-1.el9.x86_64 2024-06-10T00:19:15.438 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-10T00:19:15.438 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-10T00:19:15.438 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-10T00:19:15.438 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-10T00:19:15.438 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-10T00:19:15.438 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-10T00:19:15.439 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-10T00:19:15.439 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-10T00:19:15.439 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-10T00:19:15.439 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-10T00:19:15.439 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.439 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.439 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.439 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-10T00:19:15.439 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-10T00:19:15.439 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-10T00:19:15.439 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-10T00:19:15.439 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-10T00:19:15.439 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-10T00:19:15.440 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-10T00:19:15.440 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-10T00:19:15.440 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-10T00:19:15.440 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-10T00:19:15.440 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-10T00:19:15.440 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-10T00:19:15.440 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-10T00:19:15.440 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-10T00:19:15.440 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-10T00:19:15.440 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-10T00:19:15.440 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-10T00:19:15.440 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-10T00:19:15.440 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-10T00:19:15.440 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-10T00:19:15.441 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-10T00:19:15.441 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-10T00:19:15.441 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-10T00:19:15.441 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-10T00:19:15.441 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-10T00:19:15.441 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-10T00:19:15.441 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-10T00:19:15.441 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-10T00:19:15.441 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-10T00:19:15.441 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-10T00:19:15.441 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply-3.11-14.el9.noarch 2024-06-10T00:19:15.441 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-10T00:19:15.441 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-10T00:19:15.441 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-10T00:19:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-10T00:19:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-10T00:19:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-10T00:19:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-10T00:19:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-10T00:19:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-10T00:19:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-10T00:19:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-10T00:19:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-10T00:19:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-10T00:19:15.442 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-10T00:19:15.443 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-10T00:19:15.443 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-10T00:19:15.443 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-10T00:19:15.443 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-10T00:19:15.443 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-10T00:19:15.443 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-10T00:19:15.443 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-10T00:19:15.443 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-10T00:19:15.443 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-10T00:19:15.443 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-10T00:19:15.443 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-10T00:19:15.443 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-10T00:19:15.443 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-10T00:19:15.443 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.444 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.444 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:19:15.444 INFO:teuthology.orchestra.run.smithi062.stdout: re2-1:20211101-3.el9.x86_64 2024-06-10T00:19:15.444 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-10T00:19:15.444 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-10T00:19:15.444 INFO:teuthology.orchestra.run.smithi062.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-10T00:19:15.444 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-10T00:19:15.444 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-10T00:19:15.444 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-10T00:19:15.444 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:19:15.445 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-06-10T00:19:15.969 DEBUG:teuthology.parallel:result is None 2024-06-10T00:19:15.970 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:19:16.194 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-10T00:19:16.217 INFO:teuthology.orchestra.run.smithi062.stdout:19.0.0-2946.g3d89accf.el9 2024-06-10T00:19:16.218 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-10T00:19:16.218 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-10T00:19:16.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:19:16.431 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-10T00:19:16.456 INFO:teuthology.orchestra.run.smithi105.stdout:19.0.0-2946.g3d89accf.el9 2024-06-10T00:19:16.457 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-10T00:19:16.457 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-10T00:19:16.459 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:19:16.593 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-10T00:19:16.618 INFO:teuthology.orchestra.run.smithi138.stdout:19.0.0-2946.g3d89accf.el9 2024-06-10T00:19:16.619 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-10T00:19:16.619 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-10T00:19:16.621 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-10T00:19:16.621 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:19:16.621 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-10T00:19:16.653 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-06-10T00:19:16.653 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-10T00:19:16.684 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-06-10T00:19:16.684 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-10T00:19:16.714 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-10T00:19:16.714 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:19:16.714 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-06-10T00:19:16.743 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-10T00:19:16.812 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-06-10T00:19:16.812 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-06-10T00:19:16.839 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-10T00:19:16.906 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-06-10T00:19:16.906 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-06-10T00:19:16.935 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-10T00:19:17.003 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-10T00:19:17.003 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:19:17.003 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-10T00:19:17.033 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-10T00:19:17.102 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-06-10T00:19:17.102 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-10T00:19:17.130 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-10T00:19:17.196 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-06-10T00:19:17.196 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-10T00:19:17.220 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-10T00:19:17.284 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-10T00:19:17.284 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:19:17.284 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-06-10T00:19:17.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-10T00:19:17.386 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-06-10T00:19:17.386 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-06-10T00:19:17.412 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-10T00:19:17.478 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-06-10T00:19:17.479 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-06-10T00:19:17.502 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-10T00:19:17.566 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-10T00:19:17.662 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-06-10T00:19:17.662 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:19:17.663 INFO:tasks.cephadm:Cluster fsid is 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:19:17.663 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-10T00:19:17.664 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.62', 'mon.b': '172.21.15.105', 'mon.c': '172.21.15.138'} 2024-06-10T00:19:17.664 INFO:tasks.cephadm:First mon is mon.a on smithi062 2024-06-10T00:19:17.664 INFO:tasks.cephadm:First mgr is x 2024-06-10T00:19:17.664 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-10T00:19:17.664 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname $(hostname -s) 2024-06-10T00:19:17.695 DEBUG:teuthology.orchestra.run.smithi105:> sudo hostname $(hostname -s) 2024-06-10T00:19:17.722 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname $(hostname -s) 2024-06-10T00:19:17.746 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-10T00:19:17.746 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:19:17.962 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-10T00:19:18.173 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:19:18.174 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-10T00:19:18.174 DEBUG:teuthology.orchestra.run.smithi062:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-10T00:19:18.706 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 10 00:19 /home/ubuntu/cephtest/cephadm 2024-06-10T00:19:18.707 DEBUG:teuthology.orchestra.run.smithi105:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-10T00:19:19.220 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 10 00:19 /home/ubuntu/cephtest/cephadm 2024-06-10T00:19:19.220 DEBUG:teuthology.orchestra.run.smithi138:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-10T00:19:19.661 INFO:teuthology.orchestra.run.smithi138.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 10 00:19 /home/ubuntu/cephtest/cephadm 2024-06-10T00:19:19.661 DEBUG:teuthology.orchestra.run.smithi062:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-10T00:19:19.682 DEBUG:teuthology.orchestra.run.smithi105:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-10T00:19:19.697 DEBUG:teuthology.orchestra.run.smithi138:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-10T00:19:19.732 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-10T00:19:19.732 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-10T00:19:19.735 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-10T00:19:19.740 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-10T00:19:20.574 INFO:teuthology.orchestra.run.smithi138.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:19:20.627 INFO:teuthology.orchestra.run.smithi062.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:19:20.627 INFO:teuthology.orchestra.run.smithi105.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:19:48.718 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:19:48.719 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T00:19:48.719 INFO:teuthology.orchestra.run.smithi062.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-10T00:19:48.719 INFO:teuthology.orchestra.run.smithi062.stdout: "repo_digests": [ 2024-06-10T00:19:48.719 INFO:teuthology.orchestra.run.smithi062.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-10T00:19:48.719 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-06-10T00:19:48.719 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:20:00.584 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-06-10T00:20:00.585 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T00:20:00.585 INFO:teuthology.orchestra.run.smithi138.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-10T00:20:00.585 INFO:teuthology.orchestra.run.smithi138.stdout: "repo_digests": [ 2024-06-10T00:20:00.585 INFO:teuthology.orchestra.run.smithi138.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-10T00:20:00.585 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-06-10T00:20:00.585 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-06-10T00:20:13.127 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-06-10T00:20:13.128 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T00:20:13.128 INFO:teuthology.orchestra.run.smithi105.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-10T00:20:13.128 INFO:teuthology.orchestra.run.smithi105.stdout: "repo_digests": [ 2024-06-10T00:20:13.128 INFO:teuthology.orchestra.run.smithi105.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-10T00:20:13.128 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-06-10T00:20:13.128 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-06-10T00:20:13.156 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph 2024-06-10T00:20:13.191 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph 2024-06-10T00:20:13.219 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph 2024-06-10T00:20:13.251 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /etc/ceph 2024-06-10T00:20:13.282 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /etc/ceph 2024-06-10T00:20:13.307 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /etc/ceph 2024-06-10T00:20:13.336 INFO:tasks.cephadm:Writing seed config... 2024-06-10T00:20:13.337 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-10T00:20:13.337 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-10T00:20:13.337 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-10T00:20:13.337 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-10T00:20:13.337 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-10T00:20:13.337 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-06-10T00:20:13.337 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-10T00:20:13.338 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-10T00:20:13.338 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-10T00:20:13.338 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-10T00:20:13.338 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-10T00:20:13.338 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-10T00:20:13.338 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-10T00:20:13.338 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-10T00:20:13.338 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-10T00:20:13.338 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-10T00:20:13.338 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-10T00:20:13.338 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-06-10T00:20:13.338 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-10T00:20:13.338 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-10T00:20:13.339 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-10T00:20:13.339 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-10T00:20:13.339 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-06-10T00:20:13.339 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-10T00:20:13.339 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-10T00:20:13.339 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-10T00:20:13.339 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-10T00:20:13.339 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-10T00:20:13.339 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-10T00:20:13.339 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-10T00:20:13.339 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-10T00:20:13.339 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-10T00:20:13.339 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-10T00:20:13.339 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-10T00:20:13.340 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-10T00:20:13.340 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-10T00:20:13.340 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-10T00:20:13.340 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-10T00:20:13.340 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-06-10T00:20:13.340 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-10T00:20:13.340 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-10T00:20:13.340 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-10T00:20:13.340 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-10T00:20:13.340 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-06-10T00:20:13.340 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-10T00:20:13.340 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-10T00:20:13.340 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-10T00:20:13.340 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-06-10T00:20:13.341 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:20:13.341 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-10T00:20:13.357 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1353b2ce-26bf-11ef-bc9b-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-10T00:20:13.358 DEBUG:teuthology.orchestra.run.smithi062:mon.a> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mon.a.service 2024-06-10T00:20:13.400 DEBUG:teuthology.orchestra.run.smithi062:mgr.x> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-10T00:20:13.442 INFO:tasks.cephadm:Bootstrapping... 2024-06-10T00:20:13.442 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.62 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:20:13.641 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:20:13.642 INFO:teuthology.orchestra.run.smithi062.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', '1353b2ce-26bf-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.62', '--skip-admin-label'] 2024-06-10T00:20:13.642 INFO:teuthology.orchestra.run.smithi062.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-10T00:20:13.642 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying podman|docker is present... 2024-06-10T00:20:13.671 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 5.0.2 2024-06-10T00:20:13.671 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying lvm2 is present... 2024-06-10T00:20:13.671 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying time synchronization is in place... 2024-06-10T00:20:13.679 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-10T00:20:13.679 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-10T00:20:13.686 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-10T00:20:13.686 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-06-10T00:20:13.694 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout enabled 2024-06-10T00:20:13.701 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout active 2024-06-10T00:20:13.701 INFO:teuthology.orchestra.run.smithi062.stdout:Unit chronyd.service is enabled and running 2024-06-10T00:20:13.701 INFO:teuthology.orchestra.run.smithi062.stdout:Repeating the final host check... 2024-06-10T00:20:13.726 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 5.0.2 2024-06-10T00:20:13.727 INFO:teuthology.orchestra.run.smithi062.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-10T00:20:13.727 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl is present 2024-06-10T00:20:13.727 INFO:teuthology.orchestra.run.smithi062.stdout:lvcreate is present 2024-06-10T00:20:13.734 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-10T00:20:13.734 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-10T00:20:13.740 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-10T00:20:13.740 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-06-10T00:20:13.747 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout enabled 2024-06-10T00:20:13.755 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout active 2024-06-10T00:20:13.755 INFO:teuthology.orchestra.run.smithi062.stdout:Unit chronyd.service is enabled and running 2024-06-10T00:20:13.755 INFO:teuthology.orchestra.run.smithi062.stdout:Host looks OK 2024-06-10T00:20:13.755 INFO:teuthology.orchestra.run.smithi062.stdout:Cluster fsid: 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:20:13.755 INFO:teuthology.orchestra.run.smithi062.stdout:Acquiring lock 140189086205312 on /run/cephadm/1353b2ce-26bf-11ef-bc9b-c7b262605968.lock 2024-06-10T00:20:13.755 INFO:teuthology.orchestra.run.smithi062.stdout:Lock 140189086205312 acquired on /run/cephadm/1353b2ce-26bf-11ef-bc9b-c7b262605968.lock 2024-06-10T00:20:13.756 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying IP 172.21.15.62 port 3300 ... 2024-06-10T00:20:13.756 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying IP 172.21.15.62 port 6789 ... 2024-06-10T00:20:13.756 INFO:teuthology.orchestra.run.smithi062.stdout:Base mon IP(s) is [172.21.15.62:3300, 172.21.15.62:6789], mon addrv is [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2024-06-10T00:20:13.759 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.62 metric 100 2024-06-10T00:20:13.760 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.62 metric 100 2024-06-10T00:20:13.763 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-10T00:20:13.763 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-06-10T00:20:13.763 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1798sec hoplimit 64 pref medium 2024-06-10T00:20:13.765 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-10T00:20:13.766 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-10T00:20:13.766 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-10T00:20:13.766 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-06-10T00:20:13.766 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-06-10T00:20:13.766 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-10T00:20:13.767 INFO:teuthology.orchestra.run.smithi062.stdout:Mon IP `172.21.15.62` is in CIDR network `172.21.0.0/20` 2024-06-10T00:20:13.767 INFO:teuthology.orchestra.run.smithi062.stdout:Mon IP `172.21.15.62` is in CIDR network `172.21.0.0/20` 2024-06-10T00:20:13.767 INFO:teuthology.orchestra.run.smithi062.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-10T00:20:13.767 INFO:teuthology.orchestra.run.smithi062.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-10T00:20:13.768 INFO:teuthology.orchestra.run.smithi062.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:20:17.466 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-10T00:20:17.466 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:20:17.466 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Getting image source signatures 2024-06-10T00:20:17.466 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-10T00:20:17.466 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-10T00:20:17.466 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-10T00:20:17.467 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-10T00:20:20.361 INFO:teuthology.orchestra.run.smithi062.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-10T00:20:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-10T00:20:20.362 INFO:teuthology.orchestra.run.smithi062.stdout:Extracting ceph user uid/gid from container image... 2024-06-10T00:20:21.349 INFO:teuthology.orchestra.run.smithi062.stdout:stat: stdout 167 167 2024-06-10T00:20:21.350 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial keys... 2024-06-10T00:20:22.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQDFRmZmaETwKRAALS/9+i2HSI/+NSkb95tK/A== 2024-06-10T00:20:23.287 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQDGRmZmzBjaKhAA/sSEyypn/Ui23lRykIMMmw== 2024-06-10T00:20:24.134 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQDHRmZmSUqxIhAAwty1Qcs0Ot1o3uQOlkFDNg== 2024-06-10T00:20:24.135 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial monmap... 2024-06-10T00:20:25.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-10T00:20:25.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-10T00:20:25.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:20:25.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-10T00:20:25.466 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool for a [v2:172.21.15.62:3300,v1:172.21.15.62:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-10T00:20:25.466 INFO:teuthology.orchestra.run.smithi062.stdout:setting min_mon_release = quincy 2024-06-10T00:20:25.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: set fsid to 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:20:25.466 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-10T00:20:25.467 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:20:25.467 INFO:teuthology.orchestra.run.smithi062.stdout:Creating mon... 2024-06-10T00:20:26.412 INFO:teuthology.orchestra.run.smithi062.stdout:create mon.a on 2024-06-10T00:20:26.628 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-10T00:20:26.788 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-10T00:20:26.976 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968.target. 2024-06-10T00:20:26.977 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968.target. 2024-06-10T00:20:27.312 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mon.a 2024-06-10T00:20:27.312 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-10T00:20:27.483 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968.target.wants/ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@.service. 2024-06-10T00:20:28.054 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T00:20:28.054 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-06-10T00:20:28.063 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T00:20:28.063 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-06-10T00:20:28.063 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-06-10T00:20:28.063 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to enable service . firewalld.service is not available 2024-06-10T00:20:28.063 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mon to start... 2024-06-10T00:20:28.063 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mon... 2024-06-10T00:20:29.205 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout cluster: 2024-06-10T00:20:29.205 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout id: 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:20:29.205 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-10T00:20:29.205 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-06-10T00:20:29.205 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout services: 2024-06-10T00:20:29.205 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.639715s) 2024-06-10T00:20:29.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-10T00:20:29.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-10T00:20:29.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-06-10T00:20:29.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout data: 2024-06-10T00:20:29.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-10T00:20:29.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-10T00:20:29.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-10T00:20:29.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout pgs: 2024-06-10T00:20:29.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-06-10T00:20:29.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.537+0000 7f4412b58700 1 Processor -- start 2024-06-10T00:20:29.206 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.537+0000 7f4412b58700 1 -- start start 2024-06-10T00:20:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.537+0000 7f4412b58700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c107fb0 0x7f440c1083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.537+0000 7f4412b58700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1089a0 con 0x7f440c107fb0 2024-06-10T00:20:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.538+0000 7f44108f4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c107fb0 0x7f440c1083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.538+0000 7f44108f4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c107fb0 0x7f440c1083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38574/0 (socket says 172.21.15.62:38574) 2024-06-10T00:20:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.538+0000 7f44108f4700 1 -- 172.21.15.62:0/2572441153 learned_addr learned my addr 172.21.15.62:0/2572441153 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.538+0000 7f44108f4700 1 -- 172.21.15.62:0/2572441153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440c1091b0 con 0x7f440c107fb0 2024-06-10T00:20:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.538+0000 7f44108f4700 1 --2- 172.21.15.62:0/2572441153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c107fb0 0x7f440c1083d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009e80 tx=0x7f43fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115dabfe74174688 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.539+0000 7f440b7fe700 1 -- 172.21.15.62:0/2572441153 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43fc017070 con 0x7f440c107fb0 2024-06-10T00:20:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.539+0000 7f440b7fe700 1 -- 172.21.15.62:0/2572441153 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43fc003a20 con 0x7f440c107fb0 2024-06-10T00:20:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.539+0000 7f440b7fe700 1 -- 172.21.15.62:0/2572441153 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43fc003d20 con 0x7f440c107fb0 2024-06-10T00:20:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.539+0000 7f4412b58700 1 -- 172.21.15.62:0/2572441153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c107fb0 msgr2=0x7f440c1083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.539+0000 7f4412b58700 1 --2- 172.21.15.62:0/2572441153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c107fb0 0x7f440c1083d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009e80 tx=0x7f43fc00b330 comp rx=0 tx=0).stop 2024-06-10T00:20:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.539+0000 7f4412b58700 1 -- 172.21.15.62:0/2572441153 shutdown_connections 2024-06-10T00:20:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.539+0000 7f4412b58700 1 --2- 172.21.15.62:0/2572441153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c107fb0 0x7f440c1083d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.539+0000 7f4412b58700 1 -- 172.21.15.62:0/2572441153 >> 172.21.15.62:0/2572441153 conn(0x7f440c1035b0 msgr2=0x7f440c1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.539+0000 7f4412b58700 1 -- 172.21.15.62:0/2572441153 shutdown_connections 2024-06-10T00:20:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.539+0000 7f4412b58700 1 -- 172.21.15.62:0/2572441153 wait complete. 2024-06-10T00:20:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.540+0000 7f4412b58700 1 Processor -- start 2024-06-10T00:20:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.540+0000 7f4412b58700 1 -- start start 2024-06-10T00:20:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.540+0000 7f4412b58700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c107fb0 0x7f440c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.540+0000 7f4412b58700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1096e0 con 0x7f440c107fb0 2024-06-10T00:20:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.540+0000 7f44108f4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c107fb0 0x7f440c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.540+0000 7f44108f4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c107fb0 0x7f440c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38584/0 (socket says 172.21.15.62:38584) 2024-06-10T00:20:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.540+0000 7f44108f4700 1 -- 172.21.15.62:0/1469468400 learned_addr learned my addr 172.21.15.62:0/1469468400 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.540+0000 7f44108f4700 1 -- 172.21.15.62:0/1469468400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440c07e0b0 con 0x7f440c107fb0 2024-06-10T00:20:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.541+0000 7f44108f4700 1 --2- 172.21.15.62:0/1469468400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c107fb0 0x7f440c07db70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f43fc004000 tx=0x7f43fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.541+0000 7f4409ffb700 1 -- 172.21.15.62:0/1469468400 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43fc022020 con 0x7f440c107fb0 2024-06-10T00:20:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.541+0000 7f4409ffb700 1 -- 172.21.15.62:0/1469468400 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43fc018040 con 0x7f440c107fb0 2024-06-10T00:20:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.541+0000 7f4412b58700 1 -- 172.21.15.62:0/1469468400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440c07c2a0 con 0x7f440c107fb0 2024-06-10T00:20:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.541+0000 7f4409ffb700 1 -- 172.21.15.62:0/1469468400 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43fc017070 con 0x7f440c107fb0 2024-06-10T00:20:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.541+0000 7f4412b58700 1 -- 172.21.15.62:0/1469468400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f440c07c7e0 con 0x7f440c107fb0 2024-06-10T00:20:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.541+0000 7f4409ffb700 1 -- 172.21.15.62:0/1469468400 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f43fc025020 con 0x7f440c107fb0 2024-06-10T00:20:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.541+0000 7f4409ffb700 1 -- 172.21.15.62:0/1469468400 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f43fc00f050 con 0x7f440c107fb0 2024-06-10T00:20:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.542+0000 7f4412b58700 1 -- 172.21.15.62:0/1469468400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f440c063af0 con 0x7f440c107fb0 2024-06-10T00:20:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.544+0000 7f4409ffb700 1 -- 172.21.15.62:0/1469468400 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f43fc02c030 con 0x7f440c107fb0 2024-06-10T00:20:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.617+0000 7f4412b58700 1 -- 172.21.15.62:0/1469468400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f440c194f40 con 0x7f440c107fb0 2024-06-10T00:20:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.617+0000 7f4409ffb700 1 -- 172.21.15.62:0/1469468400 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f43fc01e480 con 0x7f440c107fb0 2024-06-10T00:20:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.618+0000 7f4412b58700 1 -- 172.21.15.62:0/1469468400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c107fb0 msgr2=0x7f440c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.618+0000 7f4412b58700 1 --2- 172.21.15.62:0/1469468400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c107fb0 0x7f440c07db70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f43fc004000 tx=0x7f43fc004ad0 comp rx=0 tx=0).stop 2024-06-10T00:20:29.210 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.618+0000 7f4412b58700 1 -- 172.21.15.62:0/1469468400 shutdown_connections 2024-06-10T00:20:29.210 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.618+0000 7f4412b58700 1 --2- 172.21.15.62:0/1469468400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f440c107fb0 0x7f440c07db70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:29.210 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.618+0000 7f4412b58700 1 -- 172.21.15.62:0/1469468400 >> 172.21.15.62:0/1469468400 conn(0x7f440c1035b0 msgr2=0x7f440c1923b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:29.210 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.618+0000 7f4412b58700 1 -- 172.21.15.62:0/1469468400 shutdown_connections 2024-06-10T00:20:29.210 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:28.618+0000 7f4412b58700 1 -- 172.21.15.62:0/1469468400 wait complete. 2024-06-10T00:20:29.210 INFO:teuthology.orchestra.run.smithi062.stdout:mon is available 2024-06-10T00:20:29.210 INFO:teuthology.orchestra.run.smithi062.stdout:Assimilating anything we can from ceph.conf... 2024-06-10T00:20:30.253 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:29 smithi062 ceph-mon[32022]: from='client.? 172.21.15.62:0/2221124351' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-10T00:20:30.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-06-10T00:20:30.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [global] 2024-06-10T00:20:30.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout fsid = 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:20:30.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2024-06-10T00:20:30.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-10T00:20:30.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-10T00:20:30.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-10T00:20:30.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-10T00:20:30.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-06-10T00:20:30.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-10T00:20:30.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-10T00:20:30.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-06-10T00:20:30.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [osd] 2024-06-10T00:20:30.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-06-10T00:20:30.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-10T00:20:30.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-10T00:20:30.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-10T00:20:30.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-10T00:20:30.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-10T00:20:30.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.638+0000 7f31da904700 1 Processor -- start 2024-06-10T00:20:30.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.638+0000 7f31da904700 1 -- start start 2024-06-10T00:20:30.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.638+0000 7f31da904700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d4108ac0 0x7f31d410aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:30.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.638+0000 7f31da904700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f31d407be70 con 0x7f31d4108ac0 2024-06-10T00:20:30.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.638+0000 7f31d3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d4108ac0 0x7f31d410aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:30.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.638+0000 7f31d3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d4108ac0 0x7f31d410aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38592/0 (socket says 172.21.15.62:38592) 2024-06-10T00:20:30.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.638+0000 7f31d3fff700 1 -- 172.21.15.62:0/3698553282 learned_addr learned my addr 172.21.15.62:0/3698553282 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:30.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.638+0000 7f31d3fff700 1 -- 172.21.15.62:0/3698553282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d410b3f0 con 0x7f31d4108ac0 2024-06-10T00:20:30.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.639+0000 7f31d3fff700 1 --2- 172.21.15.62:0/3698553282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d4108ac0 0x7f31d410aeb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f31c40098a0 tx=0x7f31c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f02bd569dba4ca server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:30.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.639+0000 7f31d2ffd700 1 -- 172.21.15.62:0/3698553282 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31c4004160 con 0x7f31d4108ac0 2024-06-10T00:20:30.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.639+0000 7f31d2ffd700 1 -- 172.21.15.62:0/3698553282 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c4004300 con 0x7f31d4108ac0 2024-06-10T00:20:30.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.639+0000 7f31d2ffd700 1 -- 172.21.15.62:0/3698553282 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31c40036a0 con 0x7f31d4108ac0 2024-06-10T00:20:30.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.639+0000 7f31da904700 1 -- 172.21.15.62:0/3698553282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d4108ac0 msgr2=0x7f31d410aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:30.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.639+0000 7f31da904700 1 --2- 172.21.15.62:0/3698553282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d4108ac0 0x7f31d410aeb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f31c40098a0 tx=0x7f31c4009bb0 comp rx=0 tx=0).stop 2024-06-10T00:20:30.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.639+0000 7f31da904700 1 -- 172.21.15.62:0/3698553282 shutdown_connections 2024-06-10T00:20:30.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.639+0000 7f31da904700 1 --2- 172.21.15.62:0/3698553282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d4108ac0 0x7f31d410aeb0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:30.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.639+0000 7f31da904700 1 -- 172.21.15.62:0/3698553282 >> 172.21.15.62:0/3698553282 conn(0x7f31d4102530 msgr2=0x7f31d4104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:30.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.639+0000 7f31da904700 1 -- 172.21.15.62:0/3698553282 shutdown_connections 2024-06-10T00:20:30.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.639+0000 7f31da904700 1 -- 172.21.15.62:0/3698553282 wait complete. 2024-06-10T00:20:30.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.640+0000 7f31da904700 1 Processor -- start 2024-06-10T00:20:30.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.640+0000 7f31da904700 1 -- start start 2024-06-10T00:20:30.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.640+0000 7f31da904700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d4108ac0 0x7f31d419b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:30.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.640+0000 7f31da904700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f31d410b860 con 0x7f31d4108ac0 2024-06-10T00:20:30.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.640+0000 7f31d3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d4108ac0 0x7f31d419b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:30.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.640+0000 7f31d3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d4108ac0 0x7f31d419b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38602/0 (socket says 172.21.15.62:38602) 2024-06-10T00:20:30.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.641+0000 7f31d3fff700 1 -- 172.21.15.62:0/2221124351 learned_addr learned my addr 172.21.15.62:0/2221124351 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:30.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.641+0000 7f31d3fff700 1 -- 172.21.15.62:0/2221124351 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d419bba0 con 0x7f31d4108ac0 2024-06-10T00:20:30.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.641+0000 7f31d3fff700 1 --2- 172.21.15.62:0/2221124351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d4108ac0 0x7f31d419b660 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f31c4000c00 tx=0x7f31c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:30.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.641+0000 7f31d17fa700 1 -- 172.21.15.62:0/2221124351 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31c4003db0 con 0x7f31d4108ac0 2024-06-10T00:20:30.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.641+0000 7f31d17fa700 1 -- 172.21.15.62:0/2221124351 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c4011630 con 0x7f31d4108ac0 2024-06-10T00:20:30.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.641+0000 7f31da904700 1 -- 172.21.15.62:0/2221124351 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d419be30 con 0x7f31d4108ac0 2024-06-10T00:20:30.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.641+0000 7f31d17fa700 1 -- 172.21.15.62:0/2221124351 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31c4011950 con 0x7f31d4108ac0 2024-06-10T00:20:30.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.641+0000 7f31da904700 1 -- 172.21.15.62:0/2221124351 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d419eb20 con 0x7f31d4108ac0 2024-06-10T00:20:30.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.641+0000 7f31d17fa700 1 -- 172.21.15.62:0/2221124351 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f31c4011af0 con 0x7f31d4108ac0 2024-06-10T00:20:30.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.641+0000 7f31d17fa700 1 -- 172.21.15.62:0/2221124351 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f31c401c450 con 0x7f31d4108ac0 2024-06-10T00:20:30.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.642+0000 7f31da904700 1 -- 172.21.15.62:0/2221124351 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31d4063af0 con 0x7f31d4108ac0 2024-06-10T00:20:30.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.644+0000 7f31d17fa700 1 -- 172.21.15.62:0/2221124351 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f31c4032020 con 0x7f31d4108ac0 2024-06-10T00:20:30.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.716+0000 7f31da904700 1 -- 172.21.15.62:0/2221124351 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f31d402deb0 con 0x7f31d4108ac0 2024-06-10T00:20:30.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.719+0000 7f31d17fa700 1 -- 172.21.15.62:0/2221124351 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+454 (secure 0 0 0) 0x7f31c4004260 con 0x7f31d4108ac0 2024-06-10T00:20:30.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.719+0000 7f31d17fa700 1 -- 172.21.15.62:0/2221124351 <== mon.0 v2:172.21.15.62:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f31c401b450 con 0x7f31d4108ac0 2024-06-10T00:20:30.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.720+0000 7f31da904700 1 -- 172.21.15.62:0/2221124351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d4108ac0 msgr2=0x7f31d419b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:30.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.720+0000 7f31da904700 1 --2- 172.21.15.62:0/2221124351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d4108ac0 0x7f31d419b660 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f31c4000c00 tx=0x7f31c4004750 comp rx=0 tx=0).stop 2024-06-10T00:20:30.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.720+0000 7f31da904700 1 -- 172.21.15.62:0/2221124351 shutdown_connections 2024-06-10T00:20:30.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.720+0000 7f31da904700 1 --2- 172.21.15.62:0/2221124351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31d4108ac0 0x7f31d419b660 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:30.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.720+0000 7f31da904700 1 -- 172.21.15.62:0/2221124351 >> 172.21.15.62:0/2221124351 conn(0x7f31d4102530 msgr2=0x7f31d41033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:30.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.720+0000 7f31da904700 1 -- 172.21.15.62:0/2221124351 shutdown_connections 2024-06-10T00:20:30.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:29.720+0000 7f31da904700 1 -- 172.21.15.62:0/2221124351 wait complete. 2024-06-10T00:20:30.266 INFO:teuthology.orchestra.run.smithi062.stdout:Generating new minimal ceph.conf... 2024-06-10T00:20:31.260 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:30 smithi062 ceph-mon[32022]: from='client.? 172.21.15.62:0/2143780682' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:20:31.267 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.710+0000 7f004a339700 1 Processor -- start 2024-06-10T00:20:31.267 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.710+0000 7f004a339700 1 -- start start 2024-06-10T00:20:31.267 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.711+0000 7f004a339700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0044107fd0 0x7f00441083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:31.267 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.711+0000 7f004a339700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f00441089c0 con 0x7f0044107fd0 2024-06-10T00:20:31.267 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.711+0000 7f0043fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0044107fd0 0x7f00441083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:31.267 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.711+0000 7f0043fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0044107fd0 0x7f00441083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38612/0 (socket says 172.21.15.62:38612) 2024-06-10T00:20:31.267 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.711+0000 7f0043fff700 1 -- 172.21.15.62:0/796310296 learned_addr learned my addr 172.21.15.62:0/796310296 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:31.267 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.711+0000 7f0043fff700 1 -- 172.21.15.62:0/796310296 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00441091d0 con 0x7f0044107fd0 2024-06-10T00:20:31.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.711+0000 7f0043fff700 1 --2- 172.21.15.62:0/796310296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0044107fd0 0x7f00441083f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f00340098a0 tx=0x7f0034009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b062b8ac810c697b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:31.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.711+0000 7f0042ffd700 1 -- 172.21.15.62:0/796310296 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0034004160 con 0x7f0044107fd0 2024-06-10T00:20:31.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.712+0000 7f0042ffd700 1 -- 172.21.15.62:0/796310296 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0034004300 con 0x7f0044107fd0 2024-06-10T00:20:31.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.712+0000 7f0042ffd700 1 -- 172.21.15.62:0/796310296 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0034011b50 con 0x7f0044107fd0 2024-06-10T00:20:31.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.712+0000 7f004a339700 1 -- 172.21.15.62:0/796310296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0044107fd0 msgr2=0x7f00441083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:31.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.712+0000 7f004a339700 1 --2- 172.21.15.62:0/796310296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0044107fd0 0x7f00441083f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f00340098a0 tx=0x7f0034009bb0 comp rx=0 tx=0).stop 2024-06-10T00:20:31.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.712+0000 7f004a339700 1 -- 172.21.15.62:0/796310296 shutdown_connections 2024-06-10T00:20:31.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.712+0000 7f004a339700 1 --2- 172.21.15.62:0/796310296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0044107fd0 0x7f00441083f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:31.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.712+0000 7f004a339700 1 -- 172.21.15.62:0/796310296 >> 172.21.15.62:0/796310296 conn(0x7f00441035b0 msgr2=0x7f00441059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:31.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.712+0000 7f004a339700 1 -- 172.21.15.62:0/796310296 shutdown_connections 2024-06-10T00:20:31.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.712+0000 7f004a339700 1 -- 172.21.15.62:0/796310296 wait complete. 2024-06-10T00:20:31.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.713+0000 7f004a339700 1 Processor -- start 2024-06-10T00:20:31.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.713+0000 7f004a339700 1 -- start start 2024-06-10T00:20:31.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.713+0000 7f004a339700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0044107fd0 0x7f004419bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:31.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.713+0000 7f004a339700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f004410c580 con 0x7f0044107fd0 2024-06-10T00:20:31.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.713+0000 7f0043fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0044107fd0 0x7f004419bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:31.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.713+0000 7f0043fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0044107fd0 0x7f004419bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38628/0 (socket says 172.21.15.62:38628) 2024-06-10T00:20:31.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.713+0000 7f0043fff700 1 -- 172.21.15.62:0/2143780682 learned_addr learned my addr 172.21.15.62:0/2143780682 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:31.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.714+0000 7f0043fff700 1 -- 172.21.15.62:0/2143780682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004419c4b0 con 0x7f0044107fd0 2024-06-10T00:20:31.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.714+0000 7f0043fff700 1 --2- 172.21.15.62:0/2143780682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0044107fd0 0x7f004419bf70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0034000c00 tx=0x7f0034011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:31.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.714+0000 7f00417fa700 1 -- 172.21.15.62:0/2143780682 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0034003b20 con 0x7f0044107fd0 2024-06-10T00:20:31.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.714+0000 7f00417fa700 1 -- 172.21.15.62:0/2143780682 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0034003cc0 con 0x7f0044107fd0 2024-06-10T00:20:31.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.714+0000 7f004a339700 1 -- 172.21.15.62:0/2143780682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004419c740 con 0x7f0044107fd0 2024-06-10T00:20:31.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.714+0000 7f00417fa700 1 -- 172.21.15.62:0/2143780682 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0034024910 con 0x7f0044107fd0 2024-06-10T00:20:31.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.714+0000 7f004a339700 1 -- 172.21.15.62:0/2143780682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004419f430 con 0x7f0044107fd0 2024-06-10T00:20:31.270 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.714+0000 7f00417fa700 1 -- 172.21.15.62:0/2143780682 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0034022020 con 0x7f0044107fd0 2024-06-10T00:20:31.270 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.714+0000 7f00417fa700 1 -- 172.21.15.62:0/2143780682 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0034011e00 con 0x7f0044107fd0 2024-06-10T00:20:31.270 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.715+0000 7f004a339700 1 -- 172.21.15.62:0/2143780682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0044063af0 con 0x7f0044107fd0 2024-06-10T00:20:31.270 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.717+0000 7f00417fa700 1 -- 172.21.15.62:0/2143780682 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f003401acd0 con 0x7f0044107fd0 2024-06-10T00:20:31.270 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.788+0000 7f004a339700 1 -- 172.21.15.62:0/2143780682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f004419f600 con 0x7f0044107fd0 2024-06-10T00:20:31.270 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.788+0000 7f00417fa700 1 -- 172.21.15.62:0/2143780682 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f0034032030 con 0x7f0044107fd0 2024-06-10T00:20:31.270 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.789+0000 7f004a339700 1 -- 172.21.15.62:0/2143780682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0044107fd0 msgr2=0x7f004419bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:31.270 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.789+0000 7f004a339700 1 --2- 172.21.15.62:0/2143780682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0044107fd0 0x7f004419bf70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0034000c00 tx=0x7f0034011c10 comp rx=0 tx=0).stop 2024-06-10T00:20:31.270 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.789+0000 7f004a339700 1 -- 172.21.15.62:0/2143780682 shutdown_connections 2024-06-10T00:20:31.270 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.789+0000 7f004a339700 1 --2- 172.21.15.62:0/2143780682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0044107fd0 0x7f004419bf70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:31.270 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.789+0000 7f004a339700 1 -- 172.21.15.62:0/2143780682 >> 172.21.15.62:0/2143780682 conn(0x7f00441035b0 msgr2=0x7f0044104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:31.270 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.790+0000 7f004a339700 1 -- 172.21.15.62:0/2143780682 shutdown_connections 2024-06-10T00:20:31.270 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:30.790+0000 7f004a339700 1 -- 172.21.15.62:0/2143780682 wait complete. 2024-06-10T00:20:31.271 INFO:teuthology.orchestra.run.smithi062.stdout:Restarting the monitor... 2024-06-10T00:20:31.531 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:31 smithi062 systemd[1]: Stopping Ceph mon.a for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:20:31.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:31 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a[32017]: 2024-06-10T00:20:31.530+0000 7f4626ea8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:20:31.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:31 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a[32017]: 2024-06-10T00:20:31.530+0000 7f4626ea8700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-10T00:20:31.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:31 smithi062 podman[32266]: 2024-06-10 00:20:31.756393766 +0000 UTC m=+0.322051972 container died 9e09bb11fc0249fe71c0dac7e446dd2613a2148b4d07edac5d35f9b43dafb13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-10T00:20:32.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 podman[32266]: 2024-06-10 00:20:32.063548259 +0000 UTC m=+0.629206464 container cleanup 9e09bb11fc0249fe71c0dac7e446dd2613a2148b4d07edac5d35f9b43dafb13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-06-10T00:20:32.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 bash[32266]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a 2024-06-10T00:20:32.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 podman[32278]: 2024-06-10 00:20:32.272775081 +0000 UTC m=+0.511884374 container remove 9e09bb11fc0249fe71c0dac7e446dd2613a2148b4d07edac5d35f9b43dafb13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:20:32.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 systemd[1]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-10T00:20:32.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 systemd[1]: Stopped Ceph mon.a for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:20:32.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 systemd[1]: Starting Ceph mon.a for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:20:32.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 podman[32371]: 2024-06-10 00:20:32.626255754 +0000 UTC m=+0.017673677 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:20:32.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 podman[32371]: 2024-06-10 00:20:32.731941334 +0000 UTC m=+0.123359243 container create 4f3ad4cc1fad860bae7ad39415da661be69dbf560e999087ce060d8e310df1b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528) 2024-06-10T00:20:32.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 podman[32371]: 2024-06-10 00:20:32.942616769 +0000 UTC m=+0.334034679 container init 4f3ad4cc1fad860bae7ad39415da661be69dbf560e999087ce060d8e310df1b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:20:32.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 podman[32371]: 2024-06-10 00:20:32.945919356 +0000 UTC m=+0.337337274 container start 4f3ad4cc1fad860bae7ad39415da661be69dbf560e999087ce060d8e310df1b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-10T00:20:32.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: set uid:gid to 167:167 (ceph:ceph) 2024-06-10T00:20:32.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-10T00:20:32.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: pidfile_write: ignore empty --pid-file 2024-06-10T00:20:32.973 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: load: jerasure load: lrc 2024-06-10T00:20:33.054 INFO:teuthology.orchestra.run.smithi062.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-10T00:20:33.251 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: RocksDB version: 7.9.2 2024-06-10T00:20:33.251 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Git sha 0 2024-06-10T00:20:33.251 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-10T00:20:33.252 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: DB SUMMARY 2024-06-10T00:20:33.252 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: DB Session ID: LOK6SVIYGW0SEPTDBP6O 2024-06-10T00:20:33.252 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: CURRENT file: CURRENT 2024-06-10T00:20:33.252 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: IDENTITY file: IDENTITY 2024-06-10T00:20:33.252 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-10T00:20:33.252 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-10T00:20:33.252 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105188 ; 2024-06-10T00:20:33.252 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.error_if_exists: 0 2024-06-10T00:20:33.252 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.create_if_missing: 0 2024-06-10T00:20:33.252 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.paranoid_checks: 1 2024-06-10T00:20:33.252 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-10T00:20:33.253 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-10T00:20:33.253 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-10T00:20:33.253 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.env: 0x558855080c60 2024-06-10T00:20:33.253 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.fs: PosixFileSystem 2024-06-10T00:20:33.253 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.info_log: 0x558856269b20 2024-06-10T00:20:33.253 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_file_opening_threads: 16 2024-06-10T00:20:33.253 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.statistics: (nil) 2024-06-10T00:20:33.253 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.use_fsync: 0 2024-06-10T00:20:33.253 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_log_file_size: 0 2024-06-10T00:20:33.253 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-10T00:20:33.253 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-10T00:20:33.254 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.keep_log_file_num: 1000 2024-06-10T00:20:33.254 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.recycle_log_file_num: 0 2024-06-10T00:20:33.254 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.allow_fallocate: 1 2024-06-10T00:20:33.254 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.allow_mmap_reads: 0 2024-06-10T00:20:33.254 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.allow_mmap_writes: 0 2024-06-10T00:20:33.254 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.use_direct_reads: 0 2024-06-10T00:20:33.254 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-10T00:20:33.254 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.create_missing_column_families: 0 2024-06-10T00:20:33.254 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.db_log_dir: 2024-06-10T00:20:33.254 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.wal_dir: 2024-06-10T00:20:33.254 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-10T00:20:33.254 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-10T00:20:33.255 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-10T00:20:33.255 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-10T00:20:33.256 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-10T00:20:33.256 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-10T00:20:33.256 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.advise_random_on_open: 1 2024-06-10T00:20:33.256 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.db_write_buffer_size: 0 2024-06-10T00:20:33.257 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.write_buffer_manager: 0x55885623a1e0 2024-06-10T00:20:33.257 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-10T00:20:33.257 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-10T00:20:33.257 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-10T00:20:33.257 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.rate_limiter: (nil) 2024-06-10T00:20:33.257 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-10T00:20:33.257 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.wal_recovery_mode: 2 2024-06-10T00:20:33.257 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.enable_thread_tracking: 0 2024-06-10T00:20:33.257 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.enable_pipelined_write: 0 2024-06-10T00:20:33.257 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.unordered_write: 0 2024-06-10T00:20:33.257 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-10T00:20:33.257 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-10T00:20:33.257 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-10T00:20:33.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-10T00:20:33.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.row_cache: None 2024-06-10T00:20:33.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.wal_filter: None 2024-06-10T00:20:33.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-10T00:20:33.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.allow_ingest_behind: 0 2024-06-10T00:20:33.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.two_write_queues: 0 2024-06-10T00:20:33.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.manual_wal_flush: 0 2024-06-10T00:20:33.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.wal_compression: 0 2024-06-10T00:20:33.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.atomic_flush: 0 2024-06-10T00:20:33.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-10T00:20:33.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-10T00:20:33.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-10T00:20:33.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.log_readahead_size: 0 2024-06-10T00:20:33.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-10T00:20:33.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.best_efforts_recovery: 0 2024-06-10T00:20:33.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-10T00:20:33.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-10T00:20:33.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.allow_data_in_errors: 0 2024-06-10T00:20:33.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.db_host_id: __hostname__ 2024-06-10T00:20:33.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-10T00:20:33.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_background_jobs: 2 2024-06-10T00:20:33.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_background_compactions: -1 2024-06-10T00:20:33.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_subcompactions: 1 2024-06-10T00:20:33.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-10T00:20:33.260 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-10T00:20:33.260 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-10T00:20:33.260 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_total_wal_size: 0 2024-06-10T00:20:33.260 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-10T00:20:33.260 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-10T00:20:33.260 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-10T00:20:33.260 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-10T00:20:33.260 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_open_files: -1 2024-06-10T00:20:33.260 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.bytes_per_sync: 0 2024-06-10T00:20:33.260 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-10T00:20:33.260 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-10T00:20:33.260 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compaction_readahead_size: 0 2024-06-10T00:20:33.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_background_flushes: -1 2024-06-10T00:20:33.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Compression algorithms supported: 2024-06-10T00:20:33.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-10T00:20:33.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: kZSTD supported: 0 2024-06-10T00:20:33.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: kXpressCompression supported: 0 2024-06-10T00:20:33.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: kLZ4HCCompression supported: 1 2024-06-10T00:20:33.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: kZlibCompression supported: 1 2024-06-10T00:20:33.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: kSnappyCompression supported: 1 2024-06-10T00:20:33.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: kLZ4Compression supported: 1 2024-06-10T00:20:33.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: kBZip2Compression supported: 0 2024-06-10T00:20:33.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-10T00:20:33.261 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-10T00:20:33.262 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-10T00:20:33.262 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-10T00:20:33.262 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-10T00:20:33.262 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.merge_operator: 2024-06-10T00:20:33.262 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compaction_filter: None 2024-06-10T00:20:33.262 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compaction_filter_factory: None 2024-06-10T00:20:33.262 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.sst_partitioner_factory: None 2024-06-10T00:20:33.262 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-10T00:20:33.262 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-10T00:20:33.262 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558856269c40) 2024-06-10T00:20:33.262 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks: 1 2024-06-10T00:20:33.262 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-10T00:20:33.262 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-10T00:20:33.263 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_top_level_index_and_filter: 1 2024-06-10T00:20:33.263 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_type: 0 2024-06-10T00:20:33.263 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_index_type: 0 2024-06-10T00:20:33.263 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_shortening: 1 2024-06-10T00:20:33.263 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-10T00:20:33.263 INFO:journalctl@ceph.mon.a.smithi062.stdout: checksum: 4 2024-06-10T00:20:33.263 INFO:journalctl@ceph.mon.a.smithi062.stdout: no_block_cache: 0 2024-06-10T00:20:33.263 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache: 0x55885626b090 2024-06-10T00:20:33.263 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_name: BinnedLRUCache 2024-06-10T00:20:33.263 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_options: 2024-06-10T00:20:33.263 INFO:journalctl@ceph.mon.a.smithi062.stdout: capacity : 536870912 2024-06-10T00:20:33.263 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_shard_bits : 4 2024-06-10T00:20:33.263 INFO:journalctl@ceph.mon.a.smithi062.stdout: strict_capacity_limit : 0 2024-06-10T00:20:33.264 INFO:journalctl@ceph.mon.a.smithi062.stdout: high_pri_pool_ratio: 0.000 2024-06-10T00:20:33.264 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_compressed: (nil) 2024-06-10T00:20:33.264 INFO:journalctl@ceph.mon.a.smithi062.stdout: persistent_cache: (nil) 2024-06-10T00:20:33.264 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size: 4096 2024-06-10T00:20:33.264 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size_deviation: 10 2024-06-10T00:20:33.264 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_restart_interval: 16 2024-06-10T00:20:33.264 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_block_restart_interval: 1 2024-06-10T00:20:33.264 INFO:journalctl@ceph.mon.a.smithi062.stdout: metadata_block_size: 4096 2024-06-10T00:20:33.264 INFO:journalctl@ceph.mon.a.smithi062.stdout: partition_filters: 0 2024-06-10T00:20:33.264 INFO:journalctl@ceph.mon.a.smithi062.stdout: use_delta_encoding: 1 2024-06-10T00:20:33.264 INFO:journalctl@ceph.mon.a.smithi062.stdout: filter_policy: bloomfilter 2024-06-10T00:20:33.264 INFO:journalctl@ceph.mon.a.smithi062.stdout: whole_key_filtering: 1 2024-06-10T00:20:33.264 INFO:journalctl@ceph.mon.a.smithi062.stdout: verify_compression: 0 2024-06-10T00:20:33.264 INFO:journalctl@ceph.mon.a.smithi062.stdout: read_amp_bytes_per_bit: 0 2024-06-10T00:20:33.265 INFO:journalctl@ceph.mon.a.smithi062.stdout: format_version: 5 2024-06-10T00:20:33.265 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_index_compression: 1 2024-06-10T00:20:33.265 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_align: 0 2024-06-10T00:20:33.265 INFO:journalctl@ceph.mon.a.smithi062.stdout: max_auto_readahead_size: 262144 2024-06-10T00:20:33.265 INFO:journalctl@ceph.mon.a.smithi062.stdout: prepopulate_block_cache: 0 2024-06-10T00:20:33.265 INFO:journalctl@ceph.mon.a.smithi062.stdout: initial_auto_readahead_size: 8192 2024-06-10T00:20:33.265 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_file_reads_for_auto_readahead: 2 2024-06-10T00:20:33.265 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.write_buffer_size: 33554432 2024-06-10T00:20:33.265 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_write_buffer_number: 2 2024-06-10T00:20:33.265 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compression: NoCompression 2024-06-10T00:20:33.265 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.bottommost_compression: Disabled 2024-06-10T00:20:33.265 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.prefix_extractor: nullptr 2024-06-10T00:20:33.265 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-10T00:20:33.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.num_levels: 7 2024-06-10T00:20:33.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-10T00:20:33.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-10T00:20:33.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-10T00:20:33.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-10T00:20:33.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-10T00:20:33.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-10T00:20:33.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-10T00:20:33.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-10T00:20:33.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-10T00:20:33.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-10T00:20:33.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-10T00:20:33.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-10T00:20:33.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-10T00:20:33.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compression_opts.level: 32767 2024-06-10T00:20:33.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compression_opts.strategy: 0 2024-06-10T00:20:33.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-10T00:20:33.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-10T00:20:33.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-10T00:20:33.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-10T00:20:33.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compression_opts.enabled: false 2024-06-10T00:20:33.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-10T00:20:33.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-10T00:20:33.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-10T00:20:33.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-10T00:20:33.267 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.target_file_size_base: 67108864 2024-06-10T00:20:33.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-10T00:20:33.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-10T00:20:33.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-10T00:20:33.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-10T00:20:33.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-10T00:20:33.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-10T00:20:33.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-10T00:20:33.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-10T00:20:33.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-10T00:20:33.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-10T00:20:33.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-10T00:20:33.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-10T00:20:33.268 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-10T00:20:33.269 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-10T00:20:33.269 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.arena_block_size: 1048576 2024-06-10T00:20:33.269 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-10T00:20:33.269 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-10T00:20:33.269 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.disable_auto_compactions: 0 2024-06-10T00:20:33.269 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-10T00:20:33.269 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-10T00:20:33.269 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-10T00:20:33.269 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-10T00:20:33.269 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-10T00:20:33.269 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-10T00:20:33.269 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-10T00:20:33.269 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-10T00:20:33.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-10T00:20:33.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-10T00:20:33.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.table_properties_collectors: 2024-06-10T00:20:33.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.inplace_update_support: 0 2024-06-10T00:20:33.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-10T00:20:33.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-10T00:20:33.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-10T00:20:33.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-10T00:20:33.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.bloom_locality: 0 2024-06-10T00:20:33.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.max_successive_merges: 0 2024-06-10T00:20:33.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-10T00:20:33.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.paranoid_file_checks: 0 2024-06-10T00:20:33.270 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.force_consistency_checks: 1 2024-06-10T00:20:33.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.report_bg_io_stats: 0 2024-06-10T00:20:33.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.ttl: 2592000 2024-06-10T00:20:33.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-10T00:20:33.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-10T00:20:33.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-10T00:20:33.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.enable_blob_files: false 2024-06-10T00:20:33.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.min_blob_size: 0 2024-06-10T00:20:33.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.blob_file_size: 268435456 2024-06-10T00:20:33.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-10T00:20:33.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-10T00:20:33.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-10T00:20:33.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-10T00:20:33.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-10T00:20:33.271 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.blob_file_starting_level: 0 2024-06-10T00:20:33.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-10T00:20:33.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-10T00:20:33.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-10T00:20:33.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 13daa37f-b6d4-467a-92ec-5fd93962ebd4 2024-06-10T00:20:33.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717978832975669, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-10T00:20:33.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-10T00:20:33.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717978832976687, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100240, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98155, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14284, "raw_average_key_size": 49, "raw_value_size": 90419, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717978832, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "13daa37f-b6d4-467a-92ec-5fd93962ebd4", "db_session_id": "LOK6SVIYGW0SEPTDBP6O", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-10T00:20:33.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717978832976771, "job": 1, "event": "recovery_finished"} 2024-06-10T00:20:33.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-10T00:20:33.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-10T00:20:33.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55885635a000 2024-06-10T00:20:33.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: DB pointer 0x558856344000 2024-06-10T00:20:33.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-10T00:20:33.273 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-10T00:20:33.273 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** DB Stats ** 2024-06-10T00:20:33.273 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-10T00:20:33.273 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-10T00:20:33.273 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-10T00:20:33.273 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-10T00:20:33.273 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-10T00:20:33.273 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-10T00:20:33.273 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-10T00:20:33.273 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-06-10T00:20:33.273 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-06-10T00:20:33.273 INFO:journalctl@ceph.mon.a.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-10T00:20:33.273 INFO:journalctl@ceph.mon.a.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-10T00:20:33.273 INFO:journalctl@ceph.mon.a.smithi062.stdout: L0 2/0 99.73 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 165.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T00:20:33.274 INFO:journalctl@ceph.mon.a.smithi062.stdout: Sum 2/0 99.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 165.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T00:20:33.274 INFO:journalctl@ceph.mon.a.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 165.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T00:20:33.274 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-06-10T00:20:33.274 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-06-10T00:20:33.274 INFO:journalctl@ceph.mon.a.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-10T00:20:33.274 INFO:journalctl@ceph.mon.a.smithi062.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-10T00:20:33.274 INFO:journalctl@ceph.mon.a.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 165.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T00:20:33.274 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-06-10T00:20:33.274 INFO:journalctl@ceph.mon.a.smithi062.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-10T00:20:33.274 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-06-10T00:20:33.274 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-10T00:20:33.274 INFO:journalctl@ceph.mon.a.smithi062.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-10T00:20:33.275 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-10T00:20:33.275 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-10T00:20:33.275 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-10T00:20:33.275 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-10T00:20:33.275 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative compaction: 0.00 GB write, 30.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-10T00:20:33.275 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval compaction: 0.00 GB write, 30.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-10T00:20:33.275 INFO:journalctl@ceph.mon.a.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-10T00:20:33.275 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache BinnedLRUCache@0x55885626b090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-06-10T00:20:33.275 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-10T00:20:33.275 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-06-10T00:20:33.275 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-10T00:20:33.275 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: starting mon.a rank 0 at public addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] at bind addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:20:33.275 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: mon.a@-1(???) e1 preinit fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:20:33.276 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: mon.a@-1(???).mds e1 new map 2024-06-10T00:20:33.276 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: mon.a@-1(???).mds e1 print_map 2024-06-10T00:20:33.276 INFO:journalctl@ceph.mon.a.smithi062.stdout: e1 2024-06-10T00:20:33.276 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-10T00:20:33.276 INFO:journalctl@ceph.mon.a.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T00:20:33.276 INFO:journalctl@ceph.mon.a.smithi062.stdout: legacy client fscid: -1 2024-06-10T00:20:33.276 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-06-10T00:20:33.276 INFO:journalctl@ceph.mon.a.smithi062.stdout: No filesystems configured 2024-06-10T00:20:33.276 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-10T00:20:33.276 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:20:33.276 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:20:33.276 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:20:33.276 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:32 smithi062 ceph-mon[32390]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-10T00:20:33.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:33 smithi062 ceph-mon[32390]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-10T00:20:33.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:33 smithi062 ceph-mon[32390]: monmap epoch 1 2024-06-10T00:20:33.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:33 smithi062 ceph-mon[32390]: fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:20:33.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:33 smithi062 ceph-mon[32390]: last_changed 2024-06-10T00:20:24.460529+0000 2024-06-10T00:20:33.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:33 smithi062 ceph-mon[32390]: created 2024-06-10T00:20:24.460529+0000 2024-06-10T00:20:33.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:33 smithi062 ceph-mon[32390]: min_mon_release 19 (squid) 2024-06-10T00:20:33.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:33 smithi062 ceph-mon[32390]: election_strategy: 1 2024-06-10T00:20:33.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:33 smithi062 ceph-mon[32390]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-06-10T00:20:33.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:33 smithi062 ceph-mon[32390]: fsmap 2024-06-10T00:20:33.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:33 smithi062 ceph-mon[32390]: osdmap e1: 0 total, 0 up, 0 in 2024-06-10T00:20:33.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:33 smithi062 ceph-mon[32390]: mgrmap e1: no daemons active 2024-06-10T00:20:33.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:33 smithi062 bash[32371]: 4f3ad4cc1fad860bae7ad39415da661be69dbf560e999087ce060d8e310df1b5 2024-06-10T00:20:33.277 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:33 smithi062 systemd[1]: Started Ceph mon.a for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:20:34.121 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.477+0000 7f90c1b7c700 1 Processor -- start 2024-06-10T00:20:34.122 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.477+0000 7f90c1b7c700 1 -- start start 2024-06-10T00:20:34.122 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.477+0000 7f90c1b7c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90bc108a70 0x7f90bc10ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:34.122 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.477+0000 7f90c1b7c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc07be70 con 0x7f90bc108a70 2024-06-10T00:20:34.122 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.477+0000 7f90bb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90bc108a70 0x7f90bc10ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:34.122 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.477+0000 7f90bb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90bc108a70 0x7f90bc10ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38636/0 (socket says 172.21.15.62:38636) 2024-06-10T00:20:34.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.477+0000 7f90bb7fe700 1 -- 172.21.15.62:0/2793646517 learned_addr learned my addr 172.21.15.62:0/2793646517 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:34.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.478+0000 7f90bb7fe700 1 -- 172.21.15.62:0/2793646517 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc10b3a0 con 0x7f90bc108a70 2024-06-10T00:20:34.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.478+0000 7f90bb7fe700 1 --2- 172.21.15.62:0/2793646517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90bc108a70 0x7f90bc10ae60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009e80 tx=0x7f90ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e6ea971ecdcc03 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:34.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.478+0000 7f90ba7fc700 1 -- 172.21.15.62:0/2793646517 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90ac017070 con 0x7f90bc108a70 2024-06-10T00:20:34.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.478+0000 7f90ba7fc700 1 -- 172.21.15.62:0/2793646517 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f90ac00b8c0 con 0x7f90bc108a70 2024-06-10T00:20:34.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.478+0000 7f90ba7fc700 1 -- 172.21.15.62:0/2793646517 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90ac0045c0 con 0x7f90bc108a70 2024-06-10T00:20:34.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.478+0000 7f90c1b7c700 1 -- 172.21.15.62:0/2793646517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90bc108a70 msgr2=0x7f90bc10ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:34.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.478+0000 7f90c1b7c700 1 --2- 172.21.15.62:0/2793646517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90bc108a70 0x7f90bc10ae60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009e80 tx=0x7f90ac00b330 comp rx=0 tx=0).stop 2024-06-10T00:20:34.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.479+0000 7f90c1b7c700 1 -- 172.21.15.62:0/2793646517 shutdown_connections 2024-06-10T00:20:34.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.479+0000 7f90c1b7c700 1 --2- 172.21.15.62:0/2793646517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90bc108a70 0x7f90bc10ae60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:34.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.479+0000 7f90c1b7c700 1 -- 172.21.15.62:0/2793646517 >> 172.21.15.62:0/2793646517 conn(0x7f90bc1024e0 msgr2=0x7f90bc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:34.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.479+0000 7f90c1b7c700 1 -- 172.21.15.62:0/2793646517 shutdown_connections 2024-06-10T00:20:34.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.479+0000 7f90c1b7c700 1 -- 172.21.15.62:0/2793646517 wait complete. 2024-06-10T00:20:34.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.479+0000 7f90c1b7c700 1 Processor -- start 2024-06-10T00:20:34.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.479+0000 7f90c1b7c700 1 -- start start 2024-06-10T00:20:34.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.480+0000 7f90c1b7c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90bc108a70 0x7f90bc19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:34.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.480+0000 7f90c1b7c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc10c3c0 con 0x7f90bc108a70 2024-06-10T00:20:34.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.480+0000 7f90bb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90bc108a70 0x7f90bc19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:34.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.480+0000 7f90bb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90bc108a70 0x7f90bc19e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38644/0 (socket says 172.21.15.62:38644) 2024-06-10T00:20:34.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.480+0000 7f90bb7fe700 1 -- 172.21.15.62:0/2765057254 learned_addr learned my addr 172.21.15.62:0/2765057254 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:34.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.480+0000 7f90bb7fe700 1 -- 172.21.15.62:0/2765057254 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc19e670 con 0x7f90bc108a70 2024-06-10T00:20:34.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.480+0000 7f90bb7fe700 1 --2- 172.21.15.62:0/2765057254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90bc108a70 0x7f90bc19e130 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f90ac004350 tx=0x7f90ac00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:34.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.480+0000 7f90b8ff9700 1 -- 172.21.15.62:0/2765057254 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90ac020020 con 0x7f90bc108a70 2024-06-10T00:20:34.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.480+0000 7f90c1b7c700 1 -- 172.21.15.62:0/2765057254 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90bc19e900 con 0x7f90bc108a70 2024-06-10T00:20:34.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.480+0000 7f90b8ff9700 1 -- 172.21.15.62:0/2765057254 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f90ac003c10 con 0x7f90bc108a70 2024-06-10T00:20:34.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.480+0000 7f90b8ff9700 1 -- 172.21.15.62:0/2765057254 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90ac017040 con 0x7f90bc108a70 2024-06-10T00:20:34.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.480+0000 7f90c1b7c700 1 -- 172.21.15.62:0/2765057254 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90bc1a15f0 con 0x7f90bc108a70 2024-06-10T00:20:34.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.481+0000 7f90b8ff9700 1 -- 172.21.15.62:0/2765057254 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f90ac023020 con 0x7f90bc108a70 2024-06-10T00:20:34.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.481+0000 7f90b8ff9700 1 -- 172.21.15.62:0/2765057254 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f90ac00f050 con 0x7f90bc108a70 2024-06-10T00:20:34.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.482+0000 7f90c1b7c700 1 -- 172.21.15.62:0/2765057254 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90bc063af0 con 0x7f90bc108a70 2024-06-10T00:20:34.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.484+0000 7f90b8ff9700 1 -- 172.21.15.62:0/2765057254 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f90ac02a020 con 0x7f90bc108a70 2024-06-10T00:20:34.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.554+0000 7f90c1b7c700 1 -- 172.21.15.62:0/2765057254 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f90bc1a17c0 con 0x7f90bc108a70 2024-06-10T00:20:34.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.556+0000 7f90b8ff9700 1 -- 172.21.15.62:0/2765057254 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f90ac01ccd0 con 0x7f90bc108a70 2024-06-10T00:20:34.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.557+0000 7f90c1b7c700 1 -- 172.21.15.62:0/2765057254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90bc108a70 msgr2=0x7f90bc19e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:34.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.557+0000 7f90c1b7c700 1 --2- 172.21.15.62:0/2765057254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90bc108a70 0x7f90bc19e130 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f90ac004350 tx=0x7f90ac00bd50 comp rx=0 tx=0).stop 2024-06-10T00:20:34.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.557+0000 7f90c1b7c700 1 -- 172.21.15.62:0/2765057254 shutdown_connections 2024-06-10T00:20:34.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.557+0000 7f90c1b7c700 1 --2- 172.21.15.62:0/2765057254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90bc108a70 0x7f90bc19e130 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:34.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.557+0000 7f90c1b7c700 1 -- 172.21.15.62:0/2765057254 >> 172.21.15.62:0/2765057254 conn(0x7f90bc1024e0 msgr2=0x7f90bc102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:34.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.557+0000 7f90c1b7c700 1 -- 172.21.15.62:0/2765057254 shutdown_connections 2024-06-10T00:20:34.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:33.557+0000 7f90c1b7c700 1 -- 172.21.15.62:0/2765057254 wait complete. 2024-06-10T00:20:34.126 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-10T00:20:34.139 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:20:34.139 INFO:teuthology.orchestra.run.smithi062.stdout:Creating mgr... 2024-06-10T00:20:34.140 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-10T00:20:34.140 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-10T00:20:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mgr.x 2024-06-10T00:20:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-06-10T00:20:34.527 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968.target.wants/ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@.service. 2024-06-10T00:20:34.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:34 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2765057254' entity='client.admin' 2024-06-10T00:20:35.041 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T00:20:35.042 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-06-10T00:20:35.049 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T00:20:35.049 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-06-10T00:20:35.049 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-06-10T00:20:35.049 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to enable service . firewalld.service is not available 2024-06-10T00:20:35.058 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T00:20:35.058 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-06-10T00:20:35.066 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T00:20:35.066 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-06-10T00:20:35.066 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-06-10T00:20:35.066 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-10T00:20:35.067 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr to start... 2024-06-10T00:20:35.067 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr... 2024-06-10T00:20:35.514 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:35 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:35.423+0000 7f1f7e96d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:20:35.846 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:35 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3063534760' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-10T00:20:35.847 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:35 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:35.757+0000 7f1f7e96d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:20:35.847 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:35 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:35.846+0000 7f1f7e96d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:20:36.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-06-10T00:20:36.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:20:36.208 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:20:36.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-06-10T00:20:36.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-10T00:20:36.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-10T00:20:36.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-10T00:20:36.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:36.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-10T00:20:36.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-10T00:20:36.209 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-06-10T00:20:36.210 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:20:36.210 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-10T00:20:36.210 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-06-10T00:20:36.210 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:20:36.210 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-06-10T00:20:36.210 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-10T00:20:36.210 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:36.210 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-10T00:20:36.210 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-10T00:20:36.211 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:36.211 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-10T00:20:36.211 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:36.211 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-10T00:20:36.211 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-10T00:20:36.211 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-10T00:20:36.219 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-10T00:20:36.219 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-10T00:20:36.219 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-10T00:20:36.219 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:36.219 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-10T00:20:36.219 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-10T00:20:36.219 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-10T00:20:36.219 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-10T00:20:36.219 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-10T00:20:36.220 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-10T00:20:36.221 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-10T00:20:36.221 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-06-10T00:20:36.221 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:20:36.221 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:20:36.221 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:36.221 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-10T00:20:36.221 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:36.221 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-06-10T00:20:27.982259+0000", 2024-06-10T00:20:36.221 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:20:36.221 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:36.221 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-10T00:20:36.221 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:20:36.221 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.518+0000 7fd3b12af700 1 Processor -- start 2024-06-10T00:20:36.222 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.518+0000 7fd3b12af700 1 -- start start 2024-06-10T00:20:36.222 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.519+0000 7fd3b12af700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3ac1062b0 0x7fd3ac1066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:36.222 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.519+0000 7fd3b12af700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac106ca0 con 0x7fd3ac1062b0 2024-06-10T00:20:36.222 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.519+0000 7fd3aaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3ac1062b0 0x7fd3ac1066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:36.222 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.519+0000 7fd3aaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3ac1062b0 0x7fd3ac1066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38660/0 (socket says 172.21.15.62:38660) 2024-06-10T00:20:36.222 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.519+0000 7fd3aaffd700 1 -- 172.21.15.62:0/3415333957 learned_addr learned my addr 172.21.15.62:0/3415333957 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:36.222 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.519+0000 7fd3aaffd700 1 -- 172.21.15.62:0/3415333957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac109240 con 0x7fd3ac1062b0 2024-06-10T00:20:36.222 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.519+0000 7fd3aaffd700 1 --2- 172.21.15.62:0/3415333957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3ac1062b0 0x7fd3ac1066d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0098a0 tx=0x7fd39c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9a5bdd97a29d25 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:36.222 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.520+0000 7fd3a9ffb700 1 -- 172.21.15.62:0/3415333957 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd39c004160 con 0x7fd3ac1062b0 2024-06-10T00:20:36.222 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.520+0000 7fd3a9ffb700 1 -- 172.21.15.62:0/3415333957 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd39c004300 con 0x7fd3ac1062b0 2024-06-10T00:20:36.222 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.520+0000 7fd3a9ffb700 1 -- 172.21.15.62:0/3415333957 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd39c011b50 con 0x7fd3ac1062b0 2024-06-10T00:20:36.222 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.520+0000 7fd3b12af700 1 -- 172.21.15.62:0/3415333957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3ac1062b0 msgr2=0x7fd3ac1066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.520+0000 7fd3b12af700 1 --2- 172.21.15.62:0/3415333957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3ac1062b0 0x7fd3ac1066d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0098a0 tx=0x7fd39c009bb0 comp rx=0 tx=0).stop 2024-06-10T00:20:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.520+0000 7fd3b12af700 1 -- 172.21.15.62:0/3415333957 shutdown_connections 2024-06-10T00:20:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.520+0000 7fd3b12af700 1 --2- 172.21.15.62:0/3415333957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3ac1062b0 0x7fd3ac1066d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.520+0000 7fd3b12af700 1 -- 172.21.15.62:0/3415333957 >> 172.21.15.62:0/3415333957 conn(0x7fd3ac101c20 msgr2=0x7fd3ac104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.520+0000 7fd3b12af700 1 -- 172.21.15.62:0/3415333957 shutdown_connections 2024-06-10T00:20:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.521+0000 7fd3b12af700 1 -- 172.21.15.62:0/3415333957 wait complete. 2024-06-10T00:20:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.521+0000 7fd3b12af700 1 Processor -- start 2024-06-10T00:20:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.522+0000 7fd3b12af700 1 -- start start 2024-06-10T00:20:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.522+0000 7fd3b12af700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3ac1062b0 0x7fd3ac19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.522+0000 7fd3b12af700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac10c5f0 con 0x7fd3ac1062b0 2024-06-10T00:20:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.522+0000 7fd3aaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3ac1062b0 0x7fd3ac19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.522+0000 7fd3aaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3ac1062b0 0x7fd3ac19bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38666/0 (socket says 172.21.15.62:38666) 2024-06-10T00:20:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.522+0000 7fd3aaffd700 1 -- 172.21.15.62:0/3063534760 learned_addr learned my addr 172.21.15.62:0/3063534760 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:36.223 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.522+0000 7fd3aaffd700 1 -- 172.21.15.62:0/3063534760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac19c4d0 con 0x7fd3ac1062b0 2024-06-10T00:20:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.522+0000 7fd3aaffd700 1 --2- 172.21.15.62:0/3063534760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3ac1062b0 0x7fd3ac19bf90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd39c011670 tx=0x7fd39c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.523+0000 7fd393fff700 1 -- 172.21.15.62:0/3063534760 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd39c0039c0 con 0x7fd3ac1062b0 2024-06-10T00:20:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.523+0000 7fd3b12af700 1 -- 172.21.15.62:0/3063534760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ac19c760 con 0x7fd3ac1062b0 2024-06-10T00:20:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.523+0000 7fd393fff700 1 -- 172.21.15.62:0/3063534760 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd39c011e50 con 0x7fd3ac1062b0 2024-06-10T00:20:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.523+0000 7fd393fff700 1 -- 172.21.15.62:0/3063534760 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd39c01a5c0 con 0x7fd3ac1062b0 2024-06-10T00:20:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.523+0000 7fd3b12af700 1 -- 172.21.15.62:0/3063534760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ac19f450 con 0x7fd3ac1062b0 2024-06-10T00:20:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.523+0000 7fd393fff700 1 -- 172.21.15.62:0/3063534760 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd39c01ac50 con 0x7fd3ac1062b0 2024-06-10T00:20:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.523+0000 7fd393fff700 1 -- 172.21.15.62:0/3063534760 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd39c019700 con 0x7fd3ac1062b0 2024-06-10T00:20:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.524+0000 7fd3b12af700 1 -- 172.21.15.62:0/3063534760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ac063af0 con 0x7fd3ac1062b0 2024-06-10T00:20:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.526+0000 7fd393fff700 1 -- 172.21.15.62:0/3063534760 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd39c003b60 con 0x7fd3ac1062b0 2024-06-10T00:20:36.224 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.601+0000 7fd3b12af700 1 -- 172.21.15.62:0/3063534760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd3ac19f620 con 0x7fd3ac1062b0 2024-06-10T00:20:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.601+0000 7fd393fff700 1 -- 172.21.15.62:0/3063534760 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd39c0199a0 con 0x7fd3ac1062b0 2024-06-10T00:20:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.602+0000 7fd3b12af700 1 -- 172.21.15.62:0/3063534760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3ac1062b0 msgr2=0x7fd3ac19bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.602+0000 7fd3b12af700 1 --2- 172.21.15.62:0/3063534760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3ac1062b0 0x7fd3ac19bf90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd39c011670 tx=0x7fd39c011b60 comp rx=0 tx=0).stop 2024-06-10T00:20:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.602+0000 7fd3b12af700 1 -- 172.21.15.62:0/3063534760 shutdown_connections 2024-06-10T00:20:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.602+0000 7fd3b12af700 1 --2- 172.21.15.62:0/3063534760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3ac1062b0 0x7fd3ac19bf90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.602+0000 7fd3b12af700 1 -- 172.21.15.62:0/3063534760 >> 172.21.15.62:0/3063534760 conn(0x7fd3ac101c20 msgr2=0x7fd3ac1026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.602+0000 7fd3b12af700 1 -- 172.21.15.62:0/3063534760 shutdown_connections 2024-06-10T00:20:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:35.602+0000 7fd3b12af700 1 -- 172.21.15.62:0/3063534760 wait complete. 2024-06-10T00:20:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (1/15)... 2024-06-10T00:20:37.407 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:37 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:37.115+0000 7f1f7e96d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:20:37.408 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:37 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:37.290+0000 7f1f7e96d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:20:37.768 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:37 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:37.469+0000 7f1f7e96d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:20:37.768 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:37 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:37.768+0000 7f1f7e96d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:20:38.078 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:37 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:37.864+0000 7f1f7e96d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:20:38.078 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:38 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:38.078+0000 7f1f7e96d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:20:38.329 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:38 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:38.286+0000 7f1f7e96d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:20:38.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:38 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1523083100' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-10T00:20:39.327 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:38 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:38.929+0000 7f1f7e96d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:20:39.332 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-06-10T00:20:39.332 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:20:39.332 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:20:39.332 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-06-10T00:20:39.332 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-10T00:20:39.332 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-10T00:20:39.332 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-10T00:20:39.332 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:39.332 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-10T00:20:39.333 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-10T00:20:39.333 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-06-10T00:20:39.333 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:20:39.333 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-10T00:20:39.333 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-06-10T00:20:39.333 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:20:39.333 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-06-10T00:20:39.333 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-10T00:20:39.333 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:39.333 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-10T00:20:39.333 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-10T00:20:39.333 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:39.333 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-10T00:20:39.334 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:39.334 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-10T00:20:39.334 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-10T00:20:39.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-10T00:20:39.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-10T00:20:39.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-10T00:20:39.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-10T00:20:39.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:39.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-10T00:20:39.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-10T00:20:39.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-10T00:20:39.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-10T00:20:39.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-10T00:20:39.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-10T00:20:39.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-10T00:20:39.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-10T00:20:39.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-10T00:20:39.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:39.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-10T00:20:39.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:39.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-10T00:20:39.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-10T00:20:39.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:39.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-10T00:20:39.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-10T00:20:39.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-10T00:20:39.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-10T00:20:39.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-10T00:20:39.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-10T00:20:39.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-06-10T00:20:39.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:20:39.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:20:39.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:39.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-10T00:20:39.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:39.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-06-10T00:20:27.982259+0000", 2024-06-10T00:20:39.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:20:39.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:39.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-10T00:20:39.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:20:39.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.636+0000 7fa35d9fb700 1 Processor -- start 2024-06-10T00:20:39.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.636+0000 7fa35d9fb700 1 -- start start 2024-06-10T00:20:39.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.636+0000 7fa35d9fb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa358108a50 0x7fa35810ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:39.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.636+0000 7fa35d9fb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa35807beb0 con 0x7fa358108a50 2024-06-10T00:20:39.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.636+0000 7fa356ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa358108a50 0x7fa35810ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:39.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.636+0000 7fa356ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa358108a50 0x7fa35810ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37172/0 (socket says 172.21.15.62:37172) 2024-06-10T00:20:39.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.637+0000 7fa356ffd700 1 -- 172.21.15.62:0/2777066904 learned_addr learned my addr 172.21.15.62:0/2777066904 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:39.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.637+0000 7fa356ffd700 1 -- 172.21.15.62:0/2777066904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35810b380 con 0x7fa358108a50 2024-06-10T00:20:39.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.637+0000 7fa356ffd700 1 --2- 172.21.15.62:0/2777066904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa358108a50 0x7fa35810ae40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa3480098a0 tx=0x7fa348009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a7d2832b3e3b9c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:39.345 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.637+0000 7fa355ffb700 1 -- 172.21.15.62:0/2777066904 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa348004160 con 0x7fa358108a50 2024-06-10T00:20:39.345 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.637+0000 7fa355ffb700 1 -- 172.21.15.62:0/2777066904 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa348004300 con 0x7fa358108a50 2024-06-10T00:20:39.345 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.637+0000 7fa355ffb700 1 -- 172.21.15.62:0/2777066904 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa348011b50 con 0x7fa358108a50 2024-06-10T00:20:39.345 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.638+0000 7fa35d9fb700 1 -- 172.21.15.62:0/2777066904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa358108a50 msgr2=0x7fa35810ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:39.345 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.638+0000 7fa35d9fb700 1 --2- 172.21.15.62:0/2777066904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa358108a50 0x7fa35810ae40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa3480098a0 tx=0x7fa348009bb0 comp rx=0 tx=0).stop 2024-06-10T00:20:39.345 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.638+0000 7fa35d9fb700 1 -- 172.21.15.62:0/2777066904 shutdown_connections 2024-06-10T00:20:39.345 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.638+0000 7fa35d9fb700 1 --2- 172.21.15.62:0/2777066904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa358108a50 0x7fa35810ae40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:39.345 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.638+0000 7fa35d9fb700 1 -- 172.21.15.62:0/2777066904 >> 172.21.15.62:0/2777066904 conn(0x7fa3581024e0 msgr2=0x7fa358104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:39.345 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.638+0000 7fa35d9fb700 1 -- 172.21.15.62:0/2777066904 shutdown_connections 2024-06-10T00:20:39.345 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.638+0000 7fa35d9fb700 1 -- 172.21.15.62:0/2777066904 wait complete. 2024-06-10T00:20:39.345 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.639+0000 7fa35d9fb700 1 Processor -- start 2024-06-10T00:20:39.345 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.639+0000 7fa35d9fb700 1 -- start start 2024-06-10T00:20:39.345 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.639+0000 7fa35d9fb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa358108a50 0x7fa35819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:39.346 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.639+0000 7fa35d9fb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa35810c3b0 con 0x7fa358108a50 2024-06-10T00:20:39.346 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.640+0000 7fa356ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa358108a50 0x7fa35819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:39.346 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.640+0000 7fa356ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa358108a50 0x7fa35819bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37180/0 (socket says 172.21.15.62:37180) 2024-06-10T00:20:39.346 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.640+0000 7fa356ffd700 1 -- 172.21.15.62:0/1523083100 learned_addr learned my addr 172.21.15.62:0/1523083100 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:39.346 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.640+0000 7fa356ffd700 1 -- 172.21.15.62:0/1523083100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35819c500 con 0x7fa358108a50 2024-06-10T00:20:39.346 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.640+0000 7fa356ffd700 1 --2- 172.21.15.62:0/1523083100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa358108a50 0x7fa35819bfc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa348009fb0 tx=0x7fa348011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:39.346 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.640+0000 7fa35c9f9700 1 -- 172.21.15.62:0/1523083100 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3480038b0 con 0x7fa358108a50 2024-06-10T00:20:39.346 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.640+0000 7fa35c9f9700 1 -- 172.21.15.62:0/1523083100 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa348011e50 con 0x7fa358108a50 2024-06-10T00:20:39.346 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.640+0000 7fa35d9fb700 1 -- 172.21.15.62:0/1523083100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35819c790 con 0x7fa358108a50 2024-06-10T00:20:39.346 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.640+0000 7fa35c9f9700 1 -- 172.21.15.62:0/1523083100 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa348019420 con 0x7fa358108a50 2024-06-10T00:20:39.346 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.640+0000 7fa35d9fb700 1 -- 172.21.15.62:0/1523083100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35819f480 con 0x7fa358108a50 2024-06-10T00:20:39.346 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.641+0000 7fa35c9f9700 1 -- 172.21.15.62:0/1523083100 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa348019a80 con 0x7fa358108a50 2024-06-10T00:20:39.347 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.641+0000 7fa35c9f9700 1 -- 172.21.15.62:0/1523083100 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa348024730 con 0x7fa358108a50 2024-06-10T00:20:39.347 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.641+0000 7fa35d9fb700 1 -- 172.21.15.62:0/1523083100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa358063af0 con 0x7fa358108a50 2024-06-10T00:20:39.347 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.643+0000 7fa35c9f9700 1 -- 172.21.15.62:0/1523083100 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa348003a50 con 0x7fa358108a50 2024-06-10T00:20:39.347 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.718+0000 7fa35d9fb700 1 -- 172.21.15.62:0/1523083100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa35819fa20 con 0x7fa358108a50 2024-06-10T00:20:39.347 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.718+0000 7fa35c9f9700 1 -- 172.21.15.62:0/1523083100 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa3480249d0 con 0x7fa358108a50 2024-06-10T00:20:39.347 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.719+0000 7fa35d9fb700 1 -- 172.21.15.62:0/1523083100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa358108a50 msgr2=0x7fa35819bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:39.347 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.719+0000 7fa35d9fb700 1 --2- 172.21.15.62:0/1523083100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa358108a50 0x7fa35819bfc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa348009fb0 tx=0x7fa348011bf0 comp rx=0 tx=0).stop 2024-06-10T00:20:39.347 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.719+0000 7fa35d9fb700 1 -- 172.21.15.62:0/1523083100 shutdown_connections 2024-06-10T00:20:39.347 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.719+0000 7fa35d9fb700 1 --2- 172.21.15.62:0/1523083100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa358108a50 0x7fa35819bfc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:39.347 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.719+0000 7fa35d9fb700 1 -- 172.21.15.62:0/1523083100 >> 172.21.15.62:0/1523083100 conn(0x7fa3581024e0 msgr2=0x7fa358103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:39.347 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.719+0000 7fa35d9fb700 1 -- 172.21.15.62:0/1523083100 shutdown_connections 2024-06-10T00:20:39.347 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:38.719+0000 7fa35d9fb700 1 -- 172.21.15.62:0/1523083100 wait complete. 2024-06-10T00:20:39.347 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (2/15)... 2024-06-10T00:20:39.607 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:39 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:39.376+0000 7f1f7e96d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:20:39.607 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:39 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:39.477+0000 7f1f7e96d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:20:40.107 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:39 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:39.689+0000 7f1f7e96d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:20:40.585 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:40 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:40.235+0000 7f1f7e96d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:20:40.585 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:40 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:40.331+0000 7f1f7e96d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:20:40.857 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:40 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:40.585+0000 7f1f7e96d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:20:40.857 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:40 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:40.726+0000 7f1f7e96d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:20:40.858 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:40 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:40.826+0000 7f1f7e96d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:20:41.259 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:40 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:40.947+0000 7f1f7e96d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:20:41.259 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:41 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:41.259+0000 7f1f7e96d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:20:42.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:41 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2097232532' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-10T00:20:42.107 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:41 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:41.912+0000 7f1f7e96d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:20:42.391 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:42 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:42.140+0000 7f1f7e96d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:20:42.397 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-06-10T00:20:42.397 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:20:42.397 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:20:42.397 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-06-10T00:20:42.397 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-10T00:20:42.397 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-10T00:20:42.397 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-10T00:20:42.397 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:42.397 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-10T00:20:42.397 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-10T00:20:42.397 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-06-10T00:20:42.397 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:20:42.397 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-10T00:20:42.398 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-06-10T00:20:42.398 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:20:42.398 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-06-10T00:20:42.398 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-10T00:20:42.398 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:42.398 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-10T00:20:42.398 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-10T00:20:42.398 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:42.398 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-10T00:20:42.398 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:42.398 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-10T00:20:42.398 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-10T00:20:42.398 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-10T00:20:42.406 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-10T00:20:42.406 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-10T00:20:42.406 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-10T00:20:42.406 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:42.406 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-10T00:20:42.406 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-10T00:20:42.406 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-10T00:20:42.407 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-10T00:20:42.407 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-10T00:20:42.407 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-10T00:20:42.407 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-10T00:20:42.407 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-10T00:20:42.407 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-10T00:20:42.407 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:42.407 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-10T00:20:42.407 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:42.407 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-10T00:20:42.407 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-10T00:20:42.407 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:42.407 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-10T00:20:42.407 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-10T00:20:42.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-10T00:20:42.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-10T00:20:42.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-10T00:20:42.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-10T00:20:42.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-06-10T00:20:42.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:20:42.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:20:42.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:42.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-10T00:20:42.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:42.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-06-10T00:20:27.982259+0000", 2024-06-10T00:20:42.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:20:42.408 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:42.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-10T00:20:42.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:20:42.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.731+0000 7fd5ba9db700 1 Processor -- start 2024-06-10T00:20:42.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.731+0000 7fd5ba9db700 1 -- start start 2024-06-10T00:20:42.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.731+0000 7fd5ba9db700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b407de60 0x7fd5b407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:42.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.731+0000 7fd5ba9db700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b407c890 con 0x7fd5b407de60 2024-06-10T00:20:42.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.731+0000 7fd5b3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b407de60 0x7fd5b407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:42.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.732+0000 7fd5b3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b407de60 0x7fd5b407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37194/0 (socket says 172.21.15.62:37194) 2024-06-10T00:20:42.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.732+0000 7fd5b3fff700 1 -- 172.21.15.62:0/3845343451 learned_addr learned my addr 172.21.15.62:0/3845343451 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:42.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.732+0000 7fd5b3fff700 1 -- 172.21.15.62:0/3845343451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b407ca10 con 0x7fd5b407de60 2024-06-10T00:20:42.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.732+0000 7fd5b3fff700 1 --2- 172.21.15.62:0/3845343451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b407de60 0x7fd5b407c2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40098a0 tx=0x7fd5a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e0adc315e07044 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:42.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.732+0000 7fd5b2ffd700 1 -- 172.21.15.62:0/3845343451 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5a4004160 con 0x7fd5b407de60 2024-06-10T00:20:42.409 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.732+0000 7fd5b2ffd700 1 -- 172.21.15.62:0/3845343451 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5a4004300 con 0x7fd5b407de60 2024-06-10T00:20:42.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.732+0000 7fd5b2ffd700 1 -- 172.21.15.62:0/3845343451 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5a4011b50 con 0x7fd5b407de60 2024-06-10T00:20:42.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.733+0000 7fd5ba9db700 1 -- 172.21.15.62:0/3845343451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b407de60 msgr2=0x7fd5b407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:42.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.733+0000 7fd5ba9db700 1 --2- 172.21.15.62:0/3845343451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b407de60 0x7fd5b407c2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40098a0 tx=0x7fd5a4009bb0 comp rx=0 tx=0).stop 2024-06-10T00:20:42.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.733+0000 7fd5ba9db700 1 -- 172.21.15.62:0/3845343451 shutdown_connections 2024-06-10T00:20:42.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.733+0000 7fd5ba9db700 1 --2- 172.21.15.62:0/3845343451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b407de60 0x7fd5b407c2c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:42.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.733+0000 7fd5ba9db700 1 -- 172.21.15.62:0/3845343451 >> 172.21.15.62:0/3845343451 conn(0x7fd5b4103630 msgr2=0x7fd5b4105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:42.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.733+0000 7fd5ba9db700 1 -- 172.21.15.62:0/3845343451 shutdown_connections 2024-06-10T00:20:42.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.733+0000 7fd5ba9db700 1 -- 172.21.15.62:0/3845343451 wait complete. 2024-06-10T00:20:42.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.734+0000 7fd5ba9db700 1 Processor -- start 2024-06-10T00:20:42.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.734+0000 7fd5ba9db700 1 -- start start 2024-06-10T00:20:42.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.734+0000 7fd5ba9db700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b407de60 0x7fd5b419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:42.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.734+0000 7fd5ba9db700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b410e5a0 con 0x7fd5b407de60 2024-06-10T00:20:42.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.734+0000 7fd5b3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b407de60 0x7fd5b419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:42.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.734+0000 7fd5b3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b407de60 0x7fd5b419e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37198/0 (socket says 172.21.15.62:37198) 2024-06-10T00:20:42.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.734+0000 7fd5b3fff700 1 -- 172.21.15.62:0/2097232532 learned_addr learned my addr 172.21.15.62:0/2097232532 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:42.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.735+0000 7fd5b3fff700 1 -- 172.21.15.62:0/2097232532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b419e6b0 con 0x7fd5b407de60 2024-06-10T00:20:42.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.735+0000 7fd5b3fff700 1 --2- 172.21.15.62:0/2097232532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b407de60 0x7fd5b419e170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4011670 tx=0x7fd5a4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:42.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.735+0000 7fd5b17fa700 1 -- 172.21.15.62:0/2097232532 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5a40039c0 con 0x7fd5b407de60 2024-06-10T00:20:42.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.735+0000 7fd5b17fa700 1 -- 172.21.15.62:0/2097232532 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5a4011e50 con 0x7fd5b407de60 2024-06-10T00:20:42.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.735+0000 7fd5ba9db700 1 -- 172.21.15.62:0/2097232532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b419e940 con 0x7fd5b407de60 2024-06-10T00:20:42.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.735+0000 7fd5b17fa700 1 -- 172.21.15.62:0/2097232532 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5a401a5c0 con 0x7fd5b407de60 2024-06-10T00:20:42.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.735+0000 7fd5ba9db700 1 -- 172.21.15.62:0/2097232532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b41a1630 con 0x7fd5b407de60 2024-06-10T00:20:42.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.735+0000 7fd5b17fa700 1 -- 172.21.15.62:0/2097232532 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd5a401ac00 con 0x7fd5b407de60 2024-06-10T00:20:42.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.735+0000 7fd5b17fa700 1 -- 172.21.15.62:0/2097232532 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd5a4019700 con 0x7fd5b407de60 2024-06-10T00:20:42.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.736+0000 7fd5ba9db700 1 -- 172.21.15.62:0/2097232532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b4063af0 con 0x7fd5b407de60 2024-06-10T00:20:42.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.738+0000 7fd5b17fa700 1 -- 172.21.15.62:0/2097232532 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd5a4003b60 con 0x7fd5b407de60 2024-06-10T00:20:42.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.812+0000 7fd5ba9db700 1 -- 172.21.15.62:0/2097232532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd5b41a1800 con 0x7fd5b407de60 2024-06-10T00:20:42.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.812+0000 7fd5b17fa700 1 -- 172.21.15.62:0/2097232532 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd5a4019900 con 0x7fd5b407de60 2024-06-10T00:20:42.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.813+0000 7fd5ba9db700 1 -- 172.21.15.62:0/2097232532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b407de60 msgr2=0x7fd5b419e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:42.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.813+0000 7fd5ba9db700 1 --2- 172.21.15.62:0/2097232532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b407de60 0x7fd5b419e170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4011670 tx=0x7fd5a4011b60 comp rx=0 tx=0).stop 2024-06-10T00:20:42.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.813+0000 7fd5ba9db700 1 -- 172.21.15.62:0/2097232532 shutdown_connections 2024-06-10T00:20:42.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.814+0000 7fd5ba9db700 1 --2- 172.21.15.62:0/2097232532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5b407de60 0x7fd5b419e170 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:42.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.814+0000 7fd5ba9db700 1 -- 172.21.15.62:0/2097232532 >> 172.21.15.62:0/2097232532 conn(0x7fd5b4103630 msgr2=0x7fd5b4104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:42.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.814+0000 7fd5ba9db700 1 -- 172.21.15.62:0/2097232532 shutdown_connections 2024-06-10T00:20:42.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:41.814+0000 7fd5ba9db700 1 -- 172.21.15.62:0/2097232532 wait complete. 2024-06-10T00:20:42.412 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (3/15)... 2024-06-10T00:20:43.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:42 smithi062 ceph-mon[32390]: Activating manager daemon x 2024-06-10T00:20:43.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:42 smithi062 ceph-mon[32390]: mgrmap e2: x(active, starting, since 0.00317442s) 2024-06-10T00:20:43.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:42 smithi062 ceph-mon[32390]: from='mgr.14100 172.21.15.62:0/1943718221' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T00:20:43.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:42 smithi062 ceph-mon[32390]: from='mgr.14100 172.21.15.62:0/1943718221' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T00:20:43.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:42 smithi062 ceph-mon[32390]: from='mgr.14100 172.21.15.62:0/1943718221' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T00:20:43.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:42 smithi062 ceph-mon[32390]: from='mgr.14100 172.21.15.62:0/1943718221' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:20:43.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:42 smithi062 ceph-mon[32390]: from='mgr.14100 172.21.15.62:0/1943718221' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T00:20:43.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:42 smithi062 ceph-mon[32390]: Manager daemon x is now available 2024-06-10T00:20:43.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:42 smithi062 ceph-mon[32390]: from='mgr.14100 172.21.15.62:0/1943718221' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T00:20:43.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:42 smithi062 ceph-mon[32390]: from='mgr.14100 172.21.15.62:0/1943718221' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T00:20:43.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:42 smithi062 ceph-mon[32390]: from='mgr.14100 172.21.15.62:0/1943718221' entity='mgr.x' 2024-06-10T00:20:43.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:42 smithi062 ceph-mon[32390]: from='mgr.14100 172.21.15.62:0/1943718221' entity='mgr.x' 2024-06-10T00:20:43.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:42 smithi062 ceph-mon[32390]: from='mgr.14100 172.21.15.62:0/1943718221' entity='mgr.x' 2024-06-10T00:20:44.527 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:44 smithi062 ceph-mon[32390]: mgrmap e3: x(active, since 1.00737s) 2024-06-10T00:20:45.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:45 smithi062 ceph-mon[32390]: mgrmap e4: x(active, since 2s) 2024-06-10T00:20:45.423 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:45 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2100028205' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-10T00:20:45.695 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-06-10T00:20:45.695 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:20:45.695 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:20:45.695 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-06-10T00:20:45.695 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-10T00:20:45.695 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-10T00:20:45.695 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-10T00:20:45.695 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:45.695 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-10T00:20:45.695 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-10T00:20:45.696 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-06-10T00:20:45.696 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:20:45.696 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-10T00:20:45.696 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-06-10T00:20:45.696 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:20:45.696 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-06-10T00:20:45.696 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-10T00:20:45.696 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:45.696 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-10T00:20:45.696 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-10T00:20:45.696 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:45.696 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-10T00:20:45.696 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:45.697 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-10T00:20:45.697 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-10T00:20:45.697 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-10T00:20:45.704 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-10T00:20:45.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-10T00:20:45.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-10T00:20:45.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:45.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-10T00:20:45.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-10T00:20:45.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-10T00:20:45.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-10T00:20:45.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-10T00:20:45.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-10T00:20:45.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-10T00:20:45.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-10T00:20:45.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-10T00:20:45.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:45.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-10T00:20:45.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:45.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-10T00:20:45.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-10T00:20:45.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:45.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-10T00:20:45.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-10T00:20:45.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-10T00:20:45.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-10T00:20:45.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-10T00:20:45.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-10T00:20:45.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-06-10T00:20:45.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:20:45.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:20:45.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:45.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-10T00:20:45.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:20:45.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-06-10T00:20:27.982259+0000", 2024-06-10T00:20:45.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:20:45.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:20:45.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-10T00:20:45.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:20:45.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.813+0000 7fc6db988700 1 Processor -- start 2024-06-10T00:20:45.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.813+0000 7fc6db988700 1 -- start start 2024-06-10T00:20:45.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.813+0000 7fc6db988700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6d407dea0 0x7fc6d407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:45.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.813+0000 7fc6db988700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d407c8d0 con 0x7fc6d407dea0 2024-06-10T00:20:45.708 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.814+0000 7fc6d9724700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6d407dea0 0x7fc6d407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:45.708 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.814+0000 7fc6d9724700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6d407dea0 0x7fc6d407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37274/0 (socket says 172.21.15.62:37274) 2024-06-10T00:20:45.708 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.814+0000 7fc6d9724700 1 -- 172.21.15.62:0/4074534498 learned_addr learned my addr 172.21.15.62:0/4074534498 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:45.708 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.814+0000 7fc6d9724700 1 -- 172.21.15.62:0/4074534498 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d407ca50 con 0x7fc6d407dea0 2024-06-10T00:20:45.708 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.814+0000 7fc6d9724700 1 --2- 172.21.15.62:0/4074534498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6d407dea0 0x7fc6d407c300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009e80 tx=0x7fc6c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af3be0784b321f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:45.708 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.814+0000 7fc6d3fff700 1 -- 172.21.15.62:0/4074534498 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6c4017070 con 0x7fc6d407dea0 2024-06-10T00:20:45.708 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.814+0000 7fc6d3fff700 1 -- 172.21.15.62:0/4074534498 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc6c400b8c0 con 0x7fc6d407dea0 2024-06-10T00:20:45.708 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.814+0000 7fc6d3fff700 1 -- 172.21.15.62:0/4074534498 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6c40045c0 con 0x7fc6d407dea0 2024-06-10T00:20:45.708 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.815+0000 7fc6db988700 1 -- 172.21.15.62:0/4074534498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6d407dea0 msgr2=0x7fc6d407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:45.708 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.815+0000 7fc6db988700 1 --2- 172.21.15.62:0/4074534498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6d407dea0 0x7fc6d407c300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009e80 tx=0x7fc6c400b330 comp rx=0 tx=0).stop 2024-06-10T00:20:45.708 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.815+0000 7fc6db988700 1 -- 172.21.15.62:0/4074534498 shutdown_connections 2024-06-10T00:20:45.708 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.815+0000 7fc6db988700 1 --2- 172.21.15.62:0/4074534498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6d407dea0 0x7fc6d407c300 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:45.709 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.815+0000 7fc6db988700 1 -- 172.21.15.62:0/4074534498 >> 172.21.15.62:0/4074534498 conn(0x7fc6d41035c0 msgr2=0x7fc6d41059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:45.709 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.815+0000 7fc6db988700 1 -- 172.21.15.62:0/4074534498 shutdown_connections 2024-06-10T00:20:45.709 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.815+0000 7fc6db988700 1 -- 172.21.15.62:0/4074534498 wait complete. 2024-06-10T00:20:45.709 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.816+0000 7fc6db988700 1 Processor -- start 2024-06-10T00:20:45.709 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.816+0000 7fc6db988700 1 -- start start 2024-06-10T00:20:45.709 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.816+0000 7fc6db988700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6d407dea0 0x7fc6d419e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:45.709 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.816+0000 7fc6db988700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d410e5b0 con 0x7fc6d407dea0 2024-06-10T00:20:45.709 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.816+0000 7fc6d9724700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6d407dea0 0x7fc6d419e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:45.709 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.816+0000 7fc6d9724700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6d407dea0 0x7fc6d419e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37284/0 (socket says 172.21.15.62:37284) 2024-06-10T00:20:45.709 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.816+0000 7fc6d9724700 1 -- 172.21.15.62:0/2100028205 learned_addr learned my addr 172.21.15.62:0/2100028205 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:45.709 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.816+0000 7fc6d9724700 1 -- 172.21.15.62:0/2100028205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d419e660 con 0x7fc6d407dea0 2024-06-10T00:20:45.709 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.816+0000 7fc6d9724700 1 --2- 172.21.15.62:0/2100028205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6d407dea0 0x7fc6d419e120 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40072a0 tx=0x7fc6c400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:45.709 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.816+0000 7fc6d27fc700 1 -- 172.21.15.62:0/2100028205 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6c4020020 con 0x7fc6d407dea0 2024-06-10T00:20:45.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.817+0000 7fc6d27fc700 1 -- 172.21.15.62:0/2100028205 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc6c4003b50 con 0x7fc6d407dea0 2024-06-10T00:20:45.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.817+0000 7fc6d27fc700 1 -- 172.21.15.62:0/2100028205 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6c4017040 con 0x7fc6d407dea0 2024-06-10T00:20:45.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.817+0000 7fc6db988700 1 -- 172.21.15.62:0/2100028205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d419e8f0 con 0x7fc6d407dea0 2024-06-10T00:20:45.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.817+0000 7fc6db988700 1 -- 172.21.15.62:0/2100028205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d41a15e0 con 0x7fc6d407dea0 2024-06-10T00:20:45.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.817+0000 7fc6d27fc700 1 -- 172.21.15.62:0/2100028205 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fc6c4023020 con 0x7fc6d407dea0 2024-06-10T00:20:45.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.818+0000 7fc6db988700 1 -- 172.21.15.62:0/2100028205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d4063af0 con 0x7fc6d407dea0 2024-06-10T00:20:45.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.818+0000 7fc6d27fc700 1 --2- 172.21.15.62:0/2100028205 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fc6bc038fb0 0x7fc6bc03b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:45.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.818+0000 7fc6d27fc700 1 -- 172.21.15.62:0/2100028205 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc6c400f050 con 0x7fc6d407dea0 2024-06-10T00:20:45.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.818+0000 7fc6d8f23700 1 --2- 172.21.15.62:0/2100028205 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fc6bc038fb0 0x7fc6bc03b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:45.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.818+0000 7fc6d8f23700 1 --2- 172.21.15.62:0/2100028205 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fc6bc038fb0 0x7fc6bc03b470 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8006fd0 tx=0x7fc6c8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:45.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:44.823+0000 7fc6d27fc700 1 -- 172.21.15.62:0/2100028205 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc6c402a020 con 0x7fc6d407dea0 2024-06-10T00:20:45.710 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:45.112+0000 7fc6db988700 1 -- 172.21.15.62:0/2100028205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc6d41a1cd0 con 0x7fc6d407dea0 2024-06-10T00:20:45.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:45.113+0000 7fc6d27fc700 1 -- 172.21.15.62:0/2100028205 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc6c4003db0 con 0x7fc6d407dea0 2024-06-10T00:20:45.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:45.115+0000 7fc6db988700 1 -- 172.21.15.62:0/2100028205 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fc6bc038fb0 msgr2=0x7fc6bc03b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:45.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:45.115+0000 7fc6db988700 1 --2- 172.21.15.62:0/2100028205 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fc6bc038fb0 0x7fc6bc03b470 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8006fd0 tx=0x7fc6c8006e40 comp rx=0 tx=0).stop 2024-06-10T00:20:45.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:45.115+0000 7fc6db988700 1 -- 172.21.15.62:0/2100028205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6d407dea0 msgr2=0x7fc6d419e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:45.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:45.115+0000 7fc6db988700 1 --2- 172.21.15.62:0/2100028205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6d407dea0 0x7fc6d419e120 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40072a0 tx=0x7fc6c400bd50 comp rx=0 tx=0).stop 2024-06-10T00:20:45.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:45.115+0000 7fc6db988700 1 -- 172.21.15.62:0/2100028205 shutdown_connections 2024-06-10T00:20:45.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:45.115+0000 7fc6db988700 1 --2- 172.21.15.62:0/2100028205 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fc6bc038fb0 0x7fc6bc03b470 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:45.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:45.115+0000 7fc6db988700 1 --2- 172.21.15.62:0/2100028205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6d407dea0 0x7fc6d419e120 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:45.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:45.115+0000 7fc6db988700 1 -- 172.21.15.62:0/2100028205 >> 172.21.15.62:0/2100028205 conn(0x7fc6d41035c0 msgr2=0x7fc6d4104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:45.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:45.115+0000 7fc6db988700 1 -- 172.21.15.62:0/2100028205 shutdown_connections 2024-06-10T00:20:45.711 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:45.115+0000 7fc6db988700 1 -- 172.21.15.62:0/2100028205 wait complete. 2024-06-10T00:20:45.711 INFO:teuthology.orchestra.run.smithi062.stdout:mgr is available 2024-06-10T00:20:46.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:46 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2066592321' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-10T00:20:47.190 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-06-10T00:20:47.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [global] 2024-06-10T00:20:47.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout fsid = 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:20:47.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-10T00:20:47.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-10T00:20:47.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-10T00:20:47.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-10T00:20:47.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-06-10T00:20:47.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-10T00:20:47.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-10T00:20:47.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-06-10T00:20:47.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [osd] 2024-06-10T00:20:47.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-06-10T00:20:47.191 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-10T00:20:47.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-10T00:20:47.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-10T00:20:47.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-10T00:20:47.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-10T00:20:47.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.346+0000 7f042017f700 1 Processor -- start 2024-06-10T00:20:47.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.346+0000 7f042017f700 1 -- start start 2024-06-10T00:20:47.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.347+0000 7f042017f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04181068d0 0x7f0418106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:47.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.347+0000 7f042017f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0418075be0 con 0x7f04181068d0 2024-06-10T00:20:47.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.347+0000 7f041df1b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04181068d0 0x7f0418106cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:47.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.347+0000 7f041df1b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04181068d0 0x7f0418106cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37296/0 (socket says 172.21.15.62:37296) 2024-06-10T00:20:47.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.347+0000 7f041df1b700 1 -- 172.21.15.62:0/3436513739 learned_addr learned my addr 172.21.15.62:0/3436513739 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:47.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.347+0000 7f041df1b700 1 -- 172.21.15.62:0/3436513739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0418107230 con 0x7f04181068d0 2024-06-10T00:20:47.192 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.347+0000 7f041df1b700 1 --2- 172.21.15.62:0/3436513739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04181068d0 0x7f0418106cf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0408009e80 tx=0x7f040800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45a4f81d8cd47c4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:47.193 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.347+0000 7f041cf19700 1 -- 172.21.15.62:0/3436513739 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0408017070 con 0x7f04181068d0 2024-06-10T00:20:47.193 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.347+0000 7f041cf19700 1 -- 172.21.15.62:0/3436513739 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f040800b8c0 con 0x7f04181068d0 2024-06-10T00:20:47.193 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.347+0000 7f041cf19700 1 -- 172.21.15.62:0/3436513739 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04080045c0 con 0x7f04181068d0 2024-06-10T00:20:47.193 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.348+0000 7f042017f700 1 -- 172.21.15.62:0/3436513739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04181068d0 msgr2=0x7f0418106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:47.193 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.348+0000 7f042017f700 1 --2- 172.21.15.62:0/3436513739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04181068d0 0x7f0418106cf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0408009e80 tx=0x7f040800b330 comp rx=0 tx=0).stop 2024-06-10T00:20:47.193 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.348+0000 7f042017f700 1 -- 172.21.15.62:0/3436513739 shutdown_connections 2024-06-10T00:20:47.193 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.348+0000 7f042017f700 1 --2- 172.21.15.62:0/3436513739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04181068d0 0x7f0418106cf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:47.193 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.348+0000 7f042017f700 1 -- 172.21.15.62:0/3436513739 >> 172.21.15.62:0/3436513739 conn(0x7f04181024b0 msgr2=0x7f04181048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:47.193 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.348+0000 7f042017f700 1 -- 172.21.15.62:0/3436513739 shutdown_connections 2024-06-10T00:20:47.193 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.348+0000 7f042017f700 1 -- 172.21.15.62:0/3436513739 wait complete. 2024-06-10T00:20:47.193 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.349+0000 7f042017f700 1 Processor -- start 2024-06-10T00:20:47.193 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.349+0000 7f042017f700 1 -- start start 2024-06-10T00:20:47.193 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.349+0000 7f042017f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04181068d0 0x7f04181a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:47.194 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.349+0000 7f042017f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f04181082c0 con 0x7f04181068d0 2024-06-10T00:20:47.194 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.349+0000 7f041df1b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04181068d0 0x7f04181a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:47.194 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.349+0000 7f041df1b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04181068d0 0x7f04181a2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37302/0 (socket says 172.21.15.62:37302) 2024-06-10T00:20:47.194 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.349+0000 7f041df1b700 1 -- 172.21.15.62:0/2066592321 learned_addr learned my addr 172.21.15.62:0/2066592321 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:47.194 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.349+0000 7f041df1b700 1 -- 172.21.15.62:0/2066592321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04181a2ad0 con 0x7f04181068d0 2024-06-10T00:20:47.194 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.350+0000 7f041df1b700 1 --2- 172.21.15.62:0/2066592321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04181068d0 0x7f04181a2590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0408009530 tx=0x7f040800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:47.194 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.350+0000 7f0416ffd700 1 -- 172.21.15.62:0/2066592321 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0408020020 con 0x7f04181068d0 2024-06-10T00:20:47.194 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.350+0000 7f0416ffd700 1 -- 172.21.15.62:0/2066592321 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0408003b50 con 0x7f04181068d0 2024-06-10T00:20:47.194 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.350+0000 7f042017f700 1 -- 172.21.15.62:0/2066592321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04181a2d60 con 0x7f04181068d0 2024-06-10T00:20:47.194 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.350+0000 7f0416ffd700 1 -- 172.21.15.62:0/2066592321 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0408017040 con 0x7f04181068d0 2024-06-10T00:20:47.194 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.350+0000 7f042017f700 1 -- 172.21.15.62:0/2066592321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04181a5a50 con 0x7f04181068d0 2024-06-10T00:20:47.194 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.351+0000 7f0416ffd700 1 -- 172.21.15.62:0/2066592321 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f0408023020 con 0x7f04181068d0 2024-06-10T00:20:47.194 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.351+0000 7f042017f700 1 -- 172.21.15.62:0/2066592321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f041810eb40 con 0x7f04181068d0 2024-06-10T00:20:47.195 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.351+0000 7f0416ffd700 1 --2- 172.21.15.62:0/2066592321 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7f0400039050 0x7f040003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:47.195 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.351+0000 7f0416ffd700 1 -- 172.21.15.62:0/2066592321 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f040800f050 con 0x7f04181068d0 2024-06-10T00:20:47.195 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.351+0000 7f041d71a700 1 --2- 172.21.15.62:0/2066592321 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7f0400039050 0x7f040003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:47.195 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.352+0000 7f041d71a700 1 --2- 172.21.15.62:0/2066592321 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7f0400039050 0x7f040003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f040c006fd0 tx=0x7f040c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:47.195 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.356+0000 7f0416ffd700 1 -- 172.21.15.62:0/2066592321 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f040802a020 con 0x7f04181068d0 2024-06-10T00:20:47.195 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.562+0000 7f042017f700 1 -- 172.21.15.62:0/2066592321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f041802e010 con 0x7f04181068d0 2024-06-10T00:20:47.195 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.563+0000 7f0416ffd700 1 -- 172.21.15.62:0/2066592321 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+398 (secure 0 0 0) 0x7f040801b730 con 0x7f04181068d0 2024-06-10T00:20:47.195 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.565+0000 7f042017f700 1 -- 172.21.15.62:0/2066592321 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7f0400039050 msgr2=0x7f040003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:47.195 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.565+0000 7f042017f700 1 --2- 172.21.15.62:0/2066592321 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7f0400039050 0x7f040003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f040c006fd0 tx=0x7f040c006e40 comp rx=0 tx=0).stop 2024-06-10T00:20:47.195 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.565+0000 7f042017f700 1 -- 172.21.15.62:0/2066592321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04181068d0 msgr2=0x7f04181a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:47.195 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.565+0000 7f042017f700 1 --2- 172.21.15.62:0/2066592321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04181068d0 0x7f04181a2590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0408009530 tx=0x7f040800bd50 comp rx=0 tx=0).stop 2024-06-10T00:20:47.195 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.565+0000 7f042017f700 1 -- 172.21.15.62:0/2066592321 shutdown_connections 2024-06-10T00:20:47.195 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.565+0000 7f042017f700 1 --2- 172.21.15.62:0/2066592321 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7f0400039050 0x7f040003b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:47.196 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.565+0000 7f042017f700 1 --2- 172.21.15.62:0/2066592321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04181068d0 0x7f04181a2590 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:47.196 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.565+0000 7f042017f700 1 -- 172.21.15.62:0/2066592321 >> 172.21.15.62:0/2066592321 conn(0x7f04181024b0 msgr2=0x7f041810bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:47.196 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.565+0000 7f042017f700 1 -- 172.21.15.62:0/2066592321 shutdown_connections 2024-06-10T00:20:47.196 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:46.565+0000 7f042017f700 1 -- 172.21.15.62:0/2066592321 wait complete. 2024-06-10T00:20:47.196 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling cephadm module... 2024-06-10T00:20:48.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:47 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2161321258' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-10T00:20:48.108 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:47 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: ignoring --setuser ceph since I am not root 2024-06-10T00:20:48.108 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:47 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: ignoring --setgroup ceph since I am not root 2024-06-10T00:20:48.108 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:48 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:48.070+0000 7fbe412a6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:20:48.479 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:48 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:48.266+0000 7fbe412a6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:20:48.479 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:48 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:48.373+0000 7fbe412a6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:20:48.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.568+0000 7fd51679e700 1 Processor -- start 2024-06-10T00:20:48.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.568+0000 7fd51679e700 1 -- start start 2024-06-10T00:20:48.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.568+0000 7fd51679e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd510107de0 0x7fd510108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:48.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.568+0000 7fd51679e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd5101087d0 con 0x7fd510107de0 2024-06-10T00:20:48.487 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.568+0000 7fd51579c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd510107de0 0x7fd510108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:48.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.568+0000 7fd51579c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd510107de0 0x7fd510108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37306/0 (socket says 172.21.15.62:37306) 2024-06-10T00:20:48.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.568+0000 7fd51579c700 1 -- 172.21.15.62:0/3156194768 learned_addr learned my addr 172.21.15.62:0/3156194768 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:48.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.569+0000 7fd51579c700 1 -- 172.21.15.62:0/3156194768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd510108fe0 con 0x7fd510107de0 2024-06-10T00:20:48.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.569+0000 7fd51579c700 1 --2- 172.21.15.62:0/3156194768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd510107de0 0x7fd510108200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0098a0 tx=0x7fd50c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e1cd4b75554e2c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:48.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.569+0000 7fd503fff700 1 -- 172.21.15.62:0/3156194768 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd50c004160 con 0x7fd510107de0 2024-06-10T00:20:48.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.569+0000 7fd503fff700 1 -- 172.21.15.62:0/3156194768 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd50c004300 con 0x7fd510107de0 2024-06-10T00:20:48.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.569+0000 7fd503fff700 1 -- 172.21.15.62:0/3156194768 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd50c011b50 con 0x7fd510107de0 2024-06-10T00:20:48.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.570+0000 7fd51679e700 1 -- 172.21.15.62:0/3156194768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd510107de0 msgr2=0x7fd510108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:48.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.570+0000 7fd51679e700 1 --2- 172.21.15.62:0/3156194768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd510107de0 0x7fd510108200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0098a0 tx=0x7fd50c009bb0 comp rx=0 tx=0).stop 2024-06-10T00:20:48.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.570+0000 7fd51679e700 1 -- 172.21.15.62:0/3156194768 shutdown_connections 2024-06-10T00:20:48.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.570+0000 7fd51679e700 1 --2- 172.21.15.62:0/3156194768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd510107de0 0x7fd510108200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:48.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.570+0000 7fd51679e700 1 -- 172.21.15.62:0/3156194768 >> 172.21.15.62:0/3156194768 conn(0x7fd510103380 msgr2=0x7fd5101057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:48.488 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.570+0000 7fd51679e700 1 -- 172.21.15.62:0/3156194768 shutdown_connections 2024-06-10T00:20:48.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.570+0000 7fd51679e700 1 -- 172.21.15.62:0/3156194768 wait complete. 2024-06-10T00:20:48.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.571+0000 7fd51679e700 1 Processor -- start 2024-06-10T00:20:48.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.571+0000 7fd51679e700 1 -- start start 2024-06-10T00:20:48.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.571+0000 7fd51679e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd510107de0 0x7fd51019be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:48.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.571+0000 7fd51679e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd51010c390 con 0x7fd510107de0 2024-06-10T00:20:48.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.571+0000 7fd51579c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd510107de0 0x7fd51019be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:48.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.571+0000 7fd51579c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd510107de0 0x7fd51019be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37314/0 (socket says 172.21.15.62:37314) 2024-06-10T00:20:48.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.571+0000 7fd51579c700 1 -- 172.21.15.62:0/2161321258 learned_addr learned my addr 172.21.15.62:0/2161321258 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:48.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.571+0000 7fd51579c700 1 -- 172.21.15.62:0/2161321258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51019c360 con 0x7fd510107de0 2024-06-10T00:20:48.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.571+0000 7fd51579c700 1 --2- 172.21.15.62:0/2161321258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd510107de0 0x7fd51019be20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009fb0 tx=0x7fd50c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:48.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.572+0000 7fd5027fc700 1 -- 172.21.15.62:0/2161321258 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd50c0038b0 con 0x7fd510107de0 2024-06-10T00:20:48.489 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.572+0000 7fd5027fc700 1 -- 172.21.15.62:0/2161321258 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd50c011e50 con 0x7fd510107de0 2024-06-10T00:20:48.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.572+0000 7fd51679e700 1 -- 172.21.15.62:0/2161321258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51019c5f0 con 0x7fd510107de0 2024-06-10T00:20:48.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.572+0000 7fd5027fc700 1 -- 172.21.15.62:0/2161321258 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd50c019550 con 0x7fd510107de0 2024-06-10T00:20:48.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.572+0000 7fd51679e700 1 -- 172.21.15.62:0/2161321258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51019f2e0 con 0x7fd510107de0 2024-06-10T00:20:48.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.573+0000 7fd51679e700 1 -- 172.21.15.62:0/2161321258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5100695d0 con 0x7fd510107de0 2024-06-10T00:20:48.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.573+0000 7fd5027fc700 1 -- 172.21.15.62:0/2161321258 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fd50c01a430 con 0x7fd510107de0 2024-06-10T00:20:48.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.573+0000 7fd5027fc700 1 --2- 172.21.15.62:0/2161321258 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd4fc038fb0 0x7fd4fc03b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:48.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.573+0000 7fd5027fc700 1 -- 172.21.15.62:0/2161321258 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd50c04d1a0 con 0x7fd510107de0 2024-06-10T00:20:48.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.573+0000 7fd514f9b700 1 --2- 172.21.15.62:0/2161321258 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd4fc038fb0 0x7fd4fc03b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:48.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.573+0000 7fd514f9b700 1 --2- 172.21.15.62:0/2161321258 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd4fc038fb0 0x7fd4fc03b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd504006fd0 tx=0x7fd504006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:48.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.578+0000 7fd5027fc700 1 -- 172.21.15.62:0/2161321258 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd50c016db0 con 0x7fd510107de0 2024-06-10T00:20:48.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.842+0000 7fd51679e700 1 -- 172.21.15.62:0/2161321258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fd51019f4b0 con 0x7fd510107de0 2024-06-10T00:20:48.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.893+0000 7fd5027fc700 1 -- 172.21.15.62:0/2161321258 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fd51019f4b0 con 0x7fd510107de0 2024-06-10T00:20:48.490 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.894+0000 7fd5027fc700 1 -- 172.21.15.62:0/2161321258 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fd50c01a720 con 0x7fd510107de0 2024-06-10T00:20:48.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.896+0000 7fd51679e700 1 -- 172.21.15.62:0/2161321258 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd4fc038fb0 msgr2=0x7fd4fc03b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:48.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.896+0000 7fd51679e700 1 --2- 172.21.15.62:0/2161321258 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd4fc038fb0 0x7fd4fc03b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd504006fd0 tx=0x7fd504006e40 comp rx=0 tx=0).stop 2024-06-10T00:20:48.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.896+0000 7fd51679e700 1 -- 172.21.15.62:0/2161321258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd510107de0 msgr2=0x7fd51019be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:48.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.896+0000 7fd51679e700 1 --2- 172.21.15.62:0/2161321258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd510107de0 0x7fd51019be20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009fb0 tx=0x7fd50c011bf0 comp rx=0 tx=0).stop 2024-06-10T00:20:48.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.896+0000 7fd51679e700 1 -- 172.21.15.62:0/2161321258 shutdown_connections 2024-06-10T00:20:48.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.896+0000 7fd51679e700 1 --2- 172.21.15.62:0/2161321258 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd4fc038fb0 0x7fd4fc03b470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:48.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.896+0000 7fd51679e700 1 --2- 172.21.15.62:0/2161321258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd510107de0 0x7fd51019be20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:48.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.896+0000 7fd51679e700 1 -- 172.21.15.62:0/2161321258 >> 172.21.15.62:0/2161321258 conn(0x7fd510103380 msgr2=0x7fd510103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:48.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.896+0000 7fd51679e700 1 -- 172.21.15.62:0/2161321258 shutdown_connections 2024-06-10T00:20:48.491 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:47.896+0000 7fd51679e700 1 -- 172.21.15.62:0/2161321258 wait complete. 2024-06-10T00:20:48.782 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:48 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:48.704+0000 7fbe412a6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:20:49.091 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:48 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:48.794+0000 7fbe412a6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:20:49.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:48 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2161321258' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-10T00:20:49.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:48 smithi062 ceph-mon[32390]: mgrmap e5: x(active, since 5s) 2024-06-10T00:20:49.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:20:49.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-10T00:20:49.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-10T00:20:49.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-10T00:20:49.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-10T00:20:49.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:20:49.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.886+0000 7f652f3f0700 1 Processor -- start 2024-06-10T00:20:49.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.886+0000 7f652f3f0700 1 -- start start 2024-06-10T00:20:49.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.886+0000 7f652f3f0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528106e80 0x7f65281072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:49.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.886+0000 7f652f3f0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f65281077e0 con 0x7f6528106e80 2024-06-10T00:20:49.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.886+0000 7f652d18c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528106e80 0x7f65281072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:49.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.886+0000 7f652d18c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528106e80 0x7f65281072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44668/0 (socket says 172.21.15.62:44668) 2024-06-10T00:20:49.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.886+0000 7f652d18c700 1 -- 172.21.15.62:0/3808918308 learned_addr learned my addr 172.21.15.62:0/3808918308 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:49.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.886+0000 7f652d18c700 1 -- 172.21.15.62:0/3808918308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6528108070 con 0x7f6528106e80 2024-06-10T00:20:49.755 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.887+0000 7f652d18c700 1 --2- 172.21.15.62:0/3808918308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528106e80 0x7f65281072a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65240098a0 tx=0x7f6524009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28c062923a376c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:49.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.887+0000 7f651ffff700 1 -- 172.21.15.62:0/3808918308 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6524004160 con 0x7f6528106e80 2024-06-10T00:20:49.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.887+0000 7f651ffff700 1 -- 172.21.15.62:0/3808918308 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6524004300 con 0x7f6528106e80 2024-06-10T00:20:49.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.887+0000 7f651ffff700 1 -- 172.21.15.62:0/3808918308 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6524011b50 con 0x7f6528106e80 2024-06-10T00:20:49.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.887+0000 7f652f3f0700 1 -- 172.21.15.62:0/3808918308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528106e80 msgr2=0x7f65281072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:49.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.887+0000 7f652f3f0700 1 --2- 172.21.15.62:0/3808918308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528106e80 0x7f65281072a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65240098a0 tx=0x7f6524009bb0 comp rx=0 tx=0).stop 2024-06-10T00:20:49.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.887+0000 7f652f3f0700 1 -- 172.21.15.62:0/3808918308 shutdown_connections 2024-06-10T00:20:49.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.887+0000 7f652f3f0700 1 --2- 172.21.15.62:0/3808918308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528106e80 0x7f65281072a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:49.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.887+0000 7f652f3f0700 1 -- 172.21.15.62:0/3808918308 >> 172.21.15.62:0/3808918308 conn(0x7f65281024b0 msgr2=0x7f65281048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:49.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.888+0000 7f652f3f0700 1 -- 172.21.15.62:0/3808918308 shutdown_connections 2024-06-10T00:20:49.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.888+0000 7f652f3f0700 1 -- 172.21.15.62:0/3808918308 wait complete. 2024-06-10T00:20:49.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.888+0000 7f652f3f0700 1 Processor -- start 2024-06-10T00:20:49.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.888+0000 7f652f3f0700 1 -- start start 2024-06-10T00:20:49.756 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.888+0000 7f652f3f0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528106e80 0x7f6528191ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:49.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.888+0000 7f652f3f0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f652810b330 con 0x7f6528106e80 2024-06-10T00:20:49.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.889+0000 7f652d18c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528106e80 0x7f6528191ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:49.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.889+0000 7f652d18c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528106e80 0x7f6528191ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44678/0 (socket says 172.21.15.62:44678) 2024-06-10T00:20:49.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.889+0000 7f652d18c700 1 -- 172.21.15.62:0/893910289 learned_addr learned my addr 172.21.15.62:0/893910289 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:49.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.889+0000 7f652d18c700 1 -- 172.21.15.62:0/893910289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6528192530 con 0x7f6528106e80 2024-06-10T00:20:49.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.889+0000 7f652d18c700 1 --2- 172.21.15.62:0/893910289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528106e80 0x7f6528191ff0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6524009fb0 tx=0x7f6524011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:49.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.889+0000 7f651e7fc700 1 -- 172.21.15.62:0/893910289 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65240038b0 con 0x7f6528106e80 2024-06-10T00:20:49.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.889+0000 7f651e7fc700 1 -- 172.21.15.62:0/893910289 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6524011e50 con 0x7f6528106e80 2024-06-10T00:20:49.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.889+0000 7f652f3f0700 1 -- 172.21.15.62:0/893910289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6528190720 con 0x7f6528106e80 2024-06-10T00:20:49.757 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.889+0000 7f651e7fc700 1 -- 172.21.15.62:0/893910289 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6524019550 con 0x7f6528106e80 2024-06-10T00:20:49.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.889+0000 7f652f3f0700 1 -- 172.21.15.62:0/893910289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6528190c60 con 0x7f6528106e80 2024-06-10T00:20:49.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.890+0000 7f652f3f0700 1 -- 172.21.15.62:0/893910289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f652810b4b0 con 0x7f6528106e80 2024-06-10T00:20:49.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.890+0000 7f651e7fc700 1 -- 172.21.15.62:0/893910289 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f652401a430 con 0x7f6528106e80 2024-06-10T00:20:49.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.890+0000 7f651e7fc700 1 --2- 172.21.15.62:0/893910289 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7f6514039000 0x7f651403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:49.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.890+0000 7f651e7fc700 1 -- 172.21.15.62:0/893910289 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f652404d210 con 0x7f6528106e80 2024-06-10T00:20:49.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.891+0000 7f652c98b700 1 -- 172.21.15.62:0/893910289 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7f6514039000 msgr2=0x7f651403b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/594215923 2024-06-10T00:20:49.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.891+0000 7f652c98b700 1 --2- 172.21.15.62:0/893910289 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7f6514039000 0x7f651403b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:20:49.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:48.895+0000 7f651e7fc700 1 -- 172.21.15.62:0/893910289 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6524029d50 con 0x7f6528106e80 2024-06-10T00:20:49.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:49.092+0000 7f652c98b700 1 -- 172.21.15.62:0/893910289 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7f6514039000 msgr2=0x7f651403b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/594215923 2024-06-10T00:20:49.758 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:49.092+0000 7f652c98b700 1 --2- 172.21.15.62:0/893910289 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7f6514039000 0x7f651403b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T00:20:49.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:49.160+0000 7f652f3f0700 1 -- 172.21.15.62:0/893910289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6528195e10 con 0x7f6528106e80 2024-06-10T00:20:49.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:49.161+0000 7f651e7fc700 1 -- 172.21.15.62:0/893910289 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f6524023390 con 0x7f6528106e80 2024-06-10T00:20:49.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:49.163+0000 7f652f3f0700 1 -- 172.21.15.62:0/893910289 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7f6514039000 msgr2=0x7f651403b4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:20:49.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:49.163+0000 7f652f3f0700 1 --2- 172.21.15.62:0/893910289 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7f6514039000 0x7f651403b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:49.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:49.163+0000 7f652f3f0700 1 -- 172.21.15.62:0/893910289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528106e80 msgr2=0x7f6528191ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:49.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:49.163+0000 7f652f3f0700 1 --2- 172.21.15.62:0/893910289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528106e80 0x7f6528191ff0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6524009fb0 tx=0x7f6524011bf0 comp rx=0 tx=0).stop 2024-06-10T00:20:49.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:49.163+0000 7f652f3f0700 1 -- 172.21.15.62:0/893910289 shutdown_connections 2024-06-10T00:20:49.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:49.163+0000 7f652f3f0700 1 --2- 172.21.15.62:0/893910289 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7f6514039000 0x7f651403b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:49.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:49.163+0000 7f652f3f0700 1 --2- 172.21.15.62:0/893910289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6528106e80 0x7f6528191ff0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:49.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:49.163+0000 7f652f3f0700 1 -- 172.21.15.62:0/893910289 >> 172.21.15.62:0/893910289 conn(0x7f65281024b0 msgr2=0x7f65281033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:49.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:49.163+0000 7f652f3f0700 1 -- 172.21.15.62:0/893910289 shutdown_connections 2024-06-10T00:20:49.759 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:49.163+0000 7f652f3f0700 1 -- 172.21.15.62:0/893910289 wait complete. 2024-06-10T00:20:49.759 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for the mgr to restart... 2024-06-10T00:20:49.760 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr epoch 5... 2024-06-10T00:20:50.070 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:49 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/893910289' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-10T00:20:50.358 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:50.070+0000 7fbe412a6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:20:50.358 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:50.252+0000 7fbe412a6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:20:50.736 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:50.432+0000 7fbe412a6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:20:51.118 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:50.736+0000 7fbe412a6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:20:51.119 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:50.833+0000 7fbe412a6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:20:51.119 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:51 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:51.044+0000 7fbe412a6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:20:51.357 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:51 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:51.247+0000 7fbe412a6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:20:52.322 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:51 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:51.879+0000 7fbe412a6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:20:52.607 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:52 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:52.321+0000 7fbe412a6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:20:52.607 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:52 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:52.423+0000 7fbe412a6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:20:53.107 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:52 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:52.634+0000 7fbe412a6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:20:53.533 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:53 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:53.182+0000 7fbe412a6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:20:53.534 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:53 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:53.279+0000 7fbe412a6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:20:53.857 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:53 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:53.534+0000 7fbe412a6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:20:53.857 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:53 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:53.674+0000 7fbe412a6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:20:53.857 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:53 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:53.774+0000 7fbe412a6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:20:54.215 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:53 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:53.895+0000 7fbe412a6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:20:54.215 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:54 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:54.215+0000 7fbe412a6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:20:55.085 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:54 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:54.857+0000 7fbe412a6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:20:55.086 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:20:55.085+0000 7fbe412a6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:20:55.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: Active manager daemon x restarted 2024-06-10T00:20:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: Activating manager daemon x 2024-06-10T00:20:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: osdmap e2: 0 total, 0 up, 0 in 2024-06-10T00:20:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: mgrmap e6: x(active, starting, since 0.00351038s) 2024-06-10T00:20:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:20:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T00:20:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T00:20:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T00:20:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T00:20:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: Manager daemon x is now available 2024-06-10T00:20:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' 2024-06-10T00:20:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' 2024-06-10T00:20:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:20:55.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:20:55.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:55 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T00:20:56.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:56 smithi062 ceph-mon[32390]: Found migration_current of "None". Setting to last migration. 2024-06-10T00:20:56.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:56 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T00:20:56.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:56 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' 2024-06-10T00:20:56.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:56 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' 2024-06-10T00:20:56.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:56 smithi062 ceph-mon[32390]: mgrmap e7: x(active, since 1.0057s) 2024-06-10T00:20:56.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:20:56.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-10T00:20:56.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-10T00:20:56.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:20:56.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.193+0000 7fd5e7915700 1 Processor -- start 2024-06-10T00:20:56.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.194+0000 7fd5e7915700 1 -- start start 2024-06-10T00:20:56.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.194+0000 7fd5e7915700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5e0106e70 0x7fd5e0107290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:56.603 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.194+0000 7fd5e7915700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01077d0 con 0x7fd5e0106e70 2024-06-10T00:20:56.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.194+0000 7fd5e56b1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5e0106e70 0x7fd5e0107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:56.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.194+0000 7fd5e56b1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5e0106e70 0x7fd5e0107290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44682/0 (socket says 172.21.15.62:44682) 2024-06-10T00:20:56.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.194+0000 7fd5e56b1700 1 -- 172.21.15.62:0/2955756557 learned_addr learned my addr 172.21.15.62:0/2955756557 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:56.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.194+0000 7fd5e56b1700 1 -- 172.21.15.62:0/2955756557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e0108060 con 0x7fd5e0106e70 2024-06-10T00:20:56.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.194+0000 7fd5e56b1700 1 --2- 172.21.15.62:0/2955756557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5e0106e70 0x7fd5e0107290 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0098a0 tx=0x7fd5dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1384af0289ebc47 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:56.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.195+0000 7fd5d7fff700 1 -- 172.21.15.62:0/2955756557 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5dc004160 con 0x7fd5e0106e70 2024-06-10T00:20:56.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.195+0000 7fd5d7fff700 1 -- 172.21.15.62:0/2955756557 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5dc004300 con 0x7fd5e0106e70 2024-06-10T00:20:56.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.195+0000 7fd5d7fff700 1 -- 172.21.15.62:0/2955756557 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5dc011b50 con 0x7fd5e0106e70 2024-06-10T00:20:56.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.195+0000 7fd5e7915700 1 -- 172.21.15.62:0/2955756557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5e0106e70 msgr2=0x7fd5e0107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:56.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.195+0000 7fd5e7915700 1 --2- 172.21.15.62:0/2955756557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5e0106e70 0x7fd5e0107290 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0098a0 tx=0x7fd5dc009bb0 comp rx=0 tx=0).stop 2024-06-10T00:20:56.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.195+0000 7fd5e7915700 1 -- 172.21.15.62:0/2955756557 shutdown_connections 2024-06-10T00:20:56.604 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.195+0000 7fd5e7915700 1 --2- 172.21.15.62:0/2955756557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5e0106e70 0x7fd5e0107290 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:56.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.195+0000 7fd5e7915700 1 -- 172.21.15.62:0/2955756557 >> 172.21.15.62:0/2955756557 conn(0x7fd5e0102440 msgr2=0x7fd5e01048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:56.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.195+0000 7fd5e7915700 1 -- 172.21.15.62:0/2955756557 shutdown_connections 2024-06-10T00:20:56.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.196+0000 7fd5e7915700 1 -- 172.21.15.62:0/2955756557 wait complete. 2024-06-10T00:20:56.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.196+0000 7fd5e7915700 1 Processor -- start 2024-06-10T00:20:56.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.196+0000 7fd5e7915700 1 -- start start 2024-06-10T00:20:56.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.196+0000 7fd5e7915700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5e0106e70 0x7fd5e0199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:56.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.196+0000 7fd5e7915700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e007afd0 con 0x7fd5e0106e70 2024-06-10T00:20:56.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.196+0000 7fd5e56b1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5e0106e70 0x7fd5e0199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:56.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.196+0000 7fd5e56b1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5e0106e70 0x7fd5e0199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44696/0 (socket says 172.21.15.62:44696) 2024-06-10T00:20:56.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.196+0000 7fd5e56b1700 1 -- 172.21.15.62:0/2203348015 learned_addr learned my addr 172.21.15.62:0/2203348015 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:56.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.197+0000 7fd5e56b1700 1 -- 172.21.15.62:0/2203348015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e019a330 con 0x7fd5e0106e70 2024-06-10T00:20:56.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.197+0000 7fd5e56b1700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5e0106e70 0x7fd5e0199df0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc011850 tx=0x7fd5dc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:56.605 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.197+0000 7fd5d67fc700 1 -- 172.21.15.62:0/2203348015 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5dc003af0 con 0x7fd5e0106e70 2024-06-10T00:20:56.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.197+0000 7fd5e7915700 1 -- 172.21.15.62:0/2203348015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e019a5c0 con 0x7fd5e0106e70 2024-06-10T00:20:56.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.197+0000 7fd5d67fc700 1 -- 172.21.15.62:0/2203348015 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5dc020070 con 0x7fd5e0106e70 2024-06-10T00:20:56.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.197+0000 7fd5d67fc700 1 -- 172.21.15.62:0/2203348015 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5dc0195a0 con 0x7fd5e0106e70 2024-06-10T00:20:56.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.197+0000 7fd5e7915700 1 -- 172.21.15.62:0/2203348015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e019d260 con 0x7fd5e0106e70 2024-06-10T00:20:56.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.198+0000 7fd5d67fc700 1 -- 172.21.15.62:0/2203348015 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fd5dc019850 con 0x7fd5e0106e70 2024-06-10T00:20:56.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.198+0000 7fd5d67fc700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd5cc03b300 0x7fd5cc03d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:56.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.198+0000 7fd5d67fc700 1 -- 172.21.15.62:0/2203348015 --> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd5cc03ded0 con 0x7fd5cc03b300 2024-06-10T00:20:56.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.198+0000 7fd5e4eb0700 1 -- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd5cc03b300 msgr2=0x7fd5cc03d7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/594215923 2024-06-10T00:20:56.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.198+0000 7fd5d67fc700 1 -- 172.21.15.62:0/2203348015 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd5dc011e00 con 0x7fd5e0106e70 2024-06-10T00:20:56.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.198+0000 7fd5e4eb0700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd5cc03b300 0x7fd5cc03d7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:20:56.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.399+0000 7fd5e4eb0700 1 -- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd5cc03b300 msgr2=0x7fd5cc03d7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/594215923 2024-06-10T00:20:56.606 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.399+0000 7fd5e4eb0700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd5cc03b300 0x7fd5cc03d7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T00:20:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.800+0000 7fd5e4eb0700 1 -- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd5cc03b300 msgr2=0x7fd5cc03d7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/594215923 2024-06-10T00:20:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:50.800+0000 7fd5e4eb0700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd5cc03b300 0x7fd5cc03d7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-10T00:20:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:51.601+0000 7fd5e4eb0700 1 -- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd5cc03b300 msgr2=0x7fd5cc03d7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/594215923 2024-06-10T00:20:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:51.601+0000 7fd5e4eb0700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd5cc03b300 0x7fd5cc03d7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-10T00:20:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:53.202+0000 7fd5e4eb0700 1 -- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd5cc03b300 msgr2=0x7fd5cc03d7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/594215923 2024-06-10T00:20:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:53.202+0000 7fd5e4eb0700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd5cc03b300 0x7fd5cc03d7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-10T00:20:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:55.089+0000 7fd5d67fc700 1 -- 172.21.15.62:0/2203348015 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fd5dc01a430 con 0x7fd5e0106e70 2024-06-10T00:20:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:55.089+0000 7fd5d67fc700 1 -- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd5cc03b300 msgr2=0x7fd5cc03d7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:20:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:55.089+0000 7fd5d67fc700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd5cc03b300 0x7fd5cc03d7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.091+0000 7fd5d67fc700 1 -- 172.21.15.62:0/2203348015 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fd5dc04eb50 con 0x7fd5e0106e70 2024-06-10T00:20:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.091+0000 7fd5d67fc700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fd5cc03ec80 0x7fd5cc041070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.091+0000 7fd5d67fc700 1 -- 172.21.15.62:0/2203348015 --> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd5cc03ded0 con 0x7fd5cc03ec80 2024-06-10T00:20:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.091+0000 7fd5e4eb0700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fd5cc03ec80 0x7fd5cc041070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.092+0000 7fd5e4eb0700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fd5cc03ec80 0x7fd5cc041070 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0003a10 tx=0x7fd5d00092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.093+0000 7fd5d67fc700 1 -- 172.21.15.62:0/2203348015 <== mgr.14120 v2:172.21.15.62:6800/2327526412 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fd5cc03ded0 con 0x7fd5cc03ec80 2024-06-10T00:20:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.099+0000 7fd5e7915700 1 -- 172.21.15.62:0/2203348015 --> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd5e00695d0 con 0x7fd5cc03ec80 2024-06-10T00:20:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.099+0000 7fd5d67fc700 1 -- 172.21.15.62:0/2203348015 <== mgr.14120 v2:172.21.15.62:6800/2327526412 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fd5e00695d0 con 0x7fd5cc03ec80 2024-06-10T00:20:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.099+0000 7fd5e7915700 1 -- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fd5cc03ec80 msgr2=0x7fd5cc041070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.099+0000 7fd5e7915700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fd5cc03ec80 0x7fd5cc041070 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0003a10 tx=0x7fd5d00092b0 comp rx=0 tx=0).stop 2024-06-10T00:20:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.099+0000 7fd5e7915700 1 -- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5e0106e70 msgr2=0x7fd5e0199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.099+0000 7fd5e7915700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5e0106e70 0x7fd5e0199df0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc011850 tx=0x7fd5dc011c10 comp rx=0 tx=0).stop 2024-06-10T00:20:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.099+0000 7fd5e7915700 1 -- 172.21.15.62:0/2203348015 shutdown_connections 2024-06-10T00:20:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.099+0000 7fd5e7915700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fd5cc03ec80 0x7fd5cc041070 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.100+0000 7fd5e7915700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5e0106e70 0x7fd5e0199df0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.100+0000 7fd5e7915700 1 --2- 172.21.15.62:0/2203348015 >> [v2:172.21.15.62:6800/594215923,v1:172.21.15.62:6801/594215923] conn(0x7fd5cc03b300 0x7fd5cc03d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.100+0000 7fd5e7915700 1 -- 172.21.15.62:0/2203348015 >> 172.21.15.62:0/2203348015 conn(0x7fd5e0102440 msgr2=0x7fd5e0190370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:56.609 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.100+0000 7fd5e7915700 1 -- 172.21.15.62:0/2203348015 shutdown_connections 2024-06-10T00:20:56.609 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:56.100+0000 7fd5e7915700 1 -- 172.21.15.62:0/2203348015 wait complete. 2024-06-10T00:20:56.609 INFO:teuthology.orchestra.run.smithi062.stdout:mgr epoch 5 is available 2024-06-10T00:20:56.609 INFO:teuthology.orchestra.run.smithi062.stdout:Setting orchestrator backend to cephadm... 2024-06-10T00:20:57.255 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:57 smithi062 ceph-mon[32390]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-10T00:20:57.255 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:57 smithi062 ceph-mon[32390]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-10T00:20:57.256 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:57 smithi062 ceph-mon[32390]: [10/Jun/2024:00:20:56] ENGINE Bus STARTING 2024-06-10T00:20:57.256 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:57 smithi062 ceph-mon[32390]: [10/Jun/2024:00:20:56] ENGINE Serving on http://172.21.15.62:8765 2024-06-10T00:20:57.256 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:57 smithi062 ceph-mon[32390]: [10/Jun/2024:00:20:56] ENGINE Serving on https://172.21.15.62:7150 2024-06-10T00:20:57.256 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:57 smithi062 ceph-mon[32390]: [10/Jun/2024:00:20:56] ENGINE Bus STARTED 2024-06-10T00:20:57.256 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:57 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:20:57.799 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.018+0000 7f227fd8f700 1 Processor -- start 2024-06-10T00:20:57.799 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.018+0000 7f227fd8f700 1 -- start start 2024-06-10T00:20:57.799 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.018+0000 7f227fd8f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2278108a70 0x7f227810ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:57.799 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.018+0000 7f227fd8f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f227807be70 con 0x7f2278108a70 2024-06-10T00:20:57.799 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.019+0000 7f227db2b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2278108a70 0x7f227810ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:57.799 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.019+0000 7f227db2b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2278108a70 0x7f227810ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44748/0 (socket says 172.21.15.62:44748) 2024-06-10T00:20:57.800 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.019+0000 7f227db2b700 1 -- 172.21.15.62:0/2974415664 learned_addr learned my addr 172.21.15.62:0/2974415664 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:57.800 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.019+0000 7f227db2b700 1 -- 172.21.15.62:0/2974415664 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227810b3a0 con 0x7f2278108a70 2024-06-10T00:20:57.800 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.019+0000 7f227db2b700 1 --2- 172.21.15.62:0/2974415664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2278108a70 0x7f227810ae60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f22740098a0 tx=0x7f2274009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f991c557bca04c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:57.800 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.019+0000 7f227cb29700 1 -- 172.21.15.62:0/2974415664 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2274004160 con 0x7f2278108a70 2024-06-10T00:20:57.800 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.019+0000 7f227cb29700 1 -- 172.21.15.62:0/2974415664 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2274004300 con 0x7f2278108a70 2024-06-10T00:20:57.800 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.019+0000 7f227cb29700 1 -- 172.21.15.62:0/2974415664 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2274011b50 con 0x7f2278108a70 2024-06-10T00:20:57.800 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.020+0000 7f227fd8f700 1 -- 172.21.15.62:0/2974415664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2278108a70 msgr2=0x7f227810ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:57.800 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.020+0000 7f227fd8f700 1 --2- 172.21.15.62:0/2974415664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2278108a70 0x7f227810ae60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f22740098a0 tx=0x7f2274009bb0 comp rx=0 tx=0).stop 2024-06-10T00:20:57.800 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.020+0000 7f227fd8f700 1 -- 172.21.15.62:0/2974415664 shutdown_connections 2024-06-10T00:20:57.800 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.020+0000 7f227fd8f700 1 --2- 172.21.15.62:0/2974415664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2278108a70 0x7f227810ae60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:57.800 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.020+0000 7f227fd8f700 1 -- 172.21.15.62:0/2974415664 >> 172.21.15.62:0/2974415664 conn(0x7f22781024e0 msgr2=0x7f2278104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:57.800 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.020+0000 7f227fd8f700 1 -- 172.21.15.62:0/2974415664 shutdown_connections 2024-06-10T00:20:57.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.020+0000 7f227fd8f700 1 -- 172.21.15.62:0/2974415664 wait complete. 2024-06-10T00:20:57.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.021+0000 7f227fd8f700 1 Processor -- start 2024-06-10T00:20:57.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.021+0000 7f227fd8f700 1 -- start start 2024-06-10T00:20:57.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.021+0000 7f227fd8f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2278108a70 0x7f227819bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:57.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.021+0000 7f227fd8f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f227810c3c0 con 0x7f2278108a70 2024-06-10T00:20:57.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.021+0000 7f227db2b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2278108a70 0x7f227819bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:57.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.021+0000 7f227db2b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2278108a70 0x7f227819bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44750/0 (socket says 172.21.15.62:44750) 2024-06-10T00:20:57.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.021+0000 7f227db2b700 1 -- 172.21.15.62:0/1345315286 learned_addr learned my addr 172.21.15.62:0/1345315286 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:57.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.021+0000 7f227db2b700 1 -- 172.21.15.62:0/1345315286 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227819c470 con 0x7f2278108a70 2024-06-10T00:20:57.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.021+0000 7f227db2b700 1 --2- 172.21.15.62:0/1345315286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2278108a70 0x7f227819bf30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2274011b60 tx=0x7f2274011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:57.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.021+0000 7f226effd700 1 -- 172.21.15.62:0/1345315286 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2274003b20 con 0x7f2278108a70 2024-06-10T00:20:57.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.021+0000 7f226effd700 1 -- 172.21.15.62:0/1345315286 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2274003cc0 con 0x7f2278108a70 2024-06-10T00:20:57.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.021+0000 7f227fd8f700 1 -- 172.21.15.62:0/1345315286 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227819c700 con 0x7f2278108a70 2024-06-10T00:20:57.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.021+0000 7f226effd700 1 -- 172.21.15.62:0/1345315286 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2274024840 con 0x7f2278108a70 2024-06-10T00:20:57.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.022+0000 7f227fd8f700 1 -- 172.21.15.62:0/1345315286 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f227819f3f0 con 0x7f2278108a70 2024-06-10T00:20:57.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.022+0000 7f226effd700 1 -- 172.21.15.62:0/1345315286 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f2274022020 con 0x7f2278108a70 2024-06-10T00:20:57.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.023+0000 7f226effd700 1 --2- 172.21.15.62:0/1345315286 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f2264038f80 0x7f226403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:57.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.023+0000 7f226effd700 1 -- 172.21.15.62:0/1345315286 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2274011e00 con 0x7f2278108a70 2024-06-10T00:20:57.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.023+0000 7f227d32a700 1 --2- 172.21.15.62:0/1345315286 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f2264038f80 0x7f226403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:57.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.023+0000 7f227fd8f700 1 -- 172.21.15.62:0/1345315286 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2278063af0 con 0x7f2278108a70 2024-06-10T00:20:57.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.023+0000 7f227d32a700 1 --2- 172.21.15.62:0/1345315286 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f2264038f80 0x7f226403b440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2268006fd0 tx=0x7f2268006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:57.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.028+0000 7f226effd700 1 -- 172.21.15.62:0/1345315286 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2274019c70 con 0x7f2278108a70 2024-06-10T00:20:57.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.142+0000 7f226effd700 1 -- 172.21.15.62:0/1345315286 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f227402d910 con 0x7f2278108a70 2024-06-10T00:20:57.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.255+0000 7f227fd8f700 1 -- 172.21.15.62:0/1345315286 --> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f2278103760 con 0x7f2264038f80 2024-06-10T00:20:57.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.263+0000 7f226effd700 1 -- 172.21.15.62:0/1345315286 <== mgr.14120 v2:172.21.15.62:6800/2327526412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2278103760 con 0x7f2264038f80 2024-06-10T00:20:57.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.265+0000 7f227fd8f700 1 -- 172.21.15.62:0/1345315286 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f2264038f80 msgr2=0x7f226403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:57.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.265+0000 7f227fd8f700 1 --2- 172.21.15.62:0/1345315286 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f2264038f80 0x7f226403b440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2268006fd0 tx=0x7f2268006e40 comp rx=0 tx=0).stop 2024-06-10T00:20:57.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.265+0000 7f227fd8f700 1 -- 172.21.15.62:0/1345315286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2278108a70 msgr2=0x7f227819bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:57.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.265+0000 7f227fd8f700 1 --2- 172.21.15.62:0/1345315286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2278108a70 0x7f227819bf30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2274011b60 tx=0x7f2274011c40 comp rx=0 tx=0).stop 2024-06-10T00:20:57.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.266+0000 7f227fd8f700 1 -- 172.21.15.62:0/1345315286 shutdown_connections 2024-06-10T00:20:57.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.266+0000 7f227fd8f700 1 --2- 172.21.15.62:0/1345315286 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f2264038f80 0x7f226403b440 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:57.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.266+0000 7f227fd8f700 1 --2- 172.21.15.62:0/1345315286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2278108a70 0x7f227819bf30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:57.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.266+0000 7f227fd8f700 1 -- 172.21.15.62:0/1345315286 >> 172.21.15.62:0/1345315286 conn(0x7f22781024e0 msgr2=0x7f2278103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:57.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.266+0000 7f227fd8f700 1 -- 172.21.15.62:0/1345315286 shutdown_connections 2024-06-10T00:20:57.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:57.266+0000 7f227fd8f700 1 -- 172.21.15.62:0/1345315286 wait complete. 2024-06-10T00:20:58.195 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:58 smithi062 ceph-mon[32390]: mgrmap e8: x(active, since 2s) 2024-06-10T00:20:58.195 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:58 smithi062 ceph-mon[32390]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:20:58.195 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:58 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' 2024-06-10T00:20:58.195 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:58 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:20:59.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-10T00:20:59.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.218+0000 7fadb368c700 1 Processor -- start 2024-06-10T00:20:59.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.218+0000 7fadb368c700 1 -- start start 2024-06-10T00:20:59.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.218+0000 7fadb368c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac107fd0 0x7fadac1083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:59.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.218+0000 7fadb368c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1089c0 con 0x7fadac107fd0 2024-06-10T00:20:59.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.218+0000 7fadb1428700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac107fd0 0x7fadac1083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:59.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.218+0000 7fadb1428700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac107fd0 0x7fadac1083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47902/0 (socket says 172.21.15.62:47902) 2024-06-10T00:20:59.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.218+0000 7fadb1428700 1 -- 172.21.15.62:0/1493850300 learned_addr learned my addr 172.21.15.62:0/1493850300 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:59.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.218+0000 7fadb1428700 1 -- 172.21.15.62:0/1493850300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac1091d0 con 0x7fadac107fd0 2024-06-10T00:20:59.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.219+0000 7fadb1428700 1 --2- 172.21.15.62:0/1493850300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac107fd0 0x7fadac1083f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fada80098a0 tx=0x7fada8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab04bbf6db907703 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:59.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.219+0000 7fada3fff700 1 -- 172.21.15.62:0/1493850300 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada8004160 con 0x7fadac107fd0 2024-06-10T00:20:59.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.219+0000 7fada3fff700 1 -- 172.21.15.62:0/1493850300 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fada8004300 con 0x7fadac107fd0 2024-06-10T00:20:59.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.219+0000 7fada3fff700 1 -- 172.21.15.62:0/1493850300 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada8011b50 con 0x7fadac107fd0 2024-06-10T00:20:59.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.219+0000 7fadb368c700 1 -- 172.21.15.62:0/1493850300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac107fd0 msgr2=0x7fadac1083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:59.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.219+0000 7fadb368c700 1 --2- 172.21.15.62:0/1493850300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac107fd0 0x7fadac1083f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fada80098a0 tx=0x7fada8009bb0 comp rx=0 tx=0).stop 2024-06-10T00:20:59.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.219+0000 7fadb368c700 1 -- 172.21.15.62:0/1493850300 shutdown_connections 2024-06-10T00:20:59.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.219+0000 7fadb368c700 1 --2- 172.21.15.62:0/1493850300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac107fd0 0x7fadac1083f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:59.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.219+0000 7fadb368c700 1 -- 172.21.15.62:0/1493850300 >> 172.21.15.62:0/1493850300 conn(0x7fadac103550 msgr2=0x7fadac1059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:59.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.220+0000 7fadb368c700 1 -- 172.21.15.62:0/1493850300 shutdown_connections 2024-06-10T00:20:59.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.220+0000 7fadb368c700 1 -- 172.21.15.62:0/1493850300 wait complete. 2024-06-10T00:20:59.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.220+0000 7fadb368c700 1 Processor -- start 2024-06-10T00:20:59.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.220+0000 7fadb368c700 1 -- start start 2024-06-10T00:20:59.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.220+0000 7fadb368c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac107fd0 0x7fadac19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.220+0000 7fadb368c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fadac10c580 con 0x7fadac107fd0 2024-06-10T00:20:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.221+0000 7fadb1428700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac107fd0 0x7fadac19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.221+0000 7fadb1428700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac107fd0 0x7fadac19bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47918/0 (socket says 172.21.15.62:47918) 2024-06-10T00:20:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.221+0000 7fadb1428700 1 -- 172.21.15.62:0/2666609464 learned_addr learned my addr 172.21.15.62:0/2666609464 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:20:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.221+0000 7fadb1428700 1 -- 172.21.15.62:0/2666609464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac19c530 con 0x7fadac107fd0 2024-06-10T00:20:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.221+0000 7fadb1428700 1 --2- 172.21.15.62:0/2666609464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac107fd0 0x7fadac19bff0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fada8011850 tx=0x7fada8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.221+0000 7fada27fc700 1 -- 172.21.15.62:0/2666609464 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada8003af0 con 0x7fadac107fd0 2024-06-10T00:20:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.221+0000 7fada27fc700 1 -- 172.21.15.62:0/2666609464 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fada8020070 con 0x7fadac107fd0 2024-06-10T00:20:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.221+0000 7fada27fc700 1 -- 172.21.15.62:0/2666609464 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada80195a0 con 0x7fadac107fd0 2024-06-10T00:20:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.221+0000 7fadb368c700 1 -- 172.21.15.62:0/2666609464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac19c7c0 con 0x7fadac107fd0 2024-06-10T00:20:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.221+0000 7fadb368c700 1 -- 172.21.15.62:0/2666609464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac19f4b0 con 0x7fadac107fd0 2024-06-10T00:20:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.222+0000 7fada27fc700 1 -- 172.21.15.62:0/2666609464 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fada80198a0 con 0x7fadac107fd0 2024-06-10T00:20:59.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.222+0000 7fadb368c700 1 -- 172.21.15.62:0/2666609464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadac0695d0 con 0x7fadac107fd0 2024-06-10T00:20:59.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.222+0000 7fada27fc700 1 --2- 172.21.15.62:0/2666609464 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fad98039050 0x7fad9803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:20:59.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.222+0000 7fada27fc700 1 -- 172.21.15.62:0/2666609464 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fada8011e00 con 0x7fadac107fd0 2024-06-10T00:20:59.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.222+0000 7fadb0c27700 1 --2- 172.21.15.62:0/2666609464 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fad98039050 0x7fad9803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:20:59.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.223+0000 7fadb0c27700 1 --2- 172.21.15.62:0/2666609464 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fad98039050 0x7fad9803b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fad9c006fd0 tx=0x7fad9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:20:59.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.227+0000 7fada27fc700 1 -- 172.21.15.62:0/2666609464 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fada800e260 con 0x7fadac107fd0 2024-06-10T00:20:59.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.434+0000 7fadb368c700 1 -- 172.21.15.62:0/2666609464 --> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fadac104830 con 0x7fad98039050 2024-06-10T00:20:59.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.435+0000 7fada27fc700 1 -- 172.21.15.62:0/2666609464 <== mgr.14120 v2:172.21.15.62:6800/2327526412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fadac104830 con 0x7fad98039050 2024-06-10T00:20:59.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.437+0000 7fadb368c700 1 -- 172.21.15.62:0/2666609464 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fad98039050 msgr2=0x7fad9803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:59.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.437+0000 7fadb368c700 1 --2- 172.21.15.62:0/2666609464 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fad98039050 0x7fad9803b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fad9c006fd0 tx=0x7fad9c006e40 comp rx=0 tx=0).stop 2024-06-10T00:20:59.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.437+0000 7fadb368c700 1 -- 172.21.15.62:0/2666609464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac107fd0 msgr2=0x7fadac19bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:20:59.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.437+0000 7fadb368c700 1 --2- 172.21.15.62:0/2666609464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac107fd0 0x7fadac19bff0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fada8011850 tx=0x7fada8011c10 comp rx=0 tx=0).stop 2024-06-10T00:20:59.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.437+0000 7fadb368c700 1 -- 172.21.15.62:0/2666609464 shutdown_connections 2024-06-10T00:20:59.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.437+0000 7fadb368c700 1 --2- 172.21.15.62:0/2666609464 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fad98039050 0x7fad9803b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:59.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.437+0000 7fadb368c700 1 --2- 172.21.15.62:0/2666609464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fadac107fd0 0x7fadac19bff0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:20:59.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.437+0000 7fadb368c700 1 -- 172.21.15.62:0/2666609464 >> 172.21.15.62:0/2666609464 conn(0x7fadac103550 msgr2=0x7fadac104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:20:59.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.437+0000 7fadb368c700 1 -- 172.21.15.62:0/2666609464 shutdown_connections 2024-06-10T00:20:59.045 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:58.438+0000 7fadb368c700 1 -- 172.21.15.62:0/2666609464 wait complete. 2024-06-10T00:20:59.045 INFO:teuthology.orchestra.run.smithi062.stdout:Generating ssh key... 2024-06-10T00:20:59.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:20:59 smithi062 ceph-mon[32390]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:21:00.347 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-mon[32390]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:21:00.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-mon[32390]: Generating ssh key... 2024-06-10T00:21:00.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' 2024-06-10T00:21:00.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' 2024-06-10T00:21:00.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: Generating public/private rsa key pair. 2024-06-10T00:21:00.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: Your identification has been saved in /tmp/tmpkv89jjpg/key. 2024-06-10T00:21:00.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: Your public key has been saved in /tmp/tmpkv89jjpg/key.pub. 2024-06-10T00:21:00.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: The key fingerprint is: 2024-06-10T00:21:00.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: SHA256:UC7TEXuYiqxpruf1w0yVhXZIlHVxhjdrBjyrxau4b7E ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:21:00.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: The key's randomart image is: 2024-06-10T00:21:00.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: +---[RSA 3072]----+ 2024-06-10T00:21:00.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: | o**..ooo | 2024-06-10T00:21:00.349 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: | +==o.=oo | 2024-06-10T00:21:00.350 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: | +.*+.. = o | 2024-06-10T00:21:00.350 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: | . . =o. + + | 2024-06-10T00:21:00.350 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: | o ..S o + | 2024-06-10T00:21:00.350 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: | o . o . | 2024-06-10T00:21:00.350 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: | + .+ . + | 2024-06-10T00:21:00.350 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: | o.. .+ . E | 2024-06-10T00:21:00.350 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: |.+o ...+. | 2024-06-10T00:21:00.350 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: +----[SHA256]-----+ 2024-06-10T00:21:00.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.508+0000 7fedcd632700 1 Processor -- start 2024-06-10T00:21:00.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.508+0000 7fedcd632700 1 -- start start 2024-06-10T00:21:00.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.508+0000 7fedcd632700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedc8108ad0 0x7fedc810aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:00.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.508+0000 7fedcd632700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fedc807be10 con 0x7fedc8108ad0 2024-06-10T00:21:00.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.508+0000 7fedc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedc8108ad0 0x7fedc810aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:00.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.508+0000 7fedc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedc8108ad0 0x7fedc810aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47920/0 (socket says 172.21.15.62:47920) 2024-06-10T00:21:00.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.508+0000 7fedc6ffd700 1 -- 172.21.15.62:0/3310671373 learned_addr learned my addr 172.21.15.62:0/3310671373 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:00.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.509+0000 7fedc6ffd700 1 -- 172.21.15.62:0/3310671373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc810b400 con 0x7fedc8108ad0 2024-06-10T00:21:00.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.509+0000 7fedc6ffd700 1 --2- 172.21.15.62:0/3310671373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedc8108ad0 0x7fedc810aec0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fedb80098a0 tx=0x7fedb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c07d41f5beba0e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:00.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.509+0000 7fedc5ffb700 1 -- 172.21.15.62:0/3310671373 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedb8004160 con 0x7fedc8108ad0 2024-06-10T00:21:00.634 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.509+0000 7fedc5ffb700 1 -- 172.21.15.62:0/3310671373 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fedb8004300 con 0x7fedc8108ad0 2024-06-10T00:21:00.634 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.509+0000 7fedc5ffb700 1 -- 172.21.15.62:0/3310671373 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedb8011b50 con 0x7fedc8108ad0 2024-06-10T00:21:00.634 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.509+0000 7fedcd632700 1 -- 172.21.15.62:0/3310671373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedc8108ad0 msgr2=0x7fedc810aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:00.634 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.509+0000 7fedcd632700 1 --2- 172.21.15.62:0/3310671373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedc8108ad0 0x7fedc810aec0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fedb80098a0 tx=0x7fedb8009bb0 comp rx=0 tx=0).stop 2024-06-10T00:21:00.634 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.510+0000 7fedcd632700 1 -- 172.21.15.62:0/3310671373 shutdown_connections 2024-06-10T00:21:00.634 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.510+0000 7fedcd632700 1 --2- 172.21.15.62:0/3310671373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedc8108ad0 0x7fedc810aec0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:00.634 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.510+0000 7fedcd632700 1 -- 172.21.15.62:0/3310671373 >> 172.21.15.62:0/3310671373 conn(0x7fedc81024e0 msgr2=0x7fedc8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:00.634 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.510+0000 7fedcd632700 1 -- 172.21.15.62:0/3310671373 shutdown_connections 2024-06-10T00:21:00.634 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.510+0000 7fedcd632700 1 -- 172.21.15.62:0/3310671373 wait complete. 2024-06-10T00:21:00.634 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.510+0000 7fedcd632700 1 Processor -- start 2024-06-10T00:21:00.635 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.511+0000 7fedcd632700 1 -- start start 2024-06-10T00:21:00.635 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.511+0000 7fedcd632700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedc8108ad0 0x7fedc8108350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:00.635 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.511+0000 7fedcd632700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fedc810c3c0 con 0x7fedc8108ad0 2024-06-10T00:21:00.635 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.511+0000 7fedc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedc8108ad0 0x7fedc8108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:00.635 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.511+0000 7fedc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedc8108ad0 0x7fedc8108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47932/0 (socket says 172.21.15.62:47932) 2024-06-10T00:21:00.635 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.511+0000 7fedc6ffd700 1 -- 172.21.15.62:0/308074138 learned_addr learned my addr 172.21.15.62:0/308074138 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:00.635 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.511+0000 7fedc6ffd700 1 -- 172.21.15.62:0/308074138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc8108890 con 0x7fedc8108ad0 2024-06-10T00:21:00.635 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.511+0000 7fedc6ffd700 1 --2- 172.21.15.62:0/308074138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedc8108ad0 0x7fedc8108350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fedb8011670 tx=0x7fedb8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:00.635 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.512+0000 7fedaffff700 1 -- 172.21.15.62:0/308074138 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedb80039c0 con 0x7fedc8108ad0 2024-06-10T00:21:00.635 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.512+0000 7fedaffff700 1 -- 172.21.15.62:0/308074138 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fedb8011e50 con 0x7fedc8108ad0 2024-06-10T00:21:00.635 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.512+0000 7fedcd632700 1 -- 172.21.15.62:0/308074138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc8106a80 con 0x7fedc8108ad0 2024-06-10T00:21:00.636 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.512+0000 7fedaffff700 1 -- 172.21.15.62:0/308074138 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedb801a5c0 con 0x7fedc8108ad0 2024-06-10T00:21:00.636 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.512+0000 7fedcd632700 1 -- 172.21.15.62:0/308074138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc8106fc0 con 0x7fedc8108ad0 2024-06-10T00:21:00.636 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.512+0000 7fedaffff700 1 -- 172.21.15.62:0/308074138 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fedb801a870 con 0x7fedc8108ad0 2024-06-10T00:21:00.636 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.513+0000 7fedcd632700 1 -- 172.21.15.62:0/308074138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc8063af0 con 0x7fedc8108ad0 2024-06-10T00:21:00.636 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.513+0000 7fedaffff700 1 --2- 172.21.15.62:0/308074138 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fedb0039050 0x7fedb003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:00.636 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.513+0000 7fedaffff700 1 -- 172.21.15.62:0/308074138 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fedb804d520 con 0x7fedc8108ad0 2024-06-10T00:21:00.636 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.513+0000 7fedc67fc700 1 --2- 172.21.15.62:0/308074138 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fedb0039050 0x7fedb003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:00.636 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.513+0000 7fedc67fc700 1 --2- 172.21.15.62:0/308074138 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fedb0039050 0x7fedb003b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fedbc006fd0 tx=0x7fedbc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:00.636 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.518+0000 7fedaffff700 1 -- 172.21.15.62:0/308074138 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fedb801ac60 con 0x7fedc8108ad0 2024-06-10T00:21:00.636 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:20:59.728+0000 7fedcd632700 1 -- 172.21.15.62:0/308074138 --> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fedc8103780 con 0x7fedb0039050 2024-06-10T00:21:00.636 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:00.037+0000 7fedaffff700 1 -- 172.21.15.62:0/308074138 <== mgr.14120 v2:172.21.15.62:6800/2327526412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fedc8103780 con 0x7fedb0039050 2024-06-10T00:21:00.636 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:00.039+0000 7fedcd632700 1 -- 172.21.15.62:0/308074138 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fedb0039050 msgr2=0x7fedb003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:00.636 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:00.040+0000 7fedcd632700 1 --2- 172.21.15.62:0/308074138 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fedb0039050 0x7fedb003b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fedbc006fd0 tx=0x7fedbc006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:00.637 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:00.040+0000 7fedcd632700 1 -- 172.21.15.62:0/308074138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedc8108ad0 msgr2=0x7fedc8108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:00.637 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:00.040+0000 7fedcd632700 1 --2- 172.21.15.62:0/308074138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedc8108ad0 0x7fedc8108350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fedb8011670 tx=0x7fedb8011b60 comp rx=0 tx=0).stop 2024-06-10T00:21:00.637 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:00.040+0000 7fedcd632700 1 -- 172.21.15.62:0/308074138 shutdown_connections 2024-06-10T00:21:00.637 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:00.040+0000 7fedcd632700 1 --2- 172.21.15.62:0/308074138 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fedb0039050 0x7fedb003b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:00.637 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:00.040+0000 7fedcd632700 1 --2- 172.21.15.62:0/308074138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedc8108ad0 0x7fedc8108350 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:00.637 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:00.040+0000 7fedcd632700 1 -- 172.21.15.62:0/308074138 >> 172.21.15.62:0/308074138 conn(0x7fedc81024e0 msgr2=0x7fedc8103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:00.637 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:00.040+0000 7fedcd632700 1 -- 172.21.15.62:0/308074138 shutdown_connections 2024-06-10T00:21:00.637 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:00.040+0000 7fedcd632700 1 -- 172.21.15.62:0/308074138 wait complete. 2024-06-10T00:21:01.789 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf18ETenDQ7sSymSPLNFGCqg6RtRIuHL22LKSheOdWrE+5w1t9VlPAlXDN63m5fdulnjPdiWmDybP/nYQfDlYAc49Y3jBCStc5P5wr/aiNMhZTHhJAntDcFX/xq5HG1l8DA2z2S5ofQ/vq2jrP6n1gMHIXGvGd8ROmw5wiwvoJv/Qknvl6AT7xsbJhgIGC7c63k8LP4uaDnoZ1IWASZdJkdA0QcSjrgs3BGUKpagx2QPA/3fXev/X6oZAhmoO6dX+iCdXVPtEbSoS1F0VVIVUBGppaOzBfEYmvs5uxMAtLYHMqxy1hHkQdIX3CQ0dp703uXUkFprva9x1PAfmc5ZYQwd0ApalduqYuj/tH7bCf7w0w9NJkotMS0Huucqo/tfyQI+GIo1PripKBw/JRZ9U0Bqv7kAvnBz4fE4BFH1pT2WsuHNXSufZPJGegI4UoKYEsqXql3yYIXOwCZFzpyzOU9llMlavOzzRLuYadBBpZxgP/ChCOVpteYG5mnTEp+eM= ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:21:01.789 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.046+0000 7f83a0ac3700 1 Processor -- start 2024-06-10T00:21:01.789 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.046+0000 7f83a0ac3700 1 -- start start 2024-06-10T00:21:01.790 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.046+0000 7f83a0ac3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839c107fb0 0x7f839c1083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:01.790 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.046+0000 7f83a0ac3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f839c1089a0 con 0x7f839c107fb0 2024-06-10T00:21:01.790 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.047+0000 7f839a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839c107fb0 0x7f839c1083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:01.790 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.047+0000 7f839a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839c107fb0 0x7f839c1083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47938/0 (socket says 172.21.15.62:47938) 2024-06-10T00:21:01.790 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.047+0000 7f839a59c700 1 -- 172.21.15.62:0/630077831 learned_addr learned my addr 172.21.15.62:0/630077831 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:01.790 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.047+0000 7f839a59c700 1 -- 172.21.15.62:0/630077831 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c1091b0 con 0x7f839c107fb0 2024-06-10T00:21:01.790 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.047+0000 7f839a59c700 1 --2- 172.21.15.62:0/630077831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839c107fb0 0x7f839c1083d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f838c009e80 tx=0x7f838c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2c092dcd6da042 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:01.791 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.047+0000 7f8399d9b700 1 -- 172.21.15.62:0/630077831 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f838c017070 con 0x7f839c107fb0 2024-06-10T00:21:01.791 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.047+0000 7f8399d9b700 1 -- 172.21.15.62:0/630077831 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f838c00b8c0 con 0x7f839c107fb0 2024-06-10T00:21:01.791 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.047+0000 7f8399d9b700 1 -- 172.21.15.62:0/630077831 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f838c0045c0 con 0x7f839c107fb0 2024-06-10T00:21:01.791 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.048+0000 7f83a0ac3700 1 -- 172.21.15.62:0/630077831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839c107fb0 msgr2=0x7f839c1083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:01.791 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.048+0000 7f83a0ac3700 1 --2- 172.21.15.62:0/630077831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839c107fb0 0x7f839c1083d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f838c009e80 tx=0x7f838c00b330 comp rx=0 tx=0).stop 2024-06-10T00:21:01.791 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.048+0000 7f83a0ac3700 1 -- 172.21.15.62:0/630077831 shutdown_connections 2024-06-10T00:21:01.791 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.048+0000 7f83a0ac3700 1 --2- 172.21.15.62:0/630077831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839c107fb0 0x7f839c1083d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:01.792 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.048+0000 7f83a0ac3700 1 -- 172.21.15.62:0/630077831 >> 172.21.15.62:0/630077831 conn(0x7f839c1035b0 msgr2=0x7f839c1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:01.792 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.048+0000 7f83a0ac3700 1 -- 172.21.15.62:0/630077831 shutdown_connections 2024-06-10T00:21:01.792 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.048+0000 7f83a0ac3700 1 -- 172.21.15.62:0/630077831 wait complete. 2024-06-10T00:21:01.792 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.049+0000 7f83a0ac3700 1 Processor -- start 2024-06-10T00:21:01.792 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.049+0000 7f83a0ac3700 1 -- start start 2024-06-10T00:21:01.792 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.049+0000 7f83a0ac3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839c107fb0 0x7f839c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:01.792 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.049+0000 7f83a0ac3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f839c10c560 con 0x7f839c107fb0 2024-06-10T00:21:01.792 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.049+0000 7f839a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839c107fb0 0x7f839c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:01.792 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.049+0000 7f839a59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839c107fb0 0x7f839c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47942/0 (socket says 172.21.15.62:47942) 2024-06-10T00:21:01.792 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.049+0000 7f839a59c700 1 -- 172.21.15.62:0/1600084425 learned_addr learned my addr 172.21.15.62:0/1600084425 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:01.793 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.050+0000 7f839a59c700 1 -- 172.21.15.62:0/1600084425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c07e0b0 con 0x7f839c107fb0 2024-06-10T00:21:01.793 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.050+0000 7f839a59c700 1 --2- 172.21.15.62:0/1600084425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839c107fb0 0x7f839c07db70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f838c0072a0 tx=0x7f838c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:01.793 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.050+0000 7f8393fff700 1 -- 172.21.15.62:0/1600084425 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f838c020020 con 0x7f839c107fb0 2024-06-10T00:21:01.793 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.050+0000 7f8393fff700 1 -- 172.21.15.62:0/1600084425 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f838c003b50 con 0x7f839c107fb0 2024-06-10T00:21:01.793 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.050+0000 7f83a0ac3700 1 -- 172.21.15.62:0/1600084425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c07c2a0 con 0x7f839c107fb0 2024-06-10T00:21:01.793 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.050+0000 7f8393fff700 1 -- 172.21.15.62:0/1600084425 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f838c017040 con 0x7f839c107fb0 2024-06-10T00:21:01.793 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.050+0000 7f83a0ac3700 1 -- 172.21.15.62:0/1600084425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c07c7e0 con 0x7f839c107fb0 2024-06-10T00:21:01.793 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.051+0000 7f8393fff700 1 -- 172.21.15.62:0/1600084425 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f838c023020 con 0x7f839c107fb0 2024-06-10T00:21:01.793 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.051+0000 7f83a0ac3700 1 -- 172.21.15.62:0/1600084425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c063af0 con 0x7f839c107fb0 2024-06-10T00:21:01.793 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.051+0000 7f8393fff700 1 --2- 172.21.15.62:0/1600084425 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f8388039050 0x7f838803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:01.794 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.051+0000 7f8393fff700 1 -- 172.21.15.62:0/1600084425 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f838c00f050 con 0x7f839c107fb0 2024-06-10T00:21:01.794 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.052+0000 7f8391d9b700 1 --2- 172.21.15.62:0/1600084425 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f8388039050 0x7f838803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:01.794 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.052+0000 7f8391d9b700 1 --2- 172.21.15.62:0/1600084425 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f8388039050 0x7f838803b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8384006fd0 tx=0x7f8384006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:01.794 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.056+0000 7f8393fff700 1 -- 172.21.15.62:0/1600084425 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f838c02a020 con 0x7f839c107fb0 2024-06-10T00:21:01.794 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.266+0000 7f83a0ac3700 1 -- 172.21.15.62:0/1600084425 --> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f839c07d040 con 0x7f8388039050 2024-06-10T00:21:01.794 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.266+0000 7f8393fff700 1 -- 172.21.15.62:0/1600084425 <== mgr.14120 v2:172.21.15.62:6800/2327526412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f839c07d040 con 0x7f8388039050 2024-06-10T00:21:01.794 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.269+0000 7f83a0ac3700 1 -- 172.21.15.62:0/1600084425 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f8388039050 msgr2=0x7f838803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:01.794 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.269+0000 7f83a0ac3700 1 --2- 172.21.15.62:0/1600084425 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f8388039050 0x7f838803b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8384006fd0 tx=0x7f8384006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:01.794 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.269+0000 7f83a0ac3700 1 -- 172.21.15.62:0/1600084425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839c107fb0 msgr2=0x7f839c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:01.794 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.269+0000 7f83a0ac3700 1 --2- 172.21.15.62:0/1600084425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839c107fb0 0x7f839c07db70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f838c0072a0 tx=0x7f838c00bd50 comp rx=0 tx=0).stop 2024-06-10T00:21:01.794 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.269+0000 7f83a0ac3700 1 -- 172.21.15.62:0/1600084425 shutdown_connections 2024-06-10T00:21:01.794 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.269+0000 7f83a0ac3700 1 --2- 172.21.15.62:0/1600084425 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f8388039050 0x7f838803b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:01.795 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.269+0000 7f83a0ac3700 1 --2- 172.21.15.62:0/1600084425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f839c107fb0 0x7f839c07db70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:01.795 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.269+0000 7f83a0ac3700 1 -- 172.21.15.62:0/1600084425 >> 172.21.15.62:0/1600084425 conn(0x7f839c1035b0 msgr2=0x7f839c104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:01.795 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.269+0000 7f83a0ac3700 1 -- 172.21.15.62:0/1600084425 shutdown_connections 2024-06-10T00:21:01.795 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:01.269+0000 7f83a0ac3700 1 -- 172.21.15.62:0/1600084425 wait complete. 2024-06-10T00:21:01.795 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-10T00:21:01.796 INFO:teuthology.orchestra.run.smithi062.stdout:Adding key to root@localhost authorized_keys... 2024-06-10T00:21:01.796 INFO:teuthology.orchestra.run.smithi062.stdout:Adding host smithi062... 2024-06-10T00:21:02.073 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:01 smithi062 ceph-mon[32390]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:21:03.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:02 smithi062 ceph-mon[32390]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi062", "addr": "172.21.15.62", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:21:04.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:03 smithi062 ceph-mon[32390]: Deploying cephadm binary to smithi062 2024-06-10T00:21:07.470 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Added host 'smithi062' with addr '172.21.15.62' 2024-06-10T00:21:07.471 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.290+0000 7f085a22e700 1 Processor -- start 2024-06-10T00:21:07.471 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.290+0000 7f085a22e700 1 -- start start 2024-06-10T00:21:07.471 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.290+0000 7f085a22e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0854105d40 0x7f0854106160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:07.471 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.290+0000 7f085a22e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f08541066a0 con 0x7f0854105d40 2024-06-10T00:21:07.471 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.290+0000 7f08537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0854105d40 0x7f0854106160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:07.471 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.290+0000 7f08537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0854105d40 0x7f0854106160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47958/0 (socket says 172.21.15.62:47958) 2024-06-10T00:21:07.471 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.290+0000 7f08537fe700 1 -- 172.21.15.62:0/2718664496 learned_addr learned my addr 172.21.15.62:0/2718664496 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:07.471 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.291+0000 7f08537fe700 1 -- 172.21.15.62:0/2718664496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0854106820 con 0x7f0854105d40 2024-06-10T00:21:07.471 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.291+0000 7f08537fe700 1 --2- 172.21.15.62:0/2718664496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0854105d40 0x7f0854106160 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f08440098a0 tx=0x7f0844009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b550f187d501307d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:07.471 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.291+0000 7f08527fc700 1 -- 172.21.15.62:0/2718664496 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0844004160 con 0x7f0854105d40 2024-06-10T00:21:07.471 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.291+0000 7f08527fc700 1 -- 172.21.15.62:0/2718664496 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0844004300 con 0x7f0854105d40 2024-06-10T00:21:07.472 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.291+0000 7f08527fc700 1 -- 172.21.15.62:0/2718664496 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0844011b50 con 0x7f0854105d40 2024-06-10T00:21:07.472 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.292+0000 7f085a22e700 1 -- 172.21.15.62:0/2718664496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0854105d40 msgr2=0x7f0854106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:07.472 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.292+0000 7f085a22e700 1 --2- 172.21.15.62:0/2718664496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0854105d40 0x7f0854106160 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f08440098a0 tx=0x7f0844009bb0 comp rx=0 tx=0).stop 2024-06-10T00:21:07.472 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.292+0000 7f085a22e700 1 -- 172.21.15.62:0/2718664496 shutdown_connections 2024-06-10T00:21:07.472 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.292+0000 7f085a22e700 1 --2- 172.21.15.62:0/2718664496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0854105d40 0x7f0854106160 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:07.472 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.292+0000 7f085a22e700 1 -- 172.21.15.62:0/2718664496 >> 172.21.15.62:0/2718664496 conn(0x7f0854101860 msgr2=0x7f0854103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:07.472 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.292+0000 7f085a22e700 1 -- 172.21.15.62:0/2718664496 shutdown_connections 2024-06-10T00:21:07.472 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.292+0000 7f085a22e700 1 -- 172.21.15.62:0/2718664496 wait complete. 2024-06-10T00:21:07.472 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.292+0000 7f085a22e700 1 Processor -- start 2024-06-10T00:21:07.472 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.293+0000 7f085a22e700 1 -- start start 2024-06-10T00:21:07.472 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.293+0000 7f085a22e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0854105d40 0x7f0854108f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:07.472 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.293+0000 7f085a22e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f085410c360 con 0x7f0854105d40 2024-06-10T00:21:07.472 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.293+0000 7f08537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0854105d40 0x7f0854108f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:07.473 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.293+0000 7f08537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0854105d40 0x7f0854108f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47960/0 (socket says 172.21.15.62:47960) 2024-06-10T00:21:07.473 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.293+0000 7f08537fe700 1 -- 172.21.15.62:0/2519989726 learned_addr learned my addr 172.21.15.62:0/2519989726 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:07.473 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.293+0000 7f08537fe700 1 -- 172.21.15.62:0/2519989726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0854109450 con 0x7f0854105d40 2024-06-10T00:21:07.473 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.293+0000 7f08537fe700 1 --2- 172.21.15.62:0/2519989726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0854105d40 0x7f0854108f10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0844011670 tx=0x7f0844011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:07.473 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.294+0000 7f0850ff9700 1 -- 172.21.15.62:0/2519989726 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08440039c0 con 0x7f0854105d40 2024-06-10T00:21:07.473 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.294+0000 7f0850ff9700 1 -- 172.21.15.62:0/2519989726 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0844011e50 con 0x7f0854105d40 2024-06-10T00:21:07.473 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.294+0000 7f085a22e700 1 -- 172.21.15.62:0/2519989726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0854107640 con 0x7f0854105d40 2024-06-10T00:21:07.473 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.294+0000 7f0850ff9700 1 -- 172.21.15.62:0/2519989726 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f084401a5c0 con 0x7f0854105d40 2024-06-10T00:21:07.473 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.294+0000 7f085a22e700 1 -- 172.21.15.62:0/2519989726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0854107b80 con 0x7f0854105d40 2024-06-10T00:21:07.474 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.295+0000 7f0850ff9700 1 -- 172.21.15.62:0/2519989726 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f084401a870 con 0x7f0854105d40 2024-06-10T00:21:07.474 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.295+0000 7f085a22e700 1 -- 172.21.15.62:0/2519989726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0854063af0 con 0x7f0854105d40 2024-06-10T00:21:07.474 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.295+0000 7f0850ff9700 1 --2- 172.21.15.62:0/2519989726 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f083c039050 0x7f083c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:07.474 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.295+0000 7f0850ff9700 1 -- 172.21.15.62:0/2519989726 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f084404d4a0 con 0x7f0854105d40 2024-06-10T00:21:07.474 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.295+0000 7f0852ffd700 1 --2- 172.21.15.62:0/2519989726 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f083c039050 0x7f083c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:07.474 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.296+0000 7f0852ffd700 1 --2- 172.21.15.62:0/2519989726 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f083c039050 0x7f083c03b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0848006fd0 tx=0x7f0848006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:07.474 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.300+0000 7f0850ff9700 1 -- 172.21.15.62:0/2519989726 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0844019c70 con 0x7f0854105d40 2024-06-10T00:21:07.474 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:02.513+0000 7f085a22e700 1 -- 172.21.15.62:0/2519989726 --> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi062", "addr": "172.21.15.62", "target": ["mon-mgr", ""]}) -- 0x7f0854102ad0 con 0x7f083c039050 2024-06-10T00:21:07.474 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:06.883+0000 7f0850ff9700 1 -- 172.21.15.62:0/2519989726 <== mgr.14120 v2:172.21.15.62:6800/2327526412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f0854102ad0 con 0x7f083c039050 2024-06-10T00:21:07.474 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:06.886+0000 7f085a22e700 1 -- 172.21.15.62:0/2519989726 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f083c039050 msgr2=0x7f083c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:07.474 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:06.886+0000 7f085a22e700 1 --2- 172.21.15.62:0/2519989726 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f083c039050 0x7f083c03b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0848006fd0 tx=0x7f0848006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:07.474 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:06.886+0000 7f085a22e700 1 -- 172.21.15.62:0/2519989726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0854105d40 msgr2=0x7f0854108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:07.475 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:06.886+0000 7f085a22e700 1 --2- 172.21.15.62:0/2519989726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0854105d40 0x7f0854108f10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0844011670 tx=0x7f0844011b60 comp rx=0 tx=0).stop 2024-06-10T00:21:07.475 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:06.886+0000 7f085a22e700 1 -- 172.21.15.62:0/2519989726 shutdown_connections 2024-06-10T00:21:07.475 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:06.886+0000 7f085a22e700 1 --2- 172.21.15.62:0/2519989726 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f083c039050 0x7f083c03b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:07.475 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:06.886+0000 7f085a22e700 1 --2- 172.21.15.62:0/2519989726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0854105d40 0x7f0854108f10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:07.475 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:06.886+0000 7f085a22e700 1 -- 172.21.15.62:0/2519989726 >> 172.21.15.62:0/2519989726 conn(0x7f0854101860 msgr2=0x7f08541023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:07.475 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:06.886+0000 7f085a22e700 1 -- 172.21.15.62:0/2519989726 shutdown_connections 2024-06-10T00:21:07.475 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:06.886+0000 7f085a22e700 1 -- 172.21.15.62:0/2519989726 wait complete. 2024-06-10T00:21:07.475 INFO:teuthology.orchestra.run.smithi062.stdout:Deploying unmanaged mon service... 2024-06-10T00:21:08.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:07 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' 2024-06-10T00:21:08.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:07 smithi062 ceph-mon[32390]: Added host smithi062 2024-06-10T00:21:08.087 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:07 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:21:08.865 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-10T00:21:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.943+0000 7f26603e6700 1 Processor -- start 2024-06-10T00:21:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.943+0000 7f26603e6700 1 -- start start 2024-06-10T00:21:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.943+0000 7f26603e6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2658107fb0 0x7f26581083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.943+0000 7f26603e6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f26581089a0 con 0x7f2658107fb0 2024-06-10T00:21:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.943+0000 7f265e182700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2658107fb0 0x7f26581083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.943+0000 7f265e182700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2658107fb0 0x7f26581083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42604/0 (socket says 172.21.15.62:42604) 2024-06-10T00:21:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.943+0000 7f265e182700 1 -- 172.21.15.62:0/3533818313 learned_addr learned my addr 172.21.15.62:0/3533818313 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.943+0000 7f265e182700 1 -- 172.21.15.62:0/3533818313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26581091b0 con 0x7f2658107fb0 2024-06-10T00:21:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.944+0000 7f265e182700 1 --2- 172.21.15.62:0/3533818313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2658107fb0 0x7f26581083d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f26540098a0 tx=0x7f2654009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c025c145dcd3c42c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:08.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.944+0000 7f265d180700 1 -- 172.21.15.62:0/3533818313 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2654004160 con 0x7f2658107fb0 2024-06-10T00:21:08.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.944+0000 7f265d180700 1 -- 172.21.15.62:0/3533818313 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2654004300 con 0x7f2658107fb0 2024-06-10T00:21:08.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.944+0000 7f265d180700 1 -- 172.21.15.62:0/3533818313 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2654011b50 con 0x7f2658107fb0 2024-06-10T00:21:08.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.944+0000 7f26603e6700 1 -- 172.21.15.62:0/3533818313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2658107fb0 msgr2=0x7f26581083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:08.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.944+0000 7f26603e6700 1 --2- 172.21.15.62:0/3533818313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2658107fb0 0x7f26581083d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f26540098a0 tx=0x7f2654009bb0 comp rx=0 tx=0).stop 2024-06-10T00:21:08.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.945+0000 7f26603e6700 1 -- 172.21.15.62:0/3533818313 shutdown_connections 2024-06-10T00:21:08.869 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.945+0000 7f26603e6700 1 --2- 172.21.15.62:0/3533818313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2658107fb0 0x7f26581083d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:08.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.945+0000 7f26603e6700 1 -- 172.21.15.62:0/3533818313 >> 172.21.15.62:0/3533818313 conn(0x7f26581035b0 msgr2=0x7f26581059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:08.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.945+0000 7f26603e6700 1 -- 172.21.15.62:0/3533818313 shutdown_connections 2024-06-10T00:21:08.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.945+0000 7f26603e6700 1 -- 172.21.15.62:0/3533818313 wait complete. 2024-06-10T00:21:08.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.945+0000 7f26603e6700 1 Processor -- start 2024-06-10T00:21:08.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.945+0000 7f26603e6700 1 -- start start 2024-06-10T00:21:08.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.945+0000 7f26603e6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2658107fb0 0x7f265819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:08.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.945+0000 7f26603e6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f265810c560 con 0x7f2658107fb0 2024-06-10T00:21:08.870 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.946+0000 7f265e182700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2658107fb0 0x7f265819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:08.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.946+0000 7f265e182700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2658107fb0 0x7f265819bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42618/0 (socket says 172.21.15.62:42618) 2024-06-10T00:21:08.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.946+0000 7f265e182700 1 -- 172.21.15.62:0/1897062620 learned_addr learned my addr 172.21.15.62:0/1897062620 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:08.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.946+0000 7f265e182700 1 -- 172.21.15.62:0/1897062620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265819c4c0 con 0x7f2658107fb0 2024-06-10T00:21:08.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.946+0000 7f265e182700 1 --2- 172.21.15.62:0/1897062620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2658107fb0 0x7f265819bf80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2654000c00 tx=0x7f2654011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:08.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.946+0000 7f264f7fe700 1 -- 172.21.15.62:0/1897062620 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2654003b20 con 0x7f2658107fb0 2024-06-10T00:21:08.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.946+0000 7f264f7fe700 1 -- 172.21.15.62:0/1897062620 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2654003cc0 con 0x7f2658107fb0 2024-06-10T00:21:08.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.946+0000 7f26603e6700 1 -- 172.21.15.62:0/1897062620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265819c750 con 0x7f2658107fb0 2024-06-10T00:21:08.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.946+0000 7f264f7fe700 1 -- 172.21.15.62:0/1897062620 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2654024910 con 0x7f2658107fb0 2024-06-10T00:21:08.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.946+0000 7f26603e6700 1 -- 172.21.15.62:0/1897062620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265819f440 con 0x7f2658107fb0 2024-06-10T00:21:08.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.947+0000 7f264f7fe700 1 -- 172.21.15.62:0/1897062620 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2654022020 con 0x7f2658107fb0 2024-06-10T00:21:08.871 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.947+0000 7f26603e6700 1 -- 172.21.15.62:0/1897062620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2658063af0 con 0x7f2658107fb0 2024-06-10T00:21:08.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.947+0000 7f264f7fe700 1 --2- 172.21.15.62:0/1897062620 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f26440390a0 0x7f264403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:08.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.947+0000 7f264f7fe700 1 -- 172.21.15.62:0/1897062620 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2654011e00 con 0x7f2658107fb0 2024-06-10T00:21:08.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.947+0000 7f265d981700 1 --2- 172.21.15.62:0/1897062620 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f26440390a0 0x7f264403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:08.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.948+0000 7f265d981700 1 --2- 172.21.15.62:0/1897062620 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f26440390a0 0x7f264403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2648006fd0 tx=0x7f2648006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:08.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:07.952+0000 7f264f7fe700 1 -- 172.21.15.62:0/1897062620 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f265401acd0 con 0x7f2658107fb0 2024-06-10T00:21:08.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:08.181+0000 7f26603e6700 1 -- 172.21.15.62:0/1897062620 --> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f26581048a0 con 0x7f26440390a0 2024-06-10T00:21:08.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:08.186+0000 7f264f7fe700 1 -- 172.21.15.62:0/1897062620 <== mgr.14120 v2:172.21.15.62:6800/2327526412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f26581048a0 con 0x7f26440390a0 2024-06-10T00:21:08.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:08.188+0000 7f26603e6700 1 -- 172.21.15.62:0/1897062620 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f26440390a0 msgr2=0x7f264403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:08.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:08.188+0000 7f26603e6700 1 --2- 172.21.15.62:0/1897062620 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f26440390a0 0x7f264403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2648006fd0 tx=0x7f2648006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:08.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:08.188+0000 7f26603e6700 1 -- 172.21.15.62:0/1897062620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2658107fb0 msgr2=0x7f265819bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:08.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:08.188+0000 7f26603e6700 1 --2- 172.21.15.62:0/1897062620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2658107fb0 0x7f265819bf80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2654000c00 tx=0x7f2654011c10 comp rx=0 tx=0).stop 2024-06-10T00:21:08.872 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:08.188+0000 7f26603e6700 1 -- 172.21.15.62:0/1897062620 shutdown_connections 2024-06-10T00:21:08.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:08.188+0000 7f26603e6700 1 --2- 172.21.15.62:0/1897062620 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f26440390a0 0x7f264403b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:08.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:08.188+0000 7f26603e6700 1 --2- 172.21.15.62:0/1897062620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2658107fb0 0x7f265819bf80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:08.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:08.188+0000 7f26603e6700 1 -- 172.21.15.62:0/1897062620 >> 172.21.15.62:0/1897062620 conn(0x7f26581035b0 msgr2=0x7f2658104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:08.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:08.188+0000 7f26603e6700 1 -- 172.21.15.62:0/1897062620 shutdown_connections 2024-06-10T00:21:08.873 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:08.188+0000 7f26603e6700 1 -- 172.21.15.62:0/1897062620 wait complete. 2024-06-10T00:21:08.873 INFO:teuthology.orchestra.run.smithi062.stdout:Deploying unmanaged mgr service... 2024-06-10T00:21:09.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:09 smithi062 ceph-mon[32390]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:21:09.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:09 smithi062 ceph-mon[32390]: Saving service mon spec with placement count:5 2024-06-10T00:21:09.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:09 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' 2024-06-10T00:21:10.611 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-10T00:21:10.611 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.696+0000 7fa9c5d28700 1 Processor -- start 2024-06-10T00:21:10.611 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.696+0000 7fa9c5d28700 1 -- start start 2024-06-10T00:21:10.611 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.696+0000 7fa9c5d28700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0107fd0 0x7fa9c01083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:10.611 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.696+0000 7fa9c5d28700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01089c0 con 0x7fa9c0107fd0 2024-06-10T00:21:10.611 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.697+0000 7fa9bf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0107fd0 0x7fa9c01083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:10.611 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.697+0000 7fa9bf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0107fd0 0x7fa9c01083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42622/0 (socket says 172.21.15.62:42622) 2024-06-10T00:21:10.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.697+0000 7fa9bf7fe700 1 -- 172.21.15.62:0/3042281047 learned_addr learned my addr 172.21.15.62:0/3042281047 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:10.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.697+0000 7fa9bf7fe700 1 -- 172.21.15.62:0/3042281047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c01091d0 con 0x7fa9c0107fd0 2024-06-10T00:21:10.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.697+0000 7fa9bf7fe700 1 --2- 172.21.15.62:0/3042281047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0107fd0 0x7fa9c01083f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00098a0 tx=0x7fa9b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4501edd7ef800db server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:10.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.697+0000 7fa9be7fc700 1 -- 172.21.15.62:0/3042281047 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9b0004160 con 0x7fa9c0107fd0 2024-06-10T00:21:10.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.697+0000 7fa9be7fc700 1 -- 172.21.15.62:0/3042281047 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa9b0004300 con 0x7fa9c0107fd0 2024-06-10T00:21:10.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.697+0000 7fa9be7fc700 1 -- 172.21.15.62:0/3042281047 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9b0011b50 con 0x7fa9c0107fd0 2024-06-10T00:21:10.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.698+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3042281047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0107fd0 msgr2=0x7fa9c01083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:10.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.698+0000 7fa9c5d28700 1 --2- 172.21.15.62:0/3042281047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0107fd0 0x7fa9c01083f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00098a0 tx=0x7fa9b0009bb0 comp rx=0 tx=0).stop 2024-06-10T00:21:10.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.698+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3042281047 shutdown_connections 2024-06-10T00:21:10.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.698+0000 7fa9c5d28700 1 --2- 172.21.15.62:0/3042281047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0107fd0 0x7fa9c01083f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:10.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.698+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3042281047 >> 172.21.15.62:0/3042281047 conn(0x7fa9c01035b0 msgr2=0x7fa9c01059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:10.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.698+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3042281047 shutdown_connections 2024-06-10T00:21:10.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.698+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3042281047 wait complete. 2024-06-10T00:21:10.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.698+0000 7fa9c5d28700 1 Processor -- start 2024-06-10T00:21:10.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.699+0000 7fa9c5d28700 1 -- start start 2024-06-10T00:21:10.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.699+0000 7fa9c5d28700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0107fd0 0x7fa9c007db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:10.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.699+0000 7fa9c5d28700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c010c580 con 0x7fa9c0107fd0 2024-06-10T00:21:10.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.699+0000 7fa9bf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0107fd0 0x7fa9c007db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:10.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.699+0000 7fa9bf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0107fd0 0x7fa9c007db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42632/0 (socket says 172.21.15.62:42632) 2024-06-10T00:21:10.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.699+0000 7fa9bf7fe700 1 -- 172.21.15.62:0/3562705785 learned_addr learned my addr 172.21.15.62:0/3562705785 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:10.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.699+0000 7fa9bf7fe700 1 -- 172.21.15.62:0/3562705785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c007e0b0 con 0x7fa9c0107fd0 2024-06-10T00:21:10.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.699+0000 7fa9bf7fe700 1 --2- 172.21.15.62:0/3562705785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0107fd0 0x7fa9c007db70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0000c00 tx=0x7fa9b0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:10.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.700+0000 7fa9bcff9700 1 -- 172.21.15.62:0/3562705785 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9b0003b20 con 0x7fa9c0107fd0 2024-06-10T00:21:10.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.700+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3562705785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c007c2a0 con 0x7fa9c0107fd0 2024-06-10T00:21:10.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.700+0000 7fa9bcff9700 1 -- 172.21.15.62:0/3562705785 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa9b0003cc0 con 0x7fa9c0107fd0 2024-06-10T00:21:10.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.700+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3562705785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c007c7e0 con 0x7fa9c0107fd0 2024-06-10T00:21:10.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.700+0000 7fa9bcff9700 1 -- 172.21.15.62:0/3562705785 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9b0024910 con 0x7fa9c0107fd0 2024-06-10T00:21:10.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.700+0000 7fa9bcff9700 1 -- 172.21.15.62:0/3562705785 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa9b0022020 con 0x7fa9c0107fd0 2024-06-10T00:21:10.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.701+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3562705785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c0063af0 con 0x7fa9c0107fd0 2024-06-10T00:21:10.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.701+0000 7fa9bcff9700 1 --2- 172.21.15.62:0/3562705785 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fa9a8039050 0x7fa9a803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:10.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.701+0000 7fa9bcff9700 1 -- 172.21.15.62:0/3562705785 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa9b0011e00 con 0x7fa9c0107fd0 2024-06-10T00:21:10.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.701+0000 7fa9beffd700 1 --2- 172.21.15.62:0/3562705785 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fa9a8039050 0x7fa9a803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:10.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.701+0000 7fa9beffd700 1 --2- 172.21.15.62:0/3562705785 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fa9a8039050 0x7fa9a803b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4006fd0 tx=0x7fa9b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:10.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.706+0000 7fa9bcff9700 1 -- 172.21.15.62:0/3562705785 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa9b000b3e0 con 0x7fa9c0107fd0 2024-06-10T00:21:10.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.930+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3562705785 --> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa9c0104820 con 0x7fa9a8039050 2024-06-10T00:21:10.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.934+0000 7fa9bcff9700 1 -- 172.21.15.62:0/3562705785 <== mgr.14120 v2:172.21.15.62:6800/2327526412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa9c0104820 con 0x7fa9a8039050 2024-06-10T00:21:10.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.936+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3562705785 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fa9a8039050 msgr2=0x7fa9a803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:10.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.936+0000 7fa9c5d28700 1 --2- 172.21.15.62:0/3562705785 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fa9a8039050 0x7fa9a803b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4006fd0 tx=0x7fa9b4006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:10.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.936+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3562705785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0107fd0 msgr2=0x7fa9c007db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:10.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.936+0000 7fa9c5d28700 1 --2- 172.21.15.62:0/3562705785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0107fd0 0x7fa9c007db70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0000c00 tx=0x7fa9b0011c10 comp rx=0 tx=0).stop 2024-06-10T00:21:10.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.936+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3562705785 shutdown_connections 2024-06-10T00:21:10.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.936+0000 7fa9c5d28700 1 --2- 172.21.15.62:0/3562705785 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fa9a8039050 0x7fa9a803b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:10.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.936+0000 7fa9c5d28700 1 --2- 172.21.15.62:0/3562705785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0107fd0 0x7fa9c007db70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:10.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.936+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3562705785 >> 172.21.15.62:0/3562705785 conn(0x7fa9c01035b0 msgr2=0x7fa9c0104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:10.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.937+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3562705785 shutdown_connections 2024-06-10T00:21:10.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:09.937+0000 7fa9c5d28700 1 -- 172.21.15.62:0/3562705785 wait complete. 2024-06-10T00:21:11.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:10 smithi062 ceph-mon[32390]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:21:11.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:10 smithi062 ceph-mon[32390]: Saving service mgr spec with placement count:2 2024-06-10T00:21:11.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:10 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' 2024-06-10T00:21:11.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:10 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' 2024-06-10T00:21:11.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:10 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' 2024-06-10T00:21:11.809 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.047+0000 7f4347f8e700 1 Processor -- start 2024-06-10T00:21:11.809 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.047+0000 7f4347f8e700 1 -- start start 2024-06-10T00:21:11.809 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.047+0000 7f4347f8e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340108ac0 0x7f434010aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:11.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.047+0000 7f4347f8e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4340075c00 con 0x7f4340108ac0 2024-06-10T00:21:11.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.048+0000 7f4345d2a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340108ac0 0x7f434010aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:11.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.048+0000 7f4345d2a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340108ac0 0x7f434010aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42638/0 (socket says 172.21.15.62:42638) 2024-06-10T00:21:11.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.048+0000 7f4345d2a700 1 -- 172.21.15.62:0/1346259342 learned_addr learned my addr 172.21.15.62:0/1346259342 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:11.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.048+0000 7f4345d2a700 1 -- 172.21.15.62:0/1346259342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434010b3f0 con 0x7f4340108ac0 2024-06-10T00:21:11.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.048+0000 7f4345d2a700 1 --2- 172.21.15.62:0/1346259342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340108ac0 0x7f434010aeb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f433c0098a0 tx=0x7f433c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557581d8556ab057 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:11.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.048+0000 7f4344d28700 1 -- 172.21.15.62:0/1346259342 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f433c004160 con 0x7f4340108ac0 2024-06-10T00:21:11.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.048+0000 7f4344d28700 1 -- 172.21.15.62:0/1346259342 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433c004300 con 0x7f4340108ac0 2024-06-10T00:21:11.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.048+0000 7f4344d28700 1 -- 172.21.15.62:0/1346259342 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f433c011b20 con 0x7f4340108ac0 2024-06-10T00:21:11.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.049+0000 7f4347f8e700 1 -- 172.21.15.62:0/1346259342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340108ac0 msgr2=0x7f434010aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:11.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.049+0000 7f4347f8e700 1 --2- 172.21.15.62:0/1346259342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340108ac0 0x7f434010aeb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f433c0098a0 tx=0x7f433c009bb0 comp rx=0 tx=0).stop 2024-06-10T00:21:11.810 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.049+0000 7f4347f8e700 1 -- 172.21.15.62:0/1346259342 shutdown_connections 2024-06-10T00:21:11.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.049+0000 7f4347f8e700 1 --2- 172.21.15.62:0/1346259342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340108ac0 0x7f434010aeb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:11.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.049+0000 7f4347f8e700 1 -- 172.21.15.62:0/1346259342 >> 172.21.15.62:0/1346259342 conn(0x7f43401024b0 msgr2=0x7f43401048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:11.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.049+0000 7f4347f8e700 1 -- 172.21.15.62:0/1346259342 shutdown_connections 2024-06-10T00:21:11.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.049+0000 7f4347f8e700 1 -- 172.21.15.62:0/1346259342 wait complete. 2024-06-10T00:21:11.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.050+0000 7f4347f8e700 1 Processor -- start 2024-06-10T00:21:11.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.050+0000 7f4347f8e700 1 -- start start 2024-06-10T00:21:11.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.050+0000 7f4347f8e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340108ac0 0x7f434019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:11.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.050+0000 7f4347f8e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f434010c3f0 con 0x7f4340108ac0 2024-06-10T00:21:11.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.050+0000 7f4345d2a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340108ac0 0x7f434019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:11.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.050+0000 7f4345d2a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340108ac0 0x7f434019bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42652/0 (socket says 172.21.15.62:42652) 2024-06-10T00:21:11.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.050+0000 7f4345d2a700 1 -- 172.21.15.62:0/322106111 learned_addr learned my addr 172.21.15.62:0/322106111 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:11.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.050+0000 7f4345d2a700 1 -- 172.21.15.62:0/322106111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434019c490 con 0x7f4340108ac0 2024-06-10T00:21:11.811 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.051+0000 7f4345d2a700 1 --2- 172.21.15.62:0/322106111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340108ac0 0x7f434019bf50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f433c011b60 tx=0x7f433c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:11.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.051+0000 7f4332ffd700 1 -- 172.21.15.62:0/322106111 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f433c003b60 con 0x7f4340108ac0 2024-06-10T00:21:11.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.051+0000 7f4332ffd700 1 -- 172.21.15.62:0/322106111 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433c01a910 con 0x7f4340108ac0 2024-06-10T00:21:11.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.051+0000 7f4347f8e700 1 -- 172.21.15.62:0/322106111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434019c720 con 0x7f4340108ac0 2024-06-10T00:21:11.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.051+0000 7f4332ffd700 1 -- 172.21.15.62:0/322106111 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f433c024930 con 0x7f4340108ac0 2024-06-10T00:21:11.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.051+0000 7f4347f8e700 1 -- 172.21.15.62:0/322106111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f434019cb80 con 0x7f4340108ac0 2024-06-10T00:21:11.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.052+0000 7f4332ffd700 1 -- 172.21.15.62:0/322106111 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f433c022020 con 0x7f4340108ac0 2024-06-10T00:21:11.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.052+0000 7f4347f8e700 1 -- 172.21.15.62:0/322106111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4340063af0 con 0x7f4340108ac0 2024-06-10T00:21:11.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.052+0000 7f4332ffd700 1 --2- 172.21.15.62:0/322106111 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f432c0390a0 0x7f432c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:11.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.052+0000 7f4332ffd700 1 -- 172.21.15.62:0/322106111 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f433c04d510 con 0x7f4340108ac0 2024-06-10T00:21:11.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.052+0000 7f4345529700 1 --2- 172.21.15.62:0/322106111 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f432c0390a0 0x7f432c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:11.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.053+0000 7f4345529700 1 --2- 172.21.15.62:0/322106111 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f432c0390a0 0x7f432c03b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4334006fd0 tx=0x7f4334006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:11.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.057+0000 7f4332ffd700 1 -- 172.21.15.62:0/322106111 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f433c00fdd0 con 0x7f4340108ac0 2024-06-10T00:21:11.812 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.290+0000 7f4347f8e700 1 -- 172.21.15.62:0/322106111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f434019f410 con 0x7f4340108ac0 2024-06-10T00:21:11.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.293+0000 7f4332ffd700 1 -- 172.21.15.62:0/322106111 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f433c00bc70 con 0x7f4340108ac0 2024-06-10T00:21:11.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.296+0000 7f4347f8e700 1 -- 172.21.15.62:0/322106111 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f432c0390a0 msgr2=0x7f432c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:11.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.296+0000 7f4347f8e700 1 --2- 172.21.15.62:0/322106111 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f432c0390a0 0x7f432c03b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4334006fd0 tx=0x7f4334006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:11.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.296+0000 7f4347f8e700 1 -- 172.21.15.62:0/322106111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340108ac0 msgr2=0x7f434019bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:11.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.296+0000 7f4347f8e700 1 --2- 172.21.15.62:0/322106111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340108ac0 0x7f434019bf50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f433c011b60 tx=0x7f433c011c40 comp rx=0 tx=0).stop 2024-06-10T00:21:11.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.296+0000 7f4347f8e700 1 -- 172.21.15.62:0/322106111 shutdown_connections 2024-06-10T00:21:11.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.296+0000 7f4347f8e700 1 --2- 172.21.15.62:0/322106111 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f432c0390a0 0x7f432c03b560 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:11.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.296+0000 7f4347f8e700 1 --2- 172.21.15.62:0/322106111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340108ac0 0x7f434019bf50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:11.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.296+0000 7f4347f8e700 1 -- 172.21.15.62:0/322106111 >> 172.21.15.62:0/322106111 conn(0x7f43401024b0 msgr2=0x7f4340102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:11.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.296+0000 7f4347f8e700 1 -- 172.21.15.62:0/322106111 shutdown_connections 2024-06-10T00:21:11.813 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:11.296+0000 7f4347f8e700 1 -- 172.21.15.62:0/322106111 wait complete. 2024-06-10T00:21:12.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:12 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/322106111' entity='client.admin' 2024-06-10T00:21:13.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:13 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' 2024-06-10T00:21:13.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:13 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2326287673' entity='client.admin' 2024-06-10T00:21:13.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.790+0000 7f53b30bb700 1 Processor -- start 2024-06-10T00:21:13.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.790+0000 7f53b30bb700 1 -- start start 2024-06-10T00:21:13.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.790+0000 7f53b30bb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ac107fb0 0x7f53ac1083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:13.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.790+0000 7f53b30bb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac1089a0 con 0x7f53ac107fb0 2024-06-10T00:21:13.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.791+0000 7f53b0e57700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ac107fb0 0x7f53ac1083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:13.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.791+0000 7f53b0e57700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ac107fb0 0x7f53ac1083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42660/0 (socket says 172.21.15.62:42660) 2024-06-10T00:21:13.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.791+0000 7f53b0e57700 1 -- 172.21.15.62:0/771942273 learned_addr learned my addr 172.21.15.62:0/771942273 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:13.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.791+0000 7f53b0e57700 1 -- 172.21.15.62:0/771942273 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ac1091b0 con 0x7f53ac107fb0 2024-06-10T00:21:13.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.791+0000 7f53b0e57700 1 --2- 172.21.15.62:0/771942273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ac107fb0 0x7f53ac1083d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53a00098a0 tx=0x7f53a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc820a3208d15701 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:13.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.791+0000 7f53ab7fe700 1 -- 172.21.15.62:0/771942273 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53a0004160 con 0x7f53ac107fb0 2024-06-10T00:21:13.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.791+0000 7f53ab7fe700 1 -- 172.21.15.62:0/771942273 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53a0004300 con 0x7f53ac107fb0 2024-06-10T00:21:13.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.792+0000 7f53ab7fe700 1 -- 172.21.15.62:0/771942273 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53a0011b20 con 0x7f53ac107fb0 2024-06-10T00:21:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.792+0000 7f53b30bb700 1 -- 172.21.15.62:0/771942273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ac107fb0 msgr2=0x7f53ac1083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.792+0000 7f53b30bb700 1 --2- 172.21.15.62:0/771942273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ac107fb0 0x7f53ac1083d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53a00098a0 tx=0x7f53a0009bb0 comp rx=0 tx=0).stop 2024-06-10T00:21:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.792+0000 7f53b30bb700 1 -- 172.21.15.62:0/771942273 shutdown_connections 2024-06-10T00:21:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.792+0000 7f53b30bb700 1 --2- 172.21.15.62:0/771942273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ac107fb0 0x7f53ac1083d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.792+0000 7f53b30bb700 1 -- 172.21.15.62:0/771942273 >> 172.21.15.62:0/771942273 conn(0x7f53ac1035b0 msgr2=0x7f53ac1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.792+0000 7f53b30bb700 1 -- 172.21.15.62:0/771942273 shutdown_connections 2024-06-10T00:21:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.792+0000 7f53b30bb700 1 -- 172.21.15.62:0/771942273 wait complete. 2024-06-10T00:21:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.793+0000 7f53b30bb700 1 Processor -- start 2024-06-10T00:21:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.793+0000 7f53b30bb700 1 -- start start 2024-06-10T00:21:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.793+0000 7f53b30bb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ac107fb0 0x7f53ac07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.793+0000 7f53b30bb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac10c560 con 0x7f53ac107fb0 2024-06-10T00:21:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.793+0000 7f53b0e57700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ac107fb0 0x7f53ac07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.793+0000 7f53b0e57700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ac107fb0 0x7f53ac07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42676/0 (socket says 172.21.15.62:42676) 2024-06-10T00:21:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.793+0000 7f53b0e57700 1 -- 172.21.15.62:0/2326287673 learned_addr learned my addr 172.21.15.62:0/2326287673 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.794+0000 7f53b0e57700 1 -- 172.21.15.62:0/2326287673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ac07e0b0 con 0x7f53ac107fb0 2024-06-10T00:21:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.794+0000 7f53b0e57700 1 --2- 172.21.15.62:0/2326287673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ac107fb0 0x7f53ac07db70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f53a00117e0 tx=0x7f53a0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.794+0000 7f53a9ffb700 1 -- 172.21.15.62:0/2326287673 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53a0003bf0 con 0x7f53ac107fb0 2024-06-10T00:21:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.794+0000 7f53a9ffb700 1 -- 172.21.15.62:0/2326287673 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53a001a910 con 0x7f53ac107fb0 2024-06-10T00:21:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.794+0000 7f53a9ffb700 1 -- 172.21.15.62:0/2326287673 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53a00249a0 con 0x7f53ac107fb0 2024-06-10T00:21:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.794+0000 7f53b30bb700 1 -- 172.21.15.62:0/2326287673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ac07c2a0 con 0x7f53ac107fb0 2024-06-10T00:21:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.794+0000 7f53b30bb700 1 -- 172.21.15.62:0/2326287673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ac07c760 con 0x7f53ac107fb0 2024-06-10T00:21:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.795+0000 7f53a9ffb700 1 -- 172.21.15.62:0/2326287673 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f53a0022020 con 0x7f53ac107fb0 2024-06-10T00:21:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.795+0000 7f53b30bb700 1 -- 172.21.15.62:0/2326287673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53ac063af0 con 0x7f53ac107fb0 2024-06-10T00:21:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.795+0000 7f53a9ffb700 1 --2- 172.21.15.62:0/2326287673 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f539c039050 0x7f539c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:13.622 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.796+0000 7f53a9ffb700 1 -- 172.21.15.62:0/2326287673 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f53a004d310 con 0x7f53ac107fb0 2024-06-10T00:21:13.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.796+0000 7f53abfff700 1 --2- 172.21.15.62:0/2326287673 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f539c039050 0x7f539c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:13.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.796+0000 7f53abfff700 1 --2- 172.21.15.62:0/2326287673 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f539c039050 0x7f539c03b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5398006fd0 tx=0x7f5398006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:13.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:12.801+0000 7f53a9ffb700 1 -- 172.21.15.62:0/2326287673 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f53a0011e20 con 0x7f53ac107fb0 2024-06-10T00:21:13.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:13.009+0000 7f53b30bb700 1 -- 172.21.15.62:0/2326287673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f53ac07caa0 con 0x7f53ac107fb0 2024-06-10T00:21:13.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:13.012+0000 7f53a9ffb700 1 -- 172.21.15.62:0/2326287673 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f53ac07caa0 con 0x7f53ac107fb0 2024-06-10T00:21:13.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:13.015+0000 7f53b30bb700 1 -- 172.21.15.62:0/2326287673 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f539c039050 msgr2=0x7f539c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:13.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:13.015+0000 7f53b30bb700 1 --2- 172.21.15.62:0/2326287673 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f539c039050 0x7f539c03b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5398006fd0 tx=0x7f5398006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:13.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:13.015+0000 7f53b30bb700 1 -- 172.21.15.62:0/2326287673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ac107fb0 msgr2=0x7f53ac07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:13.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:13.015+0000 7f53b30bb700 1 --2- 172.21.15.62:0/2326287673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ac107fb0 0x7f53ac07db70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f53a00117e0 tx=0x7f53a0011c10 comp rx=0 tx=0).stop 2024-06-10T00:21:13.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:13.015+0000 7f53b30bb700 1 -- 172.21.15.62:0/2326287673 shutdown_connections 2024-06-10T00:21:13.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:13.015+0000 7f53b30bb700 1 --2- 172.21.15.62:0/2326287673 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f539c039050 0x7f539c03b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:13.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:13.015+0000 7f53b30bb700 1 --2- 172.21.15.62:0/2326287673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53ac107fb0 0x7f53ac07db70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:13.623 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:13.015+0000 7f53b30bb700 1 -- 172.21.15.62:0/2326287673 >> 172.21.15.62:0/2326287673 conn(0x7f53ac1035b0 msgr2=0x7f53ac104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:13.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:13.015+0000 7f53b30bb700 1 -- 172.21.15.62:0/2326287673 shutdown_connections 2024-06-10T00:21:13.624 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:13.015+0000 7f53b30bb700 1 -- 172.21.15.62:0/2326287673 wait complete. 2024-06-10T00:21:13.624 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling the dashboard module... 2024-06-10T00:21:14.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:14 smithi062 ceph-mon[32390]: from='mgr.14120 172.21.15.62:0/2827779702' entity='mgr.x' 2024-06-10T00:21:14.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:14 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1520110202' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-10T00:21:15.052 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:14 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: ignoring --setuser ceph since I am not root 2024-06-10T00:21:15.052 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:14 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: ignoring --setgroup ceph since I am not root 2024-06-10T00:21:15.052 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:14 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:14.859+0000 7fb832583200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:21:15.052 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:15 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:15.052+0000 7fb832583200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:21:15.357 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:15 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:15.161+0000 7fb832583200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:21:15.371 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.107+0000 7f55b5961700 1 Processor -- start 2024-06-10T00:21:15.372 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.107+0000 7f55b5961700 1 -- start start 2024-06-10T00:21:15.372 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.107+0000 7f55b5961700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b0108a40 0x7f55b010ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:15.372 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.107+0000 7f55b5961700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55b0075c80 con 0x7f55b0108a40 2024-06-10T00:21:15.372 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.108+0000 7f55aeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b0108a40 0x7f55b010ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:15.372 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.108+0000 7f55aeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b0108a40 0x7f55b010ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42678/0 (socket says 172.21.15.62:42678) 2024-06-10T00:21:15.372 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.108+0000 7f55aeffd700 1 -- 172.21.15.62:0/878646542 learned_addr learned my addr 172.21.15.62:0/878646542 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:15.372 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.108+0000 7f55aeffd700 1 -- 172.21.15.62:0/878646542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b010b370 con 0x7f55b0108a40 2024-06-10T00:21:15.372 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.108+0000 7f55aeffd700 1 --2- 172.21.15.62:0/878646542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b0108a40 0x7f55b010ae30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f55a00098a0 tx=0x7f55a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da280de68eb46864 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:15.372 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.109+0000 7f55adffb700 1 -- 172.21.15.62:0/878646542 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a0004160 con 0x7f55b0108a40 2024-06-10T00:21:15.372 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.109+0000 7f55adffb700 1 -- 172.21.15.62:0/878646542 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a0004300 con 0x7f55b0108a40 2024-06-10T00:21:15.372 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.109+0000 7f55adffb700 1 -- 172.21.15.62:0/878646542 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a0011b20 con 0x7f55b0108a40 2024-06-10T00:21:15.372 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.109+0000 7f55b5961700 1 -- 172.21.15.62:0/878646542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b0108a40 msgr2=0x7f55b010ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:15.373 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.109+0000 7f55b5961700 1 --2- 172.21.15.62:0/878646542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b0108a40 0x7f55b010ae30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f55a00098a0 tx=0x7f55a0009bb0 comp rx=0 tx=0).stop 2024-06-10T00:21:15.373 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.109+0000 7f55b5961700 1 -- 172.21.15.62:0/878646542 shutdown_connections 2024-06-10T00:21:15.373 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.109+0000 7f55b5961700 1 --2- 172.21.15.62:0/878646542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b0108a40 0x7f55b010ae30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:15.373 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.109+0000 7f55b5961700 1 -- 172.21.15.62:0/878646542 >> 172.21.15.62:0/878646542 conn(0x7f55b01024b0 msgr2=0x7f55b01048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:15.373 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.109+0000 7f55b5961700 1 -- 172.21.15.62:0/878646542 shutdown_connections 2024-06-10T00:21:15.373 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.110+0000 7f55b5961700 1 -- 172.21.15.62:0/878646542 wait complete. 2024-06-10T00:21:15.373 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.110+0000 7f55b5961700 1 Processor -- start 2024-06-10T00:21:15.373 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.110+0000 7f55b5961700 1 -- start start 2024-06-10T00:21:15.373 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.111+0000 7f55b5961700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b0108a40 0x7f55b019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:15.373 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.111+0000 7f55b5961700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55b010c3e0 con 0x7f55b0108a40 2024-06-10T00:21:15.373 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.111+0000 7f55aeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b0108a40 0x7f55b019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:15.373 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.111+0000 7f55aeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b0108a40 0x7f55b019bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42694/0 (socket says 172.21.15.62:42694) 2024-06-10T00:21:15.374 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.111+0000 7f55aeffd700 1 -- 172.21.15.62:0/1520110202 learned_addr learned my addr 172.21.15.62:0/1520110202 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:15.374 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.111+0000 7f55aeffd700 1 -- 172.21.15.62:0/1520110202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b019c480 con 0x7f55b0108a40 2024-06-10T00:21:15.374 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.111+0000 7f55aeffd700 1 --2- 172.21.15.62:0/1520110202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b0108a40 0x7f55b019bf40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f55a00116a0 tx=0x7f55a0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:15.374 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.111+0000 7f55b495f700 1 -- 172.21.15.62:0/1520110202 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a0003a40 con 0x7f55b0108a40 2024-06-10T00:21:15.374 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.111+0000 7f55b495f700 1 -- 172.21.15.62:0/1520110202 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a0003be0 con 0x7f55b0108a40 2024-06-10T00:21:15.374 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.111+0000 7f55b5961700 1 -- 172.21.15.62:0/1520110202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b019c710 con 0x7f55b0108a40 2024-06-10T00:21:15.374 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.112+0000 7f55b495f700 1 -- 172.21.15.62:0/1520110202 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a001a750 con 0x7f55b0108a40 2024-06-10T00:21:15.374 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.112+0000 7f55b5961700 1 -- 172.21.15.62:0/1520110202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b019cb70 con 0x7f55b0108a40 2024-06-10T00:21:15.374 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.112+0000 7f55b495f700 1 -- 172.21.15.62:0/1520110202 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f55a001aa40 con 0x7f55b0108a40 2024-06-10T00:21:15.374 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.113+0000 7f55b495f700 1 --2- 172.21.15.62:0/1520110202 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f5598039050 0x7f559803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:15.374 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.113+0000 7f55b495f700 1 -- 172.21.15.62:0/1520110202 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f55a004d660 con 0x7f55b0108a40 2024-06-10T00:21:15.374 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.113+0000 7f55ae7fc700 1 --2- 172.21.15.62:0/1520110202 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f5598039050 0x7f559803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:15.374 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.113+0000 7f55b5961700 1 -- 172.21.15.62:0/1520110202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b0063af0 con 0x7f55b0108a40 2024-06-10T00:21:15.375 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.114+0000 7f55ae7fc700 1 --2- 172.21.15.62:0/1520110202 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f5598039050 0x7f559803b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f55a4006fd0 tx=0x7f55a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:15.375 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.122+0000 7f55b495f700 1 -- 172.21.15.62:0/1520110202 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f55a0011e20 con 0x7f55b0108a40 2024-06-10T00:21:15.375 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.427+0000 7f55b5961700 1 -- 172.21.15.62:0/1520110202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f55b019f4a0 con 0x7f55b0108a40 2024-06-10T00:21:15.375 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.676+0000 7f55b495f700 1 -- 172.21.15.62:0/1520110202 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f55a0020240 con 0x7f55b0108a40 2024-06-10T00:21:15.375 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.677+0000 7f55b495f700 1 -- 172.21.15.62:0/1520110202 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f55a002d910 con 0x7f55b0108a40 2024-06-10T00:21:15.375 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.679+0000 7f55b5961700 1 -- 172.21.15.62:0/1520110202 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f5598039050 msgr2=0x7f559803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:15.375 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.679+0000 7f55b5961700 1 --2- 172.21.15.62:0/1520110202 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f5598039050 0x7f559803b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f55a4006fd0 tx=0x7f55a4006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:15.375 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.679+0000 7f55b5961700 1 -- 172.21.15.62:0/1520110202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b0108a40 msgr2=0x7f55b019bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:15.375 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.679+0000 7f55b5961700 1 --2- 172.21.15.62:0/1520110202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b0108a40 0x7f55b019bf40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f55a00116a0 tx=0x7f55a0011ba0 comp rx=0 tx=0).stop 2024-06-10T00:21:15.375 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.679+0000 7f55b5961700 1 -- 172.21.15.62:0/1520110202 shutdown_connections 2024-06-10T00:21:15.375 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.679+0000 7f55b5961700 1 --2- 172.21.15.62:0/1520110202 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f5598039050 0x7f559803b510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:15.375 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.679+0000 7f55b5961700 1 --2- 172.21.15.62:0/1520110202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b0108a40 0x7f55b019bf40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:15.376 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.679+0000 7f55b5961700 1 -- 172.21.15.62:0/1520110202 >> 172.21.15.62:0/1520110202 conn(0x7f55b01024b0 msgr2=0x7f55b0103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:15.376 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.679+0000 7f55b5961700 1 -- 172.21.15.62:0/1520110202 shutdown_connections 2024-06-10T00:21:15.376 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:14.680+0000 7f55b5961700 1 -- 172.21.15.62:0/1520110202 wait complete. 2024-06-10T00:21:15.676 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:15 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:15.491+0000 7fb832583200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:21:15.676 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:15 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:15.581+0000 7fb832583200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:21:16.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:15 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1520110202' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-10T00:21:16.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:15 smithi062 ceph-mon[32390]: mgrmap e9: x(active, since 19s) 2024-06-10T00:21:16.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:16 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1818169031' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-10T00:21:17.107 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:16 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:16.848+0000 7fb832583200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:21:17.107 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:17 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:17.026+0000 7fb832583200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:21:17.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:21:17.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-10T00:21:17.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-10T00:21:17.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-10T00:21:17.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-10T00:21:17.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:21:17.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.250+0000 7fbdcc07a700 1 Processor -- start 2024-06-10T00:21:17.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.250+0000 7fbdcc07a700 1 -- start start 2024-06-10T00:21:17.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.251+0000 7fbdcc07a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc4107fb0 0x7fbdc41083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:17.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.251+0000 7fbdcc07a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc41089a0 con 0x7fbdc4107fb0 2024-06-10T00:21:17.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.251+0000 7fbdc9e16700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc4107fb0 0x7fbdc41083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:17.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.251+0000 7fbdc9e16700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc4107fb0 0x7fbdc41083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42724/0 (socket says 172.21.15.62:42724) 2024-06-10T00:21:17.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.251+0000 7fbdc9e16700 1 -- 172.21.15.62:0/153773298 learned_addr learned my addr 172.21.15.62:0/153773298 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:17.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.251+0000 7fbdc9e16700 1 -- 172.21.15.62:0/153773298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc41091b0 con 0x7fbdc4107fb0 2024-06-10T00:21:17.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.251+0000 7fbdc9e16700 1 --2- 172.21.15.62:0/153773298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc4107fb0 0x7fbdc41083d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00098a0 tx=0x7fbdc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d544137d7f63f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:17.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.252+0000 7fbdc8e14700 1 -- 172.21.15.62:0/153773298 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdc0004160 con 0x7fbdc4107fb0 2024-06-10T00:21:17.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.252+0000 7fbdc8e14700 1 -- 172.21.15.62:0/153773298 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc0004300 con 0x7fbdc4107fb0 2024-06-10T00:21:17.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.252+0000 7fbdc8e14700 1 -- 172.21.15.62:0/153773298 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdc0011b20 con 0x7fbdc4107fb0 2024-06-10T00:21:17.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.252+0000 7fbdcc07a700 1 -- 172.21.15.62:0/153773298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc4107fb0 msgr2=0x7fbdc41083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:17.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.252+0000 7fbdcc07a700 1 --2- 172.21.15.62:0/153773298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc4107fb0 0x7fbdc41083d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00098a0 tx=0x7fbdc0009bb0 comp rx=0 tx=0).stop 2024-06-10T00:21:17.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.252+0000 7fbdcc07a700 1 -- 172.21.15.62:0/153773298 shutdown_connections 2024-06-10T00:21:17.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.252+0000 7fbdcc07a700 1 --2- 172.21.15.62:0/153773298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc4107fb0 0x7fbdc41083d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:17.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.252+0000 7fbdcc07a700 1 -- 172.21.15.62:0/153773298 >> 172.21.15.62:0/153773298 conn(0x7fbdc41035b0 msgr2=0x7fbdc41059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:17.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.252+0000 7fbdcc07a700 1 -- 172.21.15.62:0/153773298 shutdown_connections 2024-06-10T00:21:17.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.252+0000 7fbdcc07a700 1 -- 172.21.15.62:0/153773298 wait complete. 2024-06-10T00:21:17.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.253+0000 7fbdcc07a700 1 Processor -- start 2024-06-10T00:21:17.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.253+0000 7fbdcc07a700 1 -- start start 2024-06-10T00:21:17.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.253+0000 7fbdcc07a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc4107fb0 0x7fbdc419bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:17.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.253+0000 7fbdcc07a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc410c560 con 0x7fbdc4107fb0 2024-06-10T00:21:17.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.253+0000 7fbdc9e16700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc4107fb0 0x7fbdc419bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:17.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.253+0000 7fbdc9e16700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc4107fb0 0x7fbdc419bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42740/0 (socket says 172.21.15.62:42740) 2024-06-10T00:21:17.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.253+0000 7fbdc9e16700 1 -- 172.21.15.62:0/1818169031 learned_addr learned my addr 172.21.15.62:0/1818169031 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:17.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.254+0000 7fbdc9e16700 1 -- 172.21.15.62:0/1818169031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc419c460 con 0x7fbdc4107fb0 2024-06-10T00:21:17.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.254+0000 7fbdc9e16700 1 --2- 172.21.15.62:0/1818169031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc4107fb0 0x7fbdc419bf20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00117e0 tx=0x7fbdc0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:17.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.254+0000 7fbdbaffd700 1 -- 172.21.15.62:0/1818169031 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdc0003bf0 con 0x7fbdc4107fb0 2024-06-10T00:21:17.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.254+0000 7fbdbaffd700 1 -- 172.21.15.62:0/1818169031 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc001a910 con 0x7fbdc4107fb0 2024-06-10T00:21:17.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.254+0000 7fbdcc07a700 1 -- 172.21.15.62:0/1818169031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc419c6f0 con 0x7fbdc4107fb0 2024-06-10T00:21:17.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.254+0000 7fbdbaffd700 1 -- 172.21.15.62:0/1818169031 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdc0024970 con 0x7fbdc4107fb0 2024-06-10T00:21:17.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.254+0000 7fbdcc07a700 1 -- 172.21.15.62:0/1818169031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc419cb50 con 0x7fbdc4107fb0 2024-06-10T00:21:17.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.255+0000 7fbdcc07a700 1 -- 172.21.15.62:0/1818169031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc4063af0 con 0x7fbdc4107fb0 2024-06-10T00:21:17.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.255+0000 7fbdbaffd700 1 -- 172.21.15.62:0/1818169031 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fbdc0022020 con 0x7fbdc4107fb0 2024-06-10T00:21:17.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.255+0000 7fbdbaffd700 1 --2- 172.21.15.62:0/1818169031 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fbdb0039100 0x7fbdb003b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:17.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.255+0000 7fbdbaffd700 1 -- 172.21.15.62:0/1818169031 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbdc004d410 con 0x7fbdc4107fb0 2024-06-10T00:21:17.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.255+0000 7fbdc9615700 1 -- 172.21.15.62:0/1818169031 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fbdb0039100 msgr2=0x7fbdb003b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/2327526412 2024-06-10T00:21:17.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.255+0000 7fbdc9615700 1 --2- 172.21.15.62:0/1818169031 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fbdb0039100 0x7fbdb003b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:21:17.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.260+0000 7fbdbaffd700 1 -- 172.21.15.62:0/1818169031 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbdc0020c10 con 0x7fbdc4107fb0 2024-06-10T00:21:17.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.456+0000 7fbdc9615700 1 -- 172.21.15.62:0/1818169031 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fbdb0039100 msgr2=0x7fbdb003b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/2327526412 2024-06-10T00:21:17.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.456+0000 7fbdc9615700 1 --2- 172.21.15.62:0/1818169031 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fbdb0039100 0x7fbdb003b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T00:21:17.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.532+0000 7fbdcc07a700 1 -- 172.21.15.62:0/1818169031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fbdc419f5c0 con 0x7fbdc4107fb0 2024-06-10T00:21:17.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.533+0000 7fbdbaffd700 1 -- 172.21.15.62:0/1818169031 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fbdc000b5a0 con 0x7fbdc4107fb0 2024-06-10T00:21:17.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.536+0000 7fbdcc07a700 1 -- 172.21.15.62:0/1818169031 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fbdb0039100 msgr2=0x7fbdb003b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:21:17.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.536+0000 7fbdcc07a700 1 --2- 172.21.15.62:0/1818169031 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fbdb0039100 0x7fbdb003b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:17.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.536+0000 7fbdcc07a700 1 -- 172.21.15.62:0/1818169031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc4107fb0 msgr2=0x7fbdc419bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:17.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.536+0000 7fbdcc07a700 1 --2- 172.21.15.62:0/1818169031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc4107fb0 0x7fbdc419bf20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00117e0 tx=0x7fbdc0011ba0 comp rx=0 tx=0).stop 2024-06-10T00:21:17.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.536+0000 7fbdcc07a700 1 -- 172.21.15.62:0/1818169031 shutdown_connections 2024-06-10T00:21:17.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.536+0000 7fbdcc07a700 1 --2- 172.21.15.62:0/1818169031 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7fbdb0039100 0x7fbdb003b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:17.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.536+0000 7fbdcc07a700 1 --2- 172.21.15.62:0/1818169031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdc4107fb0 0x7fbdc419bf20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:17.129 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.536+0000 7fbdcc07a700 1 -- 172.21.15.62:0/1818169031 >> 172.21.15.62:0/1818169031 conn(0x7fbdc41035b0 msgr2=0x7fbdc4104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:17.129 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.536+0000 7fbdcc07a700 1 -- 172.21.15.62:0/1818169031 shutdown_connections 2024-06-10T00:21:17.129 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:16.536+0000 7fbdcc07a700 1 -- 172.21.15.62:0/1818169031 wait complete. 2024-06-10T00:21:17.129 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for the mgr to restart... 2024-06-10T00:21:17.129 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr epoch 9... 2024-06-10T00:21:17.427 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:17 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:17.212+0000 7fb832583200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:21:17.737 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:17 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:17.522+0000 7fb832583200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:21:17.738 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:17 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:17.620+0000 7fb832583200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:21:18.035 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:17 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:17.833+0000 7fb832583200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:21:18.357 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:18 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:18.034+0000 7fb832583200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:21:18.939 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:18 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:18.665+0000 7fb832583200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:21:19.211 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:19 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:19.108+0000 7fb832583200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:21:19.211 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:19 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:19.210+0000 7fb832583200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:21:19.607 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:19 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:19.421+0000 7fb832583200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:21:20.319 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:19 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:19.967+0000 7fb832583200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:21:20.319 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:20 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:20.064+0000 7fb832583200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:21:20.607 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:20 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:20.318+0000 7fb832583200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:21:20.608 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:20 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:20.458+0000 7fb832583200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:21:20.608 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:20 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:20.558+0000 7fb832583200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:21:20.990 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:20 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:20.679+0000 7fb832583200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:21:20.990 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:20 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:20.989+0000 7fb832583200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:21:21.901 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:21 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:21.621+0000 7fb832583200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:21:21.902 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:21:21 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:21:21.848+0000 7fb832583200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:21:22.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:21 smithi062 ceph-mon[32390]: Active manager daemon x restarted 2024-06-10T00:21:22.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:21 smithi062 ceph-mon[32390]: Activating manager daemon x 2024-06-10T00:21:22.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:21 smithi062 ceph-mon[32390]: osdmap e3: 0 total, 0 up, 0 in 2024-06-10T00:21:22.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:21 smithi062 ceph-mon[32390]: mgrmap e10: x(active, starting, since 0.00333803s) 2024-06-10T00:21:22.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:21:22.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T00:21:22.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T00:21:22.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T00:21:22.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T00:21:22.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:21 smithi062 ceph-mon[32390]: Manager daemon x is now available 2024-06-10T00:21:22.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:21:22.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T00:21:23.092 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T00:21:23.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:22 smithi062 ceph-mon[32390]: [10/Jun/2024:00:21:22] ENGINE Bus STARTING 2024-06-10T00:21:23.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:22 smithi062 ceph-mon[32390]: [10/Jun/2024:00:21:22] ENGINE Serving on https://172.21.15.62:7150 2024-06-10T00:21:23.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:22 smithi062 ceph-mon[32390]: [10/Jun/2024:00:21:22] ENGINE Serving on http://172.21.15.62:8765 2024-06-10T00:21:23.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:22 smithi062 ceph-mon[32390]: [10/Jun/2024:00:21:22] ENGINE Bus STARTED 2024-06-10T00:21:23.093 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:22 smithi062 ceph-mon[32390]: mgrmap e11: x(active, since 1.00627s) 2024-06-10T00:21:23.777 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:21:23.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-10T00:21:23.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-10T00:21:23.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:21:23.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.531+0000 7f33d415a700 1 Processor -- start 2024-06-10T00:21:23.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.531+0000 7f33d415a700 1 -- start start 2024-06-10T00:21:23.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.531+0000 7f33d415a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33cc1068a0 0x7f33cc106cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:23.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.531+0000 7f33d415a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc07be30 con 0x7f33cc1068a0 2024-06-10T00:21:23.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.531+0000 7f33d1ef6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33cc1068a0 0x7f33cc106cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:23.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.531+0000 7f33d1ef6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33cc1068a0 0x7f33cc106cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42744/0 (socket says 172.21.15.62:42744) 2024-06-10T00:21:23.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.531+0000 7f33d1ef6700 1 -- 172.21.15.62:0/3183866863 learned_addr learned my addr 172.21.15.62:0/3183866863 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:23.778 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.532+0000 7f33d1ef6700 1 -- 172.21.15.62:0/3183866863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc107200 con 0x7f33cc1068a0 2024-06-10T00:21:23.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.532+0000 7f33d1ef6700 1 --2- 172.21.15.62:0/3183866863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33cc1068a0 0x7f33cc106cc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f33c80098a0 tx=0x7f33c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f622369c27d9ad59 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:23.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.532+0000 7f33d0ef4700 1 -- 172.21.15.62:0/3183866863 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33c8004160 con 0x7f33cc1068a0 2024-06-10T00:21:23.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.532+0000 7f33d0ef4700 1 -- 172.21.15.62:0/3183866863 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33c8004300 con 0x7f33cc1068a0 2024-06-10T00:21:23.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.532+0000 7f33d0ef4700 1 -- 172.21.15.62:0/3183866863 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33c8011b20 con 0x7f33cc1068a0 2024-06-10T00:21:23.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.533+0000 7f33d415a700 1 -- 172.21.15.62:0/3183866863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33cc1068a0 msgr2=0x7f33cc106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:23.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.533+0000 7f33d415a700 1 --2- 172.21.15.62:0/3183866863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33cc1068a0 0x7f33cc106cc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f33c80098a0 tx=0x7f33c8009bb0 comp rx=0 tx=0).stop 2024-06-10T00:21:23.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.533+0000 7f33d415a700 1 -- 172.21.15.62:0/3183866863 shutdown_connections 2024-06-10T00:21:23.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.533+0000 7f33d415a700 1 --2- 172.21.15.62:0/3183866863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33cc1068a0 0x7f33cc106cc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:23.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.533+0000 7f33d415a700 1 -- 172.21.15.62:0/3183866863 >> 172.21.15.62:0/3183866863 conn(0x7f33cc1024a0 msgr2=0x7f33cc1048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:23.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.533+0000 7f33d415a700 1 -- 172.21.15.62:0/3183866863 shutdown_connections 2024-06-10T00:21:23.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.533+0000 7f33d415a700 1 -- 172.21.15.62:0/3183866863 wait complete. 2024-06-10T00:21:23.779 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.533+0000 7f33d415a700 1 Processor -- start 2024-06-10T00:21:23.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.533+0000 7f33d415a700 1 -- start start 2024-06-10T00:21:23.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.534+0000 7f33d415a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33cc1068a0 0x7f33cc19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:23.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.534+0000 7f33d415a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1082d0 con 0x7f33cc1068a0 2024-06-10T00:21:23.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.534+0000 7f33d1ef6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33cc1068a0 0x7f33cc19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:23.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.534+0000 7f33d1ef6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33cc1068a0 0x7f33cc19e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42746/0 (socket says 172.21.15.62:42746) 2024-06-10T00:21:23.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.534+0000 7f33d1ef6700 1 -- 172.21.15.62:0/3910102649 learned_addr learned my addr 172.21.15.62:0/3910102649 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:23.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.534+0000 7f33d1ef6700 1 -- 172.21.15.62:0/3910102649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc19e650 con 0x7f33cc1068a0 2024-06-10T00:21:23.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.534+0000 7f33d1ef6700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33cc1068a0 0x7f33cc19e110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f33c80116a0 tx=0x7f33c8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:23.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.534+0000 7f33c2ffd700 1 -- 172.21.15.62:0/3910102649 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33c8003a40 con 0x7f33cc1068a0 2024-06-10T00:21:23.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.535+0000 7f33d415a700 1 -- 172.21.15.62:0/3910102649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33cc19e8e0 con 0x7f33cc1068a0 2024-06-10T00:21:23.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.535+0000 7f33c2ffd700 1 -- 172.21.15.62:0/3910102649 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33c8003be0 con 0x7f33cc1068a0 2024-06-10T00:21:23.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.535+0000 7f33c2ffd700 1 -- 172.21.15.62:0/3910102649 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33c801a750 con 0x7f33cc1068a0 2024-06-10T00:21:23.780 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.535+0000 7f33d415a700 1 -- 172.21.15.62:0/3910102649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33cc19ed40 con 0x7f33cc1068a0 2024-06-10T00:21:23.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.536+0000 7f33c2ffd700 1 -- 172.21.15.62:0/3910102649 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f33c801aa50 con 0x7f33cc1068a0 2024-06-10T00:21:23.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.536+0000 7f33c2ffd700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f33b8039050 0x7f33b803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:23.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.536+0000 7f33c2ffd700 1 -- 172.21.15.62:0/3910102649 --> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f33b803bc20 con 0x7f33b8039050 2024-06-10T00:21:23.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.536+0000 7f33c2ffd700 1 -- 172.21.15.62:0/3910102649 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f33c804d8d0 con 0x7f33cc1068a0 2024-06-10T00:21:23.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.536+0000 7f33d16f5700 1 -- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f33b8039050 msgr2=0x7f33b803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/2327526412 2024-06-10T00:21:23.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.536+0000 7f33d16f5700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f33b8039050 0x7f33b803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:21:23.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.737+0000 7f33d16f5700 1 -- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f33b8039050 msgr2=0x7f33b803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/2327526412 2024-06-10T00:21:23.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:17.737+0000 7f33d16f5700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f33b8039050 0x7f33b803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T00:21:23.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:18.138+0000 7f33d16f5700 1 -- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f33b8039050 msgr2=0x7f33b803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/2327526412 2024-06-10T00:21:23.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:18.138+0000 7f33d16f5700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f33b8039050 0x7f33b803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-10T00:21:23.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:18.939+0000 7f33d16f5700 1 -- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f33b8039050 msgr2=0x7f33b803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/2327526412 2024-06-10T00:21:23.781 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:18.939+0000 7f33d16f5700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f33b8039050 0x7f33b803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-10T00:21:23.782 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:20.540+0000 7f33d16f5700 1 -- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f33b8039050 msgr2=0x7f33b803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/2327526412 2024-06-10T00:21:23.782 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:20.540+0000 7f33d16f5700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f33b8039050 0x7f33b803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-10T00:21:23.782 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:21.851+0000 7f33c2ffd700 1 -- 172.21.15.62:0/3910102649 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f33c802d910 con 0x7f33cc1068a0 2024-06-10T00:21:23.782 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:21.851+0000 7f33c2ffd700 1 -- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f33b8039050 msgr2=0x7f33b803b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:21:23.782 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:21.851+0000 7f33c2ffd700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f33b8039050 0x7f33b803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:23.783 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.853+0000 7f33c2ffd700 1 -- 172.21.15.62:0/3910102649 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f33c804f660 con 0x7f33cc1068a0 2024-06-10T00:21:23.783 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.853+0000 7f33c2ffd700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f33b803cb10 0x7f33b803ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:23.783 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.854+0000 7f33c2ffd700 1 -- 172.21.15.62:0/3910102649 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f33c801be50 con 0x7f33b803cb10 2024-06-10T00:21:23.783 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.854+0000 7f33d16f5700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f33b803cb10 0x7f33b803ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:23.783 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.855+0000 7f33d16f5700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f33b803cb10 0x7f33b803ef00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f33bc003a10 tx=0x7f33bc0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:23.783 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.855+0000 7f33c2ffd700 1 -- 172.21.15.62:0/3910102649 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f33c801be50 con 0x7f33b803cb10 2024-06-10T00:21:23.783 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.862+0000 7f33d415a700 1 -- 172.21.15.62:0/3910102649 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f33cc0695d0 con 0x7f33b803cb10 2024-06-10T00:21:23.783 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.862+0000 7f33c2ffd700 1 -- 172.21.15.62:0/3910102649 <== mgr.14152 v2:172.21.15.62:6800/3602410963 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f33cc0695d0 con 0x7f33b803cb10 2024-06-10T00:21:23.783 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.863+0000 7f33d415a700 1 -- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f33b803cb10 msgr2=0x7f33b803ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:23.783 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.863+0000 7f33d415a700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f33b803cb10 0x7f33b803ef00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f33bc003a10 tx=0x7f33bc0092b0 comp rx=0 tx=0).stop 2024-06-10T00:21:23.783 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.863+0000 7f33d415a700 1 -- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33cc1068a0 msgr2=0x7f33cc19e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:23.784 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.863+0000 7f33d415a700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33cc1068a0 0x7f33cc19e110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f33c80116a0 tx=0x7f33c8011ba0 comp rx=0 tx=0).stop 2024-06-10T00:21:23.784 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.863+0000 7f33d415a700 1 -- 172.21.15.62:0/3910102649 shutdown_connections 2024-06-10T00:21:23.784 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.863+0000 7f33d415a700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f33b803cb10 0x7f33b803ef00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:23.784 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.863+0000 7f33d415a700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33cc1068a0 0x7f33cc19e110 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:23.784 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.863+0000 7f33d415a700 1 --2- 172.21.15.62:0/3910102649 >> [v2:172.21.15.62:6800/2327526412,v1:172.21.15.62:6801/2327526412] conn(0x7f33b8039050 0x7f33b803b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:23.784 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.863+0000 7f33d415a700 1 -- 172.21.15.62:0/3910102649 >> 172.21.15.62:0/3910102649 conn(0x7f33cc1024a0 msgr2=0x7f33cc194db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:23.784 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.863+0000 7f33d415a700 1 -- 172.21.15.62:0/3910102649 shutdown_connections 2024-06-10T00:21:23.784 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:22.863+0000 7f33d415a700 1 -- 172.21.15.62:0/3910102649 wait complete. 2024-06-10T00:21:23.784 INFO:teuthology.orchestra.run.smithi062.stdout:mgr epoch 9 is available 2024-06-10T00:21:23.784 INFO:teuthology.orchestra.run.smithi062.stdout:Generating a dashboard self-signed certificate... 2024-06-10T00:21:24.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:24 smithi062 ceph-mon[32390]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-10T00:21:24.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:24 smithi062 ceph-mon[32390]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-10T00:21:24.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:25.093 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-10T00:21:25.093 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.203+0000 7fbde5058700 1 Processor -- start 2024-06-10T00:21:25.093 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.203+0000 7fbde5058700 1 -- start start 2024-06-10T00:21:25.093 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.203+0000 7fbde5058700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde0106630 0x7fbde0106a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:25.093 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.203+0000 7fbde5058700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0107020 con 0x7fbde0106630 2024-06-10T00:21:25.093 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.203+0000 7fbdded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde0106630 0x7fbde0106a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:25.093 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.203+0000 7fbdded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde0106630 0x7fbde0106a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58064/0 (socket says 172.21.15.62:58064) 2024-06-10T00:21:25.093 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.203+0000 7fbdded9d700 1 -- 172.21.15.62:0/3681753984 learned_addr learned my addr 172.21.15.62:0/3681753984 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:25.094 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.204+0000 7fbdded9d700 1 -- 172.21.15.62:0/3681753984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde0107830 con 0x7fbde0106630 2024-06-10T00:21:25.094 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.204+0000 7fbdded9d700 1 --2- 172.21.15.62:0/3681753984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde0106630 0x7fbde0106a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00098a0 tx=0x7fbdd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52baa7996de2a3c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:25.094 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.204+0000 7fbdddd9b700 1 -- 172.21.15.62:0/3681753984 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdd0004160 con 0x7fbde0106630 2024-06-10T00:21:25.094 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.204+0000 7fbdddd9b700 1 -- 172.21.15.62:0/3681753984 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdd0004300 con 0x7fbde0106630 2024-06-10T00:21:25.094 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.204+0000 7fbdddd9b700 1 -- 172.21.15.62:0/3681753984 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdd0011b20 con 0x7fbde0106630 2024-06-10T00:21:25.094 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.205+0000 7fbde5058700 1 -- 172.21.15.62:0/3681753984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde0106630 msgr2=0x7fbde0106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:25.094 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.205+0000 7fbde5058700 1 --2- 172.21.15.62:0/3681753984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde0106630 0x7fbde0106a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00098a0 tx=0x7fbdd0009bb0 comp rx=0 tx=0).stop 2024-06-10T00:21:25.094 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.205+0000 7fbde5058700 1 -- 172.21.15.62:0/3681753984 shutdown_connections 2024-06-10T00:21:25.094 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.205+0000 7fbde5058700 1 --2- 172.21.15.62:0/3681753984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde0106630 0x7fbde0106a50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:25.094 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.205+0000 7fbde5058700 1 -- 172.21.15.62:0/3681753984 >> 172.21.15.62:0/3681753984 conn(0x7fbde0101c70 msgr2=0x7fbde0104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:25.094 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.205+0000 7fbde5058700 1 -- 172.21.15.62:0/3681753984 shutdown_connections 2024-06-10T00:21:25.094 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.205+0000 7fbde5058700 1 -- 172.21.15.62:0/3681753984 wait complete. 2024-06-10T00:21:25.095 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.206+0000 7fbde5058700 1 Processor -- start 2024-06-10T00:21:25.095 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.206+0000 7fbde5058700 1 -- start start 2024-06-10T00:21:25.095 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.206+0000 7fbde5058700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde0106630 0x7fbde0103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:25.095 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.206+0000 7fbde5058700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbde010abe0 con 0x7fbde0106630 2024-06-10T00:21:25.095 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.206+0000 7fbdded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde0106630 0x7fbde0103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:25.095 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.206+0000 7fbdded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde0106630 0x7fbde0103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58074/0 (socket says 172.21.15.62:58074) 2024-06-10T00:21:25.095 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.206+0000 7fbdded9d700 1 -- 172.21.15.62:0/549548250 learned_addr learned my addr 172.21.15.62:0/549548250 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:25.095 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.207+0000 7fbdded9d700 1 -- 172.21.15.62:0/549548250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde019ecf0 con 0x7fbde0106630 2024-06-10T00:21:25.095 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.207+0000 7fbdded9d700 1 --2- 172.21.15.62:0/549548250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde0106630 0x7fbde0103110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0000c00 tx=0x7fbdd0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:25.095 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.207+0000 7fbdc7fff700 1 -- 172.21.15.62:0/549548250 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdd0003b60 con 0x7fbde0106630 2024-06-10T00:21:25.095 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.207+0000 7fbdc7fff700 1 -- 172.21.15.62:0/549548250 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdd001a910 con 0x7fbde0106630 2024-06-10T00:21:25.095 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.207+0000 7fbde5058700 1 -- 172.21.15.62:0/549548250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde019eec0 con 0x7fbde0106630 2024-06-10T00:21:25.095 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.207+0000 7fbdc7fff700 1 -- 172.21.15.62:0/549548250 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdd0024930 con 0x7fbde0106630 2024-06-10T00:21:25.096 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.207+0000 7fbde5058700 1 -- 172.21.15.62:0/549548250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde019f260 con 0x7fbde0106630 2024-06-10T00:21:25.096 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.208+0000 7fbdc7fff700 1 -- 172.21.15.62:0/549548250 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fbdd0022020 con 0x7fbde0106630 2024-06-10T00:21:25.096 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.208+0000 7fbdc7fff700 1 --2- 172.21.15.62:0/549548250 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbdc8038ff0 0x7fbdc803b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:25.096 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.208+0000 7fbdc7fff700 1 -- 172.21.15.62:0/549548250 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbdd004d160 con 0x7fbde0106630 2024-06-10T00:21:25.096 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.208+0000 7fbde5058700 1 -- 172.21.15.62:0/549548250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde0063af0 con 0x7fbde0106630 2024-06-10T00:21:25.096 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.209+0000 7fbdde59c700 1 --2- 172.21.15.62:0/549548250 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbdc8038ff0 0x7fbdc803b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:25.096 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.209+0000 7fbdde59c700 1 --2- 172.21.15.62:0/549548250 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbdc8038ff0 0x7fbdc803b4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4006fd0 tx=0x7fbdd4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:25.096 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.214+0000 7fbdc7fff700 1 -- 172.21.15.62:0/549548250 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbdd00172b0 con 0x7fbde0106630 2024-06-10T00:21:25.096 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.431+0000 7fbde5058700 1 -- 172.21.15.62:0/549548250 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fbde002df60 con 0x7fbdc8038ff0 2024-06-10T00:21:25.096 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.457+0000 7fbdc7fff700 1 -- 172.21.15.62:0/549548250 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fbdd002d770 con 0x7fbde0106630 2024-06-10T00:21:25.096 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.546+0000 7fbdc7fff700 1 -- 172.21.15.62:0/549548250 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fbde002df60 con 0x7fbdc8038ff0 2024-06-10T00:21:25.096 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.548+0000 7fbde5058700 1 -- 172.21.15.62:0/549548250 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbdc8038ff0 msgr2=0x7fbdc803b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:25.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.548+0000 7fbde5058700 1 --2- 172.21.15.62:0/549548250 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbdc8038ff0 0x7fbdc803b4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4006fd0 tx=0x7fbdd4006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:25.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.548+0000 7fbde5058700 1 -- 172.21.15.62:0/549548250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde0106630 msgr2=0x7fbde0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:25.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.548+0000 7fbde5058700 1 --2- 172.21.15.62:0/549548250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde0106630 0x7fbde0103110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0000c00 tx=0x7fbdd0011c10 comp rx=0 tx=0).stop 2024-06-10T00:21:25.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.549+0000 7fbde5058700 1 -- 172.21.15.62:0/549548250 shutdown_connections 2024-06-10T00:21:25.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.549+0000 7fbde5058700 1 --2- 172.21.15.62:0/549548250 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbdc8038ff0 0x7fbdc803b4b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:25.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.549+0000 7fbde5058700 1 --2- 172.21.15.62:0/549548250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbde0106630 0x7fbde0103110 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:25.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.549+0000 7fbde5058700 1 -- 172.21.15.62:0/549548250 >> 172.21.15.62:0/549548250 conn(0x7fbde0101c70 msgr2=0x7fbde0104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:25.097 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.549+0000 7fbde5058700 1 -- 172.21.15.62:0/549548250 shutdown_connections 2024-06-10T00:21:25.098 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:24.549+0000 7fbde5058700 1 -- 172.21.15.62:0/549548250 wait complete. 2024-06-10T00:21:25.098 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial admin user... 2024-06-10T00:21:25.721 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:25 smithi062 ceph-mon[32390]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:21:25.722 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:25 smithi062 ceph-mon[32390]: mgrmap e12: x(active, since 2s) 2024-06-10T00:21:25.722 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:25 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:25.722 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:25 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:25.722 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:25 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:27.046 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$UPVNcPSO77LuqQ00.YUlm.nwIL9sgcRUQObOZjjY6vha8eumyEDxW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717978885, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-10T00:21:27.046 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.502+0000 7f1ede6bd700 1 Processor -- start 2024-06-10T00:21:27.046 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.502+0000 7f1ede6bd700 1 -- start start 2024-06-10T00:21:27.046 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.502+0000 7f1ede6bd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed8107fd0 0x7f1ed81083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:27.046 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.502+0000 7f1ede6bd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed81089c0 con 0x7f1ed8107fd0 2024-06-10T00:21:27.046 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.503+0000 7f1ed7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed8107fd0 0x7f1ed81083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:27.046 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.503+0000 7f1ed7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed8107fd0 0x7f1ed81083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58084/0 (socket says 172.21.15.62:58084) 2024-06-10T00:21:27.046 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.503+0000 7f1ed7fff700 1 -- 172.21.15.62:0/2637054671 learned_addr learned my addr 172.21.15.62:0/2637054671 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:27.047 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.503+0000 7f1ed7fff700 1 -- 172.21.15.62:0/2637054671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed81091d0 con 0x7f1ed8107fd0 2024-06-10T00:21:27.047 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.503+0000 7f1ed7fff700 1 --2- 172.21.15.62:0/2637054671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed8107fd0 0x7f1ed81083f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80098a0 tx=0x7f1ec8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e0580cdfebe91a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:27.047 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.503+0000 7f1ed77fe700 1 -- 172.21.15.62:0/2637054671 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ec8004160 con 0x7f1ed8107fd0 2024-06-10T00:21:27.047 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.503+0000 7f1ed77fe700 1 -- 172.21.15.62:0/2637054671 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ec8004300 con 0x7f1ed8107fd0 2024-06-10T00:21:27.047 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.503+0000 7f1ed77fe700 1 -- 172.21.15.62:0/2637054671 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ec8011b20 con 0x7f1ed8107fd0 2024-06-10T00:21:27.047 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.504+0000 7f1ede6bd700 1 -- 172.21.15.62:0/2637054671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed8107fd0 msgr2=0x7f1ed81083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:27.047 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.504+0000 7f1ede6bd700 1 --2- 172.21.15.62:0/2637054671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed8107fd0 0x7f1ed81083f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80098a0 tx=0x7f1ec8009bb0 comp rx=0 tx=0).stop 2024-06-10T00:21:27.047 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.504+0000 7f1ede6bd700 1 -- 172.21.15.62:0/2637054671 shutdown_connections 2024-06-10T00:21:27.047 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.504+0000 7f1ede6bd700 1 --2- 172.21.15.62:0/2637054671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed8107fd0 0x7f1ed81083f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:27.047 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.504+0000 7f1ede6bd700 1 -- 172.21.15.62:0/2637054671 >> 172.21.15.62:0/2637054671 conn(0x7f1ed81035b0 msgr2=0x7f1ed81059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:27.047 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.504+0000 7f1ede6bd700 1 -- 172.21.15.62:0/2637054671 shutdown_connections 2024-06-10T00:21:27.047 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.504+0000 7f1ede6bd700 1 -- 172.21.15.62:0/2637054671 wait complete. 2024-06-10T00:21:27.047 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.505+0000 7f1ede6bd700 1 Processor -- start 2024-06-10T00:21:27.048 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.505+0000 7f1ede6bd700 1 -- start start 2024-06-10T00:21:27.048 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.505+0000 7f1ede6bd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed8107fd0 0x7f1ed807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:27.048 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.505+0000 7f1ede6bd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed810c580 con 0x7f1ed8107fd0 2024-06-10T00:21:27.048 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.505+0000 7f1ed7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed8107fd0 0x7f1ed807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:27.048 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.505+0000 7f1ed7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed8107fd0 0x7f1ed807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58088/0 (socket says 172.21.15.62:58088) 2024-06-10T00:21:27.048 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.505+0000 7f1ed7fff700 1 -- 172.21.15.62:0/128970099 learned_addr learned my addr 172.21.15.62:0/128970099 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:27.048 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.505+0000 7f1ed7fff700 1 -- 172.21.15.62:0/128970099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed807e0b0 con 0x7f1ed8107fd0 2024-06-10T00:21:27.048 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.506+0000 7f1ed7fff700 1 --2- 172.21.15.62:0/128970099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed8107fd0 0x7f1ed807db70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80117e0 tx=0x7f1ec8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:27.048 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.506+0000 7f1ed5ffb700 1 -- 172.21.15.62:0/128970099 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ec8003bf0 con 0x7f1ed8107fd0 2024-06-10T00:21:27.048 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.506+0000 7f1ed5ffb700 1 -- 172.21.15.62:0/128970099 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ec801a910 con 0x7f1ed8107fd0 2024-06-10T00:21:27.048 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.506+0000 7f1ede6bd700 1 -- 172.21.15.62:0/128970099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed807c2a0 con 0x7f1ed8107fd0 2024-06-10T00:21:27.048 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.506+0000 7f1ed5ffb700 1 -- 172.21.15.62:0/128970099 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ec8024970 con 0x7f1ed8107fd0 2024-06-10T00:21:27.049 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.506+0000 7f1ede6bd700 1 -- 172.21.15.62:0/128970099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed807c760 con 0x7f1ed8107fd0 2024-06-10T00:21:27.049 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.507+0000 7f1ed5ffb700 1 -- 172.21.15.62:0/128970099 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f1ec8022020 con 0x7f1ed8107fd0 2024-06-10T00:21:27.049 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.507+0000 7f1ed5ffb700 1 --2- 172.21.15.62:0/128970099 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f1ec4039100 0x7f1ec403b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:27.049 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.507+0000 7f1ed5ffb700 1 -- 172.21.15.62:0/128970099 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1ec804d660 con 0x7f1ed8107fd0 2024-06-10T00:21:27.049 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.507+0000 7f1ede6bd700 1 -- 172.21.15.62:0/128970099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed8063af0 con 0x7f1ed8107fd0 2024-06-10T00:21:27.049 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.507+0000 7f1ecffff700 1 --2- 172.21.15.62:0/128970099 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f1ec4039100 0x7f1ec403b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:27.049 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.508+0000 7f1ecffff700 1 --2- 172.21.15.62:0/128970099 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f1ec4039100 0x7f1ec403b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0006fd0 tx=0x7f1ec0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:27.049 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.512+0000 7f1ed5ffb700 1 -- 172.21.15.62:0/128970099 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1ec802cdf0 con 0x7f1ed8107fd0 2024-06-10T00:21:27.049 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.742+0000 7f1ede6bd700 1 -- 172.21.15.62:0/128970099 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f1ed807ce00 con 0x7f1ec4039100 2024-06-10T00:21:27.049 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.988+0000 7f1ed5ffb700 1 -- 172.21.15.62:0/128970099 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f1ed807ce00 con 0x7f1ec4039100 2024-06-10T00:21:27.049 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.990+0000 7f1ede6bd700 1 -- 172.21.15.62:0/128970099 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f1ec4039100 msgr2=0x7f1ec403b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:27.049 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.990+0000 7f1ede6bd700 1 --2- 172.21.15.62:0/128970099 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f1ec4039100 0x7f1ec403b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0006fd0 tx=0x7f1ec0006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:27.049 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.990+0000 7f1ede6bd700 1 -- 172.21.15.62:0/128970099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed8107fd0 msgr2=0x7f1ed807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:27.050 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.990+0000 7f1ede6bd700 1 --2- 172.21.15.62:0/128970099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed8107fd0 0x7f1ed807db70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80117e0 tx=0x7f1ec8011ba0 comp rx=0 tx=0).stop 2024-06-10T00:21:27.050 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.990+0000 7f1ede6bd700 1 -- 172.21.15.62:0/128970099 shutdown_connections 2024-06-10T00:21:27.050 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.990+0000 7f1ede6bd700 1 --2- 172.21.15.62:0/128970099 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f1ec4039100 0x7f1ec403b5c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:27.050 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.990+0000 7f1ede6bd700 1 --2- 172.21.15.62:0/128970099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ed8107fd0 0x7f1ed807db70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:27.050 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.990+0000 7f1ede6bd700 1 -- 172.21.15.62:0/128970099 >> 172.21.15.62:0/128970099 conn(0x7f1ed81035b0 msgr2=0x7f1ed8104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:27.050 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.990+0000 7f1ede6bd700 1 -- 172.21.15.62:0/128970099 shutdown_connections 2024-06-10T00:21:27.050 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:25.990+0000 7f1ede6bd700 1 -- 172.21.15.62:0/128970099 wait complete. 2024-06-10T00:21:27.050 INFO:teuthology.orchestra.run.smithi062.stdout:Fetching dashboard port number... 2024-06-10T00:21:27.166 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:26 smithi062 ceph-mon[32390]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:21:27.167 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:26 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:28.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:27 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3081831342' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-10T00:21:28.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 8443 2024-06-10T00:21:28.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.446+0000 7f7f37b5b700 1 Processor -- start 2024-06-10T00:21:28.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.446+0000 7f7f37b5b700 1 -- start start 2024-06-10T00:21:28.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.446+0000 7f7f37b5b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f30107fa0 0x7f7f301083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:28.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.446+0000 7f7f37b5b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30108990 con 0x7f7f30107fa0 2024-06-10T00:21:28.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.446+0000 7f7f358f7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f30107fa0 0x7f7f301083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:28.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.446+0000 7f7f358f7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f30107fa0 0x7f7f301083c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58090/0 (socket says 172.21.15.62:58090) 2024-06-10T00:21:28.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.446+0000 7f7f358f7700 1 -- 172.21.15.62:0/3545526654 learned_addr learned my addr 172.21.15.62:0/3545526654 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:28.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.447+0000 7f7f358f7700 1 -- 172.21.15.62:0/3545526654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f301091a0 con 0x7f7f30107fa0 2024-06-10T00:21:28.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.447+0000 7f7f358f7700 1 --2- 172.21.15.62:0/3545526654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f30107fa0 0x7f7f301083c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0098a0 tx=0x7f7f2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc0760d8c0cd09c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:28.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.447+0000 7f7f348f5700 1 -- 172.21.15.62:0/3545526654 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f2c004160 con 0x7f7f30107fa0 2024-06-10T00:21:28.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.447+0000 7f7f348f5700 1 -- 172.21.15.62:0/3545526654 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f2c004300 con 0x7f7f30107fa0 2024-06-10T00:21:28.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.447+0000 7f7f348f5700 1 -- 172.21.15.62:0/3545526654 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f2c011b20 con 0x7f7f30107fa0 2024-06-10T00:21:28.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.447+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3545526654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f30107fa0 msgr2=0x7f7f301083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:28.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.447+0000 7f7f37b5b700 1 --2- 172.21.15.62:0/3545526654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f30107fa0 0x7f7f301083c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0098a0 tx=0x7f7f2c009bb0 comp rx=0 tx=0).stop 2024-06-10T00:21:28.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.447+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3545526654 shutdown_connections 2024-06-10T00:21:28.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.447+0000 7f7f37b5b700 1 --2- 172.21.15.62:0/3545526654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f30107fa0 0x7f7f301083c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:28.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.447+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3545526654 >> 172.21.15.62:0/3545526654 conn(0x7f7f30103580 msgr2=0x7f7f301059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:28.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.448+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3545526654 shutdown_connections 2024-06-10T00:21:28.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.448+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3545526654 wait complete. 2024-06-10T00:21:28.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.448+0000 7f7f37b5b700 1 Processor -- start 2024-06-10T00:21:28.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.448+0000 7f7f37b5b700 1 -- start start 2024-06-10T00:21:28.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.448+0000 7f7f37b5b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f30107fa0 0x7f7f3019beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:28.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.448+0000 7f7f37b5b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3010c550 con 0x7f7f30107fa0 2024-06-10T00:21:28.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.448+0000 7f7f358f7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f30107fa0 0x7f7f3019beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:28.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.448+0000 7f7f358f7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f30107fa0 0x7f7f3019beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58104/0 (socket says 172.21.15.62:58104) 2024-06-10T00:21:28.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.448+0000 7f7f358f7700 1 -- 172.21.15.62:0/3081831342 learned_addr learned my addr 172.21.15.62:0/3081831342 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:28.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.449+0000 7f7f358f7700 1 -- 172.21.15.62:0/3081831342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3019c3f0 con 0x7f7f30107fa0 2024-06-10T00:21:28.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.449+0000 7f7f358f7700 1 --2- 172.21.15.62:0/3081831342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f30107fa0 0x7f7f3019beb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0117e0 tx=0x7f7f2c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:28.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.449+0000 7f7f22ffd700 1 -- 172.21.15.62:0/3081831342 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f2c003bf0 con 0x7f7f30107fa0 2024-06-10T00:21:28.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.449+0000 7f7f22ffd700 1 -- 172.21.15.62:0/3081831342 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f2c01a910 con 0x7f7f30107fa0 2024-06-10T00:21:28.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.449+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3081831342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3019c680 con 0x7f7f30107fa0 2024-06-10T00:21:28.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.449+0000 7f7f22ffd700 1 -- 172.21.15.62:0/3081831342 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f2c024970 con 0x7f7f30107fa0 2024-06-10T00:21:28.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.449+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3081831342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3019cae0 con 0x7f7f30107fa0 2024-06-10T00:21:28.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.450+0000 7f7f22ffd700 1 -- 172.21.15.62:0/3081831342 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f7f2c022020 con 0x7f7f30107fa0 2024-06-10T00:21:28.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.450+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3081831342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f30063af0 con 0x7f7f30107fa0 2024-06-10T00:21:28.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.450+0000 7f7f22ffd700 1 --2- 172.21.15.62:0/3081831342 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7f1c039100 0x7f7f1c03b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:28.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.450+0000 7f7f22ffd700 1 -- 172.21.15.62:0/3081831342 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7f2c04d4d0 con 0x7f7f30107fa0 2024-06-10T00:21:28.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.450+0000 7f7f350f6700 1 --2- 172.21.15.62:0/3081831342 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7f1c039100 0x7f7f1c03b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:28.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.451+0000 7f7f350f6700 1 --2- 172.21.15.62:0/3081831342 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7f1c039100 0x7f7f1c03b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f24006fd0 tx=0x7f7f24006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:28.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.455+0000 7f7f22ffd700 1 -- 172.21.15.62:0/3081831342 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7f2c01ab70 con 0x7f7f30107fa0 2024-06-10T00:21:28.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.660+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3081831342 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f7f3019f570 con 0x7f7f30107fa0 2024-06-10T00:21:28.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.660+0000 7f7f22ffd700 1 -- 172.21.15.62:0/3081831342 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f7f2c02d3f0 con 0x7f7f30107fa0 2024-06-10T00:21:28.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.662+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3081831342 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7f1c039100 msgr2=0x7f7f1c03b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:28.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.662+0000 7f7f37b5b700 1 --2- 172.21.15.62:0/3081831342 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7f1c039100 0x7f7f1c03b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f24006fd0 tx=0x7f7f24006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:28.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.662+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3081831342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f30107fa0 msgr2=0x7f7f3019beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:28.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.662+0000 7f7f37b5b700 1 --2- 172.21.15.62:0/3081831342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f30107fa0 0x7f7f3019beb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0117e0 tx=0x7f7f2c011ba0 comp rx=0 tx=0).stop 2024-06-10T00:21:28.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.662+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3081831342 shutdown_connections 2024-06-10T00:21:28.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.662+0000 7f7f37b5b700 1 --2- 172.21.15.62:0/3081831342 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7f1c039100 0x7f7f1c03b5c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:28.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.662+0000 7f7f37b5b700 1 --2- 172.21.15.62:0/3081831342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f30107fa0 0x7f7f3019beb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:28.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.662+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3081831342 >> 172.21.15.62:0/3081831342 conn(0x7f7f30103580 msgr2=0x7f7f30104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:28.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.662+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3081831342 shutdown_connections 2024-06-10T00:21:28.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:27.662+0000 7f7f37b5b700 1 -- 172.21.15.62:0/3081831342 wait complete. 2024-06-10T00:21:28.264 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T00:21:28.264 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-06-10T00:21:28.272 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T00:21:28.272 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-06-10T00:21:28.272 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-06-10T00:21:28.272 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-10T00:21:28.273 INFO:teuthology.orchestra.run.smithi062.stdout:Ceph Dashboard is now available at: 2024-06-10T00:21:28.273 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:21:28.273 INFO:teuthology.orchestra.run.smithi062.stdout: URL: https://smithi062.front.sepia.ceph.com:8443/ 2024-06-10T00:21:28.273 INFO:teuthology.orchestra.run.smithi062.stdout: User: admin 2024-06-10T00:21:28.273 INFO:teuthology.orchestra.run.smithi062.stdout: Password: nexx228z50 2024-06-10T00:21:28.273 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:21:28.273 INFO:teuthology.orchestra.run.smithi062.stdout:Saving cluster configuration to /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config directory 2024-06-10T00:21:28.274 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling autotune for osd_memory_target 2024-06-10T00:21:29.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:28 smithi062 ceph-mon[32390]: mgrmap e13: x(active, since 6s) 2024-06-10T00:21:29.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.705+0000 7f7859e54700 1 Processor -- start 2024-06-10T00:21:29.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.705+0000 7f7859e54700 1 -- start start 2024-06-10T00:21:29.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.705+0000 7f7859e54700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854108000 0x7f7854108420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:29.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.705+0000 7f7859e54700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f78541089f0 con 0x7f7854108000 2024-06-10T00:21:29.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.705+0000 7f78537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854108000 0x7f7854108420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:29.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.705+0000 7f78537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854108000 0x7f7854108420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58814/0 (socket says 172.21.15.62:58814) 2024-06-10T00:21:29.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.705+0000 7f78537fe700 1 -- 172.21.15.62:0/1809987893 learned_addr learned my addr 172.21.15.62:0/1809987893 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:29.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.706+0000 7f78537fe700 1 -- 172.21.15.62:0/1809987893 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7854109200 con 0x7f7854108000 2024-06-10T00:21:29.494 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.706+0000 7f78537fe700 1 --2- 172.21.15.62:0/1809987893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854108000 0x7f7854108420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f78440098a0 tx=0x7f7844009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2f541f07bd90eb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:29.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.706+0000 7f78527fc700 1 -- 172.21.15.62:0/1809987893 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7844004160 con 0x7f7854108000 2024-06-10T00:21:29.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.706+0000 7f78527fc700 1 -- 172.21.15.62:0/1809987893 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7844004300 con 0x7f7854108000 2024-06-10T00:21:29.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.706+0000 7f78527fc700 1 -- 172.21.15.62:0/1809987893 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7844011b20 con 0x7f7854108000 2024-06-10T00:21:29.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.707+0000 7f7859e54700 1 -- 172.21.15.62:0/1809987893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854108000 msgr2=0x7f7854108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:29.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.707+0000 7f7859e54700 1 --2- 172.21.15.62:0/1809987893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854108000 0x7f7854108420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f78440098a0 tx=0x7f7844009bb0 comp rx=0 tx=0).stop 2024-06-10T00:21:29.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.707+0000 7f7859e54700 1 -- 172.21.15.62:0/1809987893 shutdown_connections 2024-06-10T00:21:29.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.707+0000 7f7859e54700 1 --2- 172.21.15.62:0/1809987893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854108000 0x7f7854108420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:29.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.707+0000 7f7859e54700 1 -- 172.21.15.62:0/1809987893 >> 172.21.15.62:0/1809987893 conn(0x7f78541035c0 msgr2=0x7f78541059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:29.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.707+0000 7f7859e54700 1 -- 172.21.15.62:0/1809987893 shutdown_connections 2024-06-10T00:21:29.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.707+0000 7f7859e54700 1 -- 172.21.15.62:0/1809987893 wait complete. 2024-06-10T00:21:29.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.707+0000 7f7859e54700 1 Processor -- start 2024-06-10T00:21:29.495 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.708+0000 7f7859e54700 1 -- start start 2024-06-10T00:21:29.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.708+0000 7f7859e54700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854108000 0x7f785419bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:29.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.708+0000 7f7859e54700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f785410c5b0 con 0x7f7854108000 2024-06-10T00:21:29.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.708+0000 7f78537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854108000 0x7f785419bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:29.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.708+0000 7f78537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854108000 0x7f785419bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58818/0 (socket says 172.21.15.62:58818) 2024-06-10T00:21:29.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.708+0000 7f78537fe700 1 -- 172.21.15.62:0/3705041725 learned_addr learned my addr 172.21.15.62:0/3705041725 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:29.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.708+0000 7f78537fe700 1 -- 172.21.15.62:0/3705041725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785419c430 con 0x7f7854108000 2024-06-10T00:21:29.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.708+0000 7f78537fe700 1 --2- 172.21.15.62:0/3705041725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854108000 0x7f785419bef0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f78440117e0 tx=0x7f7844011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:29.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.708+0000 7f7850ff9700 1 -- 172.21.15.62:0/3705041725 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7844003bf0 con 0x7f7854108000 2024-06-10T00:21:29.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.708+0000 7f7850ff9700 1 -- 172.21.15.62:0/3705041725 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f784401a910 con 0x7f7854108000 2024-06-10T00:21:29.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.708+0000 7f7859e54700 1 -- 172.21.15.62:0/3705041725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785419c6c0 con 0x7f7854108000 2024-06-10T00:21:29.496 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.709+0000 7f7850ff9700 1 -- 172.21.15.62:0/3705041725 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7844024970 con 0x7f7854108000 2024-06-10T00:21:29.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.709+0000 7f7859e54700 1 -- 172.21.15.62:0/3705041725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f785419cb20 con 0x7f7854108000 2024-06-10T00:21:29.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.709+0000 7f7850ff9700 1 -- 172.21.15.62:0/3705041725 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7844022020 con 0x7f7854108000 2024-06-10T00:21:29.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.710+0000 7f7850ff9700 1 --2- 172.21.15.62:0/3705041725 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f783c039140 0x7f783c03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:29.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.710+0000 7f7850ff9700 1 -- 172.21.15.62:0/3705041725 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f784404da60 con 0x7f7854108000 2024-06-10T00:21:29.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.710+0000 7f7852ffd700 1 --2- 172.21.15.62:0/3705041725 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f783c039140 0x7f783c03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:29.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.710+0000 7f7859e54700 1 -- 172.21.15.62:0/3705041725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7854063af0 con 0x7f7854108000 2024-06-10T00:21:29.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.710+0000 7f7852ffd700 1 --2- 172.21.15.62:0/3705041725 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f783c039140 0x7f783c03b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7848006fd0 tx=0x7f7848006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:29.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.715+0000 7f7850ff9700 1 -- 172.21.15.62:0/3705041725 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f784400b3e0 con 0x7f7854108000 2024-06-10T00:21:29.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.921+0000 7f7859e54700 1 -- 172.21.15.62:0/3705041725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f785419f510 con 0x7f7854108000 2024-06-10T00:21:29.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.921+0000 7f7850ff9700 1 -- 172.21.15.62:0/3705041725 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f7844032030 con 0x7f7854108000 2024-06-10T00:21:29.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.924+0000 7f7859e54700 1 -- 172.21.15.62:0/3705041725 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f783c039140 msgr2=0x7f783c03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:29.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.924+0000 7f7859e54700 1 --2- 172.21.15.62:0/3705041725 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f783c039140 0x7f783c03b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7848006fd0 tx=0x7f7848006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:29.498 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.924+0000 7f7859e54700 1 -- 172.21.15.62:0/3705041725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854108000 msgr2=0x7f785419bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:29.498 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.924+0000 7f7859e54700 1 --2- 172.21.15.62:0/3705041725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854108000 0x7f785419bef0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f78440117e0 tx=0x7f7844011ba0 comp rx=0 tx=0).stop 2024-06-10T00:21:29.498 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.924+0000 7f7859e54700 1 -- 172.21.15.62:0/3705041725 shutdown_connections 2024-06-10T00:21:29.498 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.924+0000 7f7859e54700 1 --2- 172.21.15.62:0/3705041725 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f783c039140 0x7f783c03b600 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:29.498 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.924+0000 7f7859e54700 1 --2- 172.21.15.62:0/3705041725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7854108000 0x7f785419bef0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:29.498 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.924+0000 7f7859e54700 1 -- 172.21.15.62:0/3705041725 >> 172.21.15.62:0/3705041725 conn(0x7f78541035c0 msgr2=0x7f7854104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:29.498 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.924+0000 7f7859e54700 1 -- 172.21.15.62:0/3705041725 shutdown_connections 2024-06-10T00:21:29.498 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:28.924+0000 7f7859e54700 1 -- 172.21.15.62:0/3705041725 wait complete. 2024-06-10T00:21:30.716 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.915+0000 7facd8293700 1 Processor -- start 2024-06-10T00:21:30.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.915+0000 7facd8293700 1 -- start start 2024-06-10T00:21:30.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.915+0000 7facd8293700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facd007dec0 0x7facd007c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:30.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.915+0000 7facd8293700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7facd007c8f0 con 0x7facd007dec0 2024-06-10T00:21:30.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.915+0000 7facd602f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facd007dec0 0x7facd007c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:30.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.915+0000 7facd602f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facd007dec0 0x7facd007c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58820/0 (socket says 172.21.15.62:58820) 2024-06-10T00:21:30.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.915+0000 7facd602f700 1 -- 172.21.15.62:0/3464471034 learned_addr learned my addr 172.21.15.62:0/3464471034 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:30.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.916+0000 7facd602f700 1 -- 172.21.15.62:0/3464471034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd007ca70 con 0x7facd007dec0 2024-06-10T00:21:30.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.916+0000 7facd602f700 1 --2- 172.21.15.62:0/3464471034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facd007dec0 0x7facd007c320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faccc0098a0 tx=0x7faccc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2771e2bf2df3f3f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:30.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.916+0000 7facd502d700 1 -- 172.21.15.62:0/3464471034 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faccc004160 con 0x7facd007dec0 2024-06-10T00:21:30.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.916+0000 7facd502d700 1 -- 172.21.15.62:0/3464471034 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faccc004300 con 0x7facd007dec0 2024-06-10T00:21:30.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.916+0000 7facd502d700 1 -- 172.21.15.62:0/3464471034 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faccc011b20 con 0x7facd007dec0 2024-06-10T00:21:30.717 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.917+0000 7facd8293700 1 -- 172.21.15.62:0/3464471034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facd007dec0 msgr2=0x7facd007c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:30.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.917+0000 7facd8293700 1 --2- 172.21.15.62:0/3464471034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facd007dec0 0x7facd007c320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faccc0098a0 tx=0x7faccc009bb0 comp rx=0 tx=0).stop 2024-06-10T00:21:30.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.917+0000 7facd8293700 1 -- 172.21.15.62:0/3464471034 shutdown_connections 2024-06-10T00:21:30.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.917+0000 7facd8293700 1 --2- 172.21.15.62:0/3464471034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facd007dec0 0x7facd007c320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:30.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.917+0000 7facd8293700 1 -- 172.21.15.62:0/3464471034 >> 172.21.15.62:0/3464471034 conn(0x7facd01035b0 msgr2=0x7facd01059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:30.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.917+0000 7facd8293700 1 -- 172.21.15.62:0/3464471034 shutdown_connections 2024-06-10T00:21:30.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.917+0000 7facd8293700 1 -- 172.21.15.62:0/3464471034 wait complete. 2024-06-10T00:21:30.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.917+0000 7facd8293700 1 Processor -- start 2024-06-10T00:21:30.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.918+0000 7facd8293700 1 -- start start 2024-06-10T00:21:30.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.918+0000 7facd8293700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facd007dec0 0x7facd019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:30.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.918+0000 7facd8293700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7facd010e610 con 0x7facd007dec0 2024-06-10T00:21:30.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.918+0000 7facd602f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facd007dec0 0x7facd019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:30.718 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.918+0000 7facd602f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facd007dec0 0x7facd019e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58834/0 (socket says 172.21.15.62:58834) 2024-06-10T00:21:30.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.918+0000 7facd602f700 1 -- 172.21.15.62:0/3991116892 learned_addr learned my addr 172.21.15.62:0/3991116892 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:30.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.918+0000 7facd602f700 1 -- 172.21.15.62:0/3991116892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd019e620 con 0x7facd007dec0 2024-06-10T00:21:30.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.918+0000 7facd602f700 1 --2- 172.21.15.62:0/3991116892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facd007dec0 0x7facd019e0e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faccc0117e0 tx=0x7faccc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:30.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.918+0000 7facc77fe700 1 -- 172.21.15.62:0/3991116892 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faccc003bf0 con 0x7facd007dec0 2024-06-10T00:21:30.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.919+0000 7facc77fe700 1 -- 172.21.15.62:0/3991116892 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faccc01a910 con 0x7facd007dec0 2024-06-10T00:21:30.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.919+0000 7facd8293700 1 -- 172.21.15.62:0/3991116892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd019e8b0 con 0x7facd007dec0 2024-06-10T00:21:30.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.919+0000 7facc77fe700 1 -- 172.21.15.62:0/3991116892 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faccc024970 con 0x7facd007dec0 2024-06-10T00:21:30.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.919+0000 7facd8293700 1 -- 172.21.15.62:0/3991116892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd019ed10 con 0x7facd007dec0 2024-06-10T00:21:30.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.919+0000 7facc77fe700 1 -- 172.21.15.62:0/3991116892 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7faccc022020 con 0x7facd007dec0 2024-06-10T00:21:30.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.920+0000 7facd8293700 1 -- 172.21.15.62:0/3991116892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd0063af0 con 0x7facd007dec0 2024-06-10T00:21:30.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.920+0000 7facc77fe700 1 --2- 172.21.15.62:0/3991116892 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7facbc039140 0x7facbc03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:30.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.920+0000 7facc77fe700 1 -- 172.21.15.62:0/3991116892 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faccc04db60 con 0x7facd007dec0 2024-06-10T00:21:30.719 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.920+0000 7facd582e700 1 --2- 172.21.15.62:0/3991116892 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7facbc039140 0x7facbc03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:30.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.920+0000 7facd582e700 1 --2- 172.21.15.62:0/3991116892 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7facbc039140 0x7facbc03b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7facc0006fd0 tx=0x7facc0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:30.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:29.925+0000 7facc77fe700 1 -- 172.21.15.62:0/3991116892 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faccc00b3e0 con 0x7facd007dec0 2024-06-10T00:21:30.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:30.247+0000 7facd8293700 1 -- 172.21.15.62:0/3991116892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7facd01a16e0 con 0x7facd007dec0 2024-06-10T00:21:30.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:30.249+0000 7facc77fe700 1 -- 172.21.15.62:0/3991116892 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7faccc02cda0 con 0x7facd007dec0 2024-06-10T00:21:30.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:30.251+0000 7facd8293700 1 -- 172.21.15.62:0/3991116892 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7facbc039140 msgr2=0x7facbc03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:30.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:30.251+0000 7facd8293700 1 --2- 172.21.15.62:0/3991116892 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7facbc039140 0x7facbc03b600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7facc0006fd0 tx=0x7facc0006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:30.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:30.251+0000 7facd8293700 1 -- 172.21.15.62:0/3991116892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facd007dec0 msgr2=0x7facd019e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:30.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:30.251+0000 7facd8293700 1 --2- 172.21.15.62:0/3991116892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facd007dec0 0x7facd019e0e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faccc0117e0 tx=0x7faccc011ba0 comp rx=0 tx=0).stop 2024-06-10T00:21:30.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:30.252+0000 7facd8293700 1 -- 172.21.15.62:0/3991116892 shutdown_connections 2024-06-10T00:21:30.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:30.252+0000 7facd8293700 1 --2- 172.21.15.62:0/3991116892 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7facbc039140 0x7facbc03b600 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:30.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:30.252+0000 7facd8293700 1 --2- 172.21.15.62:0/3991116892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7facd007dec0 0x7facd019e0e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:30.720 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:30.252+0000 7facd8293700 1 -- 172.21.15.62:0/3991116892 >> 172.21.15.62:0/3991116892 conn(0x7facd01035b0 msgr2=0x7facd0104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:30.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:30.252+0000 7facd8293700 1 -- 172.21.15.62:0/3991116892 shutdown_connections 2024-06-10T00:21:30.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-06-10T00:21:30.252+0000 7facd8293700 1 -- 172.21.15.62:0/3991116892 wait complete. 2024-06-10T00:21:30.721 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-10T00:21:30.721 INFO:teuthology.orchestra.run.smithi062.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-10T00:21:30.721 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:21:30.721 INFO:teuthology.orchestra.run.smithi062.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:21:30.721 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:21:30.721 INFO:teuthology.orchestra.run.smithi062.stdout:Or, if you are only running a single cluster on this host: 2024-06-10T00:21:30.721 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:21:30.721 INFO:teuthology.orchestra.run.smithi062.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-10T00:21:30.721 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:21:30.721 INFO:teuthology.orchestra.run.smithi062.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-10T00:21:30.721 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:21:30.721 INFO:teuthology.orchestra.run.smithi062.stdout: ceph telemetry on 2024-06-10T00:21:30.722 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:21:30.722 INFO:teuthology.orchestra.run.smithi062.stdout:For more information see: 2024-06-10T00:21:30.722 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:21:30.722 INFO:teuthology.orchestra.run.smithi062.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-10T00:21:30.722 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:21:30.722 INFO:teuthology.orchestra.run.smithi062.stdout:Bootstrap complete. 2024-06-10T00:21:30.756 INFO:tasks.cephadm:Fetching config... 2024-06-10T00:21:30.757 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:21:30.757 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-10T00:21:30.775 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-10T00:21:30.775 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:21:30.775 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-10T00:21:30.831 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-10T00:21:30.831 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:21:30.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-10T00:21:30.896 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-10T00:21:30.896 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:21:30.896 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-10T00:21:30.953 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-10T00:21:30.953 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf18ETenDQ7sSymSPLNFGCqg6RtRIuHL22LKSheOdWrE+5w1t9VlPAlXDN63m5fdulnjPdiWmDybP/nYQfDlYAc49Y3jBCStc5P5wr/aiNMhZTHhJAntDcFX/xq5HG1l8DA2z2S5ofQ/vq2jrP6n1gMHIXGvGd8ROmw5wiwvoJv/Qknvl6AT7xsbJhgIGC7c63k8LP4uaDnoZ1IWASZdJkdA0QcSjrgs3BGUKpagx2QPA/3fXev/X6oZAhmoO6dX+iCdXVPtEbSoS1F0VVIVUBGppaOzBfEYmvs5uxMAtLYHMqxy1hHkQdIX3CQ0dp703uXUkFprva9x1PAfmc5ZYQwd0ApalduqYuj/tH7bCf7w0w9NJkotMS0Huucqo/tfyQI+GIo1PripKBw/JRZ9U0Bqv7kAvnBz4fE4BFH1pT2WsuHNXSufZPJGegI4UoKYEsqXql3yYIXOwCZFzpyzOU9llMlavOzzRLuYadBBpZxgP/ChCOVpteYG5mnTEp+eM= ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-10T00:21:31.035 INFO:teuthology.orchestra.run.smithi062.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf18ETenDQ7sSymSPLNFGCqg6RtRIuHL22LKSheOdWrE+5w1t9VlPAlXDN63m5fdulnjPdiWmDybP/nYQfDlYAc49Y3jBCStc5P5wr/aiNMhZTHhJAntDcFX/xq5HG1l8DA2z2S5ofQ/vq2jrP6n1gMHIXGvGd8ROmw5wiwvoJv/Qknvl6AT7xsbJhgIGC7c63k8LP4uaDnoZ1IWASZdJkdA0QcSjrgs3BGUKpagx2QPA/3fXev/X6oZAhmoO6dX+iCdXVPtEbSoS1F0VVIVUBGppaOzBfEYmvs5uxMAtLYHMqxy1hHkQdIX3CQ0dp703uXUkFprva9x1PAfmc5ZYQwd0ApalduqYuj/tH7bCf7w0w9NJkotMS0Huucqo/tfyQI+GIo1PripKBw/JRZ9U0Bqv7kAvnBz4fE4BFH1pT2WsuHNXSufZPJGegI4UoKYEsqXql3yYIXOwCZFzpyzOU9llMlavOzzRLuYadBBpZxgP/ChCOVpteYG5mnTEp+eM= ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:21:31.047 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf18ETenDQ7sSymSPLNFGCqg6RtRIuHL22LKSheOdWrE+5w1t9VlPAlXDN63m5fdulnjPdiWmDybP/nYQfDlYAc49Y3jBCStc5P5wr/aiNMhZTHhJAntDcFX/xq5HG1l8DA2z2S5ofQ/vq2jrP6n1gMHIXGvGd8ROmw5wiwvoJv/Qknvl6AT7xsbJhgIGC7c63k8LP4uaDnoZ1IWASZdJkdA0QcSjrgs3BGUKpagx2QPA/3fXev/X6oZAhmoO6dX+iCdXVPtEbSoS1F0VVIVUBGppaOzBfEYmvs5uxMAtLYHMqxy1hHkQdIX3CQ0dp703uXUkFprva9x1PAfmc5ZYQwd0ApalduqYuj/tH7bCf7w0w9NJkotMS0Huucqo/tfyQI+GIo1PripKBw/JRZ9U0Bqv7kAvnBz4fE4BFH1pT2WsuHNXSufZPJGegI4UoKYEsqXql3yYIXOwCZFzpyzOU9llMlavOzzRLuYadBBpZxgP/ChCOVpteYG5mnTEp+eM= ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-10T00:21:31.085 INFO:teuthology.orchestra.run.smithi105.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf18ETenDQ7sSymSPLNFGCqg6RtRIuHL22LKSheOdWrE+5w1t9VlPAlXDN63m5fdulnjPdiWmDybP/nYQfDlYAc49Y3jBCStc5P5wr/aiNMhZTHhJAntDcFX/xq5HG1l8DA2z2S5ofQ/vq2jrP6n1gMHIXGvGd8ROmw5wiwvoJv/Qknvl6AT7xsbJhgIGC7c63k8LP4uaDnoZ1IWASZdJkdA0QcSjrgs3BGUKpagx2QPA/3fXev/X6oZAhmoO6dX+iCdXVPtEbSoS1F0VVIVUBGppaOzBfEYmvs5uxMAtLYHMqxy1hHkQdIX3CQ0dp703uXUkFprva9x1PAfmc5ZYQwd0ApalduqYuj/tH7bCf7w0w9NJkotMS0Huucqo/tfyQI+GIo1PripKBw/JRZ9U0Bqv7kAvnBz4fE4BFH1pT2WsuHNXSufZPJGegI4UoKYEsqXql3yYIXOwCZFzpyzOU9llMlavOzzRLuYadBBpZxgP/ChCOVpteYG5mnTEp+eM= ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:21:31.097 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf18ETenDQ7sSymSPLNFGCqg6RtRIuHL22LKSheOdWrE+5w1t9VlPAlXDN63m5fdulnjPdiWmDybP/nYQfDlYAc49Y3jBCStc5P5wr/aiNMhZTHhJAntDcFX/xq5HG1l8DA2z2S5ofQ/vq2jrP6n1gMHIXGvGd8ROmw5wiwvoJv/Qknvl6AT7xsbJhgIGC7c63k8LP4uaDnoZ1IWASZdJkdA0QcSjrgs3BGUKpagx2QPA/3fXev/X6oZAhmoO6dX+iCdXVPtEbSoS1F0VVIVUBGppaOzBfEYmvs5uxMAtLYHMqxy1hHkQdIX3CQ0dp703uXUkFprva9x1PAfmc5ZYQwd0ApalduqYuj/tH7bCf7w0w9NJkotMS0Huucqo/tfyQI+GIo1PripKBw/JRZ9U0Bqv7kAvnBz4fE4BFH1pT2WsuHNXSufZPJGegI4UoKYEsqXql3yYIXOwCZFzpyzOU9llMlavOzzRLuYadBBpZxgP/ChCOVpteYG5mnTEp+eM= ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-10T00:21:31.137 INFO:teuthology.orchestra.run.smithi138.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDf18ETenDQ7sSymSPLNFGCqg6RtRIuHL22LKSheOdWrE+5w1t9VlPAlXDN63m5fdulnjPdiWmDybP/nYQfDlYAc49Y3jBCStc5P5wr/aiNMhZTHhJAntDcFX/xq5HG1l8DA2z2S5ofQ/vq2jrP6n1gMHIXGvGd8ROmw5wiwvoJv/Qknvl6AT7xsbJhgIGC7c63k8LP4uaDnoZ1IWASZdJkdA0QcSjrgs3BGUKpagx2QPA/3fXev/X6oZAhmoO6dX+iCdXVPtEbSoS1F0VVIVUBGppaOzBfEYmvs5uxMAtLYHMqxy1hHkQdIX3CQ0dp703uXUkFprva9x1PAfmc5ZYQwd0ApalduqYuj/tH7bCf7w0w9NJkotMS0Huucqo/tfyQI+GIo1PripKBw/JRZ9U0Bqv7kAvnBz4fE4BFH1pT2WsuHNXSufZPJGegI4UoKYEsqXql3yYIXOwCZFzpyzOU9llMlavOzzRLuYadBBpZxgP/ChCOVpteYG5mnTEp+eM= ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:21:31.148 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-10T00:21:31.272 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:31 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3991116892' entity='client.admin' 2024-06-10T00:21:31.397 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:21:33.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.134+0000 7fcea986f700 1 -- 172.21.15.62:0/1121125704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcea4100df0 msgr2=0x7fcea4101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:33.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.134+0000 7fcea986f700 1 --2- 172.21.15.62:0/1121125704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcea4100df0 0x7fcea4101210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009910 tx=0x7fce8c009c20 comp rx=0 tx=0).stop 2024-06-10T00:21:33.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.134+0000 7fcea986f700 1 -- 172.21.15.62:0/1121125704 shutdown_connections 2024-06-10T00:21:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.134+0000 7fcea986f700 1 --2- 172.21.15.62:0/1121125704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcea4100df0 0x7fcea4101210 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.134+0000 7fcea986f700 1 -- 172.21.15.62:0/1121125704 >> 172.21.15.62:0/1121125704 conn(0x7fcea40fc390 msgr2=0x7fcea40fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.134+0000 7fcea986f700 1 -- 172.21.15.62:0/1121125704 shutdown_connections 2024-06-10T00:21:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.134+0000 7fcea986f700 1 -- 172.21.15.62:0/1121125704 wait complete. 2024-06-10T00:21:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.135+0000 7fcea986f700 1 Processor -- start 2024-06-10T00:21:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.135+0000 7fcea986f700 1 -- start start 2024-06-10T00:21:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.135+0000 7fcea986f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcea4100df0 0x7fcea40fe250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.135+0000 7fcea986f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4106e80 con 0x7fcea4100df0 2024-06-10T00:21:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.135+0000 7fcea2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcea4100df0 0x7fcea40fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.135+0000 7fcea2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcea4100df0 0x7fcea40fe250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58858/0 (socket says 172.21.15.62:58858) 2024-06-10T00:21:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.135+0000 7fcea2ffd700 1 -- 172.21.15.62:0/2129456718 learned_addr learned my addr 172.21.15.62:0/2129456718 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.136+0000 7fcea2ffd700 1 -- 172.21.15.62:0/2129456718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea40fe790 con 0x7fcea4100df0 2024-06-10T00:21:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.136+0000 7fcea2ffd700 1 --2- 172.21.15.62:0/2129456718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcea4100df0 0x7fcea40fe250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce8c005850 tx=0x7fce8c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.136+0000 7fcea886d700 1 -- 172.21.15.62:0/2129456718 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce8c026070 con 0x7fcea4100df0 2024-06-10T00:21:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.136+0000 7fcea886d700 1 -- 172.21.15.62:0/2129456718 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce8c0044a0 con 0x7fcea4100df0 2024-06-10T00:21:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.136+0000 7fcea986f700 1 -- 172.21.15.62:0/2129456718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea40fc980 con 0x7fcea4100df0 2024-06-10T00:21:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.136+0000 7fcea886d700 1 -- 172.21.15.62:0/2129456718 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce8c003ab0 con 0x7fcea4100df0 2024-06-10T00:21:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.136+0000 7fcea986f700 1 -- 172.21.15.62:0/2129456718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea40fcec0 con 0x7fcea4100df0 2024-06-10T00:21:33.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.137+0000 7fcea886d700 1 -- 172.21.15.62:0/2129456718 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fce8c029070 con 0x7fcea4100df0 2024-06-10T00:21:33.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.137+0000 7fcea986f700 1 -- 172.21.15.62:0/2129456718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea4107600 con 0x7fcea4100df0 2024-06-10T00:21:33.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.137+0000 7fcea886d700 1 --2- 172.21.15.62:0/2129456718 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fce90039150 0x7fce9003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:33.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.137+0000 7fcea886d700 1 -- 172.21.15.62:0/2129456718 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fce8c01e070 con 0x7fcea4100df0 2024-06-10T00:21:33.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.138+0000 7fcea27fc700 1 --2- 172.21.15.62:0/2129456718 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fce90039150 0x7fce9003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:33.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.138+0000 7fcea27fc700 1 --2- 172.21.15.62:0/2129456718 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fce90039150 0x7fce9003b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fce94006fd0 tx=0x7fce94006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:33.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.142+0000 7fcea886d700 1 -- 172.21.15.62:0/2129456718 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fce8c013070 con 0x7fcea4100df0 2024-06-10T00:21:33.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.351+0000 7fcea986f700 1 -- 172.21.15.62:0/2129456718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fcea4190b80 con 0x7fcea4100df0 2024-06-10T00:21:33.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.354+0000 7fcea886d700 1 -- 172.21.15.62:0/2129456718 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fce8c030070 con 0x7fcea4100df0 2024-06-10T00:21:33.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.356+0000 7fcea986f700 1 -- 172.21.15.62:0/2129456718 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fce90039150 msgr2=0x7fce9003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:33.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.356+0000 7fcea986f700 1 --2- 172.21.15.62:0/2129456718 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fce90039150 0x7fce9003b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fce94006fd0 tx=0x7fce94006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:33.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.356+0000 7fcea986f700 1 -- 172.21.15.62:0/2129456718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcea4100df0 msgr2=0x7fcea40fe250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:33.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.356+0000 7fcea986f700 1 --2- 172.21.15.62:0/2129456718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcea4100df0 0x7fcea40fe250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce8c005850 tx=0x7fce8c005030 comp rx=0 tx=0).stop 2024-06-10T00:21:33.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.356+0000 7fcea986f700 1 -- 172.21.15.62:0/2129456718 shutdown_connections 2024-06-10T00:21:33.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.356+0000 7fcea986f700 1 --2- 172.21.15.62:0/2129456718 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fce90039150 0x7fce9003b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:33.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.356+0000 7fcea986f700 1 --2- 172.21.15.62:0/2129456718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcea4100df0 0x7fcea40fe250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:33.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.356+0000 7fcea986f700 1 -- 172.21.15.62:0/2129456718 >> 172.21.15.62:0/2129456718 conn(0x7fcea40fc390 msgr2=0x7fcea4069cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:33.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.357+0000 7fcea986f700 1 -- 172.21.15.62:0/2129456718 shutdown_connections 2024-06-10T00:21:33.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:33.357+0000 7fcea986f700 1 -- 172.21.15.62:0/2129456718 wait complete. 2024-06-10T00:21:34.531 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-10T00:21:34.531 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-10T00:21:34.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:34 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2129456718' entity='client.admin' 2024-06-10T00:21:34.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:34.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:34.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:21:34.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:34.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:21:34.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:21:36.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.268+0000 7f94016a1700 1 -- 172.21.15.62:0/2743958823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93fc100df0 msgr2=0x7f93fc101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:36.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.268+0000 7f94016a1700 1 --2- 172.21.15.62:0/2743958823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93fc100df0 0x7f93fc101210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009910 tx=0x7f93ec009c20 comp rx=0 tx=0).stop 2024-06-10T00:21:36.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.268+0000 7f94016a1700 1 -- 172.21.15.62:0/2743958823 shutdown_connections 2024-06-10T00:21:36.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.268+0000 7f94016a1700 1 --2- 172.21.15.62:0/2743958823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93fc100df0 0x7f93fc101210 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:36.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.268+0000 7f94016a1700 1 -- 172.21.15.62:0/2743958823 >> 172.21.15.62:0/2743958823 conn(0x7f93fc0fc390 msgr2=0x7f93fc0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:36.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.268+0000 7f94016a1700 1 -- 172.21.15.62:0/2743958823 shutdown_connections 2024-06-10T00:21:36.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.268+0000 7f94016a1700 1 -- 172.21.15.62:0/2743958823 wait complete. 2024-06-10T00:21:36.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.269+0000 7f94016a1700 1 Processor -- start 2024-06-10T00:21:36.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.269+0000 7f94016a1700 1 -- start start 2024-06-10T00:21:36.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.269+0000 7f94016a1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93fc100df0 0x7f93fc197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:36.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.269+0000 7f94016a1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc106e80 con 0x7f93fc100df0 2024-06-10T00:21:36.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.269+0000 7f93faffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93fc100df0 0x7f93fc197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:36.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.269+0000 7f93faffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93fc100df0 0x7f93fc197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58872/0 (socket says 172.21.15.62:58872) 2024-06-10T00:21:36.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.269+0000 7f93faffd700 1 -- 172.21.15.62:0/2119063697 learned_addr learned my addr 172.21.15.62:0/2119063697 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:36.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.269+0000 7f93faffd700 1 -- 172.21.15.62:0/2119063697 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93fc197b70 con 0x7f93fc100df0 2024-06-10T00:21:36.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.270+0000 7f93faffd700 1 --2- 172.21.15.62:0/2119063697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93fc100df0 0x7f93fc197630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f93ec006010 tx=0x7f93ec005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:36.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.270+0000 7f93e3fff700 1 -- 172.21.15.62:0/2119063697 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93ec027070 con 0x7f93fc100df0 2024-06-10T00:21:36.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.270+0000 7f93e3fff700 1 -- 172.21.15.62:0/2119063697 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93ec00be40 con 0x7f93fc100df0 2024-06-10T00:21:36.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.270+0000 7f94016a1700 1 -- 172.21.15.62:0/2119063697 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93fc197e00 con 0x7f93fc100df0 2024-06-10T00:21:36.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.270+0000 7f93e3fff700 1 -- 172.21.15.62:0/2119063697 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93ec00f460 con 0x7f93fc100df0 2024-06-10T00:21:36.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.270+0000 7f94016a1700 1 -- 172.21.15.62:0/2119063697 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93fc19aaf0 con 0x7f93fc100df0 2024-06-10T00:21:36.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.271+0000 7f93e3fff700 1 -- 172.21.15.62:0/2119063697 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f93ec02a070 con 0x7f93fc100df0 2024-06-10T00:21:36.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.271+0000 7f93e3fff700 1 --2- 172.21.15.62:0/2119063697 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93e4039150 0x7f93e403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:36.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.271+0000 7f93e3fff700 1 -- 172.21.15.62:0/2119063697 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f93ec01e030 con 0x7f93fc100df0 2024-06-10T00:21:36.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.271+0000 7f94016a1700 1 -- 172.21.15.62:0/2119063697 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93fc107600 con 0x7f93fc100df0 2024-06-10T00:21:36.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.271+0000 7f93fa7fc700 1 --2- 172.21.15.62:0/2119063697 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93e4039150 0x7f93e403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:36.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.271+0000 7f93fa7fc700 1 --2- 172.21.15.62:0/2119063697 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93e4039150 0x7f93e403b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f93f0006fd0 tx=0x7f93f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:36.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.276+0000 7f93e3fff700 1 -- 172.21.15.62:0/2119063697 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f93ec031070 con 0x7f93fc100df0 2024-06-10T00:21:36.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.489+0000 7f94016a1700 1 -- 172.21.15.62:0/2119063697 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f93fc062960 con 0x7f93e4039150 2024-06-10T00:21:36.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.492+0000 7f93e3fff700 1 -- 172.21.15.62:0/2119063697 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f93fc062960 con 0x7f93e4039150 2024-06-10T00:21:36.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.494+0000 7f94016a1700 1 -- 172.21.15.62:0/2119063697 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93e4039150 msgr2=0x7f93e403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:36.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.494+0000 7f94016a1700 1 --2- 172.21.15.62:0/2119063697 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93e4039150 0x7f93e403b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f93f0006fd0 tx=0x7f93f0006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:36.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.494+0000 7f94016a1700 1 -- 172.21.15.62:0/2119063697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93fc100df0 msgr2=0x7f93fc197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:36.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.494+0000 7f94016a1700 1 --2- 172.21.15.62:0/2119063697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93fc100df0 0x7f93fc197630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f93ec006010 tx=0x7f93ec005050 comp rx=0 tx=0).stop 2024-06-10T00:21:36.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.495+0000 7f94016a1700 1 -- 172.21.15.62:0/2119063697 shutdown_connections 2024-06-10T00:21:36.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.495+0000 7f94016a1700 1 --2- 172.21.15.62:0/2119063697 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93e4039150 0x7f93e403b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:36.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.495+0000 7f94016a1700 1 --2- 172.21.15.62:0/2119063697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93fc100df0 0x7f93fc197630 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:36.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.495+0000 7f94016a1700 1 -- 172.21.15.62:0/2119063697 >> 172.21.15.62:0/2119063697 conn(0x7f93fc0fc390 msgr2=0x7f93fc069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:36.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.495+0000 7f94016a1700 1 -- 172.21.15.62:0/2119063697 shutdown_connections 2024-06-10T00:21:36.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:36.495+0000 7f94016a1700 1 -- 172.21.15.62:0/2119063697 wait complete. 2024-06-10T00:21:37.351 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi105 2024-06-10T00:21:37.351 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-06-10T00:21:37.351 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/etc/ceph/ceph.conf 2024-06-10T00:21:37.371 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-06-10T00:21:37.371 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:21:37.428 INFO:tasks.cephadm:Adding host smithi105 to orchestrator... 2024-06-10T00:21:37.428 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch host add smithi105 2024-06-10T00:21:37.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:37 smithi062 ceph-mon[32390]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:21:37.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:37 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:37.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:37 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:21:37.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:37 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:21:37.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:37 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:21:37.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:37 smithi062 ceph-mon[32390]: Updating smithi062:/etc/ceph/ceph.conf 2024-06-10T00:21:37.677 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:21:38.721 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:38 smithi062 ceph-mon[32390]: Updating smithi062:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:21:38.721 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:38 smithi062 ceph-mon[32390]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:21:38.721 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:38 smithi062 ceph-mon[32390]: Updating smithi062:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:21:38.721 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:38 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:38.721 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:38 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:38.721 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:38 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:39.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.195+0000 7f18a6afa700 1 -- 172.21.15.62:0/3864652505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a0100fe0 msgr2=0x7f18a01013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:39.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.195+0000 7f18a6afa700 1 --2- 172.21.15.62:0/3864652505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a0100fe0 0x7f18a01013c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1894009910 tx=0x7f1894009c20 comp rx=0 tx=0).stop 2024-06-10T00:21:39.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.195+0000 7f18a6afa700 1 -- 172.21.15.62:0/3864652505 shutdown_connections 2024-06-10T00:21:39.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.195+0000 7f18a6afa700 1 --2- 172.21.15.62:0/3864652505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a0100fe0 0x7f18a01013c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:39.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.195+0000 7f18a6afa700 1 -- 172.21.15.62:0/3864652505 >> 172.21.15.62:0/3864652505 conn(0x7f18a00fc890 msgr2=0x7f18a00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:39.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.195+0000 7f18a6afa700 1 -- 172.21.15.62:0/3864652505 shutdown_connections 2024-06-10T00:21:39.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.195+0000 7f18a6afa700 1 -- 172.21.15.62:0/3864652505 wait complete. 2024-06-10T00:21:39.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.195+0000 7f18a6afa700 1 Processor -- start 2024-06-10T00:21:39.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.195+0000 7f18a6afa700 1 -- start start 2024-06-10T00:21:39.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.196+0000 7f18a6afa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a0100fe0 0x7f18a0198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:39.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.196+0000 7f18a6afa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f18a010b3b0 con 0x7f18a0100fe0 2024-06-10T00:21:39.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.196+0000 7f18a4896700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a0100fe0 0x7f18a0198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:39.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.196+0000 7f18a4896700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a0100fe0 0x7f18a0198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53842/0 (socket says 172.21.15.62:53842) 2024-06-10T00:21:39.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.196+0000 7f18a4896700 1 -- 172.21.15.62:0/2555389528 learned_addr learned my addr 172.21.15.62:0/2555389528 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:39.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.196+0000 7f18a4896700 1 -- 172.21.15.62:0/2555389528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a0198d90 con 0x7f18a0100fe0 2024-06-10T00:21:39.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.196+0000 7f18a4896700 1 --2- 172.21.15.62:0/2555389528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a0100fe0 0x7f18a0198850 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1894005850 tx=0x7f1894005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:39.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.196+0000 7f189dffb700 1 -- 172.21.15.62:0/2555389528 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1894026070 con 0x7f18a0100fe0 2024-06-10T00:21:39.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.196+0000 7f18a6afa700 1 -- 172.21.15.62:0/2555389528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a0199080 con 0x7f18a0100fe0 2024-06-10T00:21:39.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.196+0000 7f189dffb700 1 -- 172.21.15.62:0/2555389528 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f189400bbd0 con 0x7f18a0100fe0 2024-06-10T00:21:39.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.196+0000 7f189dffb700 1 -- 172.21.15.62:0/2555389528 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1894003970 con 0x7f18a0100fe0 2024-06-10T00:21:39.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.197+0000 7f18a6afa700 1 -- 172.21.15.62:0/2555389528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a0199540 con 0x7f18a0100fe0 2024-06-10T00:21:39.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.198+0000 7f189dffb700 1 -- 172.21.15.62:0/2555389528 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1894029070 con 0x7f18a0100fe0 2024-06-10T00:21:39.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.198+0000 7f18a6afa700 1 -- 172.21.15.62:0/2555389528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a01024c0 con 0x7f18a0100fe0 2024-06-10T00:21:39.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.198+0000 7f189dffb700 1 --2- 172.21.15.62:0/2555389528 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f18900390f0 0x7f189003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:39.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.198+0000 7f189dffb700 1 -- 172.21.15.62:0/2555389528 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f189401e070 con 0x7f18a0100fe0 2024-06-10T00:21:39.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.198+0000 7f189ffff700 1 --2- 172.21.15.62:0/2555389528 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f18900390f0 0x7f189003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:39.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.198+0000 7f189ffff700 1 --2- 172.21.15.62:0/2555389528 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f18900390f0 0x7f189003b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f188c006fd0 tx=0x7f188c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:39.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.203+0000 7f189dffb700 1 -- 172.21.15.62:0/2555389528 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1894013070 con 0x7f18a0100fe0 2024-06-10T00:21:39.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:39.414+0000 7f18a6afa700 1 -- 172.21.15.62:0/2555389528 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi105", "target": ["mon-mgr", ""]}) -- 0x7f18a006a730 con 0x7f18900390f0 2024-06-10T00:21:40.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:39 smithi062 ceph-mon[32390]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi105", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:21:41.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:40 smithi062 ceph-mon[32390]: Deploying cephadm binary to smithi105 2024-06-10T00:21:43.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:42 smithi062 ceph-mon[32390]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:21:43.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:43.651+0000 7f189dffb700 1 -- 172.21.15.62:0/2555389528 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f18a006a730 con 0x7f18900390f0 2024-06-10T00:21:43.651 INFO:teuthology.orchestra.run.smithi062.stdout:Added host 'smithi105' with addr '172.21.15.105' 2024-06-10T00:21:43.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:43.654+0000 7f18a6afa700 1 -- 172.21.15.62:0/2555389528 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f18900390f0 msgr2=0x7f189003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:43.654+0000 7f18a6afa700 1 --2- 172.21.15.62:0/2555389528 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f18900390f0 0x7f189003b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f188c006fd0 tx=0x7f188c006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:43.654+0000 7f18a6afa700 1 -- 172.21.15.62:0/2555389528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a0100fe0 msgr2=0x7f18a0198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:43.654+0000 7f18a6afa700 1 --2- 172.21.15.62:0/2555389528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a0100fe0 0x7f18a0198850 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1894005850 tx=0x7f1894005030 comp rx=0 tx=0).stop 2024-06-10T00:21:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:43.654+0000 7f18a6afa700 1 -- 172.21.15.62:0/2555389528 shutdown_connections 2024-06-10T00:21:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:43.654+0000 7f18a6afa700 1 --2- 172.21.15.62:0/2555389528 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f18900390f0 0x7f189003b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:43.654+0000 7f18a6afa700 1 --2- 172.21.15.62:0/2555389528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18a0100fe0 0x7f18a0198850 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:43.654+0000 7f18a6afa700 1 -- 172.21.15.62:0/2555389528 >> 172.21.15.62:0/2555389528 conn(0x7f18a00fc890 msgr2=0x7f18a01029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:43.654+0000 7f18a6afa700 1 -- 172.21.15.62:0/2555389528 shutdown_connections 2024-06-10T00:21:43.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:43.654+0000 7f18a6afa700 1 -- 172.21.15.62:0/2555389528 wait complete. 2024-06-10T00:21:44.257 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-10T00:21:44.503 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:21:44.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:44 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:44.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:44 smithi062 ceph-mon[32390]: Added host smithi105 2024-06-10T00:21:44.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:44 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:21:44.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:44 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:45.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:45 smithi062 ceph-mon[32390]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:21:45.807 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:46.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.045+0000 7fef9c2c0700 1 -- 172.21.15.62:0/78246682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef94106e00 msgr2=0x7fef941071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:46.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.045+0000 7fef9c2c0700 1 --2- 172.21.15.62:0/78246682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef94106e00 0x7fef941071e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fef90009910 tx=0x7fef90009c20 comp rx=0 tx=0).stop 2024-06-10T00:21:46.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.045+0000 7fef9c2c0700 1 -- 172.21.15.62:0/78246682 shutdown_connections 2024-06-10T00:21:46.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.045+0000 7fef9c2c0700 1 --2- 172.21.15.62:0/78246682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef94106e00 0x7fef941071e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:46.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.045+0000 7fef9c2c0700 1 -- 172.21.15.62:0/78246682 >> 172.21.15.62:0/78246682 conn(0x7fef940768e0 msgr2=0x7fef94076cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:46.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.045+0000 7fef9c2c0700 1 -- 172.21.15.62:0/78246682 shutdown_connections 2024-06-10T00:21:46.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.045+0000 7fef9c2c0700 1 -- 172.21.15.62:0/78246682 wait complete. 2024-06-10T00:21:46.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.046+0000 7fef9c2c0700 1 Processor -- start 2024-06-10T00:21:46.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.046+0000 7fef9c2c0700 1 -- start start 2024-06-10T00:21:46.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.046+0000 7fef9c2c0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef94106e00 0x7fef9419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:46.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.046+0000 7fef9c2c0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fef9410b390 con 0x7fef94106e00 2024-06-10T00:21:46.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.046+0000 7fef9a05c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef94106e00 0x7fef9419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:46.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.046+0000 7fef9a05c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef94106e00 0x7fef9419c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53874/0 (socket says 172.21.15.62:53874) 2024-06-10T00:21:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.046+0000 7fef9a05c700 1 -- 172.21.15.62:0/1305019352 learned_addr learned my addr 172.21.15.62:0/1305019352 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.047+0000 7fef9a05c700 1 -- 172.21.15.62:0/1305019352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9419cb00 con 0x7fef94106e00 2024-06-10T00:21:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.047+0000 7fef9a05c700 1 --2- 172.21.15.62:0/1305019352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef94106e00 0x7fef9419c5c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef90005850 tx=0x7fef90005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.047+0000 7fef8b7fe700 1 -- 172.21.15.62:0/1305019352 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef90026070 con 0x7fef94106e00 2024-06-10T00:21:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.047+0000 7fef8b7fe700 1 -- 172.21.15.62:0/1305019352 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef9000bbd0 con 0x7fef94106e00 2024-06-10T00:21:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.047+0000 7fef9c2c0700 1 -- 172.21.15.62:0/1305019352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9419cdf0 con 0x7fef94106e00 2024-06-10T00:21:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.047+0000 7fef8b7fe700 1 -- 172.21.15.62:0/1305019352 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef90003970 con 0x7fef94106e00 2024-06-10T00:21:46.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.047+0000 7fef9c2c0700 1 -- 172.21.15.62:0/1305019352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef941968b0 con 0x7fef94106e00 2024-06-10T00:21:46.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.048+0000 7fef8b7fe700 1 -- 172.21.15.62:0/1305019352 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fef90029070 con 0x7fef94106e00 2024-06-10T00:21:46.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.048+0000 7fef9c2c0700 1 -- 172.21.15.62:0/1305019352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef941024b0 con 0x7fef94106e00 2024-06-10T00:21:46.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.048+0000 7fef8b7fe700 1 --2- 172.21.15.62:0/1305019352 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fef800390f0 0x7fef8003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:46.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.048+0000 7fef8b7fe700 1 -- 172.21.15.62:0/1305019352 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fef9001e070 con 0x7fef94106e00 2024-06-10T00:21:46.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.048+0000 7fef9985b700 1 --2- 172.21.15.62:0/1305019352 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fef800390f0 0x7fef8003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:46.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.049+0000 7fef9985b700 1 --2- 172.21.15.62:0/1305019352 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fef800390f0 0x7fef8003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef84006fd0 tx=0x7fef84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:46.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.053+0000 7fef8b7fe700 1 -- 172.21.15.62:0/1305019352 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fef90013070 con 0x7fef94106e00 2024-06-10T00:21:46.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.266+0000 7fef9c2c0700 1 -- 172.21.15.62:0/1305019352 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef94197bf0 con 0x7fef800390f0 2024-06-10T00:21:46.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.268+0000 7fef8b7fe700 1 -- 172.21.15.62:0/1305019352 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fef94197bf0 con 0x7fef800390f0 2024-06-10T00:21:46.267 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:21:46.268 INFO:teuthology.orchestra.run.smithi062.stdout:[{"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.105", "hostname": "smithi105", "labels": [], "status": ""}] 2024-06-10T00:21:46.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.270+0000 7fef9c2c0700 1 -- 172.21.15.62:0/1305019352 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fef800390f0 msgr2=0x7fef8003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:46.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.270+0000 7fef9c2c0700 1 --2- 172.21.15.62:0/1305019352 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fef800390f0 0x7fef8003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef84006fd0 tx=0x7fef84006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:46.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.270+0000 7fef9c2c0700 1 -- 172.21.15.62:0/1305019352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef94106e00 msgr2=0x7fef9419c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:46.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.270+0000 7fef9c2c0700 1 --2- 172.21.15.62:0/1305019352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef94106e00 0x7fef9419c5c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef90005850 tx=0x7fef90005030 comp rx=0 tx=0).stop 2024-06-10T00:21:46.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.270+0000 7fef9c2c0700 1 -- 172.21.15.62:0/1305019352 shutdown_connections 2024-06-10T00:21:46.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.270+0000 7fef9c2c0700 1 --2- 172.21.15.62:0/1305019352 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fef800390f0 0x7fef8003b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:46.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.270+0000 7fef9c2c0700 1 --2- 172.21.15.62:0/1305019352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef94106e00 0x7fef9419c5c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:46.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.270+0000 7fef9c2c0700 1 -- 172.21.15.62:0/1305019352 >> 172.21.15.62:0/1305019352 conn(0x7fef940768e0 msgr2=0x7fef940fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:46.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.270+0000 7fef9c2c0700 1 -- 172.21.15.62:0/1305019352 shutdown_connections 2024-06-10T00:21:46.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:46.270+0000 7fef9c2c0700 1 -- 172.21.15.62:0/1305019352 wait complete. 2024-06-10T00:21:46.825 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:46 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:46.853 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi138 2024-06-10T00:21:46.853 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-06-10T00:21:46.853 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.conf 2024-06-10T00:21:46.872 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-06-10T00:21:46.873 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:21:46.930 INFO:tasks.cephadm:Adding host smithi138 to orchestrator... 2024-06-10T00:21:46.931 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch host add smithi138 2024-06-10T00:21:47.174 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:21:47.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:47 smithi062 ceph-mon[32390]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:21:47.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:47 smithi062 ceph-mon[32390]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:21:48.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.714+0000 7fdcc4795700 1 -- 172.21.15.62:0/1759035057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc101040 msgr2=0x7fdcbc101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:48.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.714+0000 7fdcc4795700 1 --2- 172.21.15.62:0/1759035057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc101040 0x7fdcbc101420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009910 tx=0x7fdcb8009c20 comp rx=0 tx=0).stop 2024-06-10T00:21:48.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.714+0000 7fdcc4795700 1 -- 172.21.15.62:0/1759035057 shutdown_connections 2024-06-10T00:21:48.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.714+0000 7fdcc4795700 1 --2- 172.21.15.62:0/1759035057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc101040 0x7fdcbc101420 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:48.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.714+0000 7fdcc4795700 1 -- 172.21.15.62:0/1759035057 >> 172.21.15.62:0/1759035057 conn(0x7fdcbc0768f0 msgr2=0x7fdcbc076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:48.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.715+0000 7fdcc4795700 1 -- 172.21.15.62:0/1759035057 shutdown_connections 2024-06-10T00:21:48.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.715+0000 7fdcc4795700 1 -- 172.21.15.62:0/1759035057 wait complete. 2024-06-10T00:21:48.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.715+0000 7fdcc4795700 1 Processor -- start 2024-06-10T00:21:48.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.715+0000 7fdcc4795700 1 -- start start 2024-06-10T00:21:48.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.715+0000 7fdcc4795700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc101040 0x7fdcbc19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:48.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.715+0000 7fdcc4795700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc10b3c0 con 0x7fdcbc101040 2024-06-10T00:21:48.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.716+0000 7fdcc2531700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc101040 0x7fdcbc19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:48.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.716+0000 7fdcc2531700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc101040 0x7fdcbc19f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33110/0 (socket says 172.21.15.62:33110) 2024-06-10T00:21:48.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.716+0000 7fdcc2531700 1 -- 172.21.15.62:0/2665538999 learned_addr learned my addr 172.21.15.62:0/2665538999 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:48.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.716+0000 7fdcc2531700 1 -- 172.21.15.62:0/2665538999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc19fd80 con 0x7fdcbc101040 2024-06-10T00:21:48.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.716+0000 7fdcc2531700 1 --2- 172.21.15.62:0/2665538999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc101040 0x7fdcbc19f840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8006010 tx=0x7fdcb8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:48.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.716+0000 7fdcb37fe700 1 -- 172.21.15.62:0/2665538999 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcb8026070 con 0x7fdcbc101040 2024-06-10T00:21:48.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.716+0000 7fdcb37fe700 1 -- 172.21.15.62:0/2665538999 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb800bbd0 con 0x7fdcbc101040 2024-06-10T00:21:48.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.716+0000 7fdcc4795700 1 -- 172.21.15.62:0/2665538999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcbc1a0070 con 0x7fdcbc101040 2024-06-10T00:21:48.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.716+0000 7fdcb37fe700 1 -- 172.21.15.62:0/2665538999 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcb8003910 con 0x7fdcbc101040 2024-06-10T00:21:48.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.716+0000 7fdcc4795700 1 -- 172.21.15.62:0/2665538999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcbc199a00 con 0x7fdcbc101040 2024-06-10T00:21:48.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.717+0000 7fdcb37fe700 1 -- 172.21.15.62:0/2665538999 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fdcb8029070 con 0x7fdcbc101040 2024-06-10T00:21:48.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.717+0000 7fdcb37fe700 1 --2- 172.21.15.62:0/2665538999 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdca80390f0 0x7fdca803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:48.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.717+0000 7fdcc4795700 1 -- 172.21.15.62:0/2665538999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcbc1024d0 con 0x7fdcbc101040 2024-06-10T00:21:48.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.717+0000 7fdcb37fe700 1 -- 172.21.15.62:0/2665538999 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdcb801e070 con 0x7fdcbc101040 2024-06-10T00:21:48.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.717+0000 7fdcc1d30700 1 --2- 172.21.15.62:0/2665538999 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdca80390f0 0x7fdca803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:48.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.718+0000 7fdcc1d30700 1 --2- 172.21.15.62:0/2665538999 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdca80390f0 0x7fdca803b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdcac006fd0 tx=0x7fdcac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:48.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.723+0000 7fdcb37fe700 1 -- 172.21.15.62:0/2665538999 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdcb8013070 con 0x7fdcbc101040 2024-06-10T00:21:48.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:48.935+0000 7fdcc4795700 1 -- 172.21.15.62:0/2665538999 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}) -- 0x7fdcbc19afa0 con 0x7fdca80390f0 2024-06-10T00:21:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:49 smithi062 ceph-mon[32390]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:21:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:50.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:50.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:50.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi105", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:21:50.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:21:50.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:21:51.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:50 smithi062 ceph-mon[32390]: Updating smithi105:/etc/ceph/ceph.conf 2024-06-10T00:21:51.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:50 smithi062 ceph-mon[32390]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:21:51.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:50 smithi062 ceph-mon[32390]: Updating smithi105:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:21:51.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:50 smithi062 ceph-mon[32390]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:21:51.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:50 smithi062 ceph-mon[32390]: Deploying cephadm binary to smithi138 2024-06-10T00:21:51.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:50 smithi062 ceph-mon[32390]: Updating smithi105:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:21:51.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:50 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:51.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:50 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:51.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:50 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:52.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:51 smithi062 ceph-mon[32390]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:21:53.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:53.144+0000 7fdcb37fe700 1 -- 172.21.15.62:0/2665538999 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fdcbc19afa0 con 0x7fdca80390f0 2024-06-10T00:21:53.144 INFO:teuthology.orchestra.run.smithi062.stdout:Added host 'smithi138' with addr '172.21.15.138' 2024-06-10T00:21:53.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:53.147+0000 7fdcc4795700 1 -- 172.21.15.62:0/2665538999 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdca80390f0 msgr2=0x7fdca803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:53.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:53.147+0000 7fdcc4795700 1 --2- 172.21.15.62:0/2665538999 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdca80390f0 0x7fdca803b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdcac006fd0 tx=0x7fdcac006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:53.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:53.147+0000 7fdcc4795700 1 -- 172.21.15.62:0/2665538999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc101040 msgr2=0x7fdcbc19f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:53.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:53.147+0000 7fdcc4795700 1 --2- 172.21.15.62:0/2665538999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc101040 0x7fdcbc19f840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8006010 tx=0x7fdcb8005030 comp rx=0 tx=0).stop 2024-06-10T00:21:53.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:53.147+0000 7fdcc4795700 1 -- 172.21.15.62:0/2665538999 shutdown_connections 2024-06-10T00:21:53.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:53.147+0000 7fdcc4795700 1 --2- 172.21.15.62:0/2665538999 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdca80390f0 0x7fdca803b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:53.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:53.147+0000 7fdcc4795700 1 --2- 172.21.15.62:0/2665538999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcbc101040 0x7fdcbc19f840 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:53.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:53.147+0000 7fdcc4795700 1 -- 172.21.15.62:0/2665538999 >> 172.21.15.62:0/2665538999 conn(0x7fdcbc0768f0 msgr2=0x7fdcbc0698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:53.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:53.147+0000 7fdcc4795700 1 -- 172.21.15.62:0/2665538999 shutdown_connections 2024-06-10T00:21:53.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:53.147+0000 7fdcc4795700 1 -- 172.21.15.62:0/2665538999 wait complete. 2024-06-10T00:21:53.953 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:53 smithi062 ceph-mon[32390]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:21:53.953 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:53.953 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:21:53.953 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:53.987 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-10T00:21:54.237 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:21:54.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:54 smithi062 ceph-mon[32390]: Added host smithi138 2024-06-10T00:21:54.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:54 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.724+0000 7fe96b7ee700 1 -- 172.21.15.62:0/3665736368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe964101040 msgr2=0x7fe964101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.724+0000 7fe96b7ee700 1 --2- 172.21.15.62:0/3665736368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe964101040 0x7fe964101420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe95400b0a0 tx=0x7fe95400b3b0 comp rx=0 tx=0).stop 2024-06-10T00:21:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.724+0000 7fe96b7ee700 1 -- 172.21.15.62:0/3665736368 shutdown_connections 2024-06-10T00:21:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.724+0000 7fe96b7ee700 1 --2- 172.21.15.62:0/3665736368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe964101040 0x7fe964101420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.724+0000 7fe96b7ee700 1 -- 172.21.15.62:0/3665736368 >> 172.21.15.62:0/3665736368 conn(0x7fe9640768f0 msgr2=0x7fe964076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.725+0000 7fe96b7ee700 1 -- 172.21.15.62:0/3665736368 shutdown_connections 2024-06-10T00:21:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.725+0000 7fe96b7ee700 1 -- 172.21.15.62:0/3665736368 wait complete. 2024-06-10T00:21:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.725+0000 7fe96b7ee700 1 Processor -- start 2024-06-10T00:21:55.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.725+0000 7fe96b7ee700 1 -- start start 2024-06-10T00:21:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.725+0000 7fe96b7ee700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe964101040 0x7fe96419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.725+0000 7fe96b7ee700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe96410b3c0 con 0x7fe964101040 2024-06-10T00:21:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.726+0000 7fe96958a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe964101040 0x7fe96419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.726+0000 7fe96958a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe964101040 0x7fe96419d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33126/0 (socket says 172.21.15.62:33126) 2024-06-10T00:21:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.726+0000 7fe96958a700 1 -- 172.21.15.62:0/2342069421 learned_addr learned my addr 172.21.15.62:0/2342069421 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.726+0000 7fe96958a700 1 -- 172.21.15.62:0/2342069421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96419dbd0 con 0x7fe964101040 2024-06-10T00:21:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.726+0000 7fe96958a700 1 --2- 172.21.15.62:0/2342069421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe964101040 0x7fe96419d690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe954015040 tx=0x7fe954003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.726+0000 7fe9627fc700 1 -- 172.21.15.62:0/2342069421 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe954017040 con 0x7fe964101040 2024-06-10T00:21:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.726+0000 7fe9627fc700 1 -- 172.21.15.62:0/2342069421 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe954022070 con 0x7fe964101040 2024-06-10T00:21:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.726+0000 7fe96b7ee700 1 -- 172.21.15.62:0/2342069421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96419de60 con 0x7fe964101040 2024-06-10T00:21:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.726+0000 7fe9627fc700 1 -- 172.21.15.62:0/2342069421 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe95401d5d0 con 0x7fe964101040 2024-06-10T00:21:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.726+0000 7fe96b7ee700 1 -- 172.21.15.62:0/2342069421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe964197760 con 0x7fe964101040 2024-06-10T00:21:55.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.727+0000 7fe96b7ee700 1 -- 172.21.15.62:0/2342069421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9641024d0 con 0x7fe964101040 2024-06-10T00:21:55.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.727+0000 7fe9627fc700 1 -- 172.21.15.62:0/2342069421 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe95401d880 con 0x7fe964101040 2024-06-10T00:21:55.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.727+0000 7fe9627fc700 1 --2- 172.21.15.62:0/2342069421 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe94c039150 0x7fe94c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:55.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.728+0000 7fe9627fc700 1 -- 172.21.15.62:0/2342069421 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe95400f050 con 0x7fe964101040 2024-06-10T00:21:55.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.728+0000 7fe968d89700 1 --2- 172.21.15.62:0/2342069421 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe94c039150 0x7fe94c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:55.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.728+0000 7fe968d89700 1 --2- 172.21.15.62:0/2342069421 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe94c039150 0x7fe94c03b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe958006fd0 tx=0x7fe958006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:55.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.733+0000 7fe9627fc700 1 -- 172.21.15.62:0/2342069421 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe954007e20 con 0x7fe964101040 2024-06-10T00:21:55.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:55 smithi062 ceph-mon[32390]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:21:55.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.946+0000 7fe96b7ee700 1 -- 172.21.15.62:0/2342069421 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe964198c90 con 0x7fe94c039150 2024-06-10T00:21:55.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.947+0000 7fe9627fc700 1 -- 172.21.15.62:0/2342069421 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fe964198c90 con 0x7fe94c039150 2024-06-10T00:21:55.946 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:21:55.947 INFO:teuthology.orchestra.run.smithi062.stdout:[{"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.105", "hostname": "smithi105", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}] 2024-06-10T00:21:55.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.949+0000 7fe96b7ee700 1 -- 172.21.15.62:0/2342069421 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe94c039150 msgr2=0x7fe94c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:55.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.949+0000 7fe96b7ee700 1 --2- 172.21.15.62:0/2342069421 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe94c039150 0x7fe94c03b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe958006fd0 tx=0x7fe958006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:55.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.949+0000 7fe96b7ee700 1 -- 172.21.15.62:0/2342069421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe964101040 msgr2=0x7fe96419d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:55.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.949+0000 7fe96b7ee700 1 --2- 172.21.15.62:0/2342069421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe964101040 0x7fe96419d690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe954015040 tx=0x7fe954003d50 comp rx=0 tx=0).stop 2024-06-10T00:21:55.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.949+0000 7fe96b7ee700 1 -- 172.21.15.62:0/2342069421 shutdown_connections 2024-06-10T00:21:55.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.949+0000 7fe96b7ee700 1 --2- 172.21.15.62:0/2342069421 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe94c039150 0x7fe94c03b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:55.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.949+0000 7fe96b7ee700 1 --2- 172.21.15.62:0/2342069421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe964101040 0x7fe96419d690 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:55.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.949+0000 7fe96b7ee700 1 -- 172.21.15.62:0/2342069421 >> 172.21.15.62:0/2342069421 conn(0x7fe9640768f0 msgr2=0x7fe9640698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:55.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.949+0000 7fe96b7ee700 1 -- 172.21.15.62:0/2342069421 shutdown_connections 2024-06-10T00:21:55.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:55.949+0000 7fe96b7ee700 1 -- 172.21.15.62:0/2342069421 wait complete. 2024-06-10T00:21:56.486 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-10T00:21:56.487 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-10T00:21:56.729 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:21:57.052 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:56 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:21:57.970 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:57 smithi062 ceph-mon[32390]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:21:57.970 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:57 smithi062 ceph-mon[32390]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:21:58.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.186+0000 7f3514f64700 1 -- 172.21.15.62:0/456513892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3510101000 msgr2=0x7f35101013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:58.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.186+0000 7f3514f64700 1 --2- 172.21.15.62:0/456513892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3510101000 0x7f35101013e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3500009910 tx=0x7f3500009c20 comp rx=0 tx=0).stop 2024-06-10T00:21:58.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.186+0000 7f3514f64700 1 -- 172.21.15.62:0/456513892 shutdown_connections 2024-06-10T00:21:58.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.186+0000 7f3514f64700 1 --2- 172.21.15.62:0/456513892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3510101000 0x7f35101013e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.186+0000 7f3514f64700 1 -- 172.21.15.62:0/456513892 >> 172.21.15.62:0/456513892 conn(0x7f35100fc890 msgr2=0x7f35100fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.186+0000 7f3514f64700 1 -- 172.21.15.62:0/456513892 shutdown_connections 2024-06-10T00:21:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.186+0000 7f3514f64700 1 -- 172.21.15.62:0/456513892 wait complete. 2024-06-10T00:21:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.187+0000 7f3514f64700 1 Processor -- start 2024-06-10T00:21:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.187+0000 7f3514f64700 1 -- start start 2024-06-10T00:21:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.187+0000 7f3514f64700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3510101000 0x7f3510198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.187+0000 7f3514f64700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f351010b3d0 con 0x7f3510101000 2024-06-10T00:21:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.187+0000 7f350e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3510101000 0x7f3510198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.187+0000 7f350e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3510101000 0x7f3510198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37276/0 (socket says 172.21.15.62:37276) 2024-06-10T00:21:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.187+0000 7f350e59c700 1 -- 172.21.15.62:0/251446218 learned_addr learned my addr 172.21.15.62:0/251446218 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:21:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.188+0000 7f350e59c700 1 -- 172.21.15.62:0/251446218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3510198da0 con 0x7f3510101000 2024-06-10T00:21:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.188+0000 7f350e59c700 1 --2- 172.21.15.62:0/251446218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3510101000 0x7f3510198860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3500005850 tx=0x7f3500005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:58.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.188+0000 7f34ff7fe700 1 -- 172.21.15.62:0/251446218 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3500026070 con 0x7f3510101000 2024-06-10T00:21:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.188+0000 7f34ff7fe700 1 -- 172.21.15.62:0/251446218 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f350000bbd0 con 0x7f3510101000 2024-06-10T00:21:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.188+0000 7f3514f64700 1 -- 172.21.15.62:0/251446218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3510199090 con 0x7f3510101000 2024-06-10T00:21:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.188+0000 7f34ff7fe700 1 -- 172.21.15.62:0/251446218 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3500003970 con 0x7f3510101000 2024-06-10T00:21:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.188+0000 7f3514f64700 1 -- 172.21.15.62:0/251446218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3510199550 con 0x7f3510101000 2024-06-10T00:21:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.189+0000 7f34ff7fe700 1 -- 172.21.15.62:0/251446218 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3500029070 con 0x7f3510101000 2024-06-10T00:21:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.189+0000 7f3514f64700 1 -- 172.21.15.62:0/251446218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35101024e0 con 0x7f3510101000 2024-06-10T00:21:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.189+0000 7f34ff7fe700 1 --2- 172.21.15.62:0/251446218 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f34f8039150 0x7f34f803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:21:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.189+0000 7f34ff7fe700 1 -- 172.21.15.62:0/251446218 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f350001e070 con 0x7f3510101000 2024-06-10T00:21:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.189+0000 7f350dd9b700 1 --2- 172.21.15.62:0/251446218 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f34f8039150 0x7f34f803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:21:58.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.190+0000 7f350dd9b700 1 --2- 172.21.15.62:0/251446218 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f34f8039150 0x7f34f803b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3504006fd0 tx=0x7f3504006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:21:58.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.194+0000 7f34ff7fe700 1 -- 172.21.15.62:0/251446218 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3500013070 con 0x7f3510101000 2024-06-10T00:21:58.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.410+0000 7f3514f64700 1 -- 172.21.15.62:0/251446218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3510063af0 con 0x7f3510101000 2024-06-10T00:21:58.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.746+0000 7f34ff7fe700 1 -- 172.21.15.62:0/251446218 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f3500030020 con 0x7f3510101000 2024-06-10T00:21:58.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.749+0000 7f3514f64700 1 -- 172.21.15.62:0/251446218 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f34f8039150 msgr2=0x7f34f803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:58.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.749+0000 7f3514f64700 1 --2- 172.21.15.62:0/251446218 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f34f8039150 0x7f34f803b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3504006fd0 tx=0x7f3504006e40 comp rx=0 tx=0).stop 2024-06-10T00:21:58.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.749+0000 7f3514f64700 1 -- 172.21.15.62:0/251446218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3510101000 msgr2=0x7f3510198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:21:58.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.749+0000 7f3514f64700 1 --2- 172.21.15.62:0/251446218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3510101000 0x7f3510198860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3500005850 tx=0x7f3500005030 comp rx=0 tx=0).stop 2024-06-10T00:21:58.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.749+0000 7f3514f64700 1 -- 172.21.15.62:0/251446218 shutdown_connections 2024-06-10T00:21:58.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.749+0000 7f3514f64700 1 --2- 172.21.15.62:0/251446218 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f34f8039150 0x7f34f803b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:58.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.749+0000 7f3514f64700 1 --2- 172.21.15.62:0/251446218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3510101000 0x7f3510198860 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:21:58.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.749+0000 7f3514f64700 1 -- 172.21.15.62:0/251446218 >> 172.21.15.62:0/251446218 conn(0x7f35100fc890 msgr2=0x7f3510103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:21:58.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.749+0000 7f3514f64700 1 -- 172.21.15.62:0/251446218 shutdown_connections 2024-06-10T00:21:58.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:21:58.749+0000 7f3514f64700 1 -- 172.21.15.62:0/251446218 wait complete. 2024-06-10T00:21:58.750 INFO:teuthology.orchestra.run.smithi062.stderr:adjusted tunables profile to default 2024-06-10T00:21:59.078 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:58 smithi062 ceph-mon[32390]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:21:59.078 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:58 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/251446218' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-10T00:21:59.381 INFO:tasks.cephadm:Adding mon.a on smithi062 2024-06-10T00:21:59.381 INFO:tasks.cephadm:Adding mon.b on smithi105 2024-06-10T00:21:59.382 INFO:tasks.cephadm:Adding mon.c on smithi138 2024-06-10T00:21:59.382 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi062:172.21.15.62=a;smithi105:172.21.15.105=b;smithi138:172.21.15.138=c' 2024-06-10T00:21:59.613 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:22:00.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/251446218' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-10T00:22:00.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:22:00.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:00.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:00.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:00.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:00.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:22:00.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:00.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:00.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: Updating smithi138:/etc/ceph/ceph.conf 2024-06-10T00:22:00.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: Updating smithi138:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:22:00.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:22:00.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: Updating smithi138:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:22:00.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:00.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:00.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:21:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:01.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.106+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2145795666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c08103110 msgr2=0x7f7c081034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:01.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.106+0000 7f7c0e6f0700 1 --2- 172.21.15.138:0/2145795666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c08103110 0x7f7c081034f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009910 tx=0x7f7bf8009c20 comp rx=0 tx=0).stop 2024-06-10T00:22:01.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.106+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2145795666 shutdown_connections 2024-06-10T00:22:01.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.106+0000 7f7c0e6f0700 1 --2- 172.21.15.138:0/2145795666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c08103110 0x7f7c081034f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:01.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.106+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2145795666 >> 172.21.15.138:0/2145795666 conn(0x7f7c080fc8e0 msgr2=0x7f7c080fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:22:01.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.106+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2145795666 shutdown_connections 2024-06-10T00:22:01.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.106+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2145795666 wait complete. 2024-06-10T00:22:01.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.107+0000 7f7c0e6f0700 1 Processor -- start 2024-06-10T00:22:01.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.107+0000 7f7c0e6f0700 1 -- start start 2024-06-10T00:22:01.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.107+0000 7f7c0e6f0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c08103110 0x7f7c0819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:01.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.107+0000 7f7c0e6f0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0810c540 con 0x7f7c08103110 2024-06-10T00:22:01.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.107+0000 7f7c07fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c08103110 0x7f7c0819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:01.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.107+0000 7f7c07fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c08103110 0x7f7c0819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.138:56294/0 (socket says 172.21.15.138:56294) 2024-06-10T00:22:01.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.107+0000 7f7c07fff700 1 -- 172.21.15.138:0/2231244044 learned_addr learned my addr 172.21.15.138:0/2231244044 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-06-10T00:22:01.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.108+0000 7f7c07fff700 1 -- 172.21.15.138:0/2231244044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0819dbf0 con 0x7f7c08103110 2024-06-10T00:22:01.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.108+0000 7f7c07fff700 1 --2- 172.21.15.138:0/2231244044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c08103110 0x7f7c0819d6b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8005850 tx=0x7f7bf8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:22:01.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:00 smithi062 ceph-mon[32390]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:01.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.108+0000 7f7c057fa700 1 -- 172.21.15.138:0/2231244044 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf8026070 con 0x7f7c08103110 2024-06-10T00:22:01.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.108+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2231244044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0819de80 con 0x7f7c08103110 2024-06-10T00:22:01.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.108+0000 7f7c057fa700 1 -- 172.21.15.138:0/2231244044 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf800bc30 con 0x7f7c08103110 2024-06-10T00:22:01.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.108+0000 7f7c057fa700 1 -- 172.21.15.138:0/2231244044 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf8003a20 con 0x7f7c08103110 2024-06-10T00:22:01.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.108+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2231244044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c08197830 con 0x7f7c08103110 2024-06-10T00:22:01.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.109+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2231244044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c0806aae0 con 0x7f7c08103110 2024-06-10T00:22:01.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.109+0000 7f7c057fa700 1 -- 172.21.15.138:0/2231244044 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7bf8029070 con 0x7f7c08103110 2024-06-10T00:22:01.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.109+0000 7f7c057fa700 1 --2- 172.21.15.138:0/2231244044 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7bf00390f0 0x7f7bf003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:01.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.109+0000 7f7c057fa700 1 -- 172.21.15.138:0/2231244044 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7bf801e070 con 0x7f7c08103110 2024-06-10T00:22:01.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.110+0000 7f7c077fe700 1 --2- 172.21.15.138:0/2231244044 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7bf00390f0 0x7f7bf003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:01.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.110+0000 7f7c077fe700 1 --2- 172.21.15.138:0/2231244044 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7bf00390f0 0x7f7bf003b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc006fd0 tx=0x7f7bfc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:22:01.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.115+0000 7f7c057fa700 1 -- 172.21.15.138:0/2231244044 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7bf8013070 con 0x7f7c08103110 2024-06-10T00:22:01.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.324+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2231244044 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi105:172.21.15.105=b;smithi138:172.21.15.138=c", "target": ["mon-mgr", ""]}) -- 0x7f7c08198d60 con 0x7f7bf00390f0 2024-06-10T00:22:01.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.331+0000 7f7c057fa700 1 -- 172.21.15.138:0/2231244044 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7c08198d60 con 0x7f7bf00390f0 2024-06-10T00:22:01.331 INFO:teuthology.orchestra.run.smithi138.stdout:Scheduled mon update... 2024-06-10T00:22:01.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.333+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2231244044 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7bf00390f0 msgr2=0x7f7bf003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:01.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.333+0000 7f7c0e6f0700 1 --2- 172.21.15.138:0/2231244044 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7bf00390f0 0x7f7bf003b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc006fd0 tx=0x7f7bfc006e40 comp rx=0 tx=0).stop 2024-06-10T00:22:01.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.333+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2231244044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c08103110 msgr2=0x7f7c0819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:01.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.333+0000 7f7c0e6f0700 1 --2- 172.21.15.138:0/2231244044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c08103110 0x7f7c0819d6b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8005850 tx=0x7f7bf8005030 comp rx=0 tx=0).stop 2024-06-10T00:22:01.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.333+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2231244044 shutdown_connections 2024-06-10T00:22:01.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.333+0000 7f7c0e6f0700 1 --2- 172.21.15.138:0/2231244044 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7bf00390f0 0x7f7bf003b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:01.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.333+0000 7f7c0e6f0700 1 --2- 172.21.15.138:0/2231244044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c08103110 0x7f7c0819d6b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:01.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.333+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2231244044 >> 172.21.15.138:0/2231244044 conn(0x7f7c080fc8e0 msgr2=0x7f7c08103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:22:01.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.333+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2231244044 shutdown_connections 2024-06-10T00:22:01.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:01.333+0000 7f7c0e6f0700 1 -- 172.21.15.138:0/2231244044 wait complete. 2024-06-10T00:22:01.919 DEBUG:teuthology.orchestra.run.smithi105:mon.b> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mon.b.service 2024-06-10T00:22:01.922 DEBUG:teuthology.orchestra.run.smithi138:mon.c> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mon.c.service 2024-06-10T00:22:01.927 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-10T00:22:01.928 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-10T00:22:02.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:02 smithi062 ceph-mon[32390]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi105:172.21.15.105=b;smithi138:172.21.15.138=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:22:02.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:02 smithi062 ceph-mon[32390]: Saving service mon spec with placement smithi062:172.21.15.62=a;smithi105:172.21.15.105=b;smithi138:172.21.15.138=c;count:3 2024-06-10T00:22:02.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:02 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:02.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:02 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:02.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:02 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:02.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:02 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:02.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:02 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:02.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:02 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:22:02.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:02 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:02.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:02 smithi062 ceph-mon[32390]: Deploying daemon mon.c on smithi138 2024-06-10T00:22:03.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:03 smithi062 ceph-mon[32390]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:03.892 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:22:05.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.528+0000 7f9ff9997700 1 -- 172.21.15.138:0/1094518803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff4103130 msgr2=0x7f9ff4103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:05.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.528+0000 7f9ff9997700 1 --2- 172.21.15.138:0/1094518803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff4103130 0x7f9ff4103510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009910 tx=0x7f9fe4009c20 comp rx=0 tx=0).stop 2024-06-10T00:22:05.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.528+0000 7f9ff9997700 1 -- 172.21.15.138:0/1094518803 shutdown_connections 2024-06-10T00:22:05.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.528+0000 7f9ff9997700 1 --2- 172.21.15.138:0/1094518803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff4103130 0x7f9ff4103510 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:05.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.528+0000 7f9ff9997700 1 -- 172.21.15.138:0/1094518803 >> 172.21.15.138:0/1094518803 conn(0x7f9ff40fc940 msgr2=0x7f9ff40fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:22:05.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.528+0000 7f9ff9997700 1 -- 172.21.15.138:0/1094518803 shutdown_connections 2024-06-10T00:22:05.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.528+0000 7f9ff9997700 1 -- 172.21.15.138:0/1094518803 wait complete. 2024-06-10T00:22:05.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.529+0000 7f9ff9997700 1 Processor -- start 2024-06-10T00:22:05.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.529+0000 7f9ff9997700 1 -- start start 2024-06-10T00:22:05.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.529+0000 7f9ff9997700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff4103130 0x7f9ff419d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:05.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.529+0000 7f9ff9997700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff410c620 con 0x7f9ff4103130 2024-06-10T00:22:05.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.530+0000 7f9ff2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff4103130 0x7f9ff419d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:05.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.530+0000 7f9ff2ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff4103130 0x7f9ff419d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.138:56310/0 (socket says 172.21.15.138:56310) 2024-06-10T00:22:05.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.530+0000 7f9ff2ffd700 1 -- 172.21.15.138:0/2241994809 learned_addr learned my addr 172.21.15.138:0/2241994809 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-06-10T00:22:05.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.530+0000 7f9ff2ffd700 1 -- 172.21.15.138:0/2241994809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff419dc90 con 0x7f9ff4103130 2024-06-10T00:22:05.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.530+0000 7f9ff2ffd700 1 --2- 172.21.15.138:0/2241994809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff4103130 0x7f9ff419d750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4006010 tx=0x7f9fe4005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:22:05.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.530+0000 7f9ff8995700 1 -- 172.21.15.138:0/2241994809 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe4026070 con 0x7f9ff4103130 2024-06-10T00:22:05.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.530+0000 7f9ff9997700 1 -- 172.21.15.138:0/2241994809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff419df20 con 0x7f9ff4103130 2024-06-10T00:22:05.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.530+0000 7f9ff8995700 1 -- 172.21.15.138:0/2241994809 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe400bc30 con 0x7f9ff4103130 2024-06-10T00:22:05.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.530+0000 7f9ff8995700 1 -- 172.21.15.138:0/2241994809 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe4003ab0 con 0x7f9ff4103130 2024-06-10T00:22:05.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.530+0000 7f9ff9997700 1 -- 172.21.15.138:0/2241994809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff4197910 con 0x7f9ff4103130 2024-06-10T00:22:05.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.531+0000 7f9ff9997700 1 -- 172.21.15.138:0/2241994809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff406af80 con 0x7f9ff4103130 2024-06-10T00:22:05.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.531+0000 7f9ff8995700 1 -- 172.21.15.138:0/2241994809 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9fe4029070 con 0x7f9ff4103130 2024-06-10T00:22:05.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.531+0000 7f9ff8995700 1 --2- 172.21.15.138:0/2241994809 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9fe00390f0 0x7f9fe003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:05.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.531+0000 7f9ff8995700 1 -- 172.21.15.138:0/2241994809 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9fe401e070 con 0x7f9ff4103130 2024-06-10T00:22:05.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.532+0000 7f9fea7fc700 1 --2- 172.21.15.138:0/2241994809 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9fe00390f0 0x7f9fe003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:05.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.532+0000 7f9fea7fc700 1 --2- 172.21.15.138:0/2241994809 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9fe00390f0 0x7f9fe003b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc006fd0 tx=0x7f9fdc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:22:05.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.537+0000 7f9ff8995700 1 -- 172.21.15.138:0/2241994809 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9fe4013070 con 0x7f9ff4103130 2024-06-10T00:22:05.788 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 podman[36490]: 2024-06-10 00:22:05.678639903 +0000 UTC m=+0.343390062 container create f0c99954ecf68a5a60b29f82282765f6cd700f4c46a00b99da8d8bfda8e381c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-10T00:22:05.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.822+0000 7f9ff9997700 1 -- 172.21.15.138:0/2241994809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9ff4063af0 con 0x7f9ff4103130 2024-06-10T00:22:05.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.823+0000 7f9ff8995700 1 -- 172.21.15.138:0/2241994809 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f9fe4030020 con 0x7f9ff4103130 2024-06-10T00:22:05.825 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:22:05.825 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","modified":"2024-06-10T00:20:24.460529Z","created":"2024-06-10T00:20:24.460529Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-10T00:22:05.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.825+0000 7f9ff9997700 1 -- 172.21.15.138:0/2241994809 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9fe00390f0 msgr2=0x7f9fe003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:05.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.825+0000 7f9ff9997700 1 --2- 172.21.15.138:0/2241994809 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9fe00390f0 0x7f9fe003b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc006fd0 tx=0x7f9fdc006e40 comp rx=0 tx=0).stop 2024-06-10T00:22:05.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.825+0000 7f9ff9997700 1 -- 172.21.15.138:0/2241994809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff4103130 msgr2=0x7f9ff419d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:05.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.825+0000 7f9ff9997700 1 --2- 172.21.15.138:0/2241994809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff4103130 0x7f9ff419d750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4006010 tx=0x7f9fe4005050 comp rx=0 tx=0).stop 2024-06-10T00:22:05.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.825+0000 7f9ff9997700 1 -- 172.21.15.138:0/2241994809 shutdown_connections 2024-06-10T00:22:05.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.825+0000 7f9ff9997700 1 --2- 172.21.15.138:0/2241994809 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9fe00390f0 0x7f9fe003b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:05.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.825+0000 7f9ff9997700 1 --2- 172.21.15.138:0/2241994809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ff4103130 0x7f9ff419d750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:05.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.825+0000 7f9ff9997700 1 -- 172.21.15.138:0/2241994809 >> 172.21.15.138:0/2241994809 conn(0x7f9ff40fc940 msgr2=0x7f9ff4103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:22:05.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.826+0000 7f9ff9997700 1 -- 172.21.15.138:0/2241994809 shutdown_connections 2024-06-10T00:22:05.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:05.826+0000 7f9ff9997700 1 -- 172.21.15.138:0/2241994809 wait complete. 2024-06-10T00:22:05.827 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-06-10T00:22:05.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:05 smithi062 ceph-mon[32390]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:06.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 podman[36490]: 2024-06-10 00:22:05.85588842 +0000 UTC m=+0.520638581 container init f0c99954ecf68a5a60b29f82282765f6cd700f4c46a00b99da8d8bfda8e381c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-c, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-10T00:22:06.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 podman[36490]: 2024-06-10 00:22:05.859239173 +0000 UTC m=+0.523989332 container start f0c99954ecf68a5a60b29f82282765f6cd700f4c46a00b99da8d8bfda8e381c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-c, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-06-10T00:22:06.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: set uid:gid to 167:167 (ceph:ceph) 2024-06-10T00:22:06.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-10T00:22:06.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: pidfile_write: ignore empty --pid-file 2024-06-10T00:22:06.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: load: jerasure load: lrc 2024-06-10T00:22:06.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: RocksDB version: 7.9.2 2024-06-10T00:22:06.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Git sha 0 2024-06-10T00:22:06.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-10T00:22:06.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: DB SUMMARY 2024-06-10T00:22:06.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: DB Session ID: PI3OW9F73Q6LB326YMVG 2024-06-10T00:22:06.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: CURRENT file: CURRENT 2024-06-10T00:22:06.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: IDENTITY file: IDENTITY 2024-06-10T00:22:06.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-06-10T00:22:06.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-06-10T00:22:06.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-06-10T00:22:06.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.error_if_exists: 0 2024-06-10T00:22:06.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.create_if_missing: 0 2024-06-10T00:22:06.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.paranoid_checks: 1 2024-06-10T00:22:06.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-10T00:22:06.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-10T00:22:06.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-10T00:22:06.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.env: 0x55e424e53c60 2024-06-10T00:22:06.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.fs: PosixFileSystem 2024-06-10T00:22:06.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.info_log: 0x55e426169b20 2024-06-10T00:22:06.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_file_opening_threads: 16 2024-06-10T00:22:06.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.statistics: (nil) 2024-06-10T00:22:06.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.use_fsync: 0 2024-06-10T00:22:06.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_log_file_size: 0 2024-06-10T00:22:06.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-10T00:22:06.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-10T00:22:06.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.keep_log_file_num: 1000 2024-06-10T00:22:06.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.recycle_log_file_num: 0 2024-06-10T00:22:06.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.allow_fallocate: 1 2024-06-10T00:22:06.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.allow_mmap_reads: 0 2024-06-10T00:22:06.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.allow_mmap_writes: 0 2024-06-10T00:22:06.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.use_direct_reads: 0 2024-06-10T00:22:06.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-10T00:22:06.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.create_missing_column_families: 0 2024-06-10T00:22:06.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.db_log_dir: 2024-06-10T00:22:06.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.wal_dir: 2024-06-10T00:22:06.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-10T00:22:06.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-10T00:22:06.128 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-10T00:22:06.128 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-10T00:22:06.128 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-10T00:22:06.128 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-10T00:22:06.128 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.advise_random_on_open: 1 2024-06-10T00:22:06.128 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.db_write_buffer_size: 0 2024-06-10T00:22:06.128 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.write_buffer_manager: 0x55e42613a1e0 2024-06-10T00:22:06.128 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-10T00:22:06.128 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-10T00:22:06.128 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-10T00:22:06.128 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.rate_limiter: (nil) 2024-06-10T00:22:06.128 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-10T00:22:06.129 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.wal_recovery_mode: 2 2024-06-10T00:22:06.129 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.enable_thread_tracking: 0 2024-06-10T00:22:06.129 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.enable_pipelined_write: 0 2024-06-10T00:22:06.129 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.unordered_write: 0 2024-06-10T00:22:06.129 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-10T00:22:06.129 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-10T00:22:06.129 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-10T00:22:06.129 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-10T00:22:06.129 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.row_cache: None 2024-06-10T00:22:06.130 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.wal_filter: None 2024-06-10T00:22:06.131 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-10T00:22:06.131 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.allow_ingest_behind: 0 2024-06-10T00:22:06.131 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.two_write_queues: 0 2024-06-10T00:22:06.131 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.manual_wal_flush: 0 2024-06-10T00:22:06.131 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.wal_compression: 0 2024-06-10T00:22:06.131 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.atomic_flush: 0 2024-06-10T00:22:06.132 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-10T00:22:06.132 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-10T00:22:06.132 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-10T00:22:06.132 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.log_readahead_size: 0 2024-06-10T00:22:06.132 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-10T00:22:06.132 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.best_efforts_recovery: 0 2024-06-10T00:22:06.132 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-10T00:22:06.132 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-10T00:22:06.132 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.allow_data_in_errors: 0 2024-06-10T00:22:06.132 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.db_host_id: __hostname__ 2024-06-10T00:22:06.133 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-10T00:22:06.133 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_background_jobs: 2 2024-06-10T00:22:06.133 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_background_compactions: -1 2024-06-10T00:22:06.133 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_subcompactions: 1 2024-06-10T00:22:06.133 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-10T00:22:06.133 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-10T00:22:06.133 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-10T00:22:06.133 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_total_wal_size: 0 2024-06-10T00:22:06.133 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-10T00:22:06.133 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-10T00:22:06.133 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-10T00:22:06.134 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-10T00:22:06.134 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_open_files: -1 2024-06-10T00:22:06.134 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.bytes_per_sync: 0 2024-06-10T00:22:06.134 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-10T00:22:06.134 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-10T00:22:06.134 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compaction_readahead_size: 0 2024-06-10T00:22:06.134 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_background_flushes: -1 2024-06-10T00:22:06.134 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Compression algorithms supported: 2024-06-10T00:22:06.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-10T00:22:06.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: kZSTD supported: 0 2024-06-10T00:22:06.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: kXpressCompression supported: 0 2024-06-10T00:22:06.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: kLZ4HCCompression supported: 1 2024-06-10T00:22:06.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: kZlibCompression supported: 1 2024-06-10T00:22:06.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: kSnappyCompression supported: 1 2024-06-10T00:22:06.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: kLZ4Compression supported: 1 2024-06-10T00:22:06.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: kBZip2Compression supported: 0 2024-06-10T00:22:06.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-10T00:22:06.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-10T00:22:06.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-06-10T00:22:06.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-10T00:22:06.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-10T00:22:06.136 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.merge_operator: 2024-06-10T00:22:06.136 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compaction_filter: None 2024-06-10T00:22:06.136 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compaction_filter_factory: None 2024-06-10T00:22:06.136 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.sst_partitioner_factory: None 2024-06-10T00:22:06.136 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-10T00:22:06.136 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-10T00:22:06.136 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e426169c40) 2024-06-10T00:22:06.136 INFO:journalctl@ceph.mon.c.smithi138.stdout: cache_index_and_filter_blocks: 1 2024-06-10T00:22:06.136 INFO:journalctl@ceph.mon.c.smithi138.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-10T00:22:06.136 INFO:journalctl@ceph.mon.c.smithi138.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-10T00:22:06.136 INFO:journalctl@ceph.mon.c.smithi138.stdout: pin_top_level_index_and_filter: 1 2024-06-10T00:22:06.136 INFO:journalctl@ceph.mon.c.smithi138.stdout: index_type: 0 2024-06-10T00:22:06.137 INFO:journalctl@ceph.mon.c.smithi138.stdout: data_block_index_type: 0 2024-06-10T00:22:06.137 INFO:journalctl@ceph.mon.c.smithi138.stdout: index_shortening: 1 2024-06-10T00:22:06.137 INFO:journalctl@ceph.mon.c.smithi138.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-10T00:22:06.137 INFO:journalctl@ceph.mon.c.smithi138.stdout: checksum: 4 2024-06-10T00:22:06.137 INFO:journalctl@ceph.mon.c.smithi138.stdout: no_block_cache: 0 2024-06-10T00:22:06.137 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_cache: 0x55e42616b090 2024-06-10T00:22:06.137 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_cache_name: BinnedLRUCache 2024-06-10T00:22:06.137 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_cache_options: 2024-06-10T00:22:06.137 INFO:journalctl@ceph.mon.c.smithi138.stdout: capacity : 536870912 2024-06-10T00:22:06.137 INFO:journalctl@ceph.mon.c.smithi138.stdout: num_shard_bits : 4 2024-06-10T00:22:06.137 INFO:journalctl@ceph.mon.c.smithi138.stdout: strict_capacity_limit : 0 2024-06-10T00:22:06.137 INFO:journalctl@ceph.mon.c.smithi138.stdout: high_pri_pool_ratio: 0.000 2024-06-10T00:22:06.137 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_cache_compressed: (nil) 2024-06-10T00:22:06.138 INFO:journalctl@ceph.mon.c.smithi138.stdout: persistent_cache: (nil) 2024-06-10T00:22:06.138 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_size: 4096 2024-06-10T00:22:06.138 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_size_deviation: 10 2024-06-10T00:22:06.138 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_restart_interval: 16 2024-06-10T00:22:06.138 INFO:journalctl@ceph.mon.c.smithi138.stdout: index_block_restart_interval: 1 2024-06-10T00:22:06.138 INFO:journalctl@ceph.mon.c.smithi138.stdout: metadata_block_size: 4096 2024-06-10T00:22:06.138 INFO:journalctl@ceph.mon.c.smithi138.stdout: partition_filters: 0 2024-06-10T00:22:06.138 INFO:journalctl@ceph.mon.c.smithi138.stdout: use_delta_encoding: 1 2024-06-10T00:22:06.138 INFO:journalctl@ceph.mon.c.smithi138.stdout: filter_policy: bloomfilter 2024-06-10T00:22:06.138 INFO:journalctl@ceph.mon.c.smithi138.stdout: whole_key_filtering: 1 2024-06-10T00:22:06.138 INFO:journalctl@ceph.mon.c.smithi138.stdout: verify_compression: 0 2024-06-10T00:22:06.138 INFO:journalctl@ceph.mon.c.smithi138.stdout: read_amp_bytes_per_bit: 0 2024-06-10T00:22:06.139 INFO:journalctl@ceph.mon.c.smithi138.stdout: format_version: 5 2024-06-10T00:22:06.139 INFO:journalctl@ceph.mon.c.smithi138.stdout: enable_index_compression: 1 2024-06-10T00:22:06.139 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_align: 0 2024-06-10T00:22:06.139 INFO:journalctl@ceph.mon.c.smithi138.stdout: max_auto_readahead_size: 262144 2024-06-10T00:22:06.139 INFO:journalctl@ceph.mon.c.smithi138.stdout: prepopulate_block_cache: 0 2024-06-10T00:22:06.139 INFO:journalctl@ceph.mon.c.smithi138.stdout: initial_auto_readahead_size: 8192 2024-06-10T00:22:06.139 INFO:journalctl@ceph.mon.c.smithi138.stdout: num_file_reads_for_auto_readahead: 2 2024-06-10T00:22:06.139 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.write_buffer_size: 33554432 2024-06-10T00:22:06.139 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_write_buffer_number: 2 2024-06-10T00:22:06.139 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compression: NoCompression 2024-06-10T00:22:06.139 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.bottommost_compression: Disabled 2024-06-10T00:22:06.139 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.prefix_extractor: nullptr 2024-06-10T00:22:06.139 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-10T00:22:06.140 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.num_levels: 7 2024-06-10T00:22:06.140 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-10T00:22:06.140 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-10T00:22:06.140 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-10T00:22:06.140 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-10T00:22:06.140 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-10T00:22:06.140 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-10T00:22:06.140 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-10T00:22:06.140 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-10T00:22:06.140 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-10T00:22:06.140 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-10T00:22:06.140 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-10T00:22:06.141 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-10T00:22:06.141 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-10T00:22:06.141 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compression_opts.level: 32767 2024-06-10T00:22:06.141 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compression_opts.strategy: 0 2024-06-10T00:22:06.141 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-10T00:22:06.141 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-10T00:22:06.141 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-10T00:22:06.141 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-10T00:22:06.141 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compression_opts.enabled: false 2024-06-10T00:22:06.141 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-10T00:22:06.141 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-10T00:22:06.141 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-10T00:22:06.142 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-10T00:22:06.142 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.target_file_size_base: 67108864 2024-06-10T00:22:06.142 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-10T00:22:06.142 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-10T00:22:06.142 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-10T00:22:06.142 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-10T00:22:06.142 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-10T00:22:06.142 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-10T00:22:06.142 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-10T00:22:06.142 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-10T00:22:06.142 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-10T00:22:06.142 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-10T00:22:06.142 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-10T00:22:06.143 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-10T00:22:06.143 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-10T00:22:06.143 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-10T00:22:06.143 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.arena_block_size: 1048576 2024-06-10T00:22:06.143 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-10T00:22:06.143 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-10T00:22:06.143 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.disable_auto_compactions: 0 2024-06-10T00:22:06.143 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-10T00:22:06.143 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-10T00:22:06.143 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-10T00:22:06.143 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-10T00:22:06.143 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-10T00:22:06.143 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-10T00:22:06.144 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-10T00:22:06.144 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-10T00:22:06.144 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-10T00:22:06.144 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-10T00:22:06.144 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.table_properties_collectors: 2024-06-10T00:22:06.144 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.inplace_update_support: 0 2024-06-10T00:22:06.144 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-10T00:22:06.144 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-10T00:22:06.144 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-10T00:22:06.144 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-10T00:22:06.144 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.bloom_locality: 0 2024-06-10T00:22:06.145 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.max_successive_merges: 0 2024-06-10T00:22:06.145 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-10T00:22:06.145 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.paranoid_file_checks: 0 2024-06-10T00:22:06.145 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.force_consistency_checks: 1 2024-06-10T00:22:06.145 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.report_bg_io_stats: 0 2024-06-10T00:22:06.145 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.ttl: 2592000 2024-06-10T00:22:06.145 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-10T00:22:06.145 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-10T00:22:06.145 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-10T00:22:06.145 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.enable_blob_files: false 2024-06-10T00:22:06.145 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.min_blob_size: 0 2024-06-10T00:22:06.145 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.blob_file_size: 268435456 2024-06-10T00:22:06.145 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-10T00:22:06.146 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-10T00:22:06.146 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-10T00:22:06.146 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-10T00:22:06.146 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-10T00:22:06.146 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.blob_file_starting_level: 0 2024-06-10T00:22:06.146 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-10T00:22:06.146 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-10T00:22:06.146 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-06-10T00:22:06.146 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ddb504fa-9dd9-4605-8f56-05a49025e13c 2024-06-10T00:22:06.146 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717978925888867, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-10T00:22:06.146 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-06-10T00:22:06.146 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717978925889291, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717978925, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ddb504fa-9dd9-4605-8f56-05a49025e13c", "db_session_id": "PI3OW9F73Q6LB326YMVG", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-06-10T00:22:06.146 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717978925889369, "job": 1, "event": "recovery_finished"} 2024-06-10T00:22:06.147 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-06-10T00:22:06.147 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-10T00:22:06.147 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e42625a000 2024-06-10T00:22:06.147 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: DB pointer 0x55e426244000 2024-06-10T00:22:06.147 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-10T00:22:06.147 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-10T00:22:06.147 INFO:journalctl@ceph.mon.c.smithi138.stdout: ** DB Stats ** 2024-06-10T00:22:06.147 INFO:journalctl@ceph.mon.c.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-10T00:22:06.147 INFO:journalctl@ceph.mon.c.smithi138.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-10T00:22:06.147 INFO:journalctl@ceph.mon.c.smithi138.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-10T00:22:06.147 INFO:journalctl@ceph.mon.c.smithi138.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-10T00:22:06.147 INFO:journalctl@ceph.mon.c.smithi138.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-10T00:22:06.147 INFO:journalctl@ceph.mon.c.smithi138.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-10T00:22:06.147 INFO:journalctl@ceph.mon.c.smithi138.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-10T00:22:06.148 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-06-10T00:22:06.148 INFO:journalctl@ceph.mon.c.smithi138.stdout: ** Compaction Stats [default] ** 2024-06-10T00:22:06.148 INFO:journalctl@ceph.mon.c.smithi138.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-10T00:22:06.148 INFO:journalctl@ceph.mon.c.smithi138.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-10T00:22:06.148 INFO:journalctl@ceph.mon.c.smithi138.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-10T00:22:06.148 INFO:journalctl@ceph.mon.c.smithi138.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-10T00:22:06.148 INFO:journalctl@ceph.mon.c.smithi138.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-10T00:22:06.148 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-06-10T00:22:06.148 INFO:journalctl@ceph.mon.c.smithi138.stdout: ** Compaction Stats [default] ** 2024-06-10T00:22:06.148 INFO:journalctl@ceph.mon.c.smithi138.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-10T00:22:06.148 INFO:journalctl@ceph.mon.c.smithi138.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-10T00:22:06.148 INFO:journalctl@ceph.mon.c.smithi138.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-10T00:22:06.148 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-06-10T00:22:06.148 INFO:journalctl@ceph.mon.c.smithi138.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-10T00:22:06.149 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-06-10T00:22:06.149 INFO:journalctl@ceph.mon.c.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-10T00:22:06.149 INFO:journalctl@ceph.mon.c.smithi138.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-10T00:22:06.149 INFO:journalctl@ceph.mon.c.smithi138.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-10T00:22:06.149 INFO:journalctl@ceph.mon.c.smithi138.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-10T00:22:06.149 INFO:journalctl@ceph.mon.c.smithi138.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-10T00:22:06.149 INFO:journalctl@ceph.mon.c.smithi138.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-10T00:22:06.149 INFO:journalctl@ceph.mon.c.smithi138.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-10T00:22:06.149 INFO:journalctl@ceph.mon.c.smithi138.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-10T00:22:06.149 INFO:journalctl@ceph.mon.c.smithi138.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-10T00:22:06.149 INFO:journalctl@ceph.mon.c.smithi138.stdout: Block cache BinnedLRUCache@0x55e42616b090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-10T00:22:06.149 INFO:journalctl@ceph.mon.c.smithi138.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-06-10T00:22:06.149 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-06-10T00:22:06.150 INFO:journalctl@ceph.mon.c.smithi138.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-10T00:22:06.150 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-06-10T00:22:06.150 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: using public_addr v2:172.21.15.138:0/0 -> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] 2024-06-10T00:22:06.150 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: starting mon.c rank -1 at public addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] at bind addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:22:06.150 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c@-1(???) e0 preinit fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:22:06.150 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c@-1(synchronizing).mds e1 new map 2024-06-10T00:22:06.150 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c@-1(synchronizing).mds e1 print_map 2024-06-10T00:22:06.151 INFO:journalctl@ceph.mon.c.smithi138.stdout: e1 2024-06-10T00:22:06.151 INFO:journalctl@ceph.mon.c.smithi138.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-10T00:22:06.151 INFO:journalctl@ceph.mon.c.smithi138.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T00:22:06.151 INFO:journalctl@ceph.mon.c.smithi138.stdout: legacy client fscid: -1 2024-06-10T00:22:06.151 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-06-10T00:22:06.151 INFO:journalctl@ceph.mon.c.smithi138.stdout: No filesystems configured 2024-06-10T00:22:06.152 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-10T00:22:06.152 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-10T00:22:06.152 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-10T00:22:06.152 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-10T00:22:06.152 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-10T00:22:06.152 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-10T00:22:06.152 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-10T00:22:06.153 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:22:06.153 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:22:06.153 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:22:06.153 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:06.153 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:22:06.153 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:06.153 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.154 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.154 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.154 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.154 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi105", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:22:06.154 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:06.154 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:06.155 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: Updating smithi105:/etc/ceph/ceph.conf 2024-06-10T00:22:06.155 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:22:06.155 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: Updating smithi105:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:22:06.155 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:22:06.155 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: Deploying cephadm binary to smithi138 2024-06-10T00:22:06.156 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: Updating smithi105:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:22:06.156 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.156 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.156 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.157 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:06.157 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:06.157 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.157 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:06.157 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.157 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: Added host smithi138 2024-06-10T00:22:06.157 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.158 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:06.158 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.158 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:06.158 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:22:06.158 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:06.158 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/251446218' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-10T00:22:06.158 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/251446218' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-10T00:22:06.158 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:22:06.158 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.158 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.158 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.158 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.159 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:22:06.159 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:06.159 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:06.159 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: Updating smithi138:/etc/ceph/ceph.conf 2024-06-10T00:22:06.159 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: Updating smithi138:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:22:06.159 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:22:06.159 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: Updating smithi138:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:22:06.159 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.159 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.159 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.159 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:06.159 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi105:172.21.15.105=b;smithi138:172.21.15.138=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:22:06.159 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: Saving service mon spec with placement smithi062:172.21.15.62=a;smithi105:172.21.15.105=b;smithi138:172.21.15.138=c;count:3 2024-06-10T00:22:06.160 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.160 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:06.160 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:06.160 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:06.160 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:06.160 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:22:06.160 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:06.160 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: Deploying daemon mon.c on smithi138 2024-06-10T00:22:06.160 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:06.160 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:06.160 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:05 smithi138 ceph-mon[36549]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-10T00:22:06.568 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:06 smithi138 bash[36490]: f0c99954ecf68a5a60b29f82282765f6cd700f4c46a00b99da8d8bfda8e381c5 2024-06-10T00:22:06.568 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:06 smithi138 systemd[1]: Started Ceph mon.c for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:22:07.596 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-10T00:22:07.597 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-10T00:22:07.841 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:22:11.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:11.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:22:11.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:11.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: mon.a calling monitor election 2024-06-10T00:22:11.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:11.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:11.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:11.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: mon.c calling monitor election 2024-06-10T00:22:11.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:11.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:11.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:11.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:11.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-10T00:22:11.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: monmap epoch 2 2024-06-10T00:22:11.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:22:11.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: last_changed 2024-06-10T00:22:05.911849+0000 2024-06-10T00:22:11.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: created 2024-06-10T00:20:24.460529+0000 2024-06-10T00:22:11.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: min_mon_release 19 (squid) 2024-06-10T00:22:11.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: election_strategy: 1 2024-06-10T00:22:11.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-06-10T00:22:11.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-06-10T00:22:11.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: fsmap 2024-06-10T00:22:11.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:22:11.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: mgrmap e13: x(active, since 49s) 2024-06-10T00:22:11.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: overall HEALTH_OK 2024-06-10T00:22:11.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:11.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:11.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:11.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:22:11.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:11.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: mon.a calling monitor election 2024-06-10T00:22:11.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:11.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:11.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:11.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: mon.c calling monitor election 2024-06-10T00:22:11.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:11.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:11.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:11.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:11.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-10T00:22:11.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: monmap epoch 2 2024-06-10T00:22:11.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:22:11.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: last_changed 2024-06-10T00:22:05.911849+0000 2024-06-10T00:22:11.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: created 2024-06-10T00:20:24.460529+0000 2024-06-10T00:22:11.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: min_mon_release 19 (squid) 2024-06-10T00:22:11.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: election_strategy: 1 2024-06-10T00:22:11.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-06-10T00:22:11.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-06-10T00:22:11.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: fsmap 2024-06-10T00:22:11.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:22:11.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: mgrmap e13: x(active, since 49s) 2024-06-10T00:22:11.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: overall HEALTH_OK 2024-06-10T00:22:11.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:11.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:12.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:12.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:22:12.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:12.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:11 smithi138 ceph-mon[36549]: Deploying daemon mon.b on smithi105 2024-06-10T00:22:12.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:12.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.259+0000 7fcc03c27700 1 -- 172.21.15.138:0/1194079013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbe80035c0 msgr2=0x7fcbe8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:12.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.259+0000 7fcc03c27700 1 --2- 172.21.15.138:0/1194079013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbe80035c0 0x7fcbe8005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc100dd0 tx=0x7fcbec00feb0 comp rx=0 tx=0).stop 2024-06-10T00:22:12.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.259+0000 7fcc03c27700 1 -- 172.21.15.138:0/1194079013 shutdown_connections 2024-06-10T00:22:12.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.259+0000 7fcc03c27700 1 --2- 172.21.15.138:0/1194079013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbe80035c0 0x7fcbe8005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:12.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.259+0000 7fcc03c27700 1 -- 172.21.15.138:0/1194079013 >> 172.21.15.138:0/1194079013 conn(0x7fcbfc0fc990 msgr2=0x7fcbfc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:22:12.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.259+0000 7fcc03c27700 1 -- 172.21.15.138:0/1194079013 shutdown_connections 2024-06-10T00:22:12.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.259+0000 7fcc03c27700 1 -- 172.21.15.138:0/1194079013 wait complete. 2024-06-10T00:22:12.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.260+0000 7fcc03c27700 1 Processor -- start 2024-06-10T00:22:12.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc03c27700 1 -- start start 2024-06-10T00:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc03c27700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbe80035c0 0x7fcbfc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc03c27700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc198ef0 0x7fcbfc199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc03c27700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc108f30 con 0x7fcbe80035c0 2024-06-10T00:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc03c27700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc108db0 con 0x7fcbfc198ef0 2024-06-10T00:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc011c2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc198ef0 0x7fcbfc199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc011c2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc198ef0 0x7fcbfc199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60580/0 (socket says 172.21.15.138:60580) 2024-06-10T00:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc011c2700 1 -- 172.21.15.138:0/2961093513 learned_addr learned my addr 172.21.15.138:0/2961093513 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-06-10T00:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc019c3700 1 --2- 172.21.15.138:0/2961093513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbe80035c0 0x7fcbfc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc011c2700 1 -- 172.21.15.138:0/2961093513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc198ef0 msgr2=0x7fcbfc199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-10T00:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc011c2700 1 -- 172.21.15.138:0/2961093513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc198ef0 msgr2=0x7fcbfc199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-10T00:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc011c2700 1 --2- 172.21.15.138:0/2961093513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc198ef0 0x7fcbfc199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-10T00:22:12.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc011c2700 1 --2- 172.21.15.138:0/2961093513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc198ef0 0x7fcbfc199370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:22:12.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc019c3700 1 -- 172.21.15.138:0/2961093513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc198ef0 msgr2=0x7fcbfc199370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:22:12.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc019c3700 1 --2- 172.21.15.138:0/2961093513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc198ef0 0x7fcbfc199370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:12.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.261+0000 7fcc019c3700 1 -- 172.21.15.138:0/2961093513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc1a1a20 con 0x7fcbe80035c0 2024-06-10T00:22:12.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.262+0000 7fcc019c3700 1 --2- 172.21.15.138:0/2961093513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbe80035c0 0x7fcbfc1989b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0059d0 tx=0x7fcbec0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:22:12.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.262+0000 7fcbf2ffd700 1 -- 172.21.15.138:0/2961093513 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fcbec004b20 con 0x7fcbe80035c0 2024-06-10T00:22:12.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.262+0000 7fcbf2ffd700 1 -- 172.21.15.138:0/2961093513 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbec013e40 con 0x7fcbe80035c0 2024-06-10T00:22:12.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.262+0000 7fcc03c27700 1 -- 172.21.15.138:0/2961093513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbfc1a1bf0 con 0x7fcbe80035c0 2024-06-10T00:22:12.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.262+0000 7fcbf2ffd700 1 -- 172.21.15.138:0/2961093513 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fcbec011cd0 con 0x7fcbe80035c0 2024-06-10T00:22:12.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.262+0000 7fcc03c27700 1 -- 172.21.15.138:0/2961093513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbfc1a2120 con 0x7fcbe80035c0 2024-06-10T00:22:12.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.263+0000 7fcbf2ffd700 1 -- 172.21.15.138:0/2961093513 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fcbec023070 con 0x7fcbe80035c0 2024-06-10T00:22:12.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.263+0000 7fcbf2ffd700 1 --2- 172.21.15.138:0/2961093513 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcbe0038f30 0x7fcbe003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:12.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.263+0000 7fcc03c27700 1 -- 172.21.15.138:0/2961093513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbfc1025e0 con 0x7fcbe80035c0 2024-06-10T00:22:12.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.263+0000 7fcbf2ffd700 1 -- 172.21.15.138:0/2961093513 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcbec020050 con 0x7fcbe80035c0 2024-06-10T00:22:12.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.263+0000 7fcc011c2700 1 --2- 172.21.15.138:0/2961093513 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcbe0038f30 0x7fcbe003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:12.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.264+0000 7fcc011c2700 1 --2- 172.21.15.138:0/2961093513 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcbe0038f30 0x7fcbe003b3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009180 tx=0x7fcbf8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:22:12.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.269+0000 7fcbf2ffd700 1 -- 172.21.15.138:0/2961093513 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcbec031020 con 0x7fcbe80035c0 2024-06-10T00:22:12.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:12.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:22:12.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:12.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:11 smithi062 ceph-mon[32390]: Deploying daemon mon.b on smithi105 2024-06-10T00:22:12.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:12.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.551+0000 7fcc03c27700 1 -- 172.21.15.138:0/2961093513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcbfc02dca0 con 0x7fcbe80035c0 2024-06-10T00:22:12.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.552+0000 7fcbf2ffd700 1 -- 172.21.15.138:0/2961093513 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fcbec028030 con 0x7fcbe80035c0 2024-06-10T00:22:12.552 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:22:12.552 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":2,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","modified":"2024-06-10T00:22:05.911849Z","created":"2024-06-10T00:20:24.460529Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-10T00:22:12.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.554+0000 7fcc03c27700 1 -- 172.21.15.138:0/2961093513 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcbe0038f30 msgr2=0x7fcbe003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:12.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.554+0000 7fcc03c27700 1 --2- 172.21.15.138:0/2961093513 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcbe0038f30 0x7fcbe003b3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009180 tx=0x7fcbf8008040 comp rx=0 tx=0).stop 2024-06-10T00:22:12.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.554+0000 7fcc03c27700 1 -- 172.21.15.138:0/2961093513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbe80035c0 msgr2=0x7fcbfc1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:12.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.554+0000 7fcc03c27700 1 --2- 172.21.15.138:0/2961093513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbe80035c0 0x7fcbfc1989b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0059d0 tx=0x7fcbec0057d0 comp rx=0 tx=0).stop 2024-06-10T00:22:12.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.554+0000 7fcc03c27700 1 -- 172.21.15.138:0/2961093513 shutdown_connections 2024-06-10T00:22:12.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.554+0000 7fcc03c27700 1 --2- 172.21.15.138:0/2961093513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbe80035c0 0x7fcbfc1989b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:12.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.554+0000 7fcc03c27700 1 --2- 172.21.15.138:0/2961093513 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcbe0038f30 0x7fcbe003b3f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:12.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.554+0000 7fcc03c27700 1 --2- 172.21.15.138:0/2961093513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc198ef0 0x7fcbfc199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:12.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.554+0000 7fcc03c27700 1 -- 172.21.15.138:0/2961093513 >> 172.21.15.138:0/2961093513 conn(0x7fcbfc0fc990 msgr2=0x7fcbfc0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:22:12.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.554+0000 7fcc03c27700 1 -- 172.21.15.138:0/2961093513 shutdown_connections 2024-06-10T00:22:12.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:12.554+0000 7fcc03c27700 1 -- 172.21.15.138:0/2961093513 wait complete. 2024-06-10T00:22:12.556 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 2 2024-06-10T00:22:13.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:12 smithi138 ceph-mon[36549]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:13.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:12 smithi138 ceph-mon[36549]: from='client.? 172.21.15.138:0/2961093513' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:22:13.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:12 smithi062 ceph-mon[32390]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:13.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:12 smithi062 ceph-mon[32390]: from='client.? 172.21.15.138:0/2961093513' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:22:14.196 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-10T00:22:14.196 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-10T00:22:14.442 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:22:15.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:14 smithi105 systemd[1]: Started Ceph mon.b for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:22:20.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:22:20.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: mon.a calling monitor election 2024-06-10T00:22:20.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: mon.c calling monitor election 2024-06-10T00:22:20.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:20.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:20.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: mon.b calling monitor election 2024-06-10T00:22:20.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:20.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-10T00:22:20.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: monmap epoch 3 2024-06-10T00:22:20.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:22:20.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: last_changed 2024-06-10T00:22:14.883231+0000 2024-06-10T00:22:20.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: created 2024-06-10T00:20:24.460529+0000 2024-06-10T00:22:20.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: min_mon_release 19 (squid) 2024-06-10T00:22:20.284 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: election_strategy: 1 2024-06-10T00:22:20.284 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-06-10T00:22:20.284 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-06-10T00:22:20.284 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-06-10T00:22:20.284 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: fsmap 2024-06-10T00:22:20.284 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:22:20.284 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: mgrmap e13: x(active, since 58s) 2024-06-10T00:22:20.284 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: overall HEALTH_OK 2024-06-10T00:22:20.284 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:20.284 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:20.284 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:20.284 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.347+0000 7f6ebb289700 1 -- 172.21.15.138:0/2803749769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb41031d0 msgr2=0x7f6ea0005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.347+0000 7f6ebb289700 1 --2- 172.21.15.138:0/2803749769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb41031d0 0x7f6ea0005610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0005fa0 tx=0x7f6eb000ff70 comp rx=0 tx=0).stop 2024-06-10T00:22:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.348+0000 7f6ebb289700 1 -- 172.21.15.138:0/2803749769 shutdown_connections 2024-06-10T00:22:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.348+0000 7f6ebb289700 1 --2- 172.21.15.138:0/2803749769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb41031d0 0x7f6ea0005610 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.348+0000 7f6ebb289700 1 -- 172.21.15.138:0/2803749769 >> 172.21.15.138:0/2803749769 conn(0x7f6eb40fc940 msgr2=0x7f6eb40fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:22:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.348+0000 7f6ebb289700 1 -- 172.21.15.138:0/2803749769 shutdown_connections 2024-06-10T00:22:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.348+0000 7f6ebb289700 1 -- 172.21.15.138:0/2803749769 wait complete. 2024-06-10T00:22:20.349 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:22:20.349 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: mon.a calling monitor election 2024-06-10T00:22:20.349 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: mon.c calling monitor election 2024-06-10T00:22:20.349 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.349 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:20.349 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:20.349 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.350 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.350 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: mon.b calling monitor election 2024-06-10T00:22:20.350 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:20.350 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.350 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.350 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.350 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-10T00:22:20.350 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: monmap epoch 3 2024-06-10T00:22:20.350 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:22:20.350 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: last_changed 2024-06-10T00:22:14.883231+0000 2024-06-10T00:22:20.350 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: created 2024-06-10T00:20:24.460529+0000 2024-06-10T00:22:20.350 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: min_mon_release 19 (squid) 2024-06-10T00:22:20.351 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: election_strategy: 1 2024-06-10T00:22:20.351 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-06-10T00:22:20.351 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-06-10T00:22:20.351 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-06-10T00:22:20.351 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: fsmap 2024-06-10T00:22:20.351 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:22:20.351 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: mgrmap e13: x(active, since 58s) 2024-06-10T00:22:20.351 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: overall HEALTH_OK 2024-06-10T00:22:20.351 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:20.351 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:20.351 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:20.351 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.348+0000 7f6ebb289700 1 Processor -- start 2024-06-10T00:22:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.348+0000 7f6ebb289700 1 -- start start 2024-06-10T00:22:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6ebb289700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb41031d0 0x7f6eb419d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6ebb289700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb419dcb0 0x7f6eb4197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6ebb289700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4197e70 0x7f6eb4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6ebb289700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb4109f90 con 0x7f6eb41031d0 2024-06-10T00:22:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6ebb289700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb4109e10 con 0x7f6eb419dcb0 2024-06-10T00:22:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6ebb289700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb410a110 con 0x7f6eb4197e70 2024-06-10T00:22:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb9826700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4197e70 0x7f6eb4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb9826700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4197e70 0x7f6eb4198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48960/0 (socket says 172.21.15.138:48960) 2024-06-10T00:22:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb9826700 1 -- 172.21.15.138:0/2631559007 learned_addr learned my addr 172.21.15.138:0/2631559007 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-06-10T00:22:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb8824700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb419dcb0 0x7f6eb4197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb9025700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb41031d0 0x7f6eb419d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb9826700 1 -- 172.21.15.138:0/2631559007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4197e70 msgr2=0x7f6eb4198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-06-10T00:22:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb9826700 1 -- 172.21.15.138:0/2631559007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4197e70 msgr2=0x7f6eb4198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-10T00:22:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb9826700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4197e70 0x7f6eb4198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-10T00:22:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb9826700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4197e70 0x7f6eb4198320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:22:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb8824700 1 -- 172.21.15.138:0/2631559007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb419dcb0 msgr2=0x7f6eb4197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-10T00:22:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb8824700 1 -- 172.21.15.138:0/2631559007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb419dcb0 msgr2=0x7f6eb4197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-10T00:22:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb8824700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb419dcb0 0x7f6eb4197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-10T00:22:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb8824700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb419dcb0 0x7f6eb4197930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:22:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb9025700 1 -- 172.21.15.138:0/2631559007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4197e70 msgr2=0x7f6eb4198320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:22:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb9025700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4197e70 0x7f6eb4198320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb9025700 1 -- 172.21.15.138:0/2631559007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb419dcb0 msgr2=0x7f6eb4197930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:22:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb9025700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb419dcb0 0x7f6eb4197930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.349+0000 7f6eb9025700 1 -- 172.21.15.138:0/2631559007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb41a4640 con 0x7f6eb41031d0 2024-06-10T00:22:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.350+0000 7f6eb9025700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb41031d0 0x7f6eb419d770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0020040 tx=0x7f6eb0014e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:22:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.350+0000 7f6eaa7fc700 1 -- 172.21.15.138:0/2631559007 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb002b070 con 0x7f6eb41031d0 2024-06-10T00:22:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.350+0000 7f6eaa7fc700 1 -- 172.21.15.138:0/2631559007 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb001dc90 con 0x7f6eb41031d0 2024-06-10T00:22:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.350+0000 7f6ebb289700 1 -- 172.21.15.138:0/2631559007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb41a4810 con 0x7f6eb41031d0 2024-06-10T00:22:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.350+0000 7f6eaa7fc700 1 -- 172.21.15.138:0/2631559007 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0022040 con 0x7f6eb41031d0 2024-06-10T00:22:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.350+0000 7f6ebb289700 1 -- 172.21.15.138:0/2631559007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb41a4d50 con 0x7f6eb41031d0 2024-06-10T00:22:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.351+0000 7f6eaa7fc700 1 -- 172.21.15.138:0/2631559007 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6eb0014750 con 0x7f6eb41031d0 2024-06-10T00:22:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.351+0000 7f6ebb289700 1 -- 172.21.15.138:0/2631559007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb4063af0 con 0x7f6eb41031d0 2024-06-10T00:22:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.351+0000 7f6eaa7fc700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6e98039580 0x7f6e9803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.351+0000 7f6eaa7fc700 1 -- 172.21.15.138:0/2631559007 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6eb004e9c0 con 0x7f6eb41031d0 2024-06-10T00:22:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.352+0000 7f6eb8824700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6e98039580 0x7f6e9803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.352+0000 7f6eb8824700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6e98039580 0x7f6e9803ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009850 tx=0x7f6ea4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:22:20.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:22:20.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: mon.a calling monitor election 2024-06-10T00:22:20.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: mon.c calling monitor election 2024-06-10T00:22:20.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:22:20.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:20.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: mon.b calling monitor election 2024-06-10T00:22:20.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:20.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:20.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-10T00:22:20.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: monmap epoch 3 2024-06-10T00:22:20.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:22:20.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: last_changed 2024-06-10T00:22:14.883231+0000 2024-06-10T00:22:20.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: created 2024-06-10T00:20:24.460529+0000 2024-06-10T00:22:20.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: min_mon_release 19 (squid) 2024-06-10T00:22:20.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: election_strategy: 1 2024-06-10T00:22:20.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-06-10T00:22:20.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-06-10T00:22:20.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-06-10T00:22:20.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: fsmap 2024-06-10T00:22:20.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:22:20.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: mgrmap e13: x(active, since 58s) 2024-06-10T00:22:20.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: overall HEALTH_OK 2024-06-10T00:22:20.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:20.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:20.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:20.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:20.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.357+0000 7f6eaa7fc700 1 -- 172.21.15.138:0/2631559007 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6eb0026370 con 0x7f6eb41031d0 2024-06-10T00:22:20.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.647+0000 7f6ebb289700 1 -- 172.21.15.138:0/2631559007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6eb41a4fe0 con 0x7f6eb41031d0 2024-06-10T00:22:20.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.648+0000 7f6eaa7fc700 1 -- 172.21.15.138:0/2631559007 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f6eb0011840 con 0x7f6eb41031d0 2024-06-10T00:22:20.648 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-06-10T00:22:20.648 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":3,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","modified":"2024-06-10T00:22:14.883231Z","created":"2024-06-10T00:20:24.460529Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-06-10T00:22:20.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.650+0000 7f6ebb289700 1 -- 172.21.15.138:0/2631559007 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6e98039580 msgr2=0x7f6e9803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:20.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.650+0000 7f6ebb289700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6e98039580 0x7f6e9803ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009850 tx=0x7f6ea4008040 comp rx=0 tx=0).stop 2024-06-10T00:22:20.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.650+0000 7f6ebb289700 1 -- 172.21.15.138:0/2631559007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb41031d0 msgr2=0x7f6eb419d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:20.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.650+0000 7f6ebb289700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb41031d0 0x7f6eb419d770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0020040 tx=0x7f6eb0014e60 comp rx=0 tx=0).stop 2024-06-10T00:22:20.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.650+0000 7f6ebb289700 1 -- 172.21.15.138:0/2631559007 shutdown_connections 2024-06-10T00:22:20.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.650+0000 7f6ebb289700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6eb41031d0 0x7f6eb419d770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:20.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.650+0000 7f6ebb289700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6e98039580 0x7f6e9803ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:20.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.650+0000 7f6ebb289700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4197e70 0x7f6eb4198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:20.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.650+0000 7f6ebb289700 1 --2- 172.21.15.138:0/2631559007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6eb419dcb0 0x7f6eb4197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:20.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.650+0000 7f6ebb289700 1 -- 172.21.15.138:0/2631559007 >> 172.21.15.138:0/2631559007 conn(0x7f6eb40fc940 msgr2=0x7f6eb40fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:22:20.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.650+0000 7f6ebb289700 1 -- 172.21.15.138:0/2631559007 shutdown_connections 2024-06-10T00:22:20.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:20.650+0000 7f6ebb289700 1 -- 172.21.15.138:0/2631559007 wait complete. 2024-06-10T00:22:20.651 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 3 2024-06-10T00:22:21.164 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-10T00:22:21.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-10T00:22:21.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:20 smithi105 ceph-mon[36321]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:21.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:20 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:21.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:20 smithi105 ceph-mon[36321]: from='client.? 172.21.15.138:0/2631559007' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:22:21.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:20 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:21.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:20 smithi062 ceph-mon[32390]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:21.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:20 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:21.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:20 smithi062 ceph-mon[32390]: from='client.? 172.21.15.138:0/2631559007' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:22:21.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:20 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:21.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:20 smithi138 ceph-mon[36549]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:21.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:20 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:21.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:20 smithi138 ceph-mon[36549]: from='client.? 172.21.15.138:0/2631559007' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:22:21.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:20 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:22:21.422 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:22:22.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:22.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:22.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:22.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:22.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:22.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:22.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:22.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.997+0000 7f5eac2b5700 1 -- 172.21.15.62:0/773837136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea4101040 msgr2=0x7f5ea4101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:22.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.997+0000 7f5eac2b5700 1 --2- 172.21.15.62:0/773837136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea4101040 0x7f5ea4101420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009910 tx=0x7f5ea0009c20 comp rx=0 tx=0).stop 2024-06-10T00:22:22.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.997+0000 7f5eac2b5700 1 -- 172.21.15.62:0/773837136 shutdown_connections 2024-06-10T00:22:22.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.997+0000 7f5eac2b5700 1 --2- 172.21.15.62:0/773837136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea4101040 0x7f5ea4101420 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:22.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.997+0000 7f5eac2b5700 1 -- 172.21.15.62:0/773837136 >> 172.21.15.62:0/773837136 conn(0x7f5ea40768f0 msgr2=0x7f5ea4076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:22:22.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.997+0000 7f5eac2b5700 1 -- 172.21.15.62:0/773837136 shutdown_connections 2024-06-10T00:22:22.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.997+0000 7f5eac2b5700 1 -- 172.21.15.62:0/773837136 wait complete. 2024-06-10T00:22:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eac2b5700 1 Processor -- start 2024-06-10T00:22:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eac2b5700 1 -- start start 2024-06-10T00:22:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eac2b5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea4101040 0x7f5ea419d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eac2b5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ea419ddf0 0x7f5ea4197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eac2b5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ea4197fb0 0x7f5ea4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eac2b5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea410b3c0 con 0x7f5ea4101040 2024-06-10T00:22:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eac2b5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea410b240 con 0x7f5ea419ddf0 2024-06-10T00:22:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eac2b5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea410b540 con 0x7f5ea4197fb0 2024-06-10T00:22:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eaa051700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea4101040 0x7f5ea419d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eaa051700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea4101040 0x7f5ea419d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44454/0 (socket says 172.21.15.62:44454) 2024-06-10T00:22:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eaa051700 1 -- 172.21.15.62:0/770431116 learned_addr learned my addr 172.21.15.62:0/770431116 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:22:23.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eaa852700 1 --2- 172.21.15.62:0/770431116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ea4197fb0 0x7f5ea4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eaa051700 1 -- 172.21.15.62:0/770431116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ea4197fb0 msgr2=0x7f5ea4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5ea9850700 1 --2- 172.21.15.62:0/770431116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ea419ddf0 0x7f5ea4197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eaa051700 1 --2- 172.21.15.62:0/770431116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ea4197fb0 0x7f5ea4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eaa051700 1 -- 172.21.15.62:0/770431116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ea419ddf0 msgr2=0x7f5ea4197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eaa051700 1 --2- 172.21.15.62:0/770431116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ea419ddf0 0x7f5ea4197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.998+0000 7f5eaa051700 1 -- 172.21.15.62:0/770431116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea41a4780 con 0x7f5ea4101040 2024-06-10T00:22:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.999+0000 7f5eaa051700 1 --2- 172.21.15.62:0/770431116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea4101040 0x7f5ea419d8b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0005850 tx=0x7f5ea0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:22:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.999+0000 7f5e9b7fe700 1 -- 172.21.15.62:0/770431116 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0026070 con 0x7f5ea4101040 2024-06-10T00:22:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.999+0000 7f5eac2b5700 1 -- 172.21.15.62:0/770431116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea41a49b0 con 0x7f5ea4101040 2024-06-10T00:22:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.999+0000 7f5e9b7fe700 1 -- 172.21.15.62:0/770431116 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ea000b6f0 con 0x7f5ea4101040 2024-06-10T00:22:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.999+0000 7f5e9b7fe700 1 -- 172.21.15.62:0/770431116 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea00039e0 con 0x7f5ea4101040 2024-06-10T00:22:23.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:22.999+0000 7f5eac2b5700 1 -- 172.21.15.62:0/770431116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea41a4f40 con 0x7f5ea4101040 2024-06-10T00:22:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.000+0000 7f5e9b7fe700 1 -- 172.21.15.62:0/770431116 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5ea0029070 con 0x7f5ea4101040 2024-06-10T00:22:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.000+0000 7f5eac2b5700 1 -- 172.21.15.62:0/770431116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea4063af0 con 0x7f5ea4101040 2024-06-10T00:22:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.000+0000 7f5e9b7fe700 1 --2- 172.21.15.62:0/770431116 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f5e90039580 0x7f5e9003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.000+0000 7f5e9b7fe700 1 -- 172.21.15.62:0/770431116 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5ea004ce30 con 0x7f5ea4101040 2024-06-10T00:22:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.000+0000 7f5ea9850700 1 --2- 172.21.15.62:0/770431116 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f5e90039580 0x7f5e9003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:23.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.001+0000 7f5ea9850700 1 --2- 172.21.15.62:0/770431116 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f5e90039580 0x7f5e9003ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5e940078e0 tx=0x7f5e94008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:22:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.005+0000 7f5e9b7fe700 1 -- 172.21.15.62:0/770431116 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5ea0013070 con 0x7f5ea4101040 2024-06-10T00:22:23.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.215+0000 7f5eac2b5700 1 -- 172.21.15.62:0/770431116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f5ea41a5170 con 0x7f5ea4101040 2024-06-10T00:22:23.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.215+0000 7f5e9b7fe700 1 -- 172.21.15.62:0/770431116 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f5ea0030070 con 0x7f5ea4101040 2024-06-10T00:22:23.215 INFO:teuthology.orchestra.run.smithi062.stdout:# minimal ceph.conf for 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:22:23.216 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2024-06-10T00:22:23.216 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:22:23.216 INFO:teuthology.orchestra.run.smithi062.stdout: mon_host = [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] 2024-06-10T00:22:23.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.217+0000 7f5eac2b5700 1 -- 172.21.15.62:0/770431116 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f5e90039580 msgr2=0x7f5e9003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:23.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.217+0000 7f5eac2b5700 1 --2- 172.21.15.62:0/770431116 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f5e90039580 0x7f5e9003ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5e940078e0 tx=0x7f5e94008040 comp rx=0 tx=0).stop 2024-06-10T00:22:23.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.217+0000 7f5eac2b5700 1 -- 172.21.15.62:0/770431116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea4101040 msgr2=0x7f5ea419d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:23.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.217+0000 7f5eac2b5700 1 --2- 172.21.15.62:0/770431116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea4101040 0x7f5ea419d8b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0005850 tx=0x7f5ea0005030 comp rx=0 tx=0).stop 2024-06-10T00:22:23.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.218+0000 7f5eac2b5700 1 -- 172.21.15.62:0/770431116 shutdown_connections 2024-06-10T00:22:23.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.218+0000 7f5eac2b5700 1 --2- 172.21.15.62:0/770431116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ea4101040 0x7f5ea419d8b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:23.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.218+0000 7f5eac2b5700 1 --2- 172.21.15.62:0/770431116 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f5e90039580 0x7f5e9003ba40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:23.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.218+0000 7f5eac2b5700 1 --2- 172.21.15.62:0/770431116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ea4197fb0 0x7f5ea4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:23.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.218+0000 7f5eac2b5700 1 --2- 172.21.15.62:0/770431116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ea419ddf0 0x7f5ea4197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:23.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.218+0000 7f5eac2b5700 1 -- 172.21.15.62:0/770431116 >> 172.21.15.62:0/770431116 conn(0x7f5ea40768f0 msgr2=0x7f5ea40698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:22:23.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.218+0000 7f5eac2b5700 1 -- 172.21.15.62:0/770431116 shutdown_connections 2024-06-10T00:22:23.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:23.218+0000 7f5eac2b5700 1 -- 172.21.15.62:0/770431116 wait complete. 2024-06-10T00:22:23.601 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: Updating smithi062:/etc/ceph/ceph.conf 2024-06-10T00:22:23.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: Updating smithi105:/etc/ceph/ceph.conf 2024-06-10T00:22:23.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: Updating smithi138:/etc/ceph/ceph.conf 2024-06-10T00:22:23.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:23.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: Updating smithi138:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:22:23.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: Updating smithi062:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:22:23.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: Updating smithi105:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:22:23.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: Reconfiguring mon.a (unknown last config time)... 2024-06-10T00:22:23.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:22:23.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:22:23.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:23.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: Reconfiguring daemon mon.a on smithi062 2024-06-10T00:22:23.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:23 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/770431116' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: Updating smithi062:/etc/ceph/ceph.conf 2024-06-10T00:22:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: Updating smithi105:/etc/ceph/ceph.conf 2024-06-10T00:22:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: Updating smithi138:/etc/ceph/ceph.conf 2024-06-10T00:22:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: Updating smithi138:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:22:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: Updating smithi062:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:22:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: Updating smithi105:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:22:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: Reconfiguring mon.a (unknown last config time)... 2024-06-10T00:22:23.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:22:23.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:22:23.784 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:23.784 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: Reconfiguring daemon mon.a on smithi062 2024-06-10T00:22:23.784 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:23 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/770431116' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: Updating smithi062:/etc/ceph/ceph.conf 2024-06-10T00:22:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: Updating smithi105:/etc/ceph/ceph.conf 2024-06-10T00:22:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: Updating smithi138:/etc/ceph/ceph.conf 2024-06-10T00:22:23.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:23.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: Updating smithi138:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:22:23.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: Updating smithi062:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:22:23.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: Updating smithi105:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:22:23.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.879 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.879 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.879 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.879 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.880 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.880 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.880 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.880 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:23.880 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: Reconfiguring mon.a (unknown last config time)... 2024-06-10T00:22:23.880 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:22:23.880 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:22:23.880 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:23.880 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: Reconfiguring daemon mon.a on smithi062 2024-06-10T00:22:23.880 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:23 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/770431116' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:24.674 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-10T00:22:24.675 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:22:24.675 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.conf 2024-06-10T00:22:24.708 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:22:24.709 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:22:24.782 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-06-10T00:22:24.782 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.conf 2024-06-10T00:22:24.812 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-06-10T00:22:24.812 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:22:24.877 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-06-10T00:22:24.877 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.conf 2024-06-10T00:22:24.909 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-06-10T00:22:24.909 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:22:24.981 INFO:tasks.cephadm:Adding mgr.x on smithi062 2024-06-10T00:22:24.981 INFO:tasks.cephadm:Adding mgr.y on smithi105 2024-06-10T00:22:24.981 INFO:tasks.cephadm:Adding mgr.z on smithi138 2024-06-10T00:22:24.981 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi062=x;smithi105=y;smithi138=z' 2024-06-10T00:22:25.255 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:22:25.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:25 smithi105 ceph-mon[36321]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:25.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:25 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:25.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:25 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:25.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:25 smithi105 ceph-mon[36321]: Reconfiguring mon.b (monmap changed)... 2024-06-10T00:22:25.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:25 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:22:25.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:25 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:22:25.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:25 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:25.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:25 smithi105 ceph-mon[36321]: Reconfiguring daemon mon.b on smithi105 2024-06-10T00:22:25.571 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:25 smithi138 ceph-mon[36549]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:25.571 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:25 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:25.571 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:25 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:25.571 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:25 smithi138 ceph-mon[36549]: Reconfiguring mon.b (monmap changed)... 2024-06-10T00:22:25.571 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:25 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:22:25.571 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:25 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:22:25.571 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:25 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:25.571 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:25 smithi138 ceph-mon[36549]: Reconfiguring daemon mon.b on smithi105 2024-06-10T00:22:25.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:25 smithi062 ceph-mon[32390]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:25.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:25 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:25.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:25 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:25.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:25 smithi062 ceph-mon[32390]: Reconfiguring mon.b (monmap changed)... 2024-06-10T00:22:25.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:25 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:22:25.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:25 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:22:25.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:25 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:25.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:25 smithi062 ceph-mon[32390]: Reconfiguring daemon mon.b on smithi105 2024-06-10T00:22:26.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 -- 172.21.15.138:0/1155440044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c0100f10 msgr2=0x7f43c0101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:26.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 --2- 172.21.15.138:0/1155440044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c0100f10 0x7f43c0101330 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009910 tx=0x7f43bc009c20 comp rx=0 tx=0).stop 2024-06-10T00:22:26.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 -- 172.21.15.138:0/1155440044 shutdown_connections 2024-06-10T00:22:26.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 --2- 172.21.15.138:0/1155440044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c0100f10 0x7f43c0101330 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:26.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 -- 172.21.15.138:0/1155440044 >> 172.21.15.138:0/1155440044 conn(0x7f43c00fc4d0 msgr2=0x7f43c00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:22:26.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 -- 172.21.15.138:0/1155440044 shutdown_connections 2024-06-10T00:22:26.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 -- 172.21.15.138:0/1155440044 wait complete. 2024-06-10T00:22:26.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 Processor -- start 2024-06-10T00:22:26.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 -- start start 2024-06-10T00:22:26.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c0100f10 0x7f43c006b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:26.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43c019a5a0 0x7f43c019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:26.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c019cf20 0x7f43c019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:26.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0106ed0 con 0x7f43c0100f10 2024-06-10T00:22:26.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0106d50 con 0x7f43c019a5a0 2024-06-10T00:22:26.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.685+0000 7f43c7f22700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0107050 con 0x7f43c019cf20 2024-06-10T00:22:26.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c64bf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c019cf20 0x7f43c019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:26.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c5cbe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c0100f10 0x7f43c006b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:26.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c5cbe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c0100f10 0x7f43c006b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.138:34040/0 (socket says 172.21.15.138:34040) 2024-06-10T00:22:26.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c64bf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c019cf20 0x7f43c019f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48962/0 (socket says 172.21.15.138:48962) 2024-06-10T00:22:26.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c5cbe700 1 -- 172.21.15.138:0/981072327 learned_addr learned my addr 172.21.15.138:0/981072327 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-06-10T00:22:26.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c54bd700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43c019a5a0 0x7f43c019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:26.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c64bf700 1 -- 172.21.15.138:0/981072327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43c019a5a0 msgr2=0x7f43c019c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:26.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c64bf700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43c019a5a0 0x7f43c019c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:26.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c64bf700 1 -- 172.21.15.138:0/981072327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c0100f10 msgr2=0x7f43c006b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:26.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c64bf700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c0100f10 0x7f43c006b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:26.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c64bf700 1 -- 172.21.15.138:0/981072327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c019fc80 con 0x7f43c019cf20 2024-06-10T00:22:26.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c54bd700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43c019a5a0 0x7f43c019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:22:26.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c5cbe700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c0100f10 0x7f43c006b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:22:26.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c64bf700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c019cf20 0x7f43c019f3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f43b800bd50 tx=0x7f43b800bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:22:26.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43b6ffd700 1 -- 172.21.15.138:0/981072327 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8019070 con 0x7f43c019cf20 2024-06-10T00:22:26.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43b6ffd700 1 -- 172.21.15.138:0/981072327 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b800cc90 con 0x7f43c019cf20 2024-06-10T00:22:26.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c7f22700 1 -- 172.21.15.138:0/981072327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c019ff70 con 0x7f43c019cf20 2024-06-10T00:22:26.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43b6ffd700 1 -- 172.21.15.138:0/981072327 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8012740 con 0x7f43c019cf20 2024-06-10T00:22:26.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.686+0000 7f43c7f22700 1 -- 172.21.15.138:0/981072327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c01a0480 con 0x7f43c019cf20 2024-06-10T00:22:26.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.687+0000 7f43b6ffd700 1 -- 172.21.15.138:0/981072327 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f43b801c070 con 0x7f43c019cf20 2024-06-10T00:22:26.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.687+0000 7f43c7f22700 1 -- 172.21.15.138:0/981072327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c0063af0 con 0x7f43c019cf20 2024-06-10T00:22:26.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.687+0000 7f43b6ffd700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f43ac039580 0x7f43ac03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:26.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.687+0000 7f43b6ffd700 1 -- 172.21.15.138:0/981072327 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f43b804b990 con 0x7f43c019cf20 2024-06-10T00:22:26.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.688+0000 7f43c5cbe700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f43ac039580 0x7f43ac03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:26.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.688+0000 7f43c5cbe700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f43ac039580 0x7f43ac03ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f43bc006010 tx=0x7f43bc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:22:26.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.693+0000 7f43b6ffd700 1 -- 172.21.15.138:0/981072327 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f43b801be10 con 0x7f43c019cf20 2024-06-10T00:22:26.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.903+0000 7f43c7f22700 1 -- 172.21.15.138:0/981072327 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi105=y;smithi138=z", "target": ["mon-mgr", ""]}) -- 0x7f43c0062a20 con 0x7f43ac039580 2024-06-10T00:22:26.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.909+0000 7f43b6ffd700 1 -- 172.21.15.138:0/981072327 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f43c0062a20 con 0x7f43ac039580 2024-06-10T00:22:26.909 INFO:teuthology.orchestra.run.smithi138.stdout:Scheduled mgr update... 2024-06-10T00:22:26.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.911+0000 7f43c7f22700 1 -- 172.21.15.138:0/981072327 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f43ac039580 msgr2=0x7f43ac03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:26.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.911+0000 7f43c7f22700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f43ac039580 0x7f43ac03ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f43bc006010 tx=0x7f43bc01a040 comp rx=0 tx=0).stop 2024-06-10T00:22:26.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.911+0000 7f43c7f22700 1 -- 172.21.15.138:0/981072327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c019cf20 msgr2=0x7f43c019f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:26.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.911+0000 7f43c7f22700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c019cf20 0x7f43c019f3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f43b800bd50 tx=0x7f43b800bd80 comp rx=0 tx=0).stop 2024-06-10T00:22:26.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.912+0000 7f43c7f22700 1 -- 172.21.15.138:0/981072327 shutdown_connections 2024-06-10T00:22:26.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.912+0000 7f43c7f22700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43c0100f10 0x7f43c006b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:26.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.912+0000 7f43c7f22700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f43ac039580 0x7f43ac03ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:26.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.912+0000 7f43c7f22700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c019cf20 0x7f43c019f3b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:26.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.912+0000 7f43c7f22700 1 --2- 172.21.15.138:0/981072327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43c019a5a0 0x7f43c019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:26.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.912+0000 7f43c7f22700 1 -- 172.21.15.138:0/981072327 >> 172.21.15.138:0/981072327 conn(0x7f43c00fc4d0 msgr2=0x7f43c00fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:22:26.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.912+0000 7f43c7f22700 1 -- 172.21.15.138:0/981072327 shutdown_connections 2024-06-10T00:22:26.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:22:26.912+0000 7f43c7f22700 1 -- 172.21.15.138:0/981072327 wait complete. 2024-06-10T00:22:27.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:26 smithi138 ceph-mon[36549]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:27.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:26 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:27.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:26 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:27.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:26 smithi138 ceph-mon[36549]: Reconfiguring mon.c (monmap changed)... 2024-06-10T00:22:27.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:26 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:22:27.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:26 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:22:27.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:26 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:27.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:26 smithi138 ceph-mon[36549]: Reconfiguring daemon mon.c on smithi138 2024-06-10T00:22:27.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:26 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:26 smithi105 ceph-mon[36321]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:26 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:26 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:26 smithi105 ceph-mon[36321]: Reconfiguring mon.c (monmap changed)... 2024-06-10T00:22:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:26 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:22:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:26 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:22:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:26 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:26 smithi105 ceph-mon[36321]: Reconfiguring daemon mon.c on smithi138 2024-06-10T00:22:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:26 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:26 smithi062 ceph-mon[32390]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:26 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:26 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:26 smithi062 ceph-mon[32390]: Reconfiguring mon.c (monmap changed)... 2024-06-10T00:22:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:26 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:22:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:26 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:22:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:26 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:26 smithi062 ceph-mon[32390]: Reconfiguring daemon mon.c on smithi138 2024-06-10T00:22:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:26 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:28.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:27 smithi138 ceph-mon[36549]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi105=y;smithi138=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:22:28.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:27 smithi138 ceph-mon[36549]: Saving service mgr spec with placement smithi062=x;smithi105=y;smithi138=z;count:3 2024-06-10T00:22:28.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:27 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:28.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:27 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:28.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:27 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:28.188 DEBUG:teuthology.orchestra.run.smithi105:mgr.y> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-10T00:22:28.191 DEBUG:teuthology.orchestra.run.smithi138:mgr.z> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mgr.z.service 2024-06-10T00:22:28.193 INFO:tasks.cephadm:Deploying OSDs... 2024-06-10T00:22:28.194 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:22:28.194 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-06-10T00:22:28.210 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-10T00:22:28.210 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-06-10T00:22:28.214 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:27 smithi105 ceph-mon[36321]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi105=y;smithi138=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:22:28.214 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:27 smithi105 ceph-mon[36321]: Saving service mgr spec with placement smithi062=x;smithi105=y;smithi138=z;count:3 2024-06-10T00:22:28.214 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:27 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:28.215 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:27 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:28.215 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:27 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:28.267 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-10T00:22:28.267 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:22:28.267 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-06-10T00:22:28.267 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:22:28.267 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:22:28.267 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-06-10 00:21:34.524757636 +0000 2024-06-10T00:22:28.267 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-06-10 00:16:17.776907864 +0000 2024-06-10T00:22:28.267 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-06-10 00:16:17.776907864 +0000 2024-06-10T00:22:28.267 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-06-10 00:16:17.776907864 +0000 2024-06-10T00:22:28.267 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-10T00:22:28.328 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:27 smithi062 ceph-mon[32390]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi105=y;smithi138=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:22:28.328 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:27 smithi062 ceph-mon[32390]: Saving service mgr spec with placement smithi062=x;smithi105=y;smithi138=z;count:3 2024-06-10T00:22:28.328 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:27 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:28.329 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:27 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:28.329 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:27 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:28.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-06-10T00:22:28.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-06-10T00:22:28.331 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000198265 s, 2.6 MB/s 2024-06-10T00:22:28.332 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-10T00:22:28.390 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-06-10T00:22:28.447 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-10T00:22:28.447 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:22:28.447 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-06-10T00:22:28.447 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:22:28.447 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:22:28.447 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-06-10 00:21:34.563757089 +0000 2024-06-10T00:22:28.447 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-06-10 00:16:18.114902997 +0000 2024-06-10T00:22:28.447 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-06-10 00:16:18.114902997 +0000 2024-06-10T00:22:28.447 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-06-10 00:16:18.114902997 +0000 2024-06-10T00:22:28.448 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-10T00:22:28.511 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-06-10T00:22:28.512 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-06-10T00:22:28.512 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000244336 s, 2.1 MB/s 2024-06-10T00:22:28.513 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-10T00:22:28.570 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-06-10T00:22:28.626 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-10T00:22:28.626 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:22:28.626 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-06-10T00:22:28.626 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:22:28.626 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:22:28.626 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-06-10 00:21:34.597756611 +0000 2024-06-10T00:22:28.626 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-06-10 00:16:18.445898231 +0000 2024-06-10T00:22:28.626 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-06-10 00:16:18.445898231 +0000 2024-06-10T00:22:28.627 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-06-10 00:16:18.445898231 +0000 2024-06-10T00:22:28.627 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-10T00:22:28.690 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-06-10T00:22:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-06-10T00:22:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.00021645 s, 2.4 MB/s 2024-06-10T00:22:28.692 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-10T00:22:28.749 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-06-10T00:22:28.805 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-10T00:22:28.805 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:22:28.805 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-10T00:22:28.805 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:22:28.805 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:22:28.805 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-06-10 00:21:34.631756133 +0000 2024-06-10T00:22:28.806 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-06-10 00:16:18.784893349 +0000 2024-06-10T00:22:28.806 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-06-10 00:16:18.784893349 +0000 2024-06-10T00:22:28.806 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-06-10 00:16:18.784893349 +0000 2024-06-10T00:22:28.806 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-10T00:22:28.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-06-10T00:22:28.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-06-10T00:22:28.870 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000189502 s, 2.7 MB/s 2024-06-10T00:22:28.871 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-10T00:22:28.928 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-06-10T00:22:28.929 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-06-10T00:22:28.947 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-10T00:22:28.947 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-06-10T00:22:29.005 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-10T00:22:29.006 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:22:29.006 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-06-10T00:22:29.006 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:22:29.006 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:22:29.006 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-06-10 00:22:14.337213406 +0000 2024-06-10T00:22:29.006 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-06-10 00:16:28.129663016 +0000 2024-06-10T00:22:29.006 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-06-10 00:16:28.129663016 +0000 2024-06-10T00:22:29.006 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-06-10 00:16:28.129663016 +0000 2024-06-10T00:22:29.006 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-10T00:22:29.072 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-06-10T00:22:29.073 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-06-10T00:22:29.073 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000311376 s, 1.6 MB/s 2024-06-10T00:22:29.074 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-10T00:22:29.132 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-06-10T00:22:29.189 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-10T00:22:29.189 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:22:29.189 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-10T00:22:29.189 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:22:29.189 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:22:29.189 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-06-10 00:22:14.338213386 +0000 2024-06-10T00:22:29.190 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-06-10 00:16:28.483655901 +0000 2024-06-10T00:22:29.190 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-06-10 00:16:28.483655901 +0000 2024-06-10T00:22:29.190 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-06-10 00:16:28.483655901 +0000 2024-06-10T00:22:29.190 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-10T00:22:29.221 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:28 smithi138 ceph-mon[36549]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:29.221 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:29.221 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:29.221 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:29.222 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:22:29.222 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:22:29.222 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:22:29.222 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:29.222 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:28 smithi138 ceph-mon[36549]: Deploying daemon mgr.z on smithi138 2024-06-10T00:22:29.251 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:28 smithi105 ceph-mon[36321]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:29.252 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:29.252 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:29.252 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:29.252 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:22:29.252 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:22:29.252 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:22:29.252 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:29.252 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:28 smithi105 ceph-mon[36321]: Deploying daemon mgr.z on smithi138 2024-06-10T00:22:29.254 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-06-10T00:22:29.254 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-06-10T00:22:29.254 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000209923 s, 2.4 MB/s 2024-06-10T00:22:29.255 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-10T00:22:29.312 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-06-10T00:22:29.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:28 smithi062 ceph-mon[32390]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:29.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:29.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:29.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:29.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:22:29.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:22:29.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:22:29.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:29.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:28 smithi062 ceph-mon[32390]: Deploying daemon mgr.z on smithi138 2024-06-10T00:22:29.368 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-10T00:22:29.368 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:22:29.369 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-06-10T00:22:29.369 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:22:29.369 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:22:29.369 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-06-10 00:22:14.338213386 +0000 2024-06-10T00:22:29.369 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-06-10 00:16:28.807649389 +0000 2024-06-10T00:22:29.369 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-06-10 00:16:28.807649389 +0000 2024-06-10T00:22:29.369 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-06-10 00:16:28.807649389 +0000 2024-06-10T00:22:29.369 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-10T00:22:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-06-10T00:22:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-06-10T00:22:29.432 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.0001934 s, 2.6 MB/s 2024-06-10T00:22:29.433 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-10T00:22:29.490 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-06-10T00:22:29.545 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-10T00:22:29.545 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:22:29.546 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-06-10T00:22:29.546 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:22:29.546 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:22:29.546 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-06-10 00:22:14.339213367 +0000 2024-06-10T00:22:29.546 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-06-10 00:16:29.139642716 +0000 2024-06-10T00:22:29.546 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-06-10 00:16:29.139642716 +0000 2024-06-10T00:22:29.546 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-06-10 00:16:29.139642716 +0000 2024-06-10T00:22:29.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-10T00:22:29.607 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-06-10T00:22:29.608 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-06-10T00:22:29.608 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000218729 s, 2.3 MB/s 2024-06-10T00:22:29.609 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-10T00:22:29.664 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-06-10T00:22:29.665 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-06-10T00:22:29.680 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-10T00:22:29.680 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-06-10T00:22:29.735 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-10T00:22:29.735 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:22:29.735 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-06-10T00:22:29.735 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:22:29.736 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:22:29.736 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-06-10 00:22:04.825681264 +0000 2024-06-10T00:22:29.736 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-06-10 00:16:22.056054491 +0000 2024-06-10T00:22:29.736 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-06-10 00:16:22.056054491 +0000 2024-06-10T00:22:29.736 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-06-10 00:16:22.056054491 +0000 2024-06-10T00:22:29.736 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-10T00:22:29.798 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-06-10T00:22:29.799 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-06-10T00:22:29.799 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000189644 s, 2.7 MB/s 2024-06-10T00:22:29.800 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-10T00:22:29.855 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-06-10T00:22:29.909 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-10T00:22:29.909 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:22:29.909 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-06-10T00:22:29.909 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:22:29.909 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:22:29.910 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-06-10 00:22:04.826681245 +0000 2024-06-10T00:22:29.910 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-06-10 00:16:22.370048185 +0000 2024-06-10T00:22:29.910 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-06-10 00:16:22.370048185 +0000 2024-06-10T00:22:29.910 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-06-10 00:16:22.370048185 +0000 2024-06-10T00:22:29.910 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-10T00:22:29.972 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-06-10T00:22:29.972 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-06-10T00:22:29.972 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000190204 s, 2.7 MB/s 2024-06-10T00:22:29.973 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-10T00:22:30.036 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-06-10T00:22:30.091 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-10T00:22:30.091 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:22:30.091 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-06-10T00:22:30.091 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:22:30.092 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:22:30.092 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-06-10 00:22:04.826681245 +0000 2024-06-10T00:22:30.092 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-06-10 00:16:22.696041637 +0000 2024-06-10T00:22:30.092 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-06-10 00:16:22.696041637 +0000 2024-06-10T00:22:30.092 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-06-10 00:16:22.696041637 +0000 2024-06-10T00:22:30.092 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-10T00:22:30.158 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-06-10T00:22:30.158 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-06-10T00:22:30.159 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000182119 s, 2.8 MB/s 2024-06-10T00:22:30.159 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-10T00:22:30.217 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-06-10T00:22:30.272 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-10T00:22:30.272 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:22:30.272 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-06-10T00:22:30.272 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:22:30.272 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:22:30.272 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-06-10 00:22:30.097194104 +0000 2024-06-10T00:22:30.272 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-06-10 00:16:23.017035190 +0000 2024-06-10T00:22:30.272 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-06-10 00:16:23.017035190 +0000 2024-06-10T00:22:30.273 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-06-10 00:16:23.017035190 +0000 2024-06-10T00:22:30.273 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-10T00:22:30.338 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-06-10T00:22:30.338 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-06-10T00:22:30.338 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000182047 s, 2.8 MB/s 2024-06-10T00:22:30.339 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-10T00:22:30.395 INFO:tasks.cephadm:Deploying osd.0 on smithi062 with /dev/vg_nvme/lv_4... 2024-06-10T00:22:30.395 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-10T00:22:30.647 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:22:30.807 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:30 smithi138 podman[38365]: 2024-06-10 00:22:30.782998888 +0000 UTC m=+0.345699991 container start 2cfb57db6f25b458cf84de5df70405db286c4bdad9f6d4e33babe47ad2cd2d68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-10T00:22:30.807 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:30 smithi138 ceph-mgr[38384]: -- 172.21.15.138:0/96346810 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ce106e5a0 con 0x561ce1042c00 2024-06-10T00:22:31.077 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:30 smithi062 ceph-mon[32390]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:31.077 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:30 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:31.077 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:30 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:31.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:30 smithi138 ceph-mon[36549]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:31.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:30 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:31.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:30 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:31.123 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:30 smithi138 bash[38365]: 2cfb57db6f25b458cf84de5df70405db286c4bdad9f6d4e33babe47ad2cd2d68 2024-06-10T00:22:31.123 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:30 smithi138 systemd[1]: Started Ceph mgr.z for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:22:31.123 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:30 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:30.942+0000 7fb51d1c2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:22:31.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:30 smithi105 ceph-mon[36321]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:31.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:30 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:31.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:30 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:31.533 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:31 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:31.126+0000 7fb51d1c2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:22:31.533 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:31 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:31.226+0000 7fb51d1c2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:22:31.533 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:31 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:31.532+0000 7fb51d1c2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:22:31.873 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:31 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:31.618+0000 7fb51d1c2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:22:32.235 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:32.235 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:22:32.235 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:22:32.235 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:22:32.235 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:32.235 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:31 smithi105 ceph-mon[36321]: Deploying daemon mgr.y on smithi105 2024-06-10T00:22:32.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:32.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:22:32.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:22:32.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:22:32.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:32.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:31 smithi062 ceph-mon[32390]: Deploying daemon mgr.y on smithi105 2024-06-10T00:22:32.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:32.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:22:32.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:22:32.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:22:32.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:32.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:31 smithi138 ceph-mon[36549]: Deploying daemon mgr.y on smithi105 2024-06-10T00:22:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-10T00:22:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-10T00:22:32.516 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-06-10T00:22:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-06-10T00:22:32.516 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130592 s, 803 MB/s 2024-06-10T00:22:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-06-10T00:22:33.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:32 smithi105 ceph-mon[36321]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:33.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:32 smithi062 ceph-mon[32390]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:33.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:32 smithi138 ceph-mon[36549]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:33.123 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:32 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:32.840+0000 7fb51d1c2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:22:33.123 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:33.013+0000 7fb51d1c2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:22:33.131 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_4 2024-06-10T00:22:33.379 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:22:33.464 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:33 smithi105 ceph-mgr[37698]: -- 172.21.15.105:0/731622010 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558eb39b65a0 con 0x558eb398ac00 2024-06-10T00:22:33.485 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:33.189+0000 7fb51d1c2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:22:33.485 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:33.485+0000 7fb51d1c2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:22:33.762 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:33 smithi105 bash[37677]: 313819fe36215bef933cce694698db55433f08183a740565c647092bc992039d 2024-06-10T00:22:33.762 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:33 smithi105 systemd[1]: Started Ceph mgr.y for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:22:33.762 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:33.564+0000 7f935f03f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:22:33.762 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:33.762+0000 7f935f03f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:22:33.799 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:33.579+0000 7fb51d1c2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:22:33.799 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:33.799+0000 7fb51d1c2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:22:34.031 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:33.876+0000 7f935f03f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:22:34.123 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:34 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:34.003+0000 7fb51d1c2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:22:34.524 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:34 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:34.210+0000 7f935f03f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:22:34.525 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:34 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:34.319+0000 7f935f03f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:22:34.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:34.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:34.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:34.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:34.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:34.828 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:34.829 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:34.829 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:34.829 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:34.829 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:34.873 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:34 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:34.648+0000 7fb51d1c2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:22:34.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:34.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:34.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:34.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:34.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:34.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.935+0000 7f7bbe11e700 1 -- 172.21.15.62:0/328711419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb8102f90 msgr2=0x7f7bb8109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:34.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.935+0000 7f7bbe11e700 1 --2- 172.21.15.62:0/328711419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb8102f90 0x7f7bb8109840 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000b0d0 tx=0x7f7bb000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:22:34.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.935+0000 7f7bbe11e700 1 -- 172.21.15.62:0/328711419 shutdown_connections 2024-06-10T00:22:34.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.935+0000 7f7bbe11e700 1 --2- 172.21.15.62:0/328711419 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb8102f90 0x7f7bb8109840 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:34.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.935+0000 7f7bbe11e700 1 --2- 172.21.15.62:0/328711419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb81013f0 0x7f7bb8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:34.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.935+0000 7f7bbe11e700 1 --2- 172.21.15.62:0/328711419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bb81025f0 0x7f7bb8102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:34.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.935+0000 7f7bbe11e700 1 -- 172.21.15.62:0/328711419 >> 172.21.15.62:0/328711419 conn(0x7f7bb80fc990 msgr2=0x7f7bb80fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:22:34.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.935+0000 7f7bbe11e700 1 -- 172.21.15.62:0/328711419 shutdown_connections 2024-06-10T00:22:34.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.935+0000 7f7bbe11e700 1 -- 172.21.15.62:0/328711419 wait complete. 2024-06-10T00:22:34.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.936+0000 7f7bbe11e700 1 Processor -- start 2024-06-10T00:22:34.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.936+0000 7f7bbe11e700 1 -- start start 2024-06-10T00:22:34.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.936+0000 7f7bbe11e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bb81013f0 0x7f7bb806acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.936+0000 7f7bbe11e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb81025f0 0x7f7bb819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.936+0000 7f7bbe11e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb8102f90 0x7f7bb819fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.936+0000 7f7bbe11e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb810cfb0 con 0x7f7bb81025f0 2024-06-10T00:22:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.936+0000 7f7bbe11e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb810ce30 con 0x7f7bb81013f0 2024-06-10T00:22:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.936+0000 7f7bbe11e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb810d130 con 0x7f7bb8102f90 2024-06-10T00:22:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.936+0000 7f7baeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb81025f0 0x7f7bb819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.936+0000 7f7baeffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb81025f0 0x7f7bb819d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41862/0 (socket says 172.21.15.62:41862) 2024-06-10T00:22:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.936+0000 7f7baeffd700 1 -- 172.21.15.62:0/1742809463 learned_addr learned my addr 172.21.15.62:0/1742809463 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:22:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.936+0000 7f7bb77fe700 1 --2- 172.21.15.62:0/1742809463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bb81013f0 0x7f7bb806acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.936+0000 7f7bb7fff700 1 --2- 172.21.15.62:0/1742809463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb8102f90 0x7f7bb819fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.937+0000 7f7baeffd700 1 -- 172.21.15.62:0/1742809463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb8102f90 msgr2=0x7f7bb819fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:34.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.937+0000 7f7baeffd700 1 --2- 172.21.15.62:0/1742809463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb8102f90 0x7f7bb819fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.937+0000 7f7baeffd700 1 -- 172.21.15.62:0/1742809463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bb81013f0 msgr2=0x7f7bb806acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.937+0000 7f7baeffd700 1 --2- 172.21.15.62:0/1742809463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bb81013f0 0x7f7bb806acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.937+0000 7f7baeffd700 1 -- 172.21.15.62:0/1742809463 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb81a0330 con 0x7f7bb81025f0 2024-06-10T00:22:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.937+0000 7f7baeffd700 1 --2- 172.21.15.62:0/1742809463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb81025f0 0x7f7bb819d420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000d950 tx=0x7f7ba000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:22:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.937+0000 7f7bb57fa700 1 -- 172.21.15.62:0/1742809463 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0016070 con 0x7f7bb81025f0 2024-06-10T00:22:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.937+0000 7f7bb57fa700 1 -- 172.21.15.62:0/1742809463 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba000fd70 con 0x7f7bb81025f0 2024-06-10T00:22:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.937+0000 7f7bbe11e700 1 -- 172.21.15.62:0/1742809463 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb81a0620 con 0x7f7bb81025f0 2024-06-10T00:22:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.937+0000 7f7bb57fa700 1 -- 172.21.15.62:0/1742809463 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba000e610 con 0x7f7bb81025f0 2024-06-10T00:22:34.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.937+0000 7f7bbe11e700 1 -- 172.21.15.62:0/1742809463 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb81a0bb0 con 0x7f7bb81025f0 2024-06-10T00:22:34.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.938+0000 7f7bb57fa700 1 -- 172.21.15.62:0/1742809463 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7ba0019070 con 0x7f7bb81025f0 2024-06-10T00:22:34.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.938+0000 7f7bb57fa700 1 --2- 172.21.15.62:0/1742809463 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7ba4039330 0x7f7ba403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:22:34.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.938+0000 7f7bbe11e700 1 -- 172.21.15.62:0/1742809463 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb8069180 con 0x7f7bb81025f0 2024-06-10T00:22:34.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.938+0000 7f7bb57fa700 1 -- 172.21.15.62:0/1742809463 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7ba0010710 con 0x7f7bb81025f0 2024-06-10T00:22:34.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.939+0000 7f7bb77fe700 1 --2- 172.21.15.62:0/1742809463 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7ba4039330 0x7f7ba403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:22:34.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.939+0000 7f7bb77fe700 1 --2- 172.21.15.62:0/1742809463 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7ba4039330 0x7f7ba403b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8102450 tx=0x7f7ba8007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:22:34.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:34.944+0000 7f7bb57fa700 1 -- 172.21.15.62:0/1742809463 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7ba000ed70 con 0x7f7bb81025f0 2024-06-10T00:22:35.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:35.165+0000 7f7bbe11e700 1 -- 172.21.15.62:0/1742809463 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7bb80628d0 con 0x7f7ba4039330 2024-06-10T00:22:35.373 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:35.083+0000 7fb51d1c2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:22:35.373 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:35.183+0000 7fb51d1c2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:22:35.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:35 smithi062 ceph-mon[32390]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:35.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:35.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:35.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:22:35.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:22:35.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:35.860 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:22:35.860 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:22:35.860 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:35.873 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:35.386+0000 7fb51d1c2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:22:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-mon[36549]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:22:35.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:22:35.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:35.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:22:35.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:22:35.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:35.911 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:35.569+0000 7f935f03f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:22:35.911 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:35.740+0000 7f935f03f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:22:35.911 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:35.910+0000 7f935f03f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:22:35.911 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-mon[36321]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:35.911 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.912 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.912 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.912 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.912 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:35.912 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:35.912 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:35.912 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:22:35.912 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:22:35.913 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:35.913 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:22:35.913 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:22:35.913 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:36.266 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:35 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:35.930+0000 7fb51d1c2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:22:36.267 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:36 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:36.021+0000 7fb51d1c2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:22:36.504 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:36 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:36.196+0000 7f935f03f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:22:36.504 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:36 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:36.290+0000 7f935f03f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:22:36.504 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:36 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:36.504+0000 7f935f03f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:22:36.598 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:36 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:36.266+0000 7fb51d1c2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:22:36.598 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:36 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:36.400+0000 7fb51d1c2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:22:36.598 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:36 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:36.494+0000 7fb51d1c2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:22:36.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:36 smithi105 ceph-mon[36321]: Reconfiguring mgr.x (unknown last config time)... 2024-06-10T00:22:36.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:36 smithi105 ceph-mon[36321]: Reconfiguring daemon mgr.x on smithi062 2024-06-10T00:22:36.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:36 smithi105 ceph-mon[36321]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:22:36.782 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:36 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:36.696+0000 7f935f03f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:22:36.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:36 smithi062 ceph-mon[32390]: Reconfiguring mgr.x (unknown last config time)... 2024-06-10T00:22:36.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:36 smithi062 ceph-mon[32390]: Reconfiguring daemon mgr.x on smithi062 2024-06-10T00:22:36.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:36 smithi062 ceph-mon[32390]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:22:36.873 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:36 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:36.613+0000 7fb51d1c2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:22:36.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:36 smithi138 ceph-mon[36549]: Reconfiguring mgr.x (unknown last config time)... 2024-06-10T00:22:36.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:36 smithi138 ceph-mon[36549]: Reconfiguring daemon mgr.x on smithi062 2024-06-10T00:22:36.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:36 smithi138 ceph-mon[36549]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:22:37.373 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:36 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:36.917+0000 7fb51d1c2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:22:37.599 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:37 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:37.313+0000 7f935f03f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:22:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:37 smithi062 ceph-mon[32390]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:37 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:37.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:37 smithi138 ceph-mon[36549]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:37.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:37 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:37.873 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:37 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:37.513+0000 7fb51d1c2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:22:37.873 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:22:37 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:22:37.735+0000 7fb51d1c2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:22:38.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:37 smithi105 ceph-mon[36321]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:38.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:37 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:38.032 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:37 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:37.736+0000 7f935f03f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:22:38.032 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:37 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:37.832+0000 7f935f03f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:22:38.032 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:38.031+0000 7f935f03f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:22:38.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:38.651+0000 7f7bb57fa700 1 -- 172.21.15.62:0/1742809463 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f7ba000c2d0 con 0x7f7bb81025f0 2024-06-10T00:22:38.901 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:38.567+0000 7f935f03f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:22:38.901 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:38.662+0000 7f935f03f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:22:38.902 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:38.902 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:38.902 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:38.902 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:38.902 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:38.902 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:38.902 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-mon[36321]: Standby manager daemon z started 2024-06-10T00:22:38.902 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.138:0/3558183540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T00:22:38.902 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.138:0/3558183540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:22:38.902 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.138:0/3558183540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T00:22:38.903 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.138:0/3558183540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:22:39.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:38 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:39.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:38 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:39.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:38 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:39.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:38 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:39.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:38 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:39.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:38 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:39.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:38 smithi062 ceph-mon[32390]: Standby manager daemon z started 2024-06-10T00:22:39.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:38 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.138:0/3558183540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T00:22:39.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:38 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.138:0/3558183540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:22:39.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:38 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.138:0/3558183540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T00:22:39.104 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:38 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.138:0/3558183540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:22:39.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:38 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:39.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:38 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:39.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:38 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:39.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:38 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:39.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:38 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:39.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:38 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:39.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:38 smithi138 ceph-mon[36549]: Standby manager daemon z started 2024-06-10T00:22:39.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:38 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.138:0/3558183540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T00:22:39.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:38 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.138:0/3558183540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:22:39.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:38 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.138:0/3558183540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T00:22:39.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:38 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.138:0/3558183540' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:22:39.238 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:38 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:38.901+0000 7f935f03f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:22:39.238 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:39 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:39.033+0000 7f935f03f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:22:39.238 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:39 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:39.125+0000 7f935f03f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:22:39.238 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:39 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:39.238+0000 7f935f03f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:22:40.034 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:39 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:39.536+0000 7f935f03f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:22:40.034 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:39 smithi105 ceph-mon[36321]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:40.034 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:39 smithi105 ceph-mon[36321]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-10T00:22:40.034 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:39 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T00:22:40.034 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:39 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3504435139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a148b909-d252-4277-b842-365cb9f169d1"}]: dispatch 2024-06-10T00:22:40.034 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:39 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3504435139' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a148b909-d252-4277-b842-365cb9f169d1"}]': finished 2024-06-10T00:22:40.035 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:39 smithi105 ceph-mon[36321]: osdmap e5: 1 total, 0 up, 1 in 2024-06-10T00:22:40.035 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:39 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:40.035 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:39 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1123510428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:22:40.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:39 smithi062 ceph-mon[32390]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:39 smithi062 ceph-mon[32390]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-10T00:22:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:39 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T00:22:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:39 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3504435139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a148b909-d252-4277-b842-365cb9f169d1"}]: dispatch 2024-06-10T00:22:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:39 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3504435139' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a148b909-d252-4277-b842-365cb9f169d1"}]': finished 2024-06-10T00:22:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:39 smithi062 ceph-mon[32390]: osdmap e5: 1 total, 0 up, 1 in 2024-06-10T00:22:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:39 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:39 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1123510428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:22:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:39 smithi138 ceph-mon[36549]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:39 smithi138 ceph-mon[36549]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-10T00:22:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:39 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T00:22:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:39 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3504435139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a148b909-d252-4277-b842-365cb9f169d1"}]: dispatch 2024-06-10T00:22:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:39 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3504435139' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a148b909-d252-4277-b842-365cb9f169d1"}]': finished 2024-06-10T00:22:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:39 smithi138 ceph-mon[36549]: osdmap e5: 1 total, 0 up, 1 in 2024-06-10T00:22:40.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:39 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:40.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:39 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1123510428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:22:40.531 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:40 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:40.132+0000 7f935f03f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:22:40.531 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:22:40 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:22:40.346+0000 7f935f03f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:22:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:40.660+0000 7f7bb57fa700 1 -- 172.21.15.62:0/1742809463 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f7ba0003680 con 0x7f7bb81025f0 2024-06-10T00:22:41.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:40 smithi105 ceph-mon[36321]: Standby manager daemon y started 2024-06-10T00:22:41.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:40 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.105:0/2174441687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T00:22:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:40 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.105:0/2174441687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:22:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:40 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.105:0/2174441687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T00:22:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:40 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.105:0/2174441687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:22:41.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:40 smithi062 ceph-mon[32390]: Standby manager daemon y started 2024-06-10T00:22:41.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:40 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.105:0/2174441687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T00:22:41.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:40 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.105:0/2174441687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:22:41.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:40 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.105:0/2174441687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T00:22:41.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:40 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.105:0/2174441687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:22:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:40 smithi138 ceph-mon[36549]: Standby manager daemon y started 2024-06-10T00:22:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:40 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.105:0/2174441687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T00:22:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:40 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.105:0/2174441687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:22:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:40 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.105:0/2174441687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T00:22:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:40 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.105:0/2174441687' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:22:42.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:41 smithi105 ceph-mon[36321]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:42.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:41 smithi105 ceph-mon[36321]: mgrmap e15: x(active, since 78s), standbys: y, z 2024-06-10T00:22:42.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:41 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T00:22:42.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:41 smithi062 ceph-mon[32390]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:42.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:41 smithi062 ceph-mon[32390]: mgrmap e15: x(active, since 78s), standbys: y, z 2024-06-10T00:22:42.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:41 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T00:22:42.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:41 smithi138 ceph-mon[36549]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:42.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:41 smithi138 ceph-mon[36549]: mgrmap e15: x(active, since 78s), standbys: y, z 2024-06-10T00:22:42.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:41 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T00:22:43.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:43 smithi138 ceph-mon[36549]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:43.989 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:43 smithi105 ceph-mon[36321]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:43.994 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:43 smithi062 ceph-mon[32390]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:45.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:45 smithi062 ceph-mon[32390]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:46.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:45 smithi105 ceph-mon[36321]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:45 smithi138 ceph-mon[36549]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:46.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:46 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-10T00:22:46.986 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:46 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:47.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:46 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-10T00:22:47.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:46 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:47.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:46 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-10T00:22:47.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:46 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:48.010 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:47 smithi062 ceph-mon[32390]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:48.010 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:47 smithi062 ceph-mon[32390]: Deploying daemon osd.0 on smithi062 2024-06-10T00:22:48.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:47 smithi105 ceph-mon[36321]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:48.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:47 smithi105 ceph-mon[36321]: Deploying daemon osd.0 on smithi062 2024-06-10T00:22:48.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:47 smithi138 ceph-mon[36549]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:48.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:47 smithi138 ceph-mon[36549]: Deploying daemon osd.0 on smithi062 2024-06-10T00:22:49.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:49 smithi062 ceph-mon[32390]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:50.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:49 smithi105 ceph-mon[36321]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:50.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:49 smithi138 ceph-mon[36549]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:52.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:51 smithi105 ceph-mon[36321]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:52.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:51 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:52.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:51 smithi062 ceph-mon[32390]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:52.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:51 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:52.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:51 smithi138 ceph-mon[36549]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:52.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:51 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:52.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:52 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:52.924 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:52 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:53.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:52 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:53.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:52 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:53.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:52 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:53.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:52 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:53.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:53 smithi062 ceph-mon[32390]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:54.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:53 smithi105 ceph-mon[36321]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:54.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:53 smithi138 ceph-mon[36549]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:55.277 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 0 on host 'smithi062' 2024-06-10T00:22:55.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:55.277+0000 7f7bb57fa700 1 -- 172.21.15.62:0/1742809463 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7bb80628d0 con 0x7f7ba4039330 2024-06-10T00:22:55.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:55.279+0000 7f7bbe11e700 1 -- 172.21.15.62:0/1742809463 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7ba4039330 msgr2=0x7f7ba403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:55.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:55.279+0000 7f7bbe11e700 1 --2- 172.21.15.62:0/1742809463 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7ba4039330 0x7f7ba403b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8102450 tx=0x7f7ba8007400 comp rx=0 tx=0).stop 2024-06-10T00:22:55.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:55.279+0000 7f7bbe11e700 1 -- 172.21.15.62:0/1742809463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb81025f0 msgr2=0x7f7bb819d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:22:55.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:55.279+0000 7f7bbe11e700 1 --2- 172.21.15.62:0/1742809463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb81025f0 0x7f7bb819d420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000d950 tx=0x7f7ba000dc60 comp rx=0 tx=0).stop 2024-06-10T00:22:55.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:55.279+0000 7f7bbe11e700 1 -- 172.21.15.62:0/1742809463 shutdown_connections 2024-06-10T00:22:55.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:55.279+0000 7f7bbe11e700 1 --2- 172.21.15.62:0/1742809463 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7ba4039330 0x7f7ba403b7f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:55.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:55.279+0000 7f7bbe11e700 1 --2- 172.21.15.62:0/1742809463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb8102f90 0x7f7bb819fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:55.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:55.279+0000 7f7bbe11e700 1 --2- 172.21.15.62:0/1742809463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bb81013f0 0x7f7bb806acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:55.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:55.279+0000 7f7bbe11e700 1 --2- 172.21.15.62:0/1742809463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7bb81025f0 0x7f7bb819d420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:22:55.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:55.279+0000 7f7bbe11e700 1 -- 172.21.15.62:0/1742809463 >> 172.21.15.62:0/1742809463 conn(0x7f7bb80fc990 msgr2=0x7f7bb81058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:22:55.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:55.279+0000 7f7bbe11e700 1 -- 172.21.15.62:0/1742809463 shutdown_connections 2024-06-10T00:22:55.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:22:55.279+0000 7f7bbe11e700 1 -- 172.21.15.62:0/1742809463 wait complete. 2024-06-10T00:22:55.546 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:55 smithi062 ceph-mon[32390]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:55.546 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.546 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.546 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:55.546 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:55.546 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.546 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.546 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.546 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:55 smithi138 ceph-mon[36549]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:55.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:55 smithi105 ceph-mon[36321]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:55.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:55.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:55.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:55.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:22:55.830 DEBUG:teuthology.orchestra.run.smithi062:osd.0> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.0.service 2024-06-10T00:22:55.832 INFO:tasks.cephadm:Deploying osd.1 on smithi105 with /dev/vg_nvme/lv_4... 2024-06-10T00:22:55.832 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-10T00:22:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:22:56.469 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:56 smithi105 ceph-mon[36321]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-10T00:22:56.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:56 smithi062 ceph-mon[32390]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-10T00:22:56.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:56 smithi138 ceph-mon[36549]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-10T00:22:57.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:57 smithi105 ceph-mon[36321]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:57 smithi105 ceph-mon[36321]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-10T00:22:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:57 smithi105 ceph-mon[36321]: osdmap e6: 1 total, 0 up, 1 in 2024-06-10T00:22:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:57 smithi105 ceph-mon[36321]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-06-10T00:22:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:57 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:57.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:57 smithi062 ceph-mon[32390]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:57.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:57 smithi062 ceph-mon[32390]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-10T00:22:57.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:57 smithi062 ceph-mon[32390]: osdmap e6: 1 total, 0 up, 1 in 2024-06-10T00:22:57.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:57 smithi062 ceph-mon[32390]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-06-10T00:22:57.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:57 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:57.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:57 smithi138 ceph-mon[36549]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:57.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:57 smithi138 ceph-mon[36549]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-10T00:22:57.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:57 smithi138 ceph-mon[36549]: osdmap e6: 1 total, 0 up, 1 in 2024-06-10T00:22:57.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:57 smithi138 ceph-mon[36549]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-06-10T00:22:57.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:57 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-10T00:22:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-10T00:22:57.935 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-06-10T00:22:57.935 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-06-10T00:22:57.935 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113733 s, 922 MB/s 2024-06-10T00:22:57.936 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-06-10T00:22:58.358 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:22:58 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-0[47508]: 2024-06-10T00:22:58.101+0000 7f438a924700 -1 osd.0 0 waiting for initial osdmap 2024-06-10T00:22:58.358 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:22:58 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-0[47508]: 2024-06-10T00:22:58.104+0000 7f4381f0e700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:22:58.503 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:58 smithi105 ceph-mon[36321]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-06-10T00:22:58.504 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:58 smithi105 ceph-mon[36321]: osdmap e7: 1 total, 0 up, 1 in 2024-06-10T00:22:58.504 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:58 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:58.504 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:58 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:58.504 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:58 smithi105 ceph-mon[36321]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' 2024-06-10T00:22:58.505 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi105:vg_nvme/lv_4 2024-06-10T00:22:58.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:58 smithi138 ceph-mon[36549]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-06-10T00:22:58.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:58 smithi138 ceph-mon[36549]: osdmap e7: 1 total, 0 up, 1 in 2024-06-10T00:22:58.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:58 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:58.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:58 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:58.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:58 smithi138 ceph-mon[36549]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' 2024-06-10T00:22:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:22:58.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:58 smithi062 ceph-mon[32390]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-06-10T00:22:58.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:58 smithi062 ceph-mon[32390]: osdmap e7: 1 total, 0 up, 1 in 2024-06-10T00:22:58.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:58 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:58.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:58 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:58.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:58 smithi062 ceph-mon[32390]: from='osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246]' entity='osd.0' 2024-06-10T00:22:59.722 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:59 smithi105 ceph-mon[36321]: purged_snaps scrub starts 2024-06-10T00:22:59.723 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:59 smithi105 ceph-mon[36321]: purged_snaps scrub ok 2024-06-10T00:22:59.723 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:59 smithi105 ceph-mon[36321]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:59.723 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:59 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:59.723 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:59 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:59.723 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:59 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:59.723 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:59 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:22:59.723 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:59 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:59.723 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:59 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:59.723 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:59 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:59.723 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:59 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:59.723 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:59 smithi105 ceph-mon[36321]: osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246] boot 2024-06-10T00:22:59.723 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:59 smithi105 ceph-mon[36321]: osdmap e8: 1 total, 1 up, 1 in 2024-06-10T00:22:59.724 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:22:59 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:59.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:59 smithi062 ceph-mon[32390]: purged_snaps scrub starts 2024-06-10T00:22:59.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:59 smithi062 ceph-mon[32390]: purged_snaps scrub ok 2024-06-10T00:22:59.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:59 smithi062 ceph-mon[32390]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:59.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:59.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:59.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:59.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:22:59.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:59.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:59.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:59.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:59.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:59 smithi062 ceph-mon[32390]: osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246] boot 2024-06-10T00:22:59.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:59 smithi062 ceph-mon[32390]: osdmap e8: 1 total, 1 up, 1 in 2024-06-10T00:22:59.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:22:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:59.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:59 smithi138 ceph-mon[36549]: purged_snaps scrub starts 2024-06-10T00:22:59.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:59 smithi138 ceph-mon[36549]: purged_snaps scrub ok 2024-06-10T00:22:59.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:59 smithi138 ceph-mon[36549]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:22:59.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:59 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:22:59.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:59 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:59.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:59 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:59.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:59 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:22:59.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:59 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:59.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:59 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:22:59.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:59 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:22:59.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:59 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:22:59.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:59 smithi138 ceph-mon[36549]: osd.0 [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246] boot 2024-06-10T00:22:59.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:59 smithi138 ceph-mon[36549]: osdmap e8: 1 total, 1 up, 1 in 2024-06-10T00:22:59.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:22:59 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:23:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.223+0000 7fa182582700 1 -- 172.21.15.105:0/1527105286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c102fb0 msgr2=0x7fa17c109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:23:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.223+0000 7fa182582700 1 --2- 172.21.15.105:0/1527105286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c102fb0 0x7fa17c109860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa17400b0d0 tx=0x7fa17400b3e0 comp rx=0 tx=0).stop 2024-06-10T00:23:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.224+0000 7fa182582700 1 -- 172.21.15.105:0/1527105286 shutdown_connections 2024-06-10T00:23:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.224+0000 7fa182582700 1 --2- 172.21.15.105:0/1527105286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c102fb0 0x7fa17c109860 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.224+0000 7fa182582700 1 --2- 172.21.15.105:0/1527105286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17c101410 0x7fa17c101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.224+0000 7fa182582700 1 --2- 172.21.15.105:0/1527105286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa17c102610 0x7fa17c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.224+0000 7fa182582700 1 -- 172.21.15.105:0/1527105286 >> 172.21.15.105:0/1527105286 conn(0x7fa17c0fc9f0 msgr2=0x7fa17c0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:23:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.224+0000 7fa182582700 1 -- 172.21.15.105:0/1527105286 shutdown_connections 2024-06-10T00:23:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.224+0000 7fa182582700 1 -- 172.21.15.105:0/1527105286 wait complete. 2024-06-10T00:23:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.224+0000 7fa182582700 1 Processor -- start 2024-06-10T00:23:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.224+0000 7fa182582700 1 -- start start 2024-06-10T00:23:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.224+0000 7fa182582700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17c101410 0x7fa17c1053a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:23:00.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.224+0000 7fa182582700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c102610 0x7fa17c1039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:23:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.224+0000 7fa182582700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa17c102fb0 0x7fa17c103f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:23:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa182582700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c10d0a0 con 0x7fa17c102fb0 2024-06-10T00:23:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa182582700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c10cf20 con 0x7fa17c102610 2024-06-10T00:23:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa182582700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c10d220 con 0x7fa17c101410 2024-06-10T00:23:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa17b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c102610 0x7fa17c1039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:23:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa17b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c102610 0x7fa17c1039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45048/0 (socket says 172.21.15.105:45048) 2024-06-10T00:23:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa17b7fe700 1 -- 172.21.15.105:0/257991023 learned_addr learned my addr 172.21.15.105:0/257991023 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-06-10T00:23:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa17bfff700 1 --2- 172.21.15.105:0/257991023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17c101410 0x7fa17c1053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:23:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa180b1f700 1 --2- 172.21.15.105:0/257991023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa17c102fb0 0x7fa17c103f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:23:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa17b7fe700 1 -- 172.21.15.105:0/257991023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17c101410 msgr2=0x7fa17c1053a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:23:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa17b7fe700 1 --2- 172.21.15.105:0/257991023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17c101410 0x7fa17c1053a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa17b7fe700 1 -- 172.21.15.105:0/257991023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa17c102fb0 msgr2=0x7fa17c103f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:23:00.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa17b7fe700 1 --2- 172.21.15.105:0/257991023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa17c102fb0 0x7fa17c103f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:00.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa17b7fe700 1 -- 172.21.15.105:0/257991023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17c1047e0 con 0x7fa17c102610 2024-06-10T00:23:00.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa17b7fe700 1 --2- 172.21.15.105:0/257991023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c102610 0x7fa17c1039f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa17000b530 tx=0x7fa17000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:23:00.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa1797fa700 1 -- 172.21.15.105:0/257991023 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170015020 con 0x7fa17c102610 2024-06-10T00:23:00.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa182582700 1 -- 172.21.15.105:0/257991023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17c1a2840 con 0x7fa17c102610 2024-06-10T00:23:00.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa1797fa700 1 -- 172.21.15.105:0/257991023 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa170011d10 con 0x7fa17c102610 2024-06-10T00:23:00.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa1797fa700 1 -- 172.21.15.105:0/257991023 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17000f2e0 con 0x7fa17c102610 2024-06-10T00:23:00.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.225+0000 7fa182582700 1 -- 172.21.15.105:0/257991023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa17c1a2dd0 con 0x7fa17c102610 2024-06-10T00:23:00.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.226+0000 7fa182582700 1 -- 172.21.15.105:0/257991023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa17c069180 con 0x7fa17c102610 2024-06-10T00:23:00.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.227+0000 7fa1797fa700 1 -- 172.21.15.105:0/257991023 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fa170018020 con 0x7fa17c102610 2024-06-10T00:23:00.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.228+0000 7fa1797fa700 1 --2- 172.21.15.105:0/257991023 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa1640a2b90 0x7fa1640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:23:00.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.228+0000 7fa1797fa700 1 -- 172.21.15.105:0/257991023 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7fa1700cd200 con 0x7fa17c102610 2024-06-10T00:23:00.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.229+0000 7fa17bfff700 1 --2- 172.21.15.105:0/257991023 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa1640a2b90 0x7fa1640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:23:00.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.229+0000 7fa17bfff700 1 --2- 172.21.15.105:0/257991023 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa1640a2b90 0x7fa1640a5050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa17c102470 tx=0x7fa16c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:23:00.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.232+0000 7fa1797fa700 1 -- 172.21.15.105:0/257991023 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa17001f020 con 0x7fa17c102610 2024-06-10T00:23:00.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:00.438+0000 7fa182582700 1 -- 172.21.15.105:0/257991023 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa17c1a3090 con 0x7fa1640a2b90 2024-06-10T00:23:00.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:00 smithi105 ceph-mon[36321]: Detected new or changed devices on smithi062 2024-06-10T00:23:00.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:00 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi062 to 17088M 2024-06-10T00:23:00.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:00 smithi062 ceph-mon[32390]: Detected new or changed devices on smithi062 2024-06-10T00:23:00.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:00 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi062 to 17088M 2024-06-10T00:23:00.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:00 smithi138 ceph-mon[36549]: Detected new or changed devices on smithi062 2024-06-10T00:23:00.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:00 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi062 to 17088M 2024-06-10T00:23:01.702 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:01 smithi105 ceph-mon[36321]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:01.702 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:01 smithi105 ceph-mon[36321]: osdmap e9: 1 total, 1 up, 1 in 2024-06-10T00:23:01.702 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:01 smithi105 ceph-mon[36321]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:23:01.703 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:01 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:23:01.703 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:01 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:23:01.703 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:01 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:01.774 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:01 smithi138 ceph-mon[36549]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:01.774 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:01 smithi138 ceph-mon[36549]: osdmap e9: 1 total, 1 up, 1 in 2024-06-10T00:23:01.774 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:01 smithi138 ceph-mon[36549]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:23:01.774 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:01 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:23:01.774 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:01 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:23:01.774 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:01 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:01.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:01 smithi062 ceph-mon[32390]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:01.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:01 smithi062 ceph-mon[32390]: osdmap e9: 1 total, 1 up, 1 in 2024-06-10T00:23:01.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:01 smithi062 ceph-mon[32390]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:23:01.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:01 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:23:01.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:01 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:23:01.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:01 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:03.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:03 smithi105 ceph-mon[36321]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:03.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:03 smithi105 ceph-mon[36321]: from='client.? 172.21.15.105:0/3141010845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d83d356c-ece6-401e-8f1e-156d5076c817"}]: dispatch 2024-06-10T00:23:03.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:03 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d83d356c-ece6-401e-8f1e-156d5076c817"}]: dispatch 2024-06-10T00:23:03.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:03 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d83d356c-ece6-401e-8f1e-156d5076c817"}]': finished 2024-06-10T00:23:03.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:03 smithi105 ceph-mon[36321]: osdmap e10: 2 total, 1 up, 2 in 2024-06-10T00:23:03.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:03 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:03.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:03 smithi062 ceph-mon[32390]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:03.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:03 smithi062 ceph-mon[32390]: from='client.? 172.21.15.105:0/3141010845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d83d356c-ece6-401e-8f1e-156d5076c817"}]: dispatch 2024-06-10T00:23:03.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:03 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d83d356c-ece6-401e-8f1e-156d5076c817"}]: dispatch 2024-06-10T00:23:03.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:03 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d83d356c-ece6-401e-8f1e-156d5076c817"}]': finished 2024-06-10T00:23:03.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:03 smithi062 ceph-mon[32390]: osdmap e10: 2 total, 1 up, 2 in 2024-06-10T00:23:03.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:03 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:03 smithi138 ceph-mon[36549]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:03 smithi138 ceph-mon[36549]: from='client.? 172.21.15.105:0/3141010845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d83d356c-ece6-401e-8f1e-156d5076c817"}]: dispatch 2024-06-10T00:23:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:03 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d83d356c-ece6-401e-8f1e-156d5076c817"}]: dispatch 2024-06-10T00:23:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:03 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d83d356c-ece6-401e-8f1e-156d5076c817"}]': finished 2024-06-10T00:23:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:03 smithi138 ceph-mon[36549]: osdmap e10: 2 total, 1 up, 2 in 2024-06-10T00:23:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:03 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:04.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:04 smithi062 ceph-mon[32390]: from='client.? 172.21.15.105:0/2975148332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:23:04.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:04 smithi105 ceph-mon[36321]: from='client.? 172.21.15.105:0/2975148332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:23:04.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:04 smithi138 ceph-mon[36549]: from='client.? 172.21.15.105:0/2975148332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:23:05.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:05 smithi105 ceph-mon[36321]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:05.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:05 smithi062 ceph-mon[32390]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:05.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:05 smithi138 ceph-mon[36549]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:07.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:07 smithi105 ceph-mon[36321]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:07.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:07 smithi062 ceph-mon[32390]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:07.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:07 smithi138 ceph-mon[36549]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:09.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:09 smithi105 ceph-mon[36321]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:09.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:09 smithi062 ceph-mon[32390]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:09.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:09 smithi138 ceph-mon[36549]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:10.661 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:10 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-10T00:23:10.661 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:10 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:10.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-10T00:23:10.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:10.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-10T00:23:10.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:11.494 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:11 smithi105 ceph-mon[36321]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:11.494 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:11 smithi105 ceph-mon[36321]: Deploying daemon osd.1 on smithi105 2024-06-10T00:23:11.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:11 smithi062 ceph-mon[32390]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:11.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:11 smithi062 ceph-mon[32390]: Deploying daemon osd.1 on smithi105 2024-06-10T00:23:11.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:11 smithi138 ceph-mon[36549]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:11.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:11 smithi138 ceph-mon[36549]: Deploying daemon osd.1 on smithi105 2024-06-10T00:23:13.444 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:13 smithi105 ceph-mon[36321]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:13.673 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:13 smithi062 ceph-mon[32390]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:13.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:13 smithi138 ceph-mon[36549]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:15.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:15 smithi105 ceph-mon[36321]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:15.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:15 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:15.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:15 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:15.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:15 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:23:15.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:15 smithi062 ceph-mon[32390]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:15.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:15 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:15.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:15 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:15.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:15 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:23:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:15 smithi138 ceph-mon[36549]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:15 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:15 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:15 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:23:17.624 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:17 smithi105 ceph-mon[36321]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:17.624 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:17.624 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:17.624 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:17.624 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:23:17.624 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:17.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:17 smithi062 ceph-mon[32390]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:17.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:17.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:23:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:17 smithi138 ceph-mon[36549]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:23:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:18.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:18.809+0000 7fa1797fa700 1 -- 172.21.15.105:0/257991023 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa17c1a3090 con 0x7fa1640a2b90 2024-06-10T00:23:18.809 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 1 on host 'smithi105' 2024-06-10T00:23:18.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:18.811+0000 7fa182582700 1 -- 172.21.15.105:0/257991023 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa1640a2b90 msgr2=0x7fa1640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:23:18.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:18.811+0000 7fa182582700 1 --2- 172.21.15.105:0/257991023 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa1640a2b90 0x7fa1640a5050 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa17c102470 tx=0x7fa16c00a3b0 comp rx=0 tx=0).stop 2024-06-10T00:23:18.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:18.811+0000 7fa182582700 1 -- 172.21.15.105:0/257991023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c102610 msgr2=0x7fa17c1039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:23:18.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:18.811+0000 7fa182582700 1 --2- 172.21.15.105:0/257991023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c102610 0x7fa17c1039f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa17000b530 tx=0x7fa17000b840 comp rx=0 tx=0).stop 2024-06-10T00:23:18.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:18.811+0000 7fa182582700 1 -- 172.21.15.105:0/257991023 shutdown_connections 2024-06-10T00:23:18.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:18.811+0000 7fa182582700 1 --2- 172.21.15.105:0/257991023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa17c102fb0 0x7fa17c103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:18.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:18.811+0000 7fa182582700 1 --2- 172.21.15.105:0/257991023 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa1640a2b90 0x7fa1640a5050 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:18.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:18.811+0000 7fa182582700 1 --2- 172.21.15.105:0/257991023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa17c101410 0x7fa17c1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:18.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:18.811+0000 7fa182582700 1 --2- 172.21.15.105:0/257991023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa17c102610 0x7fa17c1039f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:18.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:18.811+0000 7fa182582700 1 -- 172.21.15.105:0/257991023 >> 172.21.15.105:0/257991023 conn(0x7fa17c0fc9f0 msgr2=0x7fa17c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:23:18.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:18.811+0000 7fa182582700 1 -- 172.21.15.105:0/257991023 shutdown_connections 2024-06-10T00:23:18.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:23:18.811+0000 7fa182582700 1 -- 172.21.15.105:0/257991023 wait complete. 2024-06-10T00:23:19.314 DEBUG:teuthology.orchestra.run.smithi105:osd.1> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.1.service 2024-06-10T00:23:19.316 INFO:tasks.cephadm:Deploying osd.2 on smithi138 with /dev/vg_nvme/lv_4... 2024-06-10T00:23:19.317 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-10T00:23:19.558 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:23:19.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:19 smithi138 ceph-mon[36549]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:19.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:19.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:19.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:23:19.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:19 smithi138 ceph-mon[36549]: from='osd.1 [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:23:19.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:19 smithi138 ceph-mon[36549]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:23:19.651 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:19 smithi105 ceph-mon[36321]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:19.651 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:19.651 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:19.651 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:23:19.651 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:19 smithi105 ceph-mon[36321]: from='osd.1 [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:23:19.651 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:19 smithi105 ceph-mon[36321]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:23:19.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:19 smithi062 ceph-mon[32390]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:19.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:19.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:23:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:19 smithi062 ceph-mon[32390]: from='osd.1 [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:23:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:19 smithi062 ceph-mon[32390]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:23:21.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:20 smithi105 ceph-mon[36321]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-10T00:23:21.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:20 smithi105 ceph-mon[36321]: osdmap e11: 2 total, 1 up, 2 in 2024-06-10T00:23:21.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:20 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:21.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:20 smithi105 ceph-mon[36321]: from='osd.1 [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:23:21.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:20 smithi105 ceph-mon[36321]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:23:21.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:20 smithi105 ceph-mon[36321]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:21.046 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:20 smithi138 ceph-mon[36549]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-10T00:23:21.047 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:20 smithi138 ceph-mon[36549]: osdmap e11: 2 total, 1 up, 2 in 2024-06-10T00:23:21.047 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:20 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:21.047 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:20 smithi138 ceph-mon[36549]: from='osd.1 [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:23:21.047 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:20 smithi138 ceph-mon[36549]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:23:21.047 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:20 smithi138 ceph-mon[36549]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:21.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:20 smithi062 ceph-mon[32390]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-10T00:23:21.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:20 smithi062 ceph-mon[32390]: osdmap e11: 2 total, 1 up, 2 in 2024-06-10T00:23:21.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:20 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:21.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:20 smithi062 ceph-mon[32390]: from='osd.1 [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:23:21.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:20 smithi062 ceph-mon[32390]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:23:21.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:20 smithi062 ceph-mon[32390]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:21.503 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-10T00:23:21.503 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-10T00:23:21.503 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-06-10T00:23:21.503 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-06-10T00:23:21.503 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129791 s, 808 MB/s 2024-06-10T00:23:21.503 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-06-10T00:23:22.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:21 smithi105 ceph-mon[36321]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-06-10T00:23:22.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:21 smithi105 ceph-mon[36321]: osdmap e12: 2 total, 1 up, 2 in 2024-06-10T00:23:22.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:21 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:22.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:21 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:22.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:21 smithi105 ceph-mon[36321]: from='osd.1 ' entity='osd.1' 2024-06-10T00:23:22.032 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:23:21 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-1[40899]: 2024-06-10T00:23:21.586+0000 7f855609e700 -1 osd.1 0 waiting for initial osdmap 2024-06-10T00:23:22.032 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:23:21 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-1[40899]: 2024-06-10T00:23:21.590+0000 7f854de89700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:23:22.107 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:21 smithi138 ceph-mon[36549]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-06-10T00:23:22.107 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:21 smithi138 ceph-mon[36549]: osdmap e12: 2 total, 1 up, 2 in 2024-06-10T00:23:22.107 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:21 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:22.107 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:21 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:22.107 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:21 smithi138 ceph-mon[36549]: from='osd.1 ' entity='osd.1' 2024-06-10T00:23:22.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:21 smithi062 ceph-mon[32390]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-06-10T00:23:22.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:21 smithi062 ceph-mon[32390]: osdmap e12: 2 total, 1 up, 2 in 2024-06-10T00:23:22.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:22.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:22.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:21 smithi062 ceph-mon[32390]: from='osd.1 ' entity='osd.1' 2024-06-10T00:23:22.134 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_4 2024-06-10T00:23:22.380 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:23:23.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: purged_snaps scrub starts 2024-06-10T00:23:23.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: purged_snaps scrub ok 2024-06-10T00:23:23.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:23.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:23.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: Detected new or changed devices on smithi105 2024-06-10T00:23:23.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:23.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:23.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:23:23.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi105 to 17088M 2024-06-10T00:23:23.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:23.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:23.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:23:23.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:23.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: osd.1 [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019] boot 2024-06-10T00:23:23.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: osdmap e13: 2 total, 2 up, 2 in 2024-06-10T00:23:23.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:23.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: purged_snaps scrub starts 2024-06-10T00:23:23.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: purged_snaps scrub ok 2024-06-10T00:23:23.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:23.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:23.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: Detected new or changed devices on smithi105 2024-06-10T00:23:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:23:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi105 to 17088M 2024-06-10T00:23:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:23:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: osd.1 [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019] boot 2024-06-10T00:23:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: osdmap e13: 2 total, 2 up, 2 in 2024-06-10T00:23:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: purged_snaps scrub starts 2024-06-10T00:23:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: purged_snaps scrub ok 2024-06-10T00:23:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:23:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: Detected new or changed devices on smithi105 2024-06-10T00:23:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:23:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi105 to 17088M 2024-06-10T00:23:23.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:23.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:23.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:23:23.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:23.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: osd.1 [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019] boot 2024-06-10T00:23:23.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: osdmap e13: 2 total, 2 up, 2 in 2024-06-10T00:23:23.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:23:23.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.819+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/2736309430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f4105880 msgr2=0x7fc5f4107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:23:23.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.819+0000 7fc5fbc2e700 1 --2- 172.21.15.138:0/2736309430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f4105880 0x7fc5f4107c70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009990 tx=0x7fc5e4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:23:23.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.819+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/2736309430 shutdown_connections 2024-06-10T00:23:23.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.819+0000 7fc5fbc2e700 1 --2- 172.21.15.138:0/2736309430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5f41081b0 0x7fc5f410a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:23.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.819+0000 7fc5fbc2e700 1 --2- 172.21.15.138:0/2736309430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f4102f50 0x7fc5f4105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:23.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.819+0000 7fc5fbc2e700 1 --2- 172.21.15.138:0/2736309430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f4105880 0x7fc5f4107c70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:23.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.819+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/2736309430 >> 172.21.15.138:0/2736309430 conn(0x7fc5f40fc980 msgr2=0x7fc5f40feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:23:23.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.819+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/2736309430 shutdown_connections 2024-06-10T00:23:23.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.819+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/2736309430 wait complete. 2024-06-10T00:23:23.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.819+0000 7fc5fbc2e700 1 Processor -- start 2024-06-10T00:23:23.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.819+0000 7fc5fbc2e700 1 -- start start 2024-06-10T00:23:23.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5fbc2e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f4102f50 0x7fc5f41028b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:23:23.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5fbc2e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f4105880 0x7fc5f4100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:23:23.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5fbc2e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5f41081b0 0x7fc5f4101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:23:23.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5fbc2e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f410e090 con 0x7fc5f41081b0 2024-06-10T00:23:23.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5fbc2e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f410df10 con 0x7fc5f4105880 2024-06-10T00:23:23.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5fbc2e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f410e210 con 0x7fc5f4102f50 2024-06-10T00:23:23.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5f99ca700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f4102f50 0x7fc5f41028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:23:23.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5f99ca700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f4102f50 0x7fc5f41028b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34378/0 (socket says 172.21.15.138:34378) 2024-06-10T00:23:23.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5f99ca700 1 -- 172.21.15.138:0/855332145 learned_addr learned my addr 172.21.15.138:0/855332145 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-06-10T00:23:23.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5fa1cb700 1 --2- 172.21.15.138:0/855332145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5f41081b0 0x7fc5f4101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:23:23.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5f91c9700 1 --2- 172.21.15.138:0/855332145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f4105880 0x7fc5f4100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:23:23.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5f99ca700 1 -- 172.21.15.138:0/855332145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f4105880 msgr2=0x7fc5f4100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:23:23.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5f99ca700 1 --2- 172.21.15.138:0/855332145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f4105880 0x7fc5f4100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:23.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5f99ca700 1 -- 172.21.15.138:0/855332145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5f41081b0 msgr2=0x7fc5f4101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:23:23.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5f99ca700 1 --2- 172.21.15.138:0/855332145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5f41081b0 0x7fc5f4101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:23.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5f99ca700 1 -- 172.21.15.138:0/855332145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f4101cf0 con 0x7fc5f4102f50 2024-06-10T00:23:23.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5f99ca700 1 --2- 172.21.15.138:0/855332145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f4102f50 0x7fc5f41028b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000c9b0 tx=0x7fc5f000ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:23:23.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5eaffd700 1 -- 172.21.15.138:0/855332145 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0007b00 con 0x7fc5f4102f50 2024-06-10T00:23:23.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5eaffd700 1 -- 172.21.15.138:0/855332145 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f000ce50 con 0x7fc5f4102f50 2024-06-10T00:23:23.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5eaffd700 1 -- 172.21.15.138:0/855332145 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0018660 con 0x7fc5f4102f50 2024-06-10T00:23:23.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/855332145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f41a27b0 con 0x7fc5f4102f50 2024-06-10T00:23:23.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.820+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/855332145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f41a2cf0 con 0x7fc5f4102f50 2024-06-10T00:23:23.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.821+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/855332145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f4069180 con 0x7fc5f4102f50 2024-06-10T00:23:23.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.822+0000 7fc5eaffd700 1 -- 172.21.15.138:0/855332145 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fc5f000f450 con 0x7fc5f4102f50 2024-06-10T00:23:23.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.823+0000 7fc5eaffd700 1 --2- 172.21.15.138:0/855332145 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5e00a2c10 0x7fc5e00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:23:23.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.823+0000 7fc5eaffd700 1 -- 172.21.15.138:0/855332145 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7fc5f00cd670 con 0x7fc5f4102f50 2024-06-10T00:23:23.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.823+0000 7fc5f91c9700 1 --2- 172.21.15.138:0/855332145 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5e00a2c10 0x7fc5e00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:23:23.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.824+0000 7fc5f91c9700 1 --2- 172.21.15.138:0/855332145 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5e00a2c10 0x7fc5e00a50d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400b5c0 tx=0x7fc5e4003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:23:23.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:23.828+0000 7fc5eaffd700 1 -- 172.21.15.138:0/855332145 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc5f0014030 con 0x7fc5f4102f50 2024-06-10T00:23:24.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:24.032+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/855332145 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc5f40628d0 con 0x7fc5e00a2c10 2024-06-10T00:23:24.936 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:24 smithi138 ceph-mon[36549]: osdmap e14: 2 total, 2 up, 2 in 2024-06-10T00:23:24.936 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:24 smithi138 ceph-mon[36549]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:24.936 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:24 smithi138 ceph-mon[36549]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:23:24.936 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:23:24.936 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:23:24.936 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:25.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:24 smithi062 ceph-mon[32390]: osdmap e14: 2 total, 2 up, 2 in 2024-06-10T00:23:25.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:24 smithi062 ceph-mon[32390]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:25.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:24 smithi062 ceph-mon[32390]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:23:25.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:23:25.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:23:25.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:24 smithi105 ceph-mon[36321]: osdmap e14: 2 total, 2 up, 2 in 2024-06-10T00:23:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:24 smithi105 ceph-mon[36321]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:24 smithi105 ceph-mon[36321]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:23:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:23:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:23:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:27.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:26 smithi138 ceph-mon[36549]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:27.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:26 smithi138 ceph-mon[36549]: from='client.? 172.21.15.138:0/4079815600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863db396-689a-4e13-8b18-534f0e38a86f"}]: dispatch 2024-06-10T00:23:27.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:26 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863db396-689a-4e13-8b18-534f0e38a86f"}]: dispatch 2024-06-10T00:23:27.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:26 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "863db396-689a-4e13-8b18-534f0e38a86f"}]': finished 2024-06-10T00:23:27.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:26 smithi138 ceph-mon[36549]: osdmap e15: 3 total, 2 up, 3 in 2024-06-10T00:23:27.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:26 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:27.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:26 smithi105 ceph-mon[36321]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:27.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:26 smithi105 ceph-mon[36321]: from='client.? 172.21.15.138:0/4079815600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863db396-689a-4e13-8b18-534f0e38a86f"}]: dispatch 2024-06-10T00:23:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:26 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863db396-689a-4e13-8b18-534f0e38a86f"}]: dispatch 2024-06-10T00:23:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:26 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "863db396-689a-4e13-8b18-534f0e38a86f"}]': finished 2024-06-10T00:23:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:26 smithi105 ceph-mon[36321]: osdmap e15: 3 total, 2 up, 3 in 2024-06-10T00:23:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:26 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:27.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:26 smithi062 ceph-mon[32390]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:27.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:26 smithi062 ceph-mon[32390]: from='client.? 172.21.15.138:0/4079815600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863db396-689a-4e13-8b18-534f0e38a86f"}]: dispatch 2024-06-10T00:23:27.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:26 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863db396-689a-4e13-8b18-534f0e38a86f"}]: dispatch 2024-06-10T00:23:27.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:26 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "863db396-689a-4e13-8b18-534f0e38a86f"}]': finished 2024-06-10T00:23:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:26 smithi062 ceph-mon[32390]: osdmap e15: 3 total, 2 up, 3 in 2024-06-10T00:23:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:26 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:28.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:27 smithi105 ceph-mon[36321]: from='client.? 172.21.15.138:0/4233804711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:23:28.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:27 smithi062 ceph-mon[32390]: from='client.? 172.21.15.138:0/4233804711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:23:28.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:27 smithi138 ceph-mon[36549]: from='client.? 172.21.15.138:0/4233804711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:23:29.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:28 smithi105 ceph-mon[36321]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:29.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:28 smithi062 ceph-mon[32390]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:29.372 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:28 smithi138 ceph-mon[36549]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:31.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:30 smithi105 ceph-mon[36321]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:31.287 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:30 smithi138 ceph-mon[36549]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:31.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:30 smithi062 ceph-mon[32390]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:33.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:32 smithi138 ceph-mon[36549]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:33.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:32 smithi105 ceph-mon[36321]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:33.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:32 smithi062 ceph-mon[32390]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:34.260 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-10T00:23:34.260 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:34.260 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:33 smithi138 ceph-mon[36549]: Deploying daemon osd.2 on smithi138 2024-06-10T00:23:34.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-10T00:23:34.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:34.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:33 smithi105 ceph-mon[36321]: Deploying daemon osd.2 on smithi138 2024-06-10T00:23:34.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-10T00:23:34.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:34.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:33 smithi062 ceph-mon[32390]: Deploying daemon osd.2 on smithi138 2024-06-10T00:23:35.224 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:34 smithi138 ceph-mon[36549]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:35.270 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:34 smithi105 ceph-mon[36321]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:35.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:34 smithi062 ceph-mon[32390]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:36.965 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:36 smithi138 ceph-mon[36549]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:37.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:36 smithi105 ceph-mon[36321]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:37.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:36 smithi062 ceph-mon[32390]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:39.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:39 smithi105 ceph-mon[36321]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:39.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:39 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:39.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:39 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:39.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:39 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:23:39.795 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:39 smithi138 ceph-mon[36549]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:39.795 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:39 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:39.795 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:39 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:39.795 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:39 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:23:39.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:39 smithi062 ceph-mon[32390]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:39.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:39 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:39.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:39 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:39.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:39 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:23:41.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:41.753+0000 7fc5eaffd700 1 -- 172.21.15.138:0/855332145 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc5f40628d0 con 0x7fc5e00a2c10 2024-06-10T00:23:41.753 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:41 smithi138 ceph-mon[36549]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:41.754 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 2 on host 'smithi138' 2024-06-10T00:23:41.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:41.755+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/855332145 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5e00a2c10 msgr2=0x7fc5e00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:23:41.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:41.755+0000 7fc5fbc2e700 1 --2- 172.21.15.138:0/855332145 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5e00a2c10 0x7fc5e00a50d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400b5c0 tx=0x7fc5e4003960 comp rx=0 tx=0).stop 2024-06-10T00:23:41.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:41.755+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/855332145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f4102f50 msgr2=0x7fc5f41028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:23:41.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:41.755+0000 7fc5fbc2e700 1 --2- 172.21.15.138:0/855332145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f4102f50 0x7fc5f41028b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000c9b0 tx=0x7fc5f000ccc0 comp rx=0 tx=0).stop 2024-06-10T00:23:41.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:41.755+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/855332145 shutdown_connections 2024-06-10T00:23:41.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:41.755+0000 7fc5fbc2e700 1 --2- 172.21.15.138:0/855332145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5f41081b0 0x7fc5f4101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:41.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:41.755+0000 7fc5fbc2e700 1 --2- 172.21.15.138:0/855332145 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5e00a2c10 0x7fc5e00a50d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:41.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:41.755+0000 7fc5fbc2e700 1 --2- 172.21.15.138:0/855332145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f4102f50 0x7fc5f41028b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:41.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:41.755+0000 7fc5fbc2e700 1 --2- 172.21.15.138:0/855332145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f4105880 0x7fc5f4100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:41.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:41.755+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/855332145 >> 172.21.15.138:0/855332145 conn(0x7fc5f40fc980 msgr2=0x7fc5f4103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:23:41.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:41.755+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/855332145 shutdown_connections 2024-06-10T00:23:41.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:23:41.756+0000 7fc5fbc2e700 1 -- 172.21.15.138:0/855332145 wait complete. 2024-06-10T00:23:41.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:41 smithi105 ceph-mon[36321]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:41.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:41 smithi062 ceph-mon[32390]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:42.386 DEBUG:teuthology.orchestra.run.smithi138:osd.2> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.2.service 2024-06-10T00:23:42.388 INFO:tasks.cephadm:Deploying osd.3 on smithi062 with /dev/vg_nvme/lv_3... 2024-06-10T00:23:42.389 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-10T00:23:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:23:42.869 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:42 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:42.869 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:42 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:42.869 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:42 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:42.869 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:42 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:42.870 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:42 smithi138 ceph-mon[36549]: from='osd.2 [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:23:42.870 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:42 smithi138 ceph-mon[36549]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:23:43.000 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:42 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:43.000 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:42 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:43.000 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:42 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:43.000 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:42 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:43.000 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:42 smithi062 ceph-mon[32390]: from='osd.2 [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:23:43.000 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:42 smithi062 ceph-mon[32390]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:23:43.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:42 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:43.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:42 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:43.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:42 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:43.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:42 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:43.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:42 smithi105 ceph-mon[36321]: from='osd.2 [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:23:43.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:42 smithi105 ceph-mon[36321]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:23:43.850 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:43 smithi138 ceph-mon[36549]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:43.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:43 smithi138 ceph-mon[36549]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-10T00:23:43.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:43 smithi138 ceph-mon[36549]: osdmap e16: 3 total, 2 up, 3 in 2024-06-10T00:23:43.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:43.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:43 smithi138 ceph-mon[36549]: from='osd.2 [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:23:43.851 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:43 smithi138 ceph-mon[36549]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:23:44.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:43 smithi105 ceph-mon[36321]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:44.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:43 smithi105 ceph-mon[36321]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-10T00:23:44.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:43 smithi105 ceph-mon[36321]: osdmap e16: 3 total, 2 up, 3 in 2024-06-10T00:23:44.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:44.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:43 smithi105 ceph-mon[36321]: from='osd.2 [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:23:44.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:43 smithi105 ceph-mon[36321]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:23:44.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:43 smithi062 ceph-mon[32390]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:44.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:43 smithi062 ceph-mon[32390]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-10T00:23:44.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:43 smithi062 ceph-mon[32390]: osdmap e16: 3 total, 2 up, 3 in 2024-06-10T00:23:44.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:44.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:43 smithi062 ceph-mon[32390]: from='osd.2 [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:23:44.033 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:43 smithi062 ceph-mon[32390]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:23:44.798 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-10T00:23:44.798 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-10T00:23:44.798 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-06-10T00:23:44.798 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-06-10T00:23:44.798 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128375 s, 817 MB/s 2024-06-10T00:23:44.799 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-06-10T00:23:45.011 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:44 smithi138 ceph-mon[36549]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-06-10T00:23:45.012 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:44 smithi138 ceph-mon[36549]: osdmap e17: 3 total, 2 up, 3 in 2024-06-10T00:23:45.012 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:44 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:45.012 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:44 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:45.012 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:44 smithi138 ceph-mon[36549]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:45.012 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:44 smithi138 ceph-mon[36549]: from='osd.2 ' entity='osd.2' 2024-06-10T00:23:45.013 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:23:44 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-2[41668]: 2024-06-10T00:23:44.640+0000 7ffa8ae75700 -1 osd.2 0 waiting for initial osdmap 2024-06-10T00:23:45.013 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:23:44 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-2[41668]: 2024-06-10T00:23:44.645+0000 7ffa82c60700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:23:45.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:44 smithi105 ceph-mon[36321]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-06-10T00:23:45.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:44 smithi105 ceph-mon[36321]: osdmap e17: 3 total, 2 up, 3 in 2024-06-10T00:23:45.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:44 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:45.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:44 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:45.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:44 smithi105 ceph-mon[36321]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:45.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:44 smithi105 ceph-mon[36321]: from='osd.2 ' entity='osd.2' 2024-06-10T00:23:45.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:44 smithi062 ceph-mon[32390]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-06-10T00:23:45.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:44 smithi062 ceph-mon[32390]: osdmap e17: 3 total, 2 up, 3 in 2024-06-10T00:23:45.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:44 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:45.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:44 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:45.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:44 smithi062 ceph-mon[32390]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:23:45.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:44 smithi062 ceph-mon[32390]: from='osd.2 ' entity='osd.2' 2024-06-10T00:23:45.491 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_3 2024-06-10T00:23:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:23:46.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: purged_snaps scrub starts 2024-06-10T00:23:46.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: purged_snaps scrub ok 2024-06-10T00:23:46.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:46.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: Detected new or changed devices on smithi138 2024-06-10T00:23:46.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:23:46.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi138 to 17088M 2024-06-10T00:23:46.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:46.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:23:46.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:23:46.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:46.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:23:46.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: osd.2 [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663] boot 2024-06-10T00:23:46.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: osdmap e18: 3 total, 3 up, 3 in 2024-06-10T00:23:46.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: purged_snaps scrub starts 2024-06-10T00:23:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: purged_snaps scrub ok 2024-06-10T00:23:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: Detected new or changed devices on smithi138 2024-06-10T00:23:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:23:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi138 to 17088M 2024-06-10T00:23:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:23:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:23:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:23:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: osd.2 [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663] boot 2024-06-10T00:23:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: osdmap e18: 3 total, 3 up, 3 in 2024-06-10T00:23:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:46.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: purged_snaps scrub starts 2024-06-10T00:23:46.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: purged_snaps scrub ok 2024-06-10T00:23:46.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:46.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: Detected new or changed devices on smithi138 2024-06-10T00:23:46.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:23:46.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi138 to 17088M 2024-06-10T00:23:46.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:46.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:23:46.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:23:46.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:46.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:23:46.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:46.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: osd.2 [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663] boot 2024-06-10T00:23:46.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: osdmap e18: 3 total, 3 up, 3 in 2024-06-10T00:23:46.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:23:47.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:46 smithi062 ceph-mon[32390]: pgmap v80: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:47.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:46 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:23:47.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:46 smithi138 ceph-mon[36549]: pgmap v80: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:47.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:46 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:23:47.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:46 smithi105 ceph-mon[36321]: pgmap v80: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:47.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:46 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:23:47.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.385+0000 7fc579f69700 1 -- 172.21.15.62:0/2733581814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc574102020 msgr2=0x7fc5741098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:23:47.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.385+0000 7fc579f69700 1 --2- 172.21.15.62:0/2733581814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc574102020 0x7fc5741098e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0099c0 tx=0x7fc56c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:23:47.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.385+0000 7fc579f69700 1 -- 172.21.15.62:0/2733581814 shutdown_connections 2024-06-10T00:23:47.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.385+0000 7fc579f69700 1 --2- 172.21.15.62:0/2733581814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc574102020 0x7fc5741098e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:47.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.385+0000 7fc579f69700 1 --2- 172.21.15.62:0/2733581814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc574102940 0x7fc574102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:47.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.385+0000 7fc579f69700 1 --2- 172.21.15.62:0/2733581814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc574101680 0x7fc574101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:47.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.385+0000 7fc579f69700 1 -- 172.21.15.62:0/2733581814 >> 172.21.15.62:0/2733581814 conn(0x7fc5740fc0f0 msgr2=0x7fc5740fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:23:47.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.385+0000 7fc579f69700 1 -- 172.21.15.62:0/2733581814 shutdown_connections 2024-06-10T00:23:47.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.385+0000 7fc579f69700 1 -- 172.21.15.62:0/2733581814 wait complete. 2024-06-10T00:23:47.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.386+0000 7fc579f69700 1 Processor -- start 2024-06-10T00:23:47.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.386+0000 7fc579f69700 1 -- start start 2024-06-10T00:23:47.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.386+0000 7fc579f69700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc574101680 0x7fc574105420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:23:47.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.386+0000 7fc579f69700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc574102020 0x7fc574103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:23:47.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.386+0000 7fc579f69700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc574102940 0x7fc574103fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:23:47.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.386+0000 7fc579f69700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc57410cf80 con 0x7fc574101680 2024-06-10T00:23:47.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.386+0000 7fc579f69700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc57410ce00 con 0x7fc574102940 2024-06-10T00:23:47.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.386+0000 7fc579f69700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc57410d100 con 0x7fc574102020 2024-06-10T00:23:47.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.386+0000 7fc5737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc574101680 0x7fc574105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:23:47.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.386+0000 7fc5737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc574101680 0x7fc574105420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38438/0 (socket says 172.21.15.62:38438) 2024-06-10T00:23:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.386+0000 7fc5737fe700 1 -- 172.21.15.62:0/53077511 learned_addr learned my addr 172.21.15.62:0/53077511 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:23:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.387+0000 7fc573fff700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc574102940 0x7fc574103fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:23:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.387+0000 7fc572ffd700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc574102020 0x7fc574103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:23:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.387+0000 7fc5737fe700 1 -- 172.21.15.62:0/53077511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc574102020 msgr2=0x7fc574103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:23:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.387+0000 7fc5737fe700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc574102020 0x7fc574103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.387+0000 7fc5737fe700 1 -- 172.21.15.62:0/53077511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc574102940 msgr2=0x7fc574103fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:23:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.387+0000 7fc5737fe700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc574102940 0x7fc574103fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:23:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.387+0000 7fc5737fe700 1 -- 172.21.15.62:0/53077511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc574104860 con 0x7fc574101680 2024-06-10T00:23:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.387+0000 7fc572ffd700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc574102020 0x7fc574103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:23:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.387+0000 7fc5737fe700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc574101680 0x7fc574105420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc56400ada0 tx=0x7fc56400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:23:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.387+0000 7fc570ff9700 1 -- 172.21.15.62:0/53077511 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564017020 con 0x7fc574101680 2024-06-10T00:23:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.387+0000 7fc579f69700 1 -- 172.21.15.62:0/53077511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5741a2810 con 0x7fc574101680 2024-06-10T00:23:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.387+0000 7fc570ff9700 1 -- 172.21.15.62:0/53077511 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc56400c980 con 0x7fc574101680 2024-06-10T00:23:47.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.387+0000 7fc570ff9700 1 -- 172.21.15.62:0/53077511 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5640044c0 con 0x7fc574101680 2024-06-10T00:23:47.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.387+0000 7fc579f69700 1 -- 172.21.15.62:0/53077511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5741a2d00 con 0x7fc574101680 2024-06-10T00:23:47.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.388+0000 7fc579f69700 1 -- 172.21.15.62:0/53077511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc574069180 con 0x7fc574101680 2024-06-10T00:23:47.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.389+0000 7fc570ff9700 1 -- 172.21.15.62:0/53077511 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fc56400cb30 con 0x7fc574101680 2024-06-10T00:23:47.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.390+0000 7fc570ff9700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5600a2b90 0x7fc5600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:23:47.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.390+0000 7fc572ffd700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5600a2b90 0x7fc5600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:23:47.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.390+0000 7fc570ff9700 1 -- 172.21.15.62:0/53077511 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7fc5640cd2e0 con 0x7fc574101680 2024-06-10T00:23:47.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.391+0000 7fc572ffd700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5600a2b90 0x7fc5600a5050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc55c005fd0 tx=0x7fc55c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:23:47.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.395+0000 7fc570ff9700 1 -- 172.21.15.62:0/53077511 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc564021020 con 0x7fc574101680 2024-06-10T00:23:47.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:47.613+0000 7fc579f69700 1 -- 172.21.15.62:0/53077511 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc5740fe220 con 0x7fc5600a2b90 2024-06-10T00:23:48.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:47 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-10T00:23:48.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:47 smithi062 ceph-mon[32390]: osdmap e19: 3 total, 3 up, 3 in 2024-06-10T00:23:48.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:47 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:23:48.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:47 smithi062 ceph-mon[32390]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:23:48.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:47 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:23:48.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:47 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:23:48.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:47 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:48.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:47 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-10T00:23:48.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:47 smithi138 ceph-mon[36549]: osdmap e19: 3 total, 3 up, 3 in 2024-06-10T00:23:48.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:47 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:23:48.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:47 smithi138 ceph-mon[36549]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:23:48.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:47 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:23:48.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:47 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:23:48.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:47 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:48.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:47 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-10T00:23:48.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:47 smithi105 ceph-mon[36321]: osdmap e19: 3 total, 3 up, 3 in 2024-06-10T00:23:48.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:47 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:23:48.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:47 smithi105 ceph-mon[36321]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:23:48.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:47 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:23:48.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:47 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:23:48.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:47 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:23:48.588 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:23:48 smithi062 sudo[51864]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:23:48.589 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:23:48 smithi062 sudo[51864]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:23:48.589 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:23:48 smithi062 sudo[51864]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:23:48.589 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:23:48 smithi062 sudo[51864]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:23:49.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:48 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-10T00:23:49.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:48 smithi062 ceph-mon[32390]: osdmap e20: 3 total, 3 up, 3 in 2024-06-10T00:23:49.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:48 smithi062 ceph-mon[32390]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:49.108 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:23:48 smithi062 sudo[51882]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:23:49.108 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:23:48 smithi062 sudo[51882]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:23:49.108 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:23:48 smithi062 sudo[51882]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:23:49.108 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:23:48 smithi062 sudo[51882]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:23:49.120 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:48 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-10T00:23:49.120 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:48 smithi105 ceph-mon[36321]: osdmap e20: 3 total, 3 up, 3 in 2024-06-10T00:23:49.121 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:48 smithi105 ceph-mon[36321]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:49.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:48 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-10T00:23:49.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:48 smithi138 ceph-mon[36549]: osdmap e20: 3 total, 3 up, 3 in 2024-06-10T00:23:49.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:48 smithi138 ceph-mon[36549]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:49.374 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:23:49 smithi105 sudo[45031]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:23:49.374 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:23:49 smithi105 sudo[45031]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:23:49.374 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:23:49 smithi105 sudo[45031]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:23:49.374 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:23:49 smithi105 sudo[45031]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:23:49.781 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:23:49 smithi105 sudo[45034]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:23:49.781 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:23:49 smithi105 sudo[45034]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:23:49.781 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:23:49 smithi105 sudo[45034]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:23:49.782 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:23:49 smithi105 sudo[45034]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:23:49.896 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:49 smithi138 ceph-mon[36549]: osdmap e21: 3 total, 3 up, 3 in 2024-06-10T00:23:49.896 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:49.896 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:49.896 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:23:49 smithi138 sudo[45418]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:23:49.896 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:23:49 smithi138 sudo[45418]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:23:49.896 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:23:49 smithi138 sudo[45418]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:23:49.896 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:23:49 smithi138 sudo[45418]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:23:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:49 smithi062 ceph-mon[32390]: osdmap e21: 3 total, 3 up, 3 in 2024-06-10T00:23:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:50.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:50.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:49 smithi105 ceph-mon[36321]: osdmap e21: 3 total, 3 up, 3 in 2024-06-10T00:23:50.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:50.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:50.373 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:23:49 smithi138 sudo[45421]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:23:50.373 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:23:49 smithi138 sudo[45421]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:23:50.373 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:23:49 smithi138 sudo[45421]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:23:50.373 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:23:49 smithi138 sudo[45421]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:23:50.833 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:50 smithi062 sudo[51955]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:23:50.833 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:50 smithi062 sudo[51955]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:23:50.833 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:50 smithi062 sudo[51955]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:23:50.833 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:50 smithi062 sudo[51955]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:23:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:50 smithi062 sudo[51958]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:23:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:50 smithi062 sudo[51958]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:23:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:50 smithi062 sudo[51958]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:23:51.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:50 smithi062 sudo[51958]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:23:51.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 sudo[45424]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:23:51.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 sudo[45424]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:23:51.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 sudo[45424]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:23:51.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 sudo[45424]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:23:51.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 sudo[45427]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:23:51.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 sudo[45427]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:23:51.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 sudo[45427]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:23:51.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 sudo[45427]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:23:51.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 sudo[45037]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:23:51.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 sudo[45037]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:23:51.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 sudo[45037]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:23:51.816 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 sudo[45037]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:23:51.817 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 sudo[45040]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:23:51.817 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 sudo[45040]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:23:51.817 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 sudo[45040]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:23:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:23:51.830+0000 7fc570ff9700 1 -- 172.21.15.62:0/53077511 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc564097700 con 0x7fc574101680 2024-06-10T00:23:52.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:51 smithi062 ceph-mon[32390]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:52.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:51 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:52.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:51 smithi062 ceph-mon[32390]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:23:52.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:51 smithi062 ceph-mon[32390]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:23:52.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:51 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:23:52.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:51 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:23:52.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:51 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:23:52.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:51 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:23:52.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:51 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:23:52.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:51 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:23:52.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 ceph-mon[36549]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:52.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:52.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 ceph-mon[36549]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:23:52.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 ceph-mon[36549]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:23:52.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:23:52.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:23:52.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:23:52.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:23:52.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:23:52.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:51 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:23:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 sudo[45040]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:23:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 ceph-mon[36321]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:23:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 ceph-mon[36321]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:23:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 ceph-mon[36321]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:23:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:23:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:23:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:23:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:23:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:23:52.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:51 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:23:53.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:52 smithi062 ceph-mon[32390]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:23:53.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:52 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2658491534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "756165fb-7807-41e8-a200-d23686c300a8"}]: dispatch 2024-06-10T00:23:53.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:52 smithi062 ceph-mon[32390]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:23:53.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:52 smithi062 ceph-mon[32390]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:23:53.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:52 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:23:53.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:52 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:23:53.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:52 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:23:53.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:52 smithi062 ceph-mon[32390]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:23:53.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:52 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2658491534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "756165fb-7807-41e8-a200-d23686c300a8"}]': finished 2024-06-10T00:23:53.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:52 smithi062 ceph-mon[32390]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-06-10T00:23:53.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:52 smithi062 ceph-mon[32390]: osdmap e22: 4 total, 3 up, 4 in 2024-06-10T00:23:53.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:52 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:23:53.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:52 smithi062 ceph-mon[32390]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:53.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:52 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3577928002' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:23:53.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:52 smithi138 ceph-mon[36549]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:23:53.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:52 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2658491534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "756165fb-7807-41e8-a200-d23686c300a8"}]: dispatch 2024-06-10T00:23:53.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:52 smithi138 ceph-mon[36549]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:23:53.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:52 smithi138 ceph-mon[36549]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:23:53.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:52 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:23:53.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:52 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:23:53.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:52 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:23:53.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:52 smithi138 ceph-mon[36549]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:23:53.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:52 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2658491534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "756165fb-7807-41e8-a200-d23686c300a8"}]': finished 2024-06-10T00:23:53.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:52 smithi138 ceph-mon[36549]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-06-10T00:23:53.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:52 smithi138 ceph-mon[36549]: osdmap e22: 4 total, 3 up, 4 in 2024-06-10T00:23:53.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:52 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:23:53.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:52 smithi138 ceph-mon[36549]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:53.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:52 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3577928002' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:23:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:52 smithi105 ceph-mon[36321]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:23:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:52 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2658491534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "756165fb-7807-41e8-a200-d23686c300a8"}]: dispatch 2024-06-10T00:23:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:52 smithi105 ceph-mon[36321]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:23:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:52 smithi105 ceph-mon[36321]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:23:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:52 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:23:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:52 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:23:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:52 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:23:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:52 smithi105 ceph-mon[36321]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:23:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:52 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2658491534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "756165fb-7807-41e8-a200-d23686c300a8"}]': finished 2024-06-10T00:23:53.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:52 smithi105 ceph-mon[36321]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-06-10T00:23:53.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:52 smithi105 ceph-mon[36321]: osdmap e22: 4 total, 3 up, 4 in 2024-06-10T00:23:53.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:52 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:23:53.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:52 smithi105 ceph-mon[36321]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:53.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:52 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3577928002' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:23:55.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:54 smithi105 ceph-mon[36321]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:55.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:54 smithi062 ceph-mon[32390]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:55.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:54 smithi138 ceph-mon[36549]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:57.081 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:56 smithi062 ceph-mon[32390]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:57.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:56 smithi105 ceph-mon[36321]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:57.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:56 smithi138 ceph-mon[36549]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:59.084 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:58 smithi062 ceph-mon[32390]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:59.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:58 smithi105 ceph-mon[36321]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:23:59.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:58 smithi138 ceph-mon[36549]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:00.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-10T00:24:00.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:00.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:23:59 smithi062 ceph-mon[32390]: Deploying daemon osd.3 on smithi062 2024-06-10T00:24:00.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:59 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-10T00:24:00.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:59 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:00.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:23:59 smithi105 ceph-mon[36321]: Deploying daemon osd.3 on smithi062 2024-06-10T00:24:00.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:59 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-10T00:24:00.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:59 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:00.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:23:59 smithi138 ceph-mon[36549]: Deploying daemon osd.3 on smithi062 2024-06-10T00:24:01.028 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:00 smithi062 ceph-mon[32390]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:01.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:00 smithi138 ceph-mon[36549]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:01.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:00 smithi105 ceph-mon[36321]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:03.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:02 smithi105 ceph-mon[36321]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:03.304 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:02 smithi138 ceph-mon[36549]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:03.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:02 smithi062 ceph-mon[32390]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:06.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:05 smithi105 ceph-mon[36321]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:06.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:06.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:06.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:06.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:05 smithi062 ceph-mon[32390]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:06.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:06.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:06.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:06.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:05 smithi138 ceph-mon[36549]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:06.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:06.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:06.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:08.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:07 smithi105 ceph-mon[36321]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:08.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:07 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:08.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:07 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:08.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:07 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:08.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:07 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:24:08.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:07 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:08.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:07 smithi062 ceph-mon[32390]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:08.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:07 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:08.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:07 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:08.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:07 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:08.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:07 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:24:08.088 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:07 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:08.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:07 smithi138 ceph-mon[36549]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:08.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:07 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:08.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:07 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:08.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:07 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:08.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:07 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:24:08.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:07 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:08.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:24:08.419+0000 7fc570ff9700 1 -- 172.21.15.62:0/53077511 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc5740fe220 con 0x7fc5600a2b90 2024-06-10T00:24:08.420 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 3 on host 'smithi062' 2024-06-10T00:24:08.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:24:08.422+0000 7fc579f69700 1 -- 172.21.15.62:0/53077511 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5600a2b90 msgr2=0x7fc5600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:24:08.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:24:08.422+0000 7fc579f69700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5600a2b90 0x7fc5600a5050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc55c005fd0 tx=0x7fc55c005dc0 comp rx=0 tx=0).stop 2024-06-10T00:24:08.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:24:08.422+0000 7fc579f69700 1 -- 172.21.15.62:0/53077511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc574101680 msgr2=0x7fc574105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:24:08.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:24:08.422+0000 7fc579f69700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc574101680 0x7fc574105420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc56400ada0 tx=0x7fc56400c330 comp rx=0 tx=0).stop 2024-06-10T00:24:08.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:24:08.422+0000 7fc579f69700 1 -- 172.21.15.62:0/53077511 shutdown_connections 2024-06-10T00:24:08.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:24:08.422+0000 7fc579f69700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc574101680 0x7fc574105420 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:08.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:24:08.422+0000 7fc579f69700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5600a2b90 0x7fc5600a5050 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:08.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:24:08.422+0000 7fc579f69700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc574102940 0x7fc574103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:08.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:24:08.422+0000 7fc579f69700 1 --2- 172.21.15.62:0/53077511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc574102020 0x7fc574103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:08.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:24:08.422+0000 7fc579f69700 1 -- 172.21.15.62:0/53077511 >> 172.21.15.62:0/53077511 conn(0x7fc5740fc0f0 msgr2=0x7fc5740fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:24:08.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:24:08.422+0000 7fc579f69700 1 -- 172.21.15.62:0/53077511 shutdown_connections 2024-06-10T00:24:08.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:24:08.422+0000 7fc579f69700 1 -- 172.21.15.62:0/53077511 wait complete. 2024-06-10T00:24:09.042 DEBUG:teuthology.orchestra.run.smithi062:osd.3> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.3.service 2024-06-10T00:24:09.044 INFO:tasks.cephadm:Deploying osd.4 on smithi105 with /dev/vg_nvme/lv_3... 2024-06-10T00:24:09.044 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-10T00:24:09.284 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:24:09.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:09 smithi062 ceph-mon[32390]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:09.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:09.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:09 smithi062 ceph-mon[32390]: from='osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-10T00:24:09.670 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:09 smithi105 ceph-mon[36321]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:09.670 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:09.670 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:09.670 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:09.670 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:09 smithi105 ceph-mon[36321]: from='osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-10T00:24:09.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:09 smithi138 ceph-mon[36549]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:09.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:09.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:09.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:09.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:09 smithi138 ceph-mon[36549]: from='osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-10T00:24:10.674 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:10 smithi105 ceph-mon[36321]: from='osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-10T00:24:10.674 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:10 smithi105 ceph-mon[36321]: osdmap e23: 4 total, 3 up, 4 in 2024-06-10T00:24:10.674 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:10 smithi105 ceph-mon[36321]: from='osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-06-10T00:24:10.674 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:10 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:24:10.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:10 smithi062 ceph-mon[32390]: from='osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-10T00:24:10.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:10 smithi062 ceph-mon[32390]: osdmap e23: 4 total, 3 up, 4 in 2024-06-10T00:24:10.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:10 smithi062 ceph-mon[32390]: from='osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-06-10T00:24:10.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:24:10.858 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:24:10 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3[53986]: 2024-06-10T00:24:10.428+0000 7f0819715700 -1 osd.3 0 waiting for initial osdmap 2024-06-10T00:24:10.859 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:24:10 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3[53986]: 2024-06-10T00:24:10.435+0000 7f0815508700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:24:10.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:10 smithi138 ceph-mon[36549]: from='osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-10T00:24:10.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:10 smithi138 ceph-mon[36549]: osdmap e23: 4 total, 3 up, 4 in 2024-06-10T00:24:10.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:10 smithi138 ceph-mon[36549]: from='osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-06-10T00:24:10.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:24:11.293 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-10T00:24:11.293 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-10T00:24:11.293 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-06-10T00:24:11.293 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-06-10T00:24:11.293 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113213 s, 926 MB/s 2024-06-10T00:24:11.293 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-06-10T00:24:11.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:11 smithi062 ceph-mon[32390]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:11.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:11 smithi062 ceph-mon[32390]: from='osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-06-10T00:24:11.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:11 smithi062 ceph-mon[32390]: osdmap e24: 4 total, 3 up, 4 in 2024-06-10T00:24:11.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:24:11.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:24:11.667 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:11 smithi105 ceph-mon[36321]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:11.667 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:11 smithi105 ceph-mon[36321]: from='osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-06-10T00:24:11.667 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:11 smithi105 ceph-mon[36321]: osdmap e24: 4 total, 3 up, 4 in 2024-06-10T00:24:11.667 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:11 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:24:11.667 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:11 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:24:11.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:11 smithi138 ceph-mon[36549]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:11.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:11 smithi138 ceph-mon[36549]: from='osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-06-10T00:24:11.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:11 smithi138 ceph-mon[36549]: osdmap e24: 4 total, 3 up, 4 in 2024-06-10T00:24:11.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:24:11.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:24:11.960 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi105:vg_nvme/lv_3 2024-06-10T00:24:12.199 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:24:12.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:12 smithi105 ceph-mon[36321]: purged_snaps scrub starts 2024-06-10T00:24:12.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:12 smithi105 ceph-mon[36321]: purged_snaps scrub ok 2024-06-10T00:24:12.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:12 smithi105 ceph-mon[36321]: osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835] boot 2024-06-10T00:24:12.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:12 smithi105 ceph-mon[36321]: osdmap e25: 4 total, 4 up, 4 in 2024-06-10T00:24:12.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:12 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:24:12.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:12 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:12.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:12 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:12.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:12 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:24:12.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:12 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:24:12.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:12 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:12.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:12 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:12.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:12 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:24:12.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:12 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:12.811 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:12 smithi138 ceph-mon[36549]: purged_snaps scrub starts 2024-06-10T00:24:12.811 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:12 smithi138 ceph-mon[36549]: purged_snaps scrub ok 2024-06-10T00:24:12.811 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:12 smithi138 ceph-mon[36549]: osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835] boot 2024-06-10T00:24:12.811 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:12 smithi138 ceph-mon[36549]: osdmap e25: 4 total, 4 up, 4 in 2024-06-10T00:24:12.811 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:12 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:24:12.811 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:12 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:12.811 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:12 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:12.811 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:12 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:24:12.811 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:12 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:24:12.811 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:12 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:12.811 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:12 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:12.811 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:12 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:24:12.812 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:12 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:12.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:12 smithi062 ceph-mon[32390]: purged_snaps scrub starts 2024-06-10T00:24:12.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:12 smithi062 ceph-mon[32390]: purged_snaps scrub ok 2024-06-10T00:24:12.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:12 smithi062 ceph-mon[32390]: osd.3 [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835] boot 2024-06-10T00:24:12.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:12 smithi062 ceph-mon[32390]: osdmap e25: 4 total, 4 up, 4 in 2024-06-10T00:24:12.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:12 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:24:12.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:12 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:12.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:12 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:12.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:12 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:24:12.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:12 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:24:12.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:12 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:12.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:12 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:12.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:12 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:24:12.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:12 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:13.580 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:13 smithi105 ceph-mon[36321]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:13.580 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:13 smithi105 ceph-mon[36321]: Detected new or changed devices on smithi062 2024-06-10T00:24:13.580 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:13 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi062 to 8544M 2024-06-10T00:24:13.580 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:13 smithi105 ceph-mon[36321]: osdmap e26: 4 total, 4 up, 4 in 2024-06-10T00:24:13.760 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:13 smithi062 ceph-mon[32390]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:13.760 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:13 smithi062 ceph-mon[32390]: Detected new or changed devices on smithi062 2024-06-10T00:24:13.760 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:13 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi062 to 8544M 2024-06-10T00:24:13.760 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:13 smithi062 ceph-mon[32390]: osdmap e26: 4 total, 4 up, 4 in 2024-06-10T00:24:13.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.777+0000 7f7563340700 1 -- 172.21.15.105:0/3856048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f755c102f90 msgr2=0x7f755c109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:24:13.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.777+0000 7f7563340700 1 --2- 172.21.15.105:0/3856048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f755c102f90 0x7f755c109840 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f75580099c0 tx=0x7f7558009cd0 comp rx=0 tx=0).stop 2024-06-10T00:24:13.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.777+0000 7f7563340700 1 -- 172.21.15.105:0/3856048 shutdown_connections 2024-06-10T00:24:13.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.777+0000 7f7563340700 1 --2- 172.21.15.105:0/3856048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f755c102f90 0x7f755c109840 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:13.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.777+0000 7f7563340700 1 --2- 172.21.15.105:0/3856048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f755c1013f0 0x7f755c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:13.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.777+0000 7f7563340700 1 --2- 172.21.15.105:0/3856048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f755c1025f0 0x7f755c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:13.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.777+0000 7f7563340700 1 -- 172.21.15.105:0/3856048 >> 172.21.15.105:0/3856048 conn(0x7f755c0fc9b0 msgr2=0x7f755c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:24:13.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.777+0000 7f7563340700 1 -- 172.21.15.105:0/3856048 shutdown_connections 2024-06-10T00:24:13.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.777+0000 7f7563340700 1 -- 172.21.15.105:0/3856048 wait complete. 2024-06-10T00:24:13.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.777+0000 7f7563340700 1 Processor -- start 2024-06-10T00:24:13.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f7563340700 1 -- start start 2024-06-10T00:24:13.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f7563340700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f755c1013f0 0x7f755c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:24:13.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f7563340700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f755c1025f0 0x7f755c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:24:13.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f7563340700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f755c102f90 0x7f755c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:24:13.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f7563340700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f755c10d050 con 0x7f755c1025f0 2024-06-10T00:24:13.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f7563340700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f755c10ced0 con 0x7f755c1013f0 2024-06-10T00:24:13.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f7563340700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f755c10d1d0 con 0x7f755c102f90 2024-06-10T00:24:13.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f75610dc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f755c1013f0 0x7f755c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:24:13.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f75610dc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f755c1013f0 0x7f755c1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53728/0 (socket says 172.21.15.105:53728) 2024-06-10T00:24:13.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f75610dc700 1 -- 172.21.15.105:0/1890277105 learned_addr learned my addr 172.21.15.105:0/1890277105 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-06-10T00:24:13.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f75618dd700 1 --2- 172.21.15.105:0/1890277105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f755c102f90 0x7f755c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:24:13.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f75608db700 1 --2- 172.21.15.105:0/1890277105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f755c1025f0 0x7f755c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:24:13.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f75610dc700 1 -- 172.21.15.105:0/1890277105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f755c102f90 msgr2=0x7f755c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:24:13.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f75610dc700 1 --2- 172.21.15.105:0/1890277105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f755c102f90 0x7f755c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:13.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f75610dc700 1 -- 172.21.15.105:0/1890277105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f755c1025f0 msgr2=0x7f755c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:24:13.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f75610dc700 1 --2- 172.21.15.105:0/1890277105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f755c1025f0 0x7f755c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:13.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f75610dc700 1 -- 172.21.15.105:0/1890277105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755c19f240 con 0x7f755c1013f0 2024-06-10T00:24:13.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f75610dc700 1 --2- 172.21.15.105:0/1890277105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f755c1013f0 0x7f755c1975e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f755400b9c0 tx=0x7f755400bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:24:13.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f75527fc700 1 -- 172.21.15.105:0/1890277105 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554019070 con 0x7f755c1013f0 2024-06-10T00:24:13.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f7563340700 1 -- 172.21.15.105:0/1890277105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755c19f530 con 0x7f755c1013f0 2024-06-10T00:24:13.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f75527fc700 1 -- 172.21.15.105:0/1890277105 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f755400ce30 con 0x7f755c1013f0 2024-06-10T00:24:13.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.778+0000 7f75527fc700 1 -- 172.21.15.105:0/1890277105 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75540123a0 con 0x7f755c1013f0 2024-06-10T00:24:13.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.779+0000 7f7563340700 1 -- 172.21.15.105:0/1890277105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f755c19faf0 con 0x7f755c1013f0 2024-06-10T00:24:13.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.780+0000 7f7563340700 1 -- 172.21.15.105:0/1890277105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f755c069180 con 0x7f755c1013f0 2024-06-10T00:24:13.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.781+0000 7f75527fc700 1 -- 172.21.15.105:0/1890277105 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f755401c070 con 0x7f755c1013f0 2024-06-10T00:24:13.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.781+0000 7f75527fc700 1 --2- 172.21.15.105:0/1890277105 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f75480a2c20 0x7f75480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:24:13.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.782+0000 7f75608db700 1 --2- 172.21.15.105:0/1890277105 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f75480a2c20 0x7f75480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:24:13.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.782+0000 7f75527fc700 1 -- 172.21.15.105:0/1890277105 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f75540cd1e0 con 0x7f755c1013f0 2024-06-10T00:24:13.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.782+0000 7f75608db700 1 --2- 172.21.15.105:0/1890277105 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f75480a2c20 0x7f75480a50e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f754c009bf0 tx=0x7f754c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:24:13.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.786+0000 7f75527fc700 1 -- 172.21.15.105:0/1890277105 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f755409b340 con 0x7f755c1013f0 2024-06-10T00:24:13.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:13 smithi138 ceph-mon[36549]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:24:13.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:13 smithi138 ceph-mon[36549]: Detected new or changed devices on smithi062 2024-06-10T00:24:13.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:13 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi062 to 8544M 2024-06-10T00:24:13.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:13 smithi138 ceph-mon[36549]: osdmap e26: 4 total, 4 up, 4 in 2024-06-10T00:24:13.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:13.993+0000 7f7563340700 1 -- 172.21.15.105:0/1890277105 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f755c0628d0 con 0x7f75480a2c20 2024-06-10T00:24:14.725 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:14 smithi105 ceph-mon[36321]: osdmap e27: 4 total, 4 up, 4 in 2024-06-10T00:24:14.725 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:14 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:24:14.725 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:14 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:24:14.725 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:14 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:14.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:14 smithi062 ceph-mon[32390]: osdmap e27: 4 total, 4 up, 4 in 2024-06-10T00:24:14.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:14 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:24:14.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:14 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:24:14.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:14 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:14.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:14 smithi138 ceph-mon[36549]: osdmap e27: 4 total, 4 up, 4 in 2024-06-10T00:24:14.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:14 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:24:14.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:14 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:24:14.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:14 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:15.625 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:15 smithi105 ceph-mon[36321]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:15.625 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:15 smithi105 ceph-mon[36321]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:24:15.625 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:15 smithi105 ceph-mon[36321]: osdmap e28: 4 total, 4 up, 4 in 2024-06-10T00:24:15.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:15 smithi062 ceph-mon[32390]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:15.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:15 smithi062 ceph-mon[32390]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:24:15.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:15 smithi062 ceph-mon[32390]: osdmap e28: 4 total, 4 up, 4 in 2024-06-10T00:24:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:15 smithi138 ceph-mon[36549]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:15 smithi138 ceph-mon[36549]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:24:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:15 smithi138 ceph-mon[36549]: osdmap e28: 4 total, 4 up, 4 in 2024-06-10T00:24:16.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:16 smithi105 ceph-mon[36321]: osdmap e29: 4 total, 4 up, 4 in 2024-06-10T00:24:16.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:16 smithi062 ceph-mon[32390]: osdmap e29: 4 total, 4 up, 4 in 2024-06-10T00:24:16.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:16 smithi138 ceph-mon[36549]: osdmap e29: 4 total, 4 up, 4 in 2024-06-10T00:24:17.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:17 smithi105 ceph-mon[36321]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:17 smithi105 ceph-mon[36321]: from='client.? 172.21.15.105:0/1639009233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73ed9b45-1a37-4078-83e4-e34607aa2835"}]: dispatch 2024-06-10T00:24:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:17 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73ed9b45-1a37-4078-83e4-e34607aa2835"}]: dispatch 2024-06-10T00:24:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:17 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73ed9b45-1a37-4078-83e4-e34607aa2835"}]': finished 2024-06-10T00:24:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:17 smithi105 ceph-mon[36321]: osdmap e30: 5 total, 4 up, 5 in 2024-06-10T00:24:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:17 smithi105 ceph-mon[36321]: from='client.? 172.21.15.105:0/2545439908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:24:17.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:17 smithi062 ceph-mon[32390]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:17.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:17 smithi062 ceph-mon[32390]: from='client.? 172.21.15.105:0/1639009233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73ed9b45-1a37-4078-83e4-e34607aa2835"}]: dispatch 2024-06-10T00:24:17.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:17 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73ed9b45-1a37-4078-83e4-e34607aa2835"}]: dispatch 2024-06-10T00:24:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:17 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73ed9b45-1a37-4078-83e4-e34607aa2835"}]': finished 2024-06-10T00:24:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:17 smithi062 ceph-mon[32390]: osdmap e30: 5 total, 4 up, 5 in 2024-06-10T00:24:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:17 smithi062 ceph-mon[32390]: from='client.? 172.21.15.105:0/2545439908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:24:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:17 smithi138 ceph-mon[36549]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:17 smithi138 ceph-mon[36549]: from='client.? 172.21.15.105:0/1639009233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73ed9b45-1a37-4078-83e4-e34607aa2835"}]: dispatch 2024-06-10T00:24:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:17 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73ed9b45-1a37-4078-83e4-e34607aa2835"}]: dispatch 2024-06-10T00:24:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:17 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73ed9b45-1a37-4078-83e4-e34607aa2835"}]': finished 2024-06-10T00:24:17.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:17 smithi138 ceph-mon[36549]: osdmap e30: 5 total, 4 up, 5 in 2024-06-10T00:24:17.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:17.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:17 smithi138 ceph-mon[36549]: from='client.? 172.21.15.105:0/2545439908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:24:19.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:19 smithi105 ceph-mon[36321]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-06-10T00:24:19.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:19 smithi062 ceph-mon[32390]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-06-10T00:24:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:19 smithi138 ceph-mon[36549]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-06-10T00:24:21.726 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:21 smithi105 ceph-mon[36321]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:24:21.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:21 smithi062 ceph-mon[32390]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:24:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:21 smithi138 ceph-mon[36549]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:24:23.716 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:23 smithi105 ceph-mon[36321]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-06-10T00:24:23.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:23 smithi062 ceph-mon[32390]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-06-10T00:24:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:23 smithi138 ceph-mon[36549]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-06-10T00:24:24.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-10T00:24:24.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:24.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-10T00:24:24.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:24.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-10T00:24:24.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:25.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:25 smithi105 ceph-mon[36321]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-06-10T00:24:25.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:25 smithi105 ceph-mon[36321]: Deploying daemon osd.4 on smithi105 2024-06-10T00:24:25.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:25 smithi062 ceph-mon[32390]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-06-10T00:24:25.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:25 smithi062 ceph-mon[32390]: Deploying daemon osd.4 on smithi105 2024-06-10T00:24:25.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:25 smithi138 ceph-mon[36549]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-06-10T00:24:25.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:25 smithi138 ceph-mon[36549]: Deploying daemon osd.4 on smithi105 2024-06-10T00:24:27.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:27 smithi105 ceph-mon[36321]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:24:27.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:27 smithi062 ceph-mon[32390]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:24:27.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:27 smithi138 ceph-mon[36549]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:24:29.676 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:29 smithi105 ceph-mon[36321]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-10T00:24:29.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:29 smithi062 ceph-mon[32390]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-10T00:24:29.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:29 smithi138 ceph-mon[36549]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-10T00:24:31.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:30 smithi105 ceph-mon[36321]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:31.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:30 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:31.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:30 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:31.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:30 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:31.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:30 smithi062 ceph-mon[32390]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:31.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:30 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:31.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:30 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:31.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:30 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:30 smithi138 ceph-mon[36549]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:30 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:30 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:30 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:33.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:33.666+0000 7f75527fc700 1 -- 172.21.15.105:0/1890277105 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f755c0628d0 con 0x7f75480a2c20 2024-06-10T00:24:33.667 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 4 on host 'smithi105' 2024-06-10T00:24:33.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:33.669+0000 7f7563340700 1 -- 172.21.15.105:0/1890277105 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f75480a2c20 msgr2=0x7f75480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:24:33.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:33.669+0000 7f7563340700 1 --2- 172.21.15.105:0/1890277105 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f75480a2c20 0x7f75480a50e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f754c009bf0 tx=0x7f754c009260 comp rx=0 tx=0).stop 2024-06-10T00:24:33.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:33.669+0000 7f7563340700 1 -- 172.21.15.105:0/1890277105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f755c1013f0 msgr2=0x7f755c1975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:24:33.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:33.669+0000 7f7563340700 1 --2- 172.21.15.105:0/1890277105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f755c1013f0 0x7f755c1975e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f755400b9c0 tx=0x7f755400bcd0 comp rx=0 tx=0).stop 2024-06-10T00:24:33.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:33.669+0000 7f7563340700 1 -- 172.21.15.105:0/1890277105 shutdown_connections 2024-06-10T00:24:33.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:33.669+0000 7f7563340700 1 --2- 172.21.15.105:0/1890277105 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f75480a2c20 0x7f75480a50e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:33.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:33.669+0000 7f7563340700 1 --2- 172.21.15.105:0/1890277105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f755c102f90 0x7f755c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:33.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:33.669+0000 7f7563340700 1 --2- 172.21.15.105:0/1890277105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f755c1013f0 0x7f755c1975e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:33.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:33.669+0000 7f7563340700 1 --2- 172.21.15.105:0/1890277105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f755c1025f0 0x7f755c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:33.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:33.669+0000 7f7563340700 1 -- 172.21.15.105:0/1890277105 >> 172.21.15.105:0/1890277105 conn(0x7f755c0fc9b0 msgr2=0x7f755c105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:24:33.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:33.669+0000 7f7563340700 1 -- 172.21.15.105:0/1890277105 shutdown_connections 2024-06-10T00:24:33.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:24:33.669+0000 7f7563340700 1 -- 172.21.15.105:0/1890277105 wait complete. 2024-06-10T00:24:34.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:33 smithi105 ceph-mon[36321]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:34.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:34.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:24:34.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:34.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:33 smithi062 ceph-mon[32390]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:34.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:34.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:24:34.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:34.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:33 smithi138 ceph-mon[36549]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:34.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:34.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:24:34.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:34.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:34.386 DEBUG:teuthology.orchestra.run.smithi105:osd.4> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.4.service 2024-06-10T00:24:34.389 INFO:tasks.cephadm:Deploying osd.5 on smithi138 with /dev/vg_nvme/lv_3... 2024-06-10T00:24:34.389 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-10T00:24:34.631 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:24:34.876 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:34 smithi105 ceph-mon[36321]: from='osd.4 [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:24:34.876 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:34 smithi105 ceph-mon[36321]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:24:35.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:34 smithi062 ceph-mon[32390]: from='osd.4 [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:24:35.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:34 smithi062 ceph-mon[32390]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:24:35.120 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:34 smithi138 ceph-mon[36549]: from='osd.4 [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:24:35.120 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:34 smithi138 ceph-mon[36549]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:24:35.964 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:35 smithi105 ceph-mon[36321]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:35.964 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:35 smithi105 ceph-mon[36321]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-10T00:24:35.964 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:35 smithi105 ceph-mon[36321]: from='osd.4 [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:24:35.964 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:35 smithi105 ceph-mon[36321]: osdmap e31: 5 total, 4 up, 5 in 2024-06-10T00:24:35.965 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:35.965 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:35 smithi105 ceph-mon[36321]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:24:35.981 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:35 smithi138 ceph-mon[36549]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:35.981 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:35 smithi138 ceph-mon[36549]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-10T00:24:35.981 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:35 smithi138 ceph-mon[36549]: from='osd.4 [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:24:35.981 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:35 smithi138 ceph-mon[36549]: osdmap e31: 5 total, 4 up, 5 in 2024-06-10T00:24:35.981 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:35.981 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:35 smithi138 ceph-mon[36549]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:24:36.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:35 smithi062 ceph-mon[32390]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:36.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:35 smithi062 ceph-mon[32390]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-10T00:24:36.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:35 smithi062 ceph-mon[32390]: from='osd.4 [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:24:36.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:35 smithi062 ceph-mon[32390]: osdmap e31: 5 total, 4 up, 5 in 2024-06-10T00:24:36.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:36.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:35 smithi062 ceph-mon[32390]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:24:36.577 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-10T00:24:36.577 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-10T00:24:36.578 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-06-10T00:24:36.578 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-06-10T00:24:36.578 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125515 s, 835 MB/s 2024-06-10T00:24:36.578 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-06-10T00:24:36.777 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:36 smithi105 ceph-mon[36321]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-06-10T00:24:36.778 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:36 smithi105 ceph-mon[36321]: osdmap e32: 5 total, 4 up, 5 in 2024-06-10T00:24:36.778 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:24:36 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-4[47821]: 2024-06-10T00:24:36.479+0000 7fb62dda6700 -1 osd.4 0 waiting for initial osdmap 2024-06-10T00:24:36.778 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:24:36 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-4[47821]: 2024-06-10T00:24:36.485+0000 7fb625b91700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:24:36.921 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:36 smithi138 ceph-mon[36549]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-06-10T00:24:36.922 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:36 smithi138 ceph-mon[36549]: osdmap e32: 5 total, 4 up, 5 in 2024-06-10T00:24:36.922 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:36 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:36.922 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:36 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:36.922 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:36 smithi138 ceph-mon[36549]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:36.922 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:36 smithi138 ceph-mon[36549]: from='osd.4 ' entity='osd.4' 2024-06-10T00:24:36.922 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:36 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:37.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:36 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:37.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:36 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:37.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:36 smithi105 ceph-mon[36321]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:37.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:36 smithi105 ceph-mon[36321]: from='osd.4 ' entity='osd.4' 2024-06-10T00:24:37.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:36 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:37.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:36 smithi062 ceph-mon[32390]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-06-10T00:24:37.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:36 smithi062 ceph-mon[32390]: osdmap e32: 5 total, 4 up, 5 in 2024-06-10T00:24:37.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:36 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:37.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:36 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:37.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:36 smithi062 ceph-mon[32390]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:24:37.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:36 smithi062 ceph-mon[32390]: from='osd.4 ' entity='osd.4' 2024-06-10T00:24:37.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:36 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:37.183 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_3 2024-06-10T00:24:37.479 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:24:37.840 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: purged_snaps scrub starts 2024-06-10T00:24:37.840 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: purged_snaps scrub ok 2024-06-10T00:24:37.840 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: osd.4 [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738] boot 2024-06-10T00:24:37.840 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: osdmap e33: 5 total, 5 up, 5 in 2024-06-10T00:24:37.840 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:37.840 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: Detected new or changed devices on smithi105 2024-06-10T00:24:37.840 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:37.840 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:37.840 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:24:37.840 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:24:37.840 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi105 to 8544M 2024-06-10T00:24:37.841 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:37.841 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:37.841 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:24:37.841 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:37 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:38.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: purged_snaps scrub starts 2024-06-10T00:24:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: purged_snaps scrub ok 2024-06-10T00:24:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: osd.4 [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738] boot 2024-06-10T00:24:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: osdmap e33: 5 total, 5 up, 5 in 2024-06-10T00:24:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: Detected new or changed devices on smithi105 2024-06-10T00:24:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:24:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:24:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi105 to 8544M 2024-06-10T00:24:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:38.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:24:38.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:37 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:38.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: purged_snaps scrub starts 2024-06-10T00:24:38.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: purged_snaps scrub ok 2024-06-10T00:24:38.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: osd.4 [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738] boot 2024-06-10T00:24:38.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: osdmap e33: 5 total, 5 up, 5 in 2024-06-10T00:24:38.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:24:38.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: Detected new or changed devices on smithi105 2024-06-10T00:24:38.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:38.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:38.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:24:38.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:24:38.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi105 to 8544M 2024-06-10T00:24:38.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:38.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:38.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:24:38.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:37 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:38.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.936+0000 7fe66132a700 1 -- 172.21.15.138:0/742807066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c1058b0 msgr2=0x7fe65c107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:24:38.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.936+0000 7fe66132a700 1 --2- 172.21.15.138:0/742807066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c1058b0 0x7fe65c107ca0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe650009990 tx=0x7fe650009ca0 comp rx=0 tx=0).stop 2024-06-10T00:24:38.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.936+0000 7fe66132a700 1 -- 172.21.15.138:0/742807066 shutdown_connections 2024-06-10T00:24:38.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.936+0000 7fe66132a700 1 --2- 172.21.15.138:0/742807066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c102f80 0x7fe65c105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:38.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.936+0000 7fe66132a700 1 --2- 172.21.15.138:0/742807066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe65c1081e0 0x7fe65c10a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:38.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.936+0000 7fe66132a700 1 --2- 172.21.15.138:0/742807066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c1058b0 0x7fe65c107ca0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:38.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.936+0000 7fe66132a700 1 -- 172.21.15.138:0/742807066 >> 172.21.15.138:0/742807066 conn(0x7fe65c0fc990 msgr2=0x7fe65c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:24:38.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.937+0000 7fe66132a700 1 -- 172.21.15.138:0/742807066 shutdown_connections 2024-06-10T00:24:38.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.937+0000 7fe66132a700 1 -- 172.21.15.138:0/742807066 wait complete. 2024-06-10T00:24:38.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.937+0000 7fe66132a700 1 Processor -- start 2024-06-10T00:24:38.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.937+0000 7fe66132a700 1 -- start start 2024-06-10T00:24:38.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.937+0000 7fe66132a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe65c102f80 0x7fe65c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:24:38.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.937+0000 7fe66132a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c1058b0 0x7fe65c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:24:38.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.937+0000 7fe66132a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c1081e0 0x7fe65c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:24:38.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.937+0000 7fe66132a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c10e100 con 0x7fe65c1081e0 2024-06-10T00:24:38.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.937+0000 7fe66132a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c10df80 con 0x7fe65c102f80 2024-06-10T00:24:38.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.937+0000 7fe66132a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c10e280 con 0x7fe65c1058b0 2024-06-10T00:24:38.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe65a7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c1058b0 0x7fe65c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:24:38.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe65a7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c1058b0 0x7fe65c198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43662/0 (socket says 172.21.15.138:43662) 2024-06-10T00:24:38.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe65a7fc700 1 -- 172.21.15.138:0/1932832734 learned_addr learned my addr 172.21.15.138:0/1932832734 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-06-10T00:24:38.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe65affd700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe65c102f80 0x7fe65c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:24:38.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe65b7fe700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c1081e0 0x7fe65c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:24:38.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe65a7fc700 1 -- 172.21.15.138:0/1932832734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe65c102f80 msgr2=0x7fe65c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:24:38.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe65a7fc700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe65c102f80 0x7fe65c198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:38.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe65a7fc700 1 -- 172.21.15.138:0/1932832734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c1081e0 msgr2=0x7fe65c19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:24:38.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe65a7fc700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c1081e0 0x7fe65c19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:38.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe65a7fc700 1 -- 172.21.15.138:0/1932832734 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65c1a0380 con 0x7fe65c1058b0 2024-06-10T00:24:38.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe65b7fe700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c1081e0 0x7fe65c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:24:38.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe65a7fc700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c1058b0 0x7fe65c198c60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe650005f20 tx=0x7fe650009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:24:38.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe643fff700 1 -- 172.21.15.138:0/1932832734 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650027070 con 0x7fe65c1058b0 2024-06-10T00:24:38.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe643fff700 1 -- 172.21.15.138:0/1932832734 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe650004150 con 0x7fe65c1058b0 2024-06-10T00:24:38.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe66132a700 1 -- 172.21.15.138:0/1932832734 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65c1a0610 con 0x7fe65c1058b0 2024-06-10T00:24:38.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe643fff700 1 -- 172.21.15.138:0/1932832734 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65000f460 con 0x7fe65c1058b0 2024-06-10T00:24:38.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.938+0000 7fe66132a700 1 -- 172.21.15.138:0/1932832734 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65c1a0ac0 con 0x7fe65c1058b0 2024-06-10T00:24:38.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.939+0000 7fe66132a700 1 -- 172.21.15.138:0/1932832734 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe65c069180 con 0x7fe65c1058b0 2024-06-10T00:24:38.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.940+0000 7fe643fff700 1 -- 172.21.15.138:0/1932832734 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe65002a070 con 0x7fe65c1058b0 2024-06-10T00:24:38.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.941+0000 7fe643fff700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe6440a2ca0 0x7fe6440a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:24:38.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.941+0000 7fe643fff700 1 -- 172.21.15.138:0/1932832734 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fe6500ceee0 con 0x7fe65c1058b0 2024-06-10T00:24:38.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.941+0000 7fe65affd700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe6440a2ca0 0x7fe6440a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:24:38.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.942+0000 7fe65affd700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe6440a2ca0 0x7fe6440a5160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00ac50 tx=0x7fe64c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:24:38.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:38.946+0000 7fe643fff700 1 -- 172.21.15.138:0/1932832734 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe65005e850 con 0x7fe65c1058b0 2024-06-10T00:24:39.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:38 smithi062 ceph-mon[32390]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:39.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:38 smithi138 ceph-mon[36549]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:39.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:39.150+0000 7fe66132a700 1 -- 172.21.15.138:0/1932832734 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe65c0628d0 con 0x7fe6440a2ca0 2024-06-10T00:24:39.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:38 smithi105 ceph-mon[36321]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:40.062 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:39 smithi138 ceph-mon[36549]: osdmap e34: 5 total, 5 up, 5 in 2024-06-10T00:24:40.062 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:39 smithi138 ceph-mon[36549]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:24:40.062 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:39 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:24:40.062 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:39 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:24:40.062 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:39 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:40.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:39 smithi062 ceph-mon[32390]: osdmap e34: 5 total, 5 up, 5 in 2024-06-10T00:24:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:39 smithi062 ceph-mon[32390]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:24:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:39 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:24:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:39 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:24:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:39 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:40.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:39 smithi105 ceph-mon[36321]: osdmap e34: 5 total, 5 up, 5 in 2024-06-10T00:24:40.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:39 smithi105 ceph-mon[36321]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:24:40.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:39 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:24:40.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:39 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:24:40.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:39 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:41.017 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:40 smithi138 ceph-mon[36549]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:41.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:40 smithi062 ceph-mon[32390]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:41.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:40 smithi105 ceph-mon[36321]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:42.209 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:41 smithi138 ceph-mon[36549]: from='client.? 172.21.15.138:0/108880164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c0fe663-9fd5-465e-808f-210bfd7ac201"}]: dispatch 2024-06-10T00:24:42.210 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:41 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c0fe663-9fd5-465e-808f-210bfd7ac201"}]: dispatch 2024-06-10T00:24:42.210 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:41 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c0fe663-9fd5-465e-808f-210bfd7ac201"}]': finished 2024-06-10T00:24:42.210 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:41 smithi138 ceph-mon[36549]: osdmap e35: 6 total, 5 up, 6 in 2024-06-10T00:24:42.210 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:41 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:24:42.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:41 smithi105 ceph-mon[36321]: from='client.? 172.21.15.138:0/108880164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c0fe663-9fd5-465e-808f-210bfd7ac201"}]: dispatch 2024-06-10T00:24:42.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:41 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c0fe663-9fd5-465e-808f-210bfd7ac201"}]: dispatch 2024-06-10T00:24:42.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:41 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c0fe663-9fd5-465e-808f-210bfd7ac201"}]': finished 2024-06-10T00:24:42.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:41 smithi105 ceph-mon[36321]: osdmap e35: 6 total, 5 up, 6 in 2024-06-10T00:24:42.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:41 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:24:42.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:41 smithi062 ceph-mon[32390]: from='client.? 172.21.15.138:0/108880164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c0fe663-9fd5-465e-808f-210bfd7ac201"}]: dispatch 2024-06-10T00:24:42.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:41 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c0fe663-9fd5-465e-808f-210bfd7ac201"}]: dispatch 2024-06-10T00:24:42.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:41 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c0fe663-9fd5-465e-808f-210bfd7ac201"}]': finished 2024-06-10T00:24:42.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:41 smithi062 ceph-mon[32390]: osdmap e35: 6 total, 5 up, 6 in 2024-06-10T00:24:42.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:41 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:24:43.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:42 smithi062 ceph-mon[32390]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:43.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:42 smithi062 ceph-mon[32390]: from='client.? 172.21.15.138:0/4123189831' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:24:43.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:42 smithi105 ceph-mon[36321]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:43.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:42 smithi105 ceph-mon[36321]: from='client.? 172.21.15.138:0/4123189831' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:24:43.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:42 smithi138 ceph-mon[36549]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:43.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:42 smithi138 ceph-mon[36549]: from='client.? 172.21.15.138:0/4123189831' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:24:45.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:44 smithi105 ceph-mon[36321]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:45.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:44 smithi062 ceph-mon[32390]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:45.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:44 smithi138 ceph-mon[36549]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:47.220 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:46 smithi138 ceph-mon[36549]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:47.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:46 smithi105 ceph-mon[36321]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:47.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:46 smithi062 ceph-mon[32390]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:49.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:48 smithi105 ceph-mon[36321]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:49.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:48 smithi062 ceph-mon[32390]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:49.372 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:48 smithi138 ceph-mon[36549]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:50.101 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-10T00:24:50.101 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:50.101 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:49 smithi138 ceph-mon[36549]: Deploying daemon osd.5 on smithi138 2024-06-10T00:24:50.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-10T00:24:50.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:50.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:49 smithi105 ceph-mon[36321]: Deploying daemon osd.5 on smithi138 2024-06-10T00:24:50.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-10T00:24:50.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:24:50.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:49 smithi062 ceph-mon[32390]: Deploying daemon osd.5 on smithi138 2024-06-10T00:24:51.048 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:50 smithi138 ceph-mon[36549]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:51.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:50 smithi105 ceph-mon[36321]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:51.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:50 smithi062 ceph-mon[32390]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:53.034 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:52 smithi138 ceph-mon[36549]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:52 smithi105 ceph-mon[36321]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:53.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:52 smithi062 ceph-mon[32390]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:55 smithi138 ceph-mon[36549]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:55.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:55.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:55 smithi062 ceph-mon[32390]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:55.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:55.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:55.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:56.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:55 smithi105 ceph-mon[36321]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:56.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:56.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:56.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:24:57.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:57 smithi062 ceph-mon[32390]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:57.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:57 smithi138 ceph-mon[36549]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:57.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:57.967+0000 7fe643fff700 1 -- 172.21.15.138:0/1932832734 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe65c0628d0 con 0x7fe6440a2ca0 2024-06-10T00:24:57.967 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 5 on host 'smithi138' 2024-06-10T00:24:57.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:57.969+0000 7fe66132a700 1 -- 172.21.15.138:0/1932832734 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe6440a2ca0 msgr2=0x7fe6440a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:24:57.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:57.969+0000 7fe66132a700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe6440a2ca0 0x7fe6440a5160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00ac50 tx=0x7fe64c00a380 comp rx=0 tx=0).stop 2024-06-10T00:24:57.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:57.970+0000 7fe66132a700 1 -- 172.21.15.138:0/1932832734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c1058b0 msgr2=0x7fe65c198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:24:57.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:57.970+0000 7fe66132a700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c1058b0 0x7fe65c198c60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe650005f20 tx=0x7fe650009ee0 comp rx=0 tx=0).stop 2024-06-10T00:24:57.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:57.970+0000 7fe66132a700 1 -- 172.21.15.138:0/1932832734 shutdown_connections 2024-06-10T00:24:57.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:57.970+0000 7fe66132a700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe65c1081e0 0x7fe65c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:57.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:57.970+0000 7fe66132a700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe6440a2ca0 0x7fe6440a5160 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:57.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:57.970+0000 7fe66132a700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe65c102f80 0x7fe65c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:57.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:57.970+0000 7fe66132a700 1 --2- 172.21.15.138:0/1932832734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c1058b0 0x7fe65c198c60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:24:57.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:57.970+0000 7fe66132a700 1 -- 172.21.15.138:0/1932832734 >> 172.21.15.138:0/1932832734 conn(0x7fe65c0fc990 msgr2=0x7fe65c1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:24:57.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:57.970+0000 7fe66132a700 1 -- 172.21.15.138:0/1932832734 shutdown_connections 2024-06-10T00:24:57.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:24:57.970+0000 7fe66132a700 1 -- 172.21.15.138:0/1932832734 wait complete. 2024-06-10T00:24:58.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:57 smithi105 ceph-mon[36321]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:58.578 DEBUG:teuthology.orchestra.run.smithi138:osd.5> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.5.service 2024-06-10T00:24:58.581 INFO:tasks.cephadm:Deploying osd.6 on smithi062 with /dev/vg_nvme/lv_2... 2024-06-10T00:24:58.581 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-10T00:24:58.841 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:24:59.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:58 smithi062 ceph-mon[32390]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:59.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:58 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:59.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:58 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:59.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:58 smithi062 ceph-mon[32390]: from='osd.5 [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:24:59.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:58 smithi062 ceph-mon[32390]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:24:59.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:58 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:59.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:58 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:59.256 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:58 smithi138 ceph-mon[36549]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:59.256 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:58 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:59.256 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:58 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:59.256 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:58 smithi138 ceph-mon[36549]: from='osd.5 [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:24:59.256 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:58 smithi138 ceph-mon[36549]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:24:59.256 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:58 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:59.257 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:58 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:59.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:58 smithi105 ceph-mon[36321]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:24:59.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:58 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:59.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:58 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:59.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:58 smithi105 ceph-mon[36321]: from='osd.5 [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:24:59.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:58 smithi105 ceph-mon[36321]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:24:59.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:58 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:24:59.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:58 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:00.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:59 smithi105 ceph-mon[36321]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-10T00:25:00.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:59 smithi105 ceph-mon[36321]: from='osd.5 [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:25:00.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:59 smithi105 ceph-mon[36321]: osdmap e36: 6 total, 5 up, 6 in 2024-06-10T00:25:00.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:59 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:25:00.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:24:59 smithi105 ceph-mon[36321]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:25:00.307 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:59 smithi138 ceph-mon[36549]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-10T00:25:00.307 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:59 smithi138 ceph-mon[36549]: from='osd.5 [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:25:00.307 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:59 smithi138 ceph-mon[36549]: osdmap e36: 6 total, 5 up, 6 in 2024-06-10T00:25:00.307 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:59 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:25:00.307 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:24:59 smithi138 ceph-mon[36549]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:25:00.308 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:24:59 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5[48299]: 2024-06-10T00:24:59.979+0000 7fd5af0b7700 -1 osd.5 0 waiting for initial osdmap 2024-06-10T00:25:00.308 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:24:59 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5[48299]: 2024-06-10T00:24:59.986+0000 7fd5aaeaa700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:25:00.347 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:59 smithi062 ceph-mon[32390]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-10T00:25:00.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:59 smithi062 ceph-mon[32390]: from='osd.5 [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:25:00.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:59 smithi062 ceph-mon[32390]: osdmap e36: 6 total, 5 up, 6 in 2024-06-10T00:25:00.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:59 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:25:00.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:24:59 smithi062 ceph-mon[32390]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:25:01.210 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-10T00:25:01.210 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-10T00:25:01.210 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-06-10T00:25:01.210 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-06-10T00:25:01.210 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111339 s, 942 MB/s 2024-06-10T00:25:01.210 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-06-10T00:25:01.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:00 smithi062 ceph-mon[32390]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:25:01.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:00 smithi062 ceph-mon[32390]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-06-10T00:25:01.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:00 smithi062 ceph-mon[32390]: osdmap e37: 6 total, 5 up, 6 in 2024-06-10T00:25:01.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:00 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:25:01.234 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:00 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:25:01.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:00 smithi105 ceph-mon[36321]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:25:01.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:00 smithi105 ceph-mon[36321]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-06-10T00:25:01.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:00 smithi105 ceph-mon[36321]: osdmap e37: 6 total, 5 up, 6 in 2024-06-10T00:25:01.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:00 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:25:01.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:00 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:25:01.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:00 smithi138 ceph-mon[36549]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:25:01.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:00 smithi138 ceph-mon[36549]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-06-10T00:25:01.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:00 smithi138 ceph-mon[36549]: osdmap e37: 6 total, 5 up, 6 in 2024-06-10T00:25:01.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:00 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:25:01.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:00 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:25:01.845 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_2 2024-06-10T00:25:02.107 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:25:02.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:01 smithi062 ceph-mon[32390]: purged_snaps scrub starts 2024-06-10T00:25:02.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:01 smithi062 ceph-mon[32390]: purged_snaps scrub ok 2024-06-10T00:25:02.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:01 smithi062 ceph-mon[32390]: osd.5 [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519] boot 2024-06-10T00:25:02.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:01 smithi062 ceph-mon[32390]: osdmap e38: 6 total, 6 up, 6 in 2024-06-10T00:25:02.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:01 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:25:02.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:01 smithi062 ceph-mon[32390]: osdmap e39: 6 total, 6 up, 6 in 2024-06-10T00:25:02.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:01 smithi105 ceph-mon[36321]: purged_snaps scrub starts 2024-06-10T00:25:02.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:01 smithi105 ceph-mon[36321]: purged_snaps scrub ok 2024-06-10T00:25:02.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:01 smithi105 ceph-mon[36321]: osd.5 [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519] boot 2024-06-10T00:25:02.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:01 smithi105 ceph-mon[36321]: osdmap e38: 6 total, 6 up, 6 in 2024-06-10T00:25:02.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:01 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:25:02.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:01 smithi105 ceph-mon[36321]: osdmap e39: 6 total, 6 up, 6 in 2024-06-10T00:25:02.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:01 smithi138 ceph-mon[36549]: purged_snaps scrub starts 2024-06-10T00:25:02.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:01 smithi138 ceph-mon[36549]: purged_snaps scrub ok 2024-06-10T00:25:02.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:01 smithi138 ceph-mon[36549]: osd.5 [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519] boot 2024-06-10T00:25:02.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:01 smithi138 ceph-mon[36549]: osdmap e38: 6 total, 6 up, 6 in 2024-06-10T00:25:02.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:01 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:25:02.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:01 smithi138 ceph-mon[36549]: osdmap e39: 6 total, 6 up, 6 in 2024-06-10T00:25:03.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:03.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: Detected new or changed devices on smithi138 2024-06-10T00:25:03.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:03.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:03.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi138 to 8544M 2024-06-10T00:25:03.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:03.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:03.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:03.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:03.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:03.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:03 smithi105 ceph-mon[36321]: osdmap e40: 6 total, 6 up, 6 in 2024-06-10T00:25:03.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:03.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: Detected new or changed devices on smithi138 2024-06-10T00:25:03.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:03.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:03.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi138 to 8544M 2024-06-10T00:25:03.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:03.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:03.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:03.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:03.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:03.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:03 smithi062 ceph-mon[32390]: osdmap e40: 6 total, 6 up, 6 in 2024-06-10T00:25:03.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:03.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: Detected new or changed devices on smithi138 2024-06-10T00:25:03.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:03.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:03.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi138 to 8544M 2024-06-10T00:25:03.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:03.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:03.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:03.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:03.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:03.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:03.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:03 smithi138 ceph-mon[36549]: osdmap e40: 6 total, 6 up, 6 in 2024-06-10T00:25:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.700+0000 7f3062eab700 1 -- 172.21.15.62:0/1131600960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f305c1036e0 msgr2=0x7f305c103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:25:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.700+0000 7f3062eab700 1 --2- 172.21.15.62:0/1131600960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f305c1036e0 0x7f305c103b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f30480099c0 tx=0x7f3048009cd0 comp rx=0 tx=0).stop 2024-06-10T00:25:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.700+0000 7f3062eab700 1 -- 172.21.15.62:0/1131600960 shutdown_connections 2024-06-10T00:25:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.700+0000 7f3062eab700 1 --2- 172.21.15.62:0/1131600960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f305c104080 0x7f305c10a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.700+0000 7f3062eab700 1 --2- 172.21.15.62:0/1131600960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f305c1024e0 0x7f305c102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.700+0000 7f3062eab700 1 --2- 172.21.15.62:0/1131600960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f305c1036e0 0x7f305c103b40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.700+0000 7f3062eab700 1 -- 172.21.15.62:0/1131600960 >> 172.21.15.62:0/1131600960 conn(0x7f305c0fda80 msgr2=0x7f305c0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:25:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.700+0000 7f3062eab700 1 -- 172.21.15.62:0/1131600960 shutdown_connections 2024-06-10T00:25:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.700+0000 7f3062eab700 1 -- 172.21.15.62:0/1131600960 wait complete. 2024-06-10T00:25:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.700+0000 7f3062eab700 1 Processor -- start 2024-06-10T00:25:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.700+0000 7f3062eab700 1 -- start start 2024-06-10T00:25:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3062eab700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f305c1024e0 0x7f305c076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:25:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3062eab700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f305c1036e0 0x7f305c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:25:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3062eab700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f305c104080 0x7f305c074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:25:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3062eab700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f305c10e0b0 con 0x7f305c1024e0 2024-06-10T00:25:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3062eab700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f305c10df30 con 0x7f305c1036e0 2024-06-10T00:25:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3062eab700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f305c10e230 con 0x7f305c104080 2024-06-10T00:25:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3060c47700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f305c1024e0 0x7f305c076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:25:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3060c47700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f305c1024e0 0x7f305c076230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55348/0 (socket says 172.21.15.62:55348) 2024-06-10T00:25:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3060c47700 1 -- 172.21.15.62:0/3147777806 learned_addr learned my addr 172.21.15.62:0/3147777806 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:25:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f305bfff700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f305c1036e0 0x7f305c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:25:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3061448700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f305c104080 0x7f305c074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:25:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3060c47700 1 -- 172.21.15.62:0/3147777806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f305c104080 msgr2=0x7f305c074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:25:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3060c47700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f305c104080 0x7f305c074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3060c47700 1 -- 172.21.15.62:0/3147777806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f305c1036e0 msgr2=0x7f305c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:25:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3060c47700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f305c1036e0 0x7f305c074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3060c47700 1 -- 172.21.15.62:0/3147777806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305c075670 con 0x7f305c1024e0 2024-06-10T00:25:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3061448700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f305c104080 0x7f305c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:25:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3060c47700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f305c1024e0 0x7f305c076230 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f305000c970 tx=0x7f305000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:25:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.701+0000 7f3059ffb700 1 -- 172.21.15.62:0/3147777806 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050004220 con 0x7f305c1024e0 2024-06-10T00:25:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.702+0000 7f3059ffb700 1 -- 172.21.15.62:0/3147777806 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3050007d80 con 0x7f305c1024e0 2024-06-10T00:25:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.702+0000 7f3062eab700 1 -- 172.21.15.62:0/3147777806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305c1a2830 con 0x7f305c1024e0 2024-06-10T00:25:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.702+0000 7f3059ffb700 1 -- 172.21.15.62:0/3147777806 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050003e50 con 0x7f305c1024e0 2024-06-10T00:25:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.702+0000 7f3062eab700 1 -- 172.21.15.62:0/3147777806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305c1a2d40 con 0x7f305c1024e0 2024-06-10T00:25:03.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.703+0000 7f3062eab700 1 -- 172.21.15.62:0/3147777806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f305c069180 con 0x7f305c1024e0 2024-06-10T00:25:03.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.704+0000 7f3059ffb700 1 -- 172.21.15.62:0/3147777806 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f30500078a0 con 0x7f305c1024e0 2024-06-10T00:25:03.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.704+0000 7f3059ffb700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f304c0a2dc0 0x7f304c0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:25:03.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.705+0000 7f3059ffb700 1 -- 172.21.15.62:0/3147777806 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f30500cdda0 con 0x7f305c1024e0 2024-06-10T00:25:03.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.705+0000 7f305bfff700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f304c0a2dc0 0x7f304c0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:25:03.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.705+0000 7f305bfff700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f304c0a2dc0 0x7f304c0a5280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3048000c00 tx=0x7f3048005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:25:03.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.709+0000 7f3059ffb700 1 -- 172.21.15.62:0/3147777806 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3050014030 con 0x7f305c1024e0 2024-06-10T00:25:03.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:03.926+0000 7f3062eab700 1 -- 172.21.15.62:0/3147777806 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f305c0628d0 con 0x7f304c0a2dc0 2024-06-10T00:25:04.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:04 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:25:04.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:04 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:25:04.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:04 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:04.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:04 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:25:04.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:04 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:25:04.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:04 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:04.379 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:04 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:25:04.379 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:04 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:25:04.379 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:04 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:05.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:05 smithi062 ceph-mon[32390]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:05.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:05 smithi062 ceph-mon[32390]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:25:05.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:05 smithi062 ceph-mon[32390]: osdmap e41: 6 total, 6 up, 6 in 2024-06-10T00:25:05.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:05 smithi138 ceph-mon[36549]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:05.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:05 smithi138 ceph-mon[36549]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:25:05.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:05 smithi138 ceph-mon[36549]: osdmap e41: 6 total, 6 up, 6 in 2024-06-10T00:25:05.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:05 smithi105 ceph-mon[36321]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:05.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:05 smithi105 ceph-mon[36321]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:25:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:05 smithi105 ceph-mon[36321]: osdmap e41: 6 total, 6 up, 6 in 2024-06-10T00:25:06.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:06 smithi062 ceph-mon[32390]: osdmap e42: 6 total, 6 up, 6 in 2024-06-10T00:25:06.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:06 smithi138 ceph-mon[36549]: osdmap e42: 6 total, 6 up, 6 in 2024-06-10T00:25:06.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:06 smithi105 ceph-mon[36321]: osdmap e42: 6 total, 6 up, 6 in 2024-06-10T00:25:07.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:07 smithi062 ceph-mon[32390]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:07.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:07 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3549540041' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e"}]: dispatch 2024-06-10T00:25:07.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:07 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3549540041' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e"}]': finished 2024-06-10T00:25:07.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:07 smithi062 ceph-mon[32390]: osdmap e43: 7 total, 6 up, 7 in 2024-06-10T00:25:07.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:07 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:07.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:07 smithi138 ceph-mon[36549]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:07.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:07 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3549540041' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e"}]: dispatch 2024-06-10T00:25:07.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:07 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3549540041' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e"}]': finished 2024-06-10T00:25:07.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:07 smithi138 ceph-mon[36549]: osdmap e43: 7 total, 6 up, 7 in 2024-06-10T00:25:07.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:07 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:07.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:07 smithi105 ceph-mon[36321]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:07.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:07 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3549540041' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e"}]: dispatch 2024-06-10T00:25:07.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:07 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3549540041' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e"}]': finished 2024-06-10T00:25:07.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:07 smithi105 ceph-mon[36321]: osdmap e43: 7 total, 6 up, 7 in 2024-06-10T00:25:07.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:07 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:08.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:08 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/732781816' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:25:08.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:08 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/732781816' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:25:08.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:08 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/732781816' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:25:09.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:09 smithi062 ceph-mon[32390]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-10T00:25:09.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:09 smithi138 ceph-mon[36549]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-10T00:25:09.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:09 smithi105 ceph-mon[36321]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-10T00:25:11.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:11 smithi062 ceph-mon[32390]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:25:11.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:11 smithi138 ceph-mon[36549]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:25:11.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:11 smithi105 ceph-mon[36321]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:25:13.172 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:13 smithi062 ceph-mon[32390]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-10T00:25:13.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:13 smithi138 ceph-mon[36549]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-10T00:25:13.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:13 smithi105 ceph-mon[36321]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-10T00:25:15.301 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:15 smithi062 ceph-mon[32390]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-10T00:25:15.301 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:15 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-10T00:25:15.301 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:15 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:15.301 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:15 smithi062 ceph-mon[32390]: Deploying daemon osd.6 on smithi062 2024-06-10T00:25:15.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:15 smithi138 ceph-mon[36549]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-10T00:25:15.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:15 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-10T00:25:15.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:15 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:15.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:15 smithi138 ceph-mon[36549]: Deploying daemon osd.6 on smithi062 2024-06-10T00:25:15.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:15 smithi105 ceph-mon[36321]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-10T00:25:15.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:15 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-10T00:25:15.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:15 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:15.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:15 smithi105 ceph-mon[36321]: Deploying daemon osd.6 on smithi062 2024-06-10T00:25:17.350 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:17 smithi062 ceph-mon[32390]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:25:17.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:17 smithi138 ceph-mon[36549]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:25:17.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:17 smithi105 ceph-mon[36321]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:25:19.260 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:19 smithi062 ceph-mon[32390]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-10T00:25:19.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:19 smithi138 ceph-mon[36549]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-10T00:25:19.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:19 smithi105 ceph-mon[36321]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-10T00:25:20.211 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:20 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:20.211 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:20 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:20.211 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:20 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:20.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:20 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:20.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:20 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:20.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:20 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:20.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:20 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:20.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:20 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:20.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:20 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:21.789 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:21 smithi062 ceph-mon[32390]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:21 smithi138 ceph-mon[36549]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:22.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:21 smithi105 ceph-mon[36321]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:23.130 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 6 on host 'smithi062' 2024-06-10T00:25:23.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:23.129+0000 7f3059ffb700 1 -- 172.21.15.62:0/3147777806 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f305c0628d0 con 0x7f304c0a2dc0 2024-06-10T00:25:23.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:23.133+0000 7f3062eab700 1 -- 172.21.15.62:0/3147777806 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f304c0a2dc0 msgr2=0x7f304c0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:25:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:23.133+0000 7f3062eab700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f304c0a2dc0 0x7f304c0a5280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3048000c00 tx=0x7f3048005fb0 comp rx=0 tx=0).stop 2024-06-10T00:25:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:23.133+0000 7f3062eab700 1 -- 172.21.15.62:0/3147777806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f305c1024e0 msgr2=0x7f305c076230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:25:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:23.133+0000 7f3062eab700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f305c1024e0 0x7f305c076230 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f305000c970 tx=0x7f305000cc80 comp rx=0 tx=0).stop 2024-06-10T00:25:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:23.133+0000 7f3062eab700 1 -- 172.21.15.62:0/3147777806 shutdown_connections 2024-06-10T00:25:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:23.133+0000 7f3062eab700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f305c1024e0 0x7f305c076230 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:23.133+0000 7f3062eab700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f304c0a2dc0 0x7f304c0a5280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:23.133+0000 7f3062eab700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f305c104080 0x7f305c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:23.133+0000 7f3062eab700 1 --2- 172.21.15.62:0/3147777806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f305c1036e0 0x7f305c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:23.133+0000 7f3062eab700 1 -- 172.21.15.62:0/3147777806 >> 172.21.15.62:0/3147777806 conn(0x7f305c0fda80 msgr2=0x7f305c106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:25:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:23.133+0000 7f3062eab700 1 -- 172.21.15.62:0/3147777806 shutdown_connections 2024-06-10T00:25:23.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:25:23.133+0000 7f3062eab700 1 -- 172.21.15.62:0/3147777806 wait complete. 2024-06-10T00:25:23.819 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:23 smithi062 ceph-mon[32390]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:23.819 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:23.819 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:23.819 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:23 smithi062 ceph-mon[32390]: from='osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-10T00:25:23.851 DEBUG:teuthology.orchestra.run.smithi062:osd.6> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.6.service 2024-06-10T00:25:23.854 INFO:tasks.cephadm:Deploying osd.7 on smithi105 with /dev/vg_nvme/lv_2... 2024-06-10T00:25:23.854 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-10T00:25:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:23 smithi138 ceph-mon[36549]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:23 smithi138 ceph-mon[36549]: from='osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-10T00:25:23.881 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:23 smithi105 ceph-mon[36321]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:23.881 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:23.881 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:23.881 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:23 smithi105 ceph-mon[36321]: from='osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-10T00:25:24.098 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:25:25.252 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:25 smithi105 ceph-mon[36321]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:25.253 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:25 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:25.253 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:25 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:25.253 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:25 smithi105 ceph-mon[36321]: from='osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-10T00:25:25.253 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:25 smithi105 ceph-mon[36321]: osdmap e44: 7 total, 6 up, 7 in 2024-06-10T00:25:25.253 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:25 smithi105 ceph-mon[36321]: from='osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-06-10T00:25:25.253 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:25 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:25.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:25 smithi062 ceph-mon[32390]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:25.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:25 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:25.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:25 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:25.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:25 smithi062 ceph-mon[32390]: from='osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-10T00:25:25.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:25 smithi062 ceph-mon[32390]: osdmap e44: 7 total, 6 up, 7 in 2024-06-10T00:25:25.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:25 smithi062 ceph-mon[32390]: from='osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-06-10T00:25:25.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:25 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:25.358 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:25:25 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-6[60913]: 2024-06-10T00:25:25.136+0000 7f7d18682700 -1 osd.6 0 waiting for initial osdmap 2024-06-10T00:25:25.359 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:25:25 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-6[60913]: 2024-06-10T00:25:25.145+0000 7f7d14475700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:25:25.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:25 smithi138 ceph-mon[36549]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:25.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:25 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:25.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:25 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:25.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:25 smithi138 ceph-mon[36549]: from='osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-10T00:25:25.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:25 smithi138 ceph-mon[36549]: osdmap e44: 7 total, 6 up, 7 in 2024-06-10T00:25:25.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:25 smithi138 ceph-mon[36549]: from='osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-06-10T00:25:25.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:25 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-10T00:25:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-10T00:25:26.298 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-06-10T00:25:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-06-10T00:25:26.298 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114112 s, 919 MB/s 2024-06-10T00:25:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-06-10T00:25:26.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:26 smithi062 ceph-mon[32390]: from='osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-06-10T00:25:26.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:26 smithi062 ceph-mon[32390]: osdmap e45: 7 total, 6 up, 7 in 2024-06-10T00:25:26.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:26 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:26.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:26 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:26.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:26 smithi105 ceph-mon[36321]: from='osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-06-10T00:25:26.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:26 smithi105 ceph-mon[36321]: osdmap e45: 7 total, 6 up, 7 in 2024-06-10T00:25:26.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:26 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:26.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:26 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:26.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:26 smithi138 ceph-mon[36549]: from='osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-06-10T00:25:26.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:26 smithi138 ceph-mon[36549]: osdmap e45: 7 total, 6 up, 7 in 2024-06-10T00:25:26.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:26 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:26.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:26 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:26.935 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi105:vg_nvme/lv_2 2024-06-10T00:25:27.218 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:25:27.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:27 smithi105 ceph-mon[36321]: purged_snaps scrub starts 2024-06-10T00:25:27.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:27 smithi105 ceph-mon[36321]: purged_snaps scrub ok 2024-06-10T00:25:27.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:27 smithi105 ceph-mon[36321]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:27.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:27 smithi105 ceph-mon[36321]: osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325] boot 2024-06-10T00:25:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:27 smithi105 ceph-mon[36321]: osdmap e46: 7 total, 7 up, 7 in 2024-06-10T00:25:27.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:27 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:27.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:27 smithi062 ceph-mon[32390]: purged_snaps scrub starts 2024-06-10T00:25:27.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:27 smithi062 ceph-mon[32390]: purged_snaps scrub ok 2024-06-10T00:25:27.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:27 smithi062 ceph-mon[32390]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:27.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:27 smithi062 ceph-mon[32390]: osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325] boot 2024-06-10T00:25:27.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:27 smithi062 ceph-mon[32390]: osdmap e46: 7 total, 7 up, 7 in 2024-06-10T00:25:27.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:27 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:27 smithi138 ceph-mon[36549]: purged_snaps scrub starts 2024-06-10T00:25:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:27 smithi138 ceph-mon[36549]: purged_snaps scrub ok 2024-06-10T00:25:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:27 smithi138 ceph-mon[36549]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:25:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:27 smithi138 ceph-mon[36549]: osd.6 [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325] boot 2024-06-10T00:25:27.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:27 smithi138 ceph-mon[36549]: osdmap e46: 7 total, 7 up, 7 in 2024-06-10T00:25:27.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:27 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:25:28.524 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: osdmap e47: 7 total, 7 up, 7 in 2024-06-10T00:25:28.525 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: Detected new or changed devices on smithi062 2024-06-10T00:25:28.525 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:28.525 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:28.525 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:28.525 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:28.526 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:28.526 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi062 to 5696M 2024-06-10T00:25:28.526 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:28.526 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:28.526 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:28.527 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:28.527 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:28.527 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:28.527 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:28.527 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:28 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:28.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: osdmap e47: 7 total, 7 up, 7 in 2024-06-10T00:25:28.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: Detected new or changed devices on smithi062 2024-06-10T00:25:28.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:28.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:28.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:28.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:28.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:28.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi062 to 5696M 2024-06-10T00:25:28.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:28.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:28.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:28.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:28.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:28.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:28.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:28.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:28 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:28.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: osdmap e47: 7 total, 7 up, 7 in 2024-06-10T00:25:28.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: Detected new or changed devices on smithi062 2024-06-10T00:25:28.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:28.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:28.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:28.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:28.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:28.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi062 to 5696M 2024-06-10T00:25:28.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:28.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:28.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:28.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:28.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:28.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:28.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:28.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:28 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:29.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.046+0000 7ffb974a0700 1 -- 172.21.15.105:0/2611921529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb90105910 msgr2=0x7ffb90107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:25:29.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.046+0000 7ffb974a0700 1 --2- 172.21.15.105:0/2611921529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb90105910 0x7ffb90107d00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009990 tx=0x7ffb80009ca0 comp rx=0 tx=0).stop 2024-06-10T00:25:29.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.046+0000 7ffb974a0700 1 -- 172.21.15.105:0/2611921529 shutdown_connections 2024-06-10T00:25:29.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.046+0000 7ffb974a0700 1 --2- 172.21.15.105:0/2611921529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb90102fe0 0x7ffb901053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:29.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.046+0000 7ffb974a0700 1 --2- 172.21.15.105:0/2611921529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb90108240 0x7ffb9010a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:29.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.046+0000 7ffb974a0700 1 --2- 172.21.15.105:0/2611921529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb90105910 0x7ffb90107d00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:29.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.046+0000 7ffb974a0700 1 -- 172.21.15.105:0/2611921529 >> 172.21.15.105:0/2611921529 conn(0x7ffb900fc990 msgr2=0x7ffb900fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:25:29.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.046+0000 7ffb974a0700 1 -- 172.21.15.105:0/2611921529 shutdown_connections 2024-06-10T00:25:29.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.046+0000 7ffb974a0700 1 -- 172.21.15.105:0/2611921529 wait complete. 2024-06-10T00:25:29.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.047+0000 7ffb974a0700 1 Processor -- start 2024-06-10T00:25:29.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.047+0000 7ffb974a0700 1 -- start start 2024-06-10T00:25:29.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.047+0000 7ffb974a0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb90102fe0 0x7ffb90102940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:25:29.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.047+0000 7ffb974a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb90105910 0x7ffb90100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:25:29.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.047+0000 7ffb974a0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb90108240 0x7ffb901014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:25:29.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.047+0000 7ffb974a0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9010e140 con 0x7ffb90105910 2024-06-10T00:25:29.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.047+0000 7ffb974a0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9010dfc0 con 0x7ffb90108240 2024-06-10T00:25:29.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.047+0000 7ffb974a0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9010e2c0 con 0x7ffb90102fe0 2024-06-10T00:25:29.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb95a3d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb90108240 0x7ffb901014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:25:29.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb94a3b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb90105910 0x7ffb90100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:25:29.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb95a3d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb90108240 0x7ffb901014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35760/0 (socket says 172.21.15.105:35760) 2024-06-10T00:25:29.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb94a3b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb90105910 0x7ffb90100f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.105:60220/0 (socket says 172.21.15.105:60220) 2024-06-10T00:25:29.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb95a3d700 1 -- 172.21.15.105:0/4202578652 learned_addr learned my addr 172.21.15.105:0/4202578652 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-06-10T00:25:29.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb9523c700 1 --2- 172.21.15.105:0/4202578652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb90102fe0 0x7ffb90102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:25:29.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb95a3d700 1 -- 172.21.15.105:0/4202578652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb90102fe0 msgr2=0x7ffb90102940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:25:29.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb95a3d700 1 --2- 172.21.15.105:0/4202578652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb90102fe0 0x7ffb90102940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:29.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb95a3d700 1 -- 172.21.15.105:0/4202578652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb90105910 msgr2=0x7ffb90100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:25:29.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb95a3d700 1 --2- 172.21.15.105:0/4202578652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb90105910 0x7ffb90100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:29.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb95a3d700 1 -- 172.21.15.105:0/4202578652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb90101d80 con 0x7ffb90108240 2024-06-10T00:25:29.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb95a3d700 1 --2- 172.21.15.105:0/4202578652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb90108240 0x7ffb901014d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800b9e0 tx=0x7ffb8800bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:25:29.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb867fc700 1 -- 172.21.15.105:0/4202578652 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8800d8e0 con 0x7ffb90108240 2024-06-10T00:25:29.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb867fc700 1 -- 172.21.15.105:0/4202578652 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb88015420 con 0x7ffb90108240 2024-06-10T00:25:29.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb974a0700 1 -- 172.21.15.105:0/4202578652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb901a2950 con 0x7ffb90108240 2024-06-10T00:25:29.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb867fc700 1 -- 172.21.15.105:0/4202578652 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88014630 con 0x7ffb90108240 2024-06-10T00:25:29.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.048+0000 7ffb974a0700 1 -- 172.21.15.105:0/4202578652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb901a2ee0 con 0x7ffb90108240 2024-06-10T00:25:29.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.049+0000 7ffb974a0700 1 -- 172.21.15.105:0/4202578652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb90069180 con 0x7ffb90108240 2024-06-10T00:25:29.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.050+0000 7ffb867fc700 1 -- 172.21.15.105:0/4202578652 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffb880155d0 con 0x7ffb90108240 2024-06-10T00:25:29.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.051+0000 7ffb867fc700 1 --2- 172.21.15.105:0/4202578652 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffb7c0a2c20 0x7ffb7c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:25:29.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.051+0000 7ffb867fc700 1 -- 172.21.15.105:0/4202578652 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7ffb880ce060 con 0x7ffb90108240 2024-06-10T00:25:29.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.051+0000 7ffb9523c700 1 --2- 172.21.15.105:0/4202578652 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffb7c0a2c20 0x7ffb7c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:25:29.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.052+0000 7ffb9523c700 1 --2- 172.21.15.105:0/4202578652 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffb7c0a2c20 0x7ffb7c0a50e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c005950 tx=0x7ffb8c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:25:29.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.055+0000 7ffb867fc700 1 -- 172.21.15.105:0/4202578652 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffb8809bcd0 con 0x7ffb90108240 2024-06-10T00:25:29.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:29.257+0000 7ffb974a0700 1 -- 172.21.15.105:0/4202578652 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ffb900628d0 con 0x7ffb7c0a2c20 2024-06-10T00:25:30.002 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:29 smithi105 ceph-mon[36321]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:30.002 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:29 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:25:30.002 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:29 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:25:30.003 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:29 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:30.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:29 smithi062 ceph-mon[32390]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:30.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:29 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:25:30.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:29 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:25:30.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:29 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:30.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:29 smithi138 ceph-mon[36549]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:30.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:29 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:25:30.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:29 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:25:30.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:29 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:31.024 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:30 smithi138 ceph-mon[36549]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:25:31.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:30 smithi105 ceph-mon[36321]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:25:31.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:30 smithi062 ceph-mon[32390]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:25:32.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:31 smithi105 ceph-mon[36321]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:32.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:31 smithi062 ceph-mon[32390]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:32.122 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:31 smithi138 ceph-mon[36549]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:33.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:32 smithi105 ceph-mon[36321]: from='client.? 172.21.15.105:0/2133116928' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59bacdeb-ed9d-4fdb-a1e9-f49732af81df"}]: dispatch 2024-06-10T00:25:33.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:32 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59bacdeb-ed9d-4fdb-a1e9-f49732af81df"}]: dispatch 2024-06-10T00:25:33.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:32 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59bacdeb-ed9d-4fdb-a1e9-f49732af81df"}]': finished 2024-06-10T00:25:33.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:32 smithi105 ceph-mon[36321]: osdmap e48: 8 total, 7 up, 8 in 2024-06-10T00:25:33.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:32 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:33.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:32 smithi105 ceph-mon[36321]: from='client.? 172.21.15.105:0/1848091493' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:25:33.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:32 smithi062 ceph-mon[32390]: from='client.? 172.21.15.105:0/2133116928' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59bacdeb-ed9d-4fdb-a1e9-f49732af81df"}]: dispatch 2024-06-10T00:25:33.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:32 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59bacdeb-ed9d-4fdb-a1e9-f49732af81df"}]: dispatch 2024-06-10T00:25:33.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:32 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59bacdeb-ed9d-4fdb-a1e9-f49732af81df"}]': finished 2024-06-10T00:25:33.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:32 smithi062 ceph-mon[32390]: osdmap e48: 8 total, 7 up, 8 in 2024-06-10T00:25:33.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:32 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:33.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:32 smithi062 ceph-mon[32390]: from='client.? 172.21.15.105:0/1848091493' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:25:33.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:32 smithi138 ceph-mon[36549]: from='client.? 172.21.15.105:0/2133116928' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59bacdeb-ed9d-4fdb-a1e9-f49732af81df"}]: dispatch 2024-06-10T00:25:33.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:32 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59bacdeb-ed9d-4fdb-a1e9-f49732af81df"}]: dispatch 2024-06-10T00:25:33.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:32 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59bacdeb-ed9d-4fdb-a1e9-f49732af81df"}]': finished 2024-06-10T00:25:33.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:32 smithi138 ceph-mon[36549]: osdmap e48: 8 total, 7 up, 8 in 2024-06-10T00:25:33.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:32 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:33.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:32 smithi138 ceph-mon[36549]: from='client.? 172.21.15.105:0/1848091493' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:25:34.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:33 smithi105 ceph-mon[36321]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:34.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:33 smithi062 ceph-mon[32390]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:34.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:33 smithi138 ceph-mon[36549]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:36.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:35 smithi105 ceph-mon[36321]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:36.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:35 smithi062 ceph-mon[32390]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:36.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:35 smithi138 ceph-mon[36549]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:37.979 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:37 smithi105 ceph-mon[36321]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:38.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:37 smithi062 ceph-mon[32390]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:38.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:37 smithi138 ceph-mon[36549]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:40.029 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:39 smithi105 ceph-mon[36321]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:40.029 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:39 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-10T00:25:40.029 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:39 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:40.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:39 smithi062 ceph-mon[32390]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:40.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:39 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-10T00:25:40.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:39 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:39 smithi138 ceph-mon[36549]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:39 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-10T00:25:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:39 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:41.029 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:40 smithi105 ceph-mon[36321]: Deploying daemon osd.7 on smithi105 2024-06-10T00:25:41.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:40 smithi062 ceph-mon[32390]: Deploying daemon osd.7 on smithi105 2024-06-10T00:25:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:40 smithi138 ceph-mon[36549]: Deploying daemon osd.7 on smithi105 2024-06-10T00:25:42.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:41 smithi105 ceph-mon[36321]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:42.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:41 smithi062 ceph-mon[32390]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:42.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:41 smithi138 ceph-mon[36549]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:44.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:43 smithi062 ceph-mon[32390]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:44.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:43 smithi138 ceph-mon[36549]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:44.183 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:43 smithi105 ceph-mon[36321]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:46.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:45 smithi105 ceph-mon[36321]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:46.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:46.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:46.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:46.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:45 smithi062 ceph-mon[32390]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:46.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:46.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:46.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:45 smithi138 ceph-mon[36549]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:47.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:46 smithi105 ceph-mon[36321]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:47.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:46 smithi062 ceph-mon[32390]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:47.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:46 smithi138 ceph-mon[36549]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:49.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:49.024+0000 7ffb867fc700 1 -- 172.21.15.105:0/4202578652 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffb900628d0 con 0x7ffb7c0a2c20 2024-06-10T00:25:49.024 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 7 on host 'smithi105' 2024-06-10T00:25:49.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:49.026+0000 7ffb974a0700 1 -- 172.21.15.105:0/4202578652 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffb7c0a2c20 msgr2=0x7ffb7c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:25:49.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:49.026+0000 7ffb974a0700 1 --2- 172.21.15.105:0/4202578652 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffb7c0a2c20 0x7ffb7c0a50e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c005950 tx=0x7ffb8c00b410 comp rx=0 tx=0).stop 2024-06-10T00:25:49.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:49.026+0000 7ffb974a0700 1 -- 172.21.15.105:0/4202578652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb90108240 msgr2=0x7ffb901014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:25:49.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:49.026+0000 7ffb974a0700 1 --2- 172.21.15.105:0/4202578652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb90108240 0x7ffb901014d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800b9e0 tx=0x7ffb8800bcf0 comp rx=0 tx=0).stop 2024-06-10T00:25:49.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:49.026+0000 7ffb974a0700 1 -- 172.21.15.105:0/4202578652 shutdown_connections 2024-06-10T00:25:49.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:49.026+0000 7ffb974a0700 1 --2- 172.21.15.105:0/4202578652 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffb7c0a2c20 0x7ffb7c0a50e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:49.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:49.026+0000 7ffb974a0700 1 --2- 172.21.15.105:0/4202578652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb90108240 0x7ffb901014d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:49.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:49.026+0000 7ffb974a0700 1 --2- 172.21.15.105:0/4202578652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb90102fe0 0x7ffb90102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:49.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:49.026+0000 7ffb974a0700 1 --2- 172.21.15.105:0/4202578652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb90105910 0x7ffb90100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:49.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:49.026+0000 7ffb974a0700 1 -- 172.21.15.105:0/4202578652 >> 172.21.15.105:0/4202578652 conn(0x7ffb900fc990 msgr2=0x7ffb90103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:25:49.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:49.026+0000 7ffb974a0700 1 -- 172.21.15.105:0/4202578652 shutdown_connections 2024-06-10T00:25:49.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:25:49.026+0000 7ffb974a0700 1 -- 172.21.15.105:0/4202578652 wait complete. 2024-06-10T00:25:49.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:49 smithi105 ceph-mon[36321]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:49.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:49.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:49.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:49 smithi105 ceph-mon[36321]: from='osd.7 [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:25:49.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:49 smithi105 ceph-mon[36321]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:25:49.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:49.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:49 smithi062 ceph-mon[32390]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:49.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:49.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:49.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:49 smithi062 ceph-mon[32390]: from='osd.7 [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:25:49.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:49 smithi062 ceph-mon[32390]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:25:49.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:49.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:49 smithi138 ceph-mon[36549]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:49.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:49.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:49.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:49 smithi138 ceph-mon[36549]: from='osd.7 [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:25:49.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:49 smithi138 ceph-mon[36549]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:25:49.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:49.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:25:49.744 DEBUG:teuthology.orchestra.run.smithi105:osd.7> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.7.service 2024-06-10T00:25:49.747 INFO:tasks.cephadm:Deploying osd.8 on smithi138 with /dev/vg_nvme/lv_2... 2024-06-10T00:25:49.748 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-10T00:25:49.992 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:25:50.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:50 smithi105 ceph-mon[36321]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-10T00:25:50.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:50 smithi105 ceph-mon[36321]: osdmap e49: 8 total, 7 up, 8 in 2024-06-10T00:25:50.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:50 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:50.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:50 smithi105 ceph-mon[36321]: from='osd.7 [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:25:50.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:50 smithi105 ceph-mon[36321]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:25:50.282 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:25:50 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-7[54492]: 2024-06-10T00:25:50.168+0000 7f7b68ff7700 -1 osd.7 0 waiting for initial osdmap 2024-06-10T00:25:50.283 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:25:50 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-7[54492]: 2024-06-10T00:25:50.176+0000 7f7b64dea700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:25:50.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:50 smithi138 ceph-mon[36549]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-10T00:25:50.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:50 smithi138 ceph-mon[36549]: osdmap e49: 8 total, 7 up, 8 in 2024-06-10T00:25:50.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:50 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:50.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:50 smithi138 ceph-mon[36549]: from='osd.7 [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:25:50.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:50 smithi138 ceph-mon[36549]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:25:50.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:50 smithi062 ceph-mon[32390]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-10T00:25:50.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:50 smithi062 ceph-mon[32390]: osdmap e49: 8 total, 7 up, 8 in 2024-06-10T00:25:50.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:50 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:50.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:50 smithi062 ceph-mon[32390]: from='osd.7 [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:25:50.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:50 smithi062 ceph-mon[32390]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:25:51.231 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:51 smithi105 ceph-mon[36321]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:51.231 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:51 smithi105 ceph-mon[36321]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-06-10T00:25:51.231 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:51 smithi105 ceph-mon[36321]: osdmap e50: 8 total, 7 up, 8 in 2024-06-10T00:25:51.231 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:51 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:51.232 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:51 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:51.291 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:51 smithi138 ceph-mon[36549]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:51.291 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:51 smithi138 ceph-mon[36549]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-06-10T00:25:51.291 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:51 smithi138 ceph-mon[36549]: osdmap e50: 8 total, 7 up, 8 in 2024-06-10T00:25:51.291 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:51 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:51.291 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:51 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:51.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:51 smithi062 ceph-mon[32390]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:25:51.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:51 smithi062 ceph-mon[32390]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-06-10T00:25:51.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:51 smithi062 ceph-mon[32390]: osdmap e50: 8 total, 7 up, 8 in 2024-06-10T00:25:51.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:51 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:51.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:51 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:51.824 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-10T00:25:51.824 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-10T00:25:51.824 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-06-10T00:25:51.824 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-06-10T00:25:51.824 INFO:teuthology.orchestra.run.smithi138.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110332 s, 950 MB/s 2024-06-10T00:25:51.824 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-06-10T00:25:52.416 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:52 smithi105 ceph-mon[36321]: purged_snaps scrub starts 2024-06-10T00:25:52.417 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:52 smithi105 ceph-mon[36321]: purged_snaps scrub ok 2024-06-10T00:25:52.417 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:52 smithi105 ceph-mon[36321]: osd.7 [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087] boot 2024-06-10T00:25:52.417 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:52 smithi105 ceph-mon[36321]: osdmap e51: 8 total, 8 up, 8 in 2024-06-10T00:25:52.417 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:52 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:52.435 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:52 smithi138 ceph-mon[36549]: purged_snaps scrub starts 2024-06-10T00:25:52.435 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:52 smithi138 ceph-mon[36549]: purged_snaps scrub ok 2024-06-10T00:25:52.435 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:52 smithi138 ceph-mon[36549]: osd.7 [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087] boot 2024-06-10T00:25:52.435 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:52 smithi138 ceph-mon[36549]: osdmap e51: 8 total, 8 up, 8 in 2024-06-10T00:25:52.436 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:52 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:52.466 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_2 2024-06-10T00:25:52.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:52 smithi062 ceph-mon[32390]: purged_snaps scrub starts 2024-06-10T00:25:52.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:52 smithi062 ceph-mon[32390]: purged_snaps scrub ok 2024-06-10T00:25:52.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:52 smithi062 ceph-mon[32390]: osd.7 [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087] boot 2024-06-10T00:25:52.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:52 smithi062 ceph-mon[32390]: osdmap e51: 8 total, 8 up, 8 in 2024-06-10T00:25:52.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:52 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:25:52.711 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:25:53.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:53 smithi138 ceph-mon[36549]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:25:53.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:53 smithi138 ceph-mon[36549]: osdmap e52: 8 total, 8 up, 8 in 2024-06-10T00:25:53.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:53.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:53.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:53.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:53.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:53.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:53.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:53.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:53.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:53 smithi105 ceph-mon[36321]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:25:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:53 smithi105 ceph-mon[36321]: osdmap e52: 8 total, 8 up, 8 in 2024-06-10T00:25:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:53.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:53.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:53.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:53.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:53 smithi062 ceph-mon[32390]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:25:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:53 smithi062 ceph-mon[32390]: osdmap e52: 8 total, 8 up, 8 in 2024-06-10T00:25:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:25:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:25:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:25:54.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.238+0000 7f6bf6a59700 1 -- 172.21.15.138:0/1119109996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bf01024e0 msgr2=0x7f6bf0102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:25:54.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.238+0000 7f6bf6a59700 1 --2- 172.21.15.138:0/1119109996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bf01024e0 0x7f6bf0102900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009990 tx=0x7f6be0009ca0 comp rx=0 tx=0).stop 2024-06-10T00:25:54.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.238+0000 7f6bf6a59700 1 -- 172.21.15.138:0/1119109996 shutdown_connections 2024-06-10T00:25:54.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.238+0000 7f6bf6a59700 1 --2- 172.21.15.138:0/1119109996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bf0104080 0x7f6bf010a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:54.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.238+0000 7f6bf6a59700 1 --2- 172.21.15.138:0/1119109996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bf01024e0 0x7f6bf0102900 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:54.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.238+0000 7f6bf6a59700 1 --2- 172.21.15.138:0/1119109996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf01036e0 0x7f6bf0103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.238+0000 7f6bf6a59700 1 -- 172.21.15.138:0/1119109996 >> 172.21.15.138:0/1119109996 conn(0x7f6bf00fda80 msgr2=0x7f6bf00ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:25:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.238+0000 7f6bf6a59700 1 -- 172.21.15.138:0/1119109996 shutdown_connections 2024-06-10T00:25:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.238+0000 7f6bf6a59700 1 -- 172.21.15.138:0/1119109996 wait complete. 2024-06-10T00:25:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.238+0000 7f6bf6a59700 1 Processor -- start 2024-06-10T00:25:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.238+0000 7f6bf6a59700 1 -- start start 2024-06-10T00:25:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf6a59700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bf01024e0 0x7f6bf01986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:25:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf6a59700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf01036e0 0x7f6bf0198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:25:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf6a59700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bf0104080 0x7f6bf019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:25:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf6a59700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf010e0c0 con 0x7f6bf01036e0 2024-06-10T00:25:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf6a59700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf010df40 con 0x7f6bf01024e0 2024-06-10T00:25:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf6a59700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf010e240 con 0x7f6bf0104080 2024-06-10T00:25:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf4ff6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bf0104080 0x7f6bf019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:25:54.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf4ff6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bf0104080 0x7f6bf019fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35454/0 (socket says 172.21.15.138:35454) 2024-06-10T00:25:54.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf4ff6700 1 -- 172.21.15.138:0/2741033440 learned_addr learned my addr 172.21.15.138:0/2741033440 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-06-10T00:25:54.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6beffff700 1 --2- 172.21.15.138:0/2741033440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bf01024e0 0x7f6bf01986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:25:54.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bef7fe700 1 --2- 172.21.15.138:0/2741033440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf01036e0 0x7f6bf0198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:25:54.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf4ff6700 1 -- 172.21.15.138:0/2741033440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bf01024e0 msgr2=0x7f6bf01986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:25:54.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf4ff6700 1 --2- 172.21.15.138:0/2741033440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bf01024e0 0x7f6bf01986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:54.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf4ff6700 1 -- 172.21.15.138:0/2741033440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf01036e0 msgr2=0x7f6bf0198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:25:54.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf4ff6700 1 --2- 172.21.15.138:0/2741033440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf01036e0 0x7f6bf0198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:25:54.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf4ff6700 1 -- 172.21.15.138:0/2741033440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf01a0300 con 0x7f6bf0104080 2024-06-10T00:25:54.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bf4ff6700 1 --2- 172.21.15.138:0/2741033440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bf0104080 0x7f6bf019fc60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6be8007d40 tx=0x7f6be800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:25:54.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.239+0000 7f6bed7fa700 1 -- 172.21.15.138:0/2741033440 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8016020 con 0x7f6bf0104080 2024-06-10T00:25:54.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.240+0000 7f6bf6a59700 1 -- 172.21.15.138:0/2741033440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf01a05f0 con 0x7f6bf0104080 2024-06-10T00:25:54.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.240+0000 7f6bed7fa700 1 -- 172.21.15.138:0/2741033440 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be800acb0 con 0x7f6bf0104080 2024-06-10T00:25:54.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.240+0000 7f6bed7fa700 1 -- 172.21.15.138:0/2741033440 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be80115e0 con 0x7f6bf0104080 2024-06-10T00:25:54.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.240+0000 7f6bf6a59700 1 -- 172.21.15.138:0/2741033440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf01a0b30 con 0x7f6bf0104080 2024-06-10T00:25:54.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.241+0000 7f6bf6a59700 1 -- 172.21.15.138:0/2741033440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf0069180 con 0x7f6bf0104080 2024-06-10T00:25:54.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.242+0000 7f6bed7fa700 1 -- 172.21.15.138:0/2741033440 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6be8019020 con 0x7f6bf0104080 2024-06-10T00:25:54.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.242+0000 7f6bed7fa700 1 --2- 172.21.15.138:0/2741033440 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6bd80a2c20 0x7f6bd80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:25:54.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.242+0000 7f6bed7fa700 1 -- 172.21.15.138:0/2741033440 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7f6be80ce410 con 0x7f6bf0104080 2024-06-10T00:25:54.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.242+0000 7f6beffff700 1 --2- 172.21.15.138:0/2741033440 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6bd80a2c20 0x7f6bd80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:25:54.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.243+0000 7f6beffff700 1 --2- 172.21.15.138:0/2741033440 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6bd80a2c20 0x7f6bd80a50e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6be0003890 tx=0x7f6be0003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:25:54.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.247+0000 7f6bed7fa700 1 -- 172.21.15.138:0/2741033440 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6be8020020 con 0x7f6bf0104080 2024-06-10T00:25:54.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:54 smithi138 ceph-mon[36549]: Detected new or changed devices on smithi105 2024-06-10T00:25:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:54 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi105 to 5696M 2024-06-10T00:25:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:54 smithi138 ceph-mon[36549]: osdmap e53: 8 total, 8 up, 8 in 2024-06-10T00:25:54.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:25:54.453+0000 7f6bf6a59700 1 -- 172.21.15.138:0/2741033440 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6bf00628d0 con 0x7f6bd80a2c20 2024-06-10T00:25:54.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:54 smithi105 ceph-mon[36321]: Detected new or changed devices on smithi105 2024-06-10T00:25:54.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:54 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi105 to 5696M 2024-06-10T00:25:54.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:54 smithi105 ceph-mon[36321]: osdmap e53: 8 total, 8 up, 8 in 2024-06-10T00:25:54.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:54 smithi062 ceph-mon[32390]: Detected new or changed devices on smithi105 2024-06-10T00:25:54.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:54 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi105 to 5696M 2024-06-10T00:25:54.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:54 smithi062 ceph-mon[32390]: osdmap e53: 8 total, 8 up, 8 in 2024-06-10T00:25:55.344 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:55 smithi138 ceph-mon[36549]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:25:55.344 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:55 smithi138 ceph-mon[36549]: osdmap e54: 8 total, 8 up, 8 in 2024-06-10T00:25:55.345 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:55 smithi138 ceph-mon[36549]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:25:55.345 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:25:55.345 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:25:55.345 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:55.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:55 smithi105 ceph-mon[36321]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:25:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:55 smithi105 ceph-mon[36321]: osdmap e54: 8 total, 8 up, 8 in 2024-06-10T00:25:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:55 smithi105 ceph-mon[36321]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:25:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:25:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:25:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:55.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:55 smithi062 ceph-mon[32390]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:25:55.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:55 smithi062 ceph-mon[32390]: osdmap e54: 8 total, 8 up, 8 in 2024-06-10T00:25:55.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:55 smithi062 ceph-mon[32390]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:25:55.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:25:55.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:25:55.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:25:56.298 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:56 smithi138 ceph-mon[36549]: osdmap e55: 8 total, 8 up, 8 in 2024-06-10T00:25:56.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:56 smithi105 ceph-mon[36321]: osdmap e55: 8 total, 8 up, 8 in 2024-06-10T00:25:56.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:56 smithi062 ceph-mon[32390]: osdmap e55: 8 total, 8 up, 8 in 2024-06-10T00:25:57.467 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:57 smithi138 ceph-mon[36549]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:25:57.467 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:57 smithi138 ceph-mon[36549]: from='client.? 172.21.15.138:0/25616278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5f449fd-35c6-4885-827a-ae7f5b104404"}]: dispatch 2024-06-10T00:25:57.467 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:57 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5f449fd-35c6-4885-827a-ae7f5b104404"}]: dispatch 2024-06-10T00:25:57.467 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:57 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5f449fd-35c6-4885-827a-ae7f5b104404"}]': finished 2024-06-10T00:25:57.467 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:57 smithi138 ceph-mon[36549]: osdmap e56: 9 total, 8 up, 9 in 2024-06-10T00:25:57.467 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:57 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:25:57.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:57 smithi105 ceph-mon[36321]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:25:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:57 smithi105 ceph-mon[36321]: from='client.? 172.21.15.138:0/25616278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5f449fd-35c6-4885-827a-ae7f5b104404"}]: dispatch 2024-06-10T00:25:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:57 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5f449fd-35c6-4885-827a-ae7f5b104404"}]: dispatch 2024-06-10T00:25:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:57 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5f449fd-35c6-4885-827a-ae7f5b104404"}]': finished 2024-06-10T00:25:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:57 smithi105 ceph-mon[36321]: osdmap e56: 9 total, 8 up, 9 in 2024-06-10T00:25:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:57 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:25:57.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:57 smithi062 ceph-mon[32390]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:25:57.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:57 smithi062 ceph-mon[32390]: from='client.? 172.21.15.138:0/25616278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5f449fd-35c6-4885-827a-ae7f5b104404"}]: dispatch 2024-06-10T00:25:57.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:57 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5f449fd-35c6-4885-827a-ae7f5b104404"}]: dispatch 2024-06-10T00:25:57.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:57 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5f449fd-35c6-4885-827a-ae7f5b104404"}]': finished 2024-06-10T00:25:57.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:57 smithi062 ceph-mon[32390]: osdmap e56: 9 total, 8 up, 9 in 2024-06-10T00:25:57.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:57 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:25:58.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:58 smithi105 ceph-mon[36321]: from='client.? 172.21.15.138:0/3454148056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:25:58.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:58 smithi062 ceph-mon[32390]: from='client.? 172.21.15.138:0/3454148056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:25:58.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:58 smithi138 ceph-mon[36549]: from='client.? 172.21.15.138:0/3454148056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:25:59.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:25:59 smithi105 ceph-mon[36321]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:25:59.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:25:59 smithi062 ceph-mon[32390]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:25:59.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:25:59 smithi138 ceph-mon[36549]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:01.367 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:01 smithi138 ceph-mon[36549]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:01.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:01 smithi105 ceph-mon[36321]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:01.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:01 smithi062 ceph-mon[32390]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:03.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:03 smithi138 ceph-mon[36549]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:03.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:03 smithi105 ceph-mon[36321]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:03.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:03 smithi062 ceph-mon[32390]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:04.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:04 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-10T00:26:04.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:04 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:04.529 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:04 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-10T00:26:04.529 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:04 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:04.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:04 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-10T00:26:04.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:04 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:05.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:05 smithi138 ceph-mon[36549]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:05.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:05 smithi138 ceph-mon[36549]: Deploying daemon osd.8 on smithi138 2024-06-10T00:26:05.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:05 smithi105 ceph-mon[36321]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:05 smithi105 ceph-mon[36321]: Deploying daemon osd.8 on smithi138 2024-06-10T00:26:05.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:05 smithi062 ceph-mon[32390]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:05.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:05 smithi062 ceph-mon[32390]: Deploying daemon osd.8 on smithi138 2024-06-10T00:26:07.345 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:07 smithi138 ceph-mon[36549]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:07.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:07 smithi105 ceph-mon[36321]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:07.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:07 smithi062 ceph-mon[32390]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:09.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:09 smithi105 ceph-mon[36321]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:09.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:09 smithi062 ceph-mon[32390]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:09.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:09 smithi138 ceph-mon[36549]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:10.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:10.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:10.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:10 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:26:10.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:10 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:10.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:10 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:10.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:10 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:26:10.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:10.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:10.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:10 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:26:11.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:11 smithi105 ceph-mon[36321]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:11.785 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:11 smithi138 ceph-mon[36549]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:11.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:11 smithi062 ceph-mon[32390]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:13.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:26:13.169+0000 7f6bed7fa700 1 -- 172.21.15.138:0/2741033440 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6bf00628d0 con 0x7f6bd80a2c20 2024-06-10T00:26:13.169 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 8 on host 'smithi138' 2024-06-10T00:26:13.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:26:13.171+0000 7f6bf6a59700 1 -- 172.21.15.138:0/2741033440 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6bd80a2c20 msgr2=0x7f6bd80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:26:13.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:26:13.171+0000 7f6bf6a59700 1 --2- 172.21.15.138:0/2741033440 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6bd80a2c20 0x7f6bd80a50e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6be0003890 tx=0x7f6be0003960 comp rx=0 tx=0).stop 2024-06-10T00:26:13.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:26:13.171+0000 7f6bf6a59700 1 -- 172.21.15.138:0/2741033440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bf0104080 msgr2=0x7f6bf019fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:26:13.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:26:13.171+0000 7f6bf6a59700 1 --2- 172.21.15.138:0/2741033440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bf0104080 0x7f6bf019fc60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6be8007d40 tx=0x7f6be800a340 comp rx=0 tx=0).stop 2024-06-10T00:26:13.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:26:13.171+0000 7f6bf6a59700 1 -- 172.21.15.138:0/2741033440 shutdown_connections 2024-06-10T00:26:13.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:26:13.171+0000 7f6bf6a59700 1 --2- 172.21.15.138:0/2741033440 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6bd80a2c20 0x7f6bd80a50e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:13.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:26:13.171+0000 7f6bf6a59700 1 --2- 172.21.15.138:0/2741033440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bf0104080 0x7f6bf019fc60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:13.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:26:13.171+0000 7f6bf6a59700 1 --2- 172.21.15.138:0/2741033440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bf01024e0 0x7f6bf01986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:13.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:26:13.171+0000 7f6bf6a59700 1 --2- 172.21.15.138:0/2741033440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf01036e0 0x7f6bf0198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:13.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:26:13.171+0000 7f6bf6a59700 1 -- 172.21.15.138:0/2741033440 >> 172.21.15.138:0/2741033440 conn(0x7f6bf00fda80 msgr2=0x7f6bf01069e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:26:13.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:26:13.171+0000 7f6bf6a59700 1 -- 172.21.15.138:0/2741033440 shutdown_connections 2024-06-10T00:26:13.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:26:13.171+0000 7f6bf6a59700 1 -- 172.21.15.138:0/2741033440 wait complete. 2024-06-10T00:26:13.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:13 smithi062 ceph-mon[32390]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:13.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:13 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:13 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:13 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:13.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:13 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:26:13.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:13 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:13 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:13 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:26:13.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:13 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:13 smithi138 ceph-mon[36549]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:13 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:13 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:13 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:13 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:26:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:13 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:13 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:13 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:26:13.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:13 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:13 smithi105 ceph-mon[36321]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:13.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:13 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:13 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:13 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:13.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:13 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:26:13.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:13 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:13 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:13 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:26:13.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:13 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:13.853 DEBUG:teuthology.orchestra.run.smithi138:osd.8> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.8.service 2024-06-10T00:26:13.855 INFO:tasks.cephadm:Deploying osd.9 on smithi062 with /dev/vg_nvme/lv_1... 2024-06-10T00:26:13.856 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-10T00:26:14.116 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:26:14.568 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:14 smithi062 ceph-mon[32390]: from='osd.8 [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:26:14.568 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:14 smithi062 ceph-mon[32390]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:26:14.599 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:14 smithi105 ceph-mon[36321]: from='osd.8 [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:26:14.599 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:14 smithi105 ceph-mon[36321]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:26:14.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:14 smithi138 ceph-mon[36549]: from='osd.8 [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:26:14.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:14 smithi138 ceph-mon[36549]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:26:15.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:15 smithi062 ceph-mon[32390]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:15.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:15 smithi062 ceph-mon[32390]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-10T00:26:15.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:15 smithi062 ceph-mon[32390]: from='osd.8 [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:26:15.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:15 smithi062 ceph-mon[32390]: osdmap e57: 9 total, 8 up, 9 in 2024-06-10T00:26:15.582 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:15 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:15.582 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:15 smithi062 ceph-mon[32390]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:26:15.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:15 smithi138 ceph-mon[36549]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:15.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:15 smithi138 ceph-mon[36549]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-10T00:26:15.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:15 smithi138 ceph-mon[36549]: from='osd.8 [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:26:15.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:15 smithi138 ceph-mon[36549]: osdmap e57: 9 total, 8 up, 9 in 2024-06-10T00:26:15.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:15 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:15.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:15 smithi138 ceph-mon[36549]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:26:15.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:15 smithi105 ceph-mon[36321]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:15.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:15 smithi105 ceph-mon[36321]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-10T00:26:15.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:15 smithi105 ceph-mon[36321]: from='osd.8 [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:26:15.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:15 smithi105 ceph-mon[36321]: osdmap e57: 9 total, 8 up, 9 in 2024-06-10T00:26:15.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:15 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:15.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:15 smithi105 ceph-mon[36321]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:26:16.167 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-10T00:26:16.167 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-10T00:26:16.167 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-06-10T00:26:16.167 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-06-10T00:26:16.167 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113166 s, 927 MB/s 2024-06-10T00:26:16.167 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-06-10T00:26:16.363 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:16 smithi138 ceph-mon[36549]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-06-10T00:26:16.363 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:16 smithi138 ceph-mon[36549]: osdmap e58: 9 total, 8 up, 9 in 2024-06-10T00:26:16.363 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:16 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:16.363 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:16 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:16.363 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:16 smithi138 ceph-mon[36549]: from='osd.8 ' entity='osd.8' 2024-06-10T00:26:16.364 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:26:16 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-8[54827]: 2024-06-10T00:26:16.075+0000 7ff0f9fd7700 -1 osd.8 0 waiting for initial osdmap 2024-06-10T00:26:16.364 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:26:16 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-8[54827]: 2024-06-10T00:26:16.085+0000 7ff0f15c1700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:26:16.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:16 smithi062 ceph-mon[32390]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-06-10T00:26:16.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:16 smithi062 ceph-mon[32390]: osdmap e58: 9 total, 8 up, 9 in 2024-06-10T00:26:16.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:16 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:16.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:16 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:16.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:16 smithi062 ceph-mon[32390]: from='osd.8 ' entity='osd.8' 2024-06-10T00:26:16.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:16 smithi105 ceph-mon[36321]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-06-10T00:26:16.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:16 smithi105 ceph-mon[36321]: osdmap e58: 9 total, 8 up, 9 in 2024-06-10T00:26:16.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:16 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:16.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:16 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:16.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:16 smithi105 ceph-mon[36321]: from='osd.8 ' entity='osd.8' 2024-06-10T00:26:16.927 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_1 2024-06-10T00:26:17.172 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:26:17.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: purged_snaps scrub starts 2024-06-10T00:26:17.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: purged_snaps scrub ok 2024-06-10T00:26:17.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:17.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:17.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: osd.8 [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531] boot 2024-06-10T00:26:17.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: osdmap e59: 9 total, 9 up, 9 in 2024-06-10T00:26:17.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:17.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:17.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:17.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:17.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:17.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:17.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:17.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:17.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:26:17.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:17 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:17.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: purged_snaps scrub starts 2024-06-10T00:26:17.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: purged_snaps scrub ok 2024-06-10T00:26:17.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:17.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:17.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: osd.8 [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531] boot 2024-06-10T00:26:17.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: osdmap e59: 9 total, 9 up, 9 in 2024-06-10T00:26:17.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:17.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:17.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:17.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:17.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:17.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:17.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:17.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:17.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:26:17.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:17 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: purged_snaps scrub starts 2024-06-10T00:26:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: purged_snaps scrub ok 2024-06-10T00:26:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:26:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: osd.8 [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531] boot 2024-06-10T00:26:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: osdmap e59: 9 total, 9 up, 9 in 2024-06-10T00:26:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:26:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:17.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:17.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:17.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:17.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:17.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:17.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:17.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:26:17.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:17 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:18.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:18 smithi062 ceph-mon[32390]: Detected new or changed devices on smithi138 2024-06-10T00:26:18.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:18 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi138 to 5696M 2024-06-10T00:26:18.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:18 smithi062 ceph-mon[32390]: osdmap e60: 9 total, 9 up, 9 in 2024-06-10T00:26:18.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:18 smithi105 ceph-mon[36321]: Detected new or changed devices on smithi138 2024-06-10T00:26:18.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:18 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi138 to 5696M 2024-06-10T00:26:18.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:18 smithi105 ceph-mon[36321]: osdmap e60: 9 total, 9 up, 9 in 2024-06-10T00:26:18.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:18 smithi138 ceph-mon[36549]: Detected new or changed devices on smithi138 2024-06-10T00:26:18.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:18 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi138 to 5696M 2024-06-10T00:26:18.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:18 smithi138 ceph-mon[36549]: osdmap e60: 9 total, 9 up, 9 in 2024-06-10T00:26:18.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.896+0000 7f88715a1700 1 -- 172.21.15.62:0/4066724620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c102f90 msgr2=0x7f886c109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:26:18.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.896+0000 7f88715a1700 1 --2- 172.21.15.62:0/4066724620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c102f90 0x7f886c109840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f886400b0d0 tx=0x7f886400b3e0 comp rx=0 tx=0).stop 2024-06-10T00:26:18.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.896+0000 7f88715a1700 1 -- 172.21.15.62:0/4066724620 shutdown_connections 2024-06-10T00:26:18.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.896+0000 7f88715a1700 1 --2- 172.21.15.62:0/4066724620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c102f90 0x7f886c109840 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:18.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.896+0000 7f88715a1700 1 --2- 172.21.15.62:0/4066724620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f886c1013f0 0x7f886c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:18.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.896+0000 7f88715a1700 1 --2- 172.21.15.62:0/4066724620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f886c1025f0 0x7f886c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:18.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.897+0000 7f88715a1700 1 -- 172.21.15.62:0/4066724620 >> 172.21.15.62:0/4066724620 conn(0x7f886c0fc990 msgr2=0x7f886c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:26:18.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.897+0000 7f88715a1700 1 -- 172.21.15.62:0/4066724620 shutdown_connections 2024-06-10T00:26:18.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.897+0000 7f88715a1700 1 -- 172.21.15.62:0/4066724620 wait complete. 2024-06-10T00:26:18.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.897+0000 7f88715a1700 1 Processor -- start 2024-06-10T00:26:18.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.897+0000 7f88715a1700 1 -- start start 2024-06-10T00:26:18.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.897+0000 7f88715a1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f886c1013f0 0x7f886c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:26:18.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.897+0000 7f88715a1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c1025f0 0x7f886c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:26:18.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f88715a1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f886c102f90 0x7f886c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:26:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f88715a1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f886c10cfb0 con 0x7f886c1025f0 2024-06-10T00:26:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f88715a1700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f886c10ce30 con 0x7f886c1013f0 2024-06-10T00:26:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f88715a1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f886c10d130 con 0x7f886c102f90 2024-06-10T00:26:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f886a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c1025f0 0x7f886c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:26:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f886a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c1025f0 0x7f886c198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32946/0 (socket says 172.21.15.62:32946) 2024-06-10T00:26:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f886a7fc700 1 -- 172.21.15.62:0/3235738388 learned_addr learned my addr 172.21.15.62:0/3235738388 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:26:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f886b7fe700 1 --2- 172.21.15.62:0/3235738388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f886c102f90 0x7f886c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:26:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f886affd700 1 --2- 172.21.15.62:0/3235738388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f886c1013f0 0x7f886c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:26:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f886a7fc700 1 -- 172.21.15.62:0/3235738388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f886c102f90 msgr2=0x7f886c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:26:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f886a7fc700 1 --2- 172.21.15.62:0/3235738388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f886c102f90 0x7f886c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f886a7fc700 1 -- 172.21.15.62:0/3235738388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f886c1013f0 msgr2=0x7f886c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:26:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f886a7fc700 1 --2- 172.21.15.62:0/3235738388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f886c1013f0 0x7f886c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:18.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f886a7fc700 1 -- 172.21.15.62:0/3235738388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f886c103810 con 0x7f886c1025f0 2024-06-10T00:26:18.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f886a7fc700 1 --2- 172.21.15.62:0/3235738388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c1025f0 0x7f886c198c00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f886000d980 tx=0x7f886000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:26:18.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f8853fff700 1 -- 172.21.15.62:0/3235738388 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860016070 con 0x7f886c1025f0 2024-06-10T00:26:18.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f88715a1700 1 -- 172.21.15.62:0/3235738388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f886c103b00 con 0x7f886c1025f0 2024-06-10T00:26:18.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f8853fff700 1 -- 172.21.15.62:0/3235738388 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8860009db0 con 0x7f886c1025f0 2024-06-10T00:26:18.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.898+0000 7f8853fff700 1 -- 172.21.15.62:0/3235738388 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886000f3c0 con 0x7f886c1025f0 2024-06-10T00:26:18.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.899+0000 7f88715a1700 1 -- 172.21.15.62:0/3235738388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f886c104040 con 0x7f886c1025f0 2024-06-10T00:26:18.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.899+0000 7f88715a1700 1 -- 172.21.15.62:0/3235738388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f886c069180 con 0x7f886c1025f0 2024-06-10T00:26:18.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.901+0000 7f8853fff700 1 -- 172.21.15.62:0/3235738388 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8860019070 con 0x7f886c1025f0 2024-06-10T00:26:18.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.901+0000 7f8853fff700 1 --2- 172.21.15.62:0/3235738388 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f88540a2c20 0x7f88540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:26:18.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.901+0000 7f886affd700 1 --2- 172.21.15.62:0/3235738388 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f88540a2c20 0x7f88540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:26:18.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.901+0000 7f8853fff700 1 -- 172.21.15.62:0/3235738388 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5858+0+0 (secure 0 0 0) 0x7f88600ce130 con 0x7f886c1025f0 2024-06-10T00:26:18.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.902+0000 7f886affd700 1 --2- 172.21.15.62:0/3235738388 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f88540a2c20 0x7f88540a50e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f886c102450 tx=0x7f885c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:26:18.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:18.905+0000 7f8853fff700 1 -- 172.21.15.62:0/3235738388 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f886009ba20 con 0x7f886c1025f0 2024-06-10T00:26:19.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:19.121+0000 7f88715a1700 1 -- 172.21.15.62:0/3235738388 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f886c0feb20 con 0x7f88540a2c20 2024-06-10T00:26:19.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:19 smithi062 ceph-mon[32390]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:19.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:19 smithi062 ceph-mon[32390]: osdmap e61: 9 total, 9 up, 9 in 2024-06-10T00:26:19.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:26:19.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:26:19.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:19.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:19 smithi105 ceph-mon[36321]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:19.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:19 smithi105 ceph-mon[36321]: osdmap e61: 9 total, 9 up, 9 in 2024-06-10T00:26:19.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:26:19.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:26:19.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:19 smithi138 ceph-mon[36549]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:19 smithi138 ceph-mon[36549]: osdmap e61: 9 total, 9 up, 9 in 2024-06-10T00:26:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:26:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:26:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:20.692 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:20 smithi062 ceph-mon[32390]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:26:20.693 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:20 smithi062 ceph-mon[32390]: osdmap e62: 9 total, 9 up, 9 in 2024-06-10T00:26:20.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:20 smithi105 ceph-mon[36321]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:26:20.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:20 smithi105 ceph-mon[36321]: osdmap e62: 9 total, 9 up, 9 in 2024-06-10T00:26:20.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:20 smithi138 ceph-mon[36549]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:26:20.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:20 smithi138 ceph-mon[36549]: osdmap e62: 9 total, 9 up, 9 in 2024-06-10T00:26:21.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:21 smithi105 ceph-mon[36321]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:21.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:21 smithi105 ceph-mon[36321]: osdmap e63: 9 total, 9 up, 9 in 2024-06-10T00:26:21.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:21 smithi062 ceph-mon[32390]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:21.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:21 smithi062 ceph-mon[32390]: osdmap e63: 9 total, 9 up, 9 in 2024-06-10T00:26:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:21 smithi138 ceph-mon[36549]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:21 smithi138 ceph-mon[36549]: osdmap e63: 9 total, 9 up, 9 in 2024-06-10T00:26:22.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:22 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2934288427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f02b059a-ffcd-4dd2-b8a1-5782cc47a378"}]: dispatch 2024-06-10T00:26:22.610 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:22 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2934288427' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f02b059a-ffcd-4dd2-b8a1-5782cc47a378"}]': finished 2024-06-10T00:26:22.610 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:22 smithi062 ceph-mon[32390]: osdmap e64: 10 total, 9 up, 10 in 2024-06-10T00:26:22.610 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:22.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:22 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2934288427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f02b059a-ffcd-4dd2-b8a1-5782cc47a378"}]: dispatch 2024-06-10T00:26:22.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:22 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2934288427' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f02b059a-ffcd-4dd2-b8a1-5782cc47a378"}]': finished 2024-06-10T00:26:22.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:22 smithi105 ceph-mon[36321]: osdmap e64: 10 total, 9 up, 10 in 2024-06-10T00:26:22.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:22.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:22 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2934288427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f02b059a-ffcd-4dd2-b8a1-5782cc47a378"}]: dispatch 2024-06-10T00:26:22.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:22 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2934288427' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f02b059a-ffcd-4dd2-b8a1-5782cc47a378"}]': finished 2024-06-10T00:26:22.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:22 smithi138 ceph-mon[36549]: osdmap e64: 10 total, 9 up, 10 in 2024-06-10T00:26:22.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:23.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:23 smithi105 ceph-mon[36321]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-10T00:26:23.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:23 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/983418039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:26:23.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:23 smithi062 ceph-mon[32390]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-10T00:26:23.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:23 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/983418039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:26:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:23 smithi138 ceph-mon[36549]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-10T00:26:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:23 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/983418039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:26:25.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:25 smithi062 ceph-mon[32390]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-06-10T00:26:25.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:25 smithi105 ceph-mon[36321]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-06-10T00:26:25.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:25 smithi138 ceph-mon[36549]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-06-10T00:26:27.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:27 smithi062 ceph-mon[32390]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-06-10T00:26:27.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:27 smithi105 ceph-mon[36321]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-06-10T00:26:27.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:27 smithi138 ceph-mon[36549]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-06-10T00:26:29.503 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:29 smithi062 ceph-mon[32390]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:26:29.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:29 smithi105 ceph-mon[36321]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:26:29.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:29 smithi138 ceph-mon[36549]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:26:30.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:30 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-10T00:26:30.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:30 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:30.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:30 smithi062 ceph-mon[32390]: Deploying daemon osd.9 on smithi062 2024-06-10T00:26:30.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:30 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-10T00:26:30.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:30 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:30.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:30 smithi105 ceph-mon[36321]: Deploying daemon osd.9 on smithi062 2024-06-10T00:26:30.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:30 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-10T00:26:30.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:30 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:30.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:30 smithi138 ceph-mon[36549]: Deploying daemon osd.9 on smithi062 2024-06-10T00:26:31.723 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:31 smithi062 ceph-mon[32390]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-06-10T00:26:31.774 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:31 smithi138 ceph-mon[36549]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-06-10T00:26:31.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:31 smithi105 ceph-mon[36321]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-06-10T00:26:33.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:33 smithi062 ceph-mon[32390]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:33.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:33 smithi105 ceph-mon[36321]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:33.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:33 smithi138 ceph-mon[36549]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:35.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:35 smithi062 ceph-mon[32390]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:35.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:35.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:26:35.661 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:35.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:35 smithi105 ceph-mon[36321]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:35.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:35.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:26:35.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:35 smithi138 ceph-mon[36549]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:26:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:37.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:37 smithi062 ceph-mon[32390]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:37.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:37 smithi105 ceph-mon[36321]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:37.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:37 smithi138 ceph-mon[36549]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:38.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:38.948+0000 7f8853fff700 1 -- 172.21.15.62:0/3235738388 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f886c0feb20 con 0x7f88540a2c20 2024-06-10T00:26:38.949 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 9 on host 'smithi062' 2024-06-10T00:26:38.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:38.950+0000 7f88715a1700 1 -- 172.21.15.62:0/3235738388 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f88540a2c20 msgr2=0x7f88540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:26:38.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:38.950+0000 7f88715a1700 1 --2- 172.21.15.62:0/3235738388 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f88540a2c20 0x7f88540a50e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f886c102450 tx=0x7f885c00b410 comp rx=0 tx=0).stop 2024-06-10T00:26:38.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:38.950+0000 7f88715a1700 1 -- 172.21.15.62:0/3235738388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c1025f0 msgr2=0x7f886c198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:26:38.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:38.950+0000 7f88715a1700 1 --2- 172.21.15.62:0/3235738388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c1025f0 0x7f886c198c00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f886000d980 tx=0x7f886000dc90 comp rx=0 tx=0).stop 2024-06-10T00:26:38.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:38.950+0000 7f88715a1700 1 -- 172.21.15.62:0/3235738388 shutdown_connections 2024-06-10T00:26:38.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:38.950+0000 7f88715a1700 1 --2- 172.21.15.62:0/3235738388 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f88540a2c20 0x7f88540a50e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:38.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:38.950+0000 7f88715a1700 1 --2- 172.21.15.62:0/3235738388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f886c102f90 0x7f886c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:38.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:38.950+0000 7f88715a1700 1 --2- 172.21.15.62:0/3235738388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f886c1013f0 0x7f886c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:38.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:38.950+0000 7f88715a1700 1 --2- 172.21.15.62:0/3235738388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c1025f0 0x7f886c198c00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:38.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:38.951+0000 7f88715a1700 1 -- 172.21.15.62:0/3235738388 >> 172.21.15.62:0/3235738388 conn(0x7f886c0fc990 msgr2=0x7f886c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:26:38.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:38.951+0000 7f88715a1700 1 -- 172.21.15.62:0/3235738388 shutdown_connections 2024-06-10T00:26:38.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:26:38.951+0000 7f88715a1700 1 -- 172.21.15.62:0/3235738388 wait complete. 2024-06-10T00:26:39.655 DEBUG:teuthology.orchestra.run.smithi062:osd.9> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.9.service 2024-06-10T00:26:39.657 INFO:tasks.cephadm:Deploying osd.10 on smithi105 with /dev/vg_nvme/lv_1... 2024-06-10T00:26:39.657 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-10T00:26:39.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:39 smithi062 ceph-mon[32390]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:39.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:39 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:39.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:39 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:39.680 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:39 smithi062 ceph-mon[32390]: from='osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-10T00:26:39.683 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:39 smithi105 ceph-mon[36321]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:39.684 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:39 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:39.684 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:39 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:39.684 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:39 smithi105 ceph-mon[36321]: from='osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-10T00:26:39.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:39 smithi138 ceph-mon[36549]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:39.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:39 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:39.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:39 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:39.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:39 smithi138 ceph-mon[36549]: from='osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-10T00:26:39.900 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:26:41.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:40 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:40 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:40 smithi105 ceph-mon[36321]: from='osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-10T00:26:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:40 smithi105 ceph-mon[36321]: osdmap e65: 10 total, 9 up, 10 in 2024-06-10T00:26:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:40 smithi105 ceph-mon[36321]: from='osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-06-10T00:26:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:40 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:40 smithi105 ceph-mon[36321]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:41.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:40 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:41.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:40 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:41.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:40 smithi062 ceph-mon[32390]: from='osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-10T00:26:41.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:40 smithi062 ceph-mon[32390]: osdmap e65: 10 total, 9 up, 10 in 2024-06-10T00:26:41.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:40 smithi062 ceph-mon[32390]: from='osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-06-10T00:26:41.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:40 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:41.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:40 smithi062 ceph-mon[32390]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:41.108 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:26:40 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:26:40.910+0000 7ff093f07700 -1 osd.9 0 waiting for initial osdmap 2024-06-10T00:26:41.108 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:26:40 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:26:40.923+0000 7ff08fcfa700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:26:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:40 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:40 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:40 smithi138 ceph-mon[36549]: from='osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-10T00:26:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:40 smithi138 ceph-mon[36549]: osdmap e65: 10 total, 9 up, 10 in 2024-06-10T00:26:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:40 smithi138 ceph-mon[36549]: from='osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-06-10T00:26:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:40 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:40 smithi138 ceph-mon[36549]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-10T00:26:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-10T00:26:42.007 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-06-10T00:26:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-06-10T00:26:42.007 INFO:teuthology.orchestra.run.smithi105.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115062 s, 911 MB/s 2024-06-10T00:26:42.007 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 2024-06-10T00:26:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-06-10T00:26:42.053 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:41 smithi105 ceph-mon[36321]: from='osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-06-10T00:26:42.054 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:41 smithi105 ceph-mon[36321]: osdmap e66: 10 total, 9 up, 10 in 2024-06-10T00:26:42.054 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:41 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:42.054 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:41 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:42.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:41 smithi062 ceph-mon[32390]: from='osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-06-10T00:26:42.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:41 smithi062 ceph-mon[32390]: osdmap e66: 10 total, 9 up, 10 in 2024-06-10T00:26:42.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:41 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:42.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:41 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:42.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:41 smithi138 ceph-mon[36549]: from='osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-06-10T00:26:42.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:41 smithi138 ceph-mon[36549]: osdmap e66: 10 total, 9 up, 10 in 2024-06-10T00:26:42.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:41 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:42.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:41 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:42.685 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi105:vg_nvme/lv_1 2024-06-10T00:26:42.963 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:26:43.233 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:42 smithi105 ceph-mon[36321]: purged_snaps scrub starts 2024-06-10T00:26:43.233 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:42 smithi105 ceph-mon[36321]: purged_snaps scrub ok 2024-06-10T00:26:43.233 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:42 smithi105 ceph-mon[36321]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:43.233 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:42 smithi105 ceph-mon[36321]: osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101] boot 2024-06-10T00:26:43.233 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:42 smithi105 ceph-mon[36321]: osdmap e67: 10 total, 10 up, 10 in 2024-06-10T00:26:43.233 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:42 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:43.327 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:42 smithi062 ceph-mon[32390]: purged_snaps scrub starts 2024-06-10T00:26:43.328 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:42 smithi062 ceph-mon[32390]: purged_snaps scrub ok 2024-06-10T00:26:43.328 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:42 smithi062 ceph-mon[32390]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:43.328 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:42 smithi062 ceph-mon[32390]: osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101] boot 2024-06-10T00:26:43.328 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:42 smithi062 ceph-mon[32390]: osdmap e67: 10 total, 10 up, 10 in 2024-06-10T00:26:43.328 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:42 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:43.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:42 smithi138 ceph-mon[36549]: purged_snaps scrub starts 2024-06-10T00:26:43.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:42 smithi138 ceph-mon[36549]: purged_snaps scrub ok 2024-06-10T00:26:43.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:42 smithi138 ceph-mon[36549]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:26:43.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:42 smithi138 ceph-mon[36549]: osd.9 [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101] boot 2024-06-10T00:26:43.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:42 smithi138 ceph-mon[36549]: osdmap e67: 10 total, 10 up, 10 in 2024-06-10T00:26:43.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:42 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:26:44.173 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: osdmap e68: 10 total, 10 up, 10 in 2024-06-10T00:26:44.173 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: Detected new or changed devices on smithi062 2024-06-10T00:26:44.173 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.173 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.173 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:44.173 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:44.173 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:44.173 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:44.173 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi062 to 4272M 2024-06-10T00:26:44.173 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.173 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:44.174 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:26:44.174 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.174 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:26:44.174 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:44.174 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:26:44.174 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: osdmap e68: 10 total, 10 up, 10 in 2024-06-10T00:26:44.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: Detected new or changed devices on smithi062 2024-06-10T00:26:44.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:44.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:44.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:44.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:44.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi062 to 4272M 2024-06-10T00:26:44.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:44.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:26:44.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:26:44.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:44.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:26:44.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: osdmap e68: 10 total, 10 up, 10 in 2024-06-10T00:26:44.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: Detected new or changed devices on smithi062 2024-06-10T00:26:44.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:44.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:44.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:44.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:26:44.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi062 to 4272M 2024-06-10T00:26:44.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:44.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:26:44.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:26:44.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:44.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:26:44.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:26:44.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.745+0000 7f104d940700 1 -- 172.21.15.105:0/2207924394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10481036e0 msgr2=0x7f1048103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:26:44.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.745+0000 7f104d940700 1 --2- 172.21.15.105:0/2207924394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10481036e0 0x7f1048103b40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f103c0099c0 tx=0x7f103c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:26:44.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.745+0000 7f104d940700 1 -- 172.21.15.105:0/2207924394 shutdown_connections 2024-06-10T00:26:44.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.745+0000 7f104d940700 1 --2- 172.21.15.105:0/2207924394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10481024e0 0x7f1048102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:44.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.745+0000 7f104d940700 1 --2- 172.21.15.105:0/2207924394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1048104080 0x7f104810a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:44.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.745+0000 7f104d940700 1 --2- 172.21.15.105:0/2207924394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10481036e0 0x7f1048103b40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.745+0000 7f104d940700 1 -- 172.21.15.105:0/2207924394 >> 172.21.15.105:0/2207924394 conn(0x7f10480fda80 msgr2=0x7f10480ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:26:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.745+0000 7f104d940700 1 -- 172.21.15.105:0/2207924394 shutdown_connections 2024-06-10T00:26:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.745+0000 7f104d940700 1 -- 172.21.15.105:0/2207924394 wait complete. 2024-06-10T00:26:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.746+0000 7f104d940700 1 Processor -- start 2024-06-10T00:26:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.746+0000 7f104d940700 1 -- start start 2024-06-10T00:26:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.746+0000 7f104d940700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10481024e0 0x7f1048198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:26:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.746+0000 7f104d940700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10481036e0 0x7f1048198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:26:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.746+0000 7f104d940700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1048104080 0x7f104819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:26:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.746+0000 7f104d940700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f104810e120 con 0x7f1048104080 2024-06-10T00:26:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.746+0000 7f104d940700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f104810dfa0 con 0x7f10481036e0 2024-06-10T00:26:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.746+0000 7f104d940700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f104810e2a0 con 0x7f10481024e0 2024-06-10T00:26:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.746+0000 7f10467fc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10481036e0 0x7f1048198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:26:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.746+0000 7f10467fc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10481036e0 0x7f1048198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40160/0 (socket says 172.21.15.105:40160) 2024-06-10T00:26:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.746+0000 7f10467fc700 1 -- 172.21.15.105:0/1287272230 learned_addr learned my addr 172.21.15.105:0/1287272230 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-06-10T00:26:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.747+0000 7f10477fe700 1 --2- 172.21.15.105:0/1287272230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1048104080 0x7f104819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:26:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.747+0000 7f1046ffd700 1 --2- 172.21.15.105:0/1287272230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10481024e0 0x7f1048198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:26:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.747+0000 7f10467fc700 1 -- 172.21.15.105:0/1287272230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10481024e0 msgr2=0x7f1048198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:26:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.747+0000 7f10467fc700 1 --2- 172.21.15.105:0/1287272230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10481024e0 0x7f1048198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.747+0000 7f10467fc700 1 -- 172.21.15.105:0/1287272230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1048104080 msgr2=0x7f104819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:26:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.747+0000 7f10467fc700 1 --2- 172.21.15.105:0/1287272230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1048104080 0x7f104819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:26:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.747+0000 7f10467fc700 1 -- 172.21.15.105:0/1287272230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10481a0370 con 0x7f10481036e0 2024-06-10T00:26:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.747+0000 7f10467fc700 1 --2- 172.21.15.105:0/1287272230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10481036e0 0x7f1048198c50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f103c00c010 tx=0x7f103c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:26:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.747+0000 7f104c93e700 1 -- 172.21.15.105:0/1287272230 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c027070 con 0x7f10481036e0 2024-06-10T00:26:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.747+0000 7f104d940700 1 -- 172.21.15.105:0/1287272230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10481a0600 con 0x7f10481036e0 2024-06-10T00:26:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.747+0000 7f104c93e700 1 -- 172.21.15.105:0/1287272230 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f103c004130 con 0x7f10481036e0 2024-06-10T00:26:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.747+0000 7f104c93e700 1 -- 172.21.15.105:0/1287272230 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c022d70 con 0x7f10481036e0 2024-06-10T00:26:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.747+0000 7f104d940700 1 -- 172.21.15.105:0/1287272230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10481a0b30 con 0x7f10481036e0 2024-06-10T00:26:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.748+0000 7f104d940700 1 -- 172.21.15.105:0/1287272230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1048069180 con 0x7f10481036e0 2024-06-10T00:26:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.750+0000 7f104c93e700 1 -- 172.21.15.105:0/1287272230 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f103c02a070 con 0x7f10481036e0 2024-06-10T00:26:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.750+0000 7f104c93e700 1 --2- 172.21.15.105:0/1287272230 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f10300a2c20 0x7f10300a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:26:44.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.751+0000 7f104c93e700 1 -- 172.21.15.105:0/1287272230 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f103c0cfc90 con 0x7f10481036e0 2024-06-10T00:26:44.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.751+0000 7f1046ffd700 1 --2- 172.21.15.105:0/1287272230 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f10300a2c20 0x7f10300a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:26:44.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.751+0000 7f1046ffd700 1 --2- 172.21.15.105:0/1287272230 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f10300a2c20 0x7f10300a50e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1038005950 tx=0x7f10380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:26:44.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.754+0000 7f104c93e700 1 -- 172.21.15.105:0/1287272230 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f103c031070 con 0x7f10481036e0 2024-06-10T00:26:44.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:26:44.960+0000 7f104d940700 1 -- 172.21.15.105:0/1287272230 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1048062990 con 0x7f10300a2c20 2024-06-10T00:26:45.829 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:45 smithi105 ceph-mon[36321]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:45.829 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:26:45.830 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:26:45.830 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:45 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:45.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:45 smithi062 ceph-mon[32390]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:45.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:26:45.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:26:45.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:45 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:45.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:45 smithi138 ceph-mon[36549]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:45.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:26:45.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:26:45.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:45 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:46.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:46 smithi062 ceph-mon[32390]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:26:46.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:46 smithi138 ceph-mon[36549]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:26:46.879 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:46 smithi105 ceph-mon[36321]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:26:47.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:47 smithi062 ceph-mon[32390]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:47.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:47 smithi138 ceph-mon[36549]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:48.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:47 smithi105 ceph-mon[36321]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:48.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:48 smithi062 ceph-mon[32390]: from='client.? 172.21.15.105:0/1919252077' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55659c31-fcfc-4881-9533-8a1d6de76bc5"}]: dispatch 2024-06-10T00:26:48.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:48 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55659c31-fcfc-4881-9533-8a1d6de76bc5"}]: dispatch 2024-06-10T00:26:48.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:48 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55659c31-fcfc-4881-9533-8a1d6de76bc5"}]': finished 2024-06-10T00:26:48.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:48 smithi062 ceph-mon[32390]: osdmap e69: 11 total, 10 up, 11 in 2024-06-10T00:26:48.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:48 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:26:48.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:48 smithi062 ceph-mon[32390]: from='client.? 172.21.15.105:0/3790451427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:26:48.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:48 smithi138 ceph-mon[36549]: from='client.? 172.21.15.105:0/1919252077' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55659c31-fcfc-4881-9533-8a1d6de76bc5"}]: dispatch 2024-06-10T00:26:48.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:48 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55659c31-fcfc-4881-9533-8a1d6de76bc5"}]: dispatch 2024-06-10T00:26:48.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:48 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55659c31-fcfc-4881-9533-8a1d6de76bc5"}]': finished 2024-06-10T00:26:48.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:48 smithi138 ceph-mon[36549]: osdmap e69: 11 total, 10 up, 11 in 2024-06-10T00:26:48.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:48 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:26:48.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:48 smithi138 ceph-mon[36549]: from='client.? 172.21.15.105:0/3790451427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:26:49.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:48 smithi105 ceph-mon[36321]: from='client.? 172.21.15.105:0/1919252077' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55659c31-fcfc-4881-9533-8a1d6de76bc5"}]: dispatch 2024-06-10T00:26:49.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:48 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55659c31-fcfc-4881-9533-8a1d6de76bc5"}]: dispatch 2024-06-10T00:26:49.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:48 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55659c31-fcfc-4881-9533-8a1d6de76bc5"}]': finished 2024-06-10T00:26:49.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:48 smithi105 ceph-mon[36321]: osdmap e69: 11 total, 10 up, 11 in 2024-06-10T00:26:49.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:48 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:26:49.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:48 smithi105 ceph-mon[36321]: from='client.? 172.21.15.105:0/3790451427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:26:49.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:49 smithi062 ceph-mon[32390]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:49 smithi138 ceph-mon[36549]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:50.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:49 smithi105 ceph-mon[36321]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:51.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:51 smithi105 ceph-mon[36321]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:51.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:51 smithi138 ceph-mon[36549]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:52.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:51 smithi062 ceph-mon[32390]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:53.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:53 smithi138 ceph-mon[36549]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:53.991 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:53 smithi105 ceph-mon[36321]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:54.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:53 smithi062 ceph-mon[32390]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:55.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:55 smithi105 ceph-mon[36321]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:55.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-10T00:26:55.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:55 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:55.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:55 smithi062 ceph-mon[32390]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:55.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-10T00:26:55.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:55 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:55.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:55 smithi138 ceph-mon[36549]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:55.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-10T00:26:55.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:55 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:26:56.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:56 smithi105 ceph-mon[36321]: Deploying daemon osd.10 on smithi105 2024-06-10T00:26:57.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:56 smithi062 ceph-mon[32390]: Deploying daemon osd.10 on smithi105 2024-06-10T00:26:57.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:56 smithi138 ceph-mon[36549]: Deploying daemon osd.10 on smithi105 2024-06-10T00:26:57.918 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:57 smithi105 ceph-mon[36321]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:58.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:57 smithi062 ceph-mon[32390]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:58.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:57 smithi138 ceph-mon[36549]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:26:59.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:26:59 smithi105 ceph-mon[36321]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:00.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:26:59 smithi062 ceph-mon[32390]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:00.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:26:59 smithi138 ceph-mon[36549]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:01.925 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:01 smithi138 ceph-mon[36549]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:01.925 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:01 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:01.925 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:01 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:01.925 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:01 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:01.927 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:01 smithi105 ceph-mon[36321]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:01.927 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:01 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:01.927 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:01 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:01.927 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:01 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:02.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:01 smithi062 ceph-mon[32390]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:02.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:01 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:02.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:01 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:02.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:01 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:03.877 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:03 smithi105 ceph-mon[36321]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:04.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:03 smithi062 ceph-mon[32390]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:04.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:03 smithi138 ceph-mon[36549]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:05.181 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 10 on host 'smithi105' 2024-06-10T00:27:05.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:27:05.180+0000 7f104c93e700 1 -- 172.21.15.105:0/1287272230 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1048062990 con 0x7f10300a2c20 2024-06-10T00:27:05.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:27:05.184+0000 7f104d940700 1 -- 172.21.15.105:0/1287272230 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f10300a2c20 msgr2=0x7f10300a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:05.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:27:05.184+0000 7f104d940700 1 --2- 172.21.15.105:0/1287272230 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f10300a2c20 0x7f10300a50e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1038005950 tx=0x7f10380058e0 comp rx=0 tx=0).stop 2024-06-10T00:27:05.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:27:05.184+0000 7f104d940700 1 -- 172.21.15.105:0/1287272230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10481036e0 msgr2=0x7f1048198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:05.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:27:05.184+0000 7f104d940700 1 --2- 172.21.15.105:0/1287272230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10481036e0 0x7f1048198c50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f103c00c010 tx=0x7f103c005e80 comp rx=0 tx=0).stop 2024-06-10T00:27:05.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:27:05.184+0000 7f104d940700 1 -- 172.21.15.105:0/1287272230 shutdown_connections 2024-06-10T00:27:05.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:27:05.184+0000 7f104d940700 1 --2- 172.21.15.105:0/1287272230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1048104080 0x7f104819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:27:05.184+0000 7f104d940700 1 --2- 172.21.15.105:0/1287272230 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f10300a2c20 0x7f10300a50e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:27:05.184+0000 7f104d940700 1 --2- 172.21.15.105:0/1287272230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10481024e0 0x7f1048198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:27:05.184+0000 7f104d940700 1 --2- 172.21.15.105:0/1287272230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10481036e0 0x7f1048198c50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:27:05.184+0000 7f104d940700 1 -- 172.21.15.105:0/1287272230 >> 172.21.15.105:0/1287272230 conn(0x7f10480fda80 msgr2=0x7f1048106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:27:05.184+0000 7f104d940700 1 -- 172.21.15.105:0/1287272230 shutdown_connections 2024-06-10T00:27:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-06-10T00:27:05.184+0000 7f104d940700 1 -- 172.21.15.105:0/1287272230 wait complete. 2024-06-10T00:27:05.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:05 smithi105 ceph-mon[36321]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:05 smithi105 ceph-mon[36321]: from='osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T00:27:05.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:05 smithi062 ceph-mon[32390]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:05.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:05.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:05.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:05.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:05 smithi062 ceph-mon[32390]: from='osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T00:27:05.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:05 smithi138 ceph-mon[36549]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:05.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:05.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:05.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:05.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:05.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:05 smithi138 ceph-mon[36549]: from='osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T00:27:05.910 DEBUG:teuthology.orchestra.run.smithi105:osd.10> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.10.service 2024-06-10T00:27:05.913 INFO:tasks.cephadm:Deploying osd.11 on smithi138 with /dev/vg_nvme/lv_1... 2024-06-10T00:27:05.914 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-10T00:27:06.163 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:27:07.479 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:07 smithi105 ceph-mon[36321]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:07.479 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:07 smithi105 ceph-mon[36321]: from='osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679]' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-10T00:27:07.479 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:07 smithi105 ceph-mon[36321]: osdmap e70: 11 total, 10 up, 11 in 2024-06-10T00:27:07.480 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:07 smithi105 ceph-mon[36321]: from='osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:27:07.480 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:07 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:27:07.480 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:27:07 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:27:07.166+0000 7f1a15a29700 -1 osd.10 0 waiting for initial osdmap 2024-06-10T00:27:07.480 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:27:07 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:27:07.182+0000 7f1a1181c700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:27:07.547 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:07 smithi138 ceph-mon[36549]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:07.547 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:07 smithi138 ceph-mon[36549]: from='osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679]' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-10T00:27:07.547 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:07 smithi138 ceph-mon[36549]: osdmap e70: 11 total, 10 up, 11 in 2024-06-10T00:27:07.547 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:07 smithi138 ceph-mon[36549]: from='osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:27:07.547 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:07 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:27:07.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:07 smithi062 ceph-mon[32390]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:07.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:07 smithi062 ceph-mon[32390]: from='osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679]' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-10T00:27:07.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:07 smithi062 ceph-mon[32390]: osdmap e70: 11 total, 10 up, 11 in 2024-06-10T00:27:07.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:07 smithi062 ceph-mon[32390]: from='osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-06-10T00:27:07.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:07 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:27:08.234 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-10T00:27:08.234 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-10T00:27:08.234 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-06-10T00:27:08.234 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-06-10T00:27:08.234 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125719 s, 834 MB/s 2024-06-10T00:27:08.234 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-06-10T00:27:08.528 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:08 smithi138 ceph-mon[36549]: from='osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679]' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-06-10T00:27:08.528 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:08 smithi138 ceph-mon[36549]: osdmap e71: 11 total, 10 up, 11 in 2024-06-10T00:27:08.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:08 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:27:08.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:08 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:27:08.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:08 smithi138 ceph-mon[36549]: osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679] boot 2024-06-10T00:27:08.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:08 smithi138 ceph-mon[36549]: osdmap e72: 11 total, 11 up, 11 in 2024-06-10T00:27:08.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:08 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:27:08.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:08 smithi105 ceph-mon[36321]: from='osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679]' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-06-10T00:27:08.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:08 smithi105 ceph-mon[36321]: osdmap e71: 11 total, 10 up, 11 in 2024-06-10T00:27:08.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:08 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:27:08.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:08 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:27:08.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:08 smithi105 ceph-mon[36321]: osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679] boot 2024-06-10T00:27:08.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:08 smithi105 ceph-mon[36321]: osdmap e72: 11 total, 11 up, 11 in 2024-06-10T00:27:08.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:08 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:27:08.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:08 smithi062 ceph-mon[32390]: from='osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679]' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-06-10T00:27:08.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:08 smithi062 ceph-mon[32390]: osdmap e71: 11 total, 10 up, 11 in 2024-06-10T00:27:08.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:08 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:27:08.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:08 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:27:08.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:08 smithi062 ceph-mon[32390]: osd.10 [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679] boot 2024-06-10T00:27:08.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:08 smithi062 ceph-mon[32390]: osdmap e72: 11 total, 11 up, 11 in 2024-06-10T00:27:08.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:08 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:27:08.979 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_1 2024-06-10T00:27:09.220 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:27:09.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:09 smithi105 ceph-mon[36321]: purged_snaps scrub starts 2024-06-10T00:27:09.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:09 smithi105 ceph-mon[36321]: purged_snaps scrub ok 2024-06-10T00:27:09.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:09 smithi105 ceph-mon[36321]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:09.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:09.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:09.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:09.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:09.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:09.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:09.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:09.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:09.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:09.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:09.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:09 smithi105 ceph-mon[36321]: osdmap e73: 11 total, 11 up, 11 in 2024-06-10T00:27:09.342 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:09 smithi138 ceph-mon[36549]: purged_snaps scrub starts 2024-06-10T00:27:09.342 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:09 smithi138 ceph-mon[36549]: purged_snaps scrub ok 2024-06-10T00:27:09.342 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:09 smithi138 ceph-mon[36549]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:09.342 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:09.342 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:09.342 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:09.342 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:09.342 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:09.342 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:09.342 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:09.342 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:09.343 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:09.343 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:09.343 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:09 smithi138 ceph-mon[36549]: osdmap e73: 11 total, 11 up, 11 in 2024-06-10T00:27:09.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:09 smithi062 ceph-mon[32390]: purged_snaps scrub starts 2024-06-10T00:27:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:09 smithi062 ceph-mon[32390]: purged_snaps scrub ok 2024-06-10T00:27:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:09 smithi062 ceph-mon[32390]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:27:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:09.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:09.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:09 smithi062 ceph-mon[32390]: osdmap e73: 11 total, 11 up, 11 in 2024-06-10T00:27:10.447 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:10 smithi138 ceph-mon[36549]: Detected new or changed devices on smithi105 2024-06-10T00:27:10.447 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:10 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi105 to 4272M 2024-06-10T00:27:10.447 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:10 smithi138 ceph-mon[36549]: osdmap e74: 11 total, 11 up, 11 in 2024-06-10T00:27:10.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:10 smithi105 ceph-mon[36321]: Detected new or changed devices on smithi105 2024-06-10T00:27:10.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:10 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi105 to 4272M 2024-06-10T00:27:10.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:10 smithi105 ceph-mon[36321]: osdmap e74: 11 total, 11 up, 11 in 2024-06-10T00:27:10.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:10 smithi062 ceph-mon[32390]: Detected new or changed devices on smithi105 2024-06-10T00:27:10.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:10 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi105 to 4272M 2024-06-10T00:27:10.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:10 smithi062 ceph-mon[32390]: osdmap e74: 11 total, 11 up, 11 in 2024-06-10T00:27:10.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.813+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/940671778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c54104150 msgr2=0x7f5c5410aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:10.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.813+0000 7f5c5c0d4700 1 --2- 172.21.15.138:0/940671778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c54104150 0x7f5c5410aa00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009990 tx=0x7f5c50009ca0 comp rx=0 tx=0).stop 2024-06-10T00:27:10.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.813+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/940671778 shutdown_connections 2024-06-10T00:27:10.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.813+0000 7f5c5c0d4700 1 --2- 172.21.15.138:0/940671778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c541007d0 0x7f5c54100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:10.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.813+0000 7f5c5c0d4700 1 --2- 172.21.15.138:0/940671778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c54104150 0x7f5c5410aa00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:10.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.813+0000 7f5c5c0d4700 1 --2- 172.21.15.138:0/940671778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c541037b0 0x7f5c54103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:10.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.813+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/940671778 >> 172.21.15.138:0/940671778 conn(0x7f5c540fc130 msgr2=0x7f5c540fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:10.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.813+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/940671778 shutdown_connections 2024-06-10T00:27:10.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.813+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/940671778 wait complete. 2024-06-10T00:27:10.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5c0d4700 1 Processor -- start 2024-06-10T00:27:10.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5c0d4700 1 -- start start 2024-06-10T00:27:10.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5c0d4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c541007d0 0x7f5c54103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:10.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5c0d4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c541037b0 0x7f5c54101760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:10.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5c0d4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c54104150 0x7f5c54101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:10.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5c0d4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5410e0d0 con 0x7f5c541007d0 2024-06-10T00:27:10.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5c0d4700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5410df50 con 0x7f5c54104150 2024-06-10T00:27:10.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5c0d4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5410e250 con 0x7f5c541037b0 2024-06-10T00:27:10.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5966f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c541037b0 0x7f5c54101760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:10.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5966f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c541037b0 0x7f5c54101760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59452/0 (socket says 172.21.15.138:59452) 2024-06-10T00:27:10.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5966f700 1 -- 172.21.15.138:0/2351530826 learned_addr learned my addr 172.21.15.138:0/2351530826 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-06-10T00:27:10.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c59e70700 1 --2- 172.21.15.138:0/2351530826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c541007d0 0x7f5c54103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:10.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5966f700 1 -- 172.21.15.138:0/2351530826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c54104150 msgr2=0x7f5c54101ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:10.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5966f700 1 --2- 172.21.15.138:0/2351530826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c54104150 0x7f5c54101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:10.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5966f700 1 -- 172.21.15.138:0/2351530826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c541007d0 msgr2=0x7f5c54103110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:10.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5966f700 1 --2- 172.21.15.138:0/2351530826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c541007d0 0x7f5c54103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:10.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5966f700 1 -- 172.21.15.138:0/2351530826 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c54102550 con 0x7f5c541037b0 2024-06-10T00:27:10.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.814+0000 7f5c5966f700 1 --2- 172.21.15.138:0/2351530826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c541037b0 0x7f5c54101760 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400c9b0 tx=0x7f5c4400cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:10.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.815+0000 7f5c4affd700 1 -- 172.21.15.138:0/2351530826 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44007b00 con 0x7f5c541037b0 2024-06-10T00:27:10.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.815+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/2351530826 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c541a2750 con 0x7f5c541037b0 2024-06-10T00:27:10.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.815+0000 7f5c4affd700 1 -- 172.21.15.138:0/2351530826 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c44007ca0 con 0x7f5c541037b0 2024-06-10T00:27:10.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.815+0000 7f5c4affd700 1 -- 172.21.15.138:0/2351530826 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44018780 con 0x7f5c541037b0 2024-06-10T00:27:10.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.815+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/2351530826 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c541a2c90 con 0x7f5c541037b0 2024-06-10T00:27:10.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.816+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/2351530826 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c54069180 con 0x7f5c541037b0 2024-06-10T00:27:10.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.817+0000 7f5c4affd700 1 -- 172.21.15.138:0/2351530826 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5c4400f450 con 0x7f5c541037b0 2024-06-10T00:27:10.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.818+0000 7f5c4affd700 1 --2- 172.21.15.138:0/2351530826 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f5c400a2c20 0x7f5c400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:10.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.818+0000 7f5c59e70700 1 --2- 172.21.15.138:0/2351530826 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f5c400a2c20 0x7f5c400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:10.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.818+0000 7f5c4affd700 1 -- 172.21.15.138:0/2351530826 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7f5c440ced60 con 0x7f5c541037b0 2024-06-10T00:27:10.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.818+0000 7f5c59e70700 1 --2- 172.21.15.138:0/2351530826 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f5c400a2c20 0x7f5c400a50e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5c5406a9e0 tx=0x7f5c4c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:10.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:10.822+0000 7f5c4affd700 1 -- 172.21.15.138:0/2351530826 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5c44014030 con 0x7f5c541037b0 2024-06-10T00:27:11.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:11.026+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/2351530826 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5c540628d0 con 0x7f5c400a2c20 2024-06-10T00:27:11.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:11 smithi138 ceph-mon[36549]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:11.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:27:11.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:27:11.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:11.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:11 smithi138 ceph-mon[36549]: osdmap e75: 11 total, 11 up, 11 in 2024-06-10T00:27:11.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:11 smithi105 ceph-mon[36321]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:11.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:11 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:27:11.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:11 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:27:11.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:11 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:11.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:11 smithi105 ceph-mon[36321]: osdmap e75: 11 total, 11 up, 11 in 2024-06-10T00:27:11.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:11 smithi062 ceph-mon[32390]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:11.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:27:11.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:27:11.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:11.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:11 smithi062 ceph-mon[32390]: osdmap e75: 11 total, 11 up, 11 in 2024-06-10T00:27:12.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:12 smithi138 ceph-mon[36549]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:27:12.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:12 smithi138 ceph-mon[36549]: osdmap e76: 11 total, 11 up, 11 in 2024-06-10T00:27:12.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:12 smithi105 ceph-mon[36321]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:27:12.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:12 smithi105 ceph-mon[36321]: osdmap e76: 11 total, 11 up, 11 in 2024-06-10T00:27:12.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:12 smithi062 ceph-mon[32390]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:27:12.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:12 smithi062 ceph-mon[32390]: osdmap e76: 11 total, 11 up, 11 in 2024-06-10T00:27:13.509 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:13 smithi062 ceph-mon[32390]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-10T00:27:13.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:13 smithi105 ceph-mon[36321]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-10T00:27:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:13 smithi138 ceph-mon[36549]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-10T00:27:14.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:14 smithi105 ceph-mon[36321]: from='client.? 172.21.15.138:0/2717901763' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c01832bc-be75-400a-a707-5fe11d996b54"}]: dispatch 2024-06-10T00:27:14.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:14 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c01832bc-be75-400a-a707-5fe11d996b54"}]: dispatch 2024-06-10T00:27:14.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:14 smithi105 ceph-mon[36321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c01832bc-be75-400a-a707-5fe11d996b54"}]': finished 2024-06-10T00:27:14.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:14 smithi105 ceph-mon[36321]: osdmap e77: 12 total, 11 up, 12 in 2024-06-10T00:27:14.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:14 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:14.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:14 smithi062 ceph-mon[32390]: from='client.? 172.21.15.138:0/2717901763' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c01832bc-be75-400a-a707-5fe11d996b54"}]: dispatch 2024-06-10T00:27:14.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:14 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c01832bc-be75-400a-a707-5fe11d996b54"}]: dispatch 2024-06-10T00:27:14.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:14 smithi062 ceph-mon[32390]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c01832bc-be75-400a-a707-5fe11d996b54"}]': finished 2024-06-10T00:27:14.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:14 smithi062 ceph-mon[32390]: osdmap e77: 12 total, 11 up, 12 in 2024-06-10T00:27:14.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:14 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:14.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:14 smithi138 ceph-mon[36549]: from='client.? 172.21.15.138:0/2717901763' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c01832bc-be75-400a-a707-5fe11d996b54"}]: dispatch 2024-06-10T00:27:14.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:14 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c01832bc-be75-400a-a707-5fe11d996b54"}]: dispatch 2024-06-10T00:27:14.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:14 smithi138 ceph-mon[36549]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c01832bc-be75-400a-a707-5fe11d996b54"}]': finished 2024-06-10T00:27:14.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:14 smithi138 ceph-mon[36549]: osdmap e77: 12 total, 11 up, 12 in 2024-06-10T00:27:14.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:14 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:15.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:15 smithi105 ceph-mon[36321]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-10T00:27:15.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:15 smithi105 ceph-mon[36321]: from='client.? 172.21.15.138:0/2207417354' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:27:15.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:15 smithi062 ceph-mon[32390]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-10T00:27:15.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:15 smithi062 ceph-mon[32390]: from='client.? 172.21.15.138:0/2207417354' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:27:15.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:15 smithi138 ceph-mon[36549]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-10T00:27:15.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:15 smithi138 ceph-mon[36549]: from='client.? 172.21.15.138:0/2207417354' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:27:17.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:17 smithi105 ceph-mon[36321]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-10T00:27:17.548 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:17 smithi138 ceph-mon[36549]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-10T00:27:17.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:17 smithi062 ceph-mon[32390]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-10T00:27:19.276 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:19 smithi138 ceph-mon[36549]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-06-10T00:27:19.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:19 smithi105 ceph-mon[36321]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-06-10T00:27:19.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:19 smithi062 ceph-mon[32390]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-06-10T00:27:21.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:21 smithi105 ceph-mon[36321]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:21.534 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:21 smithi138 ceph-mon[36549]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:21.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:21 smithi062 ceph-mon[32390]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:22.471 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-10T00:27:22.471 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:22.471 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:22 smithi138 ceph-mon[36549]: Deploying daemon osd.11 on smithi138 2024-06-10T00:27:22.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-10T00:27:22.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:22.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:22 smithi105 ceph-mon[36321]: Deploying daemon osd.11 on smithi138 2024-06-10T00:27:22.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-10T00:27:22.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:22.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:22 smithi062 ceph-mon[32390]: Deploying daemon osd.11 on smithi138 2024-06-10T00:27:23.512 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:23 smithi138 ceph-mon[36549]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:23.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:23 smithi105 ceph-mon[36321]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:23.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:23 smithi062 ceph-mon[32390]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:25.529 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:25 smithi138 ceph-mon[36549]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:25.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:25 smithi105 ceph-mon[36321]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:25.610 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:25 smithi062 ceph-mon[32390]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:27.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:27 smithi105 ceph-mon[36321]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:27 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:27 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:27 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:27.543 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:27 smithi138 ceph-mon[36549]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:27.543 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:27 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:27.543 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:27 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:27.543 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:27 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:27.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:27 smithi062 ceph-mon[32390]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:27.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:27 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:27.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:27 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:27.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:27 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:29.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:29 smithi138 ceph-mon[36549]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:29.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:29 smithi105 ceph-mon[36321]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:29.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:29 smithi062 ceph-mon[32390]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:31.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:31.143+0000 7f5c4affd700 1 -- 172.21.15.138:0/2351530826 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5c540628d0 con 0x7f5c400a2c20 2024-06-10T00:27:31.144 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 11 on host 'smithi138' 2024-06-10T00:27:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:31.146+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/2351530826 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f5c400a2c20 msgr2=0x7f5c400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:31.146+0000 7f5c5c0d4700 1 --2- 172.21.15.138:0/2351530826 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f5c400a2c20 0x7f5c400a50e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5c5406a9e0 tx=0x7f5c4c009040 comp rx=0 tx=0).stop 2024-06-10T00:27:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:31.146+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/2351530826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c541037b0 msgr2=0x7f5c54101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:31.146+0000 7f5c5c0d4700 1 --2- 172.21.15.138:0/2351530826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c541037b0 0x7f5c54101760 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400c9b0 tx=0x7f5c4400cd70 comp rx=0 tx=0).stop 2024-06-10T00:27:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:31.146+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/2351530826 shutdown_connections 2024-06-10T00:27:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:31.146+0000 7f5c5c0d4700 1 --2- 172.21.15.138:0/2351530826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c541007d0 0x7f5c54103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:31.146+0000 7f5c5c0d4700 1 --2- 172.21.15.138:0/2351530826 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f5c400a2c20 0x7f5c400a50e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:31.146+0000 7f5c5c0d4700 1 --2- 172.21.15.138:0/2351530826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c54104150 0x7f5c54101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:31.146+0000 7f5c5c0d4700 1 --2- 172.21.15.138:0/2351530826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c541037b0 0x7f5c54101760 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:31.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:31.146+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/2351530826 >> 172.21.15.138:0/2351530826 conn(0x7f5c540fc130 msgr2=0x7f5c541069e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:31.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:31.146+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/2351530826 shutdown_connections 2024-06-10T00:27:31.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:31.146+0000 7f5c5c0d4700 1 -- 172.21.15.138:0/2351530826 wait complete. 2024-06-10T00:27:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:31 smithi138 ceph-mon[36549]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:31 smithi138 ceph-mon[36549]: from='osd.11 [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T00:27:31.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:31 smithi138 ceph-mon[36549]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T00:27:31.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:31.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:31 smithi105 ceph-mon[36321]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:31.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:31.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:31.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:31 smithi105 ceph-mon[36321]: from='osd.11 [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T00:27:31.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:31 smithi105 ceph-mon[36321]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T00:27:31.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:31.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:31 smithi062 ceph-mon[32390]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:31.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:31.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:31.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:31 smithi062 ceph-mon[32390]: from='osd.11 [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T00:27:31.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:31 smithi062 ceph-mon[32390]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T00:27:31.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:31.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:31.844 DEBUG:teuthology.orchestra.run.smithi138:osd.11> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.11.service 2024-06-10T00:27:31.846 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-10T00:27:31.847 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-10T00:27:32.108 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:27:32.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:32 smithi138 ceph-mon[36549]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-10T00:27:32.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:32 smithi138 ceph-mon[36549]: osdmap e78: 12 total, 11 up, 12 in 2024-06-10T00:27:32.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:32 smithi138 ceph-mon[36549]: from='osd.11 [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:27:32.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:32 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:32.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:32 smithi138 ceph-mon[36549]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:27:32.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:32 smithi105 ceph-mon[36321]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-10T00:27:32.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:32 smithi105 ceph-mon[36321]: osdmap e78: 12 total, 11 up, 12 in 2024-06-10T00:27:32.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:32 smithi105 ceph-mon[36321]: from='osd.11 [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:27:32.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:32 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:32.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:32 smithi105 ceph-mon[36321]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:27:32.547 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:32 smithi062 ceph-mon[32390]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-10T00:27:32.548 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:32 smithi062 ceph-mon[32390]: osdmap e78: 12 total, 11 up, 12 in 2024-06-10T00:27:32.548 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:32 smithi062 ceph-mon[32390]: from='osd.11 [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:27:32.548 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:32 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:32.548 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:32 smithi062 ceph-mon[32390]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-06-10T00:27:33.067 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:27:32 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:27:32.976+0000 7f758c73e700 -1 osd.11 0 waiting for initial osdmap 2024-06-10T00:27:33.067 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:27:32 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:27:32.991+0000 7f7584529700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:27:33.357 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:33 smithi138 ceph-mon[36549]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:33.357 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:33 smithi138 ceph-mon[36549]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-06-10T00:27:33.357 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:33 smithi138 ceph-mon[36549]: osdmap e79: 12 total, 11 up, 12 in 2024-06-10T00:27:33.358 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:33.358 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:33.358 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:33 smithi138 ceph-mon[36549]: from='osd.11 ' entity='osd.11' 2024-06-10T00:27:33.358 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:33.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:33 smithi105 ceph-mon[36321]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:33.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:33 smithi105 ceph-mon[36321]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-06-10T00:27:33.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:33 smithi105 ceph-mon[36321]: osdmap e79: 12 total, 11 up, 12 in 2024-06-10T00:27:33.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:33.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:33.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:33 smithi105 ceph-mon[36321]: from='osd.11 ' entity='osd.11' 2024-06-10T00:27:33.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:33.547 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:33 smithi062 ceph-mon[32390]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:33.547 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:33 smithi062 ceph-mon[32390]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-06-10T00:27:33.547 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:33 smithi062 ceph-mon[32390]: osdmap e79: 12 total, 11 up, 12 in 2024-06-10T00:27:33.547 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:33.547 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:33.547 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:33 smithi062 ceph-mon[32390]: from='osd.11 ' entity='osd.11' 2024-06-10T00:27:33.547 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:33.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.897+0000 7fbb3ed51700 1 -- 172.21.15.62:0/3909860479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb38101710 msgr2=0x7fbb38101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:33.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.897+0000 7fbb3ed51700 1 --2- 172.21.15.62:0/3909860479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb38101710 0x7fbb38101b90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009990 tx=0x7fbb24009ca0 comp rx=0 tx=0).stop 2024-06-10T00:27:33.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.897+0000 7fbb3ed51700 1 -- 172.21.15.62:0/3909860479 shutdown_connections 2024-06-10T00:27:33.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.897+0000 7fbb3ed51700 1 --2- 172.21.15.62:0/3909860479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb381020d0 0x7fbb38102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:33.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.897+0000 7fbb3ed51700 1 --2- 172.21.15.62:0/3909860479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb38100db0 0x7fbb381011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:33.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.897+0000 7fbb3ed51700 1 --2- 172.21.15.62:0/3909860479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb38101710 0x7fbb38101b90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:33.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.897+0000 7fbb3ed51700 1 -- 172.21.15.62:0/3909860479 >> 172.21.15.62:0/3909860479 conn(0x7fbb380fc9f0 msgr2=0x7fbb380fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:33.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.897+0000 7fbb3ed51700 1 -- 172.21.15.62:0/3909860479 shutdown_connections 2024-06-10T00:27:33.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.897+0000 7fbb3ed51700 1 -- 172.21.15.62:0/3909860479 wait complete. 2024-06-10T00:27:33.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.897+0000 7fbb3ed51700 1 Processor -- start 2024-06-10T00:27:33.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.897+0000 7fbb3ed51700 1 -- start start 2024-06-10T00:27:33.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb3ed51700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb38100db0 0x7fbb3819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:33.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb3ed51700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb38101710 0x7fbb3819af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:33.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb3ed51700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb381020d0 0x7fbb381a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:33.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb3ed51700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38110380 con 0x7fbb38101710 2024-06-10T00:27:33.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb3ed51700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38110200 con 0x7fbb38100db0 2024-06-10T00:27:33.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb3ed51700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38110500 con 0x7fbb381020d0 2024-06-10T00:27:33.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb2ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb38101710 0x7fbb3819af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:33.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb2ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb38101710 0x7fbb3819af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54290/0 (socket says 172.21.15.62:54290) 2024-06-10T00:27:33.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb2ffff700 1 -- 172.21.15.62:0/2040110877 learned_addr learned my addr 172.21.15.62:0/2040110877 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:27:33.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb3caed700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb38100db0 0x7fbb3819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:33.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb3d2ee700 1 --2- 172.21.15.62:0/2040110877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb381020d0 0x7fbb381a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:33.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb2ffff700 1 -- 172.21.15.62:0/2040110877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb381020d0 msgr2=0x7fbb381a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:33.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb2ffff700 1 --2- 172.21.15.62:0/2040110877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb381020d0 0x7fbb381a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:33.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb2ffff700 1 -- 172.21.15.62:0/2040110877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb38100db0 msgr2=0x7fbb3819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:33.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb2ffff700 1 --2- 172.21.15.62:0/2040110877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb38100db0 0x7fbb3819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:33.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb2ffff700 1 -- 172.21.15.62:0/2040110877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb381a2630 con 0x7fbb38101710 2024-06-10T00:27:33.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb3caed700 1 --2- 172.21.15.62:0/2040110877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb38100db0 0x7fbb3819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:27:33.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.898+0000 7fbb2ffff700 1 --2- 172.21.15.62:0/2040110877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb38101710 0x7fbb3819af10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbb24000c00 tx=0x7fbb24005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:33.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.899+0000 7fbb367fc700 1 -- 172.21.15.62:0/2040110877 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24027070 con 0x7fbb38101710 2024-06-10T00:27:33.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.899+0000 7fbb3ed51700 1 -- 172.21.15.62:0/2040110877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb381a28c0 con 0x7fbb38101710 2024-06-10T00:27:33.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.899+0000 7fbb367fc700 1 -- 172.21.15.62:0/2040110877 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb24003f40 con 0x7fbb38101710 2024-06-10T00:27:33.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.899+0000 7fbb367fc700 1 -- 172.21.15.62:0/2040110877 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24022e30 con 0x7fbb38101710 2024-06-10T00:27:33.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.899+0000 7fbb3ed51700 1 -- 172.21.15.62:0/2040110877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb381a2d90 con 0x7fbb38101710 2024-06-10T00:27:33.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.900+0000 7fbb3ed51700 1 -- 172.21.15.62:0/2040110877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb38069180 con 0x7fbb38101710 2024-06-10T00:27:33.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.901+0000 7fbb367fc700 1 -- 172.21.15.62:0/2040110877 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbb2402a070 con 0x7fbb38101710 2024-06-10T00:27:33.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.901+0000 7fbb367fc700 1 --2- 172.21.15.62:0/2040110877 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbb1c0a2c20 0x7fbb1c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:33.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.901+0000 7fbb3caed700 1 --2- 172.21.15.62:0/2040110877 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbb1c0a2c20 0x7fbb1c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:33.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.901+0000 7fbb367fc700 1 -- 172.21.15.62:0/2040110877 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7fbb240cf640 con 0x7fbb38101710 2024-06-10T00:27:33.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.902+0000 7fbb3caed700 1 --2- 172.21.15.62:0/2040110877 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbb1c0a2c20 0x7fbb1c0a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009fd0 tx=0x7fbb28009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:33.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:33.906+0000 7fbb367fc700 1 -- 172.21.15.62:0/2040110877 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbb24013070 con 0x7fbb38101710 2024-06-10T00:27:34.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:34.113+0000 7fbb3ed51700 1 -- 172.21.15.62:0/2040110877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbb380630a0 con 0x7fbb38101710 2024-06-10T00:27:34.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:34.114+0000 7fbb367fc700 1 -- 172.21.15.62:0/2040110877 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7fbb2409cbf0 con 0x7fbb38101710 2024-06-10T00:27:34.114 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:27:34.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:34.116+0000 7fbb3ed51700 1 -- 172.21.15.62:0/2040110877 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbb1c0a2c20 msgr2=0x7fbb1c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:34.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:34.116+0000 7fbb3ed51700 1 --2- 172.21.15.62:0/2040110877 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbb1c0a2c20 0x7fbb1c0a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009fd0 tx=0x7fbb28009180 comp rx=0 tx=0).stop 2024-06-10T00:27:34.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:34.116+0000 7fbb3ed51700 1 -- 172.21.15.62:0/2040110877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb38101710 msgr2=0x7fbb3819af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:34.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:34.116+0000 7fbb3ed51700 1 --2- 172.21.15.62:0/2040110877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb38101710 0x7fbb3819af10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbb24000c00 tx=0x7fbb24005e80 comp rx=0 tx=0).stop 2024-06-10T00:27:34.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:34.116+0000 7fbb3ed51700 1 -- 172.21.15.62:0/2040110877 shutdown_connections 2024-06-10T00:27:34.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:34.116+0000 7fbb3ed51700 1 --2- 172.21.15.62:0/2040110877 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbb1c0a2c20 0x7fbb1c0a50e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:34.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:34.116+0000 7fbb3ed51700 1 --2- 172.21.15.62:0/2040110877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb381020d0 0x7fbb381a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:34.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:34.116+0000 7fbb3ed51700 1 --2- 172.21.15.62:0/2040110877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb38100db0 0x7fbb3819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:34.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:34.116+0000 7fbb3ed51700 1 --2- 172.21.15.62:0/2040110877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb38101710 0x7fbb3819af10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:34.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:34.116+0000 7fbb3ed51700 1 -- 172.21.15.62:0/2040110877 >> 172.21.15.62:0/2040110877 conn(0x7fbb380fc9f0 msgr2=0x7fbb380fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:34.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:34.116+0000 7fbb3ed51700 1 -- 172.21.15.62:0/2040110877 shutdown_connections 2024-06-10T00:27:34.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:34.116+0000 7fbb3ed51700 1 -- 172.21.15.62:0/2040110877 wait complete. 2024-06-10T00:27:34.773 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1717979253,"num_in_osds":12,"osd_in_since":1717979233,"num_remapped_pgs":0} 2024-06-10T00:27:34.774 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:27:35.042 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:27:35.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: purged_snaps scrub starts 2024-06-10T00:27:35.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: purged_snaps scrub ok 2024-06-10T00:27:35.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:35.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: osd.11 [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685] boot 2024-06-10T00:27:35.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: osdmap e80: 12 total, 12 up, 12 in 2024-06-10T00:27:35.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:35.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2040110877' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-10T00:27:35.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:35.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:35.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:35.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:35.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:35.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:35.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:35.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:35.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:35.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:34 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:35.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: purged_snaps scrub starts 2024-06-10T00:27:35.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: purged_snaps scrub ok 2024-06-10T00:27:35.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:35.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: osd.11 [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685] boot 2024-06-10T00:27:35.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: osdmap e80: 12 total, 12 up, 12 in 2024-06-10T00:27:35.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:35.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2040110877' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-10T00:27:35.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:35.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:35.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:35.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:35.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:35.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:35.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:35.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:35.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:35.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:34 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:35.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: purged_snaps scrub starts 2024-06-10T00:27:35.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: purged_snaps scrub ok 2024-06-10T00:27:35.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:27:35.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: osd.11 [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685] boot 2024-06-10T00:27:35.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: osdmap e80: 12 total, 12 up, 12 in 2024-06-10T00:27:35.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:27:35.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2040110877' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-10T00:27:35.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:35.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:35.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:35.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:35.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:35.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:27:35.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:35.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:35.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:35.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:34 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:36.156 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:35 smithi062 ceph-mon[32390]: Detected new or changed devices on smithi138 2024-06-10T00:27:36.156 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:35 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi138 to 4272M 2024-06-10T00:27:36.156 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:35 smithi062 ceph-mon[32390]: osdmap e81: 12 total, 12 up, 12 in 2024-06-10T00:27:36.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:35 smithi105 ceph-mon[36321]: Detected new or changed devices on smithi138 2024-06-10T00:27:36.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:35 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi138 to 4272M 2024-06-10T00:27:36.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:35 smithi105 ceph-mon[36321]: osdmap e81: 12 total, 12 up, 12 in 2024-06-10T00:27:36.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:35 smithi138 ceph-mon[36549]: Detected new or changed devices on smithi138 2024-06-10T00:27:36.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:35 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi138 to 4272M 2024-06-10T00:27:36.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:35 smithi138 ceph-mon[36549]: osdmap e81: 12 total, 12 up, 12 in 2024-06-10T00:27:36.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.681+0000 7fdfd3a89700 1 -- 172.21.15.62:0/3391255071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfcc108580 msgr2=0x7fdfcc108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.681+0000 7fdfd3a89700 1 --2- 172.21.15.62:0/3391255071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfcc108580 0x7fdfcc108960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80099c0 tx=0x7fdfc8009cd0 comp rx=0 tx=0).stop 2024-06-10T00:27:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.681+0000 7fdfd3a89700 1 -- 172.21.15.62:0/3391255071 shutdown_connections 2024-06-10T00:27:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.681+0000 7fdfd3a89700 1 --2- 172.21.15.62:0/3391255071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfcc108580 0x7fdfcc108960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.681+0000 7fdfd3a89700 1 --2- 172.21.15.62:0/3391255071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfcc06b620 0x7fdfcc10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.681+0000 7fdfd3a89700 1 --2- 172.21.15.62:0/3391255071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfcc06ac60 0x7fdfcc06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.681+0000 7fdfd3a89700 1 -- 172.21.15.62:0/3391255071 >> 172.21.15.62:0/3391255071 conn(0x7fdfcc078430 msgr2=0x7fdfcc078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.681+0000 7fdfd3a89700 1 -- 172.21.15.62:0/3391255071 shutdown_connections 2024-06-10T00:27:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.681+0000 7fdfd3a89700 1 -- 172.21.15.62:0/3391255071 wait complete. 2024-06-10T00:27:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.681+0000 7fdfd3a89700 1 Processor -- start 2024-06-10T00:27:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.681+0000 7fdfd3a89700 1 -- start start 2024-06-10T00:27:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd3a89700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfcc06ac60 0x7fdfcc19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd3a89700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfcc06b620 0x7fdfcc19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd3a89700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfcc108580 0x7fdfcc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd3a89700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1127c0 con 0x7fdfcc108580 2024-06-10T00:27:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd3a89700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc112640 con 0x7fdfcc06b620 2024-06-10T00:27:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd3a89700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc112940 con 0x7fdfcc06ac60 2024-06-10T00:27:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd2026700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfcc108580 0x7fdfcc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd2026700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfcc108580 0x7fdfcc1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54296/0 (socket says 172.21.15.62:54296) 2024-06-10T00:27:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd2026700 1 -- 172.21.15.62:0/551552699 learned_addr learned my addr 172.21.15.62:0/551552699 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:27:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd1825700 1 --2- 172.21.15.62:0/551552699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfcc06ac60 0x7fdfcc19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd1024700 1 --2- 172.21.15.62:0/551552699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfcc06b620 0x7fdfcc19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd2026700 1 -- 172.21.15.62:0/551552699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfcc06ac60 msgr2=0x7fdfcc19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd2026700 1 --2- 172.21.15.62:0/551552699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfcc06ac60 0x7fdfcc19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd2026700 1 -- 172.21.15.62:0/551552699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfcc06b620 msgr2=0x7fdfcc19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd2026700 1 --2- 172.21.15.62:0/551552699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfcc06b620 0x7fdfcc19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd2026700 1 -- 172.21.15.62:0/551552699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc199120 con 0x7fdfcc108580 2024-06-10T00:27:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd2026700 1 --2- 172.21.15.62:0/551552699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfcc108580 0x7fdfcc1989c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4007d70 tx=0x7fdfc400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfc2ffd700 1 -- 172.21.15.62:0/551552699 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4016020 con 0x7fdfcc108580 2024-06-10T00:27:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfc2ffd700 1 -- 172.21.15.62:0/551552699 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfc400ad40 con 0x7fdfcc108580 2024-06-10T00:27:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.682+0000 7fdfd3a89700 1 -- 172.21.15.62:0/551552699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc199410 con 0x7fdfcc108580 2024-06-10T00:27:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.683+0000 7fdfc2ffd700 1 -- 172.21.15.62:0/551552699 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4011600 con 0x7fdfcc108580 2024-06-10T00:27:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.683+0000 7fdfd3a89700 1 -- 172.21.15.62:0/551552699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc1a56d0 con 0x7fdfcc108580 2024-06-10T00:27:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.683+0000 7fdfd3a89700 1 -- 172.21.15.62:0/551552699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfcc10aa10 con 0x7fdfcc108580 2024-06-10T00:27:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.685+0000 7fdfc2ffd700 1 -- 172.21.15.62:0/551552699 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdfc4019020 con 0x7fdfcc108580 2024-06-10T00:27:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.686+0000 7fdfc2ffd700 1 --2- 172.21.15.62:0/551552699 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdfb80a2ca0 0x7fdfb80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.686+0000 7fdfc2ffd700 1 -- 172.21.15.62:0/551552699 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fdfc40cee40 con 0x7fdfcc108580 2024-06-10T00:27:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.686+0000 7fdfd1825700 1 --2- 172.21.15.62:0/551552699 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdfb80a2ca0 0x7fdfb80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:36.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.686+0000 7fdfd1825700 1 --2- 172.21.15.62:0/551552699 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdfb80a2ca0 0x7fdfb80a5160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8000c00 tx=0x7fdfc8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:36.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.689+0000 7fdfc2ffd700 1 -- 172.21.15.62:0/551552699 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdfc4020020 con 0x7fdfcc108580 2024-06-10T00:27:36.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.895+0000 7fdfd3a89700 1 -- 172.21.15.62:0/551552699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfcc10a3f0 con 0x7fdfcc108580 2024-06-10T00:27:36.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.895+0000 7fdfc2ffd700 1 -- 172.21.15.62:0/551552699 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18454 (secure 0 0 0) 0x7fdfc409c240 con 0x7fdfcc108580 2024-06-10T00:27:36.896 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:27:36.896 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":81,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:27:34.981091+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:27:36.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.898+0000 7fdfd3a89700 1 -- 172.21.15.62:0/551552699 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdfb80a2ca0 msgr2=0x7fdfb80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:36.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.898+0000 7fdfd3a89700 1 --2- 172.21.15.62:0/551552699 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdfb80a2ca0 0x7fdfb80a5160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8000c00 tx=0x7fdfc8005fb0 comp rx=0 tx=0).stop 2024-06-10T00:27:36.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.898+0000 7fdfd3a89700 1 -- 172.21.15.62:0/551552699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfcc108580 msgr2=0x7fdfcc1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:36.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.898+0000 7fdfd3a89700 1 --2- 172.21.15.62:0/551552699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfcc108580 0x7fdfcc1989c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4007d70 tx=0x7fdfc400a340 comp rx=0 tx=0).stop 2024-06-10T00:27:36.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.898+0000 7fdfd3a89700 1 -- 172.21.15.62:0/551552699 shutdown_connections 2024-06-10T00:27:36.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.898+0000 7fdfd3a89700 1 --2- 172.21.15.62:0/551552699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfcc108580 0x7fdfcc1989c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:36.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.898+0000 7fdfd3a89700 1 --2- 172.21.15.62:0/551552699 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdfb80a2ca0 0x7fdfb80a5160 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:36.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.898+0000 7fdfd3a89700 1 --2- 172.21.15.62:0/551552699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfcc06ac60 0x7fdfcc19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:36.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.898+0000 7fdfd3a89700 1 --2- 172.21.15.62:0/551552699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfcc06b620 0x7fdfcc19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:36.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.898+0000 7fdfd3a89700 1 -- 172.21.15.62:0/551552699 >> 172.21.15.62:0/551552699 conn(0x7fdfcc078430 msgr2=0x7fdfcc0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:36.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.898+0000 7fdfd3a89700 1 -- 172.21.15.62:0/551552699 shutdown_connections 2024-06-10T00:27:36.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:36.898+0000 7fdfd3a89700 1 -- 172.21.15.62:0/551552699 wait complete. 2024-06-10T00:27:37.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:36 smithi105 ceph-mon[36321]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:37.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:36 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/551552699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:27:37.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:36 smithi062 ceph-mon[32390]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:37.334 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:36 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/551552699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:27:37.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:36 smithi138 ceph-mon[36549]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:37.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:36 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/551552699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:27:37.629 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-10T00:23:46.372672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-10T00:27:37.629 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-10T00:27:37.878 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:27:39.255 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:38 smithi062 ceph-mon[32390]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:39.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:38 smithi105 ceph-mon[36321]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:39.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:38 smithi138 ceph-mon[36549]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:39.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.477+0000 7fc815b55700 1 -- 172.21.15.62:0/2195918812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8101084f0 msgr2=0x7fc8101088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:39.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.477+0000 7fc815b55700 1 --2- 172.21.15.62:0/2195918812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8101084f0 0x7fc8101088d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc800009990 tx=0x7fc800009ca0 comp rx=0 tx=0).stop 2024-06-10T00:27:39.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.477+0000 7fc815b55700 1 -- 172.21.15.62:0/2195918812 shutdown_connections 2024-06-10T00:27:39.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.477+0000 7fc815b55700 1 --2- 172.21.15.62:0/2195918812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8101084f0 0x7fc8101088d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:39.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.477+0000 7fc815b55700 1 --2- 172.21.15.62:0/2195918812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc81006b3c0 0x7fc81010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:39.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.477+0000 7fc815b55700 1 --2- 172.21.15.62:0/2195918812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc81006aa00 0x7fc81006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:39.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.477+0000 7fc815b55700 1 -- 172.21.15.62:0/2195918812 >> 172.21.15.62:0/2195918812 conn(0x7fc810078300 msgr2=0x7fc810078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:39.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.477+0000 7fc815b55700 1 -- 172.21.15.62:0/2195918812 shutdown_connections 2024-06-10T00:27:39.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.477+0000 7fc815b55700 1 -- 172.21.15.62:0/2195918812 wait complete. 2024-06-10T00:27:39.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.477+0000 7fc815b55700 1 Processor -- start 2024-06-10T00:27:39.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc815b55700 1 -- start start 2024-06-10T00:27:39.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc815b55700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc81006aa00 0x7fc81019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:39.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc815b55700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc81006b3c0 0x7fc81019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:39.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc815b55700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8101084f0 0x7fc81019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:39.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc815b55700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc810112770 con 0x7fc81006b3c0 2024-06-10T00:27:39.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc815b55700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc8101125f0 con 0x7fc8101084f0 2024-06-10T00:27:39.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc815b55700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8101128f0 con 0x7fc81006aa00 2024-06-10T00:27:39.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc80effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc81006b3c0 0x7fc81019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:39.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc80effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc81006b3c0 0x7fc81019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34180/0 (socket says 172.21.15.62:34180) 2024-06-10T00:27:39.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc80effd700 1 -- 172.21.15.62:0/4126953897 learned_addr learned my addr 172.21.15.62:0/4126953897 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:27:39.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc80ffff700 1 --2- 172.21.15.62:0/4126953897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8101084f0 0x7fc81019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:39.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc80f7fe700 1 --2- 172.21.15.62:0/4126953897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc81006aa00 0x7fc81019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:39.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc80effd700 1 -- 172.21.15.62:0/4126953897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc81006aa00 msgr2=0x7fc81019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:39.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc80effd700 1 --2- 172.21.15.62:0/4126953897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc81006aa00 0x7fc81019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:39.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc80effd700 1 -- 172.21.15.62:0/4126953897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8101084f0 msgr2=0x7fc81019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:39.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc80effd700 1 --2- 172.21.15.62:0/4126953897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8101084f0 0x7fc81019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:39.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.478+0000 7fc80effd700 1 -- 172.21.15.62:0/4126953897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81019fdd0 con 0x7fc81006b3c0 2024-06-10T00:27:39.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.479+0000 7fc80effd700 1 --2- 172.21.15.62:0/4126953897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc81006b3c0 0x7fc81019b300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c920 tx=0x7fc80400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:39.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.479+0000 7fc80cff9700 1 -- 172.21.15.62:0/4126953897 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804007b00 con 0x7fc81006b3c0 2024-06-10T00:27:39.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.479+0000 7fc815b55700 1 -- 172.21.15.62:0/4126953897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8101a00c0 con 0x7fc81006b3c0 2024-06-10T00:27:39.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.479+0000 7fc80cff9700 1 -- 172.21.15.62:0/4126953897 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc804004d10 con 0x7fc81006b3c0 2024-06-10T00:27:39.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.479+0000 7fc80cff9700 1 -- 172.21.15.62:0/4126953897 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804005630 con 0x7fc81006b3c0 2024-06-10T00:27:39.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.479+0000 7fc815b55700 1 -- 172.21.15.62:0/4126953897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8101a7940 con 0x7fc81006b3c0 2024-06-10T00:27:39.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.480+0000 7fc815b55700 1 -- 172.21.15.62:0/4126953897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc810069180 con 0x7fc81006b3c0 2024-06-10T00:27:39.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.481+0000 7fc80cff9700 1 -- 172.21.15.62:0/4126953897 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc804004750 con 0x7fc81006b3c0 2024-06-10T00:27:39.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.482+0000 7fc80cff9700 1 --2- 172.21.15.62:0/4126953897 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc7f80a2c20 0x7fc7f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:39.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.482+0000 7fc80f7fe700 1 --2- 172.21.15.62:0/4126953897 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc7f80a2c20 0x7fc7f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:39.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.482+0000 7fc80cff9700 1 -- 172.21.15.62:0/4126953897 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc8040ce730 con 0x7fc81006b3c0 2024-06-10T00:27:39.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.482+0000 7fc80f7fe700 1 --2- 172.21.15.62:0/4126953897 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc7f80a2c20 0x7fc7f80a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc800006010 tx=0x7fc80000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:39.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.486+0000 7fc80cff9700 1 -- 172.21.15.62:0/4126953897 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc804014030 con 0x7fc81006b3c0 2024-06-10T00:27:39.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.692+0000 7fc815b55700 1 -- 172.21.15.62:0/4126953897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8101a7b40 con 0x7fc81006b3c0 2024-06-10T00:27:39.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.692+0000 7fc80cff9700 1 -- 172.21.15.62:0/4126953897 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fc80409bb30 con 0x7fc81006b3c0 2024-06-10T00:27:39.692 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-06-10T00:27:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.695+0000 7fc815b55700 1 -- 172.21.15.62:0/4126953897 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc7f80a2c20 msgr2=0x7fc7f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.695+0000 7fc815b55700 1 --2- 172.21.15.62:0/4126953897 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc7f80a2c20 0x7fc7f80a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc800006010 tx=0x7fc80000b540 comp rx=0 tx=0).stop 2024-06-10T00:27:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.695+0000 7fc815b55700 1 -- 172.21.15.62:0/4126953897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc81006b3c0 msgr2=0x7fc81019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.695+0000 7fc815b55700 1 --2- 172.21.15.62:0/4126953897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc81006b3c0 0x7fc81019b300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c920 tx=0x7fc80400cc30 comp rx=0 tx=0).stop 2024-06-10T00:27:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.695+0000 7fc815b55700 1 -- 172.21.15.62:0/4126953897 shutdown_connections 2024-06-10T00:27:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.695+0000 7fc815b55700 1 --2- 172.21.15.62:0/4126953897 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc7f80a2c20 0x7fc7f80a50e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.695+0000 7fc815b55700 1 --2- 172.21.15.62:0/4126953897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8101084f0 0x7fc81019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.695+0000 7fc815b55700 1 --2- 172.21.15.62:0/4126953897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc81006aa00 0x7fc81019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.695+0000 7fc815b55700 1 --2- 172.21.15.62:0/4126953897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc81006b3c0 0x7fc81019b300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.695+0000 7fc815b55700 1 -- 172.21.15.62:0/4126953897 >> 172.21.15.62:0/4126953897 conn(0x7fc810078300 msgr2=0x7fc81010d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.695+0000 7fc815b55700 1 -- 172.21.15.62:0/4126953897 shutdown_connections 2024-06-10T00:27:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:39.695+0000 7fc815b55700 1 -- 172.21.15.62:0/4126953897 wait complete. 2024-06-10T00:27:40.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:40 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/4126953897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:27:40.332 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:40 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/4126953897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:27:40.362 INFO:tasks.cephadm:Adding mds.a on smithi062 2024-06-10T00:27:40.362 INFO:tasks.cephadm:Adding mds.d on smithi062 2024-06-10T00:27:40.362 INFO:tasks.cephadm:Adding mds.g on smithi062 2024-06-10T00:27:40.362 INFO:tasks.cephadm:Adding mds.j on smithi062 2024-06-10T00:27:40.362 INFO:tasks.cephadm:Adding mds.b on smithi105 2024-06-10T00:27:40.362 INFO:tasks.cephadm:Adding mds.e on smithi105 2024-06-10T00:27:40.363 INFO:tasks.cephadm:Adding mds.h on smithi105 2024-06-10T00:27:40.363 INFO:tasks.cephadm:Adding mds.k on smithi105 2024-06-10T00:27:40.363 INFO:tasks.cephadm:Adding mds.c on smithi138 2024-06-10T00:27:40.363 INFO:tasks.cephadm:Adding mds.f on smithi138 2024-06-10T00:27:40.363 INFO:tasks.cephadm:Adding mds.i on smithi138 2024-06-10T00:27:40.363 INFO:tasks.cephadm:Adding mds.l on smithi138 2024-06-10T00:27:40.363 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l' 2024-06-10T00:27:40.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:40 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/4126953897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:27:40.610 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:27:41.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:41 smithi105 ceph-mon[36321]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:41.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:41 smithi062 ceph-mon[32390]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:41.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:41 smithi138 ceph-mon[36549]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.066+0000 7f2998e32700 1 -- 172.21.15.138:0/364086995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29941024c0 msgr2=0x7f29941028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.066+0000 7f2998e32700 1 --2- 172.21.15.138:0/364086995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29941024c0 0x7f29941028e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2984009990 tx=0x7f2984009ca0 comp rx=0 tx=0).stop 2024-06-10T00:27:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.066+0000 7f2998e32700 1 -- 172.21.15.138:0/364086995 shutdown_connections 2024-06-10T00:27:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.066+0000 7f2998e32700 1 --2- 172.21.15.138:0/364086995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2994104060 0x7f299410a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.066+0000 7f2998e32700 1 --2- 172.21.15.138:0/364086995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29941024c0 0x7f29941028e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.066+0000 7f2998e32700 1 --2- 172.21.15.138:0/364086995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29941036c0 0x7f2994103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:42.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.066+0000 7f2998e32700 1 -- 172.21.15.138:0/364086995 >> 172.21.15.138:0/364086995 conn(0x7f29940fda60 msgr2=0x7f29940ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:42.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.066+0000 7f2998e32700 1 -- 172.21.15.138:0/364086995 shutdown_connections 2024-06-10T00:27:42.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.066+0000 7f2998e32700 1 -- 172.21.15.138:0/364086995 wait complete. 2024-06-10T00:27:42.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.066+0000 7f2998e32700 1 Processor -- start 2024-06-10T00:27:42.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2998e32700 1 -- start start 2024-06-10T00:27:42.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2998e32700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29941024c0 0x7f2994198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:42.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2998e32700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29941036c0 0x7f2994198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:42.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2998e32700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2994104060 0x7f299419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:42.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2998e32700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f299410e0a0 con 0x7f2994104060 2024-06-10T00:27:42.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2998e32700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f299410df20 con 0x7f29941024c0 2024-06-10T00:27:42.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2998e32700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f299410e220 con 0x7f29941036c0 2024-06-10T00:27:42.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2991d9b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29941036c0 0x7f2994198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:42.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2991d9b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29941036c0 0x7f2994198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48660/0 (socket says 172.21.15.138:48660) 2024-06-10T00:27:42.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2991d9b700 1 -- 172.21.15.138:0/732565221 learned_addr learned my addr 172.21.15.138:0/732565221 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-06-10T00:27:42.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2992d9d700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2994104060 0x7f299419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:42.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f299259c700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29941024c0 0x7f2994198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:42.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2991d9b700 1 -- 172.21.15.138:0/732565221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29941024c0 msgr2=0x7f2994198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:42.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2991d9b700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29941024c0 0x7f2994198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:42.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2991d9b700 1 -- 172.21.15.138:0/732565221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2994104060 msgr2=0x7f299419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:42.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2991d9b700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2994104060 0x7f299419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:42.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2991d9b700 1 -- 172.21.15.138:0/732565221 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29941a02c0 con 0x7f29941036c0 2024-06-10T00:27:42.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f299259c700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29941024c0 0x7f2994198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:27:42.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2991d9b700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29941036c0 0x7f2994198ba0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f298800c9e0 tx=0x7f298800ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:42.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f29837fe700 1 -- 172.21.15.138:0/732565221 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988007ac0 con 0x7f29941036c0 2024-06-10T00:27:42.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f29837fe700 1 -- 172.21.15.138:0/732565221 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2988007c60 con 0x7f29941036c0 2024-06-10T00:27:42.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f2998e32700 1 -- 172.21.15.138:0/732565221 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29941a05b0 con 0x7f29941036c0 2024-06-10T00:27:42.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.067+0000 7f29837fe700 1 -- 172.21.15.138:0/732565221 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988018730 con 0x7f29941036c0 2024-06-10T00:27:42.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.068+0000 7f2998e32700 1 -- 172.21.15.138:0/732565221 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29941a0b40 con 0x7f29941036c0 2024-06-10T00:27:42.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.069+0000 7f2998e32700 1 -- 172.21.15.138:0/732565221 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2994069180 con 0x7f29941036c0 2024-06-10T00:27:42.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.070+0000 7f29837fe700 1 -- 172.21.15.138:0/732565221 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f29880188d0 con 0x7f29941036c0 2024-06-10T00:27:42.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.070+0000 7f29837fe700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f297c0a2ca0 0x7f297c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:42.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.070+0000 7f29837fe700 1 -- 172.21.15.138:0/732565221 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f29880ceda0 con 0x7f29941036c0 2024-06-10T00:27:42.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.070+0000 7f299259c700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f297c0a2ca0 0x7f297c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:42.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.071+0000 7f299259c700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f297c0a2ca0 0x7f297c0a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2984000c00 tx=0x7f2984005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:42.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.074+0000 7f29837fe700 1 -- 172.21.15.138:0/732565221 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2988014030 con 0x7f29941036c0 2024-06-10T00:27:42.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.282+0000 7f2998e32700 1 -- 172.21.15.138:0/732565221 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l", "target": ["mon-mgr", ""]}) -- 0x7f2994062ad0 con 0x7f297c0a2ca0 2024-06-10T00:27:42.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.290+0000 7f29837fe700 1 -- 172.21.15.138:0/732565221 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f2994062ad0 con 0x7f297c0a2ca0 2024-06-10T00:27:42.290 INFO:teuthology.orchestra.run.smithi138.stdout:Scheduled mds.all update... 2024-06-10T00:27:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.292+0000 7f2998e32700 1 -- 172.21.15.138:0/732565221 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f297c0a2ca0 msgr2=0x7f297c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.292+0000 7f2998e32700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f297c0a2ca0 0x7f297c0a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2984000c00 tx=0x7f2984005fb0 comp rx=0 tx=0).stop 2024-06-10T00:27:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.292+0000 7f2998e32700 1 -- 172.21.15.138:0/732565221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29941036c0 msgr2=0x7f2994198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.292+0000 7f2998e32700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29941036c0 0x7f2994198ba0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f298800c9e0 tx=0x7f298800ccf0 comp rx=0 tx=0).stop 2024-06-10T00:27:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.292+0000 7f2998e32700 1 -- 172.21.15.138:0/732565221 shutdown_connections 2024-06-10T00:27:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.292+0000 7f2998e32700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2994104060 0x7f299419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.292+0000 7f2998e32700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f297c0a2ca0 0x7f297c0a5160 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.292+0000 7f2998e32700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29941024c0 0x7f2994198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.292+0000 7f2998e32700 1 --2- 172.21.15.138:0/732565221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29941036c0 0x7f2994198ba0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:42.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.292+0000 7f2998e32700 1 -- 172.21.15.138:0/732565221 >> 172.21.15.138:0/732565221 conn(0x7f29940fda60 msgr2=0x7f29941069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:42.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.292+0000 7f2998e32700 1 -- 172.21.15.138:0/732565221 shutdown_connections 2024-06-10T00:27:42.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-06-10T00:27:42.293+0000 7f2998e32700 1 -- 172.21.15.138:0/732565221 wait complete. 2024-06-10T00:27:42.904 DEBUG:teuthology.orchestra.run.smithi062:mds.a> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.a.service 2024-06-10T00:27:42.906 DEBUG:teuthology.orchestra.run.smithi062:mds.d> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.d.service 2024-06-10T00:27:42.950 DEBUG:teuthology.orchestra.run.smithi062:mds.g> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.g.service 2024-06-10T00:27:42.992 DEBUG:teuthology.orchestra.run.smithi062:mds.j> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.j.service 2024-06-10T00:27:43.036 DEBUG:teuthology.orchestra.run.smithi105:mds.b> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.b.service 2024-06-10T00:27:43.038 DEBUG:teuthology.orchestra.run.smithi105:mds.e> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.e.service 2024-06-10T00:27:43.081 DEBUG:teuthology.orchestra.run.smithi105:mds.h> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.h.service 2024-06-10T00:27:43.123 DEBUG:teuthology.orchestra.run.smithi105:mds.k> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.k.service 2024-06-10T00:27:43.166 DEBUG:teuthology.orchestra.run.smithi138:mds.c> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.c.service 2024-06-10T00:27:43.169 DEBUG:teuthology.orchestra.run.smithi138:mds.f> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.f.service 2024-06-10T00:27:43.212 DEBUG:teuthology.orchestra.run.smithi138:mds.i> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.i.service 2024-06-10T00:27:43.254 DEBUG:teuthology.orchestra.run.smithi138:mds.l> sudo journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.l.service 2024-06-10T00:27:43.297 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-10T00:27:43.298 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:27:43.502 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:43 smithi138 ceph-mon[36549]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:43.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:43 smithi138 ceph-mon[36549]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:27:43.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:43 smithi138 ceph-mon[36549]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l;count:12 2024-06-10T00:27:43.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:43.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:43.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:43.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:43.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:43.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:43.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:43.503 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:43 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:43.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:43 smithi105 ceph-mon[36321]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:43.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:43 smithi105 ceph-mon[36321]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:27:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:43 smithi105 ceph-mon[36321]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l;count:12 2024-06-10T00:27:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:43 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:43.544 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:27:43.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:43 smithi062 ceph-mon[32390]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:43.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:43 smithi062 ceph-mon[32390]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:27:43.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:43 smithi062 ceph-mon[32390]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l;count:12 2024-06-10T00:27:43.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:43.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:27:43.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:43.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:27:43.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:43.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:43.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:43.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:43 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:44.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:44 smithi105 ceph-mon[36321]: Deploying daemon mds.l on smithi138 2024-06-10T00:27:44.607 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:44 smithi138 ceph-mon[36549]: Deploying daemon mds.l on smithi138 2024-06-10T00:27:44.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:44 smithi062 ceph-mon[32390]: Deploying daemon mds.l on smithi138 2024-06-10T00:27:45.282 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:27:45 smithi138 systemd[1]: Starting Ceph mds.l for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:27:45.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.317+0000 7f360e14c700 1 -- 172.21.15.62:0/2998781782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36081013c0 msgr2=0x7f3608101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:45.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.317+0000 7f360e14c700 1 --2- 172.21.15.62:0/2998781782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36081013c0 0x7f3608101840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009990 tx=0x7f35fc009ca0 comp rx=0 tx=0).stop 2024-06-10T00:27:45.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.317+0000 7f360e14c700 1 -- 172.21.15.62:0/2998781782 shutdown_connections 2024-06-10T00:27:45.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.317+0000 7f360e14c700 1 --2- 172.21.15.62:0/2998781782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3608101d80 0x7f360810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:45.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.317+0000 7f360e14c700 1 --2- 172.21.15.62:0/2998781782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3608107440 0x7f3608107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:45.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.317+0000 7f360e14c700 1 --2- 172.21.15.62:0/2998781782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36081013c0 0x7f3608101840 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:45.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.317+0000 7f360e14c700 1 -- 172.21.15.62:0/2998781782 >> 172.21.15.62:0/2998781782 conn(0x7f3608077400 msgr2=0x7f3608077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:45.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.317+0000 7f360e14c700 1 -- 172.21.15.62:0/2998781782 shutdown_connections 2024-06-10T00:27:45.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.317+0000 7f360e14c700 1 -- 172.21.15.62:0/2998781782 wait complete. 2024-06-10T00:27:45.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.318+0000 7f360e14c700 1 Processor -- start 2024-06-10T00:27:45.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.318+0000 7f360e14c700 1 -- start start 2024-06-10T00:27:45.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.318+0000 7f360e14c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36081013c0 0x7f360819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:45.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.318+0000 7f360e14c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3608101d80 0x7f360819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:45.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.318+0000 7f360e14c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3608107440 0x7f3608198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:45.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.318+0000 7f360e14c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3608111680 con 0x7f3608107440 2024-06-10T00:27:45.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.318+0000 7f360e14c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3608111500 con 0x7f36081013c0 2024-06-10T00:27:45.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.318+0000 7f360e14c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3608111800 con 0x7f3608101d80 2024-06-10T00:27:45.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.318+0000 7f3607fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3608107440 0x7f3608198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:45.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.318+0000 7f3607fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3608107440 0x7f3608198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34196/0 (socket says 172.21.15.62:34196) 2024-06-10T00:27:45.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.318+0000 7f3607fff700 1 -- 172.21.15.62:0/2446715231 learned_addr learned my addr 172.21.15.62:0/2446715231 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:27:45.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.318+0000 7f3606ffd700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3608101d80 0x7f360819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:45.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.318+0000 7f36077fe700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36081013c0 0x7f360819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:45.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.319+0000 7f3607fff700 1 -- 172.21.15.62:0/2446715231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3608101d80 msgr2=0x7f360819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:45.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.319+0000 7f3607fff700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3608101d80 0x7f360819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:45.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.319+0000 7f3607fff700 1 -- 172.21.15.62:0/2446715231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36081013c0 msgr2=0x7f360819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:45.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.319+0000 7f3607fff700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36081013c0 0x7f360819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:45.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.319+0000 7f3607fff700 1 -- 172.21.15.62:0/2446715231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36081991c0 con 0x7f3608107440 2024-06-10T00:27:45.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.319+0000 7f3606ffd700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3608101d80 0x7f360819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:27:45.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.319+0000 7f3607fff700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3608107440 0x7f3608198a60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3600007ee0 tx=0x7f360000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:45.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.319+0000 7f3604ff9700 1 -- 172.21.15.62:0/2446715231 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360000ddd0 con 0x7f3608107440 2024-06-10T00:27:45.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.319+0000 7f3604ff9700 1 -- 172.21.15.62:0/2446715231 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3600004020 con 0x7f3608107440 2024-06-10T00:27:45.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.319+0000 7f360e14c700 1 -- 172.21.15.62:0/2446715231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36081994b0 con 0x7f3608107440 2024-06-10T00:27:45.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.319+0000 7f3604ff9700 1 -- 172.21.15.62:0/2446715231 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600015660 con 0x7f3608107440 2024-06-10T00:27:45.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.319+0000 7f360e14c700 1 -- 172.21.15.62:0/2446715231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36081a5770 con 0x7f3608107440 2024-06-10T00:27:45.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.320+0000 7f360e14c700 1 -- 172.21.15.62:0/2446715231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3608069180 con 0x7f3608107440 2024-06-10T00:27:45.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.321+0000 7f3604ff9700 1 -- 172.21.15.62:0/2446715231 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f36000041d0 con 0x7f3608107440 2024-06-10T00:27:45.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.322+0000 7f3604ff9700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f35f00a2c20 0x7f35f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:45.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.322+0000 7f36077fe700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f35f00a2c20 0x7f35f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:45.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.322+0000 7f3604ff9700 1 -- 172.21.15.62:0/2446715231 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f36000cee70 con 0x7f3608107440 2024-06-10T00:27:45.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.322+0000 7f36077fe700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f35f00a2c20 0x7f35f00a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005e10 tx=0x7f35f8005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:45.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.325+0000 7f3604ff9700 1 -- 172.21.15.62:0/2446715231 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3600010070 con 0x7f3608107440 2024-06-10T00:27:45.554 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:45 smithi138 ceph-mon[36549]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:45.555 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:27:45 smithi138 podman[65968]: 2024-06-10 00:27:45.185661372 +0000 UTC m=+0.017640366 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:27:45.555 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:27:45 smithi138 podman[65968]: 2024-06-10 00:27:45.285474629 +0000 UTC m=+0.117453611 container create f5decf1d16c409b22fd658e40889be176651fb43af3e5720657a5c0da4d93b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-10T00:27:45.555 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:27:45 smithi138 podman[65968]: 2024-06-10 00:27:45.530026052 +0000 UTC m=+0.362005042 container init f5decf1d16c409b22fd658e40889be176651fb43af3e5720657a5c0da4d93b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-l, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-06-10T00:27:45.555 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:27:45 smithi138 podman[65968]: 2024-06-10 00:27:45.532534865 +0000 UTC m=+0.364513857 container start f5decf1d16c409b22fd658e40889be176651fb43af3e5720657a5c0da4d93b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-l, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:27:45.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.579+0000 7f360e14c700 1 -- 172.21.15.62:0/2446715231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36081a5970 con 0x7f3608107440 2024-06-10T00:27:45.579 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:45 smithi062 ceph-mon[32390]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:45.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.579+0000 7f3604ff9700 1 -- 172.21.15.62:0/2446715231 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f360009c270 con 0x7f3608107440 2024-06-10T00:27:45.580 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:27:45.580 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-10T00:27:45.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.581+0000 7f360e14c700 1 -- 172.21.15.62:0/2446715231 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f35f00a2c20 msgr2=0x7f35f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:45.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.581+0000 7f360e14c700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f35f00a2c20 0x7f35f00a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005e10 tx=0x7f35f8005d40 comp rx=0 tx=0).stop 2024-06-10T00:27:45.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.581+0000 7f360e14c700 1 -- 172.21.15.62:0/2446715231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3608107440 msgr2=0x7f3608198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:45.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.581+0000 7f360e14c700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3608107440 0x7f3608198a60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3600007ee0 tx=0x7f360000d3b0 comp rx=0 tx=0).stop 2024-06-10T00:27:45.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.581+0000 7f360e14c700 1 -- 172.21.15.62:0/2446715231 shutdown_connections 2024-06-10T00:27:45.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.581+0000 7f360e14c700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3608107440 0x7f3608198a60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:45.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.581+0000 7f360e14c700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f35f00a2c20 0x7f35f00a50e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:45.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.581+0000 7f360e14c700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36081013c0 0x7f360819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:45.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.581+0000 7f360e14c700 1 --2- 172.21.15.62:0/2446715231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3608101d80 0x7f360819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:45.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.581+0000 7f360e14c700 1 -- 172.21.15.62:0/2446715231 >> 172.21.15.62:0/2446715231 conn(0x7f3608077400 msgr2=0x7f3608100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:45.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.582+0000 7f360e14c700 1 -- 172.21.15.62:0/2446715231 shutdown_connections 2024-06-10T00:27:45.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:45.582+0000 7f360e14c700 1 -- 172.21.15.62:0/2446715231 wait complete. 2024-06-10T00:27:45.583 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 2 2024-06-10T00:27:45.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:45 smithi105 ceph-mon[36321]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:45.874 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:27:45 smithi138 ceph-mds[65987]: -- 172.21.15.138:0/1717227119 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e2de3a7a40 con 0x55e2de37ac00 2024-06-10T00:27:45.874 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:27:45 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-l[65983]: starting mds.l at 2024-06-10T00:27:45.874 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:27:45 smithi138 bash[65968]: f5decf1d16c409b22fd658e40889be176651fb43af3e5720657a5c0da4d93b06 2024-06-10T00:27:45.874 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:27:45 smithi138 systemd[1]: Started Ceph mds.l for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:27:46.595 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:46 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.138:6832/2673201423,v1:172.21.15.138:6833/2673201423] up:boot 2024-06-10T00:27:46.596 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:46 smithi105 ceph-mon[36321]: fsmap 1 up:standby 2024-06-10T00:27:46.596 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:46 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T00:27:46.596 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:46 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2446715231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:27:46.596 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:46 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:46.596 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:46 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:46.596 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:46 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:46.596 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:46 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:46.596 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:46 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:46.596 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:46 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:46.596 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:46 smithi105 ceph-mon[36321]: Deploying daemon mds.k on smithi105 2024-06-10T00:27:46.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:46 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.138:6832/2673201423,v1:172.21.15.138:6833/2673201423] up:boot 2024-06-10T00:27:46.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:46 smithi062 ceph-mon[32390]: fsmap 1 up:standby 2024-06-10T00:27:46.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:46 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T00:27:46.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:46 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2446715231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:27:46.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:46 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:46.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:46 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:46.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:46 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:46.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:46 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:46.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:46 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:46.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:46 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:46.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:46 smithi062 ceph-mon[32390]: Deploying daemon mds.k on smithi105 2024-06-10T00:27:46.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:46 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.138:6832/2673201423,v1:172.21.15.138:6833/2673201423] up:boot 2024-06-10T00:27:46.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:46 smithi138 ceph-mon[36549]: fsmap 1 up:standby 2024-06-10T00:27:46.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:46 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T00:27:46.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:46 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2446715231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:27:46.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:46 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:46.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:46 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:46.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:46 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:46.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:46 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:46.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:46 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:46.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:46 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:46.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:46 smithi138 ceph-mon[36549]: Deploying daemon mds.k on smithi105 2024-06-10T00:27:48.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:47 smithi105 ceph-mon[36321]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:48.032 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:27:47 smithi105 systemd[1]: Starting Ceph mds.k for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:27:48.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:47 smithi062 ceph-mon[32390]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:48.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:47 smithi138 ceph-mon[36549]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:48.312 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:27:48.406 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:27:48 smithi105 podman[65904]: 2024-06-10 00:27:48.041926499 +0000 UTC m=+0.016712231 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:27:48.406 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:27:48 smithi105 podman[65904]: 2024-06-10 00:27:48.154223519 +0000 UTC m=+0.129009281 container create 0ccb9711c5caa2ebec3130a017830d66cb8a3bfca3aecbf482b97977a1c53d5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-k, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-10T00:27:48.406 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:27:48 smithi105 podman[65904]: 2024-06-10 00:27:48.406926923 +0000 UTC m=+0.381712641 container init 0ccb9711c5caa2ebec3130a017830d66cb8a3bfca3aecbf482b97977a1c53d5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-10T00:27:48.575 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:27:48.782 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:27:48 smithi105 podman[65904]: 2024-06-10 00:27:48.409690452 +0000 UTC m=+0.384476171 container start 0ccb9711c5caa2ebec3130a017830d66cb8a3bfca3aecbf482b97977a1c53d5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-10T00:27:48.782 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:27:48 smithi105 ceph-mds[65923]: -- 172.21.15.105:0/4262870402 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555fb951da40 con 0x555fb94f0c00 2024-06-10T00:27:48.782 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:27:48 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-k[65919]: starting mds.k at 2024-06-10T00:27:48.782 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:27:48 smithi105 bash[65904]: 0ccb9711c5caa2ebec3130a017830d66cb8a3bfca3aecbf482b97977a1c53d5e 2024-06-10T00:27:48.782 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:27:48 smithi105 systemd[1]: Started Ceph mds.k for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:27:49.825 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:49 smithi105 ceph-mon[36321]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:49.825 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:49.825 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:49.826 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:49.826 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:49.826 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:49.826 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:49.826 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:49 smithi105 ceph-mon[36321]: Deploying daemon mds.j on smithi062 2024-06-10T00:27:49.826 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:49 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.105:6832/1194987741,v1:172.21.15.105:6833/1194987741] up:boot 2024-06-10T00:27:49.826 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:49 smithi105 ceph-mon[36321]: fsmap 2 up:standby 2024-06-10T00:27:49.826 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:49 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T00:27:49.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:49 smithi062 ceph-mon[32390]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:49.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:49.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:49.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:49.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:49.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:49.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:49.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:49 smithi062 ceph-mon[32390]: Deploying daemon mds.j on smithi062 2024-06-10T00:27:49.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:49 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.105:6832/1194987741,v1:172.21.15.105:6833/1194987741] up:boot 2024-06-10T00:27:49.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:49 smithi062 ceph-mon[32390]: fsmap 2 up:standby 2024-06-10T00:27:49.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:49 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T00:27:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:49 smithi138 ceph-mon[36549]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:49 smithi138 ceph-mon[36549]: Deploying daemon mds.j on smithi062 2024-06-10T00:27:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:49 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.105:6832/1194987741,v1:172.21.15.105:6833/1194987741] up:boot 2024-06-10T00:27:49.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:49 smithi138 ceph-mon[36549]: fsmap 2 up:standby 2024-06-10T00:27:49.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:49 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T00:27:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.470+0000 7f325f2b0700 1 -- 172.21.15.62:0/2336036863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3258107840 msgr2=0x7f325810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.470+0000 7f325f2b0700 1 --2- 172.21.15.62:0/2336036863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3258107840 0x7f325810f3c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f325000b0d0 tx=0x7f325000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:27:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.470+0000 7f325f2b0700 1 -- 172.21.15.62:0/2336036863 shutdown_connections 2024-06-10T00:27:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.470+0000 7f325f2b0700 1 --2- 172.21.15.62:0/2336036863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3258107840 0x7f325810f3c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.470+0000 7f325f2b0700 1 --2- 172.21.15.62:0/2336036863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3258102740 0x7f3258102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.470+0000 7f325f2b0700 1 --2- 172.21.15.62:0/2336036863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32581030f0 0x7f3258107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.470+0000 7f325f2b0700 1 -- 172.21.15.62:0/2336036863 >> 172.21.15.62:0/2336036863 conn(0x7f32580795f0 msgr2=0x7f3258079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.470+0000 7f325f2b0700 1 -- 172.21.15.62:0/2336036863 shutdown_connections 2024-06-10T00:27:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.470+0000 7f325f2b0700 1 -- 172.21.15.62:0/2336036863 wait complete. 2024-06-10T00:27:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.470+0000 7f325f2b0700 1 Processor -- start 2024-06-10T00:27:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.470+0000 7f325f2b0700 1 -- start start 2024-06-10T00:27:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325f2b0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3258102740 0x7f325819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325f2b0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32581030f0 0x7f325819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325f2b0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3258107840 0x7f3258198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325f2b0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3258112760 con 0x7f3258107840 2024-06-10T00:27:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325f2b0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32581125e0 con 0x7f32581030f0 2024-06-10T00:27:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325f2b0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32581128e0 con 0x7f3258102740 2024-06-10T00:27:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325d84d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3258107840 0x7f3258198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325c84b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32581030f0 0x7f325819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325d84d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3258107840 0x7f3258198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49614/0 (socket says 172.21.15.62:49614) 2024-06-10T00:27:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325d84d700 1 -- 172.21.15.62:0/461561135 learned_addr learned my addr 172.21.15.62:0/461561135 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:27:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325d04c700 1 --2- 172.21.15.62:0/461561135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3258102740 0x7f325819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325d84d700 1 -- 172.21.15.62:0/461561135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3258102740 msgr2=0x7f325819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325d84d700 1 --2- 172.21.15.62:0/461561135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3258102740 0x7f325819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325d84d700 1 -- 172.21.15.62:0/461561135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32581030f0 msgr2=0x7f325819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325d84d700 1 --2- 172.21.15.62:0/461561135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32581030f0 0x7f325819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325d84d700 1 -- 172.21.15.62:0/461561135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32581990a0 con 0x7f3258107840 2024-06-10T00:27:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325c84b700 1 --2- 172.21.15.62:0/461561135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32581030f0 0x7f325819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:27:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.471+0000 7f325d84d700 1 --2- 172.21.15.62:0/461561135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3258107840 0x7f3258198940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3250006070 tx=0x7f3250007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.472+0000 7f324e7fc700 1 -- 172.21.15.62:0/461561135 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250003c00 con 0x7f3258107840 2024-06-10T00:27:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.472+0000 7f324e7fc700 1 -- 172.21.15.62:0/461561135 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3250003da0 con 0x7f3258107840 2024-06-10T00:27:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.472+0000 7f325f2b0700 1 -- 172.21.15.62:0/461561135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3258199330 con 0x7f3258107840 2024-06-10T00:27:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.472+0000 7f324e7fc700 1 -- 172.21.15.62:0/461561135 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325001b880 con 0x7f3258107840 2024-06-10T00:27:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.472+0000 7f325f2b0700 1 -- 172.21.15.62:0/461561135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32581a5650 con 0x7f3258107840 2024-06-10T00:27:50.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.473+0000 7f325f2b0700 1 -- 172.21.15.62:0/461561135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3258103bd0 con 0x7f3258107840 2024-06-10T00:27:50.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.474+0000 7f324e7fc700 1 -- 172.21.15.62:0/461561135 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f325001ba20 con 0x7f3258107840 2024-06-10T00:27:50.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.475+0000 7f324e7fc700 1 --2- 172.21.15.62:0/461561135 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f32440a2c20 0x7f32440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:50.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.475+0000 7f325d04c700 1 --2- 172.21.15.62:0/461561135 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f32440a2c20 0x7f32440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:50.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.475+0000 7f324e7fc700 1 -- 172.21.15.62:0/461561135 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f32500cfff0 con 0x7f3258107840 2024-06-10T00:27:50.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.475+0000 7f325d04c700 1 --2- 172.21.15.62:0/461561135 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f32440a2c20 0x7f32440a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3254005950 tx=0x7f325400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.479+0000 7f324e7fc700 1 -- 172.21.15.62:0/461561135 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f325009d460 con 0x7f3258107840 2024-06-10T00:27:50.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.734+0000 7f325f2b0700 1 -- 172.21.15.62:0/461561135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f325802dca0 con 0x7f3258107840 2024-06-10T00:27:50.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.735+0000 7f324e7fc700 1 -- 172.21.15.62:0/461561135 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f325009cd60 con 0x7f3258107840 2024-06-10T00:27:50.735 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:27:50.735 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-10T00:27:50.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.737+0000 7f325f2b0700 1 -- 172.21.15.62:0/461561135 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f32440a2c20 msgr2=0x7f32440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:50.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.737+0000 7f325f2b0700 1 --2- 172.21.15.62:0/461561135 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f32440a2c20 0x7f32440a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3254005950 tx=0x7f325400a300 comp rx=0 tx=0).stop 2024-06-10T00:27:50.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.737+0000 7f325f2b0700 1 -- 172.21.15.62:0/461561135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3258107840 msgr2=0x7f3258198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:50.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.737+0000 7f325f2b0700 1 --2- 172.21.15.62:0/461561135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3258107840 0x7f3258198940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3250006070 tx=0x7f3250007a20 comp rx=0 tx=0).stop 2024-06-10T00:27:50.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.737+0000 7f325f2b0700 1 -- 172.21.15.62:0/461561135 shutdown_connections 2024-06-10T00:27:50.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.737+0000 7f325f2b0700 1 --2- 172.21.15.62:0/461561135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3258107840 0x7f3258198940 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:50.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.737+0000 7f325f2b0700 1 --2- 172.21.15.62:0/461561135 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f32440a2c20 0x7f32440a50e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:50.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.737+0000 7f325f2b0700 1 --2- 172.21.15.62:0/461561135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3258102740 0x7f325819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:50.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.737+0000 7f325f2b0700 1 --2- 172.21.15.62:0/461561135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32581030f0 0x7f325819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:50.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.737+0000 7f325f2b0700 1 -- 172.21.15.62:0/461561135 >> 172.21.15.62:0/461561135 conn(0x7f32580795f0 msgr2=0x7f32581005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:50.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.737+0000 7f325f2b0700 1 -- 172.21.15.62:0/461561135 shutdown_connections 2024-06-10T00:27:50.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:50.738+0000 7f325f2b0700 1 -- 172.21.15.62:0/461561135 wait complete. 2024-06-10T00:27:50.738 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 3 2024-06-10T00:27:51.706 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:51 smithi062 ceph-mon[32390]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:51.707 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:51 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/461561135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:27:51.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:51 smithi138 ceph-mon[36549]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:51.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:51 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/461561135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:27:52.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:51 smithi105 ceph-mon[36321]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:52.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:51 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/461561135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:27:52.097 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:27:51 smithi062 systemd[1]: Starting Ceph mds.j for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:27:52.358 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:27:52 smithi062 podman[72555]: 2024-06-10 00:27:52.307454326 +0000 UTC m=+0.114334676 container create 06963e5804a5081e43c84e7af4e149132772990ba585ee5a5486d81c9746283d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-10T00:27:52.358 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:27:52 smithi062 podman[72555]: 2024-06-10 00:27:52.210555775 +0000 UTC m=+0.017436149 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:27:52.858 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:27:52 smithi062 podman[72555]: 2024-06-10 00:27:52.560270487 +0000 UTC m=+0.367150835 container init 06963e5804a5081e43c84e7af4e149132772990ba585ee5a5486d81c9746283d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:27:52.858 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:27:52 smithi062 podman[72555]: 2024-06-10 00:27:52.565025691 +0000 UTC m=+0.371906041 container start 06963e5804a5081e43c84e7af4e149132772990ba585ee5a5486d81c9746283d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-j, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-10T00:27:52.858 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:27:52 smithi062 ceph-mds[72575]: -- 172.21.15.62:0/3258452088 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5574ed349a40 con 0x5574ed31c800 2024-06-10T00:27:52.858 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:27:52 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-j[72571]: starting mds.j at 2024-06-10T00:27:52.858 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:27:52 smithi062 bash[72555]: 06963e5804a5081e43c84e7af4e149132772990ba585ee5a5486d81c9746283d 2024-06-10T00:27:52.858 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:27:52 smithi062 systemd[1]: Started Ceph mds.j for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:27:54.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:53 smithi105 ceph-mon[36321]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:54.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:54.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:54.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:54.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:54.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:54.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:54.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:53 smithi105 ceph-mon[36321]: Deploying daemon mds.d on smithi062 2024-06-10T00:27:54.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:53 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.62:6834/3839480602,v1:172.21.15.62:6835/3839480602] up:boot 2024-06-10T00:27:54.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:53 smithi105 ceph-mon[36321]: fsmap 3 up:standby 2024-06-10T00:27:54.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:53 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T00:27:54.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:53 smithi062 ceph-mon[32390]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:54.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:54.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:54.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:54.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:54.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:54.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:54.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:53 smithi062 ceph-mon[32390]: Deploying daemon mds.d on smithi062 2024-06-10T00:27:54.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:53 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.62:6834/3839480602,v1:172.21.15.62:6835/3839480602] up:boot 2024-06-10T00:27:54.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:53 smithi062 ceph-mon[32390]: fsmap 3 up:standby 2024-06-10T00:27:54.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:53 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T00:27:54.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:53 smithi138 ceph-mon[36549]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:54.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:54.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:54.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:54.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:54.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:54.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:54.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:53 smithi138 ceph-mon[36549]: Deploying daemon mds.d on smithi062 2024-06-10T00:27:54.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:53 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.62:6834/3839480602,v1:172.21.15.62:6835/3839480602] up:boot 2024-06-10T00:27:54.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:53 smithi138 ceph-mon[36549]: fsmap 3 up:standby 2024-06-10T00:27:54.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:53 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T00:27:54.127 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:27:54.557 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:27:55.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:55 smithi062 ceph-mon[32390]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:56.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:55 smithi105 ceph-mon[36321]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:56.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:55 smithi138 ceph-mon[36549]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.640+0000 7f2054055700 1 -- 172.21.15.62:0/2356525351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06aac0 msgr2=0x7f204c10ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.640+0000 7f2054055700 1 --2- 172.21.15.62:0/2356525351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06aac0 0x7f204c10ea00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f203c009990 tx=0x7f203c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:27:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.640+0000 7f2054055700 1 -- 172.21.15.62:0/2356525351 shutdown_connections 2024-06-10T00:27:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.640+0000 7f2054055700 1 --2- 172.21.15.62:0/2356525351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f204c10ef40 0x7f204c111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.640+0000 7f2054055700 1 --2- 172.21.15.62:0/2356525351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f204c06a1a0 0x7f204c06a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.640+0000 7f2054055700 1 --2- 172.21.15.62:0/2356525351 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06aac0 0x7f204c10ea00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.640+0000 7f2054055700 1 -- 172.21.15.62:0/2356525351 >> 172.21.15.62:0/2356525351 conn(0x7f204c078340 msgr2=0x7f204c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.640+0000 7f2054055700 1 -- 172.21.15.62:0/2356525351 shutdown_connections 2024-06-10T00:27:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.640+0000 7f2054055700 1 -- 172.21.15.62:0/2356525351 wait complete. 2024-06-10T00:27:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.641+0000 7f2054055700 1 Processor -- start 2024-06-10T00:27:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.641+0000 7f2054055700 1 -- start start 2024-06-10T00:27:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.641+0000 7f2054055700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06a1a0 0x7f204c19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:56.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.641+0000 7f2054055700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f204c06aac0 0x7f204c19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.641+0000 7f2054055700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f204c10ef40 0x7f204c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.641+0000 7f2054055700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1149a0 con 0x7f204c06a1a0 2024-06-10T00:27:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.641+0000 7f2054055700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f204c114820 con 0x7f204c10ef40 2024-06-10T00:27:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.641+0000 7f2054055700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f204c114b20 con 0x7f204c06aac0 2024-06-10T00:27:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.641+0000 7f2051df1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06a1a0 0x7f204c19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.641+0000 7f2051df1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06a1a0 0x7f204c19ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49664/0 (socket says 172.21.15.62:49664) 2024-06-10T00:27:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.641+0000 7f2051df1700 1 -- 172.21.15.62:0/1126664969 learned_addr learned my addr 172.21.15.62:0/1126664969 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:27:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.641+0000 7f20515f0700 1 --2- 172.21.15.62:0/1126664969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f204c06aac0 0x7f204c19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.641+0000 7f20525f2700 1 --2- 172.21.15.62:0/1126664969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f204c10ef40 0x7f204c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.642+0000 7f2051df1700 1 -- 172.21.15.62:0/1126664969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f204c06aac0 msgr2=0x7f204c19b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.642+0000 7f2051df1700 1 --2- 172.21.15.62:0/1126664969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f204c06aac0 0x7f204c19b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.642+0000 7f2051df1700 1 -- 172.21.15.62:0/1126664969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f204c10ef40 msgr2=0x7f204c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.642+0000 7f2051df1700 1 --2- 172.21.15.62:0/1126664969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f204c10ef40 0x7f204c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.642+0000 7f2051df1700 1 -- 172.21.15.62:0/1126664969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c19fea0 con 0x7f204c06a1a0 2024-06-10T00:27:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.642+0000 7f2051df1700 1 --2- 172.21.15.62:0/1126664969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06a1a0 0x7f204c19ae90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f204800d950 tx=0x7f204800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.642+0000 7f2042ffd700 1 -- 172.21.15.62:0/1126664969 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048016070 con 0x7f204c06a1a0 2024-06-10T00:27:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.642+0000 7f2054055700 1 -- 172.21.15.62:0/1126664969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204c1a0190 con 0x7f204c06a1a0 2024-06-10T00:27:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.642+0000 7f2042ffd700 1 -- 172.21.15.62:0/1126664969 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f204800fd70 con 0x7f204c06a1a0 2024-06-10T00:27:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.642+0000 7f2042ffd700 1 -- 172.21.15.62:0/1126664969 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204800e610 con 0x7f204c06a1a0 2024-06-10T00:27:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.642+0000 7f2054055700 1 -- 172.21.15.62:0/1126664969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204c1a7a10 con 0x7f204c06a1a0 2024-06-10T00:27:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.643+0000 7f2054055700 1 -- 172.21.15.62:0/1126664969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f204c069180 con 0x7f204c06a1a0 2024-06-10T00:27:56.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.644+0000 7f2042ffd700 1 -- 172.21.15.62:0/1126664969 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2048019070 con 0x7f204c06a1a0 2024-06-10T00:27:56.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.645+0000 7f2042ffd700 1 --2- 172.21.15.62:0/1126664969 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f20380a2c20 0x7f20380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:27:56.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.645+0000 7f20515f0700 1 --2- 172.21.15.62:0/1126664969 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f20380a2c20 0x7f20380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:27:56.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.645+0000 7f2042ffd700 1 -- 172.21.15.62:0/1126664969 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f20480ce630 con 0x7f204c06a1a0 2024-06-10T00:27:56.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.646+0000 7f20515f0700 1 --2- 172.21.15.62:0/1126664969 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f20380a2c20 0x7f20380a50e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f203c00b5c0 tx=0x7f203c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:27:56.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.649+0000 7f2042ffd700 1 -- 172.21.15.62:0/1126664969 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f204809ba30 con 0x7f204c06a1a0 2024-06-10T00:27:56.818 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jun 10 00:27:56 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-d[72991]: starting mds.d at 2024-06-10T00:27:56.819 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jun 10 00:27:56 smithi062 bash[72969]: 6b93b2f244ccd884db725276fbe824c217ab39cfa0ff1ce83e6ec2e92bd4c9af 2024-06-10T00:27:56.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.904+0000 7f2054055700 1 -- 172.21.15.62:0/1126664969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f204c02dca0 con 0x7f204c06a1a0 2024-06-10T00:27:56.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.904+0000 7f2042ffd700 1 -- 172.21.15.62:0/1126664969 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f204809b330 con 0x7f204c06a1a0 2024-06-10T00:27:56.904 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:27:56.904 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-10T00:27:56.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.907+0000 7f2054055700 1 -- 172.21.15.62:0/1126664969 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f20380a2c20 msgr2=0x7f20380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:56.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.907+0000 7f2054055700 1 --2- 172.21.15.62:0/1126664969 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f20380a2c20 0x7f20380a50e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f203c00b5c0 tx=0x7f203c005960 comp rx=0 tx=0).stop 2024-06-10T00:27:56.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.907+0000 7f2054055700 1 -- 172.21.15.62:0/1126664969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06a1a0 msgr2=0x7f204c19ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:27:56.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.907+0000 7f2054055700 1 --2- 172.21.15.62:0/1126664969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06a1a0 0x7f204c19ae90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f204800d950 tx=0x7f204800dc60 comp rx=0 tx=0).stop 2024-06-10T00:27:56.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.907+0000 7f2054055700 1 -- 172.21.15.62:0/1126664969 shutdown_connections 2024-06-10T00:27:56.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.907+0000 7f2054055700 1 --2- 172.21.15.62:0/1126664969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f204c06a1a0 0x7f204c19ae90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:56.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.907+0000 7f2054055700 1 --2- 172.21.15.62:0/1126664969 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f20380a2c20 0x7f20380a50e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:56.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.907+0000 7f2054055700 1 --2- 172.21.15.62:0/1126664969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f204c10ef40 0x7f204c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:56.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.907+0000 7f2054055700 1 --2- 172.21.15.62:0/1126664969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f204c06aac0 0x7f204c19b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:27:56.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.907+0000 7f2054055700 1 -- 172.21.15.62:0/1126664969 >> 172.21.15.62:0/1126664969 conn(0x7f204c078340 msgr2=0x7f204c1103f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:27:56.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.907+0000 7f2054055700 1 -- 172.21.15.62:0/1126664969 shutdown_connections 2024-06-10T00:27:56.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:27:56.907+0000 7f2054055700 1 -- 172.21.15.62:0/1126664969 wait complete. 2024-06-10T00:27:56.908 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 4 2024-06-10T00:27:57.107 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jun 10 00:27:56 smithi062 systemd[1]: Started Ceph mds.d for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:27:57.810 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:57 smithi105 ceph-mon[36321]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:57.810 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:57 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:57.810 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:57 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:57.810 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:57 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1126664969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:27:57.810 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:57 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:57.811 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:57 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:57.811 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:57 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:57.811 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:57 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:58.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:57 smithi062 ceph-mon[32390]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:58.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:57 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:58.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:57 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:58.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:57 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1126664969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:27:58.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:57 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:58.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:57 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:58.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:57 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:58.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:57 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:58.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:57 smithi138 ceph-mon[36549]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:27:58.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:57 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:58.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:57 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:58.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:57 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1126664969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:27:58.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:57 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:27:58.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:57 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:27:58.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:57 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:27:58.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:57 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:27:58.985 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:58 smithi105 ceph-mon[36321]: Deploying daemon mds.e on smithi105 2024-06-10T00:27:58.985 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:58 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.62:6836/147630858,v1:172.21.15.62:6837/147630858] up:boot 2024-06-10T00:27:58.985 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:58 smithi105 ceph-mon[36321]: fsmap 4 up:standby 2024-06-10T00:27:58.985 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:58 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T00:27:59.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:58 smithi062 ceph-mon[32390]: Deploying daemon mds.e on smithi105 2024-06-10T00:27:59.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:58 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.62:6836/147630858,v1:172.21.15.62:6837/147630858] up:boot 2024-06-10T00:27:59.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:58 smithi062 ceph-mon[32390]: fsmap 4 up:standby 2024-06-10T00:27:59.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:58 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T00:27:59.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:58 smithi138 ceph-mon[36549]: Deploying daemon mds.e on smithi105 2024-06-10T00:27:59.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:58 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.62:6836/147630858,v1:172.21.15.62:6837/147630858] up:boot 2024-06-10T00:27:59.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:58 smithi138 ceph-mon[36549]: fsmap 4 up:standby 2024-06-10T00:27:59.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:58 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T00:27:59.282 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:27:59 smithi105 systemd[1]: Starting Ceph mds.e for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:27:59.636 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:27:59 smithi105 podman[66217]: 2024-06-10 00:27:59.28870312 +0000 UTC m=+0.017368654 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:27:59.636 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:27:59 smithi105 podman[66217]: 2024-06-10 00:27:59.40960174 +0000 UTC m=+0.138267274 container create 209bf3ad6388f996d1de1666b3b0b1bde4e3a3932582756fab0cbaabe6af47bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:27:59.636 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:27:59 smithi105 podman[66217]: 2024-06-10 00:27:59.637393502 +0000 UTC m=+0.366059040 container init 209bf3ad6388f996d1de1666b3b0b1bde4e3a3932582756fab0cbaabe6af47bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-10T00:27:59.638 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:27:59.906 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:00.028 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:27:59 smithi062 ceph-mon[32390]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:00.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:27:59 smithi105 ceph-mon[36321]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:00.032 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:27:59 smithi105 podman[66217]: 2024-06-10 00:27:59.64001356 +0000 UTC m=+0.368679094 container start 209bf3ad6388f996d1de1666b3b0b1bde4e3a3932582756fab0cbaabe6af47bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-10T00:28:00.032 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:27:59 smithi105 ceph-mds[66236]: -- 172.21.15.105:0/3731333740 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55600bd83a40 con 0x55600bd56c00 2024-06-10T00:28:00.032 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:27:59 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-e[66232]: starting mds.e at 2024-06-10T00:28:00.032 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:27:59 smithi105 bash[66217]: 209bf3ad6388f996d1de1666b3b0b1bde4e3a3932582756fab0cbaabe6af47bf 2024-06-10T00:28:00.033 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:27:59 smithi105 systemd[1]: Started Ceph mds.e for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:28:00.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:27:59 smithi138 ceph-mon[36549]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:00.776 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:00 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.105:6834/588077925,v1:172.21.15.105:6835/588077925] up:boot 2024-06-10T00:28:00.776 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:00 smithi138 ceph-mon[36549]: fsmap 5 up:standby 2024-06-10T00:28:00.776 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:00 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T00:28:00.776 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:00 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:00.776 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:00 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:00.776 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:00 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:00.776 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:00 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:00.776 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:00 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:00.777 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:00 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:01.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:00 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.105:6834/588077925,v1:172.21.15.105:6835/588077925] up:boot 2024-06-10T00:28:01.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:00 smithi105 ceph-mon[36321]: fsmap 5 up:standby 2024-06-10T00:28:01.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:00 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T00:28:01.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:00 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:01.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:00 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:01.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:00 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:01.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:00 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:01.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:00 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:01.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:00 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:01.044 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:00 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.105:6834/588077925,v1:172.21.15.105:6835/588077925] up:boot 2024-06-10T00:28:01.044 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:00 smithi062 ceph-mon[32390]: fsmap 5 up:standby 2024-06-10T00:28:01.044 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:00 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T00:28:01.044 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:00 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:01.044 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:00 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:01.044 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:00 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:01.044 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:00 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:01.045 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:00 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:01.045 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:00 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:01.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.684+0000 7fbe8d554700 1 -- 172.21.15.62:0/1211133299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe88108580 msgr2=0x7fbe88108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:01.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.684+0000 7fbe8d554700 1 --2- 172.21.15.62:0/1211133299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe88108580 0x7fbe88108960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009920 tx=0x7fbe78009c30 comp rx=0 tx=0).stop 2024-06-10T00:28:01.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.684+0000 7fbe8d554700 1 -- 172.21.15.62:0/1211133299 shutdown_connections 2024-06-10T00:28:01.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.684+0000 7fbe8d554700 1 --2- 172.21.15.62:0/1211133299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe88108580 0x7fbe88108960 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:01.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.684+0000 7fbe8d554700 1 --2- 172.21.15.62:0/1211133299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8806b620 0x7fbe8810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:01.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.684+0000 7fbe8d554700 1 --2- 172.21.15.62:0/1211133299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe8806ac60 0x7fbe8806b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:01.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.684+0000 7fbe8d554700 1 -- 172.21.15.62:0/1211133299 >> 172.21.15.62:0/1211133299 conn(0x7fbe88078430 msgr2=0x7fbe88078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:01.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.684+0000 7fbe8d554700 1 -- 172.21.15.62:0/1211133299 shutdown_connections 2024-06-10T00:28:01.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.684+0000 7fbe8d554700 1 -- 172.21.15.62:0/1211133299 wait complete. 2024-06-10T00:28:01.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.685+0000 7fbe8d554700 1 Processor -- start 2024-06-10T00:28:01.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.685+0000 7fbe8d554700 1 -- start start 2024-06-10T00:28:01.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.685+0000 7fbe8d554700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8806ac60 0x7fbe8819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:01.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.685+0000 7fbe8d554700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe8806b620 0x7fbe8819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:01.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.685+0000 7fbe8d554700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe88108580 0x7fbe88198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:01.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.685+0000 7fbe8d554700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbe881127c0 con 0x7fbe88108580 2024-06-10T00:28:01.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.685+0000 7fbe8d554700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbe88112640 con 0x7fbe8806ac60 2024-06-10T00:28:01.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.685+0000 7fbe8d554700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe88112940 con 0x7fbe8806b620 2024-06-10T00:28:01.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.685+0000 7fbe877fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe88108580 0x7fbe88198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:01.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.685+0000 7fbe867fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe8806b620 0x7fbe8819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:01.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.685+0000 7fbe877fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe88108580 0x7fbe88198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40682/0 (socket says 172.21.15.62:40682) 2024-06-10T00:28:01.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.685+0000 7fbe877fe700 1 -- 172.21.15.62:0/1694706759 learned_addr learned my addr 172.21.15.62:0/1694706759 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:01.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.685+0000 7fbe86ffd700 1 --2- 172.21.15.62:0/1694706759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8806ac60 0x7fbe8819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:01.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.686+0000 7fbe877fe700 1 -- 172.21.15.62:0/1694706759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe8806b620 msgr2=0x7fbe8819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:01.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.686+0000 7fbe877fe700 1 --2- 172.21.15.62:0/1694706759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe8806b620 0x7fbe8819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:01.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.686+0000 7fbe877fe700 1 -- 172.21.15.62:0/1694706759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8806ac60 msgr2=0x7fbe8819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:01.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.686+0000 7fbe877fe700 1 --2- 172.21.15.62:0/1694706759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8806ac60 0x7fbe8819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:01.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.686+0000 7fbe877fe700 1 -- 172.21.15.62:0/1694706759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe88199260 con 0x7fbe88108580 2024-06-10T00:28:01.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.686+0000 7fbe867fc700 1 --2- 172.21.15.62:0/1694706759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe8806b620 0x7fbe8819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:28:01.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.686+0000 7fbe877fe700 1 --2- 172.21.15.62:0/1694706759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe88108580 0x7fbe88198b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe80007ee0 tx=0x7fbe8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:01.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.686+0000 7fbe6ffff700 1 -- 172.21.15.62:0/1694706759 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8000dd20 con 0x7fbe88108580 2024-06-10T00:28:01.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.686+0000 7fbe8d554700 1 -- 172.21.15.62:0/1694706759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe88199550 con 0x7fbe88108580 2024-06-10T00:28:01.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.686+0000 7fbe6ffff700 1 -- 172.21.15.62:0/1694706759 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe80004020 con 0x7fbe88108580 2024-06-10T00:28:01.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.686+0000 7fbe6ffff700 1 -- 172.21.15.62:0/1694706759 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80015660 con 0x7fbe88108580 2024-06-10T00:28:01.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.686+0000 7fbe8d554700 1 -- 172.21.15.62:0/1694706759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe88101330 con 0x7fbe88108580 2024-06-10T00:28:01.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.687+0000 7fbe8d554700 1 -- 172.21.15.62:0/1694706759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe8810aa10 con 0x7fbe88108580 2024-06-10T00:28:01.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.688+0000 7fbe6ffff700 1 -- 172.21.15.62:0/1694706759 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbe800041d0 con 0x7fbe88108580 2024-06-10T00:28:01.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.689+0000 7fbe6ffff700 1 --2- 172.21.15.62:0/1694706759 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbe700a2c20 0x7fbe700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:01.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.689+0000 7fbe86ffd700 1 --2- 172.21.15.62:0/1694706759 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbe700a2c20 0x7fbe700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:01.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.689+0000 7fbe6ffff700 1 -- 172.21.15.62:0/1694706759 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fbe800ce500 con 0x7fbe88108580 2024-06-10T00:28:01.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.690+0000 7fbe86ffd700 1 --2- 172.21.15.62:0/1694706759 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbe700a2c20 0x7fbe700a50e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800c010 tx=0x7fbe7801d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:01.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.694+0000 7fbe6ffff700 1 -- 172.21.15.62:0/1694706759 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbe80010070 con 0x7fbe88108580 2024-06-10T00:28:01.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:01 smithi062 ceph-mon[32390]: Deploying daemon mds.f on smithi138 2024-06-10T00:28:01.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:01 smithi062 ceph-mon[32390]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:01.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.944+0000 7fbe8d554700 1 -- 172.21.15.62:0/1694706759 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe88199ff0 con 0x7fbe88108580 2024-06-10T00:28:01.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.945+0000 7fbe6ffff700 1 -- 172.21.15.62:0/1694706759 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7fbe8009b900 con 0x7fbe88108580 2024-06-10T00:28:01.945 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:28:01.945 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-06-10T00:28:01.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.947+0000 7fbe8d554700 1 -- 172.21.15.62:0/1694706759 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbe700a2c20 msgr2=0x7fbe700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.947+0000 7fbe8d554700 1 --2- 172.21.15.62:0/1694706759 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbe700a2c20 0x7fbe700a50e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800c010 tx=0x7fbe7801d040 comp rx=0 tx=0).stop 2024-06-10T00:28:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.947+0000 7fbe8d554700 1 -- 172.21.15.62:0/1694706759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe88108580 msgr2=0x7fbe88198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.947+0000 7fbe8d554700 1 --2- 172.21.15.62:0/1694706759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe88108580 0x7fbe88198b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe80007ee0 tx=0x7fbe8000d3b0 comp rx=0 tx=0).stop 2024-06-10T00:28:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.947+0000 7fbe8d554700 1 -- 172.21.15.62:0/1694706759 shutdown_connections 2024-06-10T00:28:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.947+0000 7fbe8d554700 1 --2- 172.21.15.62:0/1694706759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe88108580 0x7fbe88198b00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.947+0000 7fbe8d554700 1 --2- 172.21.15.62:0/1694706759 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbe700a2c20 0x7fbe700a50e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.947+0000 7fbe8d554700 1 --2- 172.21.15.62:0/1694706759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe8806ac60 0x7fbe8819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.947+0000 7fbe8d554700 1 --2- 172.21.15.62:0/1694706759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe8806b620 0x7fbe8819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.947+0000 7fbe8d554700 1 -- 172.21.15.62:0/1694706759 >> 172.21.15.62:0/1694706759 conn(0x7fbe88078430 msgr2=0x7fbe880fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.947+0000 7fbe8d554700 1 -- 172.21.15.62:0/1694706759 shutdown_connections 2024-06-10T00:28:01.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:01.947+0000 7fbe8d554700 1 -- 172.21.15.62:0/1694706759 wait complete. 2024-06-10T00:28:01.949 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 6 2024-06-10T00:28:02.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:01 smithi105 ceph-mon[36321]: Deploying daemon mds.f on smithi138 2024-06-10T00:28:02.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:01 smithi105 ceph-mon[36321]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:02.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:01 smithi138 ceph-mon[36549]: Deploying daemon mds.f on smithi138 2024-06-10T00:28:02.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:01 smithi138 ceph-mon[36549]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:02.124 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:28:01 smithi138 systemd[1]: Starting Ceph mds.f for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:28:02.386 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:28:02 smithi138 podman[66272]: 2024-06-10 00:28:02.033959328 +0000 UTC m=+0.017032541 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:28:02.386 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:28:02 smithi138 podman[66272]: 2024-06-10 00:28:02.159028243 +0000 UTC m=+0.142101432 container create 41fcb2b3c9be39ecf2ab4f486a0d0ba2e3430b8cb056e5d8cb6c9a6e56789686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-10T00:28:02.386 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:28:02 smithi138 podman[66272]: 2024-06-10 00:28:02.386667687 +0000 UTC m=+0.369740887 container init 41fcb2b3c9be39ecf2ab4f486a0d0ba2e3430b8cb056e5d8cb6c9a6e56789686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-10T00:28:02.702 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:28:02 smithi138 podman[66272]: 2024-06-10 00:28:02.389703229 +0000 UTC m=+0.372776424 container start 41fcb2b3c9be39ecf2ab4f486a0d0ba2e3430b8cb056e5d8cb6c9a6e56789686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:28:02.702 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:28:02 smithi138 ceph-mds[66292]: -- 172.21.15.138:0/2215495655 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a6dba95a40 con 0x55a6dba68c00 2024-06-10T00:28:02.703 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:28:02 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-f[66287]: starting mds.f at 2024-06-10T00:28:02.703 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:28:02 smithi138 bash[66272]: 41fcb2b3c9be39ecf2ab4f486a0d0ba2e3430b8cb056e5d8cb6c9a6e56789686 2024-06-10T00:28:02.703 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:28:02 smithi138 systemd[1]: Started Ceph mds.f for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:28:02.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:02 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1694706759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:02.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:02 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:02.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:02 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:02.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:02 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:02.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:02 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:02.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:02 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:02.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:02 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:03.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:02 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1694706759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:03.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:02 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:03.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:02 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:03.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:02 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:03.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:02 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:03.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:02 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:03.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:02 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:03.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:02 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1694706759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:03.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:02 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:03.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:02 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:03.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:02 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:03.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:02 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:03.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:02 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:03.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:02 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:03 smithi138 ceph-mon[36549]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:03 smithi138 ceph-mon[36549]: Deploying daemon mds.c on smithi138 2024-06-10T00:28:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:03 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] up:boot 2024-06-10T00:28:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:03 smithi138 ceph-mon[36549]: fsmap 6 up:standby 2024-06-10T00:28:03.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:03 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T00:28:04.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:03 smithi105 ceph-mon[36321]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:04.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:03 smithi105 ceph-mon[36321]: Deploying daemon mds.c on smithi138 2024-06-10T00:28:04.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:03 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] up:boot 2024-06-10T00:28:04.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:03 smithi105 ceph-mon[36321]: fsmap 6 up:standby 2024-06-10T00:28:04.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:03 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T00:28:04.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:03 smithi062 ceph-mon[32390]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:04.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:03 smithi062 ceph-mon[32390]: Deploying daemon mds.c on smithi138 2024-06-10T00:28:04.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:03 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] up:boot 2024-06-10T00:28:04.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:03 smithi062 ceph-mon[32390]: fsmap 6 up:standby 2024-06-10T00:28:04.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:03 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T00:28:04.613 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:28:04.874 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:05.043 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:28:04 smithi138 systemd[1]: Starting Ceph mds.c for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:28:05.318 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:28:05 smithi138 podman[66585]: 2024-06-10 00:28:04.945890689 +0000 UTC m=+0.017249349 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:28:05.318 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:28:05 smithi138 podman[66585]: 2024-06-10 00:28:05.066489968 +0000 UTC m=+0.137848615 container create c3949f77fe3fbff4bbc66bb34e201f4e15df3d6b8c5c1f03fa7f91fe4eb37036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-10T00:28:05.318 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:28:05 smithi138 podman[66585]: 2024-06-10 00:28:05.319236385 +0000 UTC m=+0.390595038 container init c3949f77fe3fbff4bbc66bb34e201f4e15df3d6b8c5c1f03fa7f91fe4eb37036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-10T00:28:05.623 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:28:05 smithi138 podman[66585]: 2024-06-10 00:28:05.321853519 +0000 UTC m=+0.393212171 container start c3949f77fe3fbff4bbc66bb34e201f4e15df3d6b8c5c1f03fa7f91fe4eb37036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True) 2024-06-10T00:28:05.623 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:28:05 smithi138 ceph-mds[66605]: -- 172.21.15.138:0/833814404 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555e3a9e5a40 con 0x555e3a9b9000 2024-06-10T00:28:05.623 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:28:05 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-c[66601]: starting mds.c at 2024-06-10T00:28:05.623 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:28:05 smithi138 bash[66585]: c3949f77fe3fbff4bbc66bb34e201f4e15df3d6b8c5c1f03fa7f91fe4eb37036 2024-06-10T00:28:05.623 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:28:05 smithi138 systemd[1]: Started Ceph mds.c for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:28:06.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:05 smithi062 ceph-mon[32390]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:06.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:06.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:06.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:06.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:06.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:06.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:05 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:06.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:05 smithi105 ceph-mon[36321]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:06.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:06.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:06.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:06.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:06.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:06.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:05 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:06.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:05 smithi138 ceph-mon[36549]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:06.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:06.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:06.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:06.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:06.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:06.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:05 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.426+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/1463127077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc06ab30 msgr2=0x7f3edc10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.426+0000 7f3ee2fbd700 1 --2- 172.21.15.62:0/1463127077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc06ab30 0x7f3edc10f320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400b0a0 tx=0x7f3ed400b3b0 comp rx=0 tx=0).stop 2024-06-10T00:28:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.426+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/1463127077 shutdown_connections 2024-06-10T00:28:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.426+0000 7f3ee2fbd700 1 --2- 172.21.15.62:0/1463127077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc06ab30 0x7f3edc10f320 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.426+0000 7f3ee2fbd700 1 --2- 172.21.15.62:0/1463127077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3edc108500 0x7f3edc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.426+0000 7f3ee2fbd700 1 --2- 172.21.15.62:0/1463127077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3edc06a170 0x7f3edc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.426+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/1463127077 >> 172.21.15.62:0/1463127077 conn(0x7f3edc078340 msgr2=0x7f3edc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.426+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/1463127077 shutdown_connections 2024-06-10T00:28:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.426+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/1463127077 wait complete. 2024-06-10T00:28:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.426+0000 7f3ee2fbd700 1 Processor -- start 2024-06-10T00:28:06.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.426+0000 7f3ee2fbd700 1 -- start start 2024-06-10T00:28:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.426+0000 7f3ee2fbd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3edc06a170 0x7f3edc19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.426+0000 7f3ee2fbd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3edc06ab30 0x7f3edc19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.426+0000 7f3ee2fbd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc108500 0x7f3edc198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee2fbd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc112860 con 0x7f3edc108500 2024-06-10T00:28:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee2fbd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc1126e0 con 0x7f3edc06a170 2024-06-10T00:28:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee2fbd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc1129e0 con 0x7f3edc06ab30 2024-06-10T00:28:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee155a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc108500 0x7f3edc198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee155a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc108500 0x7f3edc198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40702/0 (socket says 172.21.15.62:40702) 2024-06-10T00:28:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee155a700 1 -- 172.21.15.62:0/2399176561 learned_addr learned my addr 172.21.15.62:0/2399176561 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee0d59700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3edc06a170 0x7f3edc19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3edbfff700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3edc06ab30 0x7f3edc19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee155a700 1 -- 172.21.15.62:0/2399176561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3edc06ab30 msgr2=0x7f3edc19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee155a700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3edc06ab30 0x7f3edc19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee155a700 1 -- 172.21.15.62:0/2399176561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3edc06a170 msgr2=0x7f3edc19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee155a700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3edc06a170 0x7f3edc19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee155a700 1 -- 172.21.15.62:0/2399176561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc199240 con 0x7f3edc108500 2024-06-10T00:28:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3edbfff700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3edc06ab30 0x7f3edc19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:28:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee155a700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc108500 0x7f3edc198ae0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40062a0 tx=0x7f3ed400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ed9ffb700 1 -- 172.21.15.62:0/2399176561 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4021070 con 0x7f3edc108500 2024-06-10T00:28:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/2399176561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3edc1994d0 con 0x7f3edc108500 2024-06-10T00:28:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ed9ffb700 1 -- 172.21.15.62:0/2399176561 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ed4007db0 con 0x7f3edc108500 2024-06-10T00:28:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ed9ffb700 1 -- 172.21.15.62:0/2399176561 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4005460 con 0x7f3edc108500 2024-06-10T00:28:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.427+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/2399176561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3edc1a57a0 con 0x7f3edc108500 2024-06-10T00:28:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.428+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/2399176561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3edc10a9e0 con 0x7f3edc108500 2024-06-10T00:28:06.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.430+0000 7f3ed9ffb700 1 -- 172.21.15.62:0/2399176561 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3ed4024070 con 0x7f3edc108500 2024-06-10T00:28:06.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.430+0000 7f3ed9ffb700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3ecc0a2c20 0x7f3ecc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:06.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.430+0000 7f3ee0d59700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3ecc0a2c20 0x7f3ecc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:06.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.430+0000 7f3ed9ffb700 1 -- 172.21.15.62:0/2399176561 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3ed400f020 con 0x7f3edc108500 2024-06-10T00:28:06.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.431+0000 7f3ee0d59700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3ecc0a2c20 0x7f3ecc0a50e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0006fd0 tx=0x7f3ed0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:06.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.435+0000 7f3ed9ffb700 1 -- 172.21.15.62:0/2399176561 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3ed402b020 con 0x7f3edc108500 2024-06-10T00:28:06.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.687+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/2399176561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3edc199fc0 con 0x7f3edc108500 2024-06-10T00:28:06.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.688+0000 7f3ed9ffb700 1 -- 172.21.15.62:0/2399176561 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f3ed409d840 con 0x7f3edc108500 2024-06-10T00:28:06.688 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:28:06.688 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-06-10T00:28:06.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.690+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/2399176561 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3ecc0a2c20 msgr2=0x7f3ecc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:06.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.690+0000 7f3ee2fbd700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3ecc0a2c20 0x7f3ecc0a50e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0006fd0 tx=0x7f3ed0008040 comp rx=0 tx=0).stop 2024-06-10T00:28:06.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.690+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/2399176561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc108500 msgr2=0x7f3edc198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:06.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.690+0000 7f3ee2fbd700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc108500 0x7f3edc198ae0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40062a0 tx=0x7f3ed400bf10 comp rx=0 tx=0).stop 2024-06-10T00:28:06.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.690+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/2399176561 shutdown_connections 2024-06-10T00:28:06.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.690+0000 7f3ee2fbd700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc108500 0x7f3edc198ae0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:06.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.690+0000 7f3ee2fbd700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3ecc0a2c20 0x7f3ecc0a50e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:06.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.690+0000 7f3ee2fbd700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3edc06a170 0x7f3edc19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:06.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.690+0000 7f3ee2fbd700 1 --2- 172.21.15.62:0/2399176561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3edc06ab30 0x7f3edc19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:06.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.690+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/2399176561 >> 172.21.15.62:0/2399176561 conn(0x7f3edc078340 msgr2=0x7f3edc0fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:06.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.690+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/2399176561 shutdown_connections 2024-06-10T00:28:06.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:06.690+0000 7f3ee2fbd700 1 -- 172.21.15.62:0/2399176561 wait complete. 2024-06-10T00:28:06.691 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 8 2024-06-10T00:28:07.011 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:06 smithi062 ceph-mon[32390]: Deploying daemon mds.h on smithi105 2024-06-10T00:28:07.011 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:06 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] up:boot 2024-06-10T00:28:07.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:06 smithi062 ceph-mon[32390]: fsmap 7 up:standby 2024-06-10T00:28:07.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:06 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T00:28:07.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:06 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2399176561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:07.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:06 smithi105 ceph-mon[36321]: Deploying daemon mds.h on smithi105 2024-06-10T00:28:07.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:06 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] up:boot 2024-06-10T00:28:07.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:06 smithi105 ceph-mon[36321]: fsmap 7 up:standby 2024-06-10T00:28:07.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:06 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T00:28:07.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:06 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2399176561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:07.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:06 smithi138 ceph-mon[36549]: Deploying daemon mds.h on smithi105 2024-06-10T00:28:07.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:06 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] up:boot 2024-06-10T00:28:07.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:06 smithi138 ceph-mon[36549]: fsmap 7 up:standby 2024-06-10T00:28:07.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:06 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T00:28:07.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:06 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2399176561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:08.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:07 smithi105 ceph-mon[36321]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:08.032 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:28:07 smithi105 podman[66531]: 2024-06-10 00:28:07.991032301 +0000 UTC m=+0.126814611 container create abed1b6864ba2aa29e7d5b401da20791ada0f97affa12f37addab1418c7ff5fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-h, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:28:08.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:07 smithi062 ceph-mon[32390]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:08.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:07 smithi138 ceph-mon[36549]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:08.532 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:28:08 smithi105 podman[66531]: 2024-06-10 00:28:08.193573291 +0000 UTC m=+0.329355573 container init abed1b6864ba2aa29e7d5b401da20791ada0f97affa12f37addab1418c7ff5fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-10T00:28:08.532 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:28:08 smithi105 podman[66531]: 2024-06-10 00:28:08.19602106 +0000 UTC m=+0.331803350 container start abed1b6864ba2aa29e7d5b401da20791ada0f97affa12f37addab1418c7ff5fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-h, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-10T00:28:08.532 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:28:08 smithi105 ceph-mds[66550]: -- 172.21.15.105:0/1634806204 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fbe2afba40 con 0x55fbe2acf000 2024-06-10T00:28:08.532 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:28:08 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-h[66546]: starting mds.h at 2024-06-10T00:28:08.532 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:28:08 smithi105 bash[66531]: abed1b6864ba2aa29e7d5b401da20791ada0f97affa12f37addab1418c7ff5fc 2024-06-10T00:28:08.532 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:28:08 smithi105 systemd[1]: Started Ceph mds.h for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:28:09.328 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:28:09.593 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:09 smithi062 ceph-mon[32390]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:09 smithi062 ceph-mon[32390]: Deploying daemon mds.i on smithi138 2024-06-10T00:28:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:09 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] up:boot 2024-06-10T00:28:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:09 smithi062 ceph-mon[32390]: fsmap 8 up:standby 2024-06-10T00:28:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:09 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T00:28:09.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:09 smithi138 ceph-mon[36549]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:09.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:09.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:09.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:09.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:09.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:09.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:09.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:09 smithi138 ceph-mon[36549]: Deploying daemon mds.i on smithi138 2024-06-10T00:28:09.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:09 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] up:boot 2024-06-10T00:28:09.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:09 smithi138 ceph-mon[36549]: fsmap 8 up:standby 2024-06-10T00:28:09.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:09 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T00:28:09.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:09 smithi105 ceph-mon[36321]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:09.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:09.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:09.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:09.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:09.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:09.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:09.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:09 smithi105 ceph-mon[36321]: Deploying daemon mds.i on smithi138 2024-06-10T00:28:09.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:09 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] up:boot 2024-06-10T00:28:09.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:09 smithi105 ceph-mon[36321]: fsmap 8 up:standby 2024-06-10T00:28:09.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:09 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T00:28:10.623 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:28:10 smithi138 systemd[1]: Starting Ceph mds.i for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:28:10.928 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:28:10 smithi138 podman[66888]: 2024-06-10 00:28:10.576119813 +0000 UTC m=+0.017302428 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:28:10.928 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:28:10 smithi138 podman[66888]: 2024-06-10 00:28:10.675063766 +0000 UTC m=+0.116246369 container create 13a4245f0167e1ef0ce5ab69fab8d64d6ab33954364b0ea0a59360fdc0300855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-i, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-10T00:28:10.928 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:28:10 smithi138 podman[66888]: 2024-06-10 00:28:10.928797749 +0000 UTC m=+0.369980351 container init 13a4245f0167e1ef0ce5ab69fab8d64d6ab33954364b0ea0a59360fdc0300855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-i, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:28:11.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.322+0000 7fc558665700 1 -- 172.21.15.62:0/3721184414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc550105330 msgr2=0x7fc550105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:11.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.322+0000 7fc558665700 1 --2- 172.21.15.62:0/3721184414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc550105330 0x7fc550105710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009920 tx=0x7fc54c009c30 comp rx=0 tx=0).stop 2024-06-10T00:28:11.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.322+0000 7fc558665700 1 -- 172.21.15.62:0/3721184414 shutdown_connections 2024-06-10T00:28:11.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.322+0000 7fc558665700 1 --2- 172.21.15.62:0/3721184414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc550105330 0x7fc550105710 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:11.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.322+0000 7fc558665700 1 --2- 172.21.15.62:0/3721184414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc550107b60 0x7fc55010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:11.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.322+0000 7fc558665700 1 --2- 172.21.15.62:0/3721184414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5501071a0 0x7fc550107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.322+0000 7fc558665700 1 -- 172.21.15.62:0/3721184414 >> 172.21.15.62:0/3721184414 conn(0x7fc5500766b0 msgr2=0x7fc550076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.322+0000 7fc558665700 1 -- 172.21.15.62:0/3721184414 shutdown_connections 2024-06-10T00:28:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.322+0000 7fc558665700 1 -- 172.21.15.62:0/3721184414 wait complete. 2024-06-10T00:28:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc558665700 1 Processor -- start 2024-06-10T00:28:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc558665700 1 -- start start 2024-06-10T00:28:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc558665700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc550105330 0x7fc55019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc558665700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5501071a0 0x7fc55019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc558665700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc550107b60 0x7fc550198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc558665700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc550111780 con 0x7fc5501071a0 2024-06-10T00:28:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc558665700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc550111600 con 0x7fc550107b60 2024-06-10T00:28:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc558665700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc550111900 con 0x7fc550105330 2024-06-10T00:28:11.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc555c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5501071a0 0x7fc55019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc555c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5501071a0 0x7fc55019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59146/0 (socket says 172.21.15.62:59146) 2024-06-10T00:28:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc555c00700 1 -- 172.21.15.62:0/2287466573 learned_addr learned my addr 172.21.15.62:0/2287466573 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc556401700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc550105330 0x7fc55019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc556c02700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc550107b60 0x7fc550198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc555c00700 1 -- 172.21.15.62:0/2287466573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc550105330 msgr2=0x7fc55019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc555c00700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc550105330 0x7fc55019ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc555c00700 1 -- 172.21.15.62:0/2287466573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc550107b60 msgr2=0x7fc550198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc555c00700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc550107b60 0x7fc550198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc555c00700 1 -- 172.21.15.62:0/2287466573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc550199410 con 0x7fc5501071a0 2024-06-10T00:28:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.323+0000 7fc556c02700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc550107b60 0x7fc550198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:28:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.324+0000 7fc555c00700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5501071a0 0x7fc55019efa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc54000e9c0 tx=0x7fc54000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:11.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.324+0000 7fc5477fe700 1 -- 172.21.15.62:0/2287466573 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54000cbd0 con 0x7fc5501071a0 2024-06-10T00:28:11.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.324+0000 7fc5477fe700 1 -- 172.21.15.62:0/2287466573 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc54000cd70 con 0x7fc5501071a0 2024-06-10T00:28:11.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.324+0000 7fc5477fe700 1 -- 172.21.15.62:0/2287466573 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540005660 con 0x7fc5501071a0 2024-06-10T00:28:11.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.324+0000 7fc558665700 1 -- 172.21.15.62:0/2287466573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc550199700 con 0x7fc5501071a0 2024-06-10T00:28:11.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.324+0000 7fc558665700 1 -- 172.21.15.62:0/2287466573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5501a59e0 con 0x7fc5501071a0 2024-06-10T00:28:11.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.325+0000 7fc558665700 1 -- 172.21.15.62:0/2287466573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc55006a780 con 0x7fc5501071a0 2024-06-10T00:28:11.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.326+0000 7fc5477fe700 1 -- 172.21.15.62:0/2287466573 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc540004750 con 0x7fc5501071a0 2024-06-10T00:28:11.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.327+0000 7fc5477fe700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc53c0a2ca0 0x7fc53c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:11.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.327+0000 7fc556401700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc53c0a2ca0 0x7fc53c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:11.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.327+0000 7fc5477fe700 1 -- 172.21.15.62:0/2287466573 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc5400cef30 con 0x7fc5501071a0 2024-06-10T00:28:11.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.328+0000 7fc556401700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc53c0a2ca0 0x7fc53c0a5160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009810 tx=0x7fc54c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:11.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.331+0000 7fc5477fe700 1 -- 172.21.15.62:0/2287466573 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc540053850 con 0x7fc5501071a0 2024-06-10T00:28:11.344 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:28:10 smithi138 podman[66888]: 2024-06-10 00:28:10.931413261 +0000 UTC m=+0.372595876 container start 13a4245f0167e1ef0ce5ab69fab8d64d6ab33954364b0ea0a59360fdc0300855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-i, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-06-10T00:28:11.344 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:28:10 smithi138 ceph-mds[66907]: -- 172.21.15.138:0/181952371 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c71fad7a40 con 0x55c71faaac00 2024-06-10T00:28:11.345 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:28:10 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-i[66903]: starting mds.i at 2024-06-10T00:28:11.345 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:28:11 smithi138 bash[66888]: 13a4245f0167e1ef0ce5ab69fab8d64d6ab33954364b0ea0a59360fdc0300855 2024-06-10T00:28:11.345 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:28:11 smithi138 systemd[1]: Started Ceph mds.i for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:28:11.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.585+0000 7fc558665700 1 -- 172.21.15.62:0/2287466573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc550199ff0 con 0x7fc5501071a0 2024-06-10T00:28:11.585 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:11 smithi062 ceph-mon[32390]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:11.585 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:11.585 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:11.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:11.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:11.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:11.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:11 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:11.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.586+0000 7fc5477fe700 1 -- 172.21.15.62:0/2287466573 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7fc54009c290 con 0x7fc5501071a0 2024-06-10T00:28:11.587 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:28:11.587 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-10T00:28:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.589+0000 7fc558665700 1 -- 172.21.15.62:0/2287466573 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc53c0a2ca0 msgr2=0x7fc53c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.589+0000 7fc558665700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc53c0a2ca0 0x7fc53c0a5160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009810 tx=0x7fc54c019040 comp rx=0 tx=0).stop 2024-06-10T00:28:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.589+0000 7fc558665700 1 -- 172.21.15.62:0/2287466573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5501071a0 msgr2=0x7fc55019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.589+0000 7fc558665700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5501071a0 0x7fc55019efa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc54000e9c0 tx=0x7fc54000ecd0 comp rx=0 tx=0).stop 2024-06-10T00:28:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.589+0000 7fc558665700 1 -- 172.21.15.62:0/2287466573 shutdown_connections 2024-06-10T00:28:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.589+0000 7fc558665700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc53c0a2ca0 0x7fc53c0a5160 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.589+0000 7fc558665700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc550107b60 0x7fc550198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.589+0000 7fc558665700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc550105330 0x7fc55019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.589+0000 7fc558665700 1 --2- 172.21.15.62:0/2287466573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5501071a0 0x7fc55019efa0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.589+0000 7fc558665700 1 -- 172.21.15.62:0/2287466573 >> 172.21.15.62:0/2287466573 conn(0x7fc5500766b0 msgr2=0x7fc550101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.589+0000 7fc558665700 1 -- 172.21.15.62:0/2287466573 shutdown_connections 2024-06-10T00:28:11.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:11.589+0000 7fc558665700 1 -- 172.21.15.62:0/2287466573 wait complete. 2024-06-10T00:28:11.590 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 10 2024-06-10T00:28:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:11 smithi138 ceph-mon[36549]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:11 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:11.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:11 smithi105 ceph-mon[36321]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:11.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:11 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:11.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:11 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:11.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:11 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:11.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:11 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:11.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:11 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:11.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:11 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:12.550 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:12 smithi062 ceph-mon[32390]: Deploying daemon mds.g on smithi062 2024-06-10T00:28:12.551 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:12 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] up:boot 2024-06-10T00:28:12.551 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:12 smithi062 ceph-mon[32390]: fsmap 9 up:standby 2024-06-10T00:28:12.551 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:12 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T00:28:12.551 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:12 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2287466573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:12.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:12 smithi138 ceph-mon[36549]: Deploying daemon mds.g on smithi062 2024-06-10T00:28:12.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:12 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] up:boot 2024-06-10T00:28:12.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:12 smithi138 ceph-mon[36549]: fsmap 9 up:standby 2024-06-10T00:28:12.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:12 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T00:28:12.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:12 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2287466573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:12.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:12 smithi105 ceph-mon[36321]: Deploying daemon mds.g on smithi062 2024-06-10T00:28:12.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:12 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] up:boot 2024-06-10T00:28:12.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:12 smithi105 ceph-mon[36321]: fsmap 9 up:standby 2024-06-10T00:28:12.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:12 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T00:28:12.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:12 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2287466573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:13.408 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:13 smithi062 ceph-mon[32390]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:13 smithi138 ceph-mon[36549]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:13.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:13 smithi105 ceph-mon[36321]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:14.107 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:28:13 smithi062 systemd[1]: Starting Ceph mds.g for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:28:14.287 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:28:14.372 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:28:14 smithi062 podman[73801]: 2024-06-10 00:28:14.017045689 +0000 UTC m=+0.017868007 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:28:14.372 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:28:14 smithi062 podman[73801]: 2024-06-10 00:28:14.136076678 +0000 UTC m=+0.136898992 container create 55f780cf7a1159b766eff059db86e38b85f874034c874817bd49ec9ca92af0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-g, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-10T00:28:14.372 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:28:14 smithi062 podman[73801]: 2024-06-10 00:28:14.372578785 +0000 UTC m=+0.373401128 container init 55f780cf7a1159b766eff059db86e38b85f874034c874817bd49ec9ca92af0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-10T00:28:14.570 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:14.693 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:28:14 smithi062 podman[73801]: 2024-06-10 00:28:14.37634088 +0000 UTC m=+0.377163204 container start 55f780cf7a1159b766eff059db86e38b85f874034c874817bd49ec9ca92af0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:28:14.694 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:28:14 smithi062 ceph-mds[73838]: -- 172.21.15.62:0/1377090145 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5630a1615a40 con 0x5630a15e9000 2024-06-10T00:28:14.694 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:28:14 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-g[73816]: starting mds.g at 2024-06-10T00:28:14.694 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:28:14 smithi062 bash[73801]: 55f780cf7a1159b766eff059db86e38b85f874034c874817bd49ec9ca92af0e6 2024-06-10T00:28:14.694 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:28:14 smithi062 systemd[1]: Started Ceph mds.g for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:28:15.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:15 smithi062 ceph-mon[32390]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:15.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:15 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:15.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:15 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:15.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:15 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:15.750 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:15 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:15.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:15 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:15.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:15 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:15.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:15 smithi062 ceph-mon[32390]: Deploying daemon mds.a on smithi062 2024-06-10T00:28:15.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:15 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.62:6838/4040406390,v1:172.21.15.62:6839/4040406390] up:boot 2024-06-10T00:28:15.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:15 smithi062 ceph-mon[32390]: fsmap 10 up:standby 2024-06-10T00:28:15.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:15 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T00:28:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:15 smithi138 ceph-mon[36549]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:15 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:15 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:15 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:15 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:15.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:15 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:15.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:15 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:15.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:15 smithi138 ceph-mon[36549]: Deploying daemon mds.a on smithi062 2024-06-10T00:28:15.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:15 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.62:6838/4040406390,v1:172.21.15.62:6839/4040406390] up:boot 2024-06-10T00:28:15.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:15 smithi138 ceph-mon[36549]: fsmap 10 up:standby 2024-06-10T00:28:15.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:15 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T00:28:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:15 smithi105 ceph-mon[36321]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:15 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:15 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:15 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:15 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:15 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:15 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:15 smithi105 ceph-mon[36321]: Deploying daemon mds.a on smithi062 2024-06-10T00:28:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:15 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.62:6838/4040406390,v1:172.21.15.62:6839/4040406390] up:boot 2024-06-10T00:28:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:15 smithi105 ceph-mon[36321]: fsmap 10 up:standby 2024-06-10T00:28:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:15 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T00:28:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.906+0000 7ff00bee9700 1 -- 172.21.15.62:0/1070256118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff004102740 msgr2=0x7ff004102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.906+0000 7ff00bee9700 1 --2- 172.21.15.62:0/1070256118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff004102740 0x7ff004102b20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0000099c0 tx=0x7ff000009cd0 comp rx=0 tx=0).stop 2024-06-10T00:28:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.906+0000 7ff00bee9700 1 -- 172.21.15.62:0/1070256118 shutdown_connections 2024-06-10T00:28:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.906+0000 7ff00bee9700 1 --2- 172.21.15.62:0/1070256118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff004102740 0x7ff004102b20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.906+0000 7ff00bee9700 1 --2- 172.21.15.62:0/1070256118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff004107840 0x7ff00410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.906+0000 7ff00bee9700 1 --2- 172.21.15.62:0/1070256118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0041030f0 0x7ff004107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.906+0000 7ff00bee9700 1 -- 172.21.15.62:0/1070256118 >> 172.21.15.62:0/1070256118 conn(0x7ff0040795f0 msgr2=0x7ff004079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.906+0000 7ff00bee9700 1 -- 172.21.15.62:0/1070256118 shutdown_connections 2024-06-10T00:28:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.906+0000 7ff00bee9700 1 -- 172.21.15.62:0/1070256118 wait complete. 2024-06-10T00:28:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.906+0000 7ff00bee9700 1 Processor -- start 2024-06-10T00:28:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff00bee9700 1 -- start start 2024-06-10T00:28:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff00bee9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff004102740 0x7ff004074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff00bee9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0041030f0 0x7ff004074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff00bee9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff004107840 0x7ff004075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff00bee9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff004112760 con 0x7ff004102740 2024-06-10T00:28:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff00bee9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041125e0 con 0x7ff0041030f0 2024-06-10T00:28:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff00bee9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041128e0 con 0x7ff004107840 2024-06-10T00:28:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff009c85700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff004102740 0x7ff004074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff009c85700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff004102740 0x7ff004074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59184/0 (socket says 172.21.15.62:59184) 2024-06-10T00:28:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff009c85700 1 -- 172.21.15.62:0/2398718158 learned_addr learned my addr 172.21.15.62:0/2398718158 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff009484700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0041030f0 0x7ff004074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff00a486700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff004107840 0x7ff004075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff009c85700 1 -- 172.21.15.62:0/2398718158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff004107840 msgr2=0x7ff004075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff009c85700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff004107840 0x7ff004075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff009c85700 1 -- 172.21.15.62:0/2398718158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0041030f0 msgr2=0x7ff004074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff009c85700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0041030f0 0x7ff004074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff009c85700 1 -- 172.21.15.62:0/2398718158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0041a7130 con 0x7ff004102740 2024-06-10T00:28:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.907+0000 7ff00a486700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff004107840 0x7ff004075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:28:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.908+0000 7ff009c85700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff004102740 0x7ff004074790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff000006010 tx=0x7ff000009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:16.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.908+0000 7feffaffd700 1 -- 172.21.15.62:0/2398718158 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000027070 con 0x7ff004102740 2024-06-10T00:28:16.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.908+0000 7ff00bee9700 1 -- 172.21.15.62:0/2398718158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0041a73c0 con 0x7ff004102740 2024-06-10T00:28:16.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.908+0000 7feffaffd700 1 -- 172.21.15.62:0/2398718158 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff000003780 con 0x7ff004102740 2024-06-10T00:28:16.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.908+0000 7feffaffd700 1 -- 172.21.15.62:0/2398718158 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000022da0 con 0x7ff004102740 2024-06-10T00:28:16.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.908+0000 7ff00bee9700 1 -- 172.21.15.62:0/2398718158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0041a7810 con 0x7ff004102740 2024-06-10T00:28:16.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.909+0000 7ff00bee9700 1 -- 172.21.15.62:0/2398718158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00410aa80 con 0x7ff004102740 2024-06-10T00:28:16.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.910+0000 7feffaffd700 1 -- 172.21.15.62:0/2398718158 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff00002a070 con 0x7ff004102740 2024-06-10T00:28:16.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.911+0000 7feffaffd700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7feff00a2c20 0x7feff00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:16.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.911+0000 7ff009484700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7feff00a2c20 0x7feff00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:16.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.911+0000 7feffaffd700 1 -- 172.21.15.62:0/2398718158 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff0000cf820 con 0x7ff004102740 2024-06-10T00:28:16.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.911+0000 7ff009484700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7feff00a2c20 0x7feff00a50e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff004102040 tx=0x7feff400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:16.914+0000 7feffaffd700 1 -- 172.21.15.62:0/2398718158 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff000013070 con 0x7ff004102740 2024-06-10T00:28:17.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:17.170+0000 7ff00bee9700 1 -- 172.21.15.62:0/2398718158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff00410a460 con 0x7ff004102740 2024-06-10T00:28:17.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:17.171+0000 7feffaffd700 1 -- 172.21.15.62:0/2398718158 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7ff00009ccb0 con 0x7ff004102740 2024-06-10T00:28:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:28:17.171 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-10T00:28:17.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:17.173+0000 7ff00bee9700 1 -- 172.21.15.62:0/2398718158 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7feff00a2c20 msgr2=0x7feff00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:17.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:17.173+0000 7ff00bee9700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7feff00a2c20 0x7feff00a50e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff004102040 tx=0x7feff400b410 comp rx=0 tx=0).stop 2024-06-10T00:28:17.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:17.173+0000 7ff00bee9700 1 -- 172.21.15.62:0/2398718158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff004102740 msgr2=0x7ff004074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:17.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:17.173+0000 7ff00bee9700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff004102740 0x7ff004074790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff000006010 tx=0x7ff000009f80 comp rx=0 tx=0).stop 2024-06-10T00:28:17.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:17.173+0000 7ff00bee9700 1 -- 172.21.15.62:0/2398718158 shutdown_connections 2024-06-10T00:28:17.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:17.173+0000 7ff00bee9700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff004102740 0x7ff004074790 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:17.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:17.173+0000 7ff00bee9700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7feff00a2c20 0x7feff00a50e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:17.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:17.173+0000 7ff00bee9700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff004107840 0x7ff004075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:17.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:17.173+0000 7ff00bee9700 1 --2- 172.21.15.62:0/2398718158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0041030f0 0x7ff004074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:17.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:17.173+0000 7ff00bee9700 1 -- 172.21.15.62:0/2398718158 >> 172.21.15.62:0/2398718158 conn(0x7ff0040795f0 msgr2=0x7ff004108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:17.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:17.173+0000 7ff00bee9700 1 -- 172.21.15.62:0/2398718158 shutdown_connections 2024-06-10T00:28:17.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:17.174+0000 7ff00bee9700 1 -- 172.21.15.62:0/2398718158 wait complete. 2024-06-10T00:28:17.178 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 11 2024-06-10T00:28:17.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:17 smithi062 ceph-mon[32390]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:17.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:17 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2398718158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:17 smithi138 ceph-mon[36549]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:17 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2398718158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:18.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:17 smithi105 ceph-mon[36321]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:18.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:17 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2398718158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:18.608 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:28:18 smithi062 systemd[1]: Starting Ceph mds.a for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:28:18.956 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:28:18 smithi062 podman[74259]: 2024-06-10 00:28:18.588792422 +0000 UTC m=+0.017824892 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:28:18.956 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:28:18 smithi062 podman[74259]: 2024-06-10 00:28:18.693014785 +0000 UTC m=+0.122047269 container create 9267421457af753c655b89d23a1df80c609fc2666c6f753d1193c03289a32869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:28:18.956 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:28:18 smithi062 podman[74259]: 2024-06-10 00:28:18.929132081 +0000 UTC m=+0.358164544 container init 9267421457af753c655b89d23a1df80c609fc2666c6f753d1193c03289a32869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-10T00:28:18.956 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:28:18 smithi062 podman[74259]: 2024-06-10 00:28:18.932036456 +0000 UTC m=+0.361068919 container start 9267421457af753c655b89d23a1df80c609fc2666c6f753d1193c03289a32869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-a, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-10T00:28:18.956 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:28:18 smithi062 ceph-mds[74278]: -- 172.21.15.62:0/4189723474 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5643fe8bfa40 con 0x5643fe892c00 2024-06-10T00:28:19.357 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:28:18 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-a[74274]: starting mds.a at 2024-06-10T00:28:19.358 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:28:19 smithi062 bash[74259]: 9267421457af753c655b89d23a1df80c609fc2666c6f753d1193c03289a32869 2024-06-10T00:28:19.358 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:28:19 smithi062 systemd[1]: Started Ceph mds.a for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:28:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:19 smithi062 ceph-mon[32390]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:19.871 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:28:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:19 smithi138 ceph-mon[36549]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:19.922 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:19 smithi105 ceph-mon[36321]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:19.922 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:19.922 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:19.922 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:19.922 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:28:19.922 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:28:19.923 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:20.128 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:20.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:20 smithi062 ceph-mon[32390]: Deploying daemon mds.b on smithi105 2024-06-10T00:28:20.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:20 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.62:6840/2460552589,v1:172.21.15.62:6841/2460552589] up:boot 2024-06-10T00:28:20.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:20 smithi062 ceph-mon[32390]: fsmap 11 up:standby 2024-06-10T00:28:20.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:20 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T00:28:20.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:20 smithi138 ceph-mon[36549]: Deploying daemon mds.b on smithi105 2024-06-10T00:28:20.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:20 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.62:6840/2460552589,v1:172.21.15.62:6841/2460552589] up:boot 2024-06-10T00:28:20.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:20 smithi138 ceph-mon[36549]: fsmap 11 up:standby 2024-06-10T00:28:20.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:20 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T00:28:20.935 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:20 smithi105 ceph-mon[36321]: Deploying daemon mds.b on smithi105 2024-06-10T00:28:20.935 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:20 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.62:6840/2460552589,v1:172.21.15.62:6841/2460552589] up:boot 2024-06-10T00:28:20.935 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:20 smithi105 ceph-mon[36321]: fsmap 11 up:standby 2024-06-10T00:28:20.935 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:20 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T00:28:21.532 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:28:21 smithi105 systemd[1]: Starting Ceph mds.b for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:28:21.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.799+0000 7f68878b2700 1 -- 172.21.15.62:0/3718506488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68801030f0 msgr2=0x7f6880107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:21.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.799+0000 7f68878b2700 1 --2- 172.21.15.62:0/3718506488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68801030f0 0x7f6880107110 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f687c0099c0 tx=0x7f687c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:28:21.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.799+0000 7f68878b2700 1 -- 172.21.15.62:0/3718506488 shutdown_connections 2024-06-10T00:28:21.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.799+0000 7f68878b2700 1 --2- 172.21.15.62:0/3718506488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6880107840 0x7f688010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:21.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.799+0000 7f68878b2700 1 --2- 172.21.15.62:0/3718506488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6880102740 0x7f6880102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:21.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.799+0000 7f68878b2700 1 --2- 172.21.15.62:0/3718506488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68801030f0 0x7f6880107110 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:21.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.799+0000 7f68878b2700 1 -- 172.21.15.62:0/3718506488 >> 172.21.15.62:0/3718506488 conn(0x7f68800795f0 msgr2=0x7f6880079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:21.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.800+0000 7f68878b2700 1 -- 172.21.15.62:0/3718506488 shutdown_connections 2024-06-10T00:28:21.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.800+0000 7f68878b2700 1 -- 172.21.15.62:0/3718506488 wait complete. 2024-06-10T00:28:21.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.800+0000 7f68878b2700 1 Processor -- start 2024-06-10T00:28:21.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.800+0000 7f68878b2700 1 -- start start 2024-06-10T00:28:21.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.800+0000 7f68878b2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6880102740 0x7f688019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:21.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.800+0000 7f68878b2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68801030f0 0x7f688019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:21.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.800+0000 7f68878b2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6880107840 0x7f68801989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:21.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.800+0000 7f68878b2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6880112750 con 0x7f68801030f0 2024-06-10T00:28:21.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.800+0000 7f68878b2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f68801125d0 con 0x7f6880107840 2024-06-10T00:28:21.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.800+0000 7f68878b2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68801128d0 con 0x7f6880102740 2024-06-10T00:28:21.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f6884e4d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68801030f0 0x7f688019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:21.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f6884e4d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68801030f0 0x7f688019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54604/0 (socket says 172.21.15.62:54604) 2024-06-10T00:28:21.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f6884e4d700 1 -- 172.21.15.62:0/4138502004 learned_addr learned my addr 172.21.15.62:0/4138502004 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:21.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f6885e4f700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6880107840 0x7f68801989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:21.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f688564e700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6880102740 0x7f688019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:21.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f6884e4d700 1 -- 172.21.15.62:0/4138502004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6880102740 msgr2=0x7f688019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:21.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f6884e4d700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6880102740 0x7f688019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:21.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f6884e4d700 1 -- 172.21.15.62:0/4138502004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6880107840 msgr2=0x7f68801989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:21.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f6884e4d700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6880107840 0x7f68801989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:21.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f6884e4d700 1 -- 172.21.15.62:0/4138502004 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6880199130 con 0x7f68801030f0 2024-06-10T00:28:21.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f6885e4f700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6880107840 0x7f68801989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:28:21.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f6884e4d700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68801030f0 0x7f688019ed00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f687c0059d0 tx=0x7f687c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:21.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f68727fc700 1 -- 172.21.15.62:0/4138502004 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c027070 con 0x7f68801030f0 2024-06-10T00:28:21.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f68727fc700 1 -- 172.21.15.62:0/4138502004 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f687c003680 con 0x7f68801030f0 2024-06-10T00:28:21.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f68878b2700 1 -- 172.21.15.62:0/4138502004 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68801993c0 con 0x7f68801030f0 2024-06-10T00:28:21.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f68727fc700 1 -- 172.21.15.62:0/4138502004 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c0173f0 con 0x7f68801030f0 2024-06-10T00:28:21.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.801+0000 7f68878b2700 1 -- 172.21.15.62:0/4138502004 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68801a5690 con 0x7f68801030f0 2024-06-10T00:28:21.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.802+0000 7f68878b2700 1 -- 172.21.15.62:0/4138502004 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6880069180 con 0x7f68801030f0 2024-06-10T00:28:21.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.804+0000 7f68727fc700 1 -- 172.21.15.62:0/4138502004 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f687c02a070 con 0x7f68801030f0 2024-06-10T00:28:21.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.804+0000 7f68727fc700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f686c0a2ca0 0x7f686c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:21.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.804+0000 7f688564e700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f686c0a2ca0 0x7f686c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:21.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.804+0000 7f68727fc700 1 -- 172.21.15.62:0/4138502004 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f687c0cf6e0 con 0x7f68801030f0 2024-06-10T00:28:21.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.805+0000 7f688564e700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f686c0a2ca0 0x7f686c0a5160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f68801073f0 tx=0x7f687400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:21.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:21.808+0000 7f68727fc700 1 -- 172.21.15.62:0/4138502004 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f687c013070 con 0x7f68801030f0 2024-06-10T00:28:21.808 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:21 smithi062 ceph-mon[32390]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:21 smithi138 ceph-mon[36549]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:21.890 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:21 smithi105 ceph-mon[36321]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:21.891 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:28:21 smithi105 podman[66836]: 2024-06-10 00:28:21.521181867 +0000 UTC m=+0.017086151 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:28:21.891 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:28:21 smithi105 podman[66836]: 2024-06-10 00:28:21.638597526 +0000 UTC m=+0.134501798 container create c7fd797fca0ba5e2f012820206794d56a93abe76ccea2ba06a382446f056ef6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-10T00:28:21.891 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:28:21 smithi105 podman[66836]: 2024-06-10 00:28:21.891403411 +0000 UTC m=+0.387307682 container init c7fd797fca0ba5e2f012820206794d56a93abe76ccea2ba06a382446f056ef6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True) 2024-06-10T00:28:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:22.066+0000 7f68878b2700 1 -- 172.21.15.62:0/4138502004 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6880199eb0 con 0x7f68801030f0 2024-06-10T00:28:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:22.068+0000 7f68727fc700 1 -- 172.21.15.62:0/4138502004 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f687c09cb70 con 0x7f68801030f0 2024-06-10T00:28:22.068 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:28:22.068 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-10T00:28:22.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:22.070+0000 7f68878b2700 1 -- 172.21.15.62:0/4138502004 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f686c0a2ca0 msgr2=0x7f686c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:22.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:22.070+0000 7f68878b2700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f686c0a2ca0 0x7f686c0a5160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f68801073f0 tx=0x7f687400b410 comp rx=0 tx=0).stop 2024-06-10T00:28:22.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:22.070+0000 7f68878b2700 1 -- 172.21.15.62:0/4138502004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68801030f0 msgr2=0x7f688019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:22.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:22.070+0000 7f68878b2700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68801030f0 0x7f688019ed00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f687c0059d0 tx=0x7f687c00fa00 comp rx=0 tx=0).stop 2024-06-10T00:28:22.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:22.070+0000 7f68878b2700 1 -- 172.21.15.62:0/4138502004 shutdown_connections 2024-06-10T00:28:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:22.070+0000 7f68878b2700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f686c0a2ca0 0x7f686c0a5160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:22.070+0000 7f68878b2700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6880107840 0x7f68801989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:22.070+0000 7f68878b2700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6880102740 0x7f688019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:22.070+0000 7f68878b2700 1 --2- 172.21.15.62:0/4138502004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68801030f0 0x7f688019ed00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:22.070+0000 7f68878b2700 1 -- 172.21.15.62:0/4138502004 >> 172.21.15.62:0/4138502004 conn(0x7f68800795f0 msgr2=0x7f6880101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:22.070+0000 7f68878b2700 1 -- 172.21.15.62:0/4138502004 shutdown_connections 2024-06-10T00:28:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:22.070+0000 7f68878b2700 1 -- 172.21.15.62:0/4138502004 wait complete. 2024-06-10T00:28:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 12 2024-06-10T00:28:22.281 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:28:21 smithi105 podman[66836]: 2024-06-10 00:28:21.894552899 +0000 UTC m=+0.390457179 container start c7fd797fca0ba5e2f012820206794d56a93abe76ccea2ba06a382446f056ef6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-b, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:28:22.282 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:28:21 smithi105 ceph-mds[66855]: -- 172.21.15.105:0/1898677715 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c442b21a40 con 0x55c442af4800 2024-06-10T00:28:22.282 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:28:21 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-b[66851]: starting mds.b at 2024-06-10T00:28:22.282 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:28:21 smithi105 bash[66836]: c7fd797fca0ba5e2f012820206794d56a93abe76ccea2ba06a382446f056ef6b 2024-06-10T00:28:22.282 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:28:21 smithi105 systemd[1]: Started Ceph mds.b for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:28:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:23 smithi105 ceph-mon[36321]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:23 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/4138502004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:23.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:28:23.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:23 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] up:boot 2024-06-10T00:28:23.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:23 smithi105 ceph-mon[36321]: fsmap 12 up:standby 2024-06-10T00:28:23.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T00:28:23.289 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:23 smithi062 ceph-mon[32390]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:23.289 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.289 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:23 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/4138502004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:23.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:28:23.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:23 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] up:boot 2024-06-10T00:28:23.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:23 smithi062 ceph-mon[32390]: fsmap 12 up:standby 2024-06-10T00:28:23.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T00:28:23.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:23 smithi138 ceph-mon[36549]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:23.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:23.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:23 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/4138502004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:23.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:28:23.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:23 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] up:boot 2024-06-10T00:28:23.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:23 smithi138 ceph-mon[36549]: fsmap 12 up:standby 2024-06-10T00:28:23.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T00:28:24.793 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:28:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:25.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:24 smithi062 ceph-mon[32390]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:25.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:25.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:28:25.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:28:25.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:24 smithi138 ceph-mon[36549]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:25.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:25.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:28:25.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:28:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:24 smithi105 ceph-mon[36321]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:25.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:28:25.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:25.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:28:26.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.728+0000 7fa3aacc6700 1 -- 172.21.15.62:0/2951656393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a4107840 msgr2=0x7fa3a410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:26.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.728+0000 7fa3aacc6700 1 --2- 172.21.15.62:0/2951656393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a4107840 0x7fa3a410f3c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00b0d0 tx=0x7fa39c00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:28:26.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.728+0000 7fa3aacc6700 1 -- 172.21.15.62:0/2951656393 shutdown_connections 2024-06-10T00:28:26.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.728+0000 7fa3aacc6700 1 --2- 172.21.15.62:0/2951656393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a4107840 0x7fa3a410f3c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:26.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.728+0000 7fa3aacc6700 1 --2- 172.21.15.62:0/2951656393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a4102740 0x7fa3a4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:26.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.728+0000 7fa3aacc6700 1 --2- 172.21.15.62:0/2951656393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3a41030f0 0x7fa3a4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:26.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.728+0000 7fa3aacc6700 1 -- 172.21.15.62:0/2951656393 >> 172.21.15.62:0/2951656393 conn(0x7fa3a40795f0 msgr2=0x7fa3a4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:26.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.728+0000 7fa3aacc6700 1 -- 172.21.15.62:0/2951656393 shutdown_connections 2024-06-10T00:28:26.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.728+0000 7fa3aacc6700 1 -- 172.21.15.62:0/2951656393 wait complete. 2024-06-10T00:28:26.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.728+0000 7fa3aacc6700 1 Processor -- start 2024-06-10T00:28:26.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.728+0000 7fa3aacc6700 1 -- start start 2024-06-10T00:28:26.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3aacc6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a4102740 0x7fa3a419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:26.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3aacc6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3a41030f0 0x7fa3a419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:26.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3aacc6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a4107840 0x7fa3a41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:26.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3aacc6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a4112780 con 0x7fa3a4102740 2024-06-10T00:28:26.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3aacc6700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a4112600 con 0x7fa3a41030f0 2024-06-10T00:28:26.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3aacc6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a4112900 con 0x7fa3a4107840 2024-06-10T00:28:26.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3a8a62700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a4102740 0x7fa3a419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:26.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3a8a62700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a4102740 0x7fa3a419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54622/0 (socket says 172.21.15.62:54622) 2024-06-10T00:28:26.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3a8a62700 1 -- 172.21.15.62:0/1186607902 learned_addr learned my addr 172.21.15.62:0/1186607902 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:26.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3a3fff700 1 --2- 172.21.15.62:0/1186607902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3a41030f0 0x7fa3a419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:26.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3a9263700 1 --2- 172.21.15.62:0/1186607902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a4107840 0x7fa3a41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:26.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3a8a62700 1 -- 172.21.15.62:0/1186607902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a4107840 msgr2=0x7fa3a41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:26.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3a8a62700 1 --2- 172.21.15.62:0/1186607902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a4107840 0x7fa3a41988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:26.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3a8a62700 1 -- 172.21.15.62:0/1186607902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3a41030f0 msgr2=0x7fa3a419ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:26.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3a8a62700 1 --2- 172.21.15.62:0/1186607902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3a41030f0 0x7fa3a419ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:26.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3a8a62700 1 -- 172.21.15.62:0/1186607902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a4199020 con 0x7fa3a4102740 2024-06-10T00:28:26.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3a8a62700 1 --2- 172.21.15.62:0/1186607902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a4102740 0x7fa3a419e700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa39400c970 tx=0x7fa39400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:26.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3a1ffb700 1 -- 172.21.15.62:0/1186607902 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394007aa0 con 0x7fa3a4102740 2024-06-10T00:28:26.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3aacc6700 1 -- 172.21.15.62:0/1186607902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a4199310 con 0x7fa3a4102740 2024-06-10T00:28:26.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3a1ffb700 1 -- 172.21.15.62:0/1186607902 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa394004d10 con 0x7fa3a4102740 2024-06-10T00:28:26.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.729+0000 7fa3a1ffb700 1 -- 172.21.15.62:0/1186607902 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39400f450 con 0x7fa3a4102740 2024-06-10T00:28:26.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.730+0000 7fa3aacc6700 1 -- 172.21.15.62:0/1186607902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a41a55d0 con 0x7fa3a4102740 2024-06-10T00:28:26.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.731+0000 7fa3aacc6700 1 -- 172.21.15.62:0/1186607902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a4069180 con 0x7fa3a4102740 2024-06-10T00:28:26.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.732+0000 7fa3a1ffb700 1 -- 172.21.15.62:0/1186607902 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa394004750 con 0x7fa3a4102740 2024-06-10T00:28:26.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.732+0000 7fa3a1ffb700 1 --2- 172.21.15.62:0/1186607902 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa38c0a2c20 0x7fa38c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:26.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.732+0000 7fa3a3fff700 1 --2- 172.21.15.62:0/1186607902 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa38c0a2c20 0x7fa38c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:26.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.733+0000 7fa3a1ffb700 1 -- 172.21.15.62:0/1186607902 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa3940ce980 con 0x7fa3a4102740 2024-06-10T00:28:26.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.733+0000 7fa3a3fff700 1 --2- 172.21.15.62:0/1186607902 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa38c0a2c20 0x7fa38c0a50e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4108c60 tx=0x7fa39800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:26.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.737+0000 7fa3a1ffb700 1 -- 172.21.15.62:0/1186607902 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa39409be10 con 0x7fa3a4102740 2024-06-10T00:28:26.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.992+0000 7fa3aacc6700 1 -- 172.21.15.62:0/1186607902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3a4199df0 con 0x7fa3a4102740 2024-06-10T00:28:26.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.992+0000 7fa3a1ffb700 1 -- 172.21.15.62:0/1186607902 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7fa394005700 con 0x7fa3a4102740 2024-06-10T00:28:26.994 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:28:26.994 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-06-10T00:28:26.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.995+0000 7fa3aacc6700 1 -- 172.21.15.62:0/1186607902 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa38c0a2c20 msgr2=0x7fa38c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:26.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.995+0000 7fa3aacc6700 1 --2- 172.21.15.62:0/1186607902 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa38c0a2c20 0x7fa38c0a50e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4108c60 tx=0x7fa39800a380 comp rx=0 tx=0).stop 2024-06-10T00:28:26.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.995+0000 7fa3aacc6700 1 -- 172.21.15.62:0/1186607902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a4102740 msgr2=0x7fa3a419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.995+0000 7fa3aacc6700 1 --2- 172.21.15.62:0/1186607902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a4102740 0x7fa3a419e700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa39400c970 tx=0x7fa39400cc80 comp rx=0 tx=0).stop 2024-06-10T00:28:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.995+0000 7fa3aacc6700 1 -- 172.21.15.62:0/1186607902 shutdown_connections 2024-06-10T00:28:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.995+0000 7fa3aacc6700 1 --2- 172.21.15.62:0/1186607902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a4102740 0x7fa3a419e700 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.995+0000 7fa3aacc6700 1 --2- 172.21.15.62:0/1186607902 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa38c0a2c20 0x7fa38c0a50e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.995+0000 7fa3aacc6700 1 --2- 172.21.15.62:0/1186607902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a4107840 0x7fa3a41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.995+0000 7fa3aacc6700 1 --2- 172.21.15.62:0/1186607902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3a41030f0 0x7fa3a419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.995+0000 7fa3aacc6700 1 -- 172.21.15.62:0/1186607902 >> 172.21.15.62:0/1186607902 conn(0x7fa3a40795f0 msgr2=0x7fa3a41004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.995+0000 7fa3aacc6700 1 -- 172.21.15.62:0/1186607902 shutdown_connections 2024-06-10T00:28:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:26.995+0000 7fa3aacc6700 1 -- 172.21.15.62:0/1186607902 wait complete. 2024-06-10T00:28:26.998 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 13 2024-06-10T00:28:27.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:27 smithi062 ceph-mon[32390]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:27.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:27 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:27 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:27 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:27 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:27 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:27 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1186607902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:27 smithi105 ceph-mon[36321]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:27 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:27 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:27 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:27 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:27 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:27 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1186607902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:27 smithi138 ceph-mon[36549]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:27 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:27 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:27 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:27 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:27 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:27.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:27 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1186607902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:27.738 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-10T00:28:28.101 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:29.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:29 smithi105 ceph-mon[36321]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:29.557 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:29 smithi062 ceph-mon[32390]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:29.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:29 smithi138 ceph-mon[36549]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:29.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.774+0000 7fb1abcb8700 1 -- 172.21.15.62:0/4164484806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4106b90 msgr2=0x7fb1a4106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:29.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.774+0000 7fb1abcb8700 1 --2- 172.21.15.62:0/4164484806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4106b90 0x7fb1a4106f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000b0a0 tx=0x7fb1a000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:28:29.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.774+0000 7fb1abcb8700 1 -- 172.21.15.62:0/4164484806 shutdown_connections 2024-06-10T00:28:29.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.774+0000 7fb1abcb8700 1 --2- 172.21.15.62:0/4164484806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4106b90 0x7fb1a4106f70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:29.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.774+0000 7fb1abcb8700 1 --2- 172.21.15.62:0/4164484806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1a4101510 0x7fb1a410d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:29.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.774+0000 7fb1abcb8700 1 --2- 172.21.15.62:0/4164484806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a4100b50 0x7fb1a4100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:29.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.774+0000 7fb1abcb8700 1 -- 172.21.15.62:0/4164484806 >> 172.21.15.62:0/4164484806 conn(0x7fb1a4077a00 msgr2=0x7fb1a40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:29.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.774+0000 7fb1abcb8700 1 -- 172.21.15.62:0/4164484806 shutdown_connections 2024-06-10T00:28:29.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.774+0000 7fb1abcb8700 1 -- 172.21.15.62:0/4164484806 wait complete. 2024-06-10T00:28:29.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.775+0000 7fb1abcb8700 1 Processor -- start 2024-06-10T00:28:29.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.775+0000 7fb1abcb8700 1 -- start start 2024-06-10T00:28:29.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.775+0000 7fb1abcb8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1a4100b50 0x7fb1a419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:29.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.775+0000 7fb1abcb8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4101510 0x7fb1a419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:29.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.775+0000 7fb1abcb8700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a4106b90 0x7fb1a4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:29.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.775+0000 7fb1abcb8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4110ef0 con 0x7fb1a4101510 2024-06-10T00:28:29.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.775+0000 7fb1abcb8700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4110d70 con 0x7fb1a4106b90 2024-06-10T00:28:29.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.775+0000 7fb1abcb8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4111070 con 0x7fb1a4100b50 2024-06-10T00:28:29.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.775+0000 7fb1a9253700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4101510 0x7fb1a419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:29.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.775+0000 7fb1a9253700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4101510 0x7fb1a419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57968/0 (socket says 172.21.15.62:57968) 2024-06-10T00:28:29.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.775+0000 7fb1a9253700 1 -- 172.21.15.62:0/601017560 learned_addr learned my addr 172.21.15.62:0/601017560 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:29.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.775+0000 7fb1a9a54700 1 --2- 172.21.15.62:0/601017560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1a4100b50 0x7fb1a419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:29.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.776+0000 7fb1aa255700 1 --2- 172.21.15.62:0/601017560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a4106b90 0x7fb1a4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:29.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.776+0000 7fb1a9253700 1 -- 172.21.15.62:0/601017560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1a4100b50 msgr2=0x7fb1a419e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:29.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.776+0000 7fb1a9253700 1 --2- 172.21.15.62:0/601017560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1a4100b50 0x7fb1a419e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:29.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.776+0000 7fb1a9253700 1 -- 172.21.15.62:0/601017560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a4106b90 msgr2=0x7fb1a4198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:29.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.776+0000 7fb1a9253700 1 --2- 172.21.15.62:0/601017560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a4106b90 0x7fb1a4198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:29.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.776+0000 7fb1a9253700 1 -- 172.21.15.62:0/601017560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a4199220 con 0x7fb1a4101510 2024-06-10T00:28:29.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.776+0000 7fb1a9253700 1 --2- 172.21.15.62:0/601017560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4101510 0x7fb1a419ee40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb19400b530 tx=0x7fb19400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:29.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.776+0000 7fb19affd700 1 -- 172.21.15.62:0/601017560 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194015020 con 0x7fb1a4101510 2024-06-10T00:28:29.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.776+0000 7fb19affd700 1 -- 172.21.15.62:0/601017560 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb194011d10 con 0x7fb1a4101510 2024-06-10T00:28:29.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.776+0000 7fb1abcb8700 1 -- 172.21.15.62:0/601017560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a4199510 con 0x7fb1a4101510 2024-06-10T00:28:29.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.776+0000 7fb19affd700 1 -- 172.21.15.62:0/601017560 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19400f2e0 con 0x7fb1a4101510 2024-06-10T00:28:29.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.776+0000 7fb1abcb8700 1 -- 172.21.15.62:0/601017560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a41a57d0 con 0x7fb1a4101510 2024-06-10T00:28:29.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.777+0000 7fb1abcb8700 1 -- 172.21.15.62:0/601017560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a4069180 con 0x7fb1a4101510 2024-06-10T00:28:29.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.779+0000 7fb19affd700 1 -- 172.21.15.62:0/601017560 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb194018020 con 0x7fb1a4101510 2024-06-10T00:28:29.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.779+0000 7fb19affd700 1 --2- 172.21.15.62:0/601017560 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb1900a2ca0 0x7fb1900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:29.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.779+0000 7fb1a9a54700 1 --2- 172.21.15.62:0/601017560 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb1900a2ca0 0x7fb1900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:29.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.779+0000 7fb19affd700 1 -- 172.21.15.62:0/601017560 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb1940ce7c0 con 0x7fb1a4101510 2024-06-10T00:28:29.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.780+0000 7fb1a9a54700 1 --2- 172.21.15.62:0/601017560 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb1900a2ca0 0x7fb1900a5160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000b070 tx=0x7fb1a000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:29.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.783+0000 7fb19affd700 1 -- 172.21.15.62:0/601017560 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb19401f020 con 0x7fb1a4101510 2024-06-10T00:28:29.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:29.999+0000 7fb1abcb8700 1 -- 172.21.15.62:0/601017560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb1a4199ff0 con 0x7fb1a4101510 2024-06-10T00:28:30.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:30.000+0000 7fb19affd700 1 -- 172.21.15.62:0/601017560 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7fb1a4199ff0 con 0x7fb1a4101510 2024-06-10T00:28:30.000 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:28:30.001 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-06-10T00:28:30.001 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:28:30.001 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-06-10T00:28:30.001 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-06-10T00:28:30.001 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:28:30.001 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-06-10T00:28:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:30.002+0000 7fb1abcb8700 1 -- 172.21.15.62:0/601017560 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb1900a2ca0 msgr2=0x7fb1900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:30.002+0000 7fb1abcb8700 1 --2- 172.21.15.62:0/601017560 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb1900a2ca0 0x7fb1900a5160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000b070 tx=0x7fb1a000b7b0 comp rx=0 tx=0).stop 2024-06-10T00:28:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:30.002+0000 7fb1abcb8700 1 -- 172.21.15.62:0/601017560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4101510 msgr2=0x7fb1a419ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:30.002+0000 7fb1abcb8700 1 --2- 172.21.15.62:0/601017560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4101510 0x7fb1a419ee40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb19400b530 tx=0x7fb19400b840 comp rx=0 tx=0).stop 2024-06-10T00:28:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:30.002+0000 7fb1abcb8700 1 -- 172.21.15.62:0/601017560 shutdown_connections 2024-06-10T00:28:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:30.002+0000 7fb1abcb8700 1 --2- 172.21.15.62:0/601017560 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb1900a2ca0 0x7fb1900a5160 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:30.002+0000 7fb1abcb8700 1 --2- 172.21.15.62:0/601017560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1a4106b90 0x7fb1a4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:30.002+0000 7fb1abcb8700 1 --2- 172.21.15.62:0/601017560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1a4100b50 0x7fb1a419e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:30.002+0000 7fb1abcb8700 1 --2- 172.21.15.62:0/601017560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1a4101510 0x7fb1a419ee40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:30.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:30.002+0000 7fb1abcb8700 1 -- 172.21.15.62:0/601017560 >> 172.21.15.62:0/601017560 conn(0x7fb1a4077a00 msgr2=0x7fb1a40fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:30.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:30.002+0000 7fb1abcb8700 1 -- 172.21.15.62:0/601017560 shutdown_connections 2024-06-10T00:28:30.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:30.002+0000 7fb1abcb8700 1 -- 172.21.15.62:0/601017560 wait complete. 2024-06-10T00:28:30.719 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-10T00:28:30.719 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-10T00:28:30.972 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:31.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:31 smithi062 ceph-mon[32390]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:31.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:31.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:31 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/601017560' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-10T00:28:31.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:31.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:31.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:28:31.266 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:31.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:31 smithi105 ceph-mon[36321]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:31.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:31.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:31 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/601017560' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-10T00:28:31.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:31.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:31.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:28:31.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:31 smithi138 ceph-mon[36549]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:31 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/601017560' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-10T00:28:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:28:31.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:28:31.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:28:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.666+0000 7fcb1484a700 1 -- 172.21.15.62:0/3212745537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb10106b90 msgr2=0x7fcb10106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.666+0000 7fcb1484a700 1 --2- 172.21.15.62:0/3212745537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb10106b90 0x7fcb10106f70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009990 tx=0x7fcb00009ca0 comp rx=0 tx=0).stop 2024-06-10T00:28:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.666+0000 7fcb1484a700 1 -- 172.21.15.62:0/3212745537 shutdown_connections 2024-06-10T00:28:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.666+0000 7fcb1484a700 1 --2- 172.21.15.62:0/3212745537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb10106b90 0x7fcb10106f70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.666+0000 7fcb1484a700 1 --2- 172.21.15.62:0/3212745537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb10101510 0x7fcb1010d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.666+0000 7fcb1484a700 1 --2- 172.21.15.62:0/3212745537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb10100b50 0x7fcb10100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.666+0000 7fcb1484a700 1 -- 172.21.15.62:0/3212745537 >> 172.21.15.62:0/3212745537 conn(0x7fcb10077a00 msgr2=0x7fcb100fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.666+0000 7fcb1484a700 1 -- 172.21.15.62:0/3212745537 shutdown_connections 2024-06-10T00:28:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.666+0000 7fcb1484a700 1 -- 172.21.15.62:0/3212745537 wait complete. 2024-06-10T00:28:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.667+0000 7fcb1484a700 1 Processor -- start 2024-06-10T00:28:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.667+0000 7fcb1484a700 1 -- start start 2024-06-10T00:28:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.667+0000 7fcb1484a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb10100b50 0x7fcb1019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.667+0000 7fcb1484a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb10101510 0x7fcb1019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.667+0000 7fcb1484a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb10106b90 0x7fcb10198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.667+0000 7fcb1484a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101116f0 con 0x7fcb10100b50 2024-06-10T00:28:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.667+0000 7fcb1484a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10111570 con 0x7fcb10106b90 2024-06-10T00:28:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.667+0000 7fcb1484a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10111870 con 0x7fcb10101510 2024-06-10T00:28:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.667+0000 7fcb0e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb10100b50 0x7fcb1019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.667+0000 7fcb0e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb10100b50 0x7fcb1019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57988/0 (socket says 172.21.15.62:57988) 2024-06-10T00:28:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.667+0000 7fcb0e59c700 1 -- 172.21.15.62:0/848957161 learned_addr learned my addr 172.21.15.62:0/848957161 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.667+0000 7fcb0dd9b700 1 --2- 172.21.15.62:0/848957161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb10101510 0x7fcb1019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.668+0000 7fcb0e59c700 1 -- 172.21.15.62:0/848957161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb10101510 msgr2=0x7fcb1019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:32.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.668+0000 7fcb0e59c700 1 --2- 172.21.15.62:0/848957161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb10101510 0x7fcb1019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:32.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.668+0000 7fcb0ed9d700 1 --2- 172.21.15.62:0/848957161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb10106b90 0x7fcb10198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:32.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.668+0000 7fcb0e59c700 1 -- 172.21.15.62:0/848957161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb10106b90 msgr2=0x7fcb10198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:32.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.668+0000 7fcb0e59c700 1 --2- 172.21.15.62:0/848957161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb10106b90 0x7fcb10198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:32.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.668+0000 7fcb0e59c700 1 -- 172.21.15.62:0/848957161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb101991c0 con 0x7fcb10100b50 2024-06-10T00:28:32.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.668+0000 7fcb0e59c700 1 --2- 172.21.15.62:0/848957161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb10100b50 0x7fcb1019e8a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcb00000c00 tx=0x7fcb00003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:32.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.668+0000 7fcaff7fe700 1 -- 172.21.15.62:0/848957161 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00027070 con 0x7fcb10100b50 2024-06-10T00:28:32.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.668+0000 7fcaff7fe700 1 -- 172.21.15.62:0/848957161 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb00004110 con 0x7fcb10100b50 2024-06-10T00:28:32.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.668+0000 7fcb1484a700 1 -- 172.21.15.62:0/848957161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb10199450 con 0x7fcb10100b50 2024-06-10T00:28:32.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.668+0000 7fcaff7fe700 1 -- 172.21.15.62:0/848957161 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000f460 con 0x7fcb10100b50 2024-06-10T00:28:32.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.668+0000 7fcb1484a700 1 -- 172.21.15.62:0/848957161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb101a5770 con 0x7fcb10100b50 2024-06-10T00:28:32.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.669+0000 7fcb1484a700 1 -- 172.21.15.62:0/848957161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb10069180 con 0x7fcb10100b50 2024-06-10T00:28:32.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.670+0000 7fcaff7fe700 1 -- 172.21.15.62:0/848957161 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcb0002a070 con 0x7fcb10100b50 2024-06-10T00:28:32.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.671+0000 7fcaff7fe700 1 --2- 172.21.15.62:0/848957161 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcaf80a2c20 0x7fcaf80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.671+0000 7fcb0dd9b700 1 --2- 172.21.15.62:0/848957161 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcaf80a2c20 0x7fcaf80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.671+0000 7fcaff7fe700 1 -- 172.21.15.62:0/848957161 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fcb000cfb20 con 0x7fcb10100b50 2024-06-10T00:28:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.671+0000 7fcb0dd9b700 1 --2- 172.21.15.62:0/848957161 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcaf80a2c20 0x7fcaf80a50e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcb10100020 tx=0x7fcb0400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.675+0000 7fcaff7fe700 1 -- 172.21.15.62:0/848957161 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcb0005e850 con 0x7fcb10100b50 2024-06-10T00:28:32.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:32.884+0000 7fcb1484a700 1 -- 172.21.15.62:0/848957161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcb1002dca0 con 0x7fcb10100b50 2024-06-10T00:28:33.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:33.372+0000 7fcaff7fe700 1 -- 172.21.15.62:0/848957161 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7fcb00013070 con 0x7fcb10100b50 2024-06-10T00:28:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:33.375+0000 7fcb1484a700 1 -- 172.21.15.62:0/848957161 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcaf80a2c20 msgr2=0x7fcaf80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:33.375+0000 7fcb1484a700 1 --2- 172.21.15.62:0/848957161 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcaf80a2c20 0x7fcaf80a50e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcb10100020 tx=0x7fcb0400a300 comp rx=0 tx=0).stop 2024-06-10T00:28:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:33.375+0000 7fcb1484a700 1 -- 172.21.15.62:0/848957161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb10100b50 msgr2=0x7fcb1019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:33.375+0000 7fcb1484a700 1 --2- 172.21.15.62:0/848957161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb10100b50 0x7fcb1019e8a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcb00000c00 tx=0x7fcb00003680 comp rx=0 tx=0).stop 2024-06-10T00:28:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:33.375+0000 7fcb1484a700 1 -- 172.21.15.62:0/848957161 shutdown_connections 2024-06-10T00:28:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:33.375+0000 7fcb1484a700 1 --2- 172.21.15.62:0/848957161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb10100b50 0x7fcb1019e8a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:33.375+0000 7fcb1484a700 1 --2- 172.21.15.62:0/848957161 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcaf80a2c20 0x7fcaf80a50e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:33.375+0000 7fcb1484a700 1 --2- 172.21.15.62:0/848957161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb10106b90 0x7fcb10198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:33.375+0000 7fcb1484a700 1 --2- 172.21.15.62:0/848957161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb10101510 0x7fcb1019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:33.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:33.375+0000 7fcb1484a700 1 -- 172.21.15.62:0/848957161 >> 172.21.15.62:0/848957161 conn(0x7fcb10077a00 msgr2=0x7fcb1010e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:33.375+0000 7fcb1484a700 1 -- 172.21.15.62:0/848957161 shutdown_connections 2024-06-10T00:28:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:33.375+0000 7fcb1484a700 1 -- 172.21.15.62:0/848957161 wait complete. 2024-06-10T00:28:33.376 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-06-10T00:28:33.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:33 smithi062 ceph-mon[32390]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:33.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:33 smithi138 ceph-mon[36549]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:33.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:33 smithi105 ceph-mon[36321]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:34.000 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-06-10T00:28:34.251 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:34.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:34 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/848957161' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-10T00:28:34.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:34 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/848957161' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-10T00:28:34.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:34 smithi062 ceph-mon[32390]: osdmap e82: 12 total, 12 up, 12 in 2024-06-10T00:28:34.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:34 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/848957161' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-10T00:28:34.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:34 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/848957161' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-10T00:28:34.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:34 smithi138 ceph-mon[36549]: osdmap e82: 12 total, 12 up, 12 in 2024-06-10T00:28:34.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:34 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/848957161' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-10T00:28:34.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:34 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/848957161' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-10T00:28:34.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:34 smithi105 ceph-mon[36321]: osdmap e82: 12 total, 12 up, 12 in 2024-06-10T00:28:35.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:35 smithi062 ceph-mon[32390]: pgmap v288: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:35.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:35 smithi062 ceph-mon[32390]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T00:28:35.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:35 smithi062 ceph-mon[32390]: osdmap e83: 12 total, 12 up, 12 in 2024-06-10T00:28:35.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:35 smithi138 ceph-mon[36549]: pgmap v288: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:35.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:35 smithi138 ceph-mon[36549]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T00:28:35.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:35 smithi138 ceph-mon[36549]: osdmap e83: 12 total, 12 up, 12 in 2024-06-10T00:28:35.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:35 smithi105 ceph-mon[36321]: pgmap v288: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:35.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:35 smithi105 ceph-mon[36321]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T00:28:35.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:35 smithi105 ceph-mon[36321]: osdmap e83: 12 total, 12 up, 12 in 2024-06-10T00:28:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.921+0000 7f2f19c84700 1 -- 172.21.15.62:0/2051311967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f141084e0 msgr2=0x7f2f141088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.921+0000 7f2f19c84700 1 --2- 172.21.15.62:0/2051311967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f141084e0 0x7f2f141088c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2f040099c0 tx=0x7f2f04009cd0 comp rx=0 tx=0).stop 2024-06-10T00:28:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.921+0000 7f2f19c84700 1 -- 172.21.15.62:0/2051311967 shutdown_connections 2024-06-10T00:28:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.921+0000 7f2f19c84700 1 --2- 172.21.15.62:0/2051311967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f141084e0 0x7f2f141088c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:35.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.921+0000 7f2f19c84700 1 --2- 172.21.15.62:0/2051311967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1406ab10 0x7f2f1410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:35.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.921+0000 7f2f19c84700 1 --2- 172.21.15.62:0/2051311967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f1406a170 0x7f2f1406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:35.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.921+0000 7f2f19c84700 1 -- 172.21.15.62:0/2051311967 >> 172.21.15.62:0/2051311967 conn(0x7f2f140782c0 msgr2=0x7f2f140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:35.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.921+0000 7f2f19c84700 1 -- 172.21.15.62:0/2051311967 shutdown_connections 2024-06-10T00:28:35.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.921+0000 7f2f19c84700 1 -- 172.21.15.62:0/2051311967 wait complete. 2024-06-10T00:28:35.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.922+0000 7f2f19c84700 1 Processor -- start 2024-06-10T00:28:35.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.922+0000 7f2f19c84700 1 -- start start 2024-06-10T00:28:35.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.922+0000 7f2f19c84700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f1406a170 0x7f2f1419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:35.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.922+0000 7f2f19c84700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1406ab10 0x7f2f1419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.922+0000 7f2f19c84700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f141084e0 0x7f2f14198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.922+0000 7f2f19c84700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2f14112740 con 0x7f2f1406a170 2024-06-10T00:28:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.922+0000 7f2f19c84700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141125c0 con 0x7f2f141084e0 2024-06-10T00:28:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.922+0000 7f2f19c84700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141128c0 con 0x7f2f1406ab10 2024-06-10T00:28:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f137fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f1406a170 0x7f2f1419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f137fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f1406a170 0x7f2f1419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58002/0 (socket says 172.21.15.62:58002) 2024-06-10T00:28:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f137fe700 1 -- 172.21.15.62:0/1865461553 learned_addr learned my addr 172.21.15.62:0/1865461553 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f12ffd700 1 --2- 172.21.15.62:0/1865461553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1406ab10 0x7f2f1419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f13fff700 1 --2- 172.21.15.62:0/1865461553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f141084e0 0x7f2f14198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f137fe700 1 -- 172.21.15.62:0/1865461553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1406ab10 msgr2=0x7f2f1419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f137fe700 1 --2- 172.21.15.62:0/1865461553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1406ab10 0x7f2f1419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f137fe700 1 -- 172.21.15.62:0/1865461553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f141084e0 msgr2=0x7f2f14198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f137fe700 1 --2- 172.21.15.62:0/1865461553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f141084e0 0x7f2f14198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f137fe700 1 -- 172.21.15.62:0/1865461553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f141991a0 con 0x7f2f1406a170 2024-06-10T00:28:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f137fe700 1 --2- 172.21.15.62:0/1865461553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f1406a170 0x7f2f1419e830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2f04006010 tx=0x7f2f0400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f10ff9700 1 -- 172.21.15.62:0/1865461553 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04003680 con 0x7f2f1406a170 2024-06-10T00:28:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f10ff9700 1 -- 172.21.15.62:0/1865461553 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f040055b0 con 0x7f2f1406a170 2024-06-10T00:28:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f10ff9700 1 -- 172.21.15.62:0/1865461553 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f040173f0 con 0x7f2f1406a170 2024-06-10T00:28:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f19c84700 1 -- 172.21.15.62:0/1865461553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f14199430 con 0x7f2f1406a170 2024-06-10T00:28:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.923+0000 7f2f19c84700 1 -- 172.21.15.62:0/1865461553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f141a5700 con 0x7f2f1406a170 2024-06-10T00:28:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.924+0000 7f2f19c84700 1 -- 172.21.15.62:0/1865461553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f14103b90 con 0x7f2f1406a170 2024-06-10T00:28:35.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.926+0000 7f2f10ff9700 1 -- 172.21.15.62:0/1865461553 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2f0400fca0 con 0x7f2f1406a170 2024-06-10T00:28:35.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.927+0000 7f2f10ff9700 1 --2- 172.21.15.62:0/1865461553 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2efc0a2dc0 0x7f2efc0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:35.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.927+0000 7f2f10ff9700 1 -- 172.21.15.62:0/1865461553 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7f2f040cff70 con 0x7f2f1406a170 2024-06-10T00:28:35.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.927+0000 7f2f12ffd700 1 --2- 172.21.15.62:0/1865461553 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2efc0a2dc0 0x7f2efc0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:35.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.928+0000 7f2f12ffd700 1 --2- 172.21.15.62:0/1865461553 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2efc0a2dc0 0x7f2efc0a5280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2f14078830 tx=0x7f2f08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:35.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:35.931+0000 7f2f10ff9700 1 -- 172.21.15.62:0/1865461553 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2f04013070 con 0x7f2f1406a170 2024-06-10T00:28:36.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:36.138+0000 7f2f19c84700 1 -- 172.21.15.62:0/1865461553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f2f14199f20 con 0x7f2f1406a170 2024-06-10T00:28:36.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:36.629+0000 7f2f10ff9700 1 -- 172.21.15.62:0/1865461553 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v85) ==== 171+0+0 (secure 0 0 0) 0x7f2f0409d290 con 0x7f2f1406a170 2024-06-10T00:28:36.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:36.632+0000 7f2f19c84700 1 -- 172.21.15.62:0/1865461553 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2efc0a2dc0 msgr2=0x7f2efc0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:36.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:36.632+0000 7f2f19c84700 1 --2- 172.21.15.62:0/1865461553 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2efc0a2dc0 0x7f2efc0a5280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2f14078830 tx=0x7f2f08009260 comp rx=0 tx=0).stop 2024-06-10T00:28:36.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:36.632+0000 7f2f19c84700 1 -- 172.21.15.62:0/1865461553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f1406a170 msgr2=0x7f2f1419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:36.632+0000 7f2f19c84700 1 --2- 172.21.15.62:0/1865461553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f1406a170 0x7f2f1419e830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2f04006010 tx=0x7f2f0400f690 comp rx=0 tx=0).stop 2024-06-10T00:28:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:36.632+0000 7f2f19c84700 1 -- 172.21.15.62:0/1865461553 shutdown_connections 2024-06-10T00:28:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:36.632+0000 7f2f19c84700 1 --2- 172.21.15.62:0/1865461553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f1406a170 0x7f2f1419e830 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:36.632+0000 7f2f19c84700 1 --2- 172.21.15.62:0/1865461553 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2efc0a2dc0 0x7f2efc0a5280 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:36.632+0000 7f2f19c84700 1 --2- 172.21.15.62:0/1865461553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f141084e0 0x7f2f14198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:36.632+0000 7f2f19c84700 1 --2- 172.21.15.62:0/1865461553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1406ab10 0x7f2f1419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:36.632+0000 7f2f19c84700 1 -- 172.21.15.62:0/1865461553 >> 172.21.15.62:0/1865461553 conn(0x7f2f140782c0 msgr2=0x7f2f140ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:36.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:36.632+0000 7f2f19c84700 1 -- 172.21.15.62:0/1865461553 shutdown_connections 2024-06-10T00:28:36.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:36.633+0000 7f2f19c84700 1 -- 172.21.15.62:0/1865461553 wait complete. 2024-06-10T00:28:36.635 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-06-10T00:28:36.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:36 smithi062 ceph-mon[32390]: osdmap e84: 12 total, 12 up, 12 in 2024-06-10T00:28:36.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:36 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1865461553' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-10T00:28:36.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:36 smithi138 ceph-mon[36549]: osdmap e84: 12 total, 12 up, 12 in 2024-06-10T00:28:36.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:36 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1865461553' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-10T00:28:37.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:36 smithi105 ceph-mon[36321]: osdmap e84: 12 total, 12 up, 12 in 2024-06-10T00:28:37.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:36 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1865461553' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-10T00:28:37.332 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-10T00:28:37.581 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:37.932 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:37 smithi062 ceph-mon[32390]: pgmap v291: 33 pgs: 29 unknown, 4 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:37.932 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:37 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1865461553' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-10T00:28:37.932 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:37 smithi062 ceph-mon[32390]: osdmap e85: 12 total, 12 up, 12 in 2024-06-10T00:28:38.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:37 smithi105 ceph-mon[36321]: pgmap v291: 33 pgs: 29 unknown, 4 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:37 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1865461553' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-10T00:28:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:37 smithi105 ceph-mon[36321]: osdmap e85: 12 total, 12 up, 12 in 2024-06-10T00:28:38.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:37 smithi138 ceph-mon[36549]: pgmap v291: 33 pgs: 29 unknown, 4 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:38.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:37 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1865461553' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-10T00:28:38.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:37 smithi138 ceph-mon[36549]: osdmap e85: 12 total, 12 up, 12 in 2024-06-10T00:28:38.687 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:38 smithi062 ceph-mon[32390]: osdmap e86: 12 total, 12 up, 12 in 2024-06-10T00:28:39.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:38 smithi105 ceph-mon[36321]: osdmap e86: 12 total, 12 up, 12 in 2024-06-10T00:28:39.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:38 smithi138 ceph-mon[36549]: osdmap e86: 12 total, 12 up, 12 in 2024-06-10T00:28:39.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.216+0000 7f3458fe6700 1 -- 172.21.15.62:0/2971907369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3454107830 msgr2=0x7f345410f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:39.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.216+0000 7f3458fe6700 1 --2- 172.21.15.62:0/2971907369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3454107830 0x7f345410f3b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f344c0099c0 tx=0x7f344c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:28:39.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.216+0000 7f3458fe6700 1 -- 172.21.15.62:0/2971907369 shutdown_connections 2024-06-10T00:28:39.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.216+0000 7f3458fe6700 1 --2- 172.21.15.62:0/2971907369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3454107830 0x7f345410f3b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:39.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.216+0000 7f3458fe6700 1 --2- 172.21.15.62:0/2971907369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3454102740 0x7f3454102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:39.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.216+0000 7f3458fe6700 1 --2- 172.21.15.62:0/2971907369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34541030f0 0x7f3454107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:39.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.216+0000 7f3458fe6700 1 -- 172.21.15.62:0/2971907369 >> 172.21.15.62:0/2971907369 conn(0x7f34540795b0 msgr2=0x7f34540799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:39.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.216+0000 7f3458fe6700 1 -- 172.21.15.62:0/2971907369 shutdown_connections 2024-06-10T00:28:39.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.216+0000 7f3458fe6700 1 -- 172.21.15.62:0/2971907369 wait complete. 2024-06-10T00:28:39.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.217+0000 7f3458fe6700 1 Processor -- start 2024-06-10T00:28:39.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.217+0000 7f3458fe6700 1 -- start start 2024-06-10T00:28:39.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.217+0000 7f3458fe6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3454102740 0x7f345419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:39.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.217+0000 7f3458fe6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34541030f0 0x7f345419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:39.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.217+0000 7f3458fe6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3454107830 0x7f3454198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:39.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.217+0000 7f3458fe6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3454112840 con 0x7f34541030f0 2024-06-10T00:28:39.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.217+0000 7f3458fe6700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f34541126c0 con 0x7f3454107830 2024-06-10T00:28:39.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.217+0000 7f3458fe6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34541129c0 con 0x7f3454102740 2024-06-10T00:28:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.217+0000 7f3451d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34541030f0 0x7f345419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.217+0000 7f3451d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34541030f0 0x7f345419ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60756/0 (socket says 172.21.15.62:60756) 2024-06-10T00:28:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.217+0000 7f3451d9b700 1 -- 172.21.15.62:0/3812599440 learned_addr learned my addr 172.21.15.62:0/3812599440 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f345259c700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3454102740 0x7f345419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f3452d9d700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3454107830 0x7f3454198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f3451d9b700 1 -- 172.21.15.62:0/3812599440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3454102740 msgr2=0x7f345419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f3451d9b700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3454102740 0x7f345419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f3451d9b700 1 -- 172.21.15.62:0/3812599440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3454107830 msgr2=0x7f3454198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f3451d9b700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3454107830 0x7f3454198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f3451d9b700 1 -- 172.21.15.62:0/3812599440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3454199240 con 0x7f34541030f0 2024-06-10T00:28:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f3452d9d700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3454107830 0x7f3454198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:28:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f345259c700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3454102740 0x7f345419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:28:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f3451d9b700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34541030f0 0x7f345419ee10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f343c00b530 tx=0x7f343c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f344b7fe700 1 -- 172.21.15.62:0/3812599440 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c015020 con 0x7f34541030f0 2024-06-10T00:28:39.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f3458fe6700 1 -- 172.21.15.62:0/3812599440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3454199530 con 0x7f34541030f0 2024-06-10T00:28:39.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f344b7fe700 1 -- 172.21.15.62:0/3812599440 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f343c011d10 con 0x7f34541030f0 2024-06-10T00:28:39.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f344b7fe700 1 -- 172.21.15.62:0/3812599440 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c010e50 con 0x7f34541030f0 2024-06-10T00:28:39.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.218+0000 7f3458fe6700 1 -- 172.21.15.62:0/3812599440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34541a57a0 con 0x7f34541030f0 2024-06-10T00:28:39.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.219+0000 7f3458fe6700 1 -- 172.21.15.62:0/3812599440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3454069180 con 0x7f34541030f0 2024-06-10T00:28:39.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.221+0000 7f344b7fe700 1 -- 172.21.15.62:0/3812599440 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f343c018020 con 0x7f34541030f0 2024-06-10T00:28:39.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.221+0000 7f344b7fe700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f34400a2dc0 0x7f34400a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:39.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.221+0000 7f345259c700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f34400a2dc0 0x7f34400a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:39.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.221+0000 7f344b7fe700 1 -- 172.21.15.62:0/3812599440 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7879+0+0 (secure 0 0 0) 0x7f343c0cec90 con 0x7f34541030f0 2024-06-10T00:28:39.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.222+0000 7f345259c700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f34400a2dc0 0x7f34400a5280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3454107330 tx=0x7f3444009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:39.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.225+0000 7f344b7fe700 1 -- 172.21.15.62:0/3812599440 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f343c01f020 con 0x7f34541030f0 2024-06-10T00:28:39.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.477+0000 7f3458fe6700 1 -- 172.21.15.62:0/3812599440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f345410aa70 con 0x7f34541030f0 2024-06-10T00:28:39.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.642+0000 7f344b7fe700 1 -- 172.21.15.62:0/3812599440 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f343c09bda0 con 0x7f34541030f0 2024-06-10T00:28:39.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.645+0000 7f3458fe6700 1 -- 172.21.15.62:0/3812599440 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f34400a2dc0 msgr2=0x7f34400a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:39.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.645+0000 7f3458fe6700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f34400a2dc0 0x7f34400a5280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3454107330 tx=0x7f3444009570 comp rx=0 tx=0).stop 2024-06-10T00:28:39.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.645+0000 7f3458fe6700 1 -- 172.21.15.62:0/3812599440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34541030f0 msgr2=0x7f345419ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:39.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.645+0000 7f3458fe6700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34541030f0 0x7f345419ee10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f343c00b530 tx=0x7f343c00b8f0 comp rx=0 tx=0).stop 2024-06-10T00:28:39.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.645+0000 7f3458fe6700 1 -- 172.21.15.62:0/3812599440 shutdown_connections 2024-06-10T00:28:39.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.646+0000 7f3458fe6700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f34400a2dc0 0x7f34400a5280 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:39.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.646+0000 7f3458fe6700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3454107830 0x7f3454198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:39.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.646+0000 7f3458fe6700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3454102740 0x7f345419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:39.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.646+0000 7f3458fe6700 1 --2- 172.21.15.62:0/3812599440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34541030f0 0x7f345419ee10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:39.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.646+0000 7f3458fe6700 1 -- 172.21.15.62:0/3812599440 >> 172.21.15.62:0/3812599440 conn(0x7f34540795b0 msgr2=0x7f34541006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:39.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.646+0000 7f3458fe6700 1 -- 172.21.15.62:0/3812599440 shutdown_connections 2024-06-10T00:28:39.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:39.646+0000 7f3458fe6700 1 -- 172.21.15.62:0/3812599440 wait complete. 2024-06-10T00:28:39.647 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 2 and data pool 3 2024-06-10T00:28:40.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:39 smithi062 ceph-mon[32390]: pgmap v294: 97 pgs: 12 creating+peering, 40 active+clean, 45 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:40.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:39 smithi062 ceph-mon[32390]: osdmap e87: 12 total, 12 up, 12 in 2024-06-10T00:28:40.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:39 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3812599440' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T00:28:40.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:39 smithi062 ceph-mon[32390]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-10T00:28:40.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:39 smithi105 ceph-mon[36321]: pgmap v294: 97 pgs: 12 creating+peering, 40 active+clean, 45 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:40.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:39 smithi105 ceph-mon[36321]: osdmap e87: 12 total, 12 up, 12 in 2024-06-10T00:28:40.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:39 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3812599440' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T00:28:40.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:39 smithi105 ceph-mon[36321]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-10T00:28:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:39 smithi138 ceph-mon[36549]: pgmap v294: 97 pgs: 12 creating+peering, 40 active+clean, 45 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:39 smithi138 ceph-mon[36549]: osdmap e87: 12 total, 12 up, 12 in 2024-06-10T00:28:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:39 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3812599440' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T00:28:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:39 smithi138 ceph-mon[36549]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-10T00:28:40.315 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-06-10T00:28:40.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-06-10T00:28:40.567 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:40.933 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:40 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3812599440' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-10T00:28:40.934 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:40 smithi062 ceph-mon[32390]: osdmap e88: 12 total, 12 up, 12 in 2024-06-10T00:28:40.934 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:40 smithi062 ceph-mon[32390]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-10T00:28:40.934 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:40 smithi062 ceph-mon[32390]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-10T00:28:41.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:40 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3812599440' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-10T00:28:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:40 smithi105 ceph-mon[36321]: osdmap e88: 12 total, 12 up, 12 in 2024-06-10T00:28:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:40 smithi105 ceph-mon[36321]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-10T00:28:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:40 smithi105 ceph-mon[36321]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-10T00:28:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:40 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3812599440' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-10T00:28:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:40 smithi138 ceph-mon[36549]: osdmap e88: 12 total, 12 up, 12 in 2024-06-10T00:28:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:40 smithi138 ceph-mon[36549]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-10T00:28:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:40 smithi138 ceph-mon[36549]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-10T00:28:42.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:41 smithi105 ceph-mon[36321]: pgmap v297: 97 pgs: 12 creating+peering, 62 active+clean, 23 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:42.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:41 smithi105 ceph-mon[36321]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-10T00:28:42.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:41 smithi105 ceph-mon[36321]: Cluster is now healthy 2024-06-10T00:28:42.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:41 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] up:active 2024-06-10T00:28:42.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:41 smithi105 ceph-mon[36321]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:28:42.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:41 smithi105 ceph-mon[36321]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:28:42.070 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:41 smithi138 ceph-mon[36549]: pgmap v297: 97 pgs: 12 creating+peering, 62 active+clean, 23 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:42.070 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:41 smithi138 ceph-mon[36549]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-10T00:28:42.071 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:41 smithi138 ceph-mon[36549]: Cluster is now healthy 2024-06-10T00:28:42.071 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:41 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] up:active 2024-06-10T00:28:42.071 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:41 smithi138 ceph-mon[36549]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:28:42.071 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:41 smithi138 ceph-mon[36549]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:28:42.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:41 smithi062 ceph-mon[32390]: pgmap v297: 97 pgs: 12 creating+peering, 62 active+clean, 23 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:42.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:41 smithi062 ceph-mon[32390]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-10T00:28:42.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:41 smithi062 ceph-mon[32390]: Cluster is now healthy 2024-06-10T00:28:42.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:41 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] up:active 2024-06-10T00:28:42.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:41 smithi062 ceph-mon[32390]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:28:42.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:41 smithi062 ceph-mon[32390]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:28:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.330+0000 7f941c1b4700 1 -- 172.21.15.62:0/3420246443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9414101770 msgr2=0x7f9414101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.330+0000 7f941c1b4700 1 --2- 172.21.15.62:0/3420246443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9414101770 0x7f9414101bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9408009990 tx=0x7f9408009ca0 comp rx=0 tx=0).stop 2024-06-10T00:28:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.330+0000 7f941c1b4700 1 -- 172.21.15.62:0/3420246443 shutdown_connections 2024-06-10T00:28:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.330+0000 7f941c1b4700 1 --2- 172.21.15.62:0/3420246443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9414102130 0x7f941410e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.330+0000 7f941c1b4700 1 --2- 172.21.15.62:0/3420246443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9414107550 0x7f9414107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.330+0000 7f941c1b4700 1 --2- 172.21.15.62:0/3420246443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9414101770 0x7f9414101bf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.330+0000 7f941c1b4700 1 -- 172.21.15.62:0/3420246443 >> 172.21.15.62:0/3420246443 conn(0x7f9414078340 msgr2=0x7f9414078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.330+0000 7f941c1b4700 1 -- 172.21.15.62:0/3420246443 shutdown_connections 2024-06-10T00:28:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.330+0000 7f941c1b4700 1 -- 172.21.15.62:0/3420246443 wait complete. 2024-06-10T00:28:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.331+0000 7f941c1b4700 1 Processor -- start 2024-06-10T00:28:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.331+0000 7f941c1b4700 1 -- start start 2024-06-10T00:28:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.331+0000 7f941c1b4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9414101770 0x7f941419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.331+0000 7f941c1b4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9414102130 0x7f941419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.331+0000 7f941c1b4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9414107550 0x7f9414197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.331+0000 7f941c1b4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f94141116d0 con 0x7f9414101770 2024-06-10T00:28:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.331+0000 7f941c1b4700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9414111550 con 0x7f9414102130 2024-06-10T00:28:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.331+0000 7f941c1b4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9414111850 con 0x7f9414107550 2024-06-10T00:28:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.331+0000 7f9419f50700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9414101770 0x7f941419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.331+0000 7f9419f50700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9414101770 0x7f941419d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60772/0 (socket says 172.21.15.62:60772) 2024-06-10T00:28:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.331+0000 7f9419f50700 1 -- 172.21.15.62:0/3182930444 learned_addr learned my addr 172.21.15.62:0/3182930444 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f941a751700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9414107550 0x7f9414197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f941974f700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9414102130 0x7f941419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f9419f50700 1 -- 172.21.15.62:0/3182930444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9414107550 msgr2=0x7f9414197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f9419f50700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9414107550 0x7f9414197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f9419f50700 1 -- 172.21.15.62:0/3182930444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9414102130 msgr2=0x7f941419dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f9419f50700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9414102130 0x7f941419dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f9419f50700 1 -- 172.21.15.62:0/3182930444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9414198030 con 0x7f9414101770 2024-06-10T00:28:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f941a751700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9414107550 0x7f9414197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:28:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f941974f700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9414102130 0x7f941419dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:28:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f9419f50700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9414101770 0x7f941419d6d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f940c007b50 tx=0x7f940c007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f9412ffd700 1 -- 172.21.15.62:0/3182930444 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c01a070 con 0x7f9414101770 2024-06-10T00:28:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f9412ffd700 1 -- 172.21.15.62:0/3182930444 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f940c00eaa0 con 0x7f9414101770 2024-06-10T00:28:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f941c1b4700 1 -- 172.21.15.62:0/3182930444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9414198320 con 0x7f9414101770 2024-06-10T00:28:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f9412ffd700 1 -- 172.21.15.62:0/3182930444 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c00ab40 con 0x7f9414101770 2024-06-10T00:28:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.332+0000 7f941c1b4700 1 -- 172.21.15.62:0/3182930444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f941406ab60 con 0x7f9414101770 2024-06-10T00:28:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.333+0000 7f941c1b4700 1 -- 172.21.15.62:0/3182930444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9414102ac0 con 0x7f9414101770 2024-06-10T00:28:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.334+0000 7f9412ffd700 1 -- 172.21.15.62:0/3182930444 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f940c00ace0 con 0x7f9414101770 2024-06-10T00:28:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.335+0000 7f9412ffd700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93fc0a2c20 0x7f93fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.335+0000 7f941974f700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93fc0a2c20 0x7f93fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.335+0000 7f9412ffd700 1 -- 172.21.15.62:0/3182930444 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f940c0cfd80 con 0x7f9414101770 2024-06-10T00:28:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.335+0000 7f941974f700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93fc0a2c20 0x7f93fc0a50e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9408006010 tx=0x7f940800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:42.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.339+0000 7f9412ffd700 1 -- 172.21.15.62:0/3182930444 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f940c050850 con 0x7f9414101770 2024-06-10T00:28:42.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.555+0000 7f941c1b4700 1 -- 172.21.15.62:0/3182930444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f941406b5f0 con 0x7f9414101770 2024-06-10T00:28:42.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.660+0000 7f9412ffd700 1 -- 172.21.15.62:0/3182930444 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v89) ==== 157+0+0 (secure 0 0 0) 0x7f940c09ce30 con 0x7f9414101770 2024-06-10T00:28:42.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.663+0000 7f941c1b4700 1 -- 172.21.15.62:0/3182930444 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93fc0a2c20 msgr2=0x7f93fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:42.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.663+0000 7f941c1b4700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93fc0a2c20 0x7f93fc0a50e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9408006010 tx=0x7f940800b540 comp rx=0 tx=0).stop 2024-06-10T00:28:42.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.663+0000 7f941c1b4700 1 -- 172.21.15.62:0/3182930444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9414101770 msgr2=0x7f941419d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:42.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.663+0000 7f941c1b4700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9414101770 0x7f941419d6d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f940c007b50 tx=0x7f940c007f10 comp rx=0 tx=0).stop 2024-06-10T00:28:42.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.663+0000 7f941c1b4700 1 -- 172.21.15.62:0/3182930444 shutdown_connections 2024-06-10T00:28:42.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.663+0000 7f941c1b4700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9414101770 0x7f941419d6d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:42.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.663+0000 7f941c1b4700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93fc0a2c20 0x7f93fc0a50e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:42.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.663+0000 7f941c1b4700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9414107550 0x7f9414197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:42.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.663+0000 7f941c1b4700 1 --2- 172.21.15.62:0/3182930444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9414102130 0x7f941419dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:42.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.663+0000 7f941c1b4700 1 -- 172.21.15.62:0/3182930444 >> 172.21.15.62:0/3182930444 conn(0x7f9414078340 msgr2=0x7f9414100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:42.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.663+0000 7f941c1b4700 1 -- 172.21.15.62:0/3182930444 shutdown_connections 2024-06-10T00:28:42.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:42.663+0000 7f941c1b4700 1 -- 172.21.15.62:0/3182930444 wait complete. 2024-06-10T00:28:43.030 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:42 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3182930444' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-10T00:28:43.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:42 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3182930444' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-10T00:28:43.032 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:28:42 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-b[66851]: 2024-06-10T00:28:42.581+0000 7f9075648700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-10T00:28:43.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:42 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3182930444' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-10T00:28:43.476 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-06-10T00:28:43.734 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:43.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:43 smithi062 ceph-mon[32390]: pgmap v298: 97 pgs: 12 creating+peering, 85 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-06-10T00:28:43.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:43 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3182930444' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-10T00:28:43.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:43 smithi062 ceph-mon[32390]: osdmap e89: 12 total, 12 up, 12 in 2024-06-10T00:28:44.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:43 smithi105 ceph-mon[36321]: pgmap v298: 97 pgs: 12 creating+peering, 85 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-06-10T00:28:44.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:43 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3182930444' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-10T00:28:44.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:43 smithi105 ceph-mon[36321]: osdmap e89: 12 total, 12 up, 12 in 2024-06-10T00:28:44.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:43 smithi138 ceph-mon[36549]: pgmap v298: 97 pgs: 12 creating+peering, 85 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-06-10T00:28:44.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:43 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3182930444' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-10T00:28:44.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:43 smithi138 ceph-mon[36549]: osdmap e89: 12 total, 12 up, 12 in 2024-06-10T00:28:45.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.501+0000 7f2e8e439700 1 -- 172.21.15.62:0/3018722231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e88101680 msgr2=0x7f2e88101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:45.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.501+0000 7f2e8e439700 1 --2- 172.21.15.62:0/3018722231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e88101680 0x7f2e88101a60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2e780099c0 tx=0x7f2e78009cd0 comp rx=0 tx=0).stop 2024-06-10T00:28:45.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.501+0000 7f2e8e439700 1 -- 172.21.15.62:0/3018722231 shutdown_connections 2024-06-10T00:28:45.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.501+0000 7f2e8e439700 1 --2- 172.21.15.62:0/3018722231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e88101680 0x7f2e88101a60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:45.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.501+0000 7f2e8e439700 1 --2- 172.21.15.62:0/3018722231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e881066d0 0x7f2e8810e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:45.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.501+0000 7f2e8e439700 1 --2- 172.21.15.62:0/3018722231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e88102030 0x7f2e88105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:45.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.501+0000 7f2e8e439700 1 -- 172.21.15.62:0/3018722231 >> 172.21.15.62:0/3018722231 conn(0x7f2e880773e0 msgr2=0x7f2e880777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:45.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.501+0000 7f2e8e439700 1 -- 172.21.15.62:0/3018722231 shutdown_connections 2024-06-10T00:28:45.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.501+0000 7f2e8e439700 1 -- 172.21.15.62:0/3018722231 wait complete. 2024-06-10T00:28:45.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.501+0000 7f2e8e439700 1 Processor -- start 2024-06-10T00:28:45.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.501+0000 7f2e8e439700 1 -- start start 2024-06-10T00:28:45.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e8e439700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e88101680 0x7f2e88103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:45.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e8e439700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e88102030 0x7f2e881035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:45.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e8e439700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e881066d0 0x7f2e88103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:45.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e8e439700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88111730 con 0x7f2e88101680 2024-06-10T00:28:45.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e8e439700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881115b0 con 0x7f2e881066d0 2024-06-10T00:28:45.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e8e439700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881118b0 con 0x7f2e88102030 2024-06-10T00:28:45.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e87fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e88101680 0x7f2e88103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:45.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e87fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e88101680 0x7f2e88103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60794/0 (socket says 172.21.15.62:60794) 2024-06-10T00:28:45.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e87fff700 1 -- 172.21.15.62:0/845228112 learned_addr learned my addr 172.21.15.62:0/845228112 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:45.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e8c9d6700 1 --2- 172.21.15.62:0/845228112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e881066d0 0x7f2e88103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:45.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e7ffff700 1 --2- 172.21.15.62:0/845228112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e88102030 0x7f2e881035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:45.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e87fff700 1 -- 172.21.15.62:0/845228112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e88102030 msgr2=0x7f2e881035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:45.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e87fff700 1 --2- 172.21.15.62:0/845228112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e88102030 0x7f2e881035d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:45.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e87fff700 1 -- 172.21.15.62:0/845228112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e881066d0 msgr2=0x7f2e88103b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:45.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e87fff700 1 --2- 172.21.15.62:0/845228112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e881066d0 0x7f2e88103b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:45.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e87fff700 1 -- 172.21.15.62:0/845228112 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e881a7160 con 0x7f2e88101680 2024-06-10T00:28:45.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e87fff700 1 --2- 172.21.15.62:0/845228112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e88101680 0x7f2e88103090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009990 tx=0x7f2e78009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:45.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e85ffb700 1 -- 172.21.15.62:0/845228112 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78026070 con 0x7f2e88101680 2024-06-10T00:28:45.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e85ffb700 1 -- 172.21.15.62:0/845228112 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e78003780 con 0x7f2e88101680 2024-06-10T00:28:45.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e8e439700 1 -- 172.21.15.62:0/845228112 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e881a73f0 con 0x7f2e88101680 2024-06-10T00:28:45.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.502+0000 7f2e85ffb700 1 -- 172.21.15.62:0/845228112 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7800fc90 con 0x7f2e88101680 2024-06-10T00:28:45.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.503+0000 7f2e8e439700 1 -- 172.21.15.62:0/845228112 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e881a7840 con 0x7f2e88101680 2024-06-10T00:28:45.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.503+0000 7f2e8e439700 1 -- 172.21.15.62:0/845228112 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e88050a00 con 0x7f2e88101680 2024-06-10T00:28:45.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.505+0000 7f2e85ffb700 1 -- 172.21.15.62:0/845228112 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2e78029070 con 0x7f2e88101680 2024-06-10T00:28:45.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.505+0000 7f2e85ffb700 1 --2- 172.21.15.62:0/845228112 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2e740a2cc0 0x7f2e740a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:45.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.505+0000 7f2e7ffff700 1 --2- 172.21.15.62:0/845228112 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2e740a2cc0 0x7f2e740a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:45.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.505+0000 7f2e85ffb700 1 -- 172.21.15.62:0/845228112 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8277+0+0 (secure 0 0 0) 0x7f2e780cfaa0 con 0x7f2e88101680 2024-06-10T00:28:45.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.506+0000 7f2e7ffff700 1 --2- 172.21.15.62:0/845228112 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2e740a2cc0 0x7f2e740a5180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2e70005950 tx=0x7f2e700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:45.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.510+0000 7f2e85ffb700 1 -- 172.21.15.62:0/845228112 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2e7805d850 con 0x7f2e88101680 2024-06-10T00:28:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:45.716+0000 7f2e8e439700 1 -- 172.21.15.62:0/845228112 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2e8802dd40 con 0x7f2e88101680 2024-06-10T00:28:46.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:45 smithi105 ceph-mon[36321]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-06-10T00:28:46.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:45 smithi062 ceph-mon[32390]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-06-10T00:28:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:45 smithi138 ceph-mon[36549]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-06-10T00:28:47.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:46 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/845228112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-10T00:28:47.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:46 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/845228112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-10T00:28:47.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:46 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/845228112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-10T00:28:47.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:47.678+0000 7f2e85ffb700 1 -- 172.21.15.62:0/845228112 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v91) ==== 228+0+0 (secure 0 0 0) 0x7f2e78013070 con 0x7f2e88101680 2024-06-10T00:28:47.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:47.682+0000 7f2e8e439700 1 -- 172.21.15.62:0/845228112 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2e740a2cc0 msgr2=0x7f2e740a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:47.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:47.682+0000 7f2e8e439700 1 --2- 172.21.15.62:0/845228112 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2e740a2cc0 0x7f2e740a5180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2e70005950 tx=0x7f2e700058e0 comp rx=0 tx=0).stop 2024-06-10T00:28:47.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:47.682+0000 7f2e8e439700 1 -- 172.21.15.62:0/845228112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e88101680 msgr2=0x7f2e88103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:47.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:47.682+0000 7f2e8e439700 1 --2- 172.21.15.62:0/845228112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e88101680 0x7f2e88103090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009990 tx=0x7f2e78009f80 comp rx=0 tx=0).stop 2024-06-10T00:28:47.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:47.682+0000 7f2e8e439700 1 -- 172.21.15.62:0/845228112 shutdown_connections 2024-06-10T00:28:47.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:47.682+0000 7f2e8e439700 1 --2- 172.21.15.62:0/845228112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e88101680 0x7f2e88103090 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:47.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:47.682+0000 7f2e8e439700 1 --2- 172.21.15.62:0/845228112 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2e740a2cc0 0x7f2e740a5180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:47.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:47.682+0000 7f2e8e439700 1 --2- 172.21.15.62:0/845228112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e881066d0 0x7f2e88103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:47.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:47.682+0000 7f2e8e439700 1 --2- 172.21.15.62:0/845228112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e88102030 0x7f2e881035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:47.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:47.682+0000 7f2e8e439700 1 -- 172.21.15.62:0/845228112 >> 172.21.15.62:0/845228112 conn(0x7f2e880773e0 msgr2=0x7f2e8806ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:47.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:47.682+0000 7f2e8e439700 1 -- 172.21.15.62:0/845228112 shutdown_connections 2024-06-10T00:28:47.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:47.682+0000 7f2e8e439700 1 -- 172.21.15.62:0/845228112 wait complete. 2024-06-10T00:28:47.685 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data_ec' created 2024-06-10T00:28:48.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:47 smithi062 ceph-mon[32390]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-06-10T00:28:48.016 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:47 smithi062 ceph-mon[32390]: osdmap e90: 12 total, 12 up, 12 in 2024-06-10T00:28:48.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:47 smithi105 ceph-mon[36321]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-06-10T00:28:48.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:47 smithi105 ceph-mon[36321]: osdmap e90: 12 total, 12 up, 12 in 2024-06-10T00:28:48.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:47 smithi138 ceph-mon[36549]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-06-10T00:28:48.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:47 smithi138 ceph-mon[36549]: osdmap e90: 12 total, 12 up, 12 in 2024-06-10T00:28:48.453 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-06-10T00:28:48.728 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:48.974 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:48 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/845228112' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-10T00:28:48.975 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:48 smithi062 ceph-mon[32390]: osdmap e91: 12 total, 12 up, 12 in 2024-06-10T00:28:49.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:48 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/845228112' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-10T00:28:49.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:48 smithi105 ceph-mon[36321]: osdmap e91: 12 total, 12 up, 12 in 2024-06-10T00:28:49.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:48 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/845228112' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-10T00:28:49.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:48 smithi138 ceph-mon[36549]: osdmap e91: 12 total, 12 up, 12 in 2024-06-10T00:28:49.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:49 smithi062 ceph-mon[32390]: pgmap v304: 129 pgs: 12 creating+peering, 20 unknown, 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 5 op/s 2024-06-10T00:28:49.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:49 smithi062 ceph-mon[32390]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T00:28:49.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:49 smithi062 ceph-mon[32390]: osdmap e92: 12 total, 12 up, 12 in 2024-06-10T00:28:50.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:49 smithi105 ceph-mon[36321]: pgmap v304: 129 pgs: 12 creating+peering, 20 unknown, 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 5 op/s 2024-06-10T00:28:50.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:49 smithi105 ceph-mon[36321]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T00:28:50.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:49 smithi105 ceph-mon[36321]: osdmap e92: 12 total, 12 up, 12 in 2024-06-10T00:28:50.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:49 smithi138 ceph-mon[36549]: pgmap v304: 129 pgs: 12 creating+peering, 20 unknown, 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 5 op/s 2024-06-10T00:28:50.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:49 smithi138 ceph-mon[36549]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T00:28:50.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:49 smithi138 ceph-mon[36549]: osdmap e92: 12 total, 12 up, 12 in 2024-06-10T00:28:50.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.390+0000 7faba4494700 1 -- 172.21.15.62:0/2249920895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab9c107400 msgr2=0x7fab9c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:50.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.390+0000 7faba4494700 1 --2- 172.21.15.62:0/2249920895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab9c107400 0x7fab9c1077e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fab98009990 tx=0x7fab98009ca0 comp rx=0 tx=0).stop 2024-06-10T00:28:50.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.390+0000 7faba4494700 1 -- 172.21.15.62:0/2249920895 shutdown_connections 2024-06-10T00:28:50.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.390+0000 7faba4494700 1 --2- 172.21.15.62:0/2249920895 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab9c107400 0x7fab9c1077e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:50.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.390+0000 7faba4494700 1 --2- 172.21.15.62:0/2249920895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab9c101d50 0x7fab9c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:50.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.390+0000 7faba4494700 1 --2- 172.21.15.62:0/2249920895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab9c101390 0x7fab9c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:50.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.390+0000 7faba4494700 1 -- 172.21.15.62:0/2249920895 >> 172.21.15.62:0/2249920895 conn(0x7fab9c078340 msgr2=0x7fab9c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:50.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.390+0000 7faba4494700 1 -- 172.21.15.62:0/2249920895 shutdown_connections 2024-06-10T00:28:50.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.390+0000 7faba4494700 1 -- 172.21.15.62:0/2249920895 wait complete. 2024-06-10T00:28:50.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.391+0000 7faba4494700 1 Processor -- start 2024-06-10T00:28:50.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.391+0000 7faba4494700 1 -- start start 2024-06-10T00:28:50.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.391+0000 7faba4494700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab9c101390 0x7fab9c1082e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:50.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.391+0000 7faba4494700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab9c101d50 0x7fab9c108820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:50.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.391+0000 7faba4494700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab9c107400 0x7fab9c108d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:50.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.391+0000 7faba4494700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c111770 con 0x7fab9c101390 2024-06-10T00:28:50.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.391+0000 7faba4494700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1115f0 con 0x7fab9c101d50 2024-06-10T00:28:50.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.391+0000 7faba4494700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1118f0 con 0x7fab9c107400 2024-06-10T00:28:50.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.391+0000 7faba2230700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab9c101390 0x7fab9c1082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:50.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.391+0000 7faba2230700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab9c101390 0x7fab9c1082e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59472/0 (socket says 172.21.15.62:59472) 2024-06-10T00:28:50.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.391+0000 7faba2230700 1 -- 172.21.15.62:0/957203669 learned_addr learned my addr 172.21.15.62:0/957203669 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:50.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.392+0000 7faba2230700 1 -- 172.21.15.62:0/957203669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab9c107400 msgr2=0x7fab9c108d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:50.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.392+0000 7faba2a31700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab9c107400 0x7fab9c108d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:50.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.392+0000 7faba1a2f700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab9c101d50 0x7fab9c108820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:50.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.392+0000 7faba2230700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab9c107400 0x7fab9c108d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:50.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.392+0000 7faba2230700 1 -- 172.21.15.62:0/957203669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab9c101d50 msgr2=0x7fab9c108820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:50.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.392+0000 7faba2230700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab9c101d50 0x7fab9c108820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:50.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.392+0000 7faba2230700 1 -- 172.21.15.62:0/957203669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9c1a7420 con 0x7fab9c101390 2024-06-10T00:28:50.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.392+0000 7faba1a2f700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab9c101d50 0x7fab9c108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:28:50.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.392+0000 7faba2230700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab9c101390 0x7fab9c1082e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fab98000c00 tx=0x7fab98003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:50.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.392+0000 7fab8f7fe700 1 -- 172.21.15.62:0/957203669 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98027070 con 0x7fab9c101390 2024-06-10T00:28:50.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.392+0000 7fab8f7fe700 1 -- 172.21.15.62:0/957203669 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab98004190 con 0x7fab9c101390 2024-06-10T00:28:50.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.392+0000 7faba4494700 1 -- 172.21.15.62:0/957203669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9c1a76b0 con 0x7fab9c101390 2024-06-10T00:28:50.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.392+0000 7fab8f7fe700 1 -- 172.21.15.62:0/957203669 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9800f460 con 0x7fab9c101390 2024-06-10T00:28:50.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.392+0000 7faba4494700 1 -- 172.21.15.62:0/957203669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9c1a7b60 con 0x7fab9c101390 2024-06-10T00:28:50.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.393+0000 7faba4494700 1 -- 172.21.15.62:0/957203669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab9c069180 con 0x7fab9c101390 2024-06-10T00:28:50.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.394+0000 7fab8f7fe700 1 -- 172.21.15.62:0/957203669 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fab9802a070 con 0x7fab9c101390 2024-06-10T00:28:50.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.395+0000 7fab8f7fe700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fab880a2ca0 0x7fab880a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:50.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.395+0000 7faba1a2f700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fab880a2ca0 0x7fab880a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:50.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.395+0000 7fab8f7fe700 1 -- 172.21.15.62:0/957203669 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8770+0+0 (secure 0 0 0) 0x7fab980d0530 con 0x7fab9c101390 2024-06-10T00:28:50.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.396+0000 7faba1a2f700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fab880a2ca0 0x7fab880a5160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fab9c100f90 tx=0x7fab90008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:50.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.399+0000 7fab8f7fe700 1 -- 172.21.15.62:0/957203669 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fab98013070 con 0x7fab9c101390 2024-06-10T00:28:50.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.604+0000 7faba4494700 1 -- 172.21.15.62:0/957203669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fab9c1a7e80 con 0x7fab9c101390 2024-06-10T00:28:50.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.699+0000 7fab8f7fe700 1 -- 172.21.15.62:0/957203669 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v94) ==== 169+0+0 (secure 0 0 0) 0x7fab9809d300 con 0x7fab9c101390 2024-06-10T00:28:50.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.702+0000 7faba4494700 1 -- 172.21.15.62:0/957203669 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fab880a2ca0 msgr2=0x7fab880a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:50.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.702+0000 7faba4494700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fab880a2ca0 0x7fab880a5160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fab9c100f90 tx=0x7fab90008040 comp rx=0 tx=0).stop 2024-06-10T00:28:50.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.702+0000 7faba4494700 1 -- 172.21.15.62:0/957203669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab9c101390 msgr2=0x7fab9c1082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:50.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.702+0000 7faba4494700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab9c101390 0x7fab9c1082e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fab98000c00 tx=0x7fab98003730 comp rx=0 tx=0).stop 2024-06-10T00:28:50.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.702+0000 7faba4494700 1 -- 172.21.15.62:0/957203669 shutdown_connections 2024-06-10T00:28:50.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.702+0000 7faba4494700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab9c101390 0x7fab9c1082e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:50.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.702+0000 7faba4494700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fab880a2ca0 0x7fab880a5160 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:50.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.702+0000 7faba4494700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab9c107400 0x7fab9c108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:50.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.702+0000 7faba4494700 1 --2- 172.21.15.62:0/957203669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab9c101d50 0x7fab9c108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:50.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.702+0000 7faba4494700 1 -- 172.21.15.62:0/957203669 >> 172.21.15.62:0/957203669 conn(0x7fab9c078340 msgr2=0x7fab9c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:50.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.702+0000 7faba4494700 1 -- 172.21.15.62:0/957203669 shutdown_connections 2024-06-10T00:28:50.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:50.702+0000 7faba4494700 1 -- 172.21.15.62:0/957203669 wait complete. 2024-06-10T00:28:50.703 INFO:teuthology.orchestra.run.smithi062.stderr:set pool 4 allow_ec_overwrites to true 2024-06-10T00:28:51.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:50 smithi105 ceph-mon[36321]: osdmap e93: 12 total, 12 up, 12 in 2024-06-10T00:28:51.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:50 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/957203669' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-10T00:28:51.076 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:50 smithi062 ceph-mon[32390]: osdmap e93: 12 total, 12 up, 12 in 2024-06-10T00:28:51.077 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:50 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/957203669' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-10T00:28:51.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:50 smithi138 ceph-mon[36549]: osdmap e93: 12 total, 12 up, 12 in 2024-06-10T00:28:51.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:50 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/957203669' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-10T00:28:51.423 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-06-10T00:28:51.676 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:52.005 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:51 smithi062 ceph-mon[32390]: pgmap v307: 129 pgs: 12 creating+peering, 15 unknown, 102 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:52.005 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:51 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/957203669' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-10T00:28:52.005 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:51 smithi062 ceph-mon[32390]: osdmap e94: 12 total, 12 up, 12 in 2024-06-10T00:28:52.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:51 smithi105 ceph-mon[36321]: pgmap v307: 129 pgs: 12 creating+peering, 15 unknown, 102 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:52.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:51 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/957203669' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-10T00:28:52.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:51 smithi105 ceph-mon[36321]: osdmap e94: 12 total, 12 up, 12 in 2024-06-10T00:28:52.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:51 smithi138 ceph-mon[36549]: pgmap v307: 129 pgs: 12 creating+peering, 15 unknown, 102 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:52.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:51 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/957203669' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-10T00:28:52.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:51 smithi138 ceph-mon[36549]: osdmap e94: 12 total, 12 up, 12 in 2024-06-10T00:28:53.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.313+0000 7f8d385e2700 1 -- 172.21.15.62:0/3686053033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d30101d50 msgr2=0x7f8d3010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:53.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.313+0000 7f8d385e2700 1 --2- 172.21.15.62:0/3686053033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d30101d50 0x7f8d3010e240 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800b0a0 tx=0x7f8d2800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:28:53.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.313+0000 7f8d385e2700 1 -- 172.21.15.62:0/3686053033 shutdown_connections 2024-06-10T00:28:53.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.313+0000 7f8d385e2700 1 --2- 172.21.15.62:0/3686053033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d30101d50 0x7f8d3010e240 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:53.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.313+0000 7f8d385e2700 1 --2- 172.21.15.62:0/3686053033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d30107400 0x7f8d301077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:53.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.313+0000 7f8d385e2700 1 --2- 172.21.15.62:0/3686053033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d30101390 0x7f8d30101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:53.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.313+0000 7f8d385e2700 1 -- 172.21.15.62:0/3686053033 >> 172.21.15.62:0/3686053033 conn(0x7f8d30078340 msgr2=0x7f8d30078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:53.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.313+0000 7f8d385e2700 1 -- 172.21.15.62:0/3686053033 shutdown_connections 2024-06-10T00:28:53.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.313+0000 7f8d385e2700 1 -- 172.21.15.62:0/3686053033 wait complete. 2024-06-10T00:28:53.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.314+0000 7f8d385e2700 1 Processor -- start 2024-06-10T00:28:53.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.314+0000 7f8d385e2700 1 -- start start 2024-06-10T00:28:53.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d385e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d30101390 0x7f8d30199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:53.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d385e2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d30101d50 0x7f8d30199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:53.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d385e2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d30107400 0x7f8d301a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:53.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d385e2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d301116a0 con 0x7f8d30101390 2024-06-10T00:28:53.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d385e2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d30111520 con 0x7f8d30107400 2024-06-10T00:28:53.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d385e2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d30111820 con 0x7f8d30101d50 2024-06-10T00:28:53.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d3637e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d30101390 0x7f8d30199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:53.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d3637e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d30101390 0x7f8d30199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59478/0 (socket says 172.21.15.62:59478) 2024-06-10T00:28:53.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d3637e700 1 -- 172.21.15.62:0/3447834738 learned_addr learned my addr 172.21.15.62:0/3447834738 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:53.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d35b7d700 1 --2- 172.21.15.62:0/3447834738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d30101d50 0x7f8d30199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:53.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d36b7f700 1 --2- 172.21.15.62:0/3447834738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d30107400 0x7f8d301a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:53.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d3637e700 1 -- 172.21.15.62:0/3447834738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d30101d50 msgr2=0x7f8d30199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:53.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d3637e700 1 --2- 172.21.15.62:0/3447834738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d30101d50 0x7f8d30199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:53.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d3637e700 1 -- 172.21.15.62:0/3447834738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d30107400 msgr2=0x7f8d301a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:53.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d3637e700 1 --2- 172.21.15.62:0/3447834738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d30107400 0x7f8d301a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:53.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d3637e700 1 -- 172.21.15.62:0/3447834738 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d301a5000 con 0x7f8d30101390 2024-06-10T00:28:53.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d3637e700 1 --2- 172.21.15.62:0/3447834738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d30101390 0x7f8d30199a40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400c9b0 tx=0x7f8d2400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:53.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.315+0000 7f8d237fe700 1 -- 172.21.15.62:0/3447834738 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24007b00 con 0x7f8d30101390 2024-06-10T00:28:53.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.316+0000 7f8d385e2700 1 -- 172.21.15.62:0/3447834738 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d30197890 con 0x7f8d30101390 2024-06-10T00:28:53.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.316+0000 7f8d237fe700 1 -- 172.21.15.62:0/3447834738 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d24007ca0 con 0x7f8d30101390 2024-06-10T00:28:53.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.316+0000 7f8d237fe700 1 -- 172.21.15.62:0/3447834738 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24018730 con 0x7f8d30101390 2024-06-10T00:28:53.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.316+0000 7f8d385e2700 1 -- 172.21.15.62:0/3447834738 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d30197e20 con 0x7f8d30101390 2024-06-10T00:28:53.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.317+0000 7f8d385e2700 1 -- 172.21.15.62:0/3447834738 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d30069180 con 0x7f8d30101390 2024-06-10T00:28:53.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.318+0000 7f8d237fe700 1 -- 172.21.15.62:0/3447834738 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8d240188d0 con 0x7f8d30101390 2024-06-10T00:28:53.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.319+0000 7f8d237fe700 1 --2- 172.21.15.62:0/3447834738 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8d1c0a2c20 0x7f8d1c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:53.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.319+0000 7f8d35b7d700 1 --2- 172.21.15.62:0/3447834738 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8d1c0a2c20 0x7f8d1c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:53.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.319+0000 7f8d237fe700 1 -- 172.21.15.62:0/3447834738 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8770+0+0 (secure 0 0 0) 0x7f8d240cf1f0 con 0x7f8d30101390 2024-06-10T00:28:53.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.319+0000 7f8d35b7d700 1 --2- 172.21.15.62:0/3447834738 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8d1c0a2c20 0x7f8d1c0a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8d300788a0 tx=0x7f8d2c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:53.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.323+0000 7f8d237fe700 1 -- 172.21.15.62:0/3447834738 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8d24014030 con 0x7f8d30101390 2024-06-10T00:28:53.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.576+0000 7f8d385e2700 1 -- 172.21.15.62:0/3447834738 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f8d3006a760 con 0x7f8d30101390 2024-06-10T00:28:53.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.709+0000 7f8d237fe700 1 -- 172.21.15.62:0/3447834738 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f8d2409bf30 con 0x7f8d30101390 2024-06-10T00:28:53.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.711+0000 7f8d385e2700 1 -- 172.21.15.62:0/3447834738 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8d1c0a2c20 msgr2=0x7f8d1c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:53.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.711+0000 7f8d385e2700 1 --2- 172.21.15.62:0/3447834738 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8d1c0a2c20 0x7f8d1c0a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8d300788a0 tx=0x7f8d2c00b3c0 comp rx=0 tx=0).stop 2024-06-10T00:28:53.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.711+0000 7f8d385e2700 1 -- 172.21.15.62:0/3447834738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d30101390 msgr2=0x7f8d30199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:53.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.711+0000 7f8d385e2700 1 --2- 172.21.15.62:0/3447834738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d30101390 0x7f8d30199a40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400c9b0 tx=0x7f8d2400ccc0 comp rx=0 tx=0).stop 2024-06-10T00:28:53.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.712+0000 7f8d385e2700 1 -- 172.21.15.62:0/3447834738 shutdown_connections 2024-06-10T00:28:53.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.712+0000 7f8d385e2700 1 --2- 172.21.15.62:0/3447834738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d30101390 0x7f8d30199a40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:53.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.712+0000 7f8d385e2700 1 --2- 172.21.15.62:0/3447834738 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8d1c0a2c20 0x7f8d1c0a50e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:53.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.712+0000 7f8d385e2700 1 --2- 172.21.15.62:0/3447834738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d30107400 0x7f8d301a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:53.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.712+0000 7f8d385e2700 1 --2- 172.21.15.62:0/3447834738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d30101d50 0x7f8d30199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:53.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.712+0000 7f8d385e2700 1 -- 172.21.15.62:0/3447834738 >> 172.21.15.62:0/3447834738 conn(0x7f8d30078340 msgr2=0x7f8d300ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:53.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.712+0000 7f8d385e2700 1 -- 172.21.15.62:0/3447834738 shutdown_connections 2024-06-10T00:28:53.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:53.712+0000 7f8d385e2700 1 -- 172.21.15.62:0/3447834738 wait complete. 2024-06-10T00:28:53.713 INFO:teuthology.orchestra.run.smithi062.stderr:added data pool 4 to fsmap 2024-06-10T00:28:53.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:53 smithi062 ceph-mon[32390]: pgmap v309: 129 pgs: 12 creating+peering, 3 unknown, 114 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:53.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:53 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3447834738' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-10T00:28:54.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:53 smithi105 ceph-mon[36321]: pgmap v309: 129 pgs: 12 creating+peering, 3 unknown, 114 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:54.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:53 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3447834738' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-10T00:28:54.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:53 smithi138 ceph-mon[36549]: pgmap v309: 129 pgs: 12 creating+peering, 3 unknown, 114 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:54.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:53 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3447834738' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-10T00:28:54.396 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:28:54.644 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:54.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:54 smithi062 ceph-mon[32390]: osdmap e95: 12 total, 12 up, 12 in 2024-06-10T00:28:54.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:54 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3447834738' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-10T00:28:54.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:54 smithi062 ceph-mon[32390]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:28:55.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:54 smithi105 ceph-mon[36321]: osdmap e95: 12 total, 12 up, 12 in 2024-06-10T00:28:55.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:54 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3447834738' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-10T00:28:55.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:54 smithi105 ceph-mon[36321]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:28:55.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:54 smithi138 ceph-mon[36549]: osdmap e95: 12 total, 12 up, 12 in 2024-06-10T00:28:55.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:54 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3447834738' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-10T00:28:55.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:54 smithi138 ceph-mon[36549]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:28:55.971 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:55 smithi062 ceph-mon[32390]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:55.971 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:55 smithi062 ceph-mon[32390]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T00:28:55.971 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:55 smithi062 ceph-mon[32390]: Cluster is now healthy 2024-06-10T00:28:56.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:55 smithi105 ceph-mon[36321]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:56.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:55 smithi105 ceph-mon[36321]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T00:28:56.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:55 smithi105 ceph-mon[36321]: Cluster is now healthy 2024-06-10T00:28:56.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:55 smithi138 ceph-mon[36549]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:56.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:55 smithi138 ceph-mon[36549]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T00:28:56.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:55 smithi138 ceph-mon[36549]: Cluster is now healthy 2024-06-10T00:28:56.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.538+0000 7fee453f2700 1 -- 172.21.15.62:0/3153814636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee40107840 msgr2=0x7fee4010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:56.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.538+0000 7fee453f2700 1 --2- 172.21.15.62:0/3153814636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee40107840 0x7fee4010f3c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fee3800b0d0 tx=0x7fee3800b3e0 comp rx=0 tx=0).stop 2024-06-10T00:28:56.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.538+0000 7fee453f2700 1 -- 172.21.15.62:0/3153814636 shutdown_connections 2024-06-10T00:28:56.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.538+0000 7fee453f2700 1 --2- 172.21.15.62:0/3153814636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee40107840 0x7fee4010f3c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:56.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.538+0000 7fee453f2700 1 --2- 172.21.15.62:0/3153814636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee40102740 0x7fee40102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:56.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.538+0000 7fee453f2700 1 --2- 172.21.15.62:0/3153814636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee401030f0 0x7fee40107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:56.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.538+0000 7fee453f2700 1 -- 172.21.15.62:0/3153814636 >> 172.21.15.62:0/3153814636 conn(0x7fee400795f0 msgr2=0x7fee40079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:56.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.538+0000 7fee453f2700 1 -- 172.21.15.62:0/3153814636 shutdown_connections 2024-06-10T00:28:56.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.538+0000 7fee453f2700 1 -- 172.21.15.62:0/3153814636 wait complete. 2024-06-10T00:28:56.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.539+0000 7fee453f2700 1 Processor -- start 2024-06-10T00:28:56.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.539+0000 7fee453f2700 1 -- start start 2024-06-10T00:28:56.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.539+0000 7fee453f2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee40102740 0x7fee4019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:56.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.539+0000 7fee453f2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee401030f0 0x7fee4019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:56.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.539+0000 7fee453f2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee40107840 0x7fee40198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:56.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.539+0000 7fee453f2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fee40112780 con 0x7fee401030f0 2024-06-10T00:28:56.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.539+0000 7fee453f2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fee40112600 con 0x7fee40102740 2024-06-10T00:28:56.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.539+0000 7fee453f2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee40112900 con 0x7fee40107840 2024-06-10T00:28:56.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee3e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee401030f0 0x7fee4019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:56.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee3e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee401030f0 0x7fee4019ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59486/0 (socket says 172.21.15.62:59486) 2024-06-10T00:28:56.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee3e7fc700 1 -- 172.21.15.62:0/3960199520 learned_addr learned my addr 172.21.15.62:0/3960199520 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:56.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee3f7fe700 1 --2- 172.21.15.62:0/3960199520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee40107840 0x7fee40198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:56.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee3effd700 1 --2- 172.21.15.62:0/3960199520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee40102740 0x7fee4019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:56.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee3e7fc700 1 -- 172.21.15.62:0/3960199520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee40107840 msgr2=0x7fee40198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:56.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee3e7fc700 1 --2- 172.21.15.62:0/3960199520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee40107840 0x7fee40198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:56.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee3e7fc700 1 -- 172.21.15.62:0/3960199520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee40102740 msgr2=0x7fee4019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:56.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee3e7fc700 1 --2- 172.21.15.62:0/3960199520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee40102740 0x7fee4019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:56.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee3e7fc700 1 -- 172.21.15.62:0/3960199520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee401990a0 con 0x7fee401030f0 2024-06-10T00:28:56.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee3e7fc700 1 --2- 172.21.15.62:0/3960199520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee401030f0 0x7fee4019ecc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee2800d980 tx=0x7fee2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:56.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee37fff700 1 -- 172.21.15.62:0/3960199520 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28017070 con 0x7fee401030f0 2024-06-10T00:28:56.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee453f2700 1 -- 172.21.15.62:0/3960199520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee40199390 con 0x7fee401030f0 2024-06-10T00:28:56.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee37fff700 1 -- 172.21.15.62:0/3960199520 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee28005020 con 0x7fee401030f0 2024-06-10T00:28:56.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee37fff700 1 -- 172.21.15.62:0/3960199520 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28009bc0 con 0x7fee401030f0 2024-06-10T00:28:56.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.540+0000 7fee453f2700 1 -- 172.21.15.62:0/3960199520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee401a5650 con 0x7fee401030f0 2024-06-10T00:28:56.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.541+0000 7fee453f2700 1 -- 172.21.15.62:0/3960199520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee40069180 con 0x7fee401030f0 2024-06-10T00:28:56.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.542+0000 7fee37fff700 1 -- 172.21.15.62:0/3960199520 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fee280051d0 con 0x7fee401030f0 2024-06-10T00:28:56.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.543+0000 7fee37fff700 1 --2- 172.21.15.62:0/3960199520 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fee2c0a2c20 0x7fee2c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:56.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.543+0000 7fee3effd700 1 --2- 172.21.15.62:0/3960199520 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fee2c0a2c20 0x7fee2c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:56.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.543+0000 7fee37fff700 1 -- 172.21.15.62:0/3960199520 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fee280cf1b0 con 0x7fee401030f0 2024-06-10T00:28:56.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.544+0000 7fee3effd700 1 --2- 172.21.15.62:0/3960199520 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fee2c0a2c20 0x7fee2c0a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fee30005950 tx=0x7fee30005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:56.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.548+0000 7fee37fff700 1 -- 172.21.15.62:0/3960199520 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fee28020020 con 0x7fee401030f0 2024-06-10T00:28:56.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.800+0000 7fee453f2700 1 -- 172.21.15.62:0/3960199520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee401a58e0 con 0x7fee401030f0 2024-06-10T00:28:56.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.801+0000 7fee37fff700 1 -- 172.21.15.62:0/3960199520 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11803 (secure 0 0 0) 0x7fee2809bed0 con 0x7fee401030f0 2024-06-10T00:28:56.801 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:28:56.801 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:28:53.578096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24464,"qdb_cluster":[24464]},"id":1}]} 2024-06-10T00:28:56.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.803+0000 7fee453f2700 1 -- 172.21.15.62:0/3960199520 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fee2c0a2c20 msgr2=0x7fee2c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:56.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.803+0000 7fee453f2700 1 --2- 172.21.15.62:0/3960199520 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fee2c0a2c20 0x7fee2c0a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fee30005950 tx=0x7fee30005ed0 comp rx=0 tx=0).stop 2024-06-10T00:28:56.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.803+0000 7fee453f2700 1 -- 172.21.15.62:0/3960199520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee401030f0 msgr2=0x7fee4019ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:56.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.803+0000 7fee453f2700 1 --2- 172.21.15.62:0/3960199520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee401030f0 0x7fee4019ecc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee2800d980 tx=0x7fee2800dc90 comp rx=0 tx=0).stop 2024-06-10T00:28:56.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.803+0000 7fee453f2700 1 -- 172.21.15.62:0/3960199520 shutdown_connections 2024-06-10T00:28:56.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.803+0000 7fee453f2700 1 --2- 172.21.15.62:0/3960199520 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fee2c0a2c20 0x7fee2c0a50e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:56.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.803+0000 7fee453f2700 1 --2- 172.21.15.62:0/3960199520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee40107840 0x7fee40198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:56.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.803+0000 7fee453f2700 1 --2- 172.21.15.62:0/3960199520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee40102740 0x7fee4019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:56.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.803+0000 7fee453f2700 1 --2- 172.21.15.62:0/3960199520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee401030f0 0x7fee4019ecc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:56.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.803+0000 7fee453f2700 1 -- 172.21.15.62:0/3960199520 >> 172.21.15.62:0/3960199520 conn(0x7fee400795f0 msgr2=0x7fee40100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:56.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.804+0000 7fee453f2700 1 -- 172.21.15.62:0/3960199520 shutdown_connections 2024-06-10T00:28:56.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:56.804+0000 7fee453f2700 1 -- 172.21.15.62:0/3960199520 wait complete. 2024-06-10T00:28:56.805 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 17 2024-06-10T00:28:57.529 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:28:57.782 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:28:57.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:57 smithi062 ceph-mon[32390]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:57.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:57 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3960199520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:58.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:57 smithi105 ceph-mon[36321]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:58.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:57 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3960199520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:58.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:57 smithi138 ceph-mon[36549]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:28:58.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:57 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3960199520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:28:59.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.640+0000 7f9a4ab68700 1 -- 172.21.15.62:0/3865039803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a441027a0 msgr2=0x7f9a44102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:59.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.640+0000 7f9a4ab68700 1 --2- 172.21.15.62:0/3865039803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a441027a0 0x7f9a44102b80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a380099c0 tx=0x7f9a38009cd0 comp rx=0 tx=0).stop 2024-06-10T00:28:59.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.640+0000 7f9a4ab68700 1 -- 172.21.15.62:0/3865039803 shutdown_connections 2024-06-10T00:28:59.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.640+0000 7f9a4ab68700 1 --2- 172.21.15.62:0/3865039803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a441027a0 0x7f9a44102b80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:59.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.640+0000 7f9a4ab68700 1 --2- 172.21.15.62:0/3865039803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a44107870 0x7f9a4410f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:59.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.640+0000 7f9a4ab68700 1 --2- 172.21.15.62:0/3865039803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a44103150 0x7f9a44107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:59.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.640+0000 7f9a4ab68700 1 -- 172.21.15.62:0/3865039803 >> 172.21.15.62:0/3865039803 conn(0x7f9a44079570 msgr2=0x7f9a44079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:59.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.640+0000 7f9a4ab68700 1 -- 172.21.15.62:0/3865039803 shutdown_connections 2024-06-10T00:28:59.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.640+0000 7f9a4ab68700 1 -- 172.21.15.62:0/3865039803 wait complete. 2024-06-10T00:28:59.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a4ab68700 1 Processor -- start 2024-06-10T00:28:59.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a4ab68700 1 -- start start 2024-06-10T00:28:59.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a4ab68700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a441027a0 0x7f9a4419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:59.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a4ab68700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a44103150 0x7f9a4419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:59.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a4ab68700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a44107870 0x7f9a441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a4ab68700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44112790 con 0x7f9a441027a0 2024-06-10T00:28:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a4ab68700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44112610 con 0x7f9a44103150 2024-06-10T00:28:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a4ab68700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44112910 con 0x7f9a44107870 2024-06-10T00:28:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a48904700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a441027a0 0x7f9a4419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a48904700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a441027a0 0x7f9a4419e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49930/0 (socket says 172.21.15.62:49930) 2024-06-10T00:28:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a48904700 1 -- 172.21.15.62:0/233669455 learned_addr learned my addr 172.21.15.62:0/233669455 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:28:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a49105700 1 --2- 172.21.15.62:0/233669455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a44107870 0x7f9a441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a43fff700 1 --2- 172.21.15.62:0/233669455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a44103150 0x7f9a4419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a48904700 1 -- 172.21.15.62:0/233669455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a44107870 msgr2=0x7f9a441989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a48904700 1 --2- 172.21.15.62:0/233669455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a44107870 0x7f9a441989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a48904700 1 -- 172.21.15.62:0/233669455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a44103150 msgr2=0x7f9a4419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:59.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a48904700 1 --2- 172.21.15.62:0/233669455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a44103150 0x7f9a4419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:59.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.641+0000 7f9a48904700 1 -- 172.21.15.62:0/233669455 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a44199110 con 0x7f9a441027a0 2024-06-10T00:28:59.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.642+0000 7f9a48904700 1 --2- 172.21.15.62:0/233669455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a441027a0 0x7f9a4419e7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a38006010 tx=0x7f9a380038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:59.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.642+0000 7f9a41ffb700 1 -- 172.21.15.62:0/233669455 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38027070 con 0x7f9a441027a0 2024-06-10T00:28:59.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.642+0000 7f9a41ffb700 1 -- 172.21.15.62:0/233669455 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a38004050 con 0x7f9a441027a0 2024-06-10T00:28:59.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.642+0000 7f9a4ab68700 1 -- 172.21.15.62:0/233669455 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a441993a0 con 0x7f9a441027a0 2024-06-10T00:28:59.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.642+0000 7f9a41ffb700 1 -- 172.21.15.62:0/233669455 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38022d40 con 0x7f9a441027a0 2024-06-10T00:28:59.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.642+0000 7f9a4ab68700 1 -- 172.21.15.62:0/233669455 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a441a5670 con 0x7f9a441027a0 2024-06-10T00:28:59.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.643+0000 7f9a4ab68700 1 -- 172.21.15.62:0/233669455 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a44069180 con 0x7f9a441027a0 2024-06-10T00:28:59.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.644+0000 7f9a41ffb700 1 -- 172.21.15.62:0/233669455 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9a3802a070 con 0x7f9a441027a0 2024-06-10T00:28:59.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.645+0000 7f9a41ffb700 1 --2- 172.21.15.62:0/233669455 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9a340a2ca0 0x7f9a340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:28:59.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.645+0000 7f9a43fff700 1 --2- 172.21.15.62:0/233669455 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9a340a2ca0 0x7f9a340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:28:59.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.646+0000 7f9a41ffb700 1 -- 172.21.15.62:0/233669455 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f9a380d08a0 con 0x7f9a441027a0 2024-06-10T00:28:59.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.646+0000 7f9a43fff700 1 --2- 172.21.15.62:0/233669455 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9a340a2ca0 0x7f9a340a5160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9a44108b70 tx=0x7f9a3000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:28:59.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.649+0000 7f9a41ffb700 1 -- 172.21.15.62:0/233669455 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9a38031020 con 0x7f9a441027a0 2024-06-10T00:28:59.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.856+0000 7f9a4ab68700 1 -- 172.21.15.62:0/233669455 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a4402dca0 con 0x7f9a441027a0 2024-06-10T00:28:59.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.857+0000 7f9a41ffb700 1 -- 172.21.15.62:0/233669455 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24126 (secure 0 0 0) 0x7f9a38013070 con 0x7f9a441027a0 2024-06-10T00:28:59.857 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:28:59.858 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":95,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:28:53.577919+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:28:59.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.859+0000 7f9a4ab68700 1 -- 172.21.15.62:0/233669455 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9a340a2ca0 msgr2=0x7f9a340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:59.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.859+0000 7f9a4ab68700 1 --2- 172.21.15.62:0/233669455 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9a340a2ca0 0x7f9a340a5160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9a44108b70 tx=0x7f9a3000a300 comp rx=0 tx=0).stop 2024-06-10T00:28:59.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.859+0000 7f9a4ab68700 1 -- 172.21.15.62:0/233669455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a441027a0 msgr2=0x7f9a4419e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:28:59.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.859+0000 7f9a4ab68700 1 --2- 172.21.15.62:0/233669455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a441027a0 0x7f9a4419e7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a38006010 tx=0x7f9a380038b0 comp rx=0 tx=0).stop 2024-06-10T00:28:59.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.860+0000 7f9a4ab68700 1 -- 172.21.15.62:0/233669455 shutdown_connections 2024-06-10T00:28:59.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.860+0000 7f9a4ab68700 1 --2- 172.21.15.62:0/233669455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9a441027a0 0x7f9a4419e7a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:59.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.860+0000 7f9a4ab68700 1 --2- 172.21.15.62:0/233669455 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9a340a2ca0 0x7f9a340a5160 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:59.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.860+0000 7f9a4ab68700 1 --2- 172.21.15.62:0/233669455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a44107870 0x7f9a441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:59.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.860+0000 7f9a4ab68700 1 --2- 172.21.15.62:0/233669455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a44103150 0x7f9a4419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:28:59.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.860+0000 7f9a4ab68700 1 -- 172.21.15.62:0/233669455 >> 172.21.15.62:0/233669455 conn(0x7f9a44079570 msgr2=0x7f9a44100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:28:59.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.860+0000 7f9a4ab68700 1 -- 172.21.15.62:0/233669455 shutdown_connections 2024-06-10T00:28:59.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:28:59.860+0000 7f9a4ab68700 1 -- 172.21.15.62:0/233669455 wait complete. 2024-06-10T00:29:00.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:28:59 smithi105 ceph-mon[36321]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:00.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:28:59 smithi062 ceph-mon[32390]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:00.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:28:59 smithi138 ceph-mon[36549]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:00.640 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:29:00.892 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:01.015 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:00 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/233669455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:01.024 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:00 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/233669455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:01.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:00 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/233669455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:02.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:01 smithi105 ceph-mon[36321]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:02.100 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:01 smithi062 ceph-mon[32390]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:02.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:01 smithi138 ceph-mon[36549]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:02.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.651+0000 7ff56c769700 1 -- 172.21.15.62:0/3069983178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff564105f30 msgr2=0x7ff56410da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:02.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.651+0000 7ff56c769700 1 --2- 172.21.15.62:0/3069983178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff564105f30 0x7ff56410da70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00c210 tx=0x7ff55c00c520 comp rx=0 tx=0).stop 2024-06-10T00:29:02.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.651+0000 7ff56c769700 1 -- 172.21.15.62:0/3069983178 shutdown_connections 2024-06-10T00:29:02.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.651+0000 7ff56c769700 1 --2- 172.21.15.62:0/3069983178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff564105f30 0x7ff56410da70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:02.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.651+0000 7ff56c769700 1 --2- 172.21.15.62:0/3069983178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff564101090 0x7ff564101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:02.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.651+0000 7ff56c769700 1 --2- 172.21.15.62:0/3069983178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff564101a40 0x7ff564105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:02.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.651+0000 7ff56c769700 1 -- 172.21.15.62:0/3069983178 >> 172.21.15.62:0/3069983178 conn(0x7ff5640766b0 msgr2=0x7ff564076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:02.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.651+0000 7ff56c769700 1 -- 172.21.15.62:0/3069983178 shutdown_connections 2024-06-10T00:29:02.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.651+0000 7ff56c769700 1 -- 172.21.15.62:0/3069983178 wait complete. 2024-06-10T00:29:02.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.652+0000 7ff56c769700 1 Processor -- start 2024-06-10T00:29:02.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.652+0000 7ff56c769700 1 -- start start 2024-06-10T00:29:02.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.652+0000 7ff56c769700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff564101090 0x7ff56419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:02.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.652+0000 7ff56c769700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff564101a40 0x7ff56419cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:02.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.652+0000 7ff56c769700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff564105f30 0x7ff564196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:02.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.652+0000 7ff56c769700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff564110630 con 0x7ff564101a40 2024-06-10T00:29:02.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.652+0000 7ff56c769700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5641104b0 con 0x7ff564101090 2024-06-10T00:29:02.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.652+0000 7ff56c769700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5641107b0 con 0x7ff564105f30 2024-06-10T00:29:02.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.652+0000 7ff569d04700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff564101a40 0x7ff56419cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:02.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.652+0000 7ff569d04700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff564101a40 0x7ff56419cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49948/0 (socket says 172.21.15.62:49948) 2024-06-10T00:29:02.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.652+0000 7ff569d04700 1 -- 172.21.15.62:0/2173129353 learned_addr learned my addr 172.21.15.62:0/2173129353 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:02.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.652+0000 7ff56a505700 1 --2- 172.21.15.62:0/2173129353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff564101090 0x7ff56419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:02.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.652+0000 7ff56ad06700 1 --2- 172.21.15.62:0/2173129353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff564105f30 0x7ff564196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:02.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.653+0000 7ff569d04700 1 -- 172.21.15.62:0/2173129353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff564105f30 msgr2=0x7ff564196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:02.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.653+0000 7ff569d04700 1 --2- 172.21.15.62:0/2173129353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff564105f30 0x7ff564196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:02.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.653+0000 7ff569d04700 1 -- 172.21.15.62:0/2173129353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff564101090 msgr2=0x7ff56419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:02.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.653+0000 7ff569d04700 1 --2- 172.21.15.62:0/2173129353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff564101090 0x7ff56419c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:02.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.653+0000 7ff569d04700 1 -- 172.21.15.62:0/2173129353 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff564196f70 con 0x7ff564101a40 2024-06-10T00:29:02.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.653+0000 7ff569d04700 1 --2- 172.21.15.62:0/2173129353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff564101a40 0x7ff56419cb50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff55400b4d0 tx=0x7ff55400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:02.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.653+0000 7ff55b7fe700 1 -- 172.21.15.62:0/2173129353 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554015020 con 0x7ff564101a40 2024-06-10T00:29:02.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.653+0000 7ff55b7fe700 1 -- 172.21.15.62:0/2173129353 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff554011d10 con 0x7ff564101a40 2024-06-10T00:29:02.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.653+0000 7ff56c769700 1 -- 172.21.15.62:0/2173129353 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff564197260 con 0x7ff564101a40 2024-06-10T00:29:02.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.653+0000 7ff55b7fe700 1 -- 172.21.15.62:0/2173129353 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554010d50 con 0x7ff564101a40 2024-06-10T00:29:02.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.653+0000 7ff56c769700 1 -- 172.21.15.62:0/2173129353 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5641a34f0 con 0x7ff564101a40 2024-06-10T00:29:02.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.654+0000 7ff56c769700 1 -- 172.21.15.62:0/2173129353 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff564102270 con 0x7ff564101a40 2024-06-10T00:29:02.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.656+0000 7ff55b7fe700 1 -- 172.21.15.62:0/2173129353 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff554018020 con 0x7ff564101a40 2024-06-10T00:29:02.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.656+0000 7ff55b7fe700 1 --2- 172.21.15.62:0/2173129353 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff5500a4e30 0x7ff5500a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:02.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.656+0000 7ff56a505700 1 --2- 172.21.15.62:0/2173129353 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff5500a4e30 0x7ff5500a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:02.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.656+0000 7ff55b7fe700 1 -- 172.21.15.62:0/2173129353 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7ff5540ced30 con 0x7ff564101a40 2024-06-10T00:29:02.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.657+0000 7ff56a505700 1 --2- 172.21.15.62:0/2173129353 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff5500a4e30 0x7ff5500a72f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff560005950 tx=0x7ff56000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:02.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.660+0000 7ff55b7fe700 1 -- 172.21.15.62:0/2173129353 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff55401f020 con 0x7ff564101a40 2024-06-10T00:29:02.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.870+0000 7ff56c769700 1 -- 172.21.15.62:0/2173129353 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff56402dca0 con 0x7ff564101a40 2024-06-10T00:29:02.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.871+0000 7ff55b7fe700 1 -- 172.21.15.62:0/2173129353 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24126 (secure 0 0 0) 0x7ff55409ba50 con 0x7ff564101a40 2024-06-10T00:29:02.872 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:29:02.872 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":95,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:28:53.577919+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:29:02.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.874+0000 7ff56c769700 1 -- 172.21.15.62:0/2173129353 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff5500a4e30 msgr2=0x7ff5500a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:02.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.874+0000 7ff56c769700 1 --2- 172.21.15.62:0/2173129353 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff5500a4e30 0x7ff5500a72f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff560005950 tx=0x7ff56000b410 comp rx=0 tx=0).stop 2024-06-10T00:29:02.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.874+0000 7ff56c769700 1 -- 172.21.15.62:0/2173129353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff564101a40 msgr2=0x7ff56419cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:02.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.874+0000 7ff56c769700 1 --2- 172.21.15.62:0/2173129353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff564101a40 0x7ff56419cb50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff55400b4d0 tx=0x7ff55400b7e0 comp rx=0 tx=0).stop 2024-06-10T00:29:02.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.874+0000 7ff56c769700 1 -- 172.21.15.62:0/2173129353 shutdown_connections 2024-06-10T00:29:02.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.874+0000 7ff56c769700 1 --2- 172.21.15.62:0/2173129353 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff5500a4e30 0x7ff5500a72f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:02.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.874+0000 7ff56c769700 1 --2- 172.21.15.62:0/2173129353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff564105f30 0x7ff564196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:02.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.874+0000 7ff56c769700 1 --2- 172.21.15.62:0/2173129353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff564101090 0x7ff56419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:02.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.874+0000 7ff56c769700 1 --2- 172.21.15.62:0/2173129353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff564101a40 0x7ff56419cb50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:02.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.874+0000 7ff56c769700 1 -- 172.21.15.62:0/2173129353 >> 172.21.15.62:0/2173129353 conn(0x7ff5640766b0 msgr2=0x7ff5640ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:02.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.874+0000 7ff56c769700 1 -- 172.21.15.62:0/2173129353 shutdown_connections 2024-06-10T00:29:02.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:02.874+0000 7ff56c769700 1 -- 172.21.15.62:0/2173129353 wait complete. 2024-06-10T00:29:03.701 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:29:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:04.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:03 smithi105 ceph-mon[36321]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:04.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:03 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2173129353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:04.077 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:03 smithi062 ceph-mon[32390]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:04.077 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:03 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2173129353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:04.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:03 smithi138 ceph-mon[36549]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:04.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:03 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2173129353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:05.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.611+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3691190383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d4006b620 msgr2=0x7f7d4010f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:05.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.611+0000 7f7d45cfd700 1 --2- 172.21.15.62:0/3691190383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d4006b620 0x7f7d4010f350 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800b0d0 tx=0x7f7d3800b3e0 comp rx=0 tx=0).stop 2024-06-10T00:29:05.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.611+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3691190383 shutdown_connections 2024-06-10T00:29:05.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.611+0000 7f7d45cfd700 1 --2- 172.21.15.62:0/3691190383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d4006b620 0x7f7d4010f350 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:05.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.611+0000 7f7d45cfd700 1 --2- 172.21.15.62:0/3691190383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d40108580 0x7f7d40108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:05.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.611+0000 7f7d45cfd700 1 --2- 172.21.15.62:0/3691190383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d4006ac60 0x7f7d4006b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:05.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.611+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3691190383 >> 172.21.15.62:0/3691190383 conn(0x7f7d40078430 msgr2=0x7f7d40078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:05.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.611+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3691190383 shutdown_connections 2024-06-10T00:29:05.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.611+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3691190383 wait complete. 2024-06-10T00:29:05.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.612+0000 7f7d45cfd700 1 Processor -- start 2024-06-10T00:29:05.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.612+0000 7f7d45cfd700 1 -- start start 2024-06-10T00:29:05.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.612+0000 7f7d45cfd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d4006ac60 0x7f7d4019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:05.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.612+0000 7f7d45cfd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d4006b620 0x7f7d4019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:05.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d45cfd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d40108580 0x7f7d40198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:05.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d45cfd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7d40112840 con 0x7f7d40108580 2024-06-10T00:29:05.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d45cfd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7d401126c0 con 0x7f7d4006b620 2024-06-10T00:29:05.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d45cfd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d401129c0 con 0x7f7d4006ac60 2024-06-10T00:29:05.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d3ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d40108580 0x7f7d40198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:05.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d3ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d40108580 0x7f7d40198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49972/0 (socket says 172.21.15.62:49972) 2024-06-10T00:29:05.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d3ffff700 1 -- 172.21.15.62:0/3307635812 learned_addr learned my addr 172.21.15.62:0/3307635812 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:05.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d3f7fe700 1 --2- 172.21.15.62:0/3307635812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d4006ac60 0x7f7d4019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:05.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d36ffd700 1 --2- 172.21.15.62:0/3307635812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d4006b620 0x7f7d4019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:05.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d3ffff700 1 -- 172.21.15.62:0/3307635812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d4006ac60 msgr2=0x7f7d4019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:05.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d3ffff700 1 --2- 172.21.15.62:0/3307635812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d4006ac60 0x7f7d4019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:05.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d3ffff700 1 -- 172.21.15.62:0/3307635812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d4006b620 msgr2=0x7f7d4019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:05.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d3ffff700 1 --2- 172.21.15.62:0/3307635812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d4006b620 0x7f7d4019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:05.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d3ffff700 1 -- 172.21.15.62:0/3307635812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d40199180 con 0x7f7d40108580 2024-06-10T00:29:05.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.613+0000 7f7d3ffff700 1 --2- 172.21.15.62:0/3307635812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d40108580 0x7f7d40198a20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7d38006070 tx=0x7f7d38004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:05.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.614+0000 7f7d3d7fa700 1 -- 172.21.15.62:0/3307635812 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38021070 con 0x7f7d40108580 2024-06-10T00:29:05.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.614+0000 7f7d3d7fa700 1 -- 172.21.15.62:0/3307635812 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d38007c00 con 0x7f7d40108580 2024-06-10T00:29:05.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.614+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3307635812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d40199410 con 0x7f7d40108580 2024-06-10T00:29:05.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.614+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3307635812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d401a56e0 con 0x7f7d40108580 2024-06-10T00:29:05.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.614+0000 7f7d3d7fa700 1 -- 172.21.15.62:0/3307635812 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d380052a0 con 0x7f7d40108580 2024-06-10T00:29:05.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.615+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3307635812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d4010aa10 con 0x7f7d40108580 2024-06-10T00:29:05.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.616+0000 7f7d3d7fa700 1 -- 172.21.15.62:0/3307635812 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7d38024070 con 0x7f7d40108580 2024-06-10T00:29:05.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.617+0000 7f7d3d7fa700 1 --2- 172.21.15.62:0/3307635812 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7d2c0a2ca0 0x7f7d2c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:05.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.617+0000 7f7d3d7fa700 1 -- 172.21.15.62:0/3307635812 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f7d3800f020 con 0x7f7d40108580 2024-06-10T00:29:05.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.617+0000 7f7d3f7fe700 1 --2- 172.21.15.62:0/3307635812 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7d2c0a2ca0 0x7f7d2c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:05.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.617+0000 7f7d3f7fe700 1 --2- 172.21.15.62:0/3307635812 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7d2c0a2ca0 0x7f7d2c0a5160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7d400ffa50 tx=0x7f7d30009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:05.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.620+0000 7f7d3d7fa700 1 -- 172.21.15.62:0/3307635812 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7d3802b020 con 0x7f7d40108580 2024-06-10T00:29:05.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.875+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3307635812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d40069180 con 0x7f7d40108580 2024-06-10T00:29:05.875 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:05 smithi062 ceph-mon[32390]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:05.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.876+0000 7f7d3d7fa700 1 -- 172.21.15.62:0/3307635812 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11803 (secure 0 0 0) 0x7f7d38007db0 con 0x7f7d40108580 2024-06-10T00:29:05.876 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:29:05.876 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:28:53.578096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24464,"qdb_cluster":[24464]},"id":1}]} 2024-06-10T00:29:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.878+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3307635812 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7d2c0a2ca0 msgr2=0x7f7d2c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.878+0000 7f7d45cfd700 1 --2- 172.21.15.62:0/3307635812 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7d2c0a2ca0 0x7f7d2c0a5160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7d400ffa50 tx=0x7f7d30009190 comp rx=0 tx=0).stop 2024-06-10T00:29:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.878+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3307635812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d40108580 msgr2=0x7f7d40198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.878+0000 7f7d45cfd700 1 --2- 172.21.15.62:0/3307635812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d40108580 0x7f7d40198a20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7d38006070 tx=0x7f7d38004e40 comp rx=0 tx=0).stop 2024-06-10T00:29:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.878+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3307635812 shutdown_connections 2024-06-10T00:29:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.878+0000 7f7d45cfd700 1 --2- 172.21.15.62:0/3307635812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d40108580 0x7f7d40198a20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.878+0000 7f7d45cfd700 1 --2- 172.21.15.62:0/3307635812 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7d2c0a2ca0 0x7f7d2c0a5160 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.878+0000 7f7d45cfd700 1 --2- 172.21.15.62:0/3307635812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d4006ac60 0x7f7d4019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:05.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.878+0000 7f7d45cfd700 1 --2- 172.21.15.62:0/3307635812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d4006b620 0x7f7d4019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:05.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.878+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3307635812 >> 172.21.15.62:0/3307635812 conn(0x7f7d40078430 msgr2=0x7f7d400ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:05.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.878+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3307635812 shutdown_connections 2024-06-10T00:29:05.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:05.878+0000 7f7d45cfd700 1 -- 172.21.15.62:0/3307635812 wait complete. 2024-06-10T00:29:05.879 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 17 2024-06-10T00:29:06.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:05 smithi105 ceph-mon[36321]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:06.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:05 smithi138 ceph-mon[36549]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:06.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:29:07.007 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:07.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:06 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3307635812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:29:07.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:06 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3307635812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:29:07.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:06 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3307635812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:29:08.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:07 smithi105 ceph-mon[36321]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:08.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:07 smithi062 ceph-mon[32390]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:08.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:07 smithi138 ceph-mon[36549]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:08.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.793+0000 7f049fc42700 1 -- 172.21.15.62:0/3845038530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0498107840 msgr2=0x7f049810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:08.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.793+0000 7f049fc42700 1 --2- 172.21.15.62:0/3845038530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0498107840 0x7f049810f3c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f049000b0d0 tx=0x7f049000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:29:08.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.793+0000 7f049fc42700 1 -- 172.21.15.62:0/3845038530 shutdown_connections 2024-06-10T00:29:08.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.793+0000 7f049fc42700 1 --2- 172.21.15.62:0/3845038530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0498107840 0x7f049810f3c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:08.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.793+0000 7f049fc42700 1 --2- 172.21.15.62:0/3845038530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0498102740 0x7f0498102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:08.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.793+0000 7f049fc42700 1 --2- 172.21.15.62:0/3845038530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04981030f0 0x7f0498107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:08.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.793+0000 7f049fc42700 1 -- 172.21.15.62:0/3845038530 >> 172.21.15.62:0/3845038530 conn(0x7f04980795f0 msgr2=0x7f0498079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:08.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.794+0000 7f049fc42700 1 -- 172.21.15.62:0/3845038530 shutdown_connections 2024-06-10T00:29:08.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.794+0000 7f049fc42700 1 -- 172.21.15.62:0/3845038530 wait complete. 2024-06-10T00:29:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.794+0000 7f049fc42700 1 Processor -- start 2024-06-10T00:29:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.794+0000 7f049fc42700 1 -- start start 2024-06-10T00:29:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049fc42700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0498102740 0x7f049819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049fc42700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04981030f0 0x7f049819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049fc42700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0498107840 0x7f0498198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049fc42700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0498112820 con 0x7f0498107840 2024-06-10T00:29:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049fc42700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04981126a0 con 0x7f0498102740 2024-06-10T00:29:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049fc42700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04981129a0 con 0x7f04981030f0 2024-06-10T00:29:08.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049e1df700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0498107840 0x7f0498198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:08.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049e1df700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0498107840 0x7f0498198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41882/0 (socket says 172.21.15.62:41882) 2024-06-10T00:29:08.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049d9de700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0498102740 0x7f049819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:08.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049e1df700 1 -- 172.21.15.62:0/2198442745 learned_addr learned my addr 172.21.15.62:0/2198442745 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:08.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049d1dd700 1 --2- 172.21.15.62:0/2198442745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04981030f0 0x7f049819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:08.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049e1df700 1 -- 172.21.15.62:0/2198442745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04981030f0 msgr2=0x7f049819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049e1df700 1 --2- 172.21.15.62:0/2198442745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04981030f0 0x7f049819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049e1df700 1 -- 172.21.15.62:0/2198442745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0498102740 msgr2=0x7f049819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049e1df700 1 --2- 172.21.15.62:0/2198442745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0498102740 0x7f049819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049e1df700 1 -- 172.21.15.62:0/2198442745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0498199180 con 0x7f0498107840 2024-06-10T00:29:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.795+0000 7f049e1df700 1 --2- 172.21.15.62:0/2198442745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0498107840 0x7f0498198a20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0490004dc0 tx=0x7f0490004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.796+0000 7f048affd700 1 -- 172.21.15.62:0/2198442745 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490021020 con 0x7f0498107840 2024-06-10T00:29:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.796+0000 7f048affd700 1 -- 172.21.15.62:0/2198442745 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0490007b80 con 0x7f0498107840 2024-06-10T00:29:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.796+0000 7f049fc42700 1 -- 172.21.15.62:0/2198442745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0498199410 con 0x7f0498107840 2024-06-10T00:29:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.796+0000 7f048affd700 1 -- 172.21.15.62:0/2198442745 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490005280 con 0x7f0498107840 2024-06-10T00:29:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.796+0000 7f049fc42700 1 -- 172.21.15.62:0/2198442745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04981a5730 con 0x7f0498107840 2024-06-10T00:29:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.797+0000 7f049fc42700 1 -- 172.21.15.62:0/2198442745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0498069180 con 0x7f0498107840 2024-06-10T00:29:08.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.798+0000 7f048affd700 1 -- 172.21.15.62:0/2198442745 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0490024020 con 0x7f0498107840 2024-06-10T00:29:08.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.799+0000 7f048affd700 1 --2- 172.21.15.62:0/2198442745 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04840a2ca0 0x7f04840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:08.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.799+0000 7f049d9de700 1 --2- 172.21.15.62:0/2198442745 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04840a2ca0 0x7f04840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:08.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.799+0000 7f048affd700 1 -- 172.21.15.62:0/2198442745 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f04900d0a00 con 0x7f0498107840 2024-06-10T00:29:08.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.800+0000 7f049d9de700 1 --2- 172.21.15.62:0/2198442745 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04840a2ca0 0x7f04840a5160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0494005950 tx=0x7f049400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:08.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:08.803+0000 7f048affd700 1 -- 172.21.15.62:0/2198442745 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f049002b020 con 0x7f0498107840 2024-06-10T00:29:09.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:09.009+0000 7f049fc42700 1 -- 172.21.15.62:0/2198442745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f049802dca0 con 0x7f0498107840 2024-06-10T00:29:09.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:09.011+0000 7f048affd700 1 -- 172.21.15.62:0/2198442745 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+24126 (secure 0 0 0) 0x7f049009d870 con 0x7f0498107840 2024-06-10T00:29:09.011 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:29:09.011 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":95,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:28:53.577919+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:29:09.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:09.013+0000 7f049fc42700 1 -- 172.21.15.62:0/2198442745 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04840a2ca0 msgr2=0x7f04840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:09.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:09.013+0000 7f049fc42700 1 --2- 172.21.15.62:0/2198442745 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04840a2ca0 0x7f04840a5160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0494005950 tx=0x7f049400a300 comp rx=0 tx=0).stop 2024-06-10T00:29:09.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:09.013+0000 7f049fc42700 1 -- 172.21.15.62:0/2198442745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0498107840 msgr2=0x7f0498198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:09.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:09.013+0000 7f049fc42700 1 --2- 172.21.15.62:0/2198442745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0498107840 0x7f0498198a20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0490004dc0 tx=0x7f0490004df0 comp rx=0 tx=0).stop 2024-06-10T00:29:09.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:09.013+0000 7f049fc42700 1 -- 172.21.15.62:0/2198442745 shutdown_connections 2024-06-10T00:29:09.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:09.013+0000 7f049fc42700 1 --2- 172.21.15.62:0/2198442745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0498107840 0x7f0498198a20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:09.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:09.013+0000 7f049fc42700 1 --2- 172.21.15.62:0/2198442745 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04840a2ca0 0x7f04840a5160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:09.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:09.013+0000 7f049fc42700 1 --2- 172.21.15.62:0/2198442745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0498102740 0x7f049819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:09.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:09.013+0000 7f049fc42700 1 --2- 172.21.15.62:0/2198442745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04981030f0 0x7f049819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:09.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:09.013+0000 7f049fc42700 1 -- 172.21.15.62:0/2198442745 >> 172.21.15.62:0/2198442745 conn(0x7f04980795f0 msgr2=0x7f0498100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:09.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:09.013+0000 7f049fc42700 1 -- 172.21.15.62:0/2198442745 shutdown_connections 2024-06-10T00:29:09.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:09.013+0000 7f049fc42700 1 -- 172.21.15.62:0/2198442745 wait complete. 2024-06-10T00:29:09.817 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-06-10T00:28:39.478894+0000', 'modified': '2024-06-10T00:28:53.578096+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24464}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24464': {'gid': 24464, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.105:6839/1632080154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 1632080154}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 1632080154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24464, 'qdb_cluster': [24464]} 2024-06-10T00:29:09.817 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-06-10T00:29:09.818 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-10T00:29:10.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:09 smithi105 ceph-mon[36321]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:10.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:09 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2198442745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:10.075 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:10.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:09 smithi062 ceph-mon[32390]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:10.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:09 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2198442745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:10.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:09 smithi138 ceph-mon[36549]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:10.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:09 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2198442745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:11.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.836+0000 7ffa860e3700 1 -- 172.21.15.62:0/3051290438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa80101d80 msgr2=0x7ffa8010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:11.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.836+0000 7ffa860e3700 1 --2- 172.21.15.62:0/3051290438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa80101d80 0x7ffa8010e240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800b0a0 tx=0x7ffa7800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.836+0000 7ffa860e3700 1 -- 172.21.15.62:0/3051290438 shutdown_connections 2024-06-10T00:29:11.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.836+0000 7ffa860e3700 1 --2- 172.21.15.62:0/3051290438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa80101d80 0x7ffa8010e240 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.836+0000 7ffa860e3700 1 --2- 172.21.15.62:0/3051290438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa80107440 0x7ffa80107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.836+0000 7ffa860e3700 1 --2- 172.21.15.62:0/3051290438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa801013c0 0x7ffa80101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.836+0000 7ffa860e3700 1 -- 172.21.15.62:0/3051290438 >> 172.21.15.62:0/3051290438 conn(0x7ffa80077400 msgr2=0x7ffa80077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:11.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.836+0000 7ffa860e3700 1 -- 172.21.15.62:0/3051290438 shutdown_connections 2024-06-10T00:29:11.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.836+0000 7ffa860e3700 1 -- 172.21.15.62:0/3051290438 wait complete. 2024-06-10T00:29:11.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa860e3700 1 Processor -- start 2024-06-10T00:29:11.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa860e3700 1 -- start start 2024-06-10T00:29:11.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa860e3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa801013c0 0x7ffa801999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa860e3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa80101d80 0x7ffa80199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa860e3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa80107440 0x7ffa801a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa860e3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80111680 con 0x7ffa80101d80 2024-06-10T00:29:11.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa860e3700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80111500 con 0x7ffa801013c0 2024-06-10T00:29:11.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa860e3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80111800 con 0x7ffa80107440 2024-06-10T00:29:11.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa77fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa80101d80 0x7ffa80199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa77fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa80101d80 0x7ffa80199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41906/0 (socket says 172.21.15.62:41906) 2024-06-10T00:29:11.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa77fff700 1 -- 172.21.15.62:0/3870240048 learned_addr learned my addr 172.21.15.62:0/3870240048 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:11.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa7f7fe700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa801013c0 0x7ffa801999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa77fff700 1 -- 172.21.15.62:0/3870240048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa80107440 msgr2=0x7ffa801a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:11.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa7ffff700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa80107440 0x7ffa801a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.837+0000 7ffa77fff700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa80107440 0x7ffa801a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.838+0000 7ffa77fff700 1 -- 172.21.15.62:0/3870240048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa801013c0 msgr2=0x7ffa801999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:11.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.838+0000 7ffa77fff700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa801013c0 0x7ffa801999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.838+0000 7ffa77fff700 1 -- 172.21.15.62:0/3870240048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa801a4f60 con 0x7ffa80101d80 2024-06-10T00:29:11.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.838+0000 7ffa7f7fe700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa801013c0 0x7ffa801999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:29:11.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.838+0000 7ffa7ffff700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa80107440 0x7ffa801a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:29:11.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.838+0000 7ffa77fff700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa80101d80 0x7ffa80199ee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800d950 tx=0x7ffa6800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:11.840 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:11 smithi062 ceph-mon[32390]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:11.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.838+0000 7ffa7d7fa700 1 -- 172.21.15.62:0/3870240048 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68016070 con 0x7ffa80101d80 2024-06-10T00:29:11.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.838+0000 7ffa7d7fa700 1 -- 172.21.15.62:0/3870240048 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa68009b40 con 0x7ffa80101d80 2024-06-10T00:29:11.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.838+0000 7ffa7d7fa700 1 -- 172.21.15.62:0/3870240048 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800f620 con 0x7ffa80101d80 2024-06-10T00:29:11.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.838+0000 7ffa860e3700 1 -- 172.21.15.62:0/3870240048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa801a51f0 con 0x7ffa80101d80 2024-06-10T00:29:11.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.838+0000 7ffa860e3700 1 -- 172.21.15.62:0/3870240048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa801a55f0 con 0x7ffa80101d80 2024-06-10T00:29:11.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.841+0000 7ffa7d7fa700 1 -- 172.21.15.62:0/3870240048 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffa68019070 con 0x7ffa80101d80 2024-06-10T00:29:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.842+0000 7ffa90966b80 1 Processor -- start 2024-06-10T00:29:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.842+0000 7ffa90966b80 1 -- start start 2024-06-10T00:29:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.842+0000 7ffa7d7fa700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffa6c0a2c20 0x7ffa6c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.842+0000 7ffa90966b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.842+0000 7ffa7d7fa700 1 -- 172.21.15.62:0/3870240048 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7ffa680cf270 con 0x7ffa80101d80 2024-06-10T00:29:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.842+0000 7ffa90966b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f1f10 0x55f9ac8eb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.842+0000 7ffa90966b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8ec2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.842+0000 7ffa90966b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac83d7c0 con 0x55f9ac8ebed0 2024-06-10T00:29:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.842+0000 7ffa90966b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac7e56d0 con 0x55f9ac8f15f0 2024-06-10T00:29:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.842+0000 7ffa90966b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac839780 con 0x55f9ac8f1f10 2024-06-10T00:29:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa7ffff700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffa6c0a2c20 0x7ffa6c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa7ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8ec2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa7f7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa7f7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:41722/0 (socket says 172.21.15.62:41722) 2024-06-10T00:29:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa7ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8ec2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41918/0 (socket says 172.21.15.62:41918) 2024-06-10T00:29:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa7f7fe700 1 -- 172.21.15.62:0/256814695 learned_addr learned my addr 172.21.15.62:0/256814695 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa77fff700 1 --2- 172.21.15.62:0/256814695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f1f10 0x55f9ac8eb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa7f7fe700 1 -- 172.21.15.62:0/256814695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f1f10 msgr2=0x55f9ac8eb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:29:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa7f7fe700 1 --2- 172.21.15.62:0/256814695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f1f10 0x55f9ac8eb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa7ffff700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffa6c0a2c20 0x7ffa6c0a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800b070 tx=0x7ffa780091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa7f7fe700 1 -- 172.21.15.62:0/256814695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8ebed0 msgr2=0x55f9ac8ec2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:29:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa7f7fe700 1 --2- 172.21.15.62:0/256814695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8ec2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa7f7fe700 1 -- 172.21.15.62:0/256814695 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9ac7dbf00 con 0x55f9ac8f15f0 2024-06-10T00:29:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa7ffff700 1 --2- 172.21.15.62:0/256814695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8ec2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:29:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.843+0000 7ffa7f7fe700 1 --2- 172.21.15.62:0/256814695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f19d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009c50 tx=0x7ffa70009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a82175d638fb4b82 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.844+0000 7ffa75ffb700 1 -- 172.21.15.62:0/256814695 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70016070 con 0x55f9ac8f15f0 2024-06-10T00:29:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.844+0000 7ffa75ffb700 1 -- 172.21.15.62:0/256814695 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa70011470 con 0x55f9ac8f15f0 2024-06-10T00:29:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.844+0000 7ffa75ffb700 1 -- 172.21.15.62:0/256814695 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70010610 con 0x55f9ac8f15f0 2024-06-10T00:29:11.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.844+0000 7ffa90966b80 1 -- 172.21.15.62:0/256814695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8f15f0 msgr2=0x55f9ac8f19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:11.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.844+0000 7ffa90966b80 1 --2- 172.21.15.62:0/256814695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f19d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009c50 tx=0x7ffa70009f60 comp rx=0 tx=0).stop 2024-06-10T00:29:11.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.845+0000 7ffa90966b80 1 -- 172.21.15.62:0/256814695 shutdown_connections 2024-06-10T00:29:11.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.845+0000 7ffa90966b80 1 --2- 172.21.15.62:0/256814695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8ec2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.845+0000 7ffa90966b80 1 --2- 172.21.15.62:0/256814695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f19d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.845+0000 7ffa90966b80 1 --2- 172.21.15.62:0/256814695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f1f10 0x55f9ac8eb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.845+0000 7ffa90966b80 1 -- 172.21.15.62:0/256814695 >> 172.21.15.62:0/256814695 conn(0x55f9ac816c70 msgr2=0x55f9ac8e6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:11.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.845+0000 7ffa90966b80 1 -- 172.21.15.62:0/256814695 shutdown_connections 2024-06-10T00:29:11.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.845+0000 7ffa90966b80 1 -- 172.21.15.62:0/256814695 wait complete. 2024-06-10T00:29:11.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.845+0000 7ffa90966b80 1 -- start start 2024-06-10T00:29:11.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.845+0000 7ffa90966b80 1 Processor -- start 2024-06-10T00:29:11.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa90966b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa90966b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa90966b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f1f10 0x55f9ac8f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa90966b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac8f8980 con 0x55f9ac8f15f0 2024-06-10T00:29:11.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa90966b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac7ddba0 con 0x55f9ac8ebed0 2024-06-10T00:29:11.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa90966b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac8f8b00 con 0x55f9ac8f1f10 2024-06-10T00:29:11.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa90966b80 10 client.0 _ll_register_callbacks cb 0x55f9ac7d78d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:29:11.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa77fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa77fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41932/0 (socket says 172.21.15.62:41932) 2024-06-10T00:29:11.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa77fff700 1 -- 172.21.15.62:0/1726235345 learned_addr learned my addr 172.21.15.62:0/1726235345 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:11.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa7ffff700 1 --2- 172.21.15.62:0/1726235345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f1f10 0x55f9ac8f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa7f7fe700 1 --2- 172.21.15.62:0/1726235345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa77fff700 1 -- 172.21.15.62:0/1726235345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f1f10 msgr2=0x55f9ac8f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:29:11.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa77fff700 1 --2- 172.21.15.62:0/1726235345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f1f10 0x55f9ac8f54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa77fff700 1 -- 172.21.15.62:0/1726235345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8ebed0 msgr2=0x55f9ac8f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:29:11.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa77fff700 1 --2- 172.21.15.62:0/1726235345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8f7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa77fff700 1 -- 172.21.15.62:0/1726235345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9ac78c880 con 0x55f9ac8f15f0 2024-06-10T00:29:11.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa7f7fe700 1 --2- 172.21.15.62:0/1726235345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:29:11.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa77fff700 1 --2- 172.21.15.62:0/1726235345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f4fa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffa68000c00 tx=0x7ffa6809fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4410c93743d29b50 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:11.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa5effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:29:11.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa5effd700 1 -- 172.21.15.62:0/1726235345 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68020050 con 0x55f9ac8f15f0 2024-06-10T00:29:11.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa5effd700 1 -- 172.21.15.62:0/1726235345 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7ffa680860b0 con 0x55f9ac8f15f0 2024-06-10T00:29:11.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.846+0000 7ffa90966b80 1 -- 172.21.15.62:0/1726235345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa70011470 con 0x55f9ac8f15f0 2024-06-10T00:29:11.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.847+0000 7ffa5effd700 1 -- 172.21.15.62:0/1726235345 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ffa680862b0 con 0x55f9ac8f15f0 2024-06-10T00:29:11.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.847+0000 7ffa90966b80 10 client.14733 fetch_fsmap learned FSMap version 17 2024-06-10T00:29:11.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.847+0000 7ffa90966b80 1 -- 172.21.15.62:0/1726235345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55f9ac903220 con 0x55f9ac8f15f0 2024-06-10T00:29:11.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.847+0000 7ffa5effd700 1 -- 172.21.15.62:0/1726235345 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa70011470 con 0x55f9ac8f15f0 2024-06-10T00:29:11.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.847+0000 7ffa5effd700 1 -- 172.21.15.62:0/1726235345 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7ffa6809cbc0 con 0x55f9ac8f15f0 2024-06-10T00:29:11.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.847+0000 7ffa90966b80 10 client.14733 fetch_fsmap finished waiting for FSMap version 17 2024-06-10T00:29:11.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.847+0000 7ffa90966b80 10 client.14733 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T00:29:11.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.847+0000 7ffa90966b80 20 client.14733 populate_metadata read hostname 'smithi062' 2024-06-10T00:29:11.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.847+0000 7ffa90966b80 1 --2- 172.21.15.62:0/1726235345 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f9ac905850 0x55f9ac906cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.847+0000 7ffa90966b80 4 client.14733 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:29:11.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.847+0000 7ffa90966b80 1 -- 172.21.15.62:0/1726235345 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9ac84acc0 con 0x55f9ac905850 2024-06-10T00:29:11.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.847+0000 7ffa7f7fe700 1 --2- 172.21.15.62:0/1726235345 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f9ac905850 0x55f9ac906cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.848+0000 7ffa7f7fe700 1 --2- 172.21.15.62:0/1726235345 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f9ac905850 0x55f9ac906cc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d6bb90b60d628b8 server_cookie=b70b145f14a34968 in_seq=0 out_seq=0 2024-06-10T00:29:11.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.848+0000 7ffa5effd700 10 client.14733 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:29:11.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa5effd700 1 -- 172.21.15.62:0/1726235345 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f9ac84acc0 con 0x55f9ac905850 2024-06-10T00:29:11.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa5effd700 10 client.14733 handle_command_reply: tid=0 2024-06-10T00:29:11.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa90966b80 1 client.14733 shutdown 2024-06-10T00:29:11.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa90966b80 1 -- 172.21.15.62:0/1726235345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8f15f0 msgr2=0x55f9ac8f4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:11.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa90966b80 1 --2- 172.21.15.62:0/1726235345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f4fa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffa68000c00 tx=0x7ffa6809fc40 comp rx=0 tx=0).stop 2024-06-10T00:29:11.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa90966b80 1 -- 172.21.15.62:0/1726235345 shutdown_connections 2024-06-10T00:29:11.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa90966b80 1 --2- 172.21.15.62:0/1726235345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f1f10 0x55f9ac8f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa90966b80 1 --2- 172.21.15.62:0/1726235345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa90966b80 1 --2- 172.21.15.62:0/1726235345 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f9ac905850 0x55f9ac906cc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa90966b80 1 --2- 172.21.15.62:0/1726235345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f4fa0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa5effd700 0 client.14733 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:29:11.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa90966b80 1 -- 172.21.15.62:0/1726235345 >> 172.21.15.62:0/1726235345 conn(0x55f9ac816c70 msgr2=0x55f9ac8ea980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:11.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa90966b80 1 -- 172.21.15.62:0/1726235345 shutdown_connections 2024-06-10T00:29:11.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa90966b80 1 -- 172.21.15.62:0/1726235345 wait complete. 2024-06-10T00:29:11.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.849+0000 7ffa90966b80 20 client.14733 trim_cache size 0 max 16384 2024-06-10T00:29:11.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa90966b80 1 Processor -- start 2024-06-10T00:29:11.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa90966b80 1 -- start start 2024-06-10T00:29:11.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa90966b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8eb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa90966b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa90966b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8f1f10 0x55f9ac8f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa90966b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac905000 con 0x55f9ac8f1f10 2024-06-10T00:29:11.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa90966b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac9056d0 con 0x55f9ac8ebed0 2024-06-10T00:29:11.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa90966b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac905da0 con 0x55f9ac8f15f0 2024-06-10T00:29:11.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa7ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8f1f10 0x55f9ac8f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa7ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8f1f10 0x55f9ac8f28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41934/0 (socket says 172.21.15.62:41934) 2024-06-10T00:29:11.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa7ffff700 1 -- 172.21.15.62:0/997946031 learned_addr learned my addr 172.21.15.62:0/997946031 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:11.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa77fff700 1 --2- 172.21.15.62:0/997946031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa7ffff700 1 -- 172.21.15.62:0/997946031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f15f0 msgr2=0x55f9ac8f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:29:11.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa7f7fe700 1 --2- 172.21.15.62:0/997946031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8eb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa7ffff700 1 --2- 172.21.15.62:0/997946031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa7ffff700 1 -- 172.21.15.62:0/997946031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8ebed0 msgr2=0x55f9ac8eb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:29:11.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa7ffff700 1 --2- 172.21.15.62:0/997946031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8eb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa7ffff700 1 -- 172.21.15.62:0/997946031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9ac8f3510 con 0x55f9ac8f1f10 2024-06-10T00:29:11.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.871+0000 7ffa77fff700 1 --2- 172.21.15.62:0/997946031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:29:11.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.872+0000 7ffa7ffff700 1 --2- 172.21.15.62:0/997946031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8f1f10 0x55f9ac8f28f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa78015000 tx=0x7ffa78023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3fd7cd07e282fb5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:11.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.872+0000 7ffa757fa700 1 -- 172.21.15.62:0/997946031 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa780078b0 con 0x55f9ac8f1f10 2024-06-10T00:29:11.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.872+0000 7ffa757fa700 1 -- 172.21.15.62:0/997946031 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7801f440 con 0x55f9ac8f1f10 2024-06-10T00:29:11.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.872+0000 7ffa757fa700 1 -- 172.21.15.62:0/997946031 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78005280 con 0x55f9ac8f1f10 2024-06-10T00:29:11.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.872+0000 7ffa90966b80 1 -- 172.21.15.62:0/997946031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8f1f10 msgr2=0x55f9ac8f28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:11.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.872+0000 7ffa90966b80 1 --2- 172.21.15.62:0/997946031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8f1f10 0x55f9ac8f28f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa78015000 tx=0x7ffa78023000 comp rx=0 tx=0).stop 2024-06-10T00:29:11.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.872+0000 7ffa90966b80 1 -- 172.21.15.62:0/997946031 shutdown_connections 2024-06-10T00:29:11.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.872+0000 7ffa90966b80 1 --2- 172.21.15.62:0/997946031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8f1f10 0x55f9ac8f28f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.872+0000 7ffa90966b80 1 --2- 172.21.15.62:0/997946031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8ebed0 0x55f9ac8eb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.872+0000 7ffa90966b80 1 --2- 172.21.15.62:0/997946031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f15f0 0x55f9ac8f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.872+0000 7ffa90966b80 1 -- 172.21.15.62:0/997946031 >> 172.21.15.62:0/997946031 conn(0x55f9ac816c70 msgr2=0x55f9ac8f3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:11.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.873+0000 7ffa90966b80 1 -- 172.21.15.62:0/997946031 shutdown_connections 2024-06-10T00:29:11.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.873+0000 7ffa90966b80 1 -- 172.21.15.62:0/997946031 wait complete. 2024-06-10T00:29:11.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.873+0000 7ffa90966b80 1 -- start start 2024-06-10T00:29:11.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.873+0000 7ffa90966b80 1 Processor -- start 2024-06-10T00:29:11.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.873+0000 7ffa90966b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8ebed0 0x55f9ac910140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.873+0000 7ffa90966b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f15f0 0x55f9ac910680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.873+0000 7ffa90966b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8f1f10 0x55f9ac90b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.873+0000 7ffa90966b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac906b40 con 0x55f9ac8ebed0 2024-06-10T00:29:11.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.873+0000 7ffa90966b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac906470 con 0x55f9ac8f1f10 2024-06-10T00:29:11.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.873+0000 7ffa90966b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9ac907210 con 0x55f9ac8f15f0 2024-06-10T00:29:11.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.873+0000 7ffa90966b80 10 client.0 _ll_register_callbacks cb 0x55f9ac7d78d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:29:11.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.873+0000 7ffa7f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8ebed0 0x55f9ac910140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.873+0000 7ffa7f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8ebed0 0x55f9ac910140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41936/0 (socket says 172.21.15.62:41936) 2024-06-10T00:29:11.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa7f7fe700 1 -- 172.21.15.62:0/3192366993 learned_addr learned my addr 172.21.15.62:0/3192366993 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:11.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa77fff700 1 --2- 172.21.15.62:0/3192366993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f15f0 0x55f9ac910680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa7ffff700 1 --2- 172.21.15.62:0/3192366993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8f1f10 0x55f9ac90b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa7f7fe700 1 -- 172.21.15.62:0/3192366993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f15f0 msgr2=0x55f9ac910680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:29:11.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa7f7fe700 1 --2- 172.21.15.62:0/3192366993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f15f0 0x55f9ac910680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa7f7fe700 1 -- 172.21.15.62:0/3192366993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8f1f10 msgr2=0x55f9ac90b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:29:11.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa7f7fe700 1 --2- 172.21.15.62:0/3192366993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8f1f10 0x55f9ac90b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa7f7fe700 1 -- 172.21.15.62:0/3192366993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9ac90b630 con 0x55f9ac8ebed0 2024-06-10T00:29:11.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa7f7fe700 1 --2- 172.21.15.62:0/3192366993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8ebed0 0x55f9ac910140 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa70011e10 tx=0x7ffa70005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7188c6d3cf49b1fc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:11.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa5effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:29:11.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa5effd700 1 -- 172.21.15.62:0/3192366993 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7002c030 con 0x55f9ac8ebed0 2024-06-10T00:29:11.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa5effd700 1 -- 172.21.15.62:0/3192366993 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7ffa70011970 con 0x55f9ac8ebed0 2024-06-10T00:29:11.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa90966b80 1 -- 172.21.15.62:0/3192366993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa7801f440 con 0x55f9ac8ebed0 2024-06-10T00:29:11.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa5effd700 1 -- 172.21.15.62:0/3192366993 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ffa70016040 con 0x55f9ac8ebed0 2024-06-10T00:29:11.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa90966b80 10 client.14739 fetch_fsmap learned FSMap version 17 2024-06-10T00:29:11.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.874+0000 7ffa90966b80 1 -- 172.21.15.62:0/3192366993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55f9ac9717f0 con 0x55f9ac8ebed0 2024-06-10T00:29:11.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.875+0000 7ffa5effd700 1 -- 172.21.15.62:0/3192366993 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7801f440 con 0x55f9ac8ebed0 2024-06-10T00:29:11.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.875+0000 7ffa5effd700 1 -- 172.21.15.62:0/3192366993 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7ffa7000f040 con 0x55f9ac8ebed0 2024-06-10T00:29:11.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.875+0000 7ffa90966b80 10 client.14739 fetch_fsmap finished waiting for FSMap version 17 2024-06-10T00:29:11.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.875+0000 7ffa90966b80 10 client.14739 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T00:29:11.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.875+0000 7ffa90966b80 20 client.14739 populate_metadata read hostname 'smithi062' 2024-06-10T00:29:11.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.875+0000 7ffa90966b80 1 --2- 172.21.15.62:0/3192366993 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f9ac9760e0 0x55f9ac974df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:11.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.875+0000 7ffa90966b80 4 client.14739 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:29:11.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.875+0000 7ffa90966b80 1 -- 172.21.15.62:0/3192366993 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "status"}) -- 0x55f9ac845240 con 0x55f9ac9760e0 2024-06-10T00:29:11.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.875+0000 7ffa77fff700 1 --2- 172.21.15.62:0/3192366993 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f9ac9760e0 0x55f9ac974df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:11.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa5effd700 10 client.14739 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:29:11.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa77fff700 1 --2- 172.21.15.62:0/3192366993 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f9ac9760e0 0x55f9ac974df0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e49a201e7300a0f1 server_cookie=d72ac43414c6c03a in_seq=0 out_seq=0 2024-06-10T00:29:11.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa5effd700 1 -- 172.21.15.62:0/3192366993 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55f9ac845240 con 0x55f9ac9760e0 2024-06-10T00:29:11.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa5effd700 10 client.14739 handle_command_reply: tid=0 2024-06-10T00:29:11.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa90966b80 1 client.14739 shutdown 2024-06-10T00:29:11.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa90966b80 1 -- 172.21.15.62:0/3192366993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8ebed0 msgr2=0x55f9ac910140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:11.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa90966b80 1 --2- 172.21.15.62:0/3192366993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8ebed0 0x55f9ac910140 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa70011e10 tx=0x7ffa70005ee0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa90966b80 1 -- 172.21.15.62:0/3192366993 shutdown_connections 2024-06-10T00:29:11.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa90966b80 1 --2- 172.21.15.62:0/3192366993 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f9ac9760e0 0x55f9ac974df0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa90966b80 1 --2- 172.21.15.62:0/3192366993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9ac8ebed0 0x55f9ac910140 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa90966b80 1 --2- 172.21.15.62:0/3192366993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9ac8f1f10 0x55f9ac90b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa90966b80 1 --2- 172.21.15.62:0/3192366993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9ac8f15f0 0x55f9ac910680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa90966b80 1 -- 172.21.15.62:0/3192366993 >> 172.21.15.62:0/3192366993 conn(0x55f9ac816c70 msgr2=0x55f9ac8dfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:11.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa5effd700 0 client.14739 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:29:11.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa90966b80 1 -- 172.21.15.62:0/3192366993 shutdown_connections 2024-06-10T00:29:11.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa90966b80 1 -- 172.21.15.62:0/3192366993 wait complete. 2024-06-10T00:29:11.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.876+0000 7ffa90966b80 20 client.14739 trim_cache size 0 max 16384 2024-06-10T00:29:11.882 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:29:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:29:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-06-10T00:29:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24464, 2024-06-10T00:29:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:29:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:29:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:29:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 32.231430951999997, 2024-06-10T00:29:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 17, 2024-06-10T00:29:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 95, 2024-06-10T00:29:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 95, 2024-06-10T00:29:11.882 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 49.956566537 2024-06-10T00:29:11.883 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:29:11.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.877+0000 7ffa860e3700 1 -- 172.21.15.62:0/3870240048 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffa6c0a2c20 msgr2=0x7ffa6c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:11.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.877+0000 7ffa860e3700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffa6c0a2c20 0x7ffa6c0a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800b070 tx=0x7ffa780091b0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.877+0000 7ffa860e3700 1 -- 172.21.15.62:0/3870240048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa80101d80 msgr2=0x7ffa80199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:11.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.877+0000 7ffa860e3700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa80101d80 0x7ffa80199ee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800d950 tx=0x7ffa6800dd10 comp rx=0 tx=0).stop 2024-06-10T00:29:11.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.877+0000 7ffa860e3700 1 -- 172.21.15.62:0/3870240048 shutdown_connections 2024-06-10T00:29:11.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.877+0000 7ffa860e3700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffa6c0a2c20 0x7ffa6c0a50e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.877+0000 7ffa860e3700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa80107440 0x7ffa801a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.877+0000 7ffa860e3700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa801013c0 0x7ffa801999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.877+0000 7ffa860e3700 1 --2- 172.21.15.62:0/3870240048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa80101d80 0x7ffa80199ee0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:11.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.877+0000 7ffa860e3700 1 -- 172.21.15.62:0/3870240048 >> 172.21.15.62:0/3870240048 conn(0x7ffa80077400 msgr2=0x7ffa80100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:11.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.877+0000 7ffa860e3700 1 -- 172.21.15.62:0/3870240048 shutdown_connections 2024-06-10T00:29:11.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:11.877+0000 7ffa860e3700 1 -- 172.21.15.62:0/3870240048 wait complete. 2024-06-10T00:29:12.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:11 smithi105 ceph-mon[36321]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:12.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:11 smithi138 ceph-mon[36549]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:12.573 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-10T00:29:12.573 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:29:12.573 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-10T00:29:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:14.008 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:13 smithi062 ceph-mon[32390]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:14.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:13 smithi105 ceph-mon[36321]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:14.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:13 smithi138 ceph-mon[36549]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:14.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.594+0000 7f01259d5700 1 -- 172.21.15.62:0/3572314922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0120108500 msgr2=0x7f01201088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:14.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.594+0000 7f01259d5700 1 --2- 172.21.15.62:0/3572314922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0120108500 0x7f01201088e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0108009990 tx=0x7f0108009ca0 comp rx=0 tx=0).stop 2024-06-10T00:29:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.594+0000 7f01259d5700 1 -- 172.21.15.62:0/3572314922 shutdown_connections 2024-06-10T00:29:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.594+0000 7f01259d5700 1 --2- 172.21.15.62:0/3572314922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0120108500 0x7f01201088e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.594+0000 7f01259d5700 1 --2- 172.21.15.62:0/3572314922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f012006b580 0x7f012010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.594+0000 7f01259d5700 1 --2- 172.21.15.62:0/3572314922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f012006abc0 0x7f012006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.594+0000 7f01259d5700 1 -- 172.21.15.62:0/3572314922 >> 172.21.15.62:0/3572314922 conn(0x7f0120078340 msgr2=0x7f0120078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.594+0000 7f01259d5700 1 -- 172.21.15.62:0/3572314922 shutdown_connections 2024-06-10T00:29:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.594+0000 7f01259d5700 1 -- 172.21.15.62:0/3572314922 wait complete. 2024-06-10T00:29:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.594+0000 7f01259d5700 1 Processor -- start 2024-06-10T00:29:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f01259d5700 1 -- start start 2024-06-10T00:29:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f01259d5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f012006abc0 0x7f0120101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f01259d5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f012006b580 0x7f01201019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f01259d5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0120108500 0x7f0120101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f01259d5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f01201127e0 con 0x7f0120108500 2024-06-10T00:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f01259d5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0120112660 con 0x7f012006abc0 2024-06-10T00:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f01259d5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0120112960 con 0x7f012006b580 2024-06-10T00:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f011f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0120108500 0x7f0120101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f011f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0120108500 0x7f0120101f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41960/0 (socket says 172.21.15.62:41960) 2024-06-10T00:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f011f7fe700 1 -- 172.21.15.62:0/1186852210 learned_addr learned my addr 172.21.15.62:0/1186852210 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f011e7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f012006b580 0x7f01201019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f011effd700 1 --2- 172.21.15.62:0/1186852210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f012006abc0 0x7f0120101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f011f7fe700 1 -- 172.21.15.62:0/1186852210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f012006b580 msgr2=0x7f01201019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f011f7fe700 1 --2- 172.21.15.62:0/1186852210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f012006b580 0x7f01201019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:14.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f011f7fe700 1 -- 172.21.15.62:0/1186852210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f012006abc0 msgr2=0x7f0120101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f011f7fe700 1 --2- 172.21.15.62:0/1186852210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f012006abc0 0x7f0120101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.595+0000 7f011f7fe700 1 -- 172.21.15.62:0/1186852210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01201a7160 con 0x7f0120108500 2024-06-10T00:29:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.596+0000 7f011f7fe700 1 --2- 172.21.15.62:0/1186852210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0120108500 0x7f0120101f10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0118007d70 tx=0x7f011800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.596+0000 7f01249d3700 1 -- 172.21.15.62:0/1186852210 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118016020 con 0x7f0120108500 2024-06-10T00:29:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.596+0000 7f01259d5700 1 -- 172.21.15.62:0/1186852210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01201a7450 con 0x7f0120108500 2024-06-10T00:29:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.596+0000 7f01249d3700 1 -- 172.21.15.62:0/1186852210 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011800ac90 con 0x7f0120108500 2024-06-10T00:29:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.596+0000 7f01249d3700 1 -- 172.21.15.62:0/1186852210 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01180115c0 con 0x7f0120108500 2024-06-10T00:29:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.596+0000 7f01259d5700 1 -- 172.21.15.62:0/1186852210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01201a7990 con 0x7f0120108500 2024-06-10T00:29:14.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.597+0000 7f01259d5700 1 -- 172.21.15.62:0/1186852210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f012010a9e0 con 0x7f0120108500 2024-06-10T00:29:14.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.598+0000 7f01249d3700 1 -- 172.21.15.62:0/1186852210 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0118019020 con 0x7f0120108500 2024-06-10T00:29:14.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.599+0000 7f01249d3700 1 --2- 172.21.15.62:0/1186852210 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f010c0a2c20 0x7f010c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:14.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.599+0000 7f011effd700 1 --2- 172.21.15.62:0/1186852210 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f010c0a2c20 0x7f010c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:14.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.599+0000 7f01249d3700 1 -- 172.21.15.62:0/1186852210 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7f01180cf200 con 0x7f0120108500 2024-06-10T00:29:14.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.599+0000 7f011effd700 1 --2- 172.21.15.62:0/1186852210 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f010c0a2c20 0x7f010c0a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f01080098e0 tx=0x7f0108005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:14.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.603+0000 7f01249d3700 1 -- 172.21.15.62:0/1186852210 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0118020020 con 0x7f0120108500 2024-06-10T00:29:14.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.854+0000 7f01259d5700 1 -- 172.21.15.62:0/1186852210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f012010a3c0 con 0x7f0120108500 2024-06-10T00:29:14.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.854+0000 7f01249d3700 1 -- 172.21.15.62:0/1186852210 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f011809bf20 con 0x7f0120108500 2024-06-10T00:29:14.855 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-10T00:29:14.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.856+0000 7f01259d5700 1 -- 172.21.15.62:0/1186852210 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f010c0a2c20 msgr2=0x7f010c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:14.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.856+0000 7f01259d5700 1 --2- 172.21.15.62:0/1186852210 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f010c0a2c20 0x7f010c0a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f01080098e0 tx=0x7f0108005960 comp rx=0 tx=0).stop 2024-06-10T00:29:14.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.856+0000 7f01259d5700 1 -- 172.21.15.62:0/1186852210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0120108500 msgr2=0x7f0120101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:14.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.856+0000 7f01259d5700 1 --2- 172.21.15.62:0/1186852210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0120108500 0x7f0120101f10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0118007d70 tx=0x7f011800a340 comp rx=0 tx=0).stop 2024-06-10T00:29:14.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.857+0000 7f01259d5700 1 -- 172.21.15.62:0/1186852210 shutdown_connections 2024-06-10T00:29:14.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.857+0000 7f01259d5700 1 --2- 172.21.15.62:0/1186852210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0120108500 0x7f0120101f10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:14.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.857+0000 7f01259d5700 1 --2- 172.21.15.62:0/1186852210 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f010c0a2c20 0x7f010c0a50e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:14.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.857+0000 7f01259d5700 1 --2- 172.21.15.62:0/1186852210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f012006abc0 0x7f0120101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:14.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.857+0000 7f01259d5700 1 --2- 172.21.15.62:0/1186852210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f012006b580 0x7f01201019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:14.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.857+0000 7f01259d5700 1 -- 172.21.15.62:0/1186852210 >> 172.21.15.62:0/1186852210 conn(0x7f0120078340 msgr2=0x7f0120100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:14.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.857+0000 7f01259d5700 1 -- 172.21.15.62:0/1186852210 shutdown_connections 2024-06-10T00:29:14.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:14.857+0000 7f01259d5700 1 -- 172.21.15.62:0/1186852210 wait complete. 2024-06-10T00:29:15.454 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-06-10T00:29:15.454 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-10T00:29:15.454 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-06-10T00:29:15.454 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-06-10T00:29:15.454 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:29:15.454 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-06-10T00:29:15.454 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-10T00:29:15.455 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-10T00:29:15.455 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-06-10T00:29:15.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:29:15.455 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-06-10T00:29:15.473 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-10T00:29:15.473 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-10T00:29:15.473 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-06-10T00:29:15.473 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-06-10T00:29:15.474 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-06-10T00:29:15.474 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-06-10T00:29:15.474 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T00:29:15.474 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:29:15.474 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-06-10T00:29:15.474 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T00:29:15.474 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:29:15.474 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-06-10T00:29:15.474 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T00:29:15.474 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:29:15.474 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-10T00:29:15.474 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:29:15.475 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-06-10T00:29:15.475 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42177sec preferred_lft 42177sec 2024-06-10T00:29:15.475 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-06-10T00:29:15.475 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-06-10T00:29:15.475 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-10T00:29:15.476 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:29:15.476 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-06-10T00:29:15.476 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2024-06-10T00:29:15.476 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2024-06-10T00:29:15.476 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2024-06-10T00:29:15.476 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-10T00:29:15.476 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-06-10T00:29:15.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:29:15.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:29:15.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:29:15.690 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-10T00:29:15.716 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-06-10T00:29:15.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:29:15.717 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-06-10T00:29:15.784 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-06-10T00:29:15.784 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-10T00:29:15.784 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-10T00:29:15.784 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-10T00:29:15.784 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-10T00:29:15.784 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:29:15.784 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-06-10T00:29:15.785 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-10T00:29:15.785 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-10T00:29:15.785 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-10T00:29:15.785 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-06-10T00:29:15.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:29:15.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:15 smithi062 ceph-mon[32390]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:15.988 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:15 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1186852210' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T00:29:16.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:15 smithi105 ceph-mon[36321]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:15 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1186852210' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T00:29:16.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:29:16.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:29:16.120 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-06-10T00:29:16.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:15 smithi138 ceph-mon[36549]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:16.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:15 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1186852210' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T00:29:16.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:29:16.137 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-06-10T00:29:16.193 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:29:16.193 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-06-10T00:29:16.193 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T00:29:16.193 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-06-10T00:29:16.193 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-06-10T00:29:16.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:29:16.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:29:16.314 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-06-10T00:29:16.314 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:29:16.314 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-06-10T00:29:16.314 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-06-10T00:29:16.314 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-10T00:29:16.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-06-10T00:29:16.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-06-10T00:29:16.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-06-10T00:29:16.315 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-06-10T00:29:16.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:29:16.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:29:16.444 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:29:16.444 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-06-10T00:29:16.444 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-06-10T00:29:16.444 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2024-06-10T00:29:16.444 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-06-10T00:29:16.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:29:16.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:29:16.524 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-06-10T00:29:16.524 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T00:29:16.524 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:29:16.540 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-06-10T00:29:16.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T00:29:16.540 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:29:16.596 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2024-06-10T00:29:16.663 DEBUG:teuthology.orchestra.run.smithi062:> cat /proc/self/mounts | awk '{print $2}' 2024-06-10T00:29:16.720 INFO:teuthology.orchestra.run.smithi062.stdout:/proc 2024-06-10T00:29:16.720 INFO:teuthology.orchestra.run.smithi062.stdout:/sys 2024-06-10T00:29:16.720 INFO:teuthology.orchestra.run.smithi062.stdout:/dev 2024-06-10T00:29:16.720 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/security 2024-06-10T00:29:16.720 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/shm 2024-06-10T00:29:16.720 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/pts 2024-06-10T00:29:16.720 INFO:teuthology.orchestra.run.smithi062.stdout:/run 2024-06-10T00:29:16.721 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/cgroup 2024-06-10T00:29:16.721 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/pstore 2024-06-10T00:29:16.721 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/bpf 2024-06-10T00:29:16.721 INFO:teuthology.orchestra.run.smithi062.stdout:/ 2024-06-10T00:29:16.721 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/selinux 2024-06-10T00:29:16.721 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T00:29:16.721 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/mqueue 2024-06-10T00:29:16.721 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/hugepages 2024-06-10T00:29:16.721 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/debug 2024-06-10T00:29:16.721 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/tracing 2024-06-10T00:29:16.721 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/fuse/connections 2024-06-10T00:29:16.721 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/config 2024-06-10T00:29:16.721 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-sysctl.service 2024-06-10T00:29:16.722 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-10T00:29:16.722 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-10T00:29:16.722 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/nfs/rpc_pipefs 2024-06-10T00:29:16.722 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/1000 2024-06-10T00:29:16.722 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph 2024-06-10T00:29:16.722 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay 2024-06-10T00:29:16.722 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/dae8df919e97d91128fb3fd60f33503a0cb914e3bb00706e19d85f8e6b12e643/merged 2024-06-10T00:29:16.722 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/35ee8172746cbb5f492a80ef77f198b2653acb8cf177fab53e63a6b8877c0a8b/merged 2024-06-10T00:29:16.722 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/0 2024-06-10T00:29:16.722 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T00:29:16.722 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/1ce218c0bc2ba8ae5bf636ad46e1e6113340d3887eaa868e8b6c96c1ae84a33b/merged 2024-06-10T00:29:16.723 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/43580f48ccbeab9f7c1f502ff09057bae3c4cb2066f8e5a7b865d50099928f06/merged 2024-06-10T00:29:16.723 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/16455d74031b00973401a8c64655caac6e16965dc8a5c3f35b155ac00654de7e/merged 2024-06-10T00:29:16.723 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/2f0685da7d20619c2d57a5f5d1f97312b09b04d82f55a1dcfb1edcd3bc36c001/merged 2024-06-10T00:29:16.723 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/697c719ddb58bee74a6a50323f55ec64ee12d576ba3c53b928e5b2682714baa8/merged 2024-06-10T00:29:16.723 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/7c2da8a9407c524562f530f153297e336e9deb8d8f52537fb653079c91188072/merged 2024-06-10T00:29:16.723 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/3b9cf598ecc55081b57400305e9115213388716166b7e388e99bb33de59e31b9/merged 2024-06-10T00:29:16.723 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/52dc6242116fbd800466d94408f08b30173ec7942e7cd964d60e80f753f3c11f/merged 2024-06-10T00:29:16.723 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns 2024-06-10T00:29:16.723 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T00:29:16.723 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T00:29:16.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:29:16.724 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2024-06-10T00:29:16.776 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-10T00:29:16.777 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-06-10T00:29:16.819 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2024-06-10T00:29:16.845 DEBUG:teuthology.orchestra.run.smithi062:> cat /proc/self/mounts | awk '{print $2}' 2024-06-10T00:29:16.902 INFO:teuthology.orchestra.run.smithi062.stdout:/proc 2024-06-10T00:29:16.902 INFO:teuthology.orchestra.run.smithi062.stdout:/sys 2024-06-10T00:29:16.902 INFO:teuthology.orchestra.run.smithi062.stdout:/dev 2024-06-10T00:29:16.902 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/security 2024-06-10T00:29:16.903 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/shm 2024-06-10T00:29:16.903 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/pts 2024-06-10T00:29:16.903 INFO:teuthology.orchestra.run.smithi062.stdout:/run 2024-06-10T00:29:16.903 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/cgroup 2024-06-10T00:29:16.903 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/pstore 2024-06-10T00:29:16.903 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/bpf 2024-06-10T00:29:16.903 INFO:teuthology.orchestra.run.smithi062.stdout:/ 2024-06-10T00:29:16.903 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/selinux 2024-06-10T00:29:16.903 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T00:29:16.903 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/mqueue 2024-06-10T00:29:16.903 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/hugepages 2024-06-10T00:29:16.903 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/debug 2024-06-10T00:29:16.904 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/tracing 2024-06-10T00:29:16.904 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/fuse/connections 2024-06-10T00:29:16.904 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/config 2024-06-10T00:29:16.904 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-sysctl.service 2024-06-10T00:29:16.904 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-10T00:29:16.904 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-10T00:29:16.904 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/nfs/rpc_pipefs 2024-06-10T00:29:16.904 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/1000 2024-06-10T00:29:16.904 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph 2024-06-10T00:29:16.904 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay 2024-06-10T00:29:16.904 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/dae8df919e97d91128fb3fd60f33503a0cb914e3bb00706e19d85f8e6b12e643/merged 2024-06-10T00:29:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/35ee8172746cbb5f492a80ef77f198b2653acb8cf177fab53e63a6b8877c0a8b/merged 2024-06-10T00:29:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/0 2024-06-10T00:29:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T00:29:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/1ce218c0bc2ba8ae5bf636ad46e1e6113340d3887eaa868e8b6c96c1ae84a33b/merged 2024-06-10T00:29:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/43580f48ccbeab9f7c1f502ff09057bae3c4cb2066f8e5a7b865d50099928f06/merged 2024-06-10T00:29:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/16455d74031b00973401a8c64655caac6e16965dc8a5c3f35b155ac00654de7e/merged 2024-06-10T00:29:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/2f0685da7d20619c2d57a5f5d1f97312b09b04d82f55a1dcfb1edcd3bc36c001/merged 2024-06-10T00:29:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/697c719ddb58bee74a6a50323f55ec64ee12d576ba3c53b928e5b2682714baa8/merged 2024-06-10T00:29:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/7c2da8a9407c524562f530f153297e336e9deb8d8f52537fb653079c91188072/merged 2024-06-10T00:29:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/3b9cf598ecc55081b57400305e9115213388716166b7e388e99bb33de59e31b9/merged 2024-06-10T00:29:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/52dc6242116fbd800466d94408f08b30173ec7942e7cd964d60e80f753f3c11f/merged 2024-06-10T00:29:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns 2024-06-10T00:29:16.905 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T00:29:16.906 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T00:29:16.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:29:16.906 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2024-06-10T00:29:16.930 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi062.stderr:2024-06-10T00:29:16.930+0000 7f871cd29480 -1 init, newargv = 0x5613b1fad7e0 newargc=15 2024-06-10T00:29:16.930 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi062.stderr:ceph-fuse[78289]: starting ceph client 2024-06-10T00:29:16.936 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi062.stderr:ceph-fuse[78289]: starting fuse 2024-06-10T00:29:16.945 INFO:teuthology.orchestra.run.smithi062.stdout:119 2024-06-10T00:29:16.945 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-06-10T00:29:16.946 DEBUG:teuthology.orchestra.run.smithi062:> sudo stdin-killer -- python3 -c ' 2024-06-10T00:29:16.946 DEBUG:teuthology.orchestra.run.smithi062:> import glob 2024-06-10T00:29:16.946 DEBUG:teuthology.orchestra.run.smithi062:> import re 2024-06-10T00:29:16.946 DEBUG:teuthology.orchestra.run.smithi062:> import os 2024-06-10T00:29:16.946 DEBUG:teuthology.orchestra.run.smithi062:> import subprocess 2024-06-10T00:29:16.946 DEBUG:teuthology.orchestra.run.smithi062:> 2024-06-10T00:29:16.946 DEBUG:teuthology.orchestra.run.smithi062:> def _find_admin_socket(client_name): 2024-06-10T00:29:16.946 DEBUG:teuthology.orchestra.run.smithi062:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-06-10T00:29:16.946 DEBUG:teuthology.orchestra.run.smithi062:> files = glob.glob(asok_path) 2024-06-10T00:29:16.946 DEBUG:teuthology.orchestra.run.smithi062:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-06-10T00:29:16.946 DEBUG:teuthology.orchestra.run.smithi062:> 2024-06-10T00:29:16.946 DEBUG:teuthology.orchestra.run.smithi062:> # Given a non-glob path, it better be there 2024-06-10T00:29:16.946 DEBUG:teuthology.orchestra.run.smithi062:> if "*" not in asok_path: 2024-06-10T00:29:16.946 DEBUG:teuthology.orchestra.run.smithi062:> assert(len(files) == 1) 2024-06-10T00:29:16.947 DEBUG:teuthology.orchestra.run.smithi062:> return files[0] 2024-06-10T00:29:16.947 DEBUG:teuthology.orchestra.run.smithi062:> 2024-06-10T00:29:16.947 DEBUG:teuthology.orchestra.run.smithi062:> for f in files: 2024-06-10T00:29:16.947 DEBUG:teuthology.orchestra.run.smithi062:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-10T00:29:16.947 DEBUG:teuthology.orchestra.run.smithi062:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-10T00:29:16.947 DEBUG:teuthology.orchestra.run.smithi062:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-10T00:29:16.947 DEBUG:teuthology.orchestra.run.smithi062:> contents = proc_f.read() 2024-06-10T00:29:16.947 DEBUG:teuthology.orchestra.run.smithi062:> if mountpoint in contents: 2024-06-10T00:29:16.947 DEBUG:teuthology.orchestra.run.smithi062:> return f 2024-06-10T00:29:16.947 DEBUG:teuthology.orchestra.run.smithi062:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-10T00:29:16.947 DEBUG:teuthology.orchestra.run.smithi062:> 2024-06-10T00:29:16.947 DEBUG:teuthology.orchestra.run.smithi062:> print(_find_admin_socket("client.admin")) 2024-06-10T00:29:16.947 DEBUG:teuthology.orchestra.run.smithi062:> ' 2024-06-10T00:29:17.067 INFO:teuthology.orchestra.run.smithi062.stdout:/var/run/ceph/ceph-client.admin.78289.asok 2024-06-10T00:29:17.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:29:17.078 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.78289.asok 2024-06-10T00:29:17.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:29:17.078 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.78289.asok status 2024-06-10T00:29:17.217 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:29:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "metadata": { 2024-06-10T00:29:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-10T00:29:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T00:29:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "entity_id": "admin", 2024-06-10T00:29:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": "smithi062", 2024-06-10T00:29:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-06-10T00:29:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "pid": "78289", 2024-06-10T00:29:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "root": "/" 2024-06-10T00:29:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:29:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "dentry_count": 0, 2024-06-10T00:29:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "dentry_pinned_count": 0, 2024-06-10T00:29:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 14751, 2024-06-10T00:29:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "inst": { 2024-06-10T00:29:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "name": { 2024-06-10T00:29:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "client", 2024-06-10T00:29:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "num": 14751 2024-06-10T00:29:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:29:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": { 2024-06-10T00:29:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "192.168.0.1:0", 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 1989619619 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": { 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "192.168.0.1:0", 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 1989619619 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "inst_str": "client.14751 192.168.0.1:0/1989619619", 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "addr_str": "192.168.0.1:0/1989619619", 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "inode_count": 1, 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_epoch": 17, 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_epoch": 95, 2024-06-10T00:29:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_epoch_barrier": 0, 2024-06-10T00:29:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "blocklisted": false, 2024-06-10T00:29:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs" 2024-06-10T00:29:17.220 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:29:17.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:29:17.225 DEBUG:teuthology.orchestra.run.smithi062:> stat --file-system '--printf=%T 2024-06-10T00:29:17.225 DEBUG:teuthology.orchestra.run.smithi062:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:29:17.242 INFO:teuthology.orchestra.run.smithi062.stdout:fuseblk 2024-06-10T00:29:17.242 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:29:17.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:29:17.242 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:29:17.309 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-06-10T00:29:17.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T00:29:17.424 INFO:teuthology.orchestra.run.smithi062.stdout:# file: . 2024-06-10T00:29:17.424 INFO:teuthology.orchestra.run.smithi062.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-10T00:29:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:29:17.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:29:17.430 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:29:17.430 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T00:29:17.448 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-10T00:29:17.735 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:18.009 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:17 smithi062 ceph-mon[32390]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:18.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:17 smithi105 ceph-mon[36321]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:18.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:17 smithi138 ceph-mon[36549]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:19.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.448+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2450864220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc54806b590 msgr2=0x7fc54810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:19.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.448+0000 7fc54f5e8700 1 --2- 172.21.15.62:0/2450864220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc54806b590 0x7fc54810f3c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc54000b0a0 tx=0x7fc54000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:29:19.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.448+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2450864220 shutdown_connections 2024-06-10T00:29:19.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.448+0000 7fc54f5e8700 1 --2- 172.21.15.62:0/2450864220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc54806b590 0x7fc54810f3c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:19.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.448+0000 7fc54f5e8700 1 --2- 172.21.15.62:0/2450864220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc548108630 0x7fc548108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:19.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.448+0000 7fc54f5e8700 1 --2- 172.21.15.62:0/2450864220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54806abd0 0x7fc54806b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:19.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.448+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2450864220 >> 172.21.15.62:0/2450864220 conn(0x7fc548078190 msgr2=0x7fc5480785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.448+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2450864220 shutdown_connections 2024-06-10T00:29:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.448+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2450864220 wait complete. 2024-06-10T00:29:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.449+0000 7fc54f5e8700 1 Processor -- start 2024-06-10T00:29:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.449+0000 7fc54f5e8700 1 -- start start 2024-06-10T00:29:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.449+0000 7fc54f5e8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc54806abd0 0x7fc54819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.449+0000 7fc54f5e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54806b590 0x7fc54819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.449+0000 7fc54f5e8700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc548108630 0x7fc5481989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.449+0000 7fc54f5e8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481127f0 con 0x7fc54806abd0 2024-06-10T00:29:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.449+0000 7fc54f5e8700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc548112670 con 0x7fc548108630 2024-06-10T00:29:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.449+0000 7fc54f5e8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc548112970 con 0x7fc54806b590 2024-06-10T00:29:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.449+0000 7fc54d384700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc54806abd0 0x7fc54819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.449+0000 7fc54d384700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc54806abd0 0x7fc54819e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49822/0 (socket says 172.21.15.62:49822) 2024-06-10T00:29:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.449+0000 7fc54d384700 1 -- 172.21.15.62:0/2358298167 learned_addr learned my addr 172.21.15.62:0/2358298167 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.449+0000 7fc54db85700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc548108630 0x7fc5481989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.449+0000 7fc54cb83700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54806b590 0x7fc54819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.450+0000 7fc54d384700 1 -- 172.21.15.62:0/2358298167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54806b590 msgr2=0x7fc54819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.450+0000 7fc54d384700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54806b590 0x7fc54819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.450+0000 7fc54d384700 1 -- 172.21.15.62:0/2358298167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc548108630 msgr2=0x7fc5481989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.450+0000 7fc54d384700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc548108630 0x7fc5481989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.450+0000 7fc54d384700 1 -- 172.21.15.62:0/2358298167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc548199150 con 0x7fc54806abd0 2024-06-10T00:29:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.450+0000 7fc54cb83700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54806b590 0x7fc54819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:29:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.450+0000 7fc54d384700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc54806abd0 0x7fc54819e7e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc54400e9a0 tx=0x7fc54400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.450+0000 7fc53e7fc700 1 -- 172.21.15.62:0/2358298167 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54400cbd0 con 0x7fc54806abd0 2024-06-10T00:29:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.450+0000 7fc53e7fc700 1 -- 172.21.15.62:0/2358298167 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc54400cd70 con 0x7fc54806abd0 2024-06-10T00:29:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.450+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2358298167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc548199440 con 0x7fc54806abd0 2024-06-10T00:29:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.450+0000 7fc53e7fc700 1 -- 172.21.15.62:0/2358298167 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544018750 con 0x7fc54806abd0 2024-06-10T00:29:19.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.450+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2358298167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5481a56b0 con 0x7fc54806abd0 2024-06-10T00:29:19.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.451+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2358298167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc548069180 con 0x7fc54806abd0 2024-06-10T00:29:19.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.452+0000 7fc53e7fc700 1 -- 172.21.15.62:0/2358298167 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc544010430 con 0x7fc54806abd0 2024-06-10T00:29:19.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.453+0000 7fc53e7fc700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5340a2c20 0x7fc5340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:19.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.453+0000 7fc54cb83700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5340a2c20 0x7fc5340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:19.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.453+0000 7fc53e7fc700 1 -- 172.21.15.62:0/2358298167 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fc5440cf9e0 con 0x7fc54806abd0 2024-06-10T00:29:19.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.454+0000 7fc54cb83700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5340a2c20 0x7fc5340a50e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc538005950 tx=0x7fc5380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:19.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.457+0000 7fc53e7fc700 1 -- 172.21.15.62:0/2358298167 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc54409c700 con 0x7fc54806abd0 2024-06-10T00:29:19.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.680+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2358298167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc548199ed0 con 0x7fc54806abd0 2024-06-10T00:29:19.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.680+0000 7fc53e7fc700 1 -- 172.21.15.62:0/2358298167 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v95) ==== 81+0+588 (secure 0 0 0) 0x7fc5440bc0c0 con 0x7fc54806abd0 2024-06-10T00:29:19.680 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3856763121 2024-06-11T00:21:21.851532+0000 2024-06-10T00:29:19.681 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/381111141 2024-06-11T00:21:21.851532+0000 2024-06-10T00:29:19.681 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1704528283 2024-06-11T00:21:21.851532+0000 2024-06-10T00:29:19.681 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3912577307 2024-06-11T00:20:55.088039+0000 2024-06-10T00:29:19.681 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/178507377 2024-06-11T00:20:55.088039+0000 2024-06-10T00:29:19.681 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/4137526131 2024-06-11T00:20:55.088039+0000 2024-06-10T00:29:19.681 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/2327526412 2024-06-11T00:21:21.851532+0000 2024-06-10T00:29:19.681 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/594215923 2024-06-11T00:20:55.088039+0000 2024-06-10T00:29:19.681 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/594215923 2024-06-11T00:20:55.088039+0000 2024-06-10T00:29:19.681 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/2327526412 2024-06-11T00:21:21.851532+0000 2024-06-10T00:29:19.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.683+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2358298167 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5340a2c20 msgr2=0x7fc5340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:19.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.683+0000 7fc54f5e8700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5340a2c20 0x7fc5340a50e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc538005950 tx=0x7fc5380058e0 comp rx=0 tx=0).stop 2024-06-10T00:29:19.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.683+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2358298167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc54806abd0 msgr2=0x7fc54819e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:19.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.683+0000 7fc54f5e8700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc54806abd0 0x7fc54819e7e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc54400e9a0 tx=0x7fc54400ecb0 comp rx=0 tx=0).stop 2024-06-10T00:29:19.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.683+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2358298167 shutdown_connections 2024-06-10T00:29:19.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.683+0000 7fc54f5e8700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc54806abd0 0x7fc54819e7e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:19.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.683+0000 7fc54f5e8700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fc5340a2c20 0x7fc5340a50e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:19.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.683+0000 7fc54f5e8700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc548108630 0x7fc5481989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:19.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.683+0000 7fc54f5e8700 1 --2- 172.21.15.62:0/2358298167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54806b590 0x7fc54819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:19.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.683+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2358298167 >> 172.21.15.62:0/2358298167 conn(0x7fc548078190 msgr2=0x7fc548100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:19.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.683+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2358298167 shutdown_connections 2024-06-10T00:29:19.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:19.683+0000 7fc54f5e8700 1 -- 172.21.15.62:0/2358298167 wait complete. 2024-06-10T00:29:19.684 INFO:teuthology.orchestra.run.smithi062.stderr:listed 10 entries 2024-06-10T00:29:20.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:19 smithi105 ceph-mon[36321]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:20.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:19 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2358298167' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:29:20.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:19 smithi062 ceph-mon[32390]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:20.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:19 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2358298167' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:29:20.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:19 smithi138 ceph-mon[36549]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:20.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:19 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2358298167' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:29:20.322 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:29:20.322 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T00:29:20.340 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-10T00:29:20.630 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:22.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:21 smithi105 ceph-mon[36321]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:29:22.050 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:21 smithi062 ceph-mon[32390]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:29:22.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:21 smithi138 ceph-mon[36549]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:29:22.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.288+0000 7fd4fe631700 1 -- 172.21.15.62:0/1291223504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4f8107840 msgr2=0x7fd4f810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:22.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.288+0000 7fd4fe631700 1 --2- 172.21.15.62:0/1291223504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4f8107840 0x7fd4f810f3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000b0d0 tx=0x7fd4f000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:29:22.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.288+0000 7fd4fe631700 1 -- 172.21.15.62:0/1291223504 shutdown_connections 2024-06-10T00:29:22.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.288+0000 7fd4fe631700 1 --2- 172.21.15.62:0/1291223504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4f8107840 0x7fd4f810f3c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:22.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.288+0000 7fd4fe631700 1 --2- 172.21.15.62:0/1291223504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4f8102740 0x7fd4f8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:22.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.288+0000 7fd4fe631700 1 --2- 172.21.15.62:0/1291223504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f81030f0 0x7fd4f8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:22.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.288+0000 7fd4fe631700 1 -- 172.21.15.62:0/1291223504 >> 172.21.15.62:0/1291223504 conn(0x7fd4f80795f0 msgr2=0x7fd4f8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:22.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.288+0000 7fd4fe631700 1 -- 172.21.15.62:0/1291223504 shutdown_connections 2024-06-10T00:29:22.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.288+0000 7fd4fe631700 1 -- 172.21.15.62:0/1291223504 wait complete. 2024-06-10T00:29:22.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.289+0000 7fd4fe631700 1 Processor -- start 2024-06-10T00:29:22.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.289+0000 7fd4fe631700 1 -- start start 2024-06-10T00:29:22.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.289+0000 7fd4fe631700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4f8102740 0x7fd4f819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:22.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.289+0000 7fd4fe631700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4f81030f0 0x7fd4f819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:22.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.289+0000 7fd4fe631700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f8107840 0x7fd4f81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:22.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.289+0000 7fd4fe631700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81127d0 con 0x7fd4f81030f0 2024-06-10T00:29:22.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.289+0000 7fd4fe631700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8112650 con 0x7fd4f8102740 2024-06-10T00:29:22.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.289+0000 7fd4fe631700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8112950 con 0x7fd4f8107840 2024-06-10T00:29:22.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.289+0000 7fd4effff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4f81030f0 0x7fd4f819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:22.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.289+0000 7fd4f7fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4f8102740 0x7fd4f819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:22.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.289+0000 7fd4effff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4f81030f0 0x7fd4f819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49842/0 (socket says 172.21.15.62:49842) 2024-06-10T00:29:22.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.289+0000 7fd4effff700 1 -- 172.21.15.62:0/926939731 learned_addr learned my addr 172.21.15.62:0/926939731 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:22.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.289+0000 7fd4fcbce700 1 --2- 172.21.15.62:0/926939731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f8107840 0x7fd4f81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:22.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.290+0000 7fd4effff700 1 -- 172.21.15.62:0/926939731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f8107840 msgr2=0x7fd4f81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:22.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.290+0000 7fd4effff700 1 --2- 172.21.15.62:0/926939731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f8107840 0x7fd4f81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:22.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.290+0000 7fd4effff700 1 -- 172.21.15.62:0/926939731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4f8102740 msgr2=0x7fd4f819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:22.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.290+0000 7fd4effff700 1 --2- 172.21.15.62:0/926939731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4f8102740 0x7fd4f819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:22.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.290+0000 7fd4effff700 1 -- 172.21.15.62:0/926939731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f8199130 con 0x7fd4f81030f0 2024-06-10T00:29:22.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.290+0000 7fd4effff700 1 --2- 172.21.15.62:0/926939731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4f81030f0 0x7fd4f819ed50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000d980 tx=0x7fd4e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:22.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.290+0000 7fd4f5ffb700 1 -- 172.21.15.62:0/926939731 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0017070 con 0x7fd4f81030f0 2024-06-10T00:29:22.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.290+0000 7fd4f5ffb700 1 -- 172.21.15.62:0/926939731 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e0005020 con 0x7fd4f81030f0 2024-06-10T00:29:22.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.290+0000 7fd4f5ffb700 1 -- 172.21.15.62:0/926939731 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000fbf0 con 0x7fd4f81030f0 2024-06-10T00:29:22.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.290+0000 7fd4fe631700 1 -- 172.21.15.62:0/926939731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f8199420 con 0x7fd4f81030f0 2024-06-10T00:29:22.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.290+0000 7fd4fe631700 1 -- 172.21.15.62:0/926939731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f81a56e0 con 0x7fd4f81030f0 2024-06-10T00:29:22.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.291+0000 7fd4fe631700 1 -- 172.21.15.62:0/926939731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f8069180 con 0x7fd4f81030f0 2024-06-10T00:29:22.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.292+0000 7fd4f5ffb700 1 -- 172.21.15.62:0/926939731 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd4e00051d0 con 0x7fd4f81030f0 2024-06-10T00:29:22.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.293+0000 7fd4f5ffb700 1 --2- 172.21.15.62:0/926939731 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd4e40a2ca0 0x7fd4e40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:22.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.293+0000 7fd4f7fff700 1 --2- 172.21.15.62:0/926939731 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd4e40a2ca0 0x7fd4e40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:22.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.293+0000 7fd4f5ffb700 1 -- 172.21.15.62:0/926939731 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8802+0+0 (secure 0 0 0) 0x7fd4e00cf040 con 0x7fd4f81030f0 2024-06-10T00:29:22.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.293+0000 7fd4f7fff700 1 --2- 172.21.15.62:0/926939731 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd4e40a2ca0 0x7fd4e40a5160 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8199b10 tx=0x7fd4e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:22.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.297+0000 7fd4f5ffb700 1 -- 172.21.15.62:0/926939731 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd4e0020020 con 0x7fd4f81030f0 2024-06-10T00:29:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.523+0000 7fd4fe631700 1 -- 172.21.15.62:0/926939731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd4f802dca0 con 0x7fd4f81030f0 2024-06-10T00:29:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.523+0000 7fd4f5ffb700 1 -- 172.21.15.62:0/926939731 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v95) ==== 81+0+588 (secure 0 0 0) 0x7fd4e009bd60 con 0x7fd4f81030f0 2024-06-10T00:29:22.523 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3856763121 2024-06-11T00:21:21.851532+0000 2024-06-10T00:29:22.523 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/381111141 2024-06-11T00:21:21.851532+0000 2024-06-10T00:29:22.523 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1704528283 2024-06-11T00:21:21.851532+0000 2024-06-10T00:29:22.523 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3912577307 2024-06-11T00:20:55.088039+0000 2024-06-10T00:29:22.524 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/178507377 2024-06-11T00:20:55.088039+0000 2024-06-10T00:29:22.524 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/4137526131 2024-06-11T00:20:55.088039+0000 2024-06-10T00:29:22.524 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/2327526412 2024-06-11T00:21:21.851532+0000 2024-06-10T00:29:22.524 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/594215923 2024-06-11T00:20:55.088039+0000 2024-06-10T00:29:22.524 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/594215923 2024-06-11T00:20:55.088039+0000 2024-06-10T00:29:22.524 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/2327526412 2024-06-11T00:21:21.851532+0000 2024-06-10T00:29:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.525+0000 7fd4fe631700 1 -- 172.21.15.62:0/926939731 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd4e40a2ca0 msgr2=0x7fd4e40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.525+0000 7fd4fe631700 1 --2- 172.21.15.62:0/926939731 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd4e40a2ca0 0x7fd4e40a5160 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8199b10 tx=0x7fd4e800b410 comp rx=0 tx=0).stop 2024-06-10T00:29:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.525+0000 7fd4fe631700 1 -- 172.21.15.62:0/926939731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4f81030f0 msgr2=0x7fd4f819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.526+0000 7fd4fe631700 1 --2- 172.21.15.62:0/926939731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4f81030f0 0x7fd4f819ed50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000d980 tx=0x7fd4e000dc90 comp rx=0 tx=0).stop 2024-06-10T00:29:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.526+0000 7fd4fe631700 1 -- 172.21.15.62:0/926939731 shutdown_connections 2024-06-10T00:29:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.526+0000 7fd4fe631700 1 --2- 172.21.15.62:0/926939731 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd4e40a2ca0 0x7fd4e40a5160 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.526+0000 7fd4fe631700 1 --2- 172.21.15.62:0/926939731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f8107840 0x7fd4f81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.526+0000 7fd4fe631700 1 --2- 172.21.15.62:0/926939731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4f8102740 0x7fd4f819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.526+0000 7fd4fe631700 1 --2- 172.21.15.62:0/926939731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4f81030f0 0x7fd4f819ed50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.526+0000 7fd4fe631700 1 -- 172.21.15.62:0/926939731 >> 172.21.15.62:0/926939731 conn(0x7fd4f80795f0 msgr2=0x7fd4f8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.526+0000 7fd4fe631700 1 -- 172.21.15.62:0/926939731 shutdown_connections 2024-06-10T00:29:22.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:22.526+0000 7fd4fe631700 1 -- 172.21.15.62:0/926939731 wait complete. 2024-06-10T00:29:22.527 INFO:teuthology.orchestra.run.smithi062.stderr:listed 10 entries 2024-06-10T00:29:22.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-10T00:29:22.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-06-10T00:29:22.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-06-10T00:29:22.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-06-10T00:29:22.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-06-10T00:29:22.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-06-10T00:29:22.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-06-10T00:29:22.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-06-10T00:29:22.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 10]}]: dispatch 2024-06-10T00:29:22.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:22 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-06-10T00:29:22.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:22 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/926939731' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:29:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-10T00:29:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-06-10T00:29:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-06-10T00:29:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-06-10T00:29:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-06-10T00:29:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-06-10T00:29:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-06-10T00:29:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-06-10T00:29:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 10]}]: dispatch 2024-06-10T00:29:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:22 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-06-10T00:29:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:22 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/926939731' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:29:23.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-10T00:29:23.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-06-10T00:29:23.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-06-10T00:29:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-06-10T00:29:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-06-10T00:29:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-06-10T00:29:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-06-10T00:29:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-06-10T00:29:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 10]}]: dispatch 2024-06-10T00:29:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:22 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-06-10T00:29:23.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:22 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/926939731' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:29:23.173 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi062.front.sepia.ceph.com... 2024-06-10T00:29:23.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:29:23.173 DEBUG:teuthology.orchestra.run.smithi062:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:29:23.200 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi062.stderr:ceph-fuse[78289]: fuse finished with error 0 and tester_r 0 2024-06-10T00:29:23.203 INFO:teuthology.orchestra.run:waiting for 300 2024-06-10T00:29:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:23 smithi105 ceph-mon[36321]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:29:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-10T00:29:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-06-10T00:29:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-06-10T00:29:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-06-10T00:29:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-06-10T00:29:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-06-10T00:29:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-06-10T00:29:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-06-10T00:29:24.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 10]}]': finished 2024-06-10T00:29:24.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-06-10T00:29:24.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:23 smithi105 ceph-mon[36321]: osdmap e96: 12 total, 12 up, 12 in 2024-06-10T00:29:24.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:23 smithi062 ceph-mon[32390]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:29:24.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-10T00:29:24.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-06-10T00:29:24.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-06-10T00:29:24.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-06-10T00:29:24.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-06-10T00:29:24.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-06-10T00:29:24.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-06-10T00:29:24.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-06-10T00:29:24.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 10]}]': finished 2024-06-10T00:29:24.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-06-10T00:29:24.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:23 smithi062 ceph-mon[32390]: osdmap e96: 12 total, 12 up, 12 in 2024-06-10T00:29:24.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:23 smithi138 ceph-mon[36549]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:29:24.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-10T00:29:24.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-06-10T00:29:24.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-06-10T00:29:24.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-06-10T00:29:24.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-06-10T00:29:24.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-06-10T00:29:24.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-06-10T00:29:24.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-06-10T00:29:24.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 10]}]': finished 2024-06-10T00:29:24.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-06-10T00:29:24.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:23 smithi138 ceph-mon[36549]: osdmap e96: 12 total, 12 up, 12 in 2024-06-10T00:29:25.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:24 smithi105 ceph-mon[36321]: osdmap e97: 12 total, 12 up, 12 in 2024-06-10T00:29:25.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:24 smithi062 ceph-mon[32390]: osdmap e97: 12 total, 12 up, 12 in 2024-06-10T00:29:25.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:24 smithi138 ceph-mon[36549]: osdmap e97: 12 total, 12 up, 12 in 2024-06-10T00:29:26.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:25 smithi105 ceph-mon[36321]: pgmap v328: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-10T00:29:26.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:25 smithi062 ceph-mon[32390]: pgmap v328: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-10T00:29:26.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:25 smithi138 ceph-mon[36549]: pgmap v328: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-10T00:29:28.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:27 smithi062 ceph-mon[32390]: pgmap v329: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-10T00:29:28.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:27 smithi138 ceph-mon[36549]: pgmap v329: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-10T00:29:28.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:27 smithi105 ceph-mon[36321]: pgmap v329: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-10T00:29:29.305 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-06-10T00:29:29.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:29:29.305 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-06-10T00:29:29.349 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:29:29.648 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:30.024 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:29 smithi062 ceph-mon[32390]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-06-10T00:29:30.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:29 smithi138 ceph-mon[36549]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-06-10T00:29:30.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:29 smithi105 ceph-mon[36321]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-06-10T00:29:31.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.356+0000 7f2a71fa4700 1 -- 172.21.15.62:0/2180028263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a6c108500 msgr2=0x7f2a6c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:31.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.356+0000 7f2a71fa4700 1 --2- 172.21.15.62:0/2180028263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a6c108500 0x7f2a6c1088e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009990 tx=0x7f2a5c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:29:31.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.356+0000 7f2a71fa4700 1 -- 172.21.15.62:0/2180028263 shutdown_connections 2024-06-10T00:29:31.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.356+0000 7f2a71fa4700 1 --2- 172.21.15.62:0/2180028263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a6c108500 0x7f2a6c1088e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:31.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.356+0000 7f2a71fa4700 1 --2- 172.21.15.62:0/2180028263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6c06ab30 0x7f2a6c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:31.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.356+0000 7f2a71fa4700 1 --2- 172.21.15.62:0/2180028263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a6c06a170 0x7f2a6c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:31.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.356+0000 7f2a71fa4700 1 -- 172.21.15.62:0/2180028263 >> 172.21.15.62:0/2180028263 conn(0x7f2a6c078340 msgr2=0x7f2a6c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:31.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.356+0000 7f2a71fa4700 1 -- 172.21.15.62:0/2180028263 shutdown_connections 2024-06-10T00:29:31.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.356+0000 7f2a71fa4700 1 -- 172.21.15.62:0/2180028263 wait complete. 2024-06-10T00:29:31.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.357+0000 7f2a71fa4700 1 Processor -- start 2024-06-10T00:29:31.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.357+0000 7f2a71fa4700 1 -- start start 2024-06-10T00:29:31.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.357+0000 7f2a71fa4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6c06a170 0x7f2a6c101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:31.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.357+0000 7f2a71fa4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a6c06ab30 0x7f2a6c1019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:31.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.357+0000 7f2a71fa4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a6c108500 0x7f2a6c101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:31.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.357+0000 7f2a71fa4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c112760 con 0x7f2a6c06ab30 2024-06-10T00:29:31.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.357+0000 7f2a71fa4700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1125e0 con 0x7f2a6c108500 2024-06-10T00:29:31.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.357+0000 7f2a71fa4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1128e0 con 0x7f2a6c06a170 2024-06-10T00:29:31.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a6affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a6c06ab30 0x7f2a6c1019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:31.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a6affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a6c06ab30 0x7f2a6c1019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46912/0 (socket says 172.21.15.62:46912) 2024-06-10T00:29:31.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a6b7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6c06a170 0x7f2a6c101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:31.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a6affd700 1 -- 172.21.15.62:0/359598981 learned_addr learned my addr 172.21.15.62:0/359598981 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:31.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a6bfff700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a6c108500 0x7f2a6c101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:31.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a6affd700 1 -- 172.21.15.62:0/359598981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6c06a170 msgr2=0x7f2a6c101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:31.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a6affd700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6c06a170 0x7f2a6c101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:31.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a6affd700 1 -- 172.21.15.62:0/359598981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a6c108500 msgr2=0x7f2a6c101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:31.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a6affd700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a6c108500 0x7f2a6c101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:31.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a6affd700 1 -- 172.21.15.62:0/359598981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c1a7110 con 0x7f2a6c06ab30 2024-06-10T00:29:31.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a6b7fe700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6c06a170 0x7f2a6c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:29:31.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a6bfff700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a6c108500 0x7f2a6c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:29:31.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a6affd700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a6c06ab30 0x7f2a6c1019a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400e9a0 tx=0x7f2a5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:31.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a68ff9700 1 -- 172.21.15.62:0/359598981 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400ca10 con 0x7f2a6c06ab30 2024-06-10T00:29:31.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a71fa4700 1 -- 172.21.15.62:0/359598981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6c1a7400 con 0x7f2a6c06ab30 2024-06-10T00:29:31.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a68ff9700 1 -- 172.21.15.62:0/359598981 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5400cbb0 con 0x7f2a6c06ab30 2024-06-10T00:29:31.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a68ff9700 1 -- 172.21.15.62:0/359598981 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54018680 con 0x7f2a6c06ab30 2024-06-10T00:29:31.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.358+0000 7f2a71fa4700 1 -- 172.21.15.62:0/359598981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6c1a7990 con 0x7f2a6c06ab30 2024-06-10T00:29:31.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.360+0000 7f2a71fa4700 1 -- 172.21.15.62:0/359598981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a6c10a9e0 con 0x7f2a6c06ab30 2024-06-10T00:29:31.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.361+0000 7f2a68ff9700 1 -- 172.21.15.62:0/359598981 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2a54010430 con 0x7f2a6c06ab30 2024-06-10T00:29:31.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.361+0000 7f2a68ff9700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2a580a2c20 0x7f2a580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:31.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.361+0000 7f2a6b7fe700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2a580a2c20 0x7f2a580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:31.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.361+0000 7f2a68ff9700 1 -- 172.21.15.62:0/359598981 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f2a540cf5e0 con 0x7f2a6c06ab30 2024-06-10T00:29:31.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.362+0000 7f2a6b7fe700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2a580a2c20 0x7f2a580a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0098e0 tx=0x7f2a5c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:31.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.366+0000 7f2a68ff9700 1 -- 172.21.15.62:0/359598981 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2a5409c1e0 con 0x7f2a6c06ab30 2024-06-10T00:29:31.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.574+0000 7f2a71fa4700 1 -- 172.21.15.62:0/359598981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a6c102940 con 0x7f2a6c06ab30 2024-06-10T00:29:31.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.576+0000 7f2a68ff9700 1 -- 172.21.15.62:0/359598981 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24653 (secure 0 0 0) 0x7f2a5409bae0 con 0x7f2a6c06ab30 2024-06-10T00:29:31.576 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:29:31.576 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":97,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:29:23.767202+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:29:31.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.578+0000 7f2a71fa4700 1 -- 172.21.15.62:0/359598981 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2a580a2c20 msgr2=0x7f2a580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:31.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.578+0000 7f2a71fa4700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2a580a2c20 0x7f2a580a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0098e0 tx=0x7f2a5c005960 comp rx=0 tx=0).stop 2024-06-10T00:29:31.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.578+0000 7f2a71fa4700 1 -- 172.21.15.62:0/359598981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a6c06ab30 msgr2=0x7f2a6c1019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.578+0000 7f2a71fa4700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a6c06ab30 0x7f2a6c1019a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400e9a0 tx=0x7f2a5400ecb0 comp rx=0 tx=0).stop 2024-06-10T00:29:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.578+0000 7f2a71fa4700 1 -- 172.21.15.62:0/359598981 shutdown_connections 2024-06-10T00:29:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.578+0000 7f2a71fa4700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2a580a2c20 0x7f2a580a50e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.578+0000 7f2a71fa4700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a6c108500 0x7f2a6c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.578+0000 7f2a71fa4700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6c06a170 0x7f2a6c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.578+0000 7f2a71fa4700 1 --2- 172.21.15.62:0/359598981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a6c06ab30 0x7f2a6c1019a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.578+0000 7f2a71fa4700 1 -- 172.21.15.62:0/359598981 >> 172.21.15.62:0/359598981 conn(0x7f2a6c078340 msgr2=0x7f2a6c0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.579+0000 7f2a71fa4700 1 -- 172.21.15.62:0/359598981 shutdown_connections 2024-06-10T00:29:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:31.579+0000 7f2a71fa4700 1 -- 172.21.15.62:0/359598981 wait complete. 2024-06-10T00:29:31.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:31 smithi062 ceph-mon[32390]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-06-10T00:29:31.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:31 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:29:31.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:31 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/359598981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:32.051 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:31 smithi138 ceph-mon[36549]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-06-10T00:29:32.051 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:31 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:29:32.051 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:31 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/359598981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:32.063 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:31 smithi105 ceph-mon[36321]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 72 B/s, 0 objects/s recovering 2024-06-10T00:29:32.063 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:31 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:29:32.063 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:31 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/359598981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:32.290 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:29:33.185 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:33.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:33 smithi062 ceph-mon[32390]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 111 B/s wr, 0 op/s; 63 B/s, 0 objects/s recovering 2024-06-10T00:29:33.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:33.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:33.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:33.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:33 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:34.077 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:33 smithi138 ceph-mon[36549]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 111 B/s wr, 0 op/s; 63 B/s, 0 objects/s recovering 2024-06-10T00:29:34.077 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:34.077 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:34.077 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:34.077 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:33 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:34.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:33 smithi105 ceph-mon[36321]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 111 B/s wr, 0 op/s; 63 B/s, 0 objects/s recovering 2024-06-10T00:29:34.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:34.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:34.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:34.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:33 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:34.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.917+0000 7fd25f673700 1 -- 172.21.15.62:0/2809194110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd258102030 msgr2=0x7fd258105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:34.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.917+0000 7fd25f673700 1 --2- 172.21.15.62:0/2809194110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd258102030 0x7fd258105ff0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd2480099c0 tx=0x7fd248009cd0 comp rx=0 tx=0).stop 2024-06-10T00:29:34.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.917+0000 7fd25f673700 1 -- 172.21.15.62:0/2809194110 shutdown_connections 2024-06-10T00:29:34.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.917+0000 7fd25f673700 1 --2- 172.21.15.62:0/2809194110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2581066d0 0x7fd25810e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:34.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.917+0000 7fd25f673700 1 --2- 172.21.15.62:0/2809194110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd258101680 0x7fd258101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:34.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.917+0000 7fd25f673700 1 --2- 172.21.15.62:0/2809194110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd258102030 0x7fd258105ff0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:34.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.917+0000 7fd25f673700 1 -- 172.21.15.62:0/2809194110 >> 172.21.15.62:0/2809194110 conn(0x7fd2580773e0 msgr2=0x7fd2580777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:34.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.917+0000 7fd25f673700 1 -- 172.21.15.62:0/2809194110 shutdown_connections 2024-06-10T00:29:34.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.917+0000 7fd25f673700 1 -- 172.21.15.62:0/2809194110 wait complete. 2024-06-10T00:29:34.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.917+0000 7fd25f673700 1 Processor -- start 2024-06-10T00:29:34.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.917+0000 7fd25f673700 1 -- start start 2024-06-10T00:29:34.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25f673700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd258101680 0x7fd258103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:34.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25f673700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd258102030 0x7fd2581035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:34.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25f673700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2581066d0 0x7fd258103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:34.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25f673700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd258111710 con 0x7fd258101680 2024-06-10T00:29:34.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25f673700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd258111590 con 0x7fd2581066d0 2024-06-10T00:29:34.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25f673700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd258111890 con 0x7fd258102030 2024-06-10T00:29:34.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25d40f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd258101680 0x7fd258103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:34.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25d40f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd258101680 0x7fd258103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46934/0 (socket says 172.21.15.62:46934) 2024-06-10T00:29:34.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25cc0e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd258102030 0x7fd2581035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:34.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25d40f700 1 -- 172.21.15.62:0/2279140898 learned_addr learned my addr 172.21.15.62:0/2279140898 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:34.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25dc10700 1 --2- 172.21.15.62:0/2279140898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2581066d0 0x7fd258103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:34.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25d40f700 1 -- 172.21.15.62:0/2279140898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd258102030 msgr2=0x7fd2581035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:34.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25d40f700 1 --2- 172.21.15.62:0/2279140898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd258102030 0x7fd2581035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:34.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25d40f700 1 -- 172.21.15.62:0/2279140898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2581066d0 msgr2=0x7fd258103b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:34.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25d40f700 1 --2- 172.21.15.62:0/2279140898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2581066d0 0x7fd258103b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:34.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.918+0000 7fd25d40f700 1 -- 172.21.15.62:0/2279140898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2581a7220 con 0x7fd258101680 2024-06-10T00:29:34.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.919+0000 7fd25d40f700 1 --2- 172.21.15.62:0/2279140898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd258101680 0x7fd258103090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd25400b530 tx=0x7fd25400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:34.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.919+0000 7fd24e7fc700 1 -- 172.21.15.62:0/2279140898 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254015020 con 0x7fd258101680 2024-06-10T00:29:34.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.919+0000 7fd24e7fc700 1 -- 172.21.15.62:0/2279140898 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd254011d10 con 0x7fd258101680 2024-06-10T00:29:34.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.919+0000 7fd24e7fc700 1 -- 172.21.15.62:0/2279140898 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25400f2e0 con 0x7fd258101680 2024-06-10T00:29:34.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.919+0000 7fd25f673700 1 -- 172.21.15.62:0/2279140898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2581a7510 con 0x7fd258101680 2024-06-10T00:29:34.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.919+0000 7fd25f673700 1 -- 172.21.15.62:0/2279140898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2581a79d0 con 0x7fd258101680 2024-06-10T00:29:34.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.920+0000 7fd25f673700 1 -- 172.21.15.62:0/2279140898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd258069180 con 0x7fd258101680 2024-06-10T00:29:34.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.922+0000 7fd24e7fc700 1 -- 172.21.15.62:0/2279140898 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd254018020 con 0x7fd258101680 2024-06-10T00:29:34.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.922+0000 7fd24e7fc700 1 --2- 172.21.15.62:0/2279140898 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd2440a2ca0 0x7fd2440a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:34.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.922+0000 7fd25cc0e700 1 --2- 172.21.15.62:0/2279140898 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd2440a2ca0 0x7fd2440a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:34.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.922+0000 7fd24e7fc700 1 -- 172.21.15.62:0/2279140898 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fd2540cee50 con 0x7fd258101680 2024-06-10T00:29:34.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.923+0000 7fd25cc0e700 1 --2- 172.21.15.62:0/2279140898 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd2440a2ca0 0x7fd2440a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd248000c00 tx=0x7fd248005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:34.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:34.926+0000 7fd24e7fc700 1 -- 172.21.15.62:0/2279140898 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd25401f020 con 0x7fd258101680 2024-06-10T00:29:35.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:35.142+0000 7fd25f673700 1 -- 172.21.15.62:0/2279140898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2581a7d30 con 0x7fd258101680 2024-06-10T00:29:35.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:35.144+0000 7fd24e7fc700 1 -- 172.21.15.62:0/2279140898 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24653 (secure 0 0 0) 0x7fd25409ba50 con 0x7fd258101680 2024-06-10T00:29:35.144 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:29:35.144 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":97,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:29:23.767202+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:29:35.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:35.146+0000 7fd25f673700 1 -- 172.21.15.62:0/2279140898 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd2440a2ca0 msgr2=0x7fd2440a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:35.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:35.146+0000 7fd25f673700 1 --2- 172.21.15.62:0/2279140898 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd2440a2ca0 0x7fd2440a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd248000c00 tx=0x7fd248005fb0 comp rx=0 tx=0).stop 2024-06-10T00:29:35.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:35.146+0000 7fd25f673700 1 -- 172.21.15.62:0/2279140898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd258101680 msgr2=0x7fd258103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:35.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:35.146+0000 7fd25f673700 1 --2- 172.21.15.62:0/2279140898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd258101680 0x7fd258103090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd25400b530 tx=0x7fd25400b840 comp rx=0 tx=0).stop 2024-06-10T00:29:35.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:35.146+0000 7fd25f673700 1 -- 172.21.15.62:0/2279140898 shutdown_connections 2024-06-10T00:29:35.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:35.146+0000 7fd25f673700 1 --2- 172.21.15.62:0/2279140898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd258101680 0x7fd258103090 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:35.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:35.146+0000 7fd25f673700 1 --2- 172.21.15.62:0/2279140898 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd2440a2ca0 0x7fd2440a5160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:35.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:35.146+0000 7fd25f673700 1 --2- 172.21.15.62:0/2279140898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2581066d0 0x7fd258103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:35.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:35.146+0000 7fd25f673700 1 --2- 172.21.15.62:0/2279140898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd258102030 0x7fd2581035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:35.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:35.146+0000 7fd25f673700 1 -- 172.21.15.62:0/2279140898 >> 172.21.15.62:0/2279140898 conn(0x7fd2580773e0 msgr2=0x7fd258105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:35.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:35.146+0000 7fd25f673700 1 -- 172.21.15.62:0/2279140898 shutdown_connections 2024-06-10T00:29:35.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:35.146+0000 7fd25f673700 1 -- 172.21.15.62:0/2279140898 wait complete. 2024-06-10T00:29:35.814 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-10T00:29:36.077 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:36.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:35 smithi062 ceph-mon[32390]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 100 B/s wr, 0 op/s; 57 B/s, 0 objects/s recovering 2024-06-10T00:29:36.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:36.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:35 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:36.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:35 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2279140898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:36.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:35 smithi138 ceph-mon[36549]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 100 B/s wr, 0 op/s; 57 B/s, 0 objects/s recovering 2024-06-10T00:29:36.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:36.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:35 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:36.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:35 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2279140898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:36.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:35 smithi105 ceph-mon[36321]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 100 B/s wr, 0 op/s; 57 B/s, 0 objects/s recovering 2024-06-10T00:29:36.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:36.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:35 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:36.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:35 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2279140898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:29:37.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:36 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:29:37.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:36 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:29:37.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:36 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:37.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:36 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:29:37.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:36 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:29:37.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:36 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:37.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:36 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:29:37.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:36 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:29:37.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:36 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:29:37.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.790+0000 7f1884557700 1 -- 172.21.15.62:0/175960134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c102530 msgr2=0x7f187c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:37.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.790+0000 7f1884557700 1 --2- 172.21.15.62:0/175960134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c102530 0x7f187c1029b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f186c0099c0 tx=0x7f186c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:29:37.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.790+0000 7f1884557700 1 -- 172.21.15.62:0/175960134 shutdown_connections 2024-06-10T00:29:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.790+0000 7f1884557700 1 --2- 172.21.15.62:0/175960134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f187c102ef0 0x7f187c10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.790+0000 7f1884557700 1 --2- 172.21.15.62:0/175960134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f187c1085d0 0x7f187c1089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.790+0000 7f1884557700 1 --2- 172.21.15.62:0/175960134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c102530 0x7f187c1029b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.790+0000 7f1884557700 1 -- 172.21.15.62:0/175960134 >> 172.21.15.62:0/175960134 conn(0x7f187c079640 msgr2=0x7f187c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.790+0000 7f1884557700 1 -- 172.21.15.62:0/175960134 shutdown_connections 2024-06-10T00:29:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.790+0000 7f1884557700 1 -- 172.21.15.62:0/175960134 wait complete. 2024-06-10T00:29:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1884557700 1 Processor -- start 2024-06-10T00:29:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1884557700 1 -- start start 2024-06-10T00:29:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1884557700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f187c102530 0x7f187c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1884557700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f187c102ef0 0x7f187c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1884557700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c1085d0 0x7f187c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1884557700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f187c112810 con 0x7f187c1085d0 2024-06-10T00:29:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1884557700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f187c112690 con 0x7f187c102530 2024-06-10T00:29:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1884557700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f187c112990 con 0x7f187c102ef0 2024-06-10T00:29:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1882af4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c1085d0 0x7f187c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1882af4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c1085d0 0x7f187c075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46944/0 (socket says 172.21.15.62:46944) 2024-06-10T00:29:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1881af2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f187c102ef0 0x7f187c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1882af4700 1 -- 172.21.15.62:0/2027446457 learned_addr learned my addr 172.21.15.62:0/2027446457 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f18822f3700 1 --2- 172.21.15.62:0/2027446457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f187c102530 0x7f187c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1882af4700 1 -- 172.21.15.62:0/2027446457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f187c102ef0 msgr2=0x7f187c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1882af4700 1 --2- 172.21.15.62:0/2027446457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f187c102ef0 0x7f187c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1882af4700 1 -- 172.21.15.62:0/2027446457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f187c102530 msgr2=0x7f187c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1882af4700 1 --2- 172.21.15.62:0/2027446457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f187c102530 0x7f187c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.791+0000 7f1882af4700 1 -- 172.21.15.62:0/2027446457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c1a7150 con 0x7f187c1085d0 2024-06-10T00:29:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.792+0000 7f1882af4700 1 --2- 172.21.15.62:0/2027446457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c1085d0 0x7f187c075210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1874007ee0 tx=0x7f187400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.792+0000 7f18737fe700 1 -- 172.21.15.62:0/2027446457 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874004bb0 con 0x7f187c1085d0 2024-06-10T00:29:37.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.792+0000 7f18737fe700 1 -- 172.21.15.62:0/2027446457 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1874004d50 con 0x7f187c1085d0 2024-06-10T00:29:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.792+0000 7f1884557700 1 -- 172.21.15.62:0/2027446457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187c1a7440 con 0x7f187c1085d0 2024-06-10T00:29:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.792+0000 7f18737fe700 1 -- 172.21.15.62:0/2027446457 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874005960 con 0x7f187c1085d0 2024-06-10T00:29:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.792+0000 7f1884557700 1 -- 172.21.15.62:0/2027446457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187c1a7900 con 0x7f187c1085d0 2024-06-10T00:29:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.793+0000 7f1884557700 1 -- 172.21.15.62:0/2027446457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187c103c40 con 0x7f187c1085d0 2024-06-10T00:29:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.794+0000 7f18737fe700 1 -- 172.21.15.62:0/2027446457 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f187401a020 con 0x7f187c1085d0 2024-06-10T00:29:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.795+0000 7f18737fe700 1 --2- 172.21.15.62:0/2027446457 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f18680a2c20 0x7f18680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.795+0000 7f18737fe700 1 -- 172.21.15.62:0/2027446457 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f18740ceae0 con 0x7f187c1085d0 2024-06-10T00:29:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.795+0000 7f18822f3700 1 --2- 172.21.15.62:0/2027446457 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f18680a2c20 0x7f18680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:37.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.795+0000 7f18822f3700 1 --2- 172.21.15.62:0/2027446457 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f18680a2c20 0x7f18680a50e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1878009be0 tx=0x7f1878009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:37.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:37.799+0000 7f18737fe700 1 -- 172.21.15.62:0/2027446457 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1874010070 con 0x7f187c1085d0 2024-06-10T00:29:38.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:38.060+0000 7f1884557700 1 -- 172.21.15.62:0/2027446457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f187c02de60 con 0x7f187c1085d0 2024-06-10T00:29:38.061 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:37 smithi062 ceph-mon[32390]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 48 B/s, 0 objects/s recovering 2024-06-10T00:29:38.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:37 smithi138 ceph-mon[36549]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 48 B/s, 0 objects/s recovering 2024-06-10T00:29:38.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:37 smithi105 ceph-mon[36321]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 48 B/s, 0 objects/s recovering 2024-06-10T00:29:38.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:38.817+0000 7f18737fe700 1 -- 172.21.15.62:0/2027446457 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f187409b6e0 con 0x7f187c1085d0 2024-06-10T00:29:38.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:38.822+0000 7f1884557700 1 -- 172.21.15.62:0/2027446457 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f18680a2c20 msgr2=0x7f18680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:38.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:38.822+0000 7f1884557700 1 --2- 172.21.15.62:0/2027446457 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f18680a2c20 0x7f18680a50e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1878009be0 tx=0x7f1878009260 comp rx=0 tx=0).stop 2024-06-10T00:29:38.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:38.822+0000 7f1884557700 1 -- 172.21.15.62:0/2027446457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c1085d0 msgr2=0x7f187c075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:38.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:38.822+0000 7f1884557700 1 --2- 172.21.15.62:0/2027446457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c1085d0 0x7f187c075210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1874007ee0 tx=0x7f187400d3b0 comp rx=0 tx=0).stop 2024-06-10T00:29:38.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:38.822+0000 7f1884557700 1 -- 172.21.15.62:0/2027446457 shutdown_connections 2024-06-10T00:29:38.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:38.822+0000 7f1884557700 1 --2- 172.21.15.62:0/2027446457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c1085d0 0x7f187c075210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:38.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:38.822+0000 7f1884557700 1 --2- 172.21.15.62:0/2027446457 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f18680a2c20 0x7f18680a50e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:38.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:38.822+0000 7f1884557700 1 --2- 172.21.15.62:0/2027446457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f187c102530 0x7f187c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:38.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:38.822+0000 7f1884557700 1 --2- 172.21.15.62:0/2027446457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f187c102ef0 0x7f187c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:38.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:38.822+0000 7f1884557700 1 -- 172.21.15.62:0/2027446457 >> 172.21.15.62:0/2027446457 conn(0x7f187c079640 msgr2=0x7f187c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:38.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:38.822+0000 7f1884557700 1 -- 172.21.15.62:0/2027446457 shutdown_connections 2024-06-10T00:29:38.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:38.822+0000 7f1884557700 1 -- 172.21.15.62:0/2027446457 wait complete. 2024-06-10T00:29:39.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:38 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2027446457' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-10T00:29:39.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:38 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2027446457' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-10T00:29:39.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:38 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2027446457' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-10T00:29:39.427 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-06-10T00:29:39.428 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-06-10T00:29:39.684 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:39 smithi062 ceph-mon[32390]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 48 B/s, 0 objects/s recovering 2024-06-10T00:29:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:39 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2027446457' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-10T00:29:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:39 smithi062 ceph-mon[32390]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:29:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:39 smithi138 ceph-mon[36549]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 48 B/s, 0 objects/s recovering 2024-06-10T00:29:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:39 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2027446457' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-10T00:29:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:39 smithi138 ceph-mon[36549]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:29:40.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:39 smithi105 ceph-mon[36321]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 48 B/s, 0 objects/s recovering 2024-06-10T00:29:40.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:39 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2027446457' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-10T00:29:40.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:39 smithi105 ceph-mon[36321]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:29:41.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:40 smithi062 ceph-mon[32390]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-10T00:29:41.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:40 smithi138 ceph-mon[36549]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-10T00:29:41.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:40 smithi105 ceph-mon[36321]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-10T00:29:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.294+0000 7f4a68e05700 1 -- 172.21.15.62:0/1697458624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a64108580 msgr2=0x7f4a64108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.294+0000 7f4a68e05700 1 --2- 172.21.15.62:0/1697458624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a64108580 0x7f4a64108960 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4a540099c0 tx=0x7f4a54009cd0 comp rx=0 tx=0).stop 2024-06-10T00:29:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.294+0000 7f4a68e05700 1 -- 172.21.15.62:0/1697458624 shutdown_connections 2024-06-10T00:29:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.294+0000 7f4a68e05700 1 --2- 172.21.15.62:0/1697458624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a64108580 0x7f4a64108960 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.294+0000 7f4a68e05700 1 --2- 172.21.15.62:0/1697458624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a6406b620 0x7f4a6410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.294+0000 7f4a68e05700 1 --2- 172.21.15.62:0/1697458624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a6406ac60 0x7f4a6406b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.294+0000 7f4a68e05700 1 -- 172.21.15.62:0/1697458624 >> 172.21.15.62:0/1697458624 conn(0x7f4a64078430 msgr2=0x7f4a64078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.294+0000 7f4a68e05700 1 -- 172.21.15.62:0/1697458624 shutdown_connections 2024-06-10T00:29:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.294+0000 7f4a68e05700 1 -- 172.21.15.62:0/1697458624 wait complete. 2024-06-10T00:29:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.295+0000 7f4a68e05700 1 Processor -- start 2024-06-10T00:29:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.295+0000 7f4a68e05700 1 -- start start 2024-06-10T00:29:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.295+0000 7f4a68e05700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a6406ac60 0x7f4a6419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.295+0000 7f4a68e05700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a6406b620 0x7f4a6419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.295+0000 7f4a68e05700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a64108580 0x7f4a641989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.295+0000 7f4a68e05700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4a641127d0 con 0x7f4a6406b620 2024-06-10T00:29:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.295+0000 7f4a68e05700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4a64112650 con 0x7f4a6406ac60 2024-06-10T00:29:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.295+0000 7f4a68e05700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a64112950 con 0x7f4a64108580 2024-06-10T00:29:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.295+0000 7f4a61d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a6406b620 0x7f4a6419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.295+0000 7f4a61d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a6406b620 0x7f4a6419ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57076/0 (socket says 172.21.15.62:57076) 2024-06-10T00:29:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.295+0000 7f4a61d9b700 1 -- 172.21.15.62:0/379540678 learned_addr learned my addr 172.21.15.62:0/379540678 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.295+0000 7f4a62d9d700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a64108580 0x7f4a641989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.295+0000 7f4a6259c700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a6406ac60 0x7f4a6419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.296+0000 7f4a61d9b700 1 -- 172.21.15.62:0/379540678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a64108580 msgr2=0x7f4a641989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.296+0000 7f4a61d9b700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a64108580 0x7f4a641989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.296+0000 7f4a61d9b700 1 -- 172.21.15.62:0/379540678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a6406ac60 msgr2=0x7f4a6419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.296+0000 7f4a61d9b700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a6406ac60 0x7f4a6419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.296+0000 7f4a61d9b700 1 -- 172.21.15.62:0/379540678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a64199130 con 0x7f4a6406b620 2024-06-10T00:29:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.296+0000 7f4a6259c700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a6406ac60 0x7f4a6419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:29:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.296+0000 7f4a61d9b700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a6406b620 0x7f4a6419ed00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c9b0 tx=0x7f4a5800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.296+0000 7f4a537fe700 1 -- 172.21.15.62:0/379540678 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58007b00 con 0x7f4a6406b620 2024-06-10T00:29:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.296+0000 7f4a537fe700 1 -- 172.21.15.62:0/379540678 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a58007ca0 con 0x7f4a6406b620 2024-06-10T00:29:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.296+0000 7f4a68e05700 1 -- 172.21.15.62:0/379540678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a64199420 con 0x7f4a6406b620 2024-06-10T00:29:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.296+0000 7f4a537fe700 1 -- 172.21.15.62:0/379540678 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58018730 con 0x7f4a6406b620 2024-06-10T00:29:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.296+0000 7f4a68e05700 1 -- 172.21.15.62:0/379540678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a641a5690 con 0x7f4a6406b620 2024-06-10T00:29:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.297+0000 7f4a68e05700 1 -- 172.21.15.62:0/379540678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a6410aa10 con 0x7f4a6406b620 2024-06-10T00:29:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.298+0000 7f4a537fe700 1 -- 172.21.15.62:0/379540678 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4a580188d0 con 0x7f4a6406b620 2024-06-10T00:29:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.299+0000 7f4a537fe700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f4a4c0a2ca0 0x7f4a4c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.299+0000 7f4a537fe700 1 -- 172.21.15.62:0/379540678 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f4a580cf330 con 0x7f4a6406b620 2024-06-10T00:29:41.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.299+0000 7f4a6259c700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f4a4c0a2ca0 0x7f4a4c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:41.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.299+0000 7f4a6259c700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f4a4c0a2ca0 0x7f4a4c0a5160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a54000c00 tx=0x7f4a54005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:41.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.303+0000 7f4a537fe700 1 -- 172.21.15.62:0/379540678 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4a58014030 con 0x7f4a6406b620 2024-06-10T00:29:41.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.558+0000 7f4a68e05700 1 -- 172.21.15.62:0/379540678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f4a64199eb0 con 0x7f4a6406b620 2024-06-10T00:29:41.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.834+0000 7f4a537fe700 1 -- 172.21.15.62:0/379540678 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f4a5809bf30 con 0x7f4a6406b620 2024-06-10T00:29:41.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.836+0000 7f4a68e05700 1 -- 172.21.15.62:0/379540678 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f4a4c0a2ca0 msgr2=0x7f4a4c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:41.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.836+0000 7f4a68e05700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f4a4c0a2ca0 0x7f4a4c0a5160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a54000c00 tx=0x7f4a54005940 comp rx=0 tx=0).stop 2024-06-10T00:29:41.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.836+0000 7f4a68e05700 1 -- 172.21.15.62:0/379540678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a6406b620 msgr2=0x7f4a6419ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:41.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.836+0000 7f4a68e05700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a6406b620 0x7f4a6419ed00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c9b0 tx=0x7f4a5800ccc0 comp rx=0 tx=0).stop 2024-06-10T00:29:41.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.837+0000 7f4a68e05700 1 -- 172.21.15.62:0/379540678 shutdown_connections 2024-06-10T00:29:41.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.837+0000 7f4a68e05700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f4a4c0a2ca0 0x7f4a4c0a5160 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:41.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.837+0000 7f4a68e05700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a64108580 0x7f4a641989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:41.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.837+0000 7f4a68e05700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a6406ac60 0x7f4a6419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:41.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.837+0000 7f4a68e05700 1 --2- 172.21.15.62:0/379540678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a6406b620 0x7f4a6419ed00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:41.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.837+0000 7f4a68e05700 1 -- 172.21.15.62:0/379540678 >> 172.21.15.62:0/379540678 conn(0x7f4a64078430 msgr2=0x7f4a640ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:41.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.837+0000 7f4a68e05700 1 -- 172.21.15.62:0/379540678 shutdown_connections 2024-06-10T00:29:41.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:41.837+0000 7f4a68e05700 1 -- 172.21.15.62:0/379540678 wait complete. 2024-06-10T00:29:42.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:41 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/379540678' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-10T00:29:42.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:41 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/379540678' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-10T00:29:42.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:41 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/379540678' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-10T00:29:42.616 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-10T00:29:42.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:42 smithi062 ceph-mon[32390]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-10T00:29:42.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:42 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/379540678' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-10T00:29:42.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:42 smithi062 ceph-mon[32390]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-10T00:29:42.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:42 smithi062 ceph-mon[32390]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:29:42.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:42 smithi062 ceph-mon[32390]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-10T00:29:42.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:42 smithi062 ceph-mon[32390]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-10T00:29:42.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:42 smithi062 ceph-mon[32390]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:43.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:42 smithi138 ceph-mon[36549]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-10T00:29:43.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:42 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/379540678' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-10T00:29:43.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:42 smithi138 ceph-mon[36549]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-10T00:29:43.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:42 smithi138 ceph-mon[36549]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:29:43.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:42 smithi138 ceph-mon[36549]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-10T00:29:43.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:42 smithi138 ceph-mon[36549]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-10T00:29:43.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:42 smithi138 ceph-mon[36549]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:43.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:42 smithi105 ceph-mon[36321]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-10T00:29:43.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:42 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/379540678' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-10T00:29:43.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:42 smithi105 ceph-mon[36321]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-10T00:29:43.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:42 smithi105 ceph-mon[36321]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-10T00:29:43.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:42 smithi105 ceph-mon[36321]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-10T00:29:43.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:42 smithi105 ceph-mon[36321]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-10T00:29:43.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:42 smithi105 ceph-mon[36321]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:29:43.999 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:43 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] up:active 2024-06-10T00:29:43.999 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:43 smithi062 ceph-mon[32390]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-10T00:29:43.999 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:43 smithi062 ceph-mon[32390]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-10T00:29:44.000 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:43 smithi062 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-06-10T00:29:44.000 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:43 smithi062 ceph-mon[32390]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-10T00:29:44.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:43 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] up:active 2024-06-10T00:29:44.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:43 smithi138 ceph-mon[36549]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-10T00:29:44.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:43 smithi138 ceph-mon[36549]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-10T00:29:44.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:43 smithi138 ceph-mon[36549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-06-10T00:29:44.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:43 smithi138 ceph-mon[36549]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-10T00:29:44.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:43 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] up:active 2024-06-10T00:29:44.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:43 smithi105 ceph-mon[36321]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-10T00:29:44.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:43 smithi105 ceph-mon[36321]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-10T00:29:44.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:43 smithi105 ceph-mon[36321]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-06-10T00:29:44.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:43 smithi105 ceph-mon[36321]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-10T00:29:44.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.512+0000 7f9668d74700 1 -- 172.21.15.62:0/1532934470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9664100ec0 msgr2=0x7f9664101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:44.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.512+0000 7f9668d74700 1 --2- 172.21.15.62:0/1532934470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9664100ec0 0x7f9664101340 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9658009990 tx=0x7f9658009ca0 comp rx=0 tx=0).stop 2024-06-10T00:29:44.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.512+0000 7f9668d74700 1 -- 172.21.15.62:0/1532934470 shutdown_connections 2024-06-10T00:29:44.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.512+0000 7f9668d74700 1 --2- 172.21.15.62:0/1532934470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9664101880 0x7f966410f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:44.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.512+0000 7f9668d74700 1 --2- 172.21.15.62:0/1532934470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664108670 0x7f9664108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:44.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.512+0000 7f9668d74700 1 --2- 172.21.15.62:0/1532934470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9664100ec0 0x7f9664101340 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:44.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.512+0000 7f9668d74700 1 -- 172.21.15.62:0/1532934470 >> 172.21.15.62:0/1532934470 conn(0x7f9664076600 msgr2=0x7f9664076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:44.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.512+0000 7f9668d74700 1 -- 172.21.15.62:0/1532934470 shutdown_connections 2024-06-10T00:29:44.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.512+0000 7f9668d74700 1 -- 172.21.15.62:0/1532934470 wait complete. 2024-06-10T00:29:44.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9668d74700 1 Processor -- start 2024-06-10T00:29:44.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9668d74700 1 -- start start 2024-06-10T00:29:44.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9668d74700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664100ec0 0x7f96641020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:44.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9668d74700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9664101880 0x7f96641025e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:44.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9668d74700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9664108670 0x7f9664102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:44.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9668d74700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9664112870 con 0x7f9664108670 2024-06-10T00:29:44.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9668d74700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f96641126f0 con 0x7f9664100ec0 2024-06-10T00:29:44.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9668d74700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96641129f0 con 0x7f9664101880 2024-06-10T00:29:44.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9662d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9664108670 0x7f9664102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:44.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9662d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9664108670 0x7f9664102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57104/0 (socket says 172.21.15.62:57104) 2024-06-10T00:29:44.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9662d9d700 1 -- 172.21.15.62:0/1185774490 learned_addr learned my addr 172.21.15.62:0/1185774490 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:44.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f966259c700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664100ec0 0x7f96641020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:44.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9661d9b700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9664101880 0x7f96641025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:44.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9662d9d700 1 -- 172.21.15.62:0/1185774490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9664101880 msgr2=0x7f96641025e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:44.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9662d9d700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9664101880 0x7f96641025e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:44.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.513+0000 7f9662d9d700 1 -- 172.21.15.62:0/1185774490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664100ec0 msgr2=0x7f96641020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:44.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.514+0000 7f9662d9d700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664100ec0 0x7f96641020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:44.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.514+0000 7f9662d9d700 1 -- 172.21.15.62:0/1185774490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96641a7290 con 0x7f9664108670 2024-06-10T00:29:44.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.514+0000 7f966259c700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664100ec0 0x7f96641020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:29:44.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.514+0000 7f9662d9d700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9664108670 0x7f9664102b20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f965c00b970 tx=0x7f965c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:44.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.514+0000 7f96537fe700 1 -- 172.21.15.62:0/1185774490 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c019070 con 0x7f9664108670 2024-06-10T00:29:44.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.514+0000 7f96537fe700 1 -- 172.21.15.62:0/1185774490 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f965c014440 con 0x7f9664108670 2024-06-10T00:29:44.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.514+0000 7f9668d74700 1 -- 172.21.15.62:0/1185774490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96641a7580 con 0x7f9664108670 2024-06-10T00:29:44.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.514+0000 7f96537fe700 1 -- 172.21.15.62:0/1185774490 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c0075d0 con 0x7f9664108670 2024-06-10T00:29:44.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.514+0000 7f9668d74700 1 -- 172.21.15.62:0/1185774490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96641a7b10 con 0x7f9664108670 2024-06-10T00:29:44.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.515+0000 7f9668d74700 1 -- 172.21.15.62:0/1185774490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9664069180 con 0x7f9664108670 2024-06-10T00:29:44.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.517+0000 7f96537fe700 1 -- 172.21.15.62:0/1185774490 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f965c01c070 con 0x7f9664108670 2024-06-10T00:29:44.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.517+0000 7f96537fe700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f964c0a2ca0 0x7f964c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:44.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.518+0000 7f966259c700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f964c0a2ca0 0x7f964c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:44.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.518+0000 7f96537fe700 1 -- 172.21.15.62:0/1185774490 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f965c0ce790 con 0x7f9664108670 2024-06-10T00:29:44.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.518+0000 7f966259c700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f964c0a2ca0 0x7f964c0a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f96540099d0 tx=0x7f9654008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:44.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.523+0000 7f96537fe700 1 -- 172.21.15.62:0/1185774490 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f965c09b330 con 0x7f9664108670 2024-06-10T00:29:44.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.776+0000 7f9668d74700 1 -- 172.21.15.62:0/1185774490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f966402dca0 con 0x7f9664108670 2024-06-10T00:29:44.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.866+0000 7f96537fe700 1 -- 172.21.15.62:0/1185774490 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f965c09ac30 con 0x7f9664108670 2024-06-10T00:29:44.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.868+0000 7f9668d74700 1 -- 172.21.15.62:0/1185774490 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f964c0a2ca0 msgr2=0x7f964c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:44.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.868+0000 7f9668d74700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f964c0a2ca0 0x7f964c0a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f96540099d0 tx=0x7f9654008040 comp rx=0 tx=0).stop 2024-06-10T00:29:44.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.868+0000 7f9668d74700 1 -- 172.21.15.62:0/1185774490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9664108670 msgr2=0x7f9664102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:44.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.868+0000 7f9668d74700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9664108670 0x7f9664102b20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f965c00b970 tx=0x7f965c00bd30 comp rx=0 tx=0).stop 2024-06-10T00:29:44.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.868+0000 7f9668d74700 1 -- 172.21.15.62:0/1185774490 shutdown_connections 2024-06-10T00:29:44.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.868+0000 7f9668d74700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9664108670 0x7f9664102b20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:44.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.868+0000 7f9668d74700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f964c0a2ca0 0x7f964c0a5160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:44.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.868+0000 7f9668d74700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664100ec0 0x7f96641020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:44.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.868+0000 7f9668d74700 1 --2- 172.21.15.62:0/1185774490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9664101880 0x7f96641025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:44.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.868+0000 7f9668d74700 1 -- 172.21.15.62:0/1185774490 >> 172.21.15.62:0/1185774490 conn(0x7f9664076600 msgr2=0x7f96640ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:44.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.868+0000 7f9668d74700 1 -- 172.21.15.62:0/1185774490 shutdown_connections 2024-06-10T00:29:44.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:44.868+0000 7f9668d74700 1 -- 172.21.15.62:0/1185774490 wait complete. 2024-06-10T00:29:45.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:44 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] up:active 2024-06-10T00:29:45.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:44 smithi062 ceph-mon[32390]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-10T00:29:45.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:44 smithi062 ceph-mon[32390]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-10T00:29:45.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:44 smithi062 ceph-mon[32390]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-06-10T00:29:45.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:44 smithi062 ceph-mon[32390]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-06-10T00:29:45.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:44 smithi062 ceph-mon[32390]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 3 op/s 2024-06-10T00:29:45.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:44 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1185774490' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-10T00:29:45.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:44 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] up:active 2024-06-10T00:29:45.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:44 smithi138 ceph-mon[36549]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-10T00:29:45.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:44 smithi138 ceph-mon[36549]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-10T00:29:45.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:44 smithi138 ceph-mon[36549]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-06-10T00:29:45.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:44 smithi138 ceph-mon[36549]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-06-10T00:29:45.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:44 smithi138 ceph-mon[36549]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 3 op/s 2024-06-10T00:29:45.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:44 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1185774490' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-10T00:29:45.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:44 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] up:active 2024-06-10T00:29:45.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:44 smithi105 ceph-mon[36321]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-10T00:29:45.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:44 smithi105 ceph-mon[36321]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-10T00:29:45.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:44 smithi105 ceph-mon[36321]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-06-10T00:29:45.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:44 smithi105 ceph-mon[36321]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-06-10T00:29:45.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:44 smithi105 ceph-mon[36321]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 3 op/s 2024-06-10T00:29:45.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:44 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1185774490' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-10T00:29:45.524 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-10T00:29:45.780 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:45 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1185774490' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-10T00:29:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:45 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] up:active 2024-06-10T00:29:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:45 smithi138 ceph-mon[36549]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-10T00:29:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:45 smithi138 ceph-mon[36549]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-10T00:29:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:45 smithi138 ceph-mon[36549]: Cluster is now healthy 2024-06-10T00:29:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:45 smithi138 ceph-mon[36549]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-06-10T00:29:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:45 smithi138 ceph-mon[36549]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-10T00:29:46.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:45 smithi138 ceph-mon[36549]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-06-10T00:29:46.135 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:45 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1185774490' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-10T00:29:46.135 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:45 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] up:active 2024-06-10T00:29:46.136 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:45 smithi062 ceph-mon[32390]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-10T00:29:46.136 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:45 smithi062 ceph-mon[32390]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-10T00:29:46.136 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:45 smithi062 ceph-mon[32390]: Cluster is now healthy 2024-06-10T00:29:46.136 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:45 smithi062 ceph-mon[32390]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-06-10T00:29:46.136 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:45 smithi062 ceph-mon[32390]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-10T00:29:46.136 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:45 smithi062 ceph-mon[32390]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-06-10T00:29:46.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:45 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1185774490' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-10T00:29:46.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:45 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] up:active 2024-06-10T00:29:46.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:45 smithi105 ceph-mon[36321]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-10T00:29:46.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:45 smithi105 ceph-mon[36321]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-10T00:29:46.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:45 smithi105 ceph-mon[36321]: Cluster is now healthy 2024-06-10T00:29:46.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:45 smithi105 ceph-mon[36321]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-06-10T00:29:46.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:45 smithi105 ceph-mon[36321]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-10T00:29:46.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:45 smithi105 ceph-mon[36321]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-06-10T00:29:47.198 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:46 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] up:active 2024-06-10T00:29:47.198 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:46 smithi062 ceph-mon[32390]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-10T00:29:47.198 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:46 smithi062 ceph-mon[32390]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:29:47.198 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:46 smithi062 ceph-mon[32390]: pgmap v339: 129 pgs: 129 active+clean; 583 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 5 op/s 2024-06-10T00:29:47.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:46 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] up:active 2024-06-10T00:29:47.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:46 smithi105 ceph-mon[36321]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-10T00:29:47.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:46 smithi105 ceph-mon[36321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:29:47.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:46 smithi105 ceph-mon[36321]: pgmap v339: 129 pgs: 129 active+clean; 583 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 5 op/s 2024-06-10T00:29:47.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:46 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] up:active 2024-06-10T00:29:47.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:46 smithi138 ceph-mon[36549]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-10T00:29:47.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:46 smithi138 ceph-mon[36549]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:29:47.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:46 smithi138 ceph-mon[36549]: pgmap v339: 129 pgs: 129 active+clean; 583 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 5 op/s 2024-06-10T00:29:47.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.448+0000 7f7a21870700 1 -- 172.21.15.62:0/1371276823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a1c06ac60 msgr2=0x7f7a1c06b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:47.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.448+0000 7f7a21870700 1 --2- 172.21.15.62:0/1371276823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a1c06ac60 0x7f7a1c06b0e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7a100099c0 tx=0x7f7a10009cd0 comp rx=0 tx=0).stop 2024-06-10T00:29:47.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.449+0000 7f7a21870700 1 -- 172.21.15.62:0/1371276823 shutdown_connections 2024-06-10T00:29:47.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.449+0000 7f7a21870700 1 --2- 172.21.15.62:0/1371276823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c06b620 0x7f7a1c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:47.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.449+0000 7f7a21870700 1 --2- 172.21.15.62:0/1371276823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a1c108580 0x7f7a1c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:47.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.449+0000 7f7a21870700 1 --2- 172.21.15.62:0/1371276823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a1c06ac60 0x7f7a1c06b0e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:47.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.449+0000 7f7a21870700 1 -- 172.21.15.62:0/1371276823 >> 172.21.15.62:0/1371276823 conn(0x7f7a1c078430 msgr2=0x7f7a1c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:47.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.449+0000 7f7a21870700 1 -- 172.21.15.62:0/1371276823 shutdown_connections 2024-06-10T00:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.449+0000 7f7a21870700 1 -- 172.21.15.62:0/1371276823 wait complete. 2024-06-10T00:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.449+0000 7f7a21870700 1 Processor -- start 2024-06-10T00:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a21870700 1 -- start start 2024-06-10T00:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a21870700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a1c06ac60 0x7f7a1c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a21870700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c06b620 0x7f7a1c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a21870700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a1c108580 0x7f7a1c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a21870700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c112830 con 0x7f7a1c108580 2024-06-10T00:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a21870700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1126b0 con 0x7f7a1c06ac60 2024-06-10T00:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a21870700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1129b0 con 0x7f7a1c06b620 2024-06-10T00:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a1b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a1c108580 0x7f7a1c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a1b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a1c108580 0x7f7a1c1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57134/0 (socket says 172.21.15.62:57134) 2024-06-10T00:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a1b7fe700 1 -- 172.21.15.62:0/1849087910 learned_addr learned my addr 172.21.15.62:0/1849087910 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a1affd700 1 --2- 172.21.15.62:0/1849087910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a1c06ac60 0x7f7a1c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a1a7fc700 1 --2- 172.21.15.62:0/1849087910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c06b620 0x7f7a1c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a1b7fe700 1 -- 172.21.15.62:0/1849087910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c06b620 msgr2=0x7f7a1c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a1b7fe700 1 --2- 172.21.15.62:0/1849087910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c06b620 0x7f7a1c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a1b7fe700 1 -- 172.21.15.62:0/1849087910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a1c06ac60 msgr2=0x7f7a1c19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a1b7fe700 1 --2- 172.21.15.62:0/1849087910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a1c06ac60 0x7f7a1c19e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.450+0000 7f7a1b7fe700 1 -- 172.21.15.62:0/1849087910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c199110 con 0x7f7a1c108580 2024-06-10T00:29:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.451+0000 7f7a1b7fe700 1 --2- 172.21.15.62:0/1849087910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a1c108580 0x7f7a1c1989b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400b970 tx=0x7f7a1400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.451+0000 7f7a2086e700 1 -- 172.21.15.62:0/1849087910 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1400d860 con 0x7f7a1c108580 2024-06-10T00:29:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.451+0000 7f7a21870700 1 -- 172.21.15.62:0/1849087910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1c199400 con 0x7f7a1c108580 2024-06-10T00:29:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.451+0000 7f7a21870700 1 -- 172.21.15.62:0/1849087910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1c1a56c0 con 0x7f7a1c108580 2024-06-10T00:29:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.451+0000 7f7a2086e700 1 -- 172.21.15.62:0/1849087910 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a1400e430 con 0x7f7a1c108580 2024-06-10T00:29:47.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.451+0000 7f7a2086e700 1 -- 172.21.15.62:0/1849087910 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14017660 con 0x7f7a1c108580 2024-06-10T00:29:47.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.452+0000 7f7a21870700 1 -- 172.21.15.62:0/1849087910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a08005330 con 0x7f7a1c108580 2024-06-10T00:29:47.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.453+0000 7f7a2086e700 1 -- 172.21.15.62:0/1849087910 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7a1400da00 con 0x7f7a1c108580 2024-06-10T00:29:47.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.454+0000 7f7a2086e700 1 --2- 172.21.15.62:0/1849087910 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7a040a2ca0 0x7f7a040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:47.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.454+0000 7f7a1affd700 1 --2- 172.21.15.62:0/1849087910 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7a040a2ca0 0x7f7a040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:47.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.454+0000 7f7a2086e700 1 -- 172.21.15.62:0/1849087910 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f7a140cee10 con 0x7f7a1c108580 2024-06-10T00:29:47.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.455+0000 7f7a1affd700 1 --2- 172.21.15.62:0/1849087910 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7a040a2ca0 0x7f7a040a5160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c199af0 tx=0x7f7a0c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:47.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.459+0000 7f7a2086e700 1 -- 172.21.15.62:0/1849087910 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7a14013030 con 0x7f7a1c108580 2024-06-10T00:29:47.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.711+0000 7f7a21870700 1 -- 172.21.15.62:0/1849087910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f7a08005620 con 0x7f7a1c108580 2024-06-10T00:29:47.881 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:29:47 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-b[66851]: 2024-06-10T00:29:47.586+0000 7f9075648700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-06-10T00:29:47.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.885+0000 7f7a2086e700 1 -- 172.21.15.62:0/1849087910 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7f7a1409ba10 con 0x7f7a1c108580 2024-06-10T00:29:47.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.888+0000 7f7a21870700 1 -- 172.21.15.62:0/1849087910 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7a040a2ca0 msgr2=0x7f7a040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:47.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.888+0000 7f7a21870700 1 --2- 172.21.15.62:0/1849087910 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7a040a2ca0 0x7f7a040a5160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c199af0 tx=0x7f7a0c00a380 comp rx=0 tx=0).stop 2024-06-10T00:29:47.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.888+0000 7f7a21870700 1 -- 172.21.15.62:0/1849087910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a1c108580 msgr2=0x7f7a1c1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:47.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.888+0000 7f7a21870700 1 --2- 172.21.15.62:0/1849087910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a1c108580 0x7f7a1c1989b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400b970 tx=0x7f7a1400bc80 comp rx=0 tx=0).stop 2024-06-10T00:29:47.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.888+0000 7f7a21870700 1 -- 172.21.15.62:0/1849087910 shutdown_connections 2024-06-10T00:29:47.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.888+0000 7f7a21870700 1 --2- 172.21.15.62:0/1849087910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a1c108580 0x7f7a1c1989b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:47.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.888+0000 7f7a21870700 1 --2- 172.21.15.62:0/1849087910 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7a040a2ca0 0x7f7a040a5160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:47.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.888+0000 7f7a21870700 1 --2- 172.21.15.62:0/1849087910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a1c06ac60 0x7f7a1c19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:47.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.888+0000 7f7a21870700 1 --2- 172.21.15.62:0/1849087910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c06b620 0x7f7a1c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:47.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.888+0000 7f7a21870700 1 -- 172.21.15.62:0/1849087910 >> 172.21.15.62:0/1849087910 conn(0x7f7a1c078430 msgr2=0x7f7a1c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:47.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.888+0000 7f7a21870700 1 -- 172.21.15.62:0/1849087910 shutdown_connections 2024-06-10T00:29:47.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:47.889+0000 7f7a21870700 1 -- 172.21.15.62:0/1849087910 wait complete. 2024-06-10T00:29:48.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:47 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1849087910' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-10T00:29:48.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:47 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1849087910' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-10T00:29:48.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:47 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1849087910' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-10T00:29:48.619 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-06-10T00:29:48.620 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-10T00:29:48.869 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:49.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:48 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1849087910' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-10T00:29:49.141 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:48 smithi062 ceph-mon[32390]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:29:49.142 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:48 smithi062 ceph-mon[32390]: pgmap v340: 129 pgs: 129 active+clean; 586 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-06-10T00:29:49.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:48 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1849087910' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-10T00:29:49.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:48 smithi105 ceph-mon[36321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:29:49.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:48 smithi105 ceph-mon[36321]: pgmap v340: 129 pgs: 129 active+clean; 586 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-06-10T00:29:49.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:48 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1849087910' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-10T00:29:49.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:48 smithi138 ceph-mon[36549]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:29:49.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:48 smithi138 ceph-mon[36549]: pgmap v340: 129 pgs: 129 active+clean; 586 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-06-10T00:29:50.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.594+0000 7f63f0019700 1 -- 172.21.15.62:0/2198343160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e810eec0 msgr2=0x7f63e81112b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:50.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.594+0000 7f63f0019700 1 --2- 172.21.15.62:0/2198343160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e810eec0 0x7f63e81112b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009990 tx=0x7f63e0009ca0 comp rx=0 tx=0).stop 2024-06-10T00:29:50.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.594+0000 7f63f0019700 1 -- 172.21.15.62:0/2198343160 shutdown_connections 2024-06-10T00:29:50.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.594+0000 7f63f0019700 1 --2- 172.21.15.62:0/2198343160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e810eec0 0x7f63e81112b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:50.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.594+0000 7f63f0019700 1 --2- 172.21.15.62:0/2198343160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63e80746c0 0x7f63e8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:50.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.594+0000 7f63f0019700 1 --2- 172.21.15.62:0/2198343160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e8074fe0 0x7f63e810e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:50.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.594+0000 7f63f0019700 1 -- 172.21.15.62:0/2198343160 >> 172.21.15.62:0/2198343160 conn(0x7f63e8079640 msgr2=0x7f63e8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:50.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.594+0000 7f63f0019700 1 -- 172.21.15.62:0/2198343160 shutdown_connections 2024-06-10T00:29:50.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.594+0000 7f63f0019700 1 -- 172.21.15.62:0/2198343160 wait complete. 2024-06-10T00:29:50.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.595+0000 7f63f0019700 1 Processor -- start 2024-06-10T00:29:50.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.595+0000 7f63f0019700 1 -- start start 2024-06-10T00:29:50.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63f0019700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e80746c0 0x7f63e819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:50.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63f0019700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e8074fe0 0x7f63e819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:50.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63f0019700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63e810eec0 0x7f63e819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:50.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63f0019700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f63e8114af0 con 0x7f63e80746c0 2024-06-10T00:29:50.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63f0019700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f63e8114970 con 0x7f63e8074fe0 2024-06-10T00:29:50.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63f0019700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f63e8114c70 con 0x7f63e810eec0 2024-06-10T00:29:50.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63eddb5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e80746c0 0x7f63e819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:50.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63ed5b4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e8074fe0 0x7f63e819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:50.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63eddb5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e80746c0 0x7f63e819afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40020/0 (socket says 172.21.15.62:40020) 2024-06-10T00:29:50.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63eddb5700 1 -- 172.21.15.62:0/4100546877 learned_addr learned my addr 172.21.15.62:0/4100546877 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:50.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63ee5b6700 1 --2- 172.21.15.62:0/4100546877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63e810eec0 0x7f63e819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:50.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63eddb5700 1 -- 172.21.15.62:0/4100546877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63e810eec0 msgr2=0x7f63e819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:50.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63eddb5700 1 --2- 172.21.15.62:0/4100546877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63e810eec0 0x7f63e819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:50.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63eddb5700 1 -- 172.21.15.62:0/4100546877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e8074fe0 msgr2=0x7f63e819b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:50.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63eddb5700 1 --2- 172.21.15.62:0/4100546877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e8074fe0 0x7f63e819b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:50.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63eddb5700 1 -- 172.21.15.62:0/4100546877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e819ff60 con 0x7f63e80746c0 2024-06-10T00:29:50.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.596+0000 7f63eddb5700 1 --2- 172.21.15.62:0/4100546877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e80746c0 0x7f63e819afa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f63d800ba50 tx=0x7f63d800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:50.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.597+0000 7f63e6ffd700 1 -- 172.21.15.62:0/4100546877 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8019070 con 0x7f63e80746c0 2024-06-10T00:29:50.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.597+0000 7f63f0019700 1 -- 172.21.15.62:0/4100546877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e81a0250 con 0x7f63e80746c0 2024-06-10T00:29:50.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.597+0000 7f63e6ffd700 1 -- 172.21.15.62:0/4100546877 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d800ce10 con 0x7f63e80746c0 2024-06-10T00:29:50.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.597+0000 7f63e6ffd700 1 -- 172.21.15.62:0/4100546877 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8007370 con 0x7f63e80746c0 2024-06-10T00:29:50.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.597+0000 7f63f0019700 1 -- 172.21.15.62:0/4100546877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e81a7ad0 con 0x7f63e80746c0 2024-06-10T00:29:50.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.598+0000 7f63f0019700 1 -- 172.21.15.62:0/4100546877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e8069180 con 0x7f63e80746c0 2024-06-10T00:29:50.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.599+0000 7f63e6ffd700 1 -- 172.21.15.62:0/4100546877 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f63d801c070 con 0x7f63e80746c0 2024-06-10T00:29:50.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.599+0000 7f63e6ffd700 1 --2- 172.21.15.62:0/4100546877 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f63d00a2c20 0x7f63d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:50.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.600+0000 7f63ed5b4700 1 --2- 172.21.15.62:0/4100546877 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f63d00a2c20 0x7f63d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:50.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.600+0000 7f63e6ffd700 1 -- 172.21.15.62:0/4100546877 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f63d80ce6f0 con 0x7f63e80746c0 2024-06-10T00:29:50.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.600+0000 7f63ed5b4700 1 --2- 172.21.15.62:0/4100546877 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f63d00a2c20 0x7f63d00a50e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f63e8101fd0 tx=0x7f63dc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:50.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.604+0000 7f63e6ffd700 1 -- 172.21.15.62:0/4100546877 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f63d809b2f0 con 0x7f63e80746c0 2024-06-10T00:29:50.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.842+0000 7f63f0019700 1 -- 172.21.15.62:0/4100546877 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f63e80628d0 con 0x7f63d00a2c20 2024-06-10T00:29:50.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.871+0000 7f63e6ffd700 1 -- 172.21.15.62:0/4100546877 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f63e80628d0 con 0x7f63d00a2c20 2024-06-10T00:29:50.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.874+0000 7f63f0019700 1 -- 172.21.15.62:0/4100546877 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f63d00a2c20 msgr2=0x7f63d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:50.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.874+0000 7f63f0019700 1 --2- 172.21.15.62:0/4100546877 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f63d00a2c20 0x7f63d00a50e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f63e8101fd0 tx=0x7f63dc009190 comp rx=0 tx=0).stop 2024-06-10T00:29:50.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.874+0000 7f63f0019700 1 -- 172.21.15.62:0/4100546877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e80746c0 msgr2=0x7f63e819afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:50.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.874+0000 7f63f0019700 1 --2- 172.21.15.62:0/4100546877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e80746c0 0x7f63e819afa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f63d800ba50 tx=0x7f63d800bd60 comp rx=0 tx=0).stop 2024-06-10T00:29:50.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.874+0000 7f63f0019700 1 -- 172.21.15.62:0/4100546877 shutdown_connections 2024-06-10T00:29:50.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.874+0000 7f63f0019700 1 --2- 172.21.15.62:0/4100546877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e80746c0 0x7f63e819afa0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:50.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.874+0000 7f63f0019700 1 --2- 172.21.15.62:0/4100546877 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f63d00a2c20 0x7f63d00a50e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:50.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.874+0000 7f63f0019700 1 --2- 172.21.15.62:0/4100546877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63e810eec0 0x7f63e819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:50.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.874+0000 7f63f0019700 1 --2- 172.21.15.62:0/4100546877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e8074fe0 0x7f63e819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:50.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.874+0000 7f63f0019700 1 -- 172.21.15.62:0/4100546877 >> 172.21.15.62:0/4100546877 conn(0x7f63e8079640 msgr2=0x7f63e80ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:50.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.874+0000 7f63f0019700 1 -- 172.21.15.62:0/4100546877 shutdown_connections 2024-06-10T00:29:50.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:50.874+0000 7f63f0019700 1 -- 172.21.15.62:0/4100546877 wait complete. 2024-06-10T00:29:51.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:51 smithi062 ceph-mon[32390]: pgmap v341: 129 pgs: 129 active+clean; 587 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-06-10T00:29:51.108 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:29:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:29:50.847+0000 7fb794902700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:29:51.108 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:29:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:29:50.847+0000 7fb794902700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:29:51.108 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:29:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:29:50.847+0000 7fb794902700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:29:51.108 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:29:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:29:50.847+0000 7fb794902700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:29:51.108 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:29:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:29:50.847+0000 7fb794902700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:29:51.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:51 smithi105 ceph-mon[36321]: pgmap v341: 129 pgs: 129 active+clean; 587 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-06-10T00:29:51.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:51 smithi138 ceph-mon[36549]: pgmap v341: 129 pgs: 129 active+clean; 587 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-06-10T00:29:51.571 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-10T00:29:51.824 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:52.219 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:52 smithi062 ceph-mon[32390]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:29:52.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:52 smithi105 ceph-mon[36321]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:29:52.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:52 smithi138 ceph-mon[36549]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:29:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:53 smithi105 ceph-mon[36321]: pgmap v342: 129 pgs: 129 active+clean; 587 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.1 KiB/s wr, 19 op/s 2024-06-10T00:29:53.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:53 smithi062 ceph-mon[32390]: pgmap v342: 129 pgs: 129 active+clean; 587 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.1 KiB/s wr, 19 op/s 2024-06-10T00:29:53.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:53 smithi138 ceph-mon[36549]: pgmap v342: 129 pgs: 129 active+clean; 587 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.1 KiB/s wr, 19 op/s 2024-06-10T00:29:53.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.589+0000 7f6b07cea700 1 -- 172.21.15.62:0/678611417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b00102380 msgr2=0x7f6b00105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:53.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.589+0000 7f6b07cea700 1 --2- 172.21.15.62:0/678611417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b00102380 0x7f6b00105ff0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6af00099c0 tx=0x7f6af0009cd0 comp rx=0 tx=0).stop 2024-06-10T00:29:53.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.589+0000 7f6b07cea700 1 -- 172.21.15.62:0/678611417 shutdown_connections 2024-06-10T00:29:53.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.589+0000 7f6b07cea700 1 --2- 172.21.15.62:0/678611417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b00106720 0x7f6b0010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:53.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.589+0000 7f6b07cea700 1 --2- 172.21.15.62:0/678611417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b001019d0 0x7f6b00101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:53.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.589+0000 7f6b07cea700 1 --2- 172.21.15.62:0/678611417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b00102380 0x7f6b00105ff0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:53.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.589+0000 7f6b07cea700 1 -- 172.21.15.62:0/678611417 >> 172.21.15.62:0/678611417 conn(0x7f6b00078340 msgr2=0x7f6b00078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:53.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.589+0000 7f6b07cea700 1 -- 172.21.15.62:0/678611417 shutdown_connections 2024-06-10T00:29:53.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.589+0000 7f6b07cea700 1 -- 172.21.15.62:0/678611417 wait complete. 2024-06-10T00:29:53.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.590+0000 7f6b07cea700 1 Processor -- start 2024-06-10T00:29:53.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.590+0000 7f6b07cea700 1 -- start start 2024-06-10T00:29:53.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.590+0000 7f6b07cea700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b001019d0 0x7f6b00103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:53.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.590+0000 7f6b07cea700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b00102380 0x7f6b001035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:53.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.590+0000 7f6b07cea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b00106720 0x7f6b00103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:53.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.590+0000 7f6b07cea700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b00111710 con 0x7f6b00106720 2024-06-10T00:29:53.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.590+0000 7f6b07cea700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6b00111590 con 0x7f6b001019d0 2024-06-10T00:29:53.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.590+0000 7f6b07cea700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b00111890 con 0x7f6b00102380 2024-06-10T00:29:53.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6b06287700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b00106720 0x7f6b00103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:53.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6b06287700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b00106720 0x7f6b00103b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40054/0 (socket says 172.21.15.62:40054) 2024-06-10T00:29:53.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6b06287700 1 -- 172.21.15.62:0/2310379965 learned_addr learned my addr 172.21.15.62:0/2310379965 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:53.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6b05285700 1 --2- 172.21.15.62:0/2310379965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b00102380 0x7f6b001035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:53.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6b05a86700 1 --2- 172.21.15.62:0/2310379965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b001019d0 0x7f6b00103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:53.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6b06287700 1 -- 172.21.15.62:0/2310379965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b00102380 msgr2=0x7f6b001035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:53.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6b06287700 1 --2- 172.21.15.62:0/2310379965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b00102380 0x7f6b001035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:53.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6b06287700 1 -- 172.21.15.62:0/2310379965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b001019d0 msgr2=0x7f6b00103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:53.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6b06287700 1 --2- 172.21.15.62:0/2310379965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b001019d0 0x7f6b00103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:53.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6b06287700 1 -- 172.21.15.62:0/2310379965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b001a7190 con 0x7f6b00106720 2024-06-10T00:29:53.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6b06287700 1 --2- 172.21.15.62:0/2310379965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b00106720 0x7f6b00103b10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6af800b970 tx=0x7f6af800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:53.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6af6ffd700 1 -- 172.21.15.62:0/2310379965 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8019070 con 0x7f6b00106720 2024-06-10T00:29:53.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6b07cea700 1 -- 172.21.15.62:0/2310379965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b001a7480 con 0x7f6b00106720 2024-06-10T00:29:53.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6af6ffd700 1 -- 172.21.15.62:0/2310379965 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6af800ce10 con 0x7f6b00106720 2024-06-10T00:29:53.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.591+0000 7f6af6ffd700 1 -- 172.21.15.62:0/2310379965 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af8007370 con 0x7f6b00106720 2024-06-10T00:29:53.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.592+0000 7f6b07cea700 1 -- 172.21.15.62:0/2310379965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b001a7990 con 0x7f6b00106720 2024-06-10T00:29:53.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.593+0000 7f6b07cea700 1 -- 172.21.15.62:0/2310379965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b00102ad0 con 0x7f6b00106720 2024-06-10T00:29:53.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.594+0000 7f6af6ffd700 1 -- 172.21.15.62:0/2310379965 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6af801c070 con 0x7f6b00106720 2024-06-10T00:29:53.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.594+0000 7f6af6ffd700 1 --2- 172.21.15.62:0/2310379965 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6aec0a2cb0 0x7f6aec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:53.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.594+0000 7f6b05a86700 1 --2- 172.21.15.62:0/2310379965 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6aec0a2cb0 0x7f6aec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:53.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.594+0000 7f6af6ffd700 1 -- 172.21.15.62:0/2310379965 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f6af80ce7e0 con 0x7f6b00106720 2024-06-10T00:29:53.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.595+0000 7f6b05a86700 1 --2- 172.21.15.62:0/2310379965 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6aec0a2cb0 0x7f6aec0a5170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6afc005ea0 tx=0x7f6afc005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:53.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.599+0000 7f6af6ffd700 1 -- 172.21.15.62:0/2310379965 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6af809b390 con 0x7f6b00106720 2024-06-10T00:29:53.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.838+0000 7f6b07cea700 1 -- 172.21.15.62:0/2310379965 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6b000628d0 con 0x7f6aec0a2cb0 2024-06-10T00:29:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.866+0000 7f6af6ffd700 1 -- 172.21.15.62:0/2310379965 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6b000628d0 con 0x7f6aec0a2cb0 2024-06-10T00:29:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.868+0000 7f6b07cea700 1 -- 172.21.15.62:0/2310379965 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6aec0a2cb0 msgr2=0x7f6aec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.868+0000 7f6b07cea700 1 --2- 172.21.15.62:0/2310379965 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6aec0a2cb0 0x7f6aec0a5170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6afc005ea0 tx=0x7f6afc005e30 comp rx=0 tx=0).stop 2024-06-10T00:29:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.868+0000 7f6b07cea700 1 -- 172.21.15.62:0/2310379965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b00106720 msgr2=0x7f6b00103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.868+0000 7f6b07cea700 1 --2- 172.21.15.62:0/2310379965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b00106720 0x7f6b00103b10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6af800b970 tx=0x7f6af800bc80 comp rx=0 tx=0).stop 2024-06-10T00:29:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.868+0000 7f6b07cea700 1 -- 172.21.15.62:0/2310379965 shutdown_connections 2024-06-10T00:29:53.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.868+0000 7f6b07cea700 1 --2- 172.21.15.62:0/2310379965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b00106720 0x7f6b00103b10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:53.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.868+0000 7f6b07cea700 1 --2- 172.21.15.62:0/2310379965 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6aec0a2cb0 0x7f6aec0a5170 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:53.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.868+0000 7f6b07cea700 1 --2- 172.21.15.62:0/2310379965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b001019d0 0x7f6b00103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:53.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.868+0000 7f6b07cea700 1 --2- 172.21.15.62:0/2310379965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b00102380 0x7f6b001035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:53.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.868+0000 7f6b07cea700 1 -- 172.21.15.62:0/2310379965 >> 172.21.15.62:0/2310379965 conn(0x7f6b00078340 msgr2=0x7f6b0006ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:53.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.868+0000 7f6b07cea700 1 -- 172.21.15.62:0/2310379965 shutdown_connections 2024-06-10T00:29:53.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:53.868+0000 7f6b07cea700 1 -- 172.21.15.62:0/2310379965 wait complete. 2024-06-10T00:29:54.192 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:54 smithi062 ceph-mon[32390]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-06-10T00:29:54.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:54 smithi138 ceph-mon[36549]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-06-10T00:29:54.460 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-10T00:29:54.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:54 smithi105 ceph-mon[36321]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-06-10T00:29:54.709 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:55.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:55 smithi062 ceph-mon[32390]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:29:55.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:55 smithi062 ceph-mon[32390]: pgmap v343: 129 pgs: 129 active+clean; 594 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 KiB/s wr, 22 op/s 2024-06-10T00:29:55.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:55 smithi138 ceph-mon[36549]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:29:55.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:55 smithi138 ceph-mon[36549]: pgmap v343: 129 pgs: 129 active+clean; 594 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 KiB/s wr, 22 op/s 2024-06-10T00:29:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:55 smithi105 ceph-mon[36321]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:29:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:55 smithi105 ceph-mon[36321]: pgmap v343: 129 pgs: 129 active+clean; 594 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 KiB/s wr, 22 op/s 2024-06-10T00:29:56.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.315+0000 7f6ee824f700 1 -- 172.21.15.62:0/2052007736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee0100e50 msgr2=0x7f6ee01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:56.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.315+0000 7f6ee824f700 1 --2- 172.21.15.62:0/2052007736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee0100e50 0x7f6ee01012b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6ed00099c0 tx=0x7f6ed0009cd0 comp rx=0 tx=0).stop 2024-06-10T00:29:56.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.315+0000 7f6ee824f700 1 -- 172.21.15.62:0/2052007736 shutdown_connections 2024-06-10T00:29:56.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.315+0000 7f6ee824f700 1 --2- 172.21.15.62:0/2052007736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ee01017f0 0x7f6ee01115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.315+0000 7f6ee824f700 1 --2- 172.21.15.62:0/2052007736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ee0103300 0x7f6ee01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.315+0000 7f6ee824f700 1 --2- 172.21.15.62:0/2052007736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee0100e50 0x7f6ee01012b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.315+0000 7f6ee824f700 1 -- 172.21.15.62:0/2052007736 >> 172.21.15.62:0/2052007736 conn(0x7f6ee00766b0 msgr2=0x7f6ee0076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.315+0000 7f6ee824f700 1 -- 172.21.15.62:0/2052007736 shutdown_connections 2024-06-10T00:29:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.315+0000 7f6ee824f700 1 -- 172.21.15.62:0/2052007736 wait complete. 2024-06-10T00:29:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.316+0000 7f6ee824f700 1 Processor -- start 2024-06-10T00:29:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.316+0000 7f6ee824f700 1 -- start start 2024-06-10T00:29:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.316+0000 7f6ee824f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee0100e50 0x7f6ee019aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.316+0000 7f6ee824f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ee01017f0 0x7f6ee019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:56.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.316+0000 7f6ee824f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ee0103300 0x7f6ee019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:56.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.316+0000 7f6ee824f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee0114b10 con 0x7f6ee0100e50 2024-06-10T00:29:56.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.316+0000 7f6ee824f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee0114990 con 0x7f6ee01017f0 2024-06-10T00:29:56.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.316+0000 7f6ee824f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee0114c90 con 0x7f6ee0103300 2024-06-10T00:29:56.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.316+0000 7f6ee5feb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee0100e50 0x7f6ee019aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:56.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.316+0000 7f6ee5feb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee0100e50 0x7f6ee019aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40060/0 (socket says 172.21.15.62:40060) 2024-06-10T00:29:56.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.316+0000 7f6ee5feb700 1 -- 172.21.15.62:0/3417903605 learned_addr learned my addr 172.21.15.62:0/3417903605 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:56.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.317+0000 7f6ee67ec700 1 --2- 172.21.15.62:0/3417903605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ee0103300 0x7f6ee019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:56.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.317+0000 7f6ee57ea700 1 --2- 172.21.15.62:0/3417903605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ee01017f0 0x7f6ee019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:56.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.317+0000 7f6ee5feb700 1 -- 172.21.15.62:0/3417903605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ee0103300 msgr2=0x7f6ee019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:56.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.317+0000 7f6ee5feb700 1 --2- 172.21.15.62:0/3417903605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ee0103300 0x7f6ee019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:56.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.317+0000 7f6ee5feb700 1 -- 172.21.15.62:0/3417903605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ee01017f0 msgr2=0x7f6ee019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:56.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.317+0000 7f6ee5feb700 1 --2- 172.21.15.62:0/3417903605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ee01017f0 0x7f6ee019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:56.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.317+0000 7f6ee5feb700 1 -- 172.21.15.62:0/3417903605 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee019fed0 con 0x7f6ee0100e50 2024-06-10T00:29:56.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.317+0000 7f6ee5feb700 1 --2- 172.21.15.62:0/3417903605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee0100e50 0x7f6ee019aec0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00c970 tx=0x7f6edc00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:56.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.317+0000 7f6ed6ffd700 1 -- 172.21.15.62:0/3417903605 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc004220 con 0x7f6ee0100e50 2024-06-10T00:29:56.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.317+0000 7f6ee824f700 1 -- 172.21.15.62:0/3417903605 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee01a01c0 con 0x7f6ee0100e50 2024-06-10T00:29:56.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.317+0000 7f6ed6ffd700 1 -- 172.21.15.62:0/3417903605 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6edc007d80 con 0x7f6ee0100e50 2024-06-10T00:29:56.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.317+0000 7f6ee824f700 1 -- 172.21.15.62:0/3417903605 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee01a7a40 con 0x7f6ee0100e50 2024-06-10T00:29:56.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.317+0000 7f6ed6ffd700 1 -- 172.21.15.62:0/3417903605 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc003e50 con 0x7f6ee0100e50 2024-06-10T00:29:56.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.319+0000 7f6ee824f700 1 -- 172.21.15.62:0/3417903605 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee0069180 con 0x7f6ee0100e50 2024-06-10T00:29:56.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.320+0000 7f6ed6ffd700 1 -- 172.21.15.62:0/3417903605 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6edc0078a0 con 0x7f6ee0100e50 2024-06-10T00:29:56.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.320+0000 7f6ed6ffd700 1 --2- 172.21.15.62:0/3417903605 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6ecc0a2ed0 0x7f6ecc0a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:56.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.320+0000 7f6ee57ea700 1 --2- 172.21.15.62:0/3417903605 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6ecc0a2ed0 0x7f6ecc0a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:56.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.320+0000 7f6ed6ffd700 1 -- 172.21.15.62:0/3417903605 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f6edc0cf190 con 0x7f6ee0100e50 2024-06-10T00:29:56.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.321+0000 7f6ee57ea700 1 --2- 172.21.15.62:0/3417903605 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6ecc0a2ed0 0x7f6ecc0a5390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0076b40 tx=0x7f6ed0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:56.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.325+0000 7f6ed6ffd700 1 -- 172.21.15.62:0/3417903605 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6edc014030 con 0x7f6ee0100e50 2024-06-10T00:29:56.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.568+0000 7f6ee824f700 1 -- 172.21.15.62:0/3417903605 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f6ee00ff490 con 0x7f6ecc0a2ed0 2024-06-10T00:29:56.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.573+0000 7f6ed6ffd700 1 -- 172.21.15.62:0/3417903605 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f6ee00ff490 con 0x7f6ecc0a2ed0 2024-06-10T00:29:56.573 INFO:teuthology.orchestra.run.smithi062.stdout:/volumes/qa/sv_0/3bfd6e57-521d-44a8-bcbb-9e5de945cb30 2024-06-10T00:29:56.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.576+0000 7f6ee824f700 1 -- 172.21.15.62:0/3417903605 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6ecc0a2ed0 msgr2=0x7f6ecc0a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:56.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.576+0000 7f6ee824f700 1 --2- 172.21.15.62:0/3417903605 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6ecc0a2ed0 0x7f6ecc0a5390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0076b40 tx=0x7f6ed0005fd0 comp rx=0 tx=0).stop 2024-06-10T00:29:56.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.576+0000 7f6ee824f700 1 -- 172.21.15.62:0/3417903605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee0100e50 msgr2=0x7f6ee019aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:56.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.576+0000 7f6ee824f700 1 --2- 172.21.15.62:0/3417903605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee0100e50 0x7f6ee019aec0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00c970 tx=0x7f6edc00cd30 comp rx=0 tx=0).stop 2024-06-10T00:29:56.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.576+0000 7f6ee824f700 1 -- 172.21.15.62:0/3417903605 shutdown_connections 2024-06-10T00:29:56.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.576+0000 7f6ee824f700 1 --2- 172.21.15.62:0/3417903605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee0100e50 0x7f6ee019aec0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:56.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.576+0000 7f6ee824f700 1 --2- 172.21.15.62:0/3417903605 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6ecc0a2ed0 0x7f6ecc0a5390 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:56.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.576+0000 7f6ee824f700 1 --2- 172.21.15.62:0/3417903605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ee0103300 0x7f6ee019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:56.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.576+0000 7f6ee824f700 1 --2- 172.21.15.62:0/3417903605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ee01017f0 0x7f6ee019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:56.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.576+0000 7f6ee824f700 1 -- 172.21.15.62:0/3417903605 >> 172.21.15.62:0/3417903605 conn(0x7f6ee00766b0 msgr2=0x7f6ee00fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:56.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.576+0000 7f6ee824f700 1 -- 172.21.15.62:0/3417903605 shutdown_connections 2024-06-10T00:29:56.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:56.576+0000 7f6ee824f700 1 -- 172.21.15.62:0/3417903605 wait complete. 2024-06-10T00:29:57.256 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:57 smithi062 ceph-mon[32390]: pgmap v344: 129 pgs: 129 active+clean; 594 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.3 KiB/s wr, 19 op/s 2024-06-10T00:29:57.256 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:57 smithi062 ceph-mon[32390]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:29:57.284 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-10T00:29:57.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:57 smithi138 ceph-mon[36549]: pgmap v344: 129 pgs: 129 active+clean; 594 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.3 KiB/s wr, 19 op/s 2024-06-10T00:29:57.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:57 smithi138 ceph-mon[36549]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:29:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:57 smithi105 ceph-mon[36321]: pgmap v344: 129 pgs: 129 active+clean; 594 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.3 KiB/s wr, 19 op/s 2024-06-10T00:29:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:57 smithi105 ceph-mon[36321]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:29:57.532 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:29:59.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.304+0000 7f935bc51700 1 -- 172.21.15.62:0/3748590002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9354102510 msgr2=0x7f9354102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:59.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.304+0000 7f935bc51700 1 --2- 172.21.15.62:0/3748590002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9354102510 0x7f9354102990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9344009990 tx=0x7f9344009ca0 comp rx=0 tx=0).stop 2024-06-10T00:29:59.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.304+0000 7f935bc51700 1 -- 172.21.15.62:0/3748590002 shutdown_connections 2024-06-10T00:29:59.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.304+0000 7f935bc51700 1 --2- 172.21.15.62:0/3748590002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9354102ed0 0x7f935410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:59.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.304+0000 7f935bc51700 1 --2- 172.21.15.62:0/3748590002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9354108510 0x7f93541088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:59.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.304+0000 7f935bc51700 1 --2- 172.21.15.62:0/3748590002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9354102510 0x7f9354102990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:59.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.304+0000 7f935bc51700 1 -- 172.21.15.62:0/3748590002 >> 172.21.15.62:0/3748590002 conn(0x7f93540795b0 msgr2=0x7f93540799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:59.305 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:29:59 smithi062 ceph-mon[32390]: pgmap v345: 129 pgs: 129 active+clean; 594 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-06-10T00:29:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.304+0000 7f935bc51700 1 -- 172.21.15.62:0/3748590002 shutdown_connections 2024-06-10T00:29:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.304+0000 7f935bc51700 1 -- 172.21.15.62:0/3748590002 wait complete. 2024-06-10T00:29:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.305+0000 7f935bc51700 1 Processor -- start 2024-06-10T00:29:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.305+0000 7f935bc51700 1 -- start start 2024-06-10T00:29:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.305+0000 7f935bc51700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9354102510 0x7f935419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.305+0000 7f935bc51700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9354102ed0 0x7f935419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.305+0000 7f935bc51700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9354108510 0x7f935419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.305+0000 7f935bc51700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f93541127e0 con 0x7f9354108510 2024-06-10T00:29:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.305+0000 7f935bc51700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9354112660 con 0x7f9354102510 2024-06-10T00:29:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.305+0000 7f935bc51700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9354112960 con 0x7f9354102ed0 2024-06-10T00:29:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.305+0000 7f935a1ee700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9354108510 0x7f935419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.305+0000 7f935a1ee700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9354108510 0x7f935419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40804/0 (socket says 172.21.15.62:40804) 2024-06-10T00:29:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.305+0000 7f935a1ee700 1 -- 172.21.15.62:0/1513242222 learned_addr learned my addr 172.21.15.62:0/1513242222 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:29:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.306+0000 7f93599ed700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9354102510 0x7f935419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.306+0000 7f93591ec700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9354102ed0 0x7f935419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.306+0000 7f935a1ee700 1 -- 172.21.15.62:0/1513242222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9354102ed0 msgr2=0x7f935419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.306+0000 7f935a1ee700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9354102ed0 0x7f935419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.306+0000 7f935a1ee700 1 -- 172.21.15.62:0/1513242222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9354102510 msgr2=0x7f935419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.306+0000 7f935a1ee700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9354102510 0x7f935419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.306+0000 7f935a1ee700 1 -- 172.21.15.62:0/1513242222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935419fdf0 con 0x7f9354108510 2024-06-10T00:29:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.306+0000 7f93591ec700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9354102ed0 0x7f935419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:29:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.306+0000 7f935a1ee700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9354108510 0x7f935419f6b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f934c00b970 tx=0x7f934c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:59.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.306+0000 7f934affd700 1 -- 172.21.15.62:0/1513242222 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c019070 con 0x7f9354108510 2024-06-10T00:29:59.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.306+0000 7f935bc51700 1 -- 172.21.15.62:0/1513242222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93541a00e0 con 0x7f9354108510 2024-06-10T00:29:59.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.306+0000 7f934affd700 1 -- 172.21.15.62:0/1513242222 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f934c00ce10 con 0x7f9354108510 2024-06-10T00:29:59.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.306+0000 7f934affd700 1 -- 172.21.15.62:0/1513242222 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c007370 con 0x7f9354108510 2024-06-10T00:29:59.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.306+0000 7f935bc51700 1 -- 172.21.15.62:0/1513242222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93541a7960 con 0x7f9354108510 2024-06-10T00:29:59.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.307+0000 7f935bc51700 1 -- 172.21.15.62:0/1513242222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9354069180 con 0x7f9354108510 2024-06-10T00:29:59.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.309+0000 7f934affd700 1 -- 172.21.15.62:0/1513242222 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f934c01c070 con 0x7f9354108510 2024-06-10T00:29:59.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.309+0000 7f934affd700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93400a2cb0 0x7f93400a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:29:59.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.309+0000 7f93599ed700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93400a2cb0 0x7f93400a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:29:59.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.309+0000 7f934affd700 1 -- 172.21.15.62:0/1513242222 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f934c0ceca0 con 0x7f9354108510 2024-06-10T00:29:59.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.310+0000 7f93599ed700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93400a2cb0 0x7f93400a5170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f935000ac50 tx=0x7f935000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:29:59.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.313+0000 7f934affd700 1 -- 172.21.15.62:0/1513242222 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f934c09b850 con 0x7f9354108510 2024-06-10T00:29:59.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:29:59 smithi138 ceph-mon[36549]: pgmap v345: 129 pgs: 129 active+clean; 594 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-06-10T00:29:59.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:29:59 smithi105 ceph-mon[36321]: pgmap v345: 129 pgs: 129 active+clean; 594 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-06-10T00:29:59.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.550+0000 7f935bc51700 1 -- 172.21.15.62:0/1513242222 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9354075de0 con 0x7f93400a2cb0 2024-06-10T00:29:59.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.580+0000 7f934affd700 1 -- 172.21.15.62:0/1513242222 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9354075de0 con 0x7f93400a2cb0 2024-06-10T00:29:59.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.582+0000 7f935bc51700 1 -- 172.21.15.62:0/1513242222 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93400a2cb0 msgr2=0x7f93400a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:59.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.582+0000 7f935bc51700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93400a2cb0 0x7f93400a5170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f935000ac50 tx=0x7f935000a380 comp rx=0 tx=0).stop 2024-06-10T00:29:59.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.582+0000 7f935bc51700 1 -- 172.21.15.62:0/1513242222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9354108510 msgr2=0x7f935419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:29:59.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.582+0000 7f935bc51700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9354108510 0x7f935419f6b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f934c00b970 tx=0x7f934c00bc80 comp rx=0 tx=0).stop 2024-06-10T00:29:59.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.582+0000 7f935bc51700 1 -- 172.21.15.62:0/1513242222 shutdown_connections 2024-06-10T00:29:59.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.582+0000 7f935bc51700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9354108510 0x7f935419f6b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:59.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.582+0000 7f935bc51700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f93400a2cb0 0x7f93400a5170 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:59.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.582+0000 7f935bc51700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9354102510 0x7f935419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:59.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.582+0000 7f935bc51700 1 --2- 172.21.15.62:0/1513242222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9354102ed0 0x7f935419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:29:59.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.582+0000 7f935bc51700 1 -- 172.21.15.62:0/1513242222 >> 172.21.15.62:0/1513242222 conn(0x7f93540795b0 msgr2=0x7f93540746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:29:59.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.582+0000 7f935bc51700 1 -- 172.21.15.62:0/1513242222 shutdown_connections 2024-06-10T00:29:59.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:29:59.582+0000 7f935bc51700 1 -- 172.21.15.62:0/1513242222 wait complete. 2024-06-10T00:30:00.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:00 smithi062 ceph-mon[32390]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:30:00.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:00 smithi062 ceph-mon[32390]: overall HEALTH_OK 2024-06-10T00:30:00.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:00 smithi138 ceph-mon[36549]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:30:00.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:00 smithi138 ceph-mon[36549]: overall HEALTH_OK 2024-06-10T00:30:00.411 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-10T00:30:00.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:00 smithi105 ceph-mon[36321]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:30:00.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:00 smithi105 ceph-mon[36321]: overall HEALTH_OK 2024-06-10T00:30:00.664 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:01.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:01 smithi062 ceph-mon[32390]: pgmap v346: 129 pgs: 129 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-06-10T00:30:01.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:01 smithi138 ceph-mon[36549]: pgmap v346: 129 pgs: 129 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-06-10T00:30:01.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:01 smithi105 ceph-mon[36321]: pgmap v346: 129 pgs: 129 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-06-10T00:30:02.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.471+0000 7fb0dddfe700 1 -- 172.21.15.62:0/1531072275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d81030f0 msgr2=0x7fb0d8107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:02.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.471+0000 7fb0dddfe700 1 --2- 172.21.15.62:0/1531072275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d81030f0 0x7fb0d8107110 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0099c0 tx=0x7fb0cc009cd0 comp rx=0 tx=0).stop 2024-06-10T00:30:02.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.471+0000 7fb0dddfe700 1 -- 172.21.15.62:0/1531072275 shutdown_connections 2024-06-10T00:30:02.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.471+0000 7fb0dddfe700 1 --2- 172.21.15.62:0/1531072275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0d8107840 0x7fb0d810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:02.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.471+0000 7fb0dddfe700 1 --2- 172.21.15.62:0/1531072275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0d8102740 0x7fb0d8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:02.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.471+0000 7fb0dddfe700 1 --2- 172.21.15.62:0/1531072275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d81030f0 0x7fb0d8107110 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.471+0000 7fb0dddfe700 1 -- 172.21.15.62:0/1531072275 >> 172.21.15.62:0/1531072275 conn(0x7fb0d80795f0 msgr2=0x7fb0d8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.471+0000 7fb0dddfe700 1 -- 172.21.15.62:0/1531072275 shutdown_connections 2024-06-10T00:30:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.471+0000 7fb0dddfe700 1 -- 172.21.15.62:0/1531072275 wait complete. 2024-06-10T00:30:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.471+0000 7fb0dddfe700 1 Processor -- start 2024-06-10T00:30:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0dddfe700 1 -- start start 2024-06-10T00:30:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0dddfe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0d8102740 0x7fb0d8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0dddfe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d81030f0 0x7fb0d8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0dddfe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0d8107840 0x7fb0d8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0dddfe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d8112770 con 0x7fb0d81030f0 2024-06-10T00:30:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0dddfe700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d81125f0 con 0x7fb0d8107840 2024-06-10T00:30:02.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0dddfe700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d81128f0 con 0x7fb0d8102740 2024-06-10T00:30:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0d6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d81030f0 0x7fb0d8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0d6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d81030f0 0x7fb0d8074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40812/0 (socket says 172.21.15.62:40812) 2024-06-10T00:30:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0d6ffd700 1 -- 172.21.15.62:0/213039603 learned_addr learned my addr 172.21.15.62:0/213039603 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0d77fe700 1 --2- 172.21.15.62:0/213039603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0d8102740 0x7fb0d8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0d7fff700 1 --2- 172.21.15.62:0/213039603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0d8107840 0x7fb0d8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0d6ffd700 1 -- 172.21.15.62:0/213039603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0d8102740 msgr2=0x7fb0d8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0d6ffd700 1 --2- 172.21.15.62:0/213039603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0d8102740 0x7fb0d8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0d6ffd700 1 -- 172.21.15.62:0/213039603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0d8107840 msgr2=0x7fb0d8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0d6ffd700 1 --2- 172.21.15.62:0/213039603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0d8107840 0x7fb0d8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.472+0000 7fb0d6ffd700 1 -- 172.21.15.62:0/213039603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d81a70a0 con 0x7fb0d81030f0 2024-06-10T00:30:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.473+0000 7fb0d6ffd700 1 --2- 172.21.15.62:0/213039603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d81030f0 0x7fb0d8074cd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0059d0 tx=0x7fb0cc0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.473+0000 7fb0d4ff9700 1 -- 172.21.15.62:0/213039603 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc027070 con 0x7fb0d81030f0 2024-06-10T00:30:02.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.473+0000 7fb0dddfe700 1 -- 172.21.15.62:0/213039603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d81a7330 con 0x7fb0d81030f0 2024-06-10T00:30:02.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.473+0000 7fb0d4ff9700 1 -- 172.21.15.62:0/213039603 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0cc004050 con 0x7fb0d81030f0 2024-06-10T00:30:02.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.473+0000 7fb0d4ff9700 1 -- 172.21.15.62:0/213039603 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc022db0 con 0x7fb0d81030f0 2024-06-10T00:30:02.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.473+0000 7fb0dddfe700 1 -- 172.21.15.62:0/213039603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d81a77e0 con 0x7fb0d81030f0 2024-06-10T00:30:02.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.474+0000 7fb0dddfe700 1 -- 172.21.15.62:0/213039603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d8069180 con 0x7fb0d81030f0 2024-06-10T00:30:02.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.475+0000 7fb0d4ff9700 1 -- 172.21.15.62:0/213039603 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb0cc02a070 con 0x7fb0d81030f0 2024-06-10T00:30:02.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.476+0000 7fb0d4ff9700 1 --2- 172.21.15.62:0/213039603 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb0c00a2cb0 0x7fb0c00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:02.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.476+0000 7fb0d77fe700 1 --2- 172.21.15.62:0/213039603 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb0c00a2cb0 0x7fb0c00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:02.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.476+0000 7fb0d4ff9700 1 -- 172.21.15.62:0/213039603 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fb0cc0d02b0 con 0x7fb0d81030f0 2024-06-10T00:30:02.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.476+0000 7fb0d77fe700 1 --2- 172.21.15.62:0/213039603 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb0c00a2cb0 0x7fb0c00a5170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8005950 tx=0x7fb0c800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:02.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.480+0000 7fb0d4ff9700 1 -- 172.21.15.62:0/213039603 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb0cc05e850 con 0x7fb0d81030f0 2024-06-10T00:30:02.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.720+0000 7fb0dddfe700 1 -- 172.21.15.62:0/213039603 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb0d8075c70 con 0x7fb0c00a2cb0 2024-06-10T00:30:02.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.726+0000 7fb0d4ff9700 1 -- 172.21.15.62:0/213039603 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb0d8075c70 con 0x7fb0c00a2cb0 2024-06-10T00:30:02.726 INFO:teuthology.orchestra.run.smithi062.stdout:/volumes/qa/sv_1/4ba2ab46-adb2-4be3-aedd-0c97cb2421cc 2024-06-10T00:30:02.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.728+0000 7fb0dddfe700 1 -- 172.21.15.62:0/213039603 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb0c00a2cb0 msgr2=0x7fb0c00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:02.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.728+0000 7fb0dddfe700 1 --2- 172.21.15.62:0/213039603 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb0c00a2cb0 0x7fb0c00a5170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8005950 tx=0x7fb0c800b410 comp rx=0 tx=0).stop 2024-06-10T00:30:02.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.728+0000 7fb0dddfe700 1 -- 172.21.15.62:0/213039603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d81030f0 msgr2=0x7fb0d8074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:02.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.728+0000 7fb0dddfe700 1 --2- 172.21.15.62:0/213039603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d81030f0 0x7fb0d8074cd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0059d0 tx=0x7fb0cc0038b0 comp rx=0 tx=0).stop 2024-06-10T00:30:02.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.728+0000 7fb0dddfe700 1 -- 172.21.15.62:0/213039603 shutdown_connections 2024-06-10T00:30:02.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.728+0000 7fb0dddfe700 1 --2- 172.21.15.62:0/213039603 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb0c00a2cb0 0x7fb0c00a5170 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:02.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.728+0000 7fb0dddfe700 1 --2- 172.21.15.62:0/213039603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0d8107840 0x7fb0d8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:02.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.728+0000 7fb0dddfe700 1 --2- 172.21.15.62:0/213039603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0d8102740 0x7fb0d8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:02.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.728+0000 7fb0dddfe700 1 --2- 172.21.15.62:0/213039603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0d81030f0 0x7fb0d8074cd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:02.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.728+0000 7fb0dddfe700 1 -- 172.21.15.62:0/213039603 >> 172.21.15.62:0/213039603 conn(0x7fb0d80795f0 msgr2=0x7fb0d81006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:02.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.728+0000 7fb0dddfe700 1 -- 172.21.15.62:0/213039603 shutdown_connections 2024-06-10T00:30:02.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:02.728+0000 7fb0dddfe700 1 -- 172.21.15.62:0/213039603 wait complete. 2024-06-10T00:30:03.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:03 smithi062 ceph-mon[32390]: pgmap v347: 129 pgs: 129 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-06-10T00:30:03.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:03 smithi062 ceph-mon[32390]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:30:03.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:03 smithi138 ceph-mon[36549]: pgmap v347: 129 pgs: 129 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-06-10T00:30:03.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:03 smithi138 ceph-mon[36549]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:30:03.523 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:30:03.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:03 smithi105 ceph-mon[36321]: pgmap v347: 129 pgs: 129 active+clean; 613 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-06-10T00:30:03.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:03 smithi105 ceph-mon[36321]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:30:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:05.307 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:05 smithi062 ceph-mon[32390]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-06-10T00:30:05.334 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:05 smithi105 ceph-mon[36321]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-06-10T00:30:05.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:05 smithi138 ceph-mon[36549]: pgmap v348: 129 pgs: 129 active+clean; 640 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-06-10T00:30:05.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.550+0000 7fd88d5f0700 1 -- 172.21.15.62:0/216586943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd888107400 msgr2=0x7fd8881077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:05.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.550+0000 7fd88d5f0700 1 --2- 172.21.15.62:0/216586943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd888107400 0x7fd8881077e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd87800b0a0 tx=0x7fd87800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:30:05.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.550+0000 7fd88d5f0700 1 -- 172.21.15.62:0/216586943 shutdown_connections 2024-06-10T00:30:05.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.550+0000 7fd88d5f0700 1 --2- 172.21.15.62:0/216586943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd888107400 0x7fd8881077e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:05.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.550+0000 7fd88d5f0700 1 --2- 172.21.15.62:0/216586943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd888101d50 0x7fd88810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:05.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.550+0000 7fd88d5f0700 1 --2- 172.21.15.62:0/216586943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd888101390 0x7fd888101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:05.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.550+0000 7fd88d5f0700 1 -- 172.21.15.62:0/216586943 >> 172.21.15.62:0/216586943 conn(0x7fd888078340 msgr2=0x7fd888078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:05.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.550+0000 7fd88d5f0700 1 -- 172.21.15.62:0/216586943 shutdown_connections 2024-06-10T00:30:05.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.550+0000 7fd88d5f0700 1 -- 172.21.15.62:0/216586943 wait complete. 2024-06-10T00:30:05.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.551+0000 7fd88d5f0700 1 Processor -- start 2024-06-10T00:30:05.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.551+0000 7fd88d5f0700 1 -- start start 2024-06-10T00:30:05.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.551+0000 7fd88d5f0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd888101390 0x7fd88819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:05.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.551+0000 7fd88d5f0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd888101d50 0x7fd88819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:05.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.551+0000 7fd88d5f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd888107400 0x7fd88819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:05.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.551+0000 7fd88d5f0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd888111770 con 0x7fd888101d50 2024-06-10T00:30:05.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.551+0000 7fd88d5f0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881115f0 con 0x7fd888101390 2024-06-10T00:30:05.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.551+0000 7fd88d5f0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881118f0 con 0x7fd888107400 2024-06-10T00:30:05.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.551+0000 7fd8867fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd888101d50 0x7fd88819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:05.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.551+0000 7fd8867fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd888101d50 0x7fd88819b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40828/0 (socket says 172.21.15.62:40828) 2024-06-10T00:30:05.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.551+0000 7fd8867fc700 1 -- 172.21.15.62:0/200031614 learned_addr learned my addr 172.21.15.62:0/200031614 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:05.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.551+0000 7fd8877fe700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd888107400 0x7fd88819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:05.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.551+0000 7fd886ffd700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd888101390 0x7fd88819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:05.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.552+0000 7fd8867fc700 1 -- 172.21.15.62:0/200031614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd888107400 msgr2=0x7fd88819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:05.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.552+0000 7fd8867fc700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd888107400 0x7fd88819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:05.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.552+0000 7fd8867fc700 1 -- 172.21.15.62:0/200031614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd888101390 msgr2=0x7fd88819aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:05.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.552+0000 7fd8867fc700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd888101390 0x7fd88819aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:05.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.552+0000 7fd8867fc700 1 -- 172.21.15.62:0/200031614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88819ff00 con 0x7fd888101d50 2024-06-10T00:30:05.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.552+0000 7fd886ffd700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd888101390 0x7fd88819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:30:05.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.552+0000 7fd8867fc700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd888101d50 0x7fd88819b430 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd87000b530 tx=0x7fd87000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:05.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.552+0000 7fd87ffff700 1 -- 172.21.15.62:0/200031614 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870015020 con 0x7fd888101d50 2024-06-10T00:30:05.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.552+0000 7fd88d5f0700 1 -- 172.21.15.62:0/200031614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8881a01f0 con 0x7fd888101d50 2024-06-10T00:30:05.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.552+0000 7fd87ffff700 1 -- 172.21.15.62:0/200031614 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd870011d10 con 0x7fd888101d50 2024-06-10T00:30:05.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.552+0000 7fd87ffff700 1 -- 172.21.15.62:0/200031614 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87000f2e0 con 0x7fd888101d50 2024-06-10T00:30:05.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.552+0000 7fd88d5f0700 1 -- 172.21.15.62:0/200031614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8881a7a70 con 0x7fd888101d50 2024-06-10T00:30:05.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.553+0000 7fd88d5f0700 1 -- 172.21.15.62:0/200031614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd888069180 con 0x7fd888101d50 2024-06-10T00:30:05.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.554+0000 7fd87ffff700 1 -- 172.21.15.62:0/200031614 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd870018020 con 0x7fd888101d50 2024-06-10T00:30:05.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.555+0000 7fd87ffff700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd8740a2cb0 0x7fd8740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:05.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.555+0000 7fd886ffd700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd8740a2cb0 0x7fd8740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:05.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.555+0000 7fd87ffff700 1 -- 172.21.15.62:0/200031614 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fd8700cf0a0 con 0x7fd888101d50 2024-06-10T00:30:05.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.555+0000 7fd886ffd700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd8740a2cb0 0x7fd8740a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd87800b830 tx=0x7fd878006060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:05.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.559+0000 7fd87ffff700 1 -- 172.21.15.62:0/200031614 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd87001f020 con 0x7fd888101d50 2024-06-10T00:30:05.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.812+0000 7fd88d5f0700 1 -- 172.21.15.62:0/200031614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8881a7d60 con 0x7fd888101d50 2024-06-10T00:30:05.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.812+0000 7fd87ffff700 1 -- 172.21.15.62:0/200031614 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7fd87009bc50 con 0x7fd888101d50 2024-06-10T00:30:05.813 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:30:05.813 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:29:47.712789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24431,24424,24418,24433,24464]},"id":1}]} 2024-06-10T00:30:05.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.815+0000 7fd88d5f0700 1 -- 172.21.15.62:0/200031614 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd8740a2cb0 msgr2=0x7fd8740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:05.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.815+0000 7fd88d5f0700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd8740a2cb0 0x7fd8740a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd87800b830 tx=0x7fd878006060 comp rx=0 tx=0).stop 2024-06-10T00:30:05.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.815+0000 7fd88d5f0700 1 -- 172.21.15.62:0/200031614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd888101d50 msgr2=0x7fd88819b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:05.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.815+0000 7fd88d5f0700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd888101d50 0x7fd88819b430 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd87000b530 tx=0x7fd87000b840 comp rx=0 tx=0).stop 2024-06-10T00:30:05.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.815+0000 7fd88d5f0700 1 -- 172.21.15.62:0/200031614 shutdown_connections 2024-06-10T00:30:05.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.815+0000 7fd88d5f0700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd8740a2cb0 0x7fd8740a5170 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:05.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.815+0000 7fd88d5f0700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd888107400 0x7fd88819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:05.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.815+0000 7fd88d5f0700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd888101390 0x7fd88819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:05.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.815+0000 7fd88d5f0700 1 --2- 172.21.15.62:0/200031614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd888101d50 0x7fd88819b430 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:05.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.815+0000 7fd88d5f0700 1 -- 172.21.15.62:0/200031614 >> 172.21.15.62:0/200031614 conn(0x7fd888078340 msgr2=0x7fd88806a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:05.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.815+0000 7fd88d5f0700 1 -- 172.21.15.62:0/200031614 shutdown_connections 2024-06-10T00:30:05.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:05.815+0000 7fd88d5f0700 1 -- 172.21.15.62:0/200031614 wait complete. 2024-06-10T00:30:05.816 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 29 2024-06-10T00:30:06.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:06 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/200031614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:30:06.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:06 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/200031614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:30:06.528 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:30:06.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:06 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/200031614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:30:06.777 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:07.228 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:07 smithi062 ceph-mon[32390]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-10T00:30:07.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:07 smithi138 ceph-mon[36549]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-10T00:30:07.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:07 smithi105 ceph-mon[36321]: pgmap v349: 129 pgs: 129 active+clean; 640 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-10T00:30:08.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.460+0000 7f16afb0b700 1 -- 172.21.15.62:0/602765687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a81085c0 msgr2=0x7f16a81089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:08.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.460+0000 7f16afb0b700 1 --2- 172.21.15.62:0/602765687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a81085c0 0x7f16a81089a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f16a40099c0 tx=0x7f16a4009cd0 comp rx=0 tx=0).stop 2024-06-10T00:30:08.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.460+0000 7f16afb0b700 1 -- 172.21.15.62:0/602765687 shutdown_connections 2024-06-10T00:30:08.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.460+0000 7f16afb0b700 1 --2- 172.21.15.62:0/602765687 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a81085c0 0x7f16a81089a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:08.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.460+0000 7f16afb0b700 1 --2- 172.21.15.62:0/602765687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16a8102ee0 0x7f16a810f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:08.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.460+0000 7f16afb0b700 1 --2- 172.21.15.62:0/602765687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a8102520 0x7f16a81029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:08.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.460+0000 7f16afb0b700 1 -- 172.21.15.62:0/602765687 >> 172.21.15.62:0/602765687 conn(0x7f16a8079680 msgr2=0x7f16a8079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:08.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.460+0000 7f16afb0b700 1 -- 172.21.15.62:0/602765687 shutdown_connections 2024-06-10T00:30:08.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.460+0000 7f16afb0b700 1 -- 172.21.15.62:0/602765687 wait complete. 2024-06-10T00:30:08.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.461+0000 7f16afb0b700 1 Processor -- start 2024-06-10T00:30:08.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.461+0000 7f16afb0b700 1 -- start start 2024-06-10T00:30:08.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.461+0000 7f16afb0b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16a8102520 0x7f16a8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:08.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.461+0000 7f16afb0b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a8102ee0 0x7f16a8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:08.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.461+0000 7f16afb0b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a81085c0 0x7f16a8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:08.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.461+0000 7f16afb0b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8112890 con 0x7f16a81085c0 2024-06-10T00:30:08.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.461+0000 7f16afb0b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8112710 con 0x7f16a8102520 2024-06-10T00:30:08.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.461+0000 7f16afb0b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8112a10 con 0x7f16a8102ee0 2024-06-10T00:30:08.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.461+0000 7f16ae0a8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a81085c0 0x7f16a8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:08.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.461+0000 7f16ae0a8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a81085c0 0x7f16a8075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46474/0 (socket says 172.21.15.62:46474) 2024-06-10T00:30:08.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.461+0000 7f16ae0a8700 1 -- 172.21.15.62:0/1240584831 learned_addr learned my addr 172.21.15.62:0/1240584831 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:08.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.461+0000 7f16ad0a6700 1 --2- 172.21.15.62:0/1240584831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a8102ee0 0x7f16a8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:08.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.462+0000 7f16ad8a7700 1 --2- 172.21.15.62:0/1240584831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16a8102520 0x7f16a8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:08.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.462+0000 7f16ae0a8700 1 -- 172.21.15.62:0/1240584831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a8102ee0 msgr2=0x7f16a8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:08.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.462+0000 7f16ae0a8700 1 --2- 172.21.15.62:0/1240584831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a8102ee0 0x7f16a8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:08.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.462+0000 7f16ae0a8700 1 -- 172.21.15.62:0/1240584831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16a8102520 msgr2=0x7f16a8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:08.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.462+0000 7f16ae0a8700 1 --2- 172.21.15.62:0/1240584831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16a8102520 0x7f16a8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:08.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.462+0000 7f16ae0a8700 1 -- 172.21.15.62:0/1240584831 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a81a7250 con 0x7f16a81085c0 2024-06-10T00:30:08.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.462+0000 7f16ae0a8700 1 --2- 172.21.15.62:0/1240584831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a81085c0 0x7f16a8075210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b970 tx=0x7f16a000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:08.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.462+0000 7f169effd700 1 -- 172.21.15.62:0/1240584831 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0019070 con 0x7f16a81085c0 2024-06-10T00:30:08.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.462+0000 7f169effd700 1 -- 172.21.15.62:0/1240584831 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a000ce10 con 0x7f16a81085c0 2024-06-10T00:30:08.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.462+0000 7f16afb0b700 1 -- 172.21.15.62:0/1240584831 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a81a7540 con 0x7f16a81085c0 2024-06-10T00:30:08.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.462+0000 7f169effd700 1 -- 172.21.15.62:0/1240584831 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0007370 con 0x7f16a81085c0 2024-06-10T00:30:08.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.462+0000 7f16afb0b700 1 -- 172.21.15.62:0/1240584831 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a81a7a30 con 0x7f16a81085c0 2024-06-10T00:30:08.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.463+0000 7f16afb0b700 1 -- 172.21.15.62:0/1240584831 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a810aa50 con 0x7f16a81085c0 2024-06-10T00:30:08.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.464+0000 7f169effd700 1 -- 172.21.15.62:0/1240584831 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f16a001c070 con 0x7f16a81085c0 2024-06-10T00:30:08.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.465+0000 7f169effd700 1 --2- 172.21.15.62:0/1240584831 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f16940a2cb0 0x7f16940a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:08.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.465+0000 7f16ad8a7700 1 --2- 172.21.15.62:0/1240584831 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f16940a2cb0 0x7f16940a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:08.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.466+0000 7f169effd700 1 -- 172.21.15.62:0/1240584831 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f16a00ced00 con 0x7f16a81085c0 2024-06-10T00:30:08.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.466+0000 7f16ad8a7700 1 --2- 172.21.15.62:0/1240584831 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f16940a2cb0 0x7f16940a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16a4000c00 tx=0x7f16a4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:08.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.469+0000 7f169effd700 1 -- 172.21.15.62:0/1240584831 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f16a009b850 con 0x7f16a81085c0 2024-06-10T00:30:08.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.680+0000 7f16afb0b700 1 -- 172.21.15.62:0/1240584831 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16a8075cc0 con 0x7f16a81085c0 2024-06-10T00:30:08.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.682+0000 7f169effd700 1 -- 172.21.15.62:0/1240584831 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24653 (secure 0 0 0) 0x7f16a009b150 con 0x7f16a81085c0 2024-06-10T00:30:08.682 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:30:08.682 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":97,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:29:23.767202+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:30:08.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.684+0000 7f16afb0b700 1 -- 172.21.15.62:0/1240584831 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f16940a2cb0 msgr2=0x7f16940a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:08.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.684+0000 7f16afb0b700 1 --2- 172.21.15.62:0/1240584831 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f16940a2cb0 0x7f16940a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16a4000c00 tx=0x7f16a4005fd0 comp rx=0 tx=0).stop 2024-06-10T00:30:08.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.684+0000 7f16afb0b700 1 -- 172.21.15.62:0/1240584831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a81085c0 msgr2=0x7f16a8075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:08.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.684+0000 7f16afb0b700 1 --2- 172.21.15.62:0/1240584831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a81085c0 0x7f16a8075210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b970 tx=0x7f16a000bc80 comp rx=0 tx=0).stop 2024-06-10T00:30:08.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.684+0000 7f16afb0b700 1 -- 172.21.15.62:0/1240584831 shutdown_connections 2024-06-10T00:30:08.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.684+0000 7f16afb0b700 1 --2- 172.21.15.62:0/1240584831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16a81085c0 0x7f16a8075210 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:08.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.684+0000 7f16afb0b700 1 --2- 172.21.15.62:0/1240584831 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f16940a2cb0 0x7f16940a5170 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:08.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.684+0000 7f16afb0b700 1 --2- 172.21.15.62:0/1240584831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16a8102520 0x7f16a8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:08.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.684+0000 7f16afb0b700 1 --2- 172.21.15.62:0/1240584831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a8102ee0 0x7f16a8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:08.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.684+0000 7f16afb0b700 1 -- 172.21.15.62:0/1240584831 >> 172.21.15.62:0/1240584831 conn(0x7f16a8079680 msgr2=0x7f16a8101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:08.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.684+0000 7f16afb0b700 1 -- 172.21.15.62:0/1240584831 shutdown_connections 2024-06-10T00:30:08.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:08.684+0000 7f16afb0b700 1 -- 172.21.15.62:0/1240584831 wait complete. 2024-06-10T00:30:09.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:09 smithi062 ceph-mon[32390]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-06-10T00:30:09.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:09 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1240584831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:30:09.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:09 smithi138 ceph-mon[36549]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-06-10T00:30:09.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:09 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1240584831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:30:09.396 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-10T00:30:09.396 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T00:30:09.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:09 smithi105 ceph-mon[36321]: pgmap v350: 129 pgs: 129 active+clean; 640 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-06-10T00:30:09.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:09 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1240584831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:30:09.645 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:11.187 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:11 smithi062 ceph-mon[32390]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-06-10T00:30:11.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:11 smithi138 ceph-mon[36549]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-06-10T00:30:11.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.425+0000 7f857b8de700 1 -- 172.21.15.62:0/1378576934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85741015d0 msgr2=0x7f85741019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:11.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.425+0000 7f857b8de700 1 --2- 172.21.15.62:0/1378576934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85741015d0 0x7f85741019b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f85700099c0 tx=0x7f8570009cd0 comp rx=0 tx=0).stop 2024-06-10T00:30:11.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.425+0000 7f857b8de700 1 -- 172.21.15.62:0/1378576934 shutdown_connections 2024-06-10T00:30:11.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.425+0000 7f857b8de700 1 --2- 172.21.15.62:0/1378576934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85741015d0 0x7f85741019b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:11.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.425+0000 7f857b8de700 1 --2- 172.21.15.62:0/1378576934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85741066b0 0x7f857410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:11.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.425+0000 7f857b8de700 1 --2- 172.21.15.62:0/1378576934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8574101f80 0x7f8574105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:11.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.425+0000 7f857b8de700 1 -- 172.21.15.62:0/1378576934 >> 172.21.15.62:0/1378576934 conn(0x7f8574078430 msgr2=0x7f8574078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:11.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.425+0000 7f857b8de700 1 -- 172.21.15.62:0/1378576934 shutdown_connections 2024-06-10T00:30:11.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.425+0000 7f857b8de700 1 -- 172.21.15.62:0/1378576934 wait complete. 2024-06-10T00:30:11.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.425+0000 7f857b8de700 1 Processor -- start 2024-06-10T00:30:11.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.425+0000 7f857b8de700 1 -- start start 2024-06-10T00:30:11.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f857b8de700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85741015d0 0x7f8574103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:11.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f857b8de700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8574101f80 0x7f85741035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:11.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f857b8de700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85741066b0 0x7f8574103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:11.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f857b8de700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8574111710 con 0x7f8574101f80 2024-06-10T00:30:11.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f857b8de700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8574111590 con 0x7f85741015d0 2024-06-10T00:30:11.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f857b8de700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8574111890 con 0x7f85741066b0 2024-06-10T00:30:11.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f8578e79700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8574101f80 0x7f85741035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:11.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f8578e79700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8574101f80 0x7f85741035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46486/0 (socket says 172.21.15.62:46486) 2024-06-10T00:30:11.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f8578e79700 1 -- 172.21.15.62:0/1353141770 learned_addr learned my addr 172.21.15.62:0/1353141770 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:11.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f8579e7b700 1 --2- 172.21.15.62:0/1353141770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85741066b0 0x7f8574103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:11.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f857967a700 1 --2- 172.21.15.62:0/1353141770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85741015d0 0x7f8574103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:11.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f8578e79700 1 -- 172.21.15.62:0/1353141770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85741066b0 msgr2=0x7f8574103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:11.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f8578e79700 1 --2- 172.21.15.62:0/1353141770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85741066b0 0x7f8574103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:11.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f8578e79700 1 -- 172.21.15.62:0/1353141770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85741015d0 msgr2=0x7f8574103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:11.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f8578e79700 1 --2- 172.21.15.62:0/1353141770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85741015d0 0x7f8574103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:11.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f8578e79700 1 -- 172.21.15.62:0/1353141770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85741a7240 con 0x7f8574101f80 2024-06-10T00:30:11.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.426+0000 7f8578e79700 1 --2- 172.21.15.62:0/1353141770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8574101f80 0x7f85741035b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f856400b500 tx=0x7f856400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:11.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.427+0000 7f856a7fc700 1 -- 172.21.15.62:0/1353141770 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8564015020 con 0x7f8574101f80 2024-06-10T00:30:11.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.427+0000 7f857b8de700 1 -- 172.21.15.62:0/1353141770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85741a7530 con 0x7f8574101f80 2024-06-10T00:30:11.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.427+0000 7f856a7fc700 1 -- 172.21.15.62:0/1353141770 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8564011d10 con 0x7f8574101f80 2024-06-10T00:30:11.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.427+0000 7f856a7fc700 1 -- 172.21.15.62:0/1353141770 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8564010de0 con 0x7f8574101f80 2024-06-10T00:30:11.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.427+0000 7f857b8de700 1 -- 172.21.15.62:0/1353141770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85741a7ac0 con 0x7f8574101f80 2024-06-10T00:30:11.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.428+0000 7f857b8de700 1 -- 172.21.15.62:0/1353141770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8574069180 con 0x7f8574101f80 2024-06-10T00:30:11.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.429+0000 7f856a7fc700 1 -- 172.21.15.62:0/1353141770 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8564018020 con 0x7f8574101f80 2024-06-10T00:30:11.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.429+0000 7f856a7fc700 1 --2- 172.21.15.62:0/1353141770 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f85600a2cb0 0x7f85600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:11.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.430+0000 7f857967a700 1 --2- 172.21.15.62:0/1353141770 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f85600a2cb0 0x7f85600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:11.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.430+0000 7f856a7fc700 1 -- 172.21.15.62:0/1353141770 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f85640ceae0 con 0x7f8574101f80 2024-06-10T00:30:11.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.430+0000 7f857967a700 1 --2- 172.21.15.62:0/1353141770 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f85600a2cb0 0x7f85600a5170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8570000c00 tx=0x7f8570005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.434+0000 7f856a7fc700 1 -- 172.21.15.62:0/1353141770 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f856401f020 con 0x7f8574101f80 2024-06-10T00:30:11.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:11 smithi105 ceph-mon[36321]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-06-10T00:30:11.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.636+0000 7f857b8de700 1 -- 172.21.15.62:0/1353141770 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85740628d0 con 0x7f85600a2cb0 2024-06-10T00:30:11.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.646+0000 7f856a7fc700 1 -- 172.21.15.62:0/1353141770 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340416 (secure 0 0 0) 0x7f85740628d0 con 0x7f85600a2cb0 2024-06-10T00:30:11.651 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:30:11.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.649+0000 7f857b8de700 1 -- 172.21.15.62:0/1353141770 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f85600a2cb0 msgr2=0x7f85600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:11.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.649+0000 7f857b8de700 1 --2- 172.21.15.62:0/1353141770 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f85600a2cb0 0x7f85600a5170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8570000c00 tx=0x7f8570005fb0 comp rx=0 tx=0).stop 2024-06-10T00:30:11.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.649+0000 7f857b8de700 1 -- 172.21.15.62:0/1353141770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8574101f80 msgr2=0x7f85741035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:11.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.649+0000 7f857b8de700 1 --2- 172.21.15.62:0/1353141770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8574101f80 0x7f85741035b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f856400b500 tx=0x7f856400b810 comp rx=0 tx=0).stop 2024-06-10T00:30:11.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.649+0000 7f857b8de700 1 -- 172.21.15.62:0/1353141770 shutdown_connections 2024-06-10T00:30:11.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.649+0000 7f857b8de700 1 --2- 172.21.15.62:0/1353141770 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f85600a2cb0 0x7f85600a5170 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:11.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.649+0000 7f857b8de700 1 --2- 172.21.15.62:0/1353141770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85741066b0 0x7f8574103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:11.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.650+0000 7f857b8de700 1 --2- 172.21.15.62:0/1353141770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85741015d0 0x7f8574103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:11.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.650+0000 7f857b8de700 1 --2- 172.21.15.62:0/1353141770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8574101f80 0x7f85741035b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:11.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.650+0000 7f857b8de700 1 -- 172.21.15.62:0/1353141770 >> 172.21.15.62:0/1353141770 conn(0x7f8574078430 msgr2=0x7f857406af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:11.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.650+0000 7f857b8de700 1 -- 172.21.15.62:0/1353141770 shutdown_connections 2024-06-10T00:30:11.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:11.650+0000 7f857b8de700 1 -- 172.21.15.62:0/1353141770 wait complete. 2024-06-10T00:30:11.655 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-06-10T00:30:12.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:12 smithi062 ceph-mon[32390]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:30:12.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:12 smithi138 ceph-mon[36549]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:30:12.422 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-06-10T00:30:09.960647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656194,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":396,"num_read_kb":325,"num_write":428,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380976,"kb_used_data":15184,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560848,"statfs":{"total":1151940427776,"available":1151550308352,"internally_reserved":0,"allocated":15548416,"data_stored":8555266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":48067,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":125,"num_read_kb":114,"num_write":25,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003844"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780586+0000","last_change":"2024-06-10T00:28:37.657234+0000","last_active":"2024-06-10T00:29:23.780586+0000","last_peered":"2024-06-10T00:29:23.780586+0000","last_clean":"2024-06-10T00:29:23.780586+0000","last_became_active":"2024-06-10T00:28:37.656879+0000","last_became_peered":"2024-06-10T00:28:37.656879+0000","last_unstale":"2024-06-10T00:29:23.780586+0000","last_undegraded":"2024-06-10T00:29:23.780586+0000","last_fullsized":"2024-06-10T00:29:23.780586+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:01:02.907597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777728+0000","last_change":"2024-06-10T00:28:37.656440+0000","last_active":"2024-06-10T00:29:23.777728+0000","last_peered":"2024-06-10T00:29:23.777728+0000","last_clean":"2024-06-10T00:29:23.777728+0000","last_became_active":"2024-06-10T00:28:37.656113+0000","last_became_peered":"2024-06-10T00:28:37.656113+0000","last_unstale":"2024-06-10T00:29:23.777728+0000","last_undegraded":"2024-06-10T00:29:23.777728+0000","last_fullsized":"2024-06-10T00:29:23.777728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:49:31.447951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777164+0000","last_change":"2024-06-10T00:28:37.659114+0000","last_active":"2024-06-10T00:29:23.777164+0000","last_peered":"2024-06-10T00:29:23.777164+0000","last_clean":"2024-06-10T00:29:23.777164+0000","last_became_active":"2024-06-10T00:28:37.658763+0000","last_became_peered":"2024-06-10T00:28:37.658763+0000","last_unstale":"2024-06-10T00:29:23.777164+0000","last_undegraded":"2024-06-10T00:29:23.777164+0000","last_fullsized":"2024-06-10T00:29:23.777164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:39:54.963590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.775384+0000","last_change":"2024-06-10T00:28:37.656786+0000","last_active":"2024-06-10T00:29:23.775384+0000","last_peered":"2024-06-10T00:29:23.775384+0000","last_clean":"2024-06-10T00:29:23.775384+0000","last_became_active":"2024-06-10T00:28:37.656073+0000","last_became_peered":"2024-06-10T00:28:37.656073+0000","last_unstale":"2024-06-10T00:29:23.775384+0000","last_undegraded":"2024-06-10T00:29:23.775384+0000","last_fullsized":"2024-06-10T00:29:23.775384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:19.176608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779481+0000","last_change":"2024-06-10T00:28:37.650250+0000","last_active":"2024-06-10T00:29:23.779481+0000","last_peered":"2024-06-10T00:29:23.779481+0000","last_clean":"2024-06-10T00:29:23.779481+0000","last_became_active":"2024-06-10T00:28:37.649852+0000","last_became_peered":"2024-06-10T00:28:37.649852+0000","last_unstale":"2024-06-10T00:29:23.779481+0000","last_undegraded":"2024-06-10T00:29:23.779481+0000","last_fullsized":"2024-06-10T00:29:23.779481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:39.492884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.856618+0000","last_change":"2024-06-10T00:28:37.646042+0000","last_active":"2024-06-10T00:29:23.856618+0000","last_peered":"2024-06-10T00:29:23.856618+0000","last_clean":"2024-06-10T00:29:23.856618+0000","last_became_active":"2024-06-10T00:28:37.644591+0000","last_became_peered":"2024-06-10T00:28:37.644591+0000","last_unstale":"2024-06-10T00:29:23.856618+0000","last_undegraded":"2024-06-10T00:29:23.856618+0000","last_fullsized":"2024-06-10T00:29:23.856618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:29.932303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777517+0000","last_change":"2024-06-10T00:28:37.664076+0000","last_active":"2024-06-10T00:29:23.777517+0000","last_peered":"2024-06-10T00:29:23.777517+0000","last_clean":"2024-06-10T00:29:23.777517+0000","last_became_active":"2024-06-10T00:28:37.663741+0000","last_became_peered":"2024-06-10T00:28:37.663741+0000","last_unstale":"2024-06-10T00:29:23.777517+0000","last_undegraded":"2024-06-10T00:29:23.777517+0000","last_fullsized":"2024-06-10T00:29:23.777517+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:14.581409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779934+0000","last_change":"2024-06-10T00:28:37.648758+0000","last_active":"2024-06-10T00:29:23.779934+0000","last_peered":"2024-06-10T00:29:23.779934+0000","last_clean":"2024-06-10T00:29:23.779934+0000","last_became_active":"2024-06-10T00:28:37.648387+0000","last_became_peered":"2024-06-10T00:28:37.648387+0000","last_unstale":"2024-06-10T00:29:23.779934+0000","last_undegraded":"2024-06-10T00:29:23.779934+0000","last_fullsized":"2024-06-10T00:29:23.779934+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:17.187265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855175+0000","last_change":"2024-06-10T00:28:37.664650+0000","last_active":"2024-06-10T00:29:23.855175+0000","last_peered":"2024-06-10T00:29:23.855175+0000","last_clean":"2024-06-10T00:29:23.855175+0000","last_became_active":"2024-06-10T00:28:37.663178+0000","last_became_peered":"2024-06-10T00:28:37.663178+0000","last_unstale":"2024-06-10T00:29:23.855175+0000","last_undegraded":"2024-06-10T00:29:23.855175+0000","last_fullsized":"2024-06-10T00:29:23.855175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:46.032078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.856399+0000","last_change":"2024-06-10T00:28:37.658353+0000","last_active":"2024-06-10T00:29:23.856399+0000","last_peered":"2024-06-10T00:29:23.856399+0000","last_clean":"2024-06-10T00:29:23.856399+0000","last_became_active":"2024-06-10T00:28:37.657961+0000","last_became_peered":"2024-06-10T00:28:37.657961+0000","last_unstale":"2024-06-10T00:29:23.856399+0000","last_undegraded":"2024-06-10T00:29:23.856399+0000","last_fullsized":"2024-06-10T00:29:23.856399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:22:27.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777983+0000","last_change":"2024-06-10T00:28:37.663823+0000","last_active":"2024-06-10T00:29:23.777983+0000","last_peered":"2024-06-10T00:29:23.777983+0000","last_clean":"2024-06-10T00:29:23.777983+0000","last_became_active":"2024-06-10T00:28:37.663499+0000","last_became_peered":"2024-06-10T00:28:37.663499+0000","last_unstale":"2024-06-10T00:29:23.777983+0000","last_undegraded":"2024-06-10T00:29:23.777983+0000","last_fullsized":"2024-06-10T00:29:23.777983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.100923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.775337+0000","last_change":"2024-06-10T00:28:37.655424+0000","last_active":"2024-06-10T00:29:23.775337+0000","last_peered":"2024-06-10T00:29:23.775337+0000","last_clean":"2024-06-10T00:29:23.775337+0000","last_became_active":"2024-06-10T00:28:37.655059+0000","last_became_peered":"2024-06-10T00:28:37.655059+0000","last_unstale":"2024-06-10T00:29:23.775337+0000","last_undegraded":"2024-06-10T00:29:23.775337+0000","last_fullsized":"2024-06-10T00:29:23.775337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:35:04.631794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777872+0000","last_change":"2024-06-10T00:28:37.649997+0000","last_active":"2024-06-10T00:29:23.777872+0000","last_peered":"2024-06-10T00:29:23.777872+0000","last_clean":"2024-06-10T00:29:23.777872+0000","last_became_active":"2024-06-10T00:28:37.649494+0000","last_became_peered":"2024-06-10T00:28:37.649494+0000","last_unstale":"2024-06-10T00:29:23.777872+0000","last_undegraded":"2024-06-10T00:29:23.777872+0000","last_fullsized":"2024-06-10T00:29:23.777872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:44:07.467206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.776851+0000","last_change":"2024-06-10T00:28:37.653064+0000","last_active":"2024-06-10T00:29:23.776851+0000","last_peered":"2024-06-10T00:29:23.776851+0000","last_clean":"2024-06-10T00:29:23.776851+0000","last_became_active":"2024-06-10T00:28:37.652684+0000","last_became_peered":"2024-06-10T00:28:37.652684+0000","last_unstale":"2024-06-10T00:29:23.776851+0000","last_undegraded":"2024-06-10T00:29:23.776851+0000","last_fullsized":"2024-06-10T00:29:23.776851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:30.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777473+0000","last_change":"2024-06-10T00:28:37.655570+0000","last_active":"2024-06-10T00:29:23.777473+0000","last_peered":"2024-06-10T00:29:23.777473+0000","last_clean":"2024-06-10T00:29:23.777473+0000","last_became_active":"2024-06-10T00:28:37.655241+0000","last_became_peered":"2024-06-10T00:28:37.655241+0000","last_unstale":"2024-06-10T00:29:23.777473+0000","last_undegraded":"2024-06-10T00:29:23.777473+0000","last_fullsized":"2024-06-10T00:29:23.777473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:07:03.543064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854322+0000","last_change":"2024-06-10T00:28:37.665607+0000","last_active":"2024-06-10T00:29:23.854322+0000","last_peered":"2024-06-10T00:29:23.854322+0000","last_clean":"2024-06-10T00:29:23.854322+0000","last_became_active":"2024-06-10T00:28:37.664943+0000","last_became_peered":"2024-06-10T00:28:37.664943+0000","last_unstale":"2024-06-10T00:29:23.854322+0000","last_undegraded":"2024-06-10T00:29:23.854322+0000","last_fullsized":"2024-06-10T00:29:23.854322+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:43:02.940298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780365+0000","last_change":"2024-06-10T00:28:37.658214+0000","last_active":"2024-06-10T00:29:23.780365+0000","last_peered":"2024-06-10T00:29:23.780365+0000","last_clean":"2024-06-10T00:29:23.780365+0000","last_became_active":"2024-06-10T00:28:37.657295+0000","last_became_peered":"2024-06-10T00:28:37.657295+0000","last_unstale":"2024-06-10T00:29:23.780365+0000","last_undegraded":"2024-06-10T00:29:23.780365+0000","last_fullsized":"2024-06-10T00:29:23.780365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:20.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774705+0000","last_change":"2024-06-10T00:28:37.656645+0000","last_active":"2024-06-10T00:29:23.774705+0000","last_peered":"2024-06-10T00:29:23.774705+0000","last_clean":"2024-06-10T00:29:23.774705+0000","last_became_active":"2024-06-10T00:28:37.656157+0000","last_became_peered":"2024-06-10T00:28:37.656157+0000","last_unstale":"2024-06-10T00:29:23.774705+0000","last_undegraded":"2024-06-10T00:29:23.774705+0000","last_fullsized":"2024-06-10T00:29:23.774705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:00.953804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.775389+0000","last_change":"2024-06-10T00:28:37.664336+0000","last_active":"2024-06-10T00:29:23.775389+0000","last_peered":"2024-06-10T00:29:23.775389+0000","last_clean":"2024-06-10T00:29:23.775389+0000","last_became_active":"2024-06-10T00:28:37.663960+0000","last_became_peered":"2024-06-10T00:28:37.663960+0000","last_unstale":"2024-06-10T00:29:23.775389+0000","last_undegraded":"2024-06-10T00:29:23.775389+0000","last_fullsized":"2024-06-10T00:29:23.775389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:59:32.774442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774631+0000","last_change":"2024-06-10T00:28:48.705930+0000","last_active":"2024-06-10T00:29:23.774631+0000","last_peered":"2024-06-10T00:29:23.774631+0000","last_clean":"2024-06-10T00:29:23.774631+0000","last_became_active":"2024-06-10T00:28:48.705558+0000","last_became_peered":"2024-06-10T00:28:48.705558+0000","last_unstale":"2024-06-10T00:29:23.774631+0000","last_undegraded":"2024-06-10T00:29:23.774631+0000","last_fullsized":"2024-06-10T00:29:23.774631+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:41:44.914378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'13","reported_seq":89,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:06.915208+0000","last_change":"2024-06-10T00:29:23.862141+0000","last_active":"2024-06-10T00:30:06.915208+0000","last_peered":"2024-06-10T00:30:06.915208+0000","last_clean":"2024-06-10T00:30:06.915208+0000","last_became_active":"2024-06-10T00:29:23.857051+0000","last_became_peered":"2024-06-10T00:29:23.857051+0000","last_unstale":"2024-06-10T00:30:06.915208+0000","last_undegraded":"2024-06-10T00:30:06.915208+0000","last_fullsized":"2024-06-10T00:30:06.915208+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:41:41.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779865+0000","last_change":"2024-06-10T00:28:37.653938+0000","last_active":"2024-06-10T00:29:23.779865+0000","last_peered":"2024-06-10T00:29:23.779865+0000","last_clean":"2024-06-10T00:29:23.779865+0000","last_became_active":"2024-06-10T00:28:37.653629+0000","last_became_peered":"2024-06-10T00:28:37.653629+0000","last_unstale":"2024-06-10T00:29:23.779865+0000","last_undegraded":"2024-06-10T00:29:23.779865+0000","last_fullsized":"2024-06-10T00:29:23.779865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:21:39.285961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774474+0000","last_change":"2024-06-10T00:28:48.704413+0000","last_active":"2024-06-10T00:29:23.774474+0000","last_peered":"2024-06-10T00:29:23.774474+0000","last_clean":"2024-06-10T00:29:23.774474+0000","last_became_active":"2024-06-10T00:28:48.703972+0000","last_became_peered":"2024-06-10T00:28:48.703972+0000","last_unstale":"2024-06-10T00:29:23.774474+0000","last_undegraded":"2024-06-10T00:29:23.774474+0000","last_fullsized":"2024-06-10T00:29:23.774474+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:51.927435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780310+0000","last_change":"2024-06-10T00:28:34.445413+0000","last_active":"2024-06-10T00:29:23.780310+0000","last_peered":"2024-06-10T00:29:23.780310+0000","last_clean":"2024-06-10T00:29:23.780310+0000","last_became_active":"2024-06-10T00:28:34.445077+0000","last_became_peered":"2024-06-10T00:28:34.445077+0000","last_unstale":"2024-06-10T00:29:23.780310+0000","last_undegraded":"2024-06-10T00:29:23.780310+0000","last_fullsized":"2024-06-10T00:29:23.780310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:19:19.535976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777738+0000","last_change":"2024-06-10T00:28:37.657123+0000","last_active":"2024-06-10T00:29:23.777738+0000","last_peered":"2024-06-10T00:29:23.777738+0000","last_clean":"2024-06-10T00:29:23.777738+0000","last_became_active":"2024-06-10T00:28:37.656791+0000","last_became_peered":"2024-06-10T00:28:37.656791+0000","last_unstale":"2024-06-10T00:29:23.777738+0000","last_undegraded":"2024-06-10T00:29:23.777738+0000","last_fullsized":"2024-06-10T00:29:23.777738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:11.102478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774581+0000","last_change":"2024-06-10T00:28:48.705989+0000","last_active":"2024-06-10T00:29:23.774581+0000","last_peered":"2024-06-10T00:29:23.774581+0000","last_clean":"2024-06-10T00:29:23.774581+0000","last_became_active":"2024-06-10T00:28:48.705555+0000","last_became_peered":"2024-06-10T00:28:48.705555+0000","last_unstale":"2024-06-10T00:29:23.774581+0000","last_undegraded":"2024-06-10T00:29:23.774581+0000","last_fullsized":"2024-06-10T00:29:23.774581+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:36.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'12","reported_seq":53,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:45.878023+0000","last_change":"2024-06-10T00:28:34.400117+0000","last_active":"2024-06-10T00:29:45.878023+0000","last_peered":"2024-06-10T00:29:45.878023+0000","last_clean":"2024-06-10T00:29:45.878023+0000","last_became_active":"2024-06-10T00:28:34.399743+0000","last_became_peered":"2024-06-10T00:28:34.399743+0000","last_unstale":"2024-06-10T00:29:45.878023+0000","last_undegraded":"2024-06-10T00:29:45.878023+0000","last_fullsized":"2024-06-10T00:29:45.878023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:20:57.580146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854581+0000","last_change":"2024-06-10T00:28:37.664005+0000","last_active":"2024-06-10T00:29:23.854581+0000","last_peered":"2024-06-10T00:29:23.854581+0000","last_clean":"2024-06-10T00:29:23.854581+0000","last_became_active":"2024-06-10T00:28:37.661297+0000","last_became_peered":"2024-06-10T00:28:37.661297+0000","last_unstale":"2024-06-10T00:29:23.854581+0000","last_undegraded":"2024-06-10T00:29:23.854581+0000","last_fullsized":"2024-06-10T00:29:23.854581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:57.795080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.859091+0000","last_change":"2024-06-10T00:29:23.859091+0000","last_active":"2024-06-10T00:29:23.859091+0000","last_peered":"2024-06-10T00:29:23.859091+0000","last_clean":"2024-06-10T00:29:23.859091+0000","last_became_active":"2024-06-10T00:29:23.858485+0000","last_became_peered":"2024-06-10T00:29:23.858485+0000","last_unstale":"2024-06-10T00:29:23.859091+0000","last_undegraded":"2024-06-10T00:29:23.859091+0000","last_fullsized":"2024-06-10T00:29:23.859091+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:07:10.925323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.884067+0000","last_change":"2024-06-10T00:28:34.400588+0000","last_active":"2024-06-10T00:29:44.884067+0000","last_peered":"2024-06-10T00:29:44.884067+0000","last_clean":"2024-06-10T00:29:44.884067+0000","last_became_active":"2024-06-10T00:28:34.400027+0000","last_became_peered":"2024-06-10T00:28:34.400027+0000","last_unstale":"2024-06-10T00:29:44.884067+0000","last_undegraded":"2024-06-10T00:29:44.884067+0000","last_fullsized":"2024-06-10T00:29:44.884067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:25:59.890076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855404+0000","last_change":"2024-06-10T00:28:37.659401+0000","last_active":"2024-06-10T00:29:23.855404+0000","last_peered":"2024-06-10T00:29:23.855404+0000","last_clean":"2024-06-10T00:29:23.855404+0000","last_became_active":"2024-06-10T00:28:37.659056+0000","last_became_peered":"2024-06-10T00:28:37.659056+0000","last_unstale":"2024-06-10T00:29:23.855404+0000","last_undegraded":"2024-06-10T00:29:23.855404+0000","last_fullsized":"2024-06-10T00:29:23.855404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:35:01.415630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.775032+0000","last_change":"2024-06-10T00:28:48.702961+0000","last_active":"2024-06-10T00:29:23.775032+0000","last_peered":"2024-06-10T00:29:23.775032+0000","last_clean":"2024-06-10T00:29:23.775032+0000","last_became_active":"2024-06-10T00:28:48.702502+0000","last_became_peered":"2024-06-10T00:28:48.702502+0000","last_unstale":"2024-06-10T00:29:23.775032+0000","last_undegraded":"2024-06-10T00:29:23.775032+0000","last_fullsized":"2024-06-10T00:29:23.775032+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:54.881595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'6","reported_seq":69,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:04.915136+0000","last_change":"2024-06-10T00:28:34.396310+0000","last_active":"2024-06-10T00:30:04.915136+0000","last_peered":"2024-06-10T00:30:04.915136+0000","last_clean":"2024-06-10T00:30:04.915136+0000","last_became_active":"2024-06-10T00:28:34.395401+0000","last_became_peered":"2024-06-10T00:28:34.395401+0000","last_unstale":"2024-06-10T00:30:04.915136+0000","last_undegraded":"2024-06-10T00:30:04.915136+0000","last_fullsized":"2024-06-10T00:30:04.915136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:33:54.300308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.778708+0000","last_change":"2024-06-10T00:28:37.651427+0000","last_active":"2024-06-10T00:29:23.778708+0000","last_peered":"2024-06-10T00:29:23.778708+0000","last_clean":"2024-06-10T00:29:23.778708+0000","last_became_active":"2024-06-10T00:28:37.650510+0000","last_became_peered":"2024-06-10T00:28:37.650510+0000","last_unstale":"2024-06-10T00:29:23.778708+0000","last_undegraded":"2024-06-10T00:29:23.778708+0000","last_fullsized":"2024-06-10T00:29:23.778708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:42.953919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774776+0000","last_change":"2024-06-10T00:28:48.705918+0000","last_active":"2024-06-10T00:29:23.774776+0000","last_peered":"2024-06-10T00:29:23.774776+0000","last_clean":"2024-06-10T00:29:23.774776+0000","last_became_active":"2024-06-10T00:28:48.705462+0000","last_became_peered":"2024-06-10T00:28:48.705462+0000","last_unstale":"2024-06-10T00:29:23.774776+0000","last_undegraded":"2024-06-10T00:29:23.774776+0000","last_fullsized":"2024-06-10T00:29:23.774776+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:59:23.176511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:43.873216+0000","last_change":"2024-06-10T00:28:34.393199+0000","last_active":"2024-06-10T00:29:43.873216+0000","last_peered":"2024-06-10T00:29:43.873216+0000","last_clean":"2024-06-10T00:29:43.873216+0000","last_became_active":"2024-06-10T00:28:34.392777+0000","last_became_peered":"2024-06-10T00:28:34.392777+0000","last_unstale":"2024-06-10T00:29:43.873216+0000","last_undegraded":"2024-06-10T00:29:43.873216+0000","last_fullsized":"2024-06-10T00:29:43.873216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:29.614242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855186+0000","last_change":"2024-06-10T00:28:37.647732+0000","last_active":"2024-06-10T00:29:23.855186+0000","last_peered":"2024-06-10T00:29:23.855186+0000","last_clean":"2024-06-10T00:29:23.855186+0000","last_became_active":"2024-06-10T00:28:37.647391+0000","last_became_peered":"2024-06-10T00:28:37.647391+0000","last_unstale":"2024-06-10T00:29:23.855186+0000","last_undegraded":"2024-06-10T00:29:23.855186+0000","last_fullsized":"2024-06-10T00:29:23.855186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:05:25.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780971+0000","last_change":"2024-06-10T00:29:23.780971+0000","last_active":"2024-06-10T00:29:23.780971+0000","last_peered":"2024-06-10T00:29:23.780971+0000","last_clean":"2024-06-10T00:29:23.780971+0000","last_became_active":"2024-06-10T00:29:23.780034+0000","last_became_peered":"2024-06-10T00:29:23.780034+0000","last_unstale":"2024-06-10T00:29:23.780971+0000","last_undegraded":"2024-06-10T00:29:23.780971+0000","last_fullsized":"2024-06-10T00:29:23.780971+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:47:06.444181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:42.861995+0000","last_change":"2024-06-10T00:28:34.392486+0000","last_active":"2024-06-10T00:29:42.861995+0000","last_peered":"2024-06-10T00:29:42.861995+0000","last_clean":"2024-06-10T00:29:42.861995+0000","last_became_active":"2024-06-10T00:28:34.392141+0000","last_became_peered":"2024-06-10T00:28:34.392141+0000","last_unstale":"2024-06-10T00:29:42.861995+0000","last_undegraded":"2024-06-10T00:29:42.861995+0000","last_fullsized":"2024-06-10T00:29:42.861995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:10:01.534195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779823+0000","last_change":"2024-06-10T00:28:37.651033+0000","last_active":"2024-06-10T00:29:23.779823+0000","last_peered":"2024-06-10T00:29:23.779823+0000","last_clean":"2024-06-10T00:29:23.779823+0000","last_became_active":"2024-06-10T00:28:37.650505+0000","last_became_peered":"2024-06-10T00:28:37.650505+0000","last_unstale":"2024-06-10T00:29:23.779823+0000","last_undegraded":"2024-06-10T00:29:23.779823+0000","last_fullsized":"2024-06-10T00:29:23.779823+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:54.965865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777647+0000","last_change":"2024-06-10T00:28:48.710748+0000","last_active":"2024-06-10T00:29:23.777647+0000","last_peered":"2024-06-10T00:29:23.777647+0000","last_clean":"2024-06-10T00:29:23.777647+0000","last_became_active":"2024-06-10T00:28:48.710353+0000","last_became_peered":"2024-06-10T00:28:48.710353+0000","last_unstale":"2024-06-10T00:29:23.777647+0000","last_undegraded":"2024-06-10T00:29:23.777647+0000","last_fullsized":"2024-06-10T00:29:23.777647+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:30:47.546650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.882907+0000","last_change":"2024-06-10T00:28:34.398911+0000","last_active":"2024-06-10T00:29:44.882907+0000","last_peered":"2024-06-10T00:29:44.882907+0000","last_clean":"2024-06-10T00:29:44.882907+0000","last_became_active":"2024-06-10T00:28:34.398373+0000","last_became_peered":"2024-06-10T00:28:34.398373+0000","last_unstale":"2024-06-10T00:29:44.882907+0000","last_undegraded":"2024-06-10T00:29:44.882907+0000","last_fullsized":"2024-06-10T00:29:44.882907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:05:34.352825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774104+0000","last_change":"2024-06-10T00:28:37.647492+0000","last_active":"2024-06-10T00:29:23.774104+0000","last_peered":"2024-06-10T00:29:23.774104+0000","last_clean":"2024-06-10T00:29:23.774104+0000","last_became_active":"2024-06-10T00:28:37.647103+0000","last_became_peered":"2024-06-10T00:28:37.647103+0000","last_unstale":"2024-06-10T00:29:23.774104+0000","last_undegraded":"2024-06-10T00:29:23.774104+0000","last_fullsized":"2024-06-10T00:29:23.774104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:10.429256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780428+0000","last_change":"2024-06-10T00:28:48.703095+0000","last_active":"2024-06-10T00:29:23.780428+0000","last_peered":"2024-06-10T00:29:23.780428+0000","last_clean":"2024-06-10T00:29:23.780428+0000","last_became_active":"2024-06-10T00:28:48.702687+0000","last_became_peered":"2024-06-10T00:28:48.702687+0000","last_unstale":"2024-06-10T00:29:23.780428+0000","last_undegraded":"2024-06-10T00:29:23.780428+0000","last_fullsized":"2024-06-10T00:29:23.780428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:10:20.074173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'4","reported_seq":89,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:07.918133+0000","last_change":"2024-06-10T00:28:34.394807+0000","last_active":"2024-06-10T00:30:07.918133+0000","last_peered":"2024-06-10T00:30:07.918133+0000","last_clean":"2024-06-10T00:30:07.918133+0000","last_became_active":"2024-06-10T00:28:34.394022+0000","last_became_peered":"2024-06-10T00:28:34.394022+0000","last_unstale":"2024-06-10T00:30:07.918133+0000","last_undegraded":"2024-06-10T00:30:07.918133+0000","last_fullsized":"2024-06-10T00:30:07.918133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:21.626079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855254+0000","last_change":"2024-06-10T00:28:37.662231+0000","last_active":"2024-06-10T00:29:23.855254+0000","last_peered":"2024-06-10T00:29:23.855254+0000","last_clean":"2024-06-10T00:29:23.855254+0000","last_became_active":"2024-06-10T00:28:37.661861+0000","last_became_peered":"2024-06-10T00:28:37.661861+0000","last_unstale":"2024-06-10T00:29:23.855254+0000","last_undegraded":"2024-06-10T00:29:23.855254+0000","last_fullsized":"2024-06-10T00:29:23.855254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:09.705349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774088+0000","last_change":"2024-06-10T00:28:48.706643+0000","last_active":"2024-06-10T00:29:23.774088+0000","last_peered":"2024-06-10T00:29:23.774088+0000","last_clean":"2024-06-10T00:29:23.774088+0000","last_became_active":"2024-06-10T00:28:48.706127+0000","last_became_peered":"2024-06-10T00:28:48.706127+0000","last_unstale":"2024-06-10T00:29:23.774088+0000","last_undegraded":"2024-06-10T00:29:23.774088+0000","last_fullsized":"2024-06-10T00:29:23.774088+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:55.031303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.883455+0000","last_change":"2024-06-10T00:28:34.392808+0000","last_active":"2024-06-10T00:29:44.883455+0000","last_peered":"2024-06-10T00:29:44.883455+0000","last_clean":"2024-06-10T00:29:44.883455+0000","last_became_active":"2024-06-10T00:28:34.392450+0000","last_became_peered":"2024-06-10T00:28:34.392450+0000","last_unstale":"2024-06-10T00:29:44.883455+0000","last_undegraded":"2024-06-10T00:29:44.883455+0000","last_fullsized":"2024-06-10T00:29:44.883455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:06:00.493316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777850+0000","last_change":"2024-06-10T00:28:37.656099+0000","last_active":"2024-06-10T00:29:23.777850+0000","last_peered":"2024-06-10T00:29:23.777850+0000","last_clean":"2024-06-10T00:29:23.777850+0000","last_became_active":"2024-06-10T00:28:37.655640+0000","last_became_peered":"2024-06-10T00:28:37.655640+0000","last_unstale":"2024-06-10T00:29:23.777850+0000","last_undegraded":"2024-06-10T00:29:23.777850+0000","last_fullsized":"2024-06-10T00:29:23.777850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:59:26.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774852+0000","last_change":"2024-06-10T00:28:48.702628+0000","last_active":"2024-06-10T00:29:23.774852+0000","last_peered":"2024-06-10T00:29:23.774852+0000","last_clean":"2024-06-10T00:29:23.774852+0000","last_became_active":"2024-06-10T00:28:48.702109+0000","last_became_peered":"2024-06-10T00:28:48.702109+0000","last_unstale":"2024-06-10T00:29:23.774852+0000","last_undegraded":"2024-06-10T00:29:23.774852+0000","last_fullsized":"2024-06-10T00:29:23.774852+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:25.262348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:43.872188+0000","last_change":"2024-06-10T00:28:34.446788+0000","last_active":"2024-06-10T00:29:43.872188+0000","last_peered":"2024-06-10T00:29:43.872188+0000","last_clean":"2024-06-10T00:29:43.872188+0000","last_became_active":"2024-06-10T00:28:34.446377+0000","last_became_peered":"2024-06-10T00:28:34.446377+0000","last_unstale":"2024-06-10T00:29:43.872188+0000","last_undegraded":"2024-06-10T00:29:43.872188+0000","last_fullsized":"2024-06-10T00:29:43.872188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:16.914244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855534+0000","last_change":"2024-06-10T00:28:37.666159+0000","last_active":"2024-06-10T00:29:23.855534+0000","last_peered":"2024-06-10T00:29:23.855534+0000","last_clean":"2024-06-10T00:29:23.855534+0000","last_became_active":"2024-06-10T00:28:37.665711+0000","last_became_peered":"2024-06-10T00:28:37.665711+0000","last_unstale":"2024-06-10T00:29:23.855534+0000","last_undegraded":"2024-06-10T00:29:23.855534+0000","last_fullsized":"2024-06-10T00:29:23.855534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:03.982375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774073+0000","last_change":"2024-06-10T00:28:48.705071+0000","last_active":"2024-06-10T00:29:23.774073+0000","last_peered":"2024-06-10T00:29:23.774073+0000","last_clean":"2024-06-10T00:29:23.774073+0000","last_became_active":"2024-06-10T00:28:48.702211+0000","last_became_peered":"2024-06-10T00:28:48.702211+0000","last_unstale":"2024-06-10T00:29:23.774073+0000","last_undegraded":"2024-06-10T00:29:23.774073+0000","last_fullsized":"2024-06-10T00:29:23.774073+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:13.361985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'33","reported_seq":126,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:07.918881+0000","last_change":"2024-06-10T00:28:34.389784+0000","last_active":"2024-06-10T00:30:07.918881+0000","last_peered":"2024-06-10T00:30:07.918881+0000","last_clean":"2024-06-10T00:30:07.918881+0000","last_became_active":"2024-06-10T00:28:34.389416+0000","last_became_peered":"2024-06-10T00:28:34.389416+0000","last_unstale":"2024-06-10T00:30:07.918881+0000","last_undegraded":"2024-06-10T00:30:07.918881+0000","last_fullsized":"2024-06-10T00:30:07.918881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:23:05.803503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58390,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":84,"num_write":35,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777759+0000","last_change":"2024-06-10T00:28:37.656174+0000","last_active":"2024-06-10T00:29:23.777759+0000","last_peered":"2024-06-10T00:29:23.777759+0000","last_clean":"2024-06-10T00:29:23.777759+0000","last_became_active":"2024-06-10T00:28:37.655794+0000","last_became_peered":"2024-06-10T00:28:37.655794+0000","last_unstale":"2024-06-10T00:29:23.777759+0000","last_undegraded":"2024-06-10T00:29:23.777759+0000","last_fullsized":"2024-06-10T00:29:23.777759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:22:00.310309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855645+0000","last_change":"2024-06-10T00:28:48.709915+0000","last_active":"2024-06-10T00:29:23.855645+0000","last_peered":"2024-06-10T00:29:23.855645+0000","last_clean":"2024-06-10T00:29:23.855645+0000","last_became_active":"2024-06-10T00:28:48.709426+0000","last_became_peered":"2024-06-10T00:28:48.709426+0000","last_unstale":"2024-06-10T00:29:23.855645+0000","last_undegraded":"2024-06-10T00:29:23.855645+0000","last_fullsized":"2024-06-10T00:29:23.855645+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:55.150617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:45.883835+0000","last_change":"2024-06-10T00:28:34.389136+0000","last_active":"2024-06-10T00:29:45.883835+0000","last_peered":"2024-06-10T00:29:45.883835+0000","last_clean":"2024-06-10T00:29:45.883835+0000","last_became_active":"2024-06-10T00:28:34.388701+0000","last_became_peered":"2024-06-10T00:28:34.388701+0000","last_unstale":"2024-06-10T00:29:45.883835+0000","last_undegraded":"2024-06-10T00:29:45.883835+0000","last_fullsized":"2024-06-10T00:29:45.883835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:13:24.907824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777188+0000","last_change":"2024-06-10T00:28:37.659048+0000","last_active":"2024-06-10T00:29:23.777188+0000","last_peered":"2024-06-10T00:29:23.777188+0000","last_clean":"2024-06-10T00:29:23.777188+0000","last_became_active":"2024-06-10T00:28:37.658710+0000","last_became_peered":"2024-06-10T00:28:37.658710+0000","last_unstale":"2024-06-10T00:29:23.777188+0000","last_undegraded":"2024-06-10T00:29:23.777188+0000","last_fullsized":"2024-06-10T00:29:23.777188+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:44.314559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779228+0000","last_change":"2024-06-10T00:28:48.709683+0000","last_active":"2024-06-10T00:29:23.779228+0000","last_peered":"2024-06-10T00:29:23.779228+0000","last_clean":"2024-06-10T00:29:23.779228+0000","last_became_active":"2024-06-10T00:28:48.709241+0000","last_became_peered":"2024-06-10T00:28:48.709241+0000","last_unstale":"2024-06-10T00:29:23.779228+0000","last_undegraded":"2024-06-10T00:29:23.779228+0000","last_fullsized":"2024-06-10T00:29:23.779228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:25.190043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.883378+0000","last_change":"2024-06-10T00:29:23.782029+0000","last_active":"2024-06-10T00:29:44.883378+0000","last_peered":"2024-06-10T00:29:44.883378+0000","last_clean":"2024-06-10T00:29:44.883378+0000","last_became_active":"2024-06-10T00:29:23.781565+0000","last_became_peered":"2024-06-10T00:29:23.781565+0000","last_unstale":"2024-06-10T00:29:44.883378+0000","last_undegraded":"2024-06-10T00:29:44.883378+0000","last_fullsized":"2024-06-10T00:29:44.883378+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:53.258760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777619+0000","last_change":"2024-06-10T00:28:37.653385+0000","last_active":"2024-06-10T00:29:23.777619+0000","last_peered":"2024-06-10T00:29:23.777619+0000","last_clean":"2024-06-10T00:29:23.777619+0000","last_became_active":"2024-06-10T00:28:37.653020+0000","last_became_peered":"2024-06-10T00:28:37.653020+0000","last_unstale":"2024-06-10T00:29:23.777619+0000","last_undegraded":"2024-06-10T00:29:23.777619+0000","last_fullsized":"2024-06-10T00:29:23.777619+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:57:54.469544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774495+0000","last_change":"2024-06-10T00:28:48.707540+0000","last_active":"2024-06-10T00:29:23.774495+0000","last_peered":"2024-06-10T00:29:23.774495+0000","last_clean":"2024-06-10T00:29:23.774495+0000","last_became_active":"2024-06-10T00:28:48.707143+0000","last_became_peered":"2024-06-10T00:28:48.707143+0000","last_unstale":"2024-06-10T00:29:23.774495+0000","last_undegraded":"2024-06-10T00:29:23.774495+0000","last_fullsized":"2024-06-10T00:29:23.774495+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:20:29.344808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'6","reported_seq":71,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:05.914398+0000","last_change":"2024-06-10T00:28:34.397496+0000","last_active":"2024-06-10T00:30:05.914398+0000","last_peered":"2024-06-10T00:30:05.914398+0000","last_clean":"2024-06-10T00:30:05.914398+0000","last_became_active":"2024-06-10T00:28:34.397124+0000","last_became_peered":"2024-06-10T00:28:34.397124+0000","last_unstale":"2024-06-10T00:30:05.914398+0000","last_undegraded":"2024-06-10T00:30:05.914398+0000","last_fullsized":"2024-06-10T00:30:05.914398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:41:48.609354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777477+0000","last_change":"2024-06-10T00:28:37.658338+0000","last_active":"2024-06-10T00:29:23.777477+0000","last_peered":"2024-06-10T00:29:23.777477+0000","last_clean":"2024-06-10T00:29:23.777477+0000","last_became_active":"2024-06-10T00:28:37.655910+0000","last_became_peered":"2024-06-10T00:28:37.655910+0000","last_unstale":"2024-06-10T00:29:23.777477+0000","last_undegraded":"2024-06-10T00:29:23.777477+0000","last_fullsized":"2024-06-10T00:29:23.777477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:38.501335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777562+0000","last_change":"2024-06-10T00:28:48.711151+0000","last_active":"2024-06-10T00:29:23.777562+0000","last_peered":"2024-06-10T00:29:23.777562+0000","last_clean":"2024-06-10T00:29:23.777562+0000","last_became_active":"2024-06-10T00:28:48.710768+0000","last_became_peered":"2024-06-10T00:28:48.710768+0000","last_unstale":"2024-06-10T00:29:23.777562+0000","last_undegraded":"2024-06-10T00:29:23.777562+0000","last_fullsized":"2024-06-10T00:29:23.777562+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:15:22.791839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":58,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.882578+0000","last_change":"2024-06-10T00:28:34.445812+0000","last_active":"2024-06-10T00:29:44.882578+0000","last_peered":"2024-06-10T00:29:44.882578+0000","last_clean":"2024-06-10T00:29:44.882578+0000","last_became_active":"2024-06-10T00:28:34.445472+0000","last_became_peered":"2024-06-10T00:28:34.445472+0000","last_unstale":"2024-06-10T00:29:44.882578+0000","last_undegraded":"2024-06-10T00:29:44.882578+0000","last_fullsized":"2024-06-10T00:29:44.882578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:00.912050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854943+0000","last_change":"2024-06-10T00:28:37.655345+0000","last_active":"2024-06-10T00:29:23.854943+0000","last_peered":"2024-06-10T00:29:23.854943+0000","last_clean":"2024-06-10T00:29:23.854943+0000","last_became_active":"2024-06-10T00:28:37.654954+0000","last_became_peered":"2024-06-10T00:28:37.654954+0000","last_unstale":"2024-06-10T00:29:23.854943+0000","last_undegraded":"2024-06-10T00:29:23.854943+0000","last_fullsized":"2024-06-10T00:29:23.854943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:39:27.403006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854755+0000","last_change":"2024-06-10T00:28:48.705247+0000","last_active":"2024-06-10T00:29:23.854755+0000","last_peered":"2024-06-10T00:29:23.854755+0000","last_clean":"2024-06-10T00:29:23.854755+0000","last_became_active":"2024-06-10T00:28:48.704822+0000","last_became_peered":"2024-06-10T00:28:48.704822+0000","last_unstale":"2024-06-10T00:29:23.854755+0000","last_undegraded":"2024-06-10T00:29:23.854755+0000","last_fullsized":"2024-06-10T00:29:23.854755+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:24:59.991491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.857565+0000","last_change":"2024-06-10T00:28:34.396792+0000","last_active":"2024-06-10T00:29:44.857565+0000","last_peered":"2024-06-10T00:29:44.857565+0000","last_clean":"2024-06-10T00:29:44.857565+0000","last_became_active":"2024-06-10T00:28:34.396442+0000","last_became_peered":"2024-06-10T00:28:34.396442+0000","last_unstale":"2024-06-10T00:29:44.857565+0000","last_undegraded":"2024-06-10T00:29:44.857565+0000","last_fullsized":"2024-06-10T00:29:44.857565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:46.436198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780106+0000","last_change":"2024-06-10T00:28:37.657698+0000","last_active":"2024-06-10T00:29:23.780106+0000","last_peered":"2024-06-10T00:29:23.780106+0000","last_clean":"2024-06-10T00:29:23.780106+0000","last_became_active":"2024-06-10T00:28:37.657364+0000","last_became_peered":"2024-06-10T00:28:37.657364+0000","last_unstale":"2024-06-10T00:29:23.780106+0000","last_undegraded":"2024-06-10T00:29:23.780106+0000","last_fullsized":"2024-06-10T00:29:23.780106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:23.097015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.778567+0000","last_change":"2024-06-10T00:28:48.727567+0000","last_active":"2024-06-10T00:29:23.778567+0000","last_peered":"2024-06-10T00:29:23.778567+0000","last_clean":"2024-06-10T00:29:23.778567+0000","last_became_active":"2024-06-10T00:28:48.727108+0000","last_became_peered":"2024-06-10T00:28:48.727108+0000","last_unstale":"2024-06-10T00:29:23.778567+0000","last_undegraded":"2024-06-10T00:29:23.778567+0000","last_fullsized":"2024-06-10T00:29:23.778567+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:35:00.824177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774644+0000","last_change":"2024-06-10T00:28:34.446832+0000","last_active":"2024-06-10T00:29:23.774644+0000","last_peered":"2024-06-10T00:29:23.774644+0000","last_clean":"2024-06-10T00:29:23.774644+0000","last_became_active":"2024-06-10T00:28:34.446435+0000","last_became_peered":"2024-06-10T00:28:34.446435+0000","last_unstale":"2024-06-10T00:29:23.774644+0000","last_undegraded":"2024-06-10T00:29:23.774644+0000","last_fullsized":"2024-06-10T00:29:23.774644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:58.296674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.856039+0000","last_change":"2024-06-10T00:28:37.647671+0000","last_active":"2024-06-10T00:29:23.856039+0000","last_peered":"2024-06-10T00:29:23.856039+0000","last_clean":"2024-06-10T00:29:23.856039+0000","last_became_active":"2024-06-10T00:28:37.647319+0000","last_became_peered":"2024-06-10T00:28:37.647319+0000","last_unstale":"2024-06-10T00:29:23.856039+0000","last_undegraded":"2024-06-10T00:29:23.856039+0000","last_fullsized":"2024-06-10T00:29:23.856039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:22:51.910596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777130+0000","last_change":"2024-06-10T00:28:48.704683+0000","last_active":"2024-06-10T00:29:23.777130+0000","last_peered":"2024-06-10T00:29:23.777130+0000","last_clean":"2024-06-10T00:29:23.777130+0000","last_became_active":"2024-06-10T00:28:48.704297+0000","last_became_peered":"2024-06-10T00:28:48.704297+0000","last_unstale":"2024-06-10T00:29:23.777130+0000","last_undegraded":"2024-06-10T00:29:23.777130+0000","last_fullsized":"2024-06-10T00:29:23.777130+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:44.980544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'5","reported_seq":66,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:04.913070+0000","last_change":"2024-06-10T00:28:34.393328+0000","last_active":"2024-06-10T00:30:04.913070+0000","last_peered":"2024-06-10T00:30:04.913070+0000","last_clean":"2024-06-10T00:30:04.913070+0000","last_became_active":"2024-06-10T00:28:34.391968+0000","last_became_peered":"2024-06-10T00:28:34.391968+0000","last_unstale":"2024-06-10T00:30:04.913070+0000","last_undegraded":"2024-06-10T00:30:04.913070+0000","last_fullsized":"2024-06-10T00:30:04.913070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:18.898180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.856246+0000","last_change":"2024-06-10T00:28:37.652555+0000","last_active":"2024-06-10T00:29:23.856246+0000","last_peered":"2024-06-10T00:29:23.856246+0000","last_clean":"2024-06-10T00:29:23.856246+0000","last_became_active":"2024-06-10T00:28:37.651476+0000","last_became_peered":"2024-06-10T00:28:37.651476+0000","last_unstale":"2024-06-10T00:29:23.856246+0000","last_undegraded":"2024-06-10T00:29:23.856246+0000","last_fullsized":"2024-06-10T00:29:23.856246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:04:57.823224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.778229+0000","last_change":"2024-06-10T00:28:48.727568+0000","last_active":"2024-06-10T00:29:23.778229+0000","last_peered":"2024-06-10T00:29:23.778229+0000","last_clean":"2024-06-10T00:29:23.778229+0000","last_became_active":"2024-06-10T00:28:48.727018+0000","last_became_peered":"2024-06-10T00:28:48.727018+0000","last_unstale":"2024-06-10T00:29:23.778229+0000","last_undegraded":"2024-06-10T00:29:23.778229+0000","last_fullsized":"2024-06-10T00:29:23.778229+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:02.025212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854952+0000","last_change":"2024-06-10T00:28:37.645264+0000","last_active":"2024-06-10T00:29:23.854952+0000","last_peered":"2024-06-10T00:29:23.854952+0000","last_clean":"2024-06-10T00:29:23.854952+0000","last_became_active":"2024-06-10T00:28:37.644860+0000","last_became_peered":"2024-06-10T00:28:37.644860+0000","last_unstale":"2024-06-10T00:29:23.854952+0000","last_undegraded":"2024-06-10T00:29:23.854952+0000","last_fullsized":"2024-06-10T00:29:23.854952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:28.309172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'5","reported_seq":66,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:04.912590+0000","last_change":"2024-06-10T00:28:34.445501+0000","last_active":"2024-06-10T00:30:04.912590+0000","last_peered":"2024-06-10T00:30:04.912590+0000","last_clean":"2024-06-10T00:30:04.912590+0000","last_became_active":"2024-06-10T00:28:34.445128+0000","last_became_peered":"2024-06-10T00:28:34.445128+0000","last_unstale":"2024-06-10T00:30:04.912590+0000","last_undegraded":"2024-06-10T00:30:04.912590+0000","last_fullsized":"2024-06-10T00:30:04.912590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:59.231585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777876+0000","last_change":"2024-06-10T00:28:48.709852+0000","last_active":"2024-06-10T00:29:23.777876+0000","last_peered":"2024-06-10T00:29:23.777876+0000","last_clean":"2024-06-10T00:29:23.777876+0000","last_became_active":"2024-06-10T00:28:48.709423+0000","last_became_peered":"2024-06-10T00:28:48.709423+0000","last_unstale":"2024-06-10T00:29:23.777876+0000","last_undegraded":"2024-06-10T00:29:23.777876+0000","last_fullsized":"2024-06-10T00:29:23.777876+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:42.123563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780357+0000","last_change":"2024-06-10T00:28:37.657423+0000","last_active":"2024-06-10T00:29:23.780357+0000","last_peered":"2024-06-10T00:29:23.780357+0000","last_clean":"2024-06-10T00:29:23.780357+0000","last_became_active":"2024-06-10T00:28:37.657140+0000","last_became_peered":"2024-06-10T00:28:37.657140+0000","last_unstale":"2024-06-10T00:29:23.780357+0000","last_undegraded":"2024-06-10T00:29:23.780357+0000","last_fullsized":"2024-06-10T00:29:23.780357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:20:08.943066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:45.884674+0000","last_change":"2024-06-10T00:28:34.392585+0000","last_active":"2024-06-10T00:29:45.884674+0000","last_peered":"2024-06-10T00:29:45.884674+0000","last_clean":"2024-06-10T00:29:45.884674+0000","last_became_active":"2024-06-10T00:28:34.392229+0000","last_became_peered":"2024-06-10T00:28:34.392229+0000","last_unstale":"2024-06-10T00:29:45.884674+0000","last_undegraded":"2024-06-10T00:29:45.884674+0000","last_fullsized":"2024-06-10T00:29:45.884674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:11.275761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":373,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777128+0000","last_change":"2024-06-10T00:27:12.072577+0000","last_active":"2024-06-10T00:29:23.777128+0000","last_peered":"2024-06-10T00:29:23.777128+0000","last_clean":"2024-06-10T00:29:23.777128+0000","last_became_active":"2024-06-10T00:27:12.072193+0000","last_became_peered":"2024-06-10T00:27:12.072193+0000","last_unstale":"2024-06-10T00:29:23.777128+0000","last_undegraded":"2024-06-10T00:29:23.777128+0000","last_fullsized":"2024-06-10T00:29:23.777128+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_clean_scrub_stamp":"2024-06-10T00:23:46.826438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:57.782124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855452+0000","last_change":"2024-06-10T00:28:48.727913+0000","last_active":"2024-06-10T00:29:23.855452+0000","last_peered":"2024-06-10T00:29:23.855452+0000","last_clean":"2024-06-10T00:29:23.855452+0000","last_became_active":"2024-06-10T00:28:48.727402+0000","last_became_peered":"2024-06-10T00:28:48.727402+0000","last_unstale":"2024-06-10T00:29:23.855452+0000","last_undegraded":"2024-06-10T00:29:23.855452+0000","last_fullsized":"2024-06-10T00:29:23.855452+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:10:25.051296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.778107+0000","last_change":"2024-06-10T00:28:37.659901+0000","last_active":"2024-06-10T00:29:23.778107+0000","last_peered":"2024-06-10T00:29:23.778107+0000","last_clean":"2024-06-10T00:29:23.778107+0000","last_became_active":"2024-06-10T00:28:37.659553+0000","last_became_peered":"2024-06-10T00:28:37.659553+0000","last_unstale":"2024-06-10T00:29:23.778107+0000","last_undegraded":"2024-06-10T00:29:23.778107+0000","last_fullsized":"2024-06-10T00:29:23.778107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:25.694823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:42.861159+0000","last_change":"2024-06-10T00:28:34.398155+0000","last_active":"2024-06-10T00:29:42.861159+0000","last_peered":"2024-06-10T00:29:42.861159+0000","last_clean":"2024-06-10T00:29:42.861159+0000","last_became_active":"2024-06-10T00:28:34.397793+0000","last_became_peered":"2024-06-10T00:28:34.397793+0000","last_unstale":"2024-06-10T00:29:42.861159+0000","last_undegraded":"2024-06-10T00:29:42.861159+0000","last_fullsized":"2024-06-10T00:29:42.861159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:21.188139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777637+0000","last_change":"2024-06-10T00:28:48.709736+0000","last_active":"2024-06-10T00:29:23.777637+0000","last_peered":"2024-06-10T00:29:23.777637+0000","last_clean":"2024-06-10T00:29:23.777637+0000","last_became_active":"2024-06-10T00:28:48.709293+0000","last_became_peered":"2024-06-10T00:28:48.709293+0000","last_unstale":"2024-06-10T00:29:23.777637+0000","last_undegraded":"2024-06-10T00:29:23.777637+0000","last_fullsized":"2024-06-10T00:29:23.777637+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:30:04.840656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.881597+0000","last_change":"2024-06-10T00:29:23.780030+0000","last_active":"2024-06-10T00:29:44.881597+0000","last_peered":"2024-06-10T00:29:44.881597+0000","last_clean":"2024-06-10T00:29:44.881597+0000","last_became_active":"2024-06-10T00:29:23.779589+0000","last_became_peered":"2024-06-10T00:29:23.779589+0000","last_unstale":"2024-06-10T00:29:44.881597+0000","last_undegraded":"2024-06-10T00:29:44.881597+0000","last_fullsized":"2024-06-10T00:29:44.881597+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:53.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777314+0000","last_change":"2024-06-10T00:28:37.658330+0000","last_active":"2024-06-10T00:29:23.777314+0000","last_peered":"2024-06-10T00:29:23.777314+0000","last_clean":"2024-06-10T00:29:23.777314+0000","last_became_active":"2024-06-10T00:28:37.657343+0000","last_became_peered":"2024-06-10T00:28:37.657343+0000","last_unstale":"2024-06-10T00:29:23.777314+0000","last_undegraded":"2024-06-10T00:29:23.777314+0000","last_fullsized":"2024-06-10T00:29:23.777314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:31:52.167402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779704+0000","last_change":"2024-06-10T00:28:48.727506+0000","last_active":"2024-06-10T00:29:23.779704+0000","last_peered":"2024-06-10T00:29:23.779704+0000","last_clean":"2024-06-10T00:29:23.779704+0000","last_became_active":"2024-06-10T00:28:48.727068+0000","last_became_peered":"2024-06-10T00:28:48.727068+0000","last_unstale":"2024-06-10T00:29:23.779704+0000","last_undegraded":"2024-06-10T00:29:23.779704+0000","last_fullsized":"2024-06-10T00:29:23.779704+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:42.147908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'14","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:02.663214+0000","last_change":"2024-06-10T00:28:34.399872+0000","last_active":"2024-06-10T00:30:02.663214+0000","last_peered":"2024-06-10T00:30:02.663214+0000","last_clean":"2024-06-10T00:30:02.663214+0000","last_became_active":"2024-06-10T00:28:34.399493+0000","last_became_peered":"2024-06-10T00:28:34.399493+0000","last_unstale":"2024-06-10T00:30:02.663214+0000","last_undegraded":"2024-06-10T00:30:02.663214+0000","last_fullsized":"2024-06-10T00:30:02.663214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:24:10.018110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780712+0000","last_change":"2024-06-10T00:28:37.656597+0000","last_active":"2024-06-10T00:29:23.780712+0000","last_peered":"2024-06-10T00:29:23.780712+0000","last_clean":"2024-06-10T00:29:23.780712+0000","last_became_active":"2024-06-10T00:28:37.654356+0000","last_became_peered":"2024-06-10T00:28:37.654356+0000","last_unstale":"2024-06-10T00:29:23.780712+0000","last_undegraded":"2024-06-10T00:29:23.780712+0000","last_fullsized":"2024-06-10T00:29:23.780712+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:07:12.950683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774497+0000","last_change":"2024-06-10T00:28:48.702686+0000","last_active":"2024-06-10T00:29:23.774497+0000","last_peered":"2024-06-10T00:29:23.774497+0000","last_clean":"2024-06-10T00:29:23.774497+0000","last_became_active":"2024-06-10T00:28:48.702273+0000","last_became_peered":"2024-06-10T00:28:48.702273+0000","last_unstale":"2024-06-10T00:29:23.774497+0000","last_undegraded":"2024-06-10T00:29:23.774497+0000","last_fullsized":"2024-06-10T00:29:23.774497+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:00.078232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.883862+0000","last_change":"2024-06-10T00:28:34.447549+0000","last_active":"2024-06-10T00:29:44.883862+0000","last_peered":"2024-06-10T00:29:44.883862+0000","last_clean":"2024-06-10T00:29:44.883862+0000","last_became_active":"2024-06-10T00:28:34.447197+0000","last_became_peered":"2024-06-10T00:28:34.447197+0000","last_unstale":"2024-06-10T00:29:44.883862+0000","last_undegraded":"2024-06-10T00:29:44.883862+0000","last_fullsized":"2024-06-10T00:29:44.883862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:47.525153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777770+0000","last_change":"2024-06-10T00:28:37.655585+0000","last_active":"2024-06-10T00:29:23.777770+0000","last_peered":"2024-06-10T00:29:23.777770+0000","last_clean":"2024-06-10T00:29:23.777770+0000","last_became_active":"2024-06-10T00:28:37.655225+0000","last_became_peered":"2024-06-10T00:28:37.655225+0000","last_unstale":"2024-06-10T00:29:23.777770+0000","last_undegraded":"2024-06-10T00:29:23.777770+0000","last_fullsized":"2024-06-10T00:29:23.777770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:57:25.184924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.857447+0000","last_change":"2024-06-10T00:29:23.857447+0000","last_active":"2024-06-10T00:29:23.857447+0000","last_peered":"2024-06-10T00:29:23.857447+0000","last_clean":"2024-06-10T00:29:23.857447+0000","last_became_active":"2024-06-10T00:29:23.856943+0000","last_became_peered":"2024-06-10T00:29:23.856943+0000","last_unstale":"2024-06-10T00:29:23.857447+0000","last_undegraded":"2024-06-10T00:29:23.857447+0000","last_fullsized":"2024-06-10T00:29:23.857447+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:58:44.548818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:41.854544+0000","last_change":"2024-06-10T00:28:34.445637+0000","last_active":"2024-06-10T00:29:41.854544+0000","last_peered":"2024-06-10T00:29:41.854544+0000","last_clean":"2024-06-10T00:29:41.854544+0000","last_became_active":"2024-06-10T00:28:34.445324+0000","last_became_peered":"2024-06-10T00:28:34.445324+0000","last_unstale":"2024-06-10T00:29:41.854544+0000","last_undegraded":"2024-06-10T00:29:41.854544+0000","last_fullsized":"2024-06-10T00:29:41.854544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:05.713557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777101+0000","last_change":"2024-06-10T00:28:37.659037+0000","last_active":"2024-06-10T00:29:23.777101+0000","last_peered":"2024-06-10T00:29:23.777101+0000","last_clean":"2024-06-10T00:29:23.777101+0000","last_became_active":"2024-06-10T00:28:37.658705+0000","last_became_peered":"2024-06-10T00:28:37.658705+0000","last_unstale":"2024-06-10T00:29:23.777101+0000","last_undegraded":"2024-06-10T00:29:23.777101+0000","last_fullsized":"2024-06-10T00:29:23.777101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:20:52.797237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774629+0000","last_change":"2024-06-10T00:28:48.704130+0000","last_active":"2024-06-10T00:29:23.774629+0000","last_peered":"2024-06-10T00:29:23.774629+0000","last_clean":"2024-06-10T00:29:23.774629+0000","last_became_active":"2024-06-10T00:28:48.703738+0000","last_became_peered":"2024-06-10T00:28:48.703738+0000","last_unstale":"2024-06-10T00:29:23.774629+0000","last_undegraded":"2024-06-10T00:29:23.774629+0000","last_fullsized":"2024-06-10T00:29:23.774629+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:01:40.436710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.881279+0000","last_change":"2024-06-10T00:28:34.386667+0000","last_active":"2024-06-10T00:29:44.881279+0000","last_peered":"2024-06-10T00:29:44.881279+0000","last_clean":"2024-06-10T00:29:44.881279+0000","last_became_active":"2024-06-10T00:28:34.386363+0000","last_became_peered":"2024-06-10T00:28:34.386363+0000","last_unstale":"2024-06-10T00:29:44.881279+0000","last_undegraded":"2024-06-10T00:29:44.881279+0000","last_fullsized":"2024-06-10T00:29:44.881279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:11.850162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777197+0000","last_change":"2024-06-10T00:28:37.664754+0000","last_active":"2024-06-10T00:29:23.777197+0000","last_peered":"2024-06-10T00:29:23.777197+0000","last_clean":"2024-06-10T00:29:23.777197+0000","last_became_active":"2024-06-10T00:28:37.658574+0000","last_became_peered":"2024-06-10T00:28:37.658574+0000","last_unstale":"2024-06-10T00:29:23.777197+0000","last_undegraded":"2024-06-10T00:29:23.777197+0000","last_fullsized":"2024-06-10T00:29:23.777197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:16:47.437822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.775310+0000","last_change":"2024-06-10T00:28:48.706560+0000","last_active":"2024-06-10T00:29:23.775310+0000","last_peered":"2024-06-10T00:29:23.775310+0000","last_clean":"2024-06-10T00:29:23.775310+0000","last_became_active":"2024-06-10T00:28:48.705986+0000","last_became_peered":"2024-06-10T00:28:48.705986+0000","last_unstale":"2024-06-10T00:29:23.775310+0000","last_undegraded":"2024-06-10T00:29:23.775310+0000","last_fullsized":"2024-06-10T00:29:23.775310+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:21:26.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'3","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:43.872209+0000","last_change":"2024-06-10T00:28:34.394509+0000","last_active":"2024-06-10T00:29:43.872209+0000","last_peered":"2024-06-10T00:29:43.872209+0000","last_clean":"2024-06-10T00:29:43.872209+0000","last_became_active":"2024-06-10T00:28:34.394105+0000","last_became_peered":"2024-06-10T00:28:34.394105+0000","last_unstale":"2024-06-10T00:29:43.872209+0000","last_undegraded":"2024-06-10T00:29:43.872209+0000","last_fullsized":"2024-06-10T00:29:43.872209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:13.415544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780855+0000","last_change":"2024-06-10T00:28:37.653192+0000","last_active":"2024-06-10T00:29:23.780855+0000","last_peered":"2024-06-10T00:29:23.780855+0000","last_clean":"2024-06-10T00:29:23.780855+0000","last_became_active":"2024-06-10T00:28:37.652868+0000","last_became_peered":"2024-06-10T00:28:37.652868+0000","last_unstale":"2024-06-10T00:29:23.780855+0000","last_undegraded":"2024-06-10T00:29:23.780855+0000","last_fullsized":"2024-06-10T00:29:23.780855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:21.614915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.775035+0000","last_change":"2024-06-10T00:28:48.727615+0000","last_active":"2024-06-10T00:29:23.775035+0000","last_peered":"2024-06-10T00:29:23.775035+0000","last_clean":"2024-06-10T00:29:23.775035+0000","last_became_active":"2024-06-10T00:28:48.727190+0000","last_became_peered":"2024-06-10T00:28:48.727190+0000","last_unstale":"2024-06-10T00:29:23.775035+0000","last_undegraded":"2024-06-10T00:29:23.775035+0000","last_fullsized":"2024-06-10T00:29:23.775035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:57.498183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":55,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:41.853891+0000","last_change":"2024-06-10T00:29:23.784131+0000","last_active":"2024-06-10T00:29:41.853891+0000","last_peered":"2024-06-10T00:29:41.853891+0000","last_clean":"2024-06-10T00:29:41.853891+0000","last_became_active":"2024-06-10T00:29:23.780909+0000","last_became_peered":"2024-06-10T00:29:23.780909+0000","last_unstale":"2024-06-10T00:29:41.853891+0000","last_undegraded":"2024-06-10T00:29:41.853891+0000","last_fullsized":"2024-06-10T00:29:41.853891+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.774217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777897+0000","last_change":"2024-06-10T00:28:37.660399+0000","last_active":"2024-06-10T00:29:23.777897+0000","last_peered":"2024-06-10T00:29:23.777897+0000","last_clean":"2024-06-10T00:29:23.777897+0000","last_became_active":"2024-06-10T00:28:37.660069+0000","last_became_peered":"2024-06-10T00:28:37.660069+0000","last_unstale":"2024-06-10T00:29:23.777897+0000","last_undegraded":"2024-06-10T00:29:23.777897+0000","last_fullsized":"2024-06-10T00:29:23.777897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:59.727883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854444+0000","last_change":"2024-06-10T00:28:48.713267+0000","last_active":"2024-06-10T00:29:23.854444+0000","last_peered":"2024-06-10T00:29:23.854444+0000","last_clean":"2024-06-10T00:29:23.854444+0000","last_became_active":"2024-06-10T00:28:48.712870+0000","last_became_peered":"2024-06-10T00:28:48.712870+0000","last_unstale":"2024-06-10T00:29:23.854444+0000","last_undegraded":"2024-06-10T00:29:23.854444+0000","last_fullsized":"2024-06-10T00:29:23.854444+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:59.210101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'5","reported_seq":66,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:03.913314+0000","last_change":"2024-06-10T00:28:34.448652+0000","last_active":"2024-06-10T00:30:03.913314+0000","last_peered":"2024-06-10T00:30:03.913314+0000","last_clean":"2024-06-10T00:30:03.913314+0000","last_became_active":"2024-06-10T00:28:34.448243+0000","last_became_peered":"2024-06-10T00:28:34.448243+0000","last_unstale":"2024-06-10T00:30:03.913314+0000","last_undegraded":"2024-06-10T00:30:03.913314+0000","last_fullsized":"2024-06-10T00:30:03.913314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:51:40.003429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.778119+0000","last_change":"2024-06-10T00:28:37.662307+0000","last_active":"2024-06-10T00:29:23.778119+0000","last_peered":"2024-06-10T00:29:23.778119+0000","last_clean":"2024-06-10T00:29:23.778119+0000","last_became_active":"2024-06-10T00:28:37.661727+0000","last_became_peered":"2024-06-10T00:28:37.661727+0000","last_unstale":"2024-06-10T00:29:23.778119+0000","last_undegraded":"2024-06-10T00:29:23.778119+0000","last_fullsized":"2024-06-10T00:29:23.778119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:23.989429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.857626+0000","last_change":"2024-06-10T00:29:23.857626+0000","last_active":"2024-06-10T00:29:23.857626+0000","last_peered":"2024-06-10T00:29:23.857626+0000","last_clean":"2024-06-10T00:29:23.857626+0000","last_became_active":"2024-06-10T00:29:23.857057+0000","last_became_peered":"2024-06-10T00:29:23.857057+0000","last_unstale":"2024-06-10T00:29:23.857626+0000","last_undegraded":"2024-06-10T00:29:23.857626+0000","last_fullsized":"2024-06-10T00:29:23.857626+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:25:10.340053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.881073+0000","last_change":"2024-06-10T00:28:34.399253+0000","last_active":"2024-06-10T00:29:44.881073+0000","last_peered":"2024-06-10T00:29:44.881073+0000","last_clean":"2024-06-10T00:29:44.881073+0000","last_became_active":"2024-06-10T00:28:34.397763+0000","last_became_peered":"2024-06-10T00:28:34.397763+0000","last_unstale":"2024-06-10T00:29:44.881073+0000","last_undegraded":"2024-06-10T00:29:44.881073+0000","last_fullsized":"2024-06-10T00:29:44.881073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:05:57.371625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777468+0000","last_change":"2024-06-10T00:28:37.658027+0000","last_active":"2024-06-10T00:29:23.777468+0000","last_peered":"2024-06-10T00:29:23.777468+0000","last_clean":"2024-06-10T00:29:23.777468+0000","last_became_active":"2024-06-10T00:28:37.655513+0000","last_became_peered":"2024-06-10T00:28:37.655513+0000","last_unstale":"2024-06-10T00:29:23.777468+0000","last_undegraded":"2024-06-10T00:29:23.777468+0000","last_fullsized":"2024-06-10T00:29:23.777468+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:50:23.525855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854462+0000","last_change":"2024-06-10T00:28:48.706997+0000","last_active":"2024-06-10T00:29:23.854462+0000","last_peered":"2024-06-10T00:29:23.854462+0000","last_clean":"2024-06-10T00:29:23.854462+0000","last_became_active":"2024-06-10T00:28:48.706597+0000","last_became_peered":"2024-06-10T00:28:48.706597+0000","last_unstale":"2024-06-10T00:29:23.854462+0000","last_undegraded":"2024-06-10T00:29:23.854462+0000","last_fullsized":"2024-06-10T00:29:23.854462+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:16.385312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.880832+0000","last_change":"2024-06-10T00:29:23.785774+0000","last_active":"2024-06-10T00:29:44.880832+0000","last_peered":"2024-06-10T00:29:44.880832+0000","last_clean":"2024-06-10T00:29:44.880832+0000","last_became_active":"2024-06-10T00:29:23.781768+0000","last_became_peered":"2024-06-10T00:29:23.781768+0000","last_unstale":"2024-06-10T00:29:44.880832+0000","last_undegraded":"2024-06-10T00:29:44.880832+0000","last_fullsized":"2024-06-10T00:29:44.880832+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:27.348264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777755+0000","last_change":"2024-06-10T00:28:37.657820+0000","last_active":"2024-06-10T00:29:23.777755+0000","last_peered":"2024-06-10T00:29:23.777755+0000","last_clean":"2024-06-10T00:29:23.777755+0000","last_became_active":"2024-06-10T00:28:37.657487+0000","last_became_peered":"2024-06-10T00:28:37.657487+0000","last_unstale":"2024-06-10T00:29:23.777755+0000","last_undegraded":"2024-06-10T00:29:23.777755+0000","last_fullsized":"2024-06-10T00:29:23.777755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:44:35.930401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777344+0000","last_change":"2024-06-10T00:28:37.664543+0000","last_active":"2024-06-10T00:29:23.777344+0000","last_peered":"2024-06-10T00:29:23.777344+0000","last_clean":"2024-06-10T00:29:23.777344+0000","last_became_active":"2024-06-10T00:28:37.659015+0000","last_became_peered":"2024-06-10T00:28:37.659015+0000","last_unstale":"2024-06-10T00:29:23.777344+0000","last_undegraded":"2024-06-10T00:29:23.777344+0000","last_fullsized":"2024-06-10T00:29:23.777344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:06.555472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774751+0000","last_change":"2024-06-10T00:28:37.659389+0000","last_active":"2024-06-10T00:29:23.774751+0000","last_peered":"2024-06-10T00:29:23.774751+0000","last_clean":"2024-06-10T00:29:23.774751+0000","last_became_active":"2024-06-10T00:28:37.658990+0000","last_became_peered":"2024-06-10T00:28:37.658990+0000","last_unstale":"2024-06-10T00:29:23.774751+0000","last_undegraded":"2024-06-10T00:29:23.774751+0000","last_fullsized":"2024-06-10T00:29:23.774751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:29.532779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774557+0000","last_change":"2024-06-10T00:28:37.655621+0000","last_active":"2024-06-10T00:29:23.774557+0000","last_peered":"2024-06-10T00:29:23.774557+0000","last_clean":"2024-06-10T00:29:23.774557+0000","last_became_active":"2024-06-10T00:28:37.653599+0000","last_became_peered":"2024-06-10T00:28:37.653599+0000","last_unstale":"2024-06-10T00:29:23.774557+0000","last_undegraded":"2024-06-10T00:29:23.774557+0000","last_fullsized":"2024-06-10T00:29:23.774557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:22.165851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.782727+0000","last_change":"2024-06-10T00:29:23.782727+0000","last_active":"2024-06-10T00:29:23.782727+0000","last_peered":"2024-06-10T00:29:23.782727+0000","last_clean":"2024-06-10T00:29:23.782727+0000","last_became_active":"2024-06-10T00:29:23.782309+0000","last_became_peered":"2024-06-10T00:29:23.782309+0000","last_unstale":"2024-06-10T00:29:23.782727+0000","last_undegraded":"2024-06-10T00:29:23.782727+0000","last_fullsized":"2024-06-10T00:29:23.782727+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:20.585955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777419+0000","last_change":"2024-06-10T00:28:37.646750+0000","last_active":"2024-06-10T00:29:23.777419+0000","last_peered":"2024-06-10T00:29:23.777419+0000","last_clean":"2024-06-10T00:29:23.777419+0000","last_became_active":"2024-06-10T00:28:37.646372+0000","last_became_peered":"2024-06-10T00:28:37.646372+0000","last_unstale":"2024-06-10T00:29:23.777419+0000","last_undegraded":"2024-06-10T00:29:23.777419+0000","last_fullsized":"2024-06-10T00:29:23.777419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:45.762956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779106+0000","last_change":"2024-06-10T00:28:37.662278+0000","last_active":"2024-06-10T00:29:23.779106+0000","last_peered":"2024-06-10T00:29:23.779106+0000","last_clean":"2024-06-10T00:29:23.779106+0000","last_became_active":"2024-06-10T00:28:37.661855+0000","last_became_peered":"2024-06-10T00:28:37.661855+0000","last_unstale":"2024-06-10T00:29:23.779106+0000","last_undegraded":"2024-06-10T00:29:23.779106+0000","last_fullsized":"2024-06-10T00:29:23.779106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:20.074208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.775489+0000","last_change":"2024-06-10T00:28:37.656738+0000","last_active":"2024-06-10T00:29:23.775489+0000","last_peered":"2024-06-10T00:29:23.775489+0000","last_clean":"2024-06-10T00:29:23.775489+0000","last_became_active":"2024-06-10T00:28:37.656375+0000","last_became_peered":"2024-06-10T00:28:37.656375+0000","last_unstale":"2024-06-10T00:29:23.775489+0000","last_undegraded":"2024-06-10T00:29:23.775489+0000","last_fullsized":"2024-06-10T00:29:23.775489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:49:17.779670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780920+0000","last_change":"2024-06-10T00:28:37.656849+0000","last_active":"2024-06-10T00:29:23.780920+0000","last_peered":"2024-06-10T00:29:23.780920+0000","last_clean":"2024-06-10T00:29:23.780920+0000","last_became_active":"2024-06-10T00:28:37.656261+0000","last_became_peered":"2024-06-10T00:28:37.656261+0000","last_unstale":"2024-06-10T00:29:23.780920+0000","last_undegraded":"2024-06-10T00:29:23.780920+0000","last_fullsized":"2024-06-10T00:29:23.780920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:13.152165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854724+0000","last_change":"2024-06-10T00:28:37.648384+0000","last_active":"2024-06-10T00:29:23.854724+0000","last_peered":"2024-06-10T00:29:23.854724+0000","last_clean":"2024-06-10T00:29:23.854724+0000","last_became_active":"2024-06-10T00:28:37.648009+0000","last_became_peered":"2024-06-10T00:28:37.648009+0000","last_unstale":"2024-06-10T00:29:23.854724+0000","last_undegraded":"2024-06-10T00:29:23.854724+0000","last_fullsized":"2024-06-10T00:29:23.854724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:45:55.808932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780879+0000","last_change":"2024-06-10T00:28:37.655828+0000","last_active":"2024-06-10T00:29:23.780879+0000","last_peered":"2024-06-10T00:29:23.780879+0000","last_clean":"2024-06-10T00:29:23.780879+0000","last_became_active":"2024-06-10T00:28:37.655269+0000","last_became_peered":"2024-06-10T00:28:37.655269+0000","last_unstale":"2024-06-10T00:29:23.780879+0000","last_undegraded":"2024-06-10T00:29:23.780879+0000","last_fullsized":"2024-06-10T00:29:23.780879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:56.618469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855445+0000","last_change":"2024-06-10T00:28:37.663756+0000","last_active":"2024-06-10T00:29:23.855445+0000","last_peered":"2024-06-10T00:29:23.855445+0000","last_clean":"2024-06-10T00:29:23.855445+0000","last_became_active":"2024-06-10T00:28:37.663380+0000","last_became_peered":"2024-06-10T00:28:37.663380+0000","last_unstale":"2024-06-10T00:29:23.855445+0000","last_undegraded":"2024-06-10T00:29:23.855445+0000","last_fullsized":"2024-06-10T00:29:23.855445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:45.534648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777983+0000","last_change":"2024-06-10T00:28:37.655246+0000","last_active":"2024-06-10T00:29:23.777983+0000","last_peered":"2024-06-10T00:29:23.777983+0000","last_clean":"2024-06-10T00:29:23.777983+0000","last_became_active":"2024-06-10T00:28:37.654605+0000","last_became_peered":"2024-06-10T00:28:37.654605+0000","last_unstale":"2024-06-10T00:29:23.777983+0000","last_undegraded":"2024-06-10T00:29:23.777983+0000","last_fullsized":"2024-06-10T00:29:23.777983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:15.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779394+0000","last_change":"2024-06-10T00:28:37.662485+0000","last_active":"2024-06-10T00:29:23.779394+0000","last_peered":"2024-06-10T00:29:23.779394+0000","last_clean":"2024-06-10T00:29:23.779394+0000","last_became_active":"2024-06-10T00:28:37.661900+0000","last_became_peered":"2024-06-10T00:28:37.661900+0000","last_unstale":"2024-06-10T00:29:23.779394+0000","last_undegraded":"2024-06-10T00:29:23.779394+0000","last_fullsized":"2024-06-10T00:29:23.779394+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:30:03.210634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65592,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":260,"num_read_kb":206,"num_write":245,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":195638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1126400,"data_stored":551803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28780,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716372,"statfs":{"total":95995035648,"available":95965564928,"internally_reserved":0,"allocated":1806336,"data_stored":1197197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":555627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":551125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1732608,"data_stored":1148795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":547035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1187840,"data_stored":607127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":548491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36504,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708648,"statfs":{"total":95995035648,"available":95957655552,"internally_reserved":0,"allocated":1196032,"data_stored":605927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":546743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":551499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28712,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716440,"statfs":{"total":95995035648,"available":95965634560,"internally_reserved":0,"allocated":1736704,"data_stored":1143897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:30:12.430 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T00:30:12.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:12 smithi105 ceph-mon[36321]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:30:12.686 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:13.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:13 smithi062 ceph-mon[32390]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-06-10T00:30:13.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:13 smithi138 ceph-mon[36549]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-06-10T00:30:13.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:13 smithi105 ceph-mon[36321]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-06-10T00:30:14.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.588+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/3960169922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e18107400 msgr2=0x7f0e181077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:14.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.588+0000 7f0e1fe9f700 1 --2- 172.21.15.62:0/3960169922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e18107400 0x7f0e181077e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009990 tx=0x7f0e14009ca0 comp rx=0 tx=0).stop 2024-06-10T00:30:14.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.588+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/3960169922 shutdown_connections 2024-06-10T00:30:14.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.588+0000 7f0e1fe9f700 1 --2- 172.21.15.62:0/3960169922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e18107400 0x7f0e181077e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:14.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.588+0000 7f0e1fe9f700 1 --2- 172.21.15.62:0/3960169922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e18101d50 0x7f0e1810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:14.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.588+0000 7f0e1fe9f700 1 --2- 172.21.15.62:0/3960169922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e18101390 0x7f0e18101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:14.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.588+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/3960169922 >> 172.21.15.62:0/3960169922 conn(0x7f0e18078340 msgr2=0x7f0e18078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:14.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.589+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/3960169922 shutdown_connections 2024-06-10T00:30:14.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.589+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/3960169922 wait complete. 2024-06-10T00:30:14.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.589+0000 7f0e1fe9f700 1 Processor -- start 2024-06-10T00:30:14.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.589+0000 7f0e1fe9f700 1 -- start start 2024-06-10T00:30:14.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.589+0000 7f0e1fe9f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e18101390 0x7f0e18199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.589+0000 7f0e1fe9f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e18101d50 0x7f0e18199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.589+0000 7f0e1fe9f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e18107400 0x7f0e181a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.589+0000 7f0e1fe9f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0e18111670 con 0x7f0e18101d50 2024-06-10T00:30:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.589+0000 7f0e1fe9f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181114f0 con 0x7f0e18107400 2024-06-10T00:30:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.589+0000 7f0e1fe9f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181117f0 con 0x7f0e18101390 2024-06-10T00:30:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e1d43a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e18101d50 0x7f0e18199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e1d43a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e18101d50 0x7f0e18199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46504/0 (socket says 172.21.15.62:46504) 2024-06-10T00:30:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e1d43a700 1 -- 172.21.15.62:0/700633027 learned_addr learned my addr 172.21.15.62:0/700633027 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e1dc3b700 1 --2- 172.21.15.62:0/700633027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e18101390 0x7f0e18199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e1e43c700 1 --2- 172.21.15.62:0/700633027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e18107400 0x7f0e181a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e1d43a700 1 -- 172.21.15.62:0/700633027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e18101390 msgr2=0x7f0e18199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:14.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e1d43a700 1 --2- 172.21.15.62:0/700633027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e18101390 0x7f0e18199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e1d43a700 1 -- 172.21.15.62:0/700633027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e18107400 msgr2=0x7f0e181a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e1d43a700 1 --2- 172.21.15.62:0/700633027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e18107400 0x7f0e181a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e1d43a700 1 -- 172.21.15.62:0/700633027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e181a4ff0 con 0x7f0e18101d50 2024-06-10T00:30:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e1d43a700 1 --2- 172.21.15.62:0/700633027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e18101d50 0x7f0e18199f70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800d950 tx=0x7f0e0800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e0effd700 1 -- 172.21.15.62:0/700633027 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08016070 con 0x7f0e18101d50 2024-06-10T00:30:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/700633027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e181a5220 con 0x7f0e18101d50 2024-06-10T00:30:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e0effd700 1 -- 172.21.15.62:0/700633027 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e0800fd70 con 0x7f0e18101d50 2024-06-10T00:30:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e0effd700 1 -- 172.21.15.62:0/700633027 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0800e610 con 0x7f0e18101d50 2024-06-10T00:30:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.590+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/700633027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e181a57b0 con 0x7f0e18101d50 2024-06-10T00:30:14.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.591+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/700633027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e18069180 con 0x7f0e18101d50 2024-06-10T00:30:14.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.593+0000 7f0e0effd700 1 -- 172.21.15.62:0/700633027 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0e08019070 con 0x7f0e18101d50 2024-06-10T00:30:14.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.593+0000 7f0e0effd700 1 --2- 172.21.15.62:0/700633027 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0e040a2cb0 0x7f0e040a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:14.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.593+0000 7f0e1dc3b700 1 --2- 172.21.15.62:0/700633027 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0e040a2cb0 0x7f0e040a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:14.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.593+0000 7f0e0effd700 1 -- 172.21.15.62:0/700633027 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f0e080cefc0 con 0x7f0e18101d50 2024-06-10T00:30:14.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.594+0000 7f0e1dc3b700 1 --2- 172.21.15.62:0/700633027 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0e040a2cb0 0x7f0e040a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400b5c0 tx=0x7f0e14005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:14.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.598+0000 7f0e0effd700 1 -- 172.21.15.62:0/700633027 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0e0809ba60 con 0x7f0e18101d50 2024-06-10T00:30:14.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.799+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/700633027 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e180628d0 con 0x7f0e040a2cb0 2024-06-10T00:30:14.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.808+0000 7f0e0effd700 1 -- 172.21.15.62:0/700633027 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340417 (secure 0 0 0) 0x7f0e180628d0 con 0x7f0e040a2cb0 2024-06-10T00:30:14.812 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:30:14.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.811+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/700633027 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0e040a2cb0 msgr2=0x7f0e040a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.811+0000 7f0e1fe9f700 1 --2- 172.21.15.62:0/700633027 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0e040a2cb0 0x7f0e040a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400b5c0 tx=0x7f0e14005960 comp rx=0 tx=0).stop 2024-06-10T00:30:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.811+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/700633027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e18101d50 msgr2=0x7f0e18199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.811+0000 7f0e1fe9f700 1 --2- 172.21.15.62:0/700633027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e18101d50 0x7f0e18199f70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800d950 tx=0x7f0e0800dc60 comp rx=0 tx=0).stop 2024-06-10T00:30:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.811+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/700633027 shutdown_connections 2024-06-10T00:30:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.811+0000 7f0e1fe9f700 1 --2- 172.21.15.62:0/700633027 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0e040a2cb0 0x7f0e040a5170 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.811+0000 7f0e1fe9f700 1 --2- 172.21.15.62:0/700633027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e18107400 0x7f0e181a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.811+0000 7f0e1fe9f700 1 --2- 172.21.15.62:0/700633027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e18101390 0x7f0e18199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.811+0000 7f0e1fe9f700 1 --2- 172.21.15.62:0/700633027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e18101d50 0x7f0e18199f70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.811+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/700633027 >> 172.21.15.62:0/700633027 conn(0x7f0e18078340 msgr2=0x7f0e180fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.812+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/700633027 shutdown_connections 2024-06-10T00:30:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:14.812+0000 7f0e1fe9f700 1 -- 172.21.15.62:0/700633027 wait complete. 2024-06-10T00:30:14.815 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-06-10T00:30:15.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:15 smithi062 ceph-mon[32390]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-06-10T00:30:15.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:15 smithi062 ceph-mon[32390]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:30:15.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:15 smithi138 ceph-mon[36549]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-06-10T00:30:15.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:15 smithi138 ceph-mon[36549]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:30:15.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:15 smithi105 ceph-mon[36321]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-06-10T00:30:15.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:15 smithi105 ceph-mon[36321]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:30:15.594 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-06-10T00:30:13.961723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660459,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":447,"num_read_kb":355,"num_write":431,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":352,"ondisk_log_size":352,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380988,"kb_used_data":15196,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560836,"statfs":{"total":1151940427776,"available":1151550296064,"internally_reserved":0,"allocated":15560704,"data_stored":8564934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32429,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":134,"num_read_kb":104,"num_write":22,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003433"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780586+0000","last_change":"2024-06-10T00:28:37.657234+0000","last_active":"2024-06-10T00:29:23.780586+0000","last_peered":"2024-06-10T00:29:23.780586+0000","last_clean":"2024-06-10T00:29:23.780586+0000","last_became_active":"2024-06-10T00:28:37.656879+0000","last_became_peered":"2024-06-10T00:28:37.656879+0000","last_unstale":"2024-06-10T00:29:23.780586+0000","last_undegraded":"2024-06-10T00:29:23.780586+0000","last_fullsized":"2024-06-10T00:29:23.780586+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:01:02.907597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777728+0000","last_change":"2024-06-10T00:28:37.656440+0000","last_active":"2024-06-10T00:29:23.777728+0000","last_peered":"2024-06-10T00:29:23.777728+0000","last_clean":"2024-06-10T00:29:23.777728+0000","last_became_active":"2024-06-10T00:28:37.656113+0000","last_became_peered":"2024-06-10T00:28:37.656113+0000","last_unstale":"2024-06-10T00:29:23.777728+0000","last_undegraded":"2024-06-10T00:29:23.777728+0000","last_fullsized":"2024-06-10T00:29:23.777728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:49:31.447951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777164+0000","last_change":"2024-06-10T00:28:37.659114+0000","last_active":"2024-06-10T00:29:23.777164+0000","last_peered":"2024-06-10T00:29:23.777164+0000","last_clean":"2024-06-10T00:29:23.777164+0000","last_became_active":"2024-06-10T00:28:37.658763+0000","last_became_peered":"2024-06-10T00:28:37.658763+0000","last_unstale":"2024-06-10T00:29:23.777164+0000","last_undegraded":"2024-06-10T00:29:23.777164+0000","last_fullsized":"2024-06-10T00:29:23.777164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:39:54.963590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.775384+0000","last_change":"2024-06-10T00:28:37.656786+0000","last_active":"2024-06-10T00:29:23.775384+0000","last_peered":"2024-06-10T00:29:23.775384+0000","last_clean":"2024-06-10T00:29:23.775384+0000","last_became_active":"2024-06-10T00:28:37.656073+0000","last_became_peered":"2024-06-10T00:28:37.656073+0000","last_unstale":"2024-06-10T00:29:23.775384+0000","last_undegraded":"2024-06-10T00:29:23.775384+0000","last_fullsized":"2024-06-10T00:29:23.775384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:19.176608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779481+0000","last_change":"2024-06-10T00:28:37.650250+0000","last_active":"2024-06-10T00:29:23.779481+0000","last_peered":"2024-06-10T00:29:23.779481+0000","last_clean":"2024-06-10T00:29:23.779481+0000","last_became_active":"2024-06-10T00:28:37.649852+0000","last_became_peered":"2024-06-10T00:28:37.649852+0000","last_unstale":"2024-06-10T00:29:23.779481+0000","last_undegraded":"2024-06-10T00:29:23.779481+0000","last_fullsized":"2024-06-10T00:29:23.779481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:39.492884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.856618+0000","last_change":"2024-06-10T00:28:37.646042+0000","last_active":"2024-06-10T00:29:23.856618+0000","last_peered":"2024-06-10T00:29:23.856618+0000","last_clean":"2024-06-10T00:29:23.856618+0000","last_became_active":"2024-06-10T00:28:37.644591+0000","last_became_peered":"2024-06-10T00:28:37.644591+0000","last_unstale":"2024-06-10T00:29:23.856618+0000","last_undegraded":"2024-06-10T00:29:23.856618+0000","last_fullsized":"2024-06-10T00:29:23.856618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:29.932303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777517+0000","last_change":"2024-06-10T00:28:37.664076+0000","last_active":"2024-06-10T00:29:23.777517+0000","last_peered":"2024-06-10T00:29:23.777517+0000","last_clean":"2024-06-10T00:29:23.777517+0000","last_became_active":"2024-06-10T00:28:37.663741+0000","last_became_peered":"2024-06-10T00:28:37.663741+0000","last_unstale":"2024-06-10T00:29:23.777517+0000","last_undegraded":"2024-06-10T00:29:23.777517+0000","last_fullsized":"2024-06-10T00:29:23.777517+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:14.581409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779934+0000","last_change":"2024-06-10T00:28:37.648758+0000","last_active":"2024-06-10T00:29:23.779934+0000","last_peered":"2024-06-10T00:29:23.779934+0000","last_clean":"2024-06-10T00:29:23.779934+0000","last_became_active":"2024-06-10T00:28:37.648387+0000","last_became_peered":"2024-06-10T00:28:37.648387+0000","last_unstale":"2024-06-10T00:29:23.779934+0000","last_undegraded":"2024-06-10T00:29:23.779934+0000","last_fullsized":"2024-06-10T00:29:23.779934+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:17.187265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855175+0000","last_change":"2024-06-10T00:28:37.664650+0000","last_active":"2024-06-10T00:29:23.855175+0000","last_peered":"2024-06-10T00:29:23.855175+0000","last_clean":"2024-06-10T00:29:23.855175+0000","last_became_active":"2024-06-10T00:28:37.663178+0000","last_became_peered":"2024-06-10T00:28:37.663178+0000","last_unstale":"2024-06-10T00:29:23.855175+0000","last_undegraded":"2024-06-10T00:29:23.855175+0000","last_fullsized":"2024-06-10T00:29:23.855175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:46.032078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.856399+0000","last_change":"2024-06-10T00:28:37.658353+0000","last_active":"2024-06-10T00:29:23.856399+0000","last_peered":"2024-06-10T00:29:23.856399+0000","last_clean":"2024-06-10T00:29:23.856399+0000","last_became_active":"2024-06-10T00:28:37.657961+0000","last_became_peered":"2024-06-10T00:28:37.657961+0000","last_unstale":"2024-06-10T00:29:23.856399+0000","last_undegraded":"2024-06-10T00:29:23.856399+0000","last_fullsized":"2024-06-10T00:29:23.856399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:22:27.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777983+0000","last_change":"2024-06-10T00:28:37.663823+0000","last_active":"2024-06-10T00:29:23.777983+0000","last_peered":"2024-06-10T00:29:23.777983+0000","last_clean":"2024-06-10T00:29:23.777983+0000","last_became_active":"2024-06-10T00:28:37.663499+0000","last_became_peered":"2024-06-10T00:28:37.663499+0000","last_unstale":"2024-06-10T00:29:23.777983+0000","last_undegraded":"2024-06-10T00:29:23.777983+0000","last_fullsized":"2024-06-10T00:29:23.777983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.100923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.775337+0000","last_change":"2024-06-10T00:28:37.655424+0000","last_active":"2024-06-10T00:29:23.775337+0000","last_peered":"2024-06-10T00:29:23.775337+0000","last_clean":"2024-06-10T00:29:23.775337+0000","last_became_active":"2024-06-10T00:28:37.655059+0000","last_became_peered":"2024-06-10T00:28:37.655059+0000","last_unstale":"2024-06-10T00:29:23.775337+0000","last_undegraded":"2024-06-10T00:29:23.775337+0000","last_fullsized":"2024-06-10T00:29:23.775337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:35:04.631794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777872+0000","last_change":"2024-06-10T00:28:37.649997+0000","last_active":"2024-06-10T00:29:23.777872+0000","last_peered":"2024-06-10T00:29:23.777872+0000","last_clean":"2024-06-10T00:29:23.777872+0000","last_became_active":"2024-06-10T00:28:37.649494+0000","last_became_peered":"2024-06-10T00:28:37.649494+0000","last_unstale":"2024-06-10T00:29:23.777872+0000","last_undegraded":"2024-06-10T00:29:23.777872+0000","last_fullsized":"2024-06-10T00:29:23.777872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:44:07.467206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.776851+0000","last_change":"2024-06-10T00:28:37.653064+0000","last_active":"2024-06-10T00:29:23.776851+0000","last_peered":"2024-06-10T00:29:23.776851+0000","last_clean":"2024-06-10T00:29:23.776851+0000","last_became_active":"2024-06-10T00:28:37.652684+0000","last_became_peered":"2024-06-10T00:28:37.652684+0000","last_unstale":"2024-06-10T00:29:23.776851+0000","last_undegraded":"2024-06-10T00:29:23.776851+0000","last_fullsized":"2024-06-10T00:29:23.776851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:30.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777473+0000","last_change":"2024-06-10T00:28:37.655570+0000","last_active":"2024-06-10T00:29:23.777473+0000","last_peered":"2024-06-10T00:29:23.777473+0000","last_clean":"2024-06-10T00:29:23.777473+0000","last_became_active":"2024-06-10T00:28:37.655241+0000","last_became_peered":"2024-06-10T00:28:37.655241+0000","last_unstale":"2024-06-10T00:29:23.777473+0000","last_undegraded":"2024-06-10T00:29:23.777473+0000","last_fullsized":"2024-06-10T00:29:23.777473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:07:03.543064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854322+0000","last_change":"2024-06-10T00:28:37.665607+0000","last_active":"2024-06-10T00:29:23.854322+0000","last_peered":"2024-06-10T00:29:23.854322+0000","last_clean":"2024-06-10T00:29:23.854322+0000","last_became_active":"2024-06-10T00:28:37.664943+0000","last_became_peered":"2024-06-10T00:28:37.664943+0000","last_unstale":"2024-06-10T00:29:23.854322+0000","last_undegraded":"2024-06-10T00:29:23.854322+0000","last_fullsized":"2024-06-10T00:29:23.854322+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:43:02.940298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780365+0000","last_change":"2024-06-10T00:28:37.658214+0000","last_active":"2024-06-10T00:29:23.780365+0000","last_peered":"2024-06-10T00:29:23.780365+0000","last_clean":"2024-06-10T00:29:23.780365+0000","last_became_active":"2024-06-10T00:28:37.657295+0000","last_became_peered":"2024-06-10T00:28:37.657295+0000","last_unstale":"2024-06-10T00:29:23.780365+0000","last_undegraded":"2024-06-10T00:29:23.780365+0000","last_fullsized":"2024-06-10T00:29:23.780365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:20.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774705+0000","last_change":"2024-06-10T00:28:37.656645+0000","last_active":"2024-06-10T00:29:23.774705+0000","last_peered":"2024-06-10T00:29:23.774705+0000","last_clean":"2024-06-10T00:29:23.774705+0000","last_became_active":"2024-06-10T00:28:37.656157+0000","last_became_peered":"2024-06-10T00:28:37.656157+0000","last_unstale":"2024-06-10T00:29:23.774705+0000","last_undegraded":"2024-06-10T00:29:23.774705+0000","last_fullsized":"2024-06-10T00:29:23.774705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:00.953804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.775389+0000","last_change":"2024-06-10T00:28:37.664336+0000","last_active":"2024-06-10T00:29:23.775389+0000","last_peered":"2024-06-10T00:29:23.775389+0000","last_clean":"2024-06-10T00:29:23.775389+0000","last_became_active":"2024-06-10T00:28:37.663960+0000","last_became_peered":"2024-06-10T00:28:37.663960+0000","last_unstale":"2024-06-10T00:29:23.775389+0000","last_undegraded":"2024-06-10T00:29:23.775389+0000","last_fullsized":"2024-06-10T00:29:23.775389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:59:32.774442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774631+0000","last_change":"2024-06-10T00:28:48.705930+0000","last_active":"2024-06-10T00:29:23.774631+0000","last_peered":"2024-06-10T00:29:23.774631+0000","last_clean":"2024-06-10T00:29:23.774631+0000","last_became_active":"2024-06-10T00:28:48.705558+0000","last_became_peered":"2024-06-10T00:28:48.705558+0000","last_unstale":"2024-06-10T00:29:23.774631+0000","last_undegraded":"2024-06-10T00:29:23.774631+0000","last_fullsized":"2024-06-10T00:29:23.774631+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:41:44.914378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'13","reported_seq":94,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:11.922027+0000","last_change":"2024-06-10T00:29:23.862141+0000","last_active":"2024-06-10T00:30:11.922027+0000","last_peered":"2024-06-10T00:30:11.922027+0000","last_clean":"2024-06-10T00:30:11.922027+0000","last_became_active":"2024-06-10T00:29:23.857051+0000","last_became_peered":"2024-06-10T00:29:23.857051+0000","last_unstale":"2024-06-10T00:30:11.922027+0000","last_undegraded":"2024-06-10T00:30:11.922027+0000","last_fullsized":"2024-06-10T00:30:11.922027+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:41:41.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779865+0000","last_change":"2024-06-10T00:28:37.653938+0000","last_active":"2024-06-10T00:29:23.779865+0000","last_peered":"2024-06-10T00:29:23.779865+0000","last_clean":"2024-06-10T00:29:23.779865+0000","last_became_active":"2024-06-10T00:28:37.653629+0000","last_became_peered":"2024-06-10T00:28:37.653629+0000","last_unstale":"2024-06-10T00:29:23.779865+0000","last_undegraded":"2024-06-10T00:29:23.779865+0000","last_fullsized":"2024-06-10T00:29:23.779865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:21:39.285961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774474+0000","last_change":"2024-06-10T00:28:48.704413+0000","last_active":"2024-06-10T00:29:23.774474+0000","last_peered":"2024-06-10T00:29:23.774474+0000","last_clean":"2024-06-10T00:29:23.774474+0000","last_became_active":"2024-06-10T00:28:48.703972+0000","last_became_peered":"2024-06-10T00:28:48.703972+0000","last_unstale":"2024-06-10T00:29:23.774474+0000","last_undegraded":"2024-06-10T00:29:23.774474+0000","last_fullsized":"2024-06-10T00:29:23.774474+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:51.927435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780310+0000","last_change":"2024-06-10T00:28:34.445413+0000","last_active":"2024-06-10T00:29:23.780310+0000","last_peered":"2024-06-10T00:29:23.780310+0000","last_clean":"2024-06-10T00:29:23.780310+0000","last_became_active":"2024-06-10T00:28:34.445077+0000","last_became_peered":"2024-06-10T00:28:34.445077+0000","last_unstale":"2024-06-10T00:29:23.780310+0000","last_undegraded":"2024-06-10T00:29:23.780310+0000","last_fullsized":"2024-06-10T00:29:23.780310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:19:19.535976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777738+0000","last_change":"2024-06-10T00:28:37.657123+0000","last_active":"2024-06-10T00:29:23.777738+0000","last_peered":"2024-06-10T00:29:23.777738+0000","last_clean":"2024-06-10T00:29:23.777738+0000","last_became_active":"2024-06-10T00:28:37.656791+0000","last_became_peered":"2024-06-10T00:28:37.656791+0000","last_unstale":"2024-06-10T00:29:23.777738+0000","last_undegraded":"2024-06-10T00:29:23.777738+0000","last_fullsized":"2024-06-10T00:29:23.777738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:11.102478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774581+0000","last_change":"2024-06-10T00:28:48.705989+0000","last_active":"2024-06-10T00:29:23.774581+0000","last_peered":"2024-06-10T00:29:23.774581+0000","last_clean":"2024-06-10T00:29:23.774581+0000","last_became_active":"2024-06-10T00:28:48.705555+0000","last_became_peered":"2024-06-10T00:28:48.705555+0000","last_unstale":"2024-06-10T00:29:23.774581+0000","last_undegraded":"2024-06-10T00:29:23.774581+0000","last_fullsized":"2024-06-10T00:29:23.774581+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:36.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'12","reported_seq":53,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:45.878023+0000","last_change":"2024-06-10T00:28:34.400117+0000","last_active":"2024-06-10T00:29:45.878023+0000","last_peered":"2024-06-10T00:29:45.878023+0000","last_clean":"2024-06-10T00:29:45.878023+0000","last_became_active":"2024-06-10T00:28:34.399743+0000","last_became_peered":"2024-06-10T00:28:34.399743+0000","last_unstale":"2024-06-10T00:29:45.878023+0000","last_undegraded":"2024-06-10T00:29:45.878023+0000","last_fullsized":"2024-06-10T00:29:45.878023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:20:57.580146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854581+0000","last_change":"2024-06-10T00:28:37.664005+0000","last_active":"2024-06-10T00:29:23.854581+0000","last_peered":"2024-06-10T00:29:23.854581+0000","last_clean":"2024-06-10T00:29:23.854581+0000","last_became_active":"2024-06-10T00:28:37.661297+0000","last_became_peered":"2024-06-10T00:28:37.661297+0000","last_unstale":"2024-06-10T00:29:23.854581+0000","last_undegraded":"2024-06-10T00:29:23.854581+0000","last_fullsized":"2024-06-10T00:29:23.854581+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:57.795080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.859091+0000","last_change":"2024-06-10T00:29:23.859091+0000","last_active":"2024-06-10T00:29:23.859091+0000","last_peered":"2024-06-10T00:29:23.859091+0000","last_clean":"2024-06-10T00:29:23.859091+0000","last_became_active":"2024-06-10T00:29:23.858485+0000","last_became_peered":"2024-06-10T00:29:23.858485+0000","last_unstale":"2024-06-10T00:29:23.859091+0000","last_undegraded":"2024-06-10T00:29:23.859091+0000","last_fullsized":"2024-06-10T00:29:23.859091+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:07:10.925323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.884067+0000","last_change":"2024-06-10T00:28:34.400588+0000","last_active":"2024-06-10T00:29:44.884067+0000","last_peered":"2024-06-10T00:29:44.884067+0000","last_clean":"2024-06-10T00:29:44.884067+0000","last_became_active":"2024-06-10T00:28:34.400027+0000","last_became_peered":"2024-06-10T00:28:34.400027+0000","last_unstale":"2024-06-10T00:29:44.884067+0000","last_undegraded":"2024-06-10T00:29:44.884067+0000","last_fullsized":"2024-06-10T00:29:44.884067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:25:59.890076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855404+0000","last_change":"2024-06-10T00:28:37.659401+0000","last_active":"2024-06-10T00:29:23.855404+0000","last_peered":"2024-06-10T00:29:23.855404+0000","last_clean":"2024-06-10T00:29:23.855404+0000","last_became_active":"2024-06-10T00:28:37.659056+0000","last_became_peered":"2024-06-10T00:28:37.659056+0000","last_unstale":"2024-06-10T00:29:23.855404+0000","last_undegraded":"2024-06-10T00:29:23.855404+0000","last_fullsized":"2024-06-10T00:29:23.855404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:35:01.415630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.775032+0000","last_change":"2024-06-10T00:28:48.702961+0000","last_active":"2024-06-10T00:29:23.775032+0000","last_peered":"2024-06-10T00:29:23.775032+0000","last_clean":"2024-06-10T00:29:23.775032+0000","last_became_active":"2024-06-10T00:28:48.702502+0000","last_became_peered":"2024-06-10T00:28:48.702502+0000","last_unstale":"2024-06-10T00:29:23.775032+0000","last_undegraded":"2024-06-10T00:29:23.775032+0000","last_fullsized":"2024-06-10T00:29:23.775032+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:54.881595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'6","reported_seq":74,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:09.921520+0000","last_change":"2024-06-10T00:28:34.396310+0000","last_active":"2024-06-10T00:30:09.921520+0000","last_peered":"2024-06-10T00:30:09.921520+0000","last_clean":"2024-06-10T00:30:09.921520+0000","last_became_active":"2024-06-10T00:28:34.395401+0000","last_became_peered":"2024-06-10T00:28:34.395401+0000","last_unstale":"2024-06-10T00:30:09.921520+0000","last_undegraded":"2024-06-10T00:30:09.921520+0000","last_fullsized":"2024-06-10T00:30:09.921520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:33:54.300308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.778708+0000","last_change":"2024-06-10T00:28:37.651427+0000","last_active":"2024-06-10T00:29:23.778708+0000","last_peered":"2024-06-10T00:29:23.778708+0000","last_clean":"2024-06-10T00:29:23.778708+0000","last_became_active":"2024-06-10T00:28:37.650510+0000","last_became_peered":"2024-06-10T00:28:37.650510+0000","last_unstale":"2024-06-10T00:29:23.778708+0000","last_undegraded":"2024-06-10T00:29:23.778708+0000","last_fullsized":"2024-06-10T00:29:23.778708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:42.953919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774776+0000","last_change":"2024-06-10T00:28:48.705918+0000","last_active":"2024-06-10T00:29:23.774776+0000","last_peered":"2024-06-10T00:29:23.774776+0000","last_clean":"2024-06-10T00:29:23.774776+0000","last_became_active":"2024-06-10T00:28:48.705462+0000","last_became_peered":"2024-06-10T00:28:48.705462+0000","last_unstale":"2024-06-10T00:29:23.774776+0000","last_undegraded":"2024-06-10T00:29:23.774776+0000","last_fullsized":"2024-06-10T00:29:23.774776+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:59:23.176511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:43.873216+0000","last_change":"2024-06-10T00:28:34.393199+0000","last_active":"2024-06-10T00:29:43.873216+0000","last_peered":"2024-06-10T00:29:43.873216+0000","last_clean":"2024-06-10T00:29:43.873216+0000","last_became_active":"2024-06-10T00:28:34.392777+0000","last_became_peered":"2024-06-10T00:28:34.392777+0000","last_unstale":"2024-06-10T00:29:43.873216+0000","last_undegraded":"2024-06-10T00:29:43.873216+0000","last_fullsized":"2024-06-10T00:29:43.873216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:29.614242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855186+0000","last_change":"2024-06-10T00:28:37.647732+0000","last_active":"2024-06-10T00:29:23.855186+0000","last_peered":"2024-06-10T00:29:23.855186+0000","last_clean":"2024-06-10T00:29:23.855186+0000","last_became_active":"2024-06-10T00:28:37.647391+0000","last_became_peered":"2024-06-10T00:28:37.647391+0000","last_unstale":"2024-06-10T00:29:23.855186+0000","last_undegraded":"2024-06-10T00:29:23.855186+0000","last_fullsized":"2024-06-10T00:29:23.855186+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:05:25.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780971+0000","last_change":"2024-06-10T00:29:23.780971+0000","last_active":"2024-06-10T00:29:23.780971+0000","last_peered":"2024-06-10T00:29:23.780971+0000","last_clean":"2024-06-10T00:29:23.780971+0000","last_became_active":"2024-06-10T00:29:23.780034+0000","last_became_peered":"2024-06-10T00:29:23.780034+0000","last_unstale":"2024-06-10T00:29:23.780971+0000","last_undegraded":"2024-06-10T00:29:23.780971+0000","last_fullsized":"2024-06-10T00:29:23.780971+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:47:06.444181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:42.861995+0000","last_change":"2024-06-10T00:28:34.392486+0000","last_active":"2024-06-10T00:29:42.861995+0000","last_peered":"2024-06-10T00:29:42.861995+0000","last_clean":"2024-06-10T00:29:42.861995+0000","last_became_active":"2024-06-10T00:28:34.392141+0000","last_became_peered":"2024-06-10T00:28:34.392141+0000","last_unstale":"2024-06-10T00:29:42.861995+0000","last_undegraded":"2024-06-10T00:29:42.861995+0000","last_fullsized":"2024-06-10T00:29:42.861995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:10:01.534195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779823+0000","last_change":"2024-06-10T00:28:37.651033+0000","last_active":"2024-06-10T00:29:23.779823+0000","last_peered":"2024-06-10T00:29:23.779823+0000","last_clean":"2024-06-10T00:29:23.779823+0000","last_became_active":"2024-06-10T00:28:37.650505+0000","last_became_peered":"2024-06-10T00:28:37.650505+0000","last_unstale":"2024-06-10T00:29:23.779823+0000","last_undegraded":"2024-06-10T00:29:23.779823+0000","last_fullsized":"2024-06-10T00:29:23.779823+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:54.965865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777647+0000","last_change":"2024-06-10T00:28:48.710748+0000","last_active":"2024-06-10T00:29:23.777647+0000","last_peered":"2024-06-10T00:29:23.777647+0000","last_clean":"2024-06-10T00:29:23.777647+0000","last_became_active":"2024-06-10T00:28:48.710353+0000","last_became_peered":"2024-06-10T00:28:48.710353+0000","last_unstale":"2024-06-10T00:29:23.777647+0000","last_undegraded":"2024-06-10T00:29:23.777647+0000","last_fullsized":"2024-06-10T00:29:23.777647+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:30:47.546650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.882907+0000","last_change":"2024-06-10T00:28:34.398911+0000","last_active":"2024-06-10T00:29:44.882907+0000","last_peered":"2024-06-10T00:29:44.882907+0000","last_clean":"2024-06-10T00:29:44.882907+0000","last_became_active":"2024-06-10T00:28:34.398373+0000","last_became_peered":"2024-06-10T00:28:34.398373+0000","last_unstale":"2024-06-10T00:29:44.882907+0000","last_undegraded":"2024-06-10T00:29:44.882907+0000","last_fullsized":"2024-06-10T00:29:44.882907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:05:34.352825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774104+0000","last_change":"2024-06-10T00:28:37.647492+0000","last_active":"2024-06-10T00:29:23.774104+0000","last_peered":"2024-06-10T00:29:23.774104+0000","last_clean":"2024-06-10T00:29:23.774104+0000","last_became_active":"2024-06-10T00:28:37.647103+0000","last_became_peered":"2024-06-10T00:28:37.647103+0000","last_unstale":"2024-06-10T00:29:23.774104+0000","last_undegraded":"2024-06-10T00:29:23.774104+0000","last_fullsized":"2024-06-10T00:29:23.774104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:10.429256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780428+0000","last_change":"2024-06-10T00:28:48.703095+0000","last_active":"2024-06-10T00:29:23.780428+0000","last_peered":"2024-06-10T00:29:23.780428+0000","last_clean":"2024-06-10T00:29:23.780428+0000","last_became_active":"2024-06-10T00:28:48.702687+0000","last_became_peered":"2024-06-10T00:28:48.702687+0000","last_unstale":"2024-06-10T00:29:23.780428+0000","last_undegraded":"2024-06-10T00:29:23.780428+0000","last_fullsized":"2024-06-10T00:29:23.780428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:10:20.074173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"97'4","reported_seq":99,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:12.924572+0000","last_change":"2024-06-10T00:28:34.394807+0000","last_active":"2024-06-10T00:30:12.924572+0000","last_peered":"2024-06-10T00:30:12.924572+0000","last_clean":"2024-06-10T00:30:12.924572+0000","last_became_active":"2024-06-10T00:28:34.394022+0000","last_became_peered":"2024-06-10T00:28:34.394022+0000","last_unstale":"2024-06-10T00:30:12.924572+0000","last_undegraded":"2024-06-10T00:30:12.924572+0000","last_fullsized":"2024-06-10T00:30:12.924572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:21.626079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855254+0000","last_change":"2024-06-10T00:28:37.662231+0000","last_active":"2024-06-10T00:29:23.855254+0000","last_peered":"2024-06-10T00:29:23.855254+0000","last_clean":"2024-06-10T00:29:23.855254+0000","last_became_active":"2024-06-10T00:28:37.661861+0000","last_became_peered":"2024-06-10T00:28:37.661861+0000","last_unstale":"2024-06-10T00:29:23.855254+0000","last_undegraded":"2024-06-10T00:29:23.855254+0000","last_fullsized":"2024-06-10T00:29:23.855254+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:09.705349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774088+0000","last_change":"2024-06-10T00:28:48.706643+0000","last_active":"2024-06-10T00:29:23.774088+0000","last_peered":"2024-06-10T00:29:23.774088+0000","last_clean":"2024-06-10T00:29:23.774088+0000","last_became_active":"2024-06-10T00:28:48.706127+0000","last_became_peered":"2024-06-10T00:28:48.706127+0000","last_unstale":"2024-06-10T00:29:23.774088+0000","last_undegraded":"2024-06-10T00:29:23.774088+0000","last_fullsized":"2024-06-10T00:29:23.774088+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:55.031303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.883455+0000","last_change":"2024-06-10T00:28:34.392808+0000","last_active":"2024-06-10T00:29:44.883455+0000","last_peered":"2024-06-10T00:29:44.883455+0000","last_clean":"2024-06-10T00:29:44.883455+0000","last_became_active":"2024-06-10T00:28:34.392450+0000","last_became_peered":"2024-06-10T00:28:34.392450+0000","last_unstale":"2024-06-10T00:29:44.883455+0000","last_undegraded":"2024-06-10T00:29:44.883455+0000","last_fullsized":"2024-06-10T00:29:44.883455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:06:00.493316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777850+0000","last_change":"2024-06-10T00:28:37.656099+0000","last_active":"2024-06-10T00:29:23.777850+0000","last_peered":"2024-06-10T00:29:23.777850+0000","last_clean":"2024-06-10T00:29:23.777850+0000","last_became_active":"2024-06-10T00:28:37.655640+0000","last_became_peered":"2024-06-10T00:28:37.655640+0000","last_unstale":"2024-06-10T00:29:23.777850+0000","last_undegraded":"2024-06-10T00:29:23.777850+0000","last_fullsized":"2024-06-10T00:29:23.777850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:59:26.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774852+0000","last_change":"2024-06-10T00:28:48.702628+0000","last_active":"2024-06-10T00:29:23.774852+0000","last_peered":"2024-06-10T00:29:23.774852+0000","last_clean":"2024-06-10T00:29:23.774852+0000","last_became_active":"2024-06-10T00:28:48.702109+0000","last_became_peered":"2024-06-10T00:28:48.702109+0000","last_unstale":"2024-06-10T00:29:23.774852+0000","last_undegraded":"2024-06-10T00:29:23.774852+0000","last_fullsized":"2024-06-10T00:29:23.774852+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:25.262348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:43.872188+0000","last_change":"2024-06-10T00:28:34.446788+0000","last_active":"2024-06-10T00:29:43.872188+0000","last_peered":"2024-06-10T00:29:43.872188+0000","last_clean":"2024-06-10T00:29:43.872188+0000","last_became_active":"2024-06-10T00:28:34.446377+0000","last_became_peered":"2024-06-10T00:28:34.446377+0000","last_unstale":"2024-06-10T00:29:43.872188+0000","last_undegraded":"2024-06-10T00:29:43.872188+0000","last_fullsized":"2024-06-10T00:29:43.872188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:16.914244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855534+0000","last_change":"2024-06-10T00:28:37.666159+0000","last_active":"2024-06-10T00:29:23.855534+0000","last_peered":"2024-06-10T00:29:23.855534+0000","last_clean":"2024-06-10T00:29:23.855534+0000","last_became_active":"2024-06-10T00:28:37.665711+0000","last_became_peered":"2024-06-10T00:28:37.665711+0000","last_unstale":"2024-06-10T00:29:23.855534+0000","last_undegraded":"2024-06-10T00:29:23.855534+0000","last_fullsized":"2024-06-10T00:29:23.855534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:03.982375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774073+0000","last_change":"2024-06-10T00:28:48.705071+0000","last_active":"2024-06-10T00:29:23.774073+0000","last_peered":"2024-06-10T00:29:23.774073+0000","last_clean":"2024-06-10T00:29:23.774073+0000","last_became_active":"2024-06-10T00:28:48.702211+0000","last_became_peered":"2024-06-10T00:28:48.702211+0000","last_unstale":"2024-06-10T00:29:23.774073+0000","last_undegraded":"2024-06-10T00:29:23.774073+0000","last_fullsized":"2024-06-10T00:29:23.774073+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:13.361985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'34","reported_seq":138,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:12.925084+0000","last_change":"2024-06-10T00:28:34.389784+0000","last_active":"2024-06-10T00:30:12.925084+0000","last_peered":"2024-06-10T00:30:12.925084+0000","last_clean":"2024-06-10T00:30:12.925084+0000","last_became_active":"2024-06-10T00:28:34.389416+0000","last_became_peered":"2024-06-10T00:28:34.389416+0000","last_unstale":"2024-06-10T00:30:12.925084+0000","last_undegraded":"2024-06-10T00:30:12.925084+0000","last_fullsized":"2024-06-10T00:30:12.925084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:23:05.803503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62655,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":94,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777759+0000","last_change":"2024-06-10T00:28:37.656174+0000","last_active":"2024-06-10T00:29:23.777759+0000","last_peered":"2024-06-10T00:29:23.777759+0000","last_clean":"2024-06-10T00:29:23.777759+0000","last_became_active":"2024-06-10T00:28:37.655794+0000","last_became_peered":"2024-06-10T00:28:37.655794+0000","last_unstale":"2024-06-10T00:29:23.777759+0000","last_undegraded":"2024-06-10T00:29:23.777759+0000","last_fullsized":"2024-06-10T00:29:23.777759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:22:00.310309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855645+0000","last_change":"2024-06-10T00:28:48.709915+0000","last_active":"2024-06-10T00:29:23.855645+0000","last_peered":"2024-06-10T00:29:23.855645+0000","last_clean":"2024-06-10T00:29:23.855645+0000","last_became_active":"2024-06-10T00:28:48.709426+0000","last_became_peered":"2024-06-10T00:28:48.709426+0000","last_unstale":"2024-06-10T00:29:23.855645+0000","last_undegraded":"2024-06-10T00:29:23.855645+0000","last_fullsized":"2024-06-10T00:29:23.855645+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:55.150617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:45.883835+0000","last_change":"2024-06-10T00:28:34.389136+0000","last_active":"2024-06-10T00:29:45.883835+0000","last_peered":"2024-06-10T00:29:45.883835+0000","last_clean":"2024-06-10T00:29:45.883835+0000","last_became_active":"2024-06-10T00:28:34.388701+0000","last_became_peered":"2024-06-10T00:28:34.388701+0000","last_unstale":"2024-06-10T00:29:45.883835+0000","last_undegraded":"2024-06-10T00:29:45.883835+0000","last_fullsized":"2024-06-10T00:29:45.883835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:13:24.907824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777188+0000","last_change":"2024-06-10T00:28:37.659048+0000","last_active":"2024-06-10T00:29:23.777188+0000","last_peered":"2024-06-10T00:29:23.777188+0000","last_clean":"2024-06-10T00:29:23.777188+0000","last_became_active":"2024-06-10T00:28:37.658710+0000","last_became_peered":"2024-06-10T00:28:37.658710+0000","last_unstale":"2024-06-10T00:29:23.777188+0000","last_undegraded":"2024-06-10T00:29:23.777188+0000","last_fullsized":"2024-06-10T00:29:23.777188+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:44.314559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779228+0000","last_change":"2024-06-10T00:28:48.709683+0000","last_active":"2024-06-10T00:29:23.779228+0000","last_peered":"2024-06-10T00:29:23.779228+0000","last_clean":"2024-06-10T00:29:23.779228+0000","last_became_active":"2024-06-10T00:28:48.709241+0000","last_became_peered":"2024-06-10T00:28:48.709241+0000","last_unstale":"2024-06-10T00:29:23.779228+0000","last_undegraded":"2024-06-10T00:29:23.779228+0000","last_fullsized":"2024-06-10T00:29:23.779228+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:25.190043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.883378+0000","last_change":"2024-06-10T00:29:23.782029+0000","last_active":"2024-06-10T00:29:44.883378+0000","last_peered":"2024-06-10T00:29:44.883378+0000","last_clean":"2024-06-10T00:29:44.883378+0000","last_became_active":"2024-06-10T00:29:23.781565+0000","last_became_peered":"2024-06-10T00:29:23.781565+0000","last_unstale":"2024-06-10T00:29:44.883378+0000","last_undegraded":"2024-06-10T00:29:44.883378+0000","last_fullsized":"2024-06-10T00:29:44.883378+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:53.258760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777619+0000","last_change":"2024-06-10T00:28:37.653385+0000","last_active":"2024-06-10T00:29:23.777619+0000","last_peered":"2024-06-10T00:29:23.777619+0000","last_clean":"2024-06-10T00:29:23.777619+0000","last_became_active":"2024-06-10T00:28:37.653020+0000","last_became_peered":"2024-06-10T00:28:37.653020+0000","last_unstale":"2024-06-10T00:29:23.777619+0000","last_undegraded":"2024-06-10T00:29:23.777619+0000","last_fullsized":"2024-06-10T00:29:23.777619+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:57:54.469544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774495+0000","last_change":"2024-06-10T00:28:48.707540+0000","last_active":"2024-06-10T00:29:23.774495+0000","last_peered":"2024-06-10T00:29:23.774495+0000","last_clean":"2024-06-10T00:29:23.774495+0000","last_became_active":"2024-06-10T00:28:48.707143+0000","last_became_peered":"2024-06-10T00:28:48.707143+0000","last_unstale":"2024-06-10T00:29:23.774495+0000","last_undegraded":"2024-06-10T00:29:23.774495+0000","last_fullsized":"2024-06-10T00:29:23.774495+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:20:29.344808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'6","reported_seq":76,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:10.921377+0000","last_change":"2024-06-10T00:28:34.397496+0000","last_active":"2024-06-10T00:30:10.921377+0000","last_peered":"2024-06-10T00:30:10.921377+0000","last_clean":"2024-06-10T00:30:10.921377+0000","last_became_active":"2024-06-10T00:28:34.397124+0000","last_became_peered":"2024-06-10T00:28:34.397124+0000","last_unstale":"2024-06-10T00:30:10.921377+0000","last_undegraded":"2024-06-10T00:30:10.921377+0000","last_fullsized":"2024-06-10T00:30:10.921377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:41:48.609354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777477+0000","last_change":"2024-06-10T00:28:37.658338+0000","last_active":"2024-06-10T00:29:23.777477+0000","last_peered":"2024-06-10T00:29:23.777477+0000","last_clean":"2024-06-10T00:29:23.777477+0000","last_became_active":"2024-06-10T00:28:37.655910+0000","last_became_peered":"2024-06-10T00:28:37.655910+0000","last_unstale":"2024-06-10T00:29:23.777477+0000","last_undegraded":"2024-06-10T00:29:23.777477+0000","last_fullsized":"2024-06-10T00:29:23.777477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:38.501335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777562+0000","last_change":"2024-06-10T00:28:48.711151+0000","last_active":"2024-06-10T00:29:23.777562+0000","last_peered":"2024-06-10T00:29:23.777562+0000","last_clean":"2024-06-10T00:29:23.777562+0000","last_became_active":"2024-06-10T00:28:48.710768+0000","last_became_peered":"2024-06-10T00:28:48.710768+0000","last_unstale":"2024-06-10T00:29:23.777562+0000","last_undegraded":"2024-06-10T00:29:23.777562+0000","last_fullsized":"2024-06-10T00:29:23.777562+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:15:22.791839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":58,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.882578+0000","last_change":"2024-06-10T00:28:34.445812+0000","last_active":"2024-06-10T00:29:44.882578+0000","last_peered":"2024-06-10T00:29:44.882578+0000","last_clean":"2024-06-10T00:29:44.882578+0000","last_became_active":"2024-06-10T00:28:34.445472+0000","last_became_peered":"2024-06-10T00:28:34.445472+0000","last_unstale":"2024-06-10T00:29:44.882578+0000","last_undegraded":"2024-06-10T00:29:44.882578+0000","last_fullsized":"2024-06-10T00:29:44.882578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:00.912050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854943+0000","last_change":"2024-06-10T00:28:37.655345+0000","last_active":"2024-06-10T00:29:23.854943+0000","last_peered":"2024-06-10T00:29:23.854943+0000","last_clean":"2024-06-10T00:29:23.854943+0000","last_became_active":"2024-06-10T00:28:37.654954+0000","last_became_peered":"2024-06-10T00:28:37.654954+0000","last_unstale":"2024-06-10T00:29:23.854943+0000","last_undegraded":"2024-06-10T00:29:23.854943+0000","last_fullsized":"2024-06-10T00:29:23.854943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:39:27.403006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854755+0000","last_change":"2024-06-10T00:28:48.705247+0000","last_active":"2024-06-10T00:29:23.854755+0000","last_peered":"2024-06-10T00:29:23.854755+0000","last_clean":"2024-06-10T00:29:23.854755+0000","last_became_active":"2024-06-10T00:28:48.704822+0000","last_became_peered":"2024-06-10T00:28:48.704822+0000","last_unstale":"2024-06-10T00:29:23.854755+0000","last_undegraded":"2024-06-10T00:29:23.854755+0000","last_fullsized":"2024-06-10T00:29:23.854755+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:24:59.991491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.857565+0000","last_change":"2024-06-10T00:28:34.396792+0000","last_active":"2024-06-10T00:29:44.857565+0000","last_peered":"2024-06-10T00:29:44.857565+0000","last_clean":"2024-06-10T00:29:44.857565+0000","last_became_active":"2024-06-10T00:28:34.396442+0000","last_became_peered":"2024-06-10T00:28:34.396442+0000","last_unstale":"2024-06-10T00:29:44.857565+0000","last_undegraded":"2024-06-10T00:29:44.857565+0000","last_fullsized":"2024-06-10T00:29:44.857565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:46.436198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780106+0000","last_change":"2024-06-10T00:28:37.657698+0000","last_active":"2024-06-10T00:29:23.780106+0000","last_peered":"2024-06-10T00:29:23.780106+0000","last_clean":"2024-06-10T00:29:23.780106+0000","last_became_active":"2024-06-10T00:28:37.657364+0000","last_became_peered":"2024-06-10T00:28:37.657364+0000","last_unstale":"2024-06-10T00:29:23.780106+0000","last_undegraded":"2024-06-10T00:29:23.780106+0000","last_fullsized":"2024-06-10T00:29:23.780106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:23.097015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.778567+0000","last_change":"2024-06-10T00:28:48.727567+0000","last_active":"2024-06-10T00:29:23.778567+0000","last_peered":"2024-06-10T00:29:23.778567+0000","last_clean":"2024-06-10T00:29:23.778567+0000","last_became_active":"2024-06-10T00:28:48.727108+0000","last_became_peered":"2024-06-10T00:28:48.727108+0000","last_unstale":"2024-06-10T00:29:23.778567+0000","last_undegraded":"2024-06-10T00:29:23.778567+0000","last_fullsized":"2024-06-10T00:29:23.778567+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:35:00.824177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774644+0000","last_change":"2024-06-10T00:28:34.446832+0000","last_active":"2024-06-10T00:29:23.774644+0000","last_peered":"2024-06-10T00:29:23.774644+0000","last_clean":"2024-06-10T00:29:23.774644+0000","last_became_active":"2024-06-10T00:28:34.446435+0000","last_became_peered":"2024-06-10T00:28:34.446435+0000","last_unstale":"2024-06-10T00:29:23.774644+0000","last_undegraded":"2024-06-10T00:29:23.774644+0000","last_fullsized":"2024-06-10T00:29:23.774644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:58.296674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.856039+0000","last_change":"2024-06-10T00:28:37.647671+0000","last_active":"2024-06-10T00:29:23.856039+0000","last_peered":"2024-06-10T00:29:23.856039+0000","last_clean":"2024-06-10T00:29:23.856039+0000","last_became_active":"2024-06-10T00:28:37.647319+0000","last_became_peered":"2024-06-10T00:28:37.647319+0000","last_unstale":"2024-06-10T00:29:23.856039+0000","last_undegraded":"2024-06-10T00:29:23.856039+0000","last_fullsized":"2024-06-10T00:29:23.856039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:22:51.910596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777130+0000","last_change":"2024-06-10T00:28:48.704683+0000","last_active":"2024-06-10T00:29:23.777130+0000","last_peered":"2024-06-10T00:29:23.777130+0000","last_clean":"2024-06-10T00:29:23.777130+0000","last_became_active":"2024-06-10T00:28:48.704297+0000","last_became_peered":"2024-06-10T00:28:48.704297+0000","last_unstale":"2024-06-10T00:29:23.777130+0000","last_undegraded":"2024-06-10T00:29:23.777130+0000","last_fullsized":"2024-06-10T00:29:23.777130+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:44.980544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'5","reported_seq":71,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:09.919907+0000","last_change":"2024-06-10T00:28:34.393328+0000","last_active":"2024-06-10T00:30:09.919907+0000","last_peered":"2024-06-10T00:30:09.919907+0000","last_clean":"2024-06-10T00:30:09.919907+0000","last_became_active":"2024-06-10T00:28:34.391968+0000","last_became_peered":"2024-06-10T00:28:34.391968+0000","last_unstale":"2024-06-10T00:30:09.919907+0000","last_undegraded":"2024-06-10T00:30:09.919907+0000","last_fullsized":"2024-06-10T00:30:09.919907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:18.898180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.856246+0000","last_change":"2024-06-10T00:28:37.652555+0000","last_active":"2024-06-10T00:29:23.856246+0000","last_peered":"2024-06-10T00:29:23.856246+0000","last_clean":"2024-06-10T00:29:23.856246+0000","last_became_active":"2024-06-10T00:28:37.651476+0000","last_became_peered":"2024-06-10T00:28:37.651476+0000","last_unstale":"2024-06-10T00:29:23.856246+0000","last_undegraded":"2024-06-10T00:29:23.856246+0000","last_fullsized":"2024-06-10T00:29:23.856246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:04:57.823224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.778229+0000","last_change":"2024-06-10T00:28:48.727568+0000","last_active":"2024-06-10T00:29:23.778229+0000","last_peered":"2024-06-10T00:29:23.778229+0000","last_clean":"2024-06-10T00:29:23.778229+0000","last_became_active":"2024-06-10T00:28:48.727018+0000","last_became_peered":"2024-06-10T00:28:48.727018+0000","last_unstale":"2024-06-10T00:29:23.778229+0000","last_undegraded":"2024-06-10T00:29:23.778229+0000","last_fullsized":"2024-06-10T00:29:23.778229+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:02.025212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854952+0000","last_change":"2024-06-10T00:28:37.645264+0000","last_active":"2024-06-10T00:29:23.854952+0000","last_peered":"2024-06-10T00:29:23.854952+0000","last_clean":"2024-06-10T00:29:23.854952+0000","last_became_active":"2024-06-10T00:28:37.644860+0000","last_became_peered":"2024-06-10T00:28:37.644860+0000","last_unstale":"2024-06-10T00:29:23.854952+0000","last_undegraded":"2024-06-10T00:29:23.854952+0000","last_fullsized":"2024-06-10T00:29:23.854952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:28.309172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"97'5","reported_seq":71,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:09.919496+0000","last_change":"2024-06-10T00:28:34.445501+0000","last_active":"2024-06-10T00:30:09.919496+0000","last_peered":"2024-06-10T00:30:09.919496+0000","last_clean":"2024-06-10T00:30:09.919496+0000","last_became_active":"2024-06-10T00:28:34.445128+0000","last_became_peered":"2024-06-10T00:28:34.445128+0000","last_unstale":"2024-06-10T00:30:09.919496+0000","last_undegraded":"2024-06-10T00:30:09.919496+0000","last_fullsized":"2024-06-10T00:30:09.919496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:59.231585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777876+0000","last_change":"2024-06-10T00:28:48.709852+0000","last_active":"2024-06-10T00:29:23.777876+0000","last_peered":"2024-06-10T00:29:23.777876+0000","last_clean":"2024-06-10T00:29:23.777876+0000","last_became_active":"2024-06-10T00:28:48.709423+0000","last_became_peered":"2024-06-10T00:28:48.709423+0000","last_unstale":"2024-06-10T00:29:23.777876+0000","last_undegraded":"2024-06-10T00:29:23.777876+0000","last_fullsized":"2024-06-10T00:29:23.777876+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:42.123563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780357+0000","last_change":"2024-06-10T00:28:37.657423+0000","last_active":"2024-06-10T00:29:23.780357+0000","last_peered":"2024-06-10T00:29:23.780357+0000","last_clean":"2024-06-10T00:29:23.780357+0000","last_became_active":"2024-06-10T00:28:37.657140+0000","last_became_peered":"2024-06-10T00:28:37.657140+0000","last_unstale":"2024-06-10T00:29:23.780357+0000","last_undegraded":"2024-06-10T00:29:23.780357+0000","last_fullsized":"2024-06-10T00:29:23.780357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:20:08.943066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:45.884674+0000","last_change":"2024-06-10T00:28:34.392585+0000","last_active":"2024-06-10T00:29:45.884674+0000","last_peered":"2024-06-10T00:29:45.884674+0000","last_clean":"2024-06-10T00:29:45.884674+0000","last_became_active":"2024-06-10T00:28:34.392229+0000","last_became_peered":"2024-06-10T00:28:34.392229+0000","last_unstale":"2024-06-10T00:29:45.884674+0000","last_undegraded":"2024-06-10T00:29:45.884674+0000","last_fullsized":"2024-06-10T00:29:45.884674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:11.275761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":373,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777128+0000","last_change":"2024-06-10T00:27:12.072577+0000","last_active":"2024-06-10T00:29:23.777128+0000","last_peered":"2024-06-10T00:29:23.777128+0000","last_clean":"2024-06-10T00:29:23.777128+0000","last_became_active":"2024-06-10T00:27:12.072193+0000","last_became_peered":"2024-06-10T00:27:12.072193+0000","last_unstale":"2024-06-10T00:29:23.777128+0000","last_undegraded":"2024-06-10T00:29:23.777128+0000","last_fullsized":"2024-06-10T00:29:23.777128+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_clean_scrub_stamp":"2024-06-10T00:23:46.826438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:57.782124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855452+0000","last_change":"2024-06-10T00:28:48.727913+0000","last_active":"2024-06-10T00:29:23.855452+0000","last_peered":"2024-06-10T00:29:23.855452+0000","last_clean":"2024-06-10T00:29:23.855452+0000","last_became_active":"2024-06-10T00:28:48.727402+0000","last_became_peered":"2024-06-10T00:28:48.727402+0000","last_unstale":"2024-06-10T00:29:23.855452+0000","last_undegraded":"2024-06-10T00:29:23.855452+0000","last_fullsized":"2024-06-10T00:29:23.855452+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:10:25.051296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.778107+0000","last_change":"2024-06-10T00:28:37.659901+0000","last_active":"2024-06-10T00:29:23.778107+0000","last_peered":"2024-06-10T00:29:23.778107+0000","last_clean":"2024-06-10T00:29:23.778107+0000","last_became_active":"2024-06-10T00:28:37.659553+0000","last_became_peered":"2024-06-10T00:28:37.659553+0000","last_unstale":"2024-06-10T00:29:23.778107+0000","last_undegraded":"2024-06-10T00:29:23.778107+0000","last_fullsized":"2024-06-10T00:29:23.778107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:25.694823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:42.861159+0000","last_change":"2024-06-10T00:28:34.398155+0000","last_active":"2024-06-10T00:29:42.861159+0000","last_peered":"2024-06-10T00:29:42.861159+0000","last_clean":"2024-06-10T00:29:42.861159+0000","last_became_active":"2024-06-10T00:28:34.397793+0000","last_became_peered":"2024-06-10T00:28:34.397793+0000","last_unstale":"2024-06-10T00:29:42.861159+0000","last_undegraded":"2024-06-10T00:29:42.861159+0000","last_fullsized":"2024-06-10T00:29:42.861159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:21.188139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777637+0000","last_change":"2024-06-10T00:28:48.709736+0000","last_active":"2024-06-10T00:29:23.777637+0000","last_peered":"2024-06-10T00:29:23.777637+0000","last_clean":"2024-06-10T00:29:23.777637+0000","last_became_active":"2024-06-10T00:28:48.709293+0000","last_became_peered":"2024-06-10T00:28:48.709293+0000","last_unstale":"2024-06-10T00:29:23.777637+0000","last_undegraded":"2024-06-10T00:29:23.777637+0000","last_fullsized":"2024-06-10T00:29:23.777637+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:30:04.840656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.881597+0000","last_change":"2024-06-10T00:29:23.780030+0000","last_active":"2024-06-10T00:29:44.881597+0000","last_peered":"2024-06-10T00:29:44.881597+0000","last_clean":"2024-06-10T00:29:44.881597+0000","last_became_active":"2024-06-10T00:29:23.779589+0000","last_became_peered":"2024-06-10T00:29:23.779589+0000","last_unstale":"2024-06-10T00:29:44.881597+0000","last_undegraded":"2024-06-10T00:29:44.881597+0000","last_fullsized":"2024-06-10T00:29:44.881597+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:53.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777314+0000","last_change":"2024-06-10T00:28:37.658330+0000","last_active":"2024-06-10T00:29:23.777314+0000","last_peered":"2024-06-10T00:29:23.777314+0000","last_clean":"2024-06-10T00:29:23.777314+0000","last_became_active":"2024-06-10T00:28:37.657343+0000","last_became_peered":"2024-06-10T00:28:37.657343+0000","last_unstale":"2024-06-10T00:29:23.777314+0000","last_undegraded":"2024-06-10T00:29:23.777314+0000","last_fullsized":"2024-06-10T00:29:23.777314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:31:52.167402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779704+0000","last_change":"2024-06-10T00:28:48.727506+0000","last_active":"2024-06-10T00:29:23.779704+0000","last_peered":"2024-06-10T00:29:23.779704+0000","last_clean":"2024-06-10T00:29:23.779704+0000","last_became_active":"2024-06-10T00:28:48.727068+0000","last_became_peered":"2024-06-10T00:28:48.727068+0000","last_unstale":"2024-06-10T00:29:23.779704+0000","last_undegraded":"2024-06-10T00:29:23.779704+0000","last_fullsized":"2024-06-10T00:29:23.779704+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:42.147908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'15","reported_seq":55,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:05.663885+0000","last_change":"2024-06-10T00:28:34.399872+0000","last_active":"2024-06-10T00:30:05.663885+0000","last_peered":"2024-06-10T00:30:05.663885+0000","last_clean":"2024-06-10T00:30:05.663885+0000","last_became_active":"2024-06-10T00:28:34.399493+0000","last_became_peered":"2024-06-10T00:28:34.399493+0000","last_unstale":"2024-06-10T00:30:05.663885+0000","last_undegraded":"2024-06-10T00:30:05.663885+0000","last_fullsized":"2024-06-10T00:30:05.663885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:24:10.018110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780712+0000","last_change":"2024-06-10T00:28:37.656597+0000","last_active":"2024-06-10T00:29:23.780712+0000","last_peered":"2024-06-10T00:29:23.780712+0000","last_clean":"2024-06-10T00:29:23.780712+0000","last_became_active":"2024-06-10T00:28:37.654356+0000","last_became_peered":"2024-06-10T00:28:37.654356+0000","last_unstale":"2024-06-10T00:29:23.780712+0000","last_undegraded":"2024-06-10T00:29:23.780712+0000","last_fullsized":"2024-06-10T00:29:23.780712+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:07:12.950683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774497+0000","last_change":"2024-06-10T00:28:48.702686+0000","last_active":"2024-06-10T00:29:23.774497+0000","last_peered":"2024-06-10T00:29:23.774497+0000","last_clean":"2024-06-10T00:29:23.774497+0000","last_became_active":"2024-06-10T00:28:48.702273+0000","last_became_peered":"2024-06-10T00:28:48.702273+0000","last_unstale":"2024-06-10T00:29:23.774497+0000","last_undegraded":"2024-06-10T00:29:23.774497+0000","last_fullsized":"2024-06-10T00:29:23.774497+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:00.078232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.883862+0000","last_change":"2024-06-10T00:28:34.447549+0000","last_active":"2024-06-10T00:29:44.883862+0000","last_peered":"2024-06-10T00:29:44.883862+0000","last_clean":"2024-06-10T00:29:44.883862+0000","last_became_active":"2024-06-10T00:28:34.447197+0000","last_became_peered":"2024-06-10T00:28:34.447197+0000","last_unstale":"2024-06-10T00:29:44.883862+0000","last_undegraded":"2024-06-10T00:29:44.883862+0000","last_fullsized":"2024-06-10T00:29:44.883862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:47.525153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777770+0000","last_change":"2024-06-10T00:28:37.655585+0000","last_active":"2024-06-10T00:29:23.777770+0000","last_peered":"2024-06-10T00:29:23.777770+0000","last_clean":"2024-06-10T00:29:23.777770+0000","last_became_active":"2024-06-10T00:28:37.655225+0000","last_became_peered":"2024-06-10T00:28:37.655225+0000","last_unstale":"2024-06-10T00:29:23.777770+0000","last_undegraded":"2024-06-10T00:29:23.777770+0000","last_fullsized":"2024-06-10T00:29:23.777770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:57:25.184924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.857447+0000","last_change":"2024-06-10T00:29:23.857447+0000","last_active":"2024-06-10T00:29:23.857447+0000","last_peered":"2024-06-10T00:29:23.857447+0000","last_clean":"2024-06-10T00:29:23.857447+0000","last_became_active":"2024-06-10T00:29:23.856943+0000","last_became_peered":"2024-06-10T00:29:23.856943+0000","last_unstale":"2024-06-10T00:29:23.857447+0000","last_undegraded":"2024-06-10T00:29:23.857447+0000","last_fullsized":"2024-06-10T00:29:23.857447+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:58:44.548818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:41.854544+0000","last_change":"2024-06-10T00:28:34.445637+0000","last_active":"2024-06-10T00:29:41.854544+0000","last_peered":"2024-06-10T00:29:41.854544+0000","last_clean":"2024-06-10T00:29:41.854544+0000","last_became_active":"2024-06-10T00:28:34.445324+0000","last_became_peered":"2024-06-10T00:28:34.445324+0000","last_unstale":"2024-06-10T00:29:41.854544+0000","last_undegraded":"2024-06-10T00:29:41.854544+0000","last_fullsized":"2024-06-10T00:29:41.854544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:05.713557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777101+0000","last_change":"2024-06-10T00:28:37.659037+0000","last_active":"2024-06-10T00:29:23.777101+0000","last_peered":"2024-06-10T00:29:23.777101+0000","last_clean":"2024-06-10T00:29:23.777101+0000","last_became_active":"2024-06-10T00:28:37.658705+0000","last_became_peered":"2024-06-10T00:28:37.658705+0000","last_unstale":"2024-06-10T00:29:23.777101+0000","last_undegraded":"2024-06-10T00:29:23.777101+0000","last_fullsized":"2024-06-10T00:29:23.777101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:20:52.797237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774629+0000","last_change":"2024-06-10T00:28:48.704130+0000","last_active":"2024-06-10T00:29:23.774629+0000","last_peered":"2024-06-10T00:29:23.774629+0000","last_clean":"2024-06-10T00:29:23.774629+0000","last_became_active":"2024-06-10T00:28:48.703738+0000","last_became_peered":"2024-06-10T00:28:48.703738+0000","last_unstale":"2024-06-10T00:29:23.774629+0000","last_undegraded":"2024-06-10T00:29:23.774629+0000","last_fullsized":"2024-06-10T00:29:23.774629+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:01:40.436710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.881279+0000","last_change":"2024-06-10T00:28:34.386667+0000","last_active":"2024-06-10T00:29:44.881279+0000","last_peered":"2024-06-10T00:29:44.881279+0000","last_clean":"2024-06-10T00:29:44.881279+0000","last_became_active":"2024-06-10T00:28:34.386363+0000","last_became_peered":"2024-06-10T00:28:34.386363+0000","last_unstale":"2024-06-10T00:29:44.881279+0000","last_undegraded":"2024-06-10T00:29:44.881279+0000","last_fullsized":"2024-06-10T00:29:44.881279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:11.850162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777197+0000","last_change":"2024-06-10T00:28:37.664754+0000","last_active":"2024-06-10T00:29:23.777197+0000","last_peered":"2024-06-10T00:29:23.777197+0000","last_clean":"2024-06-10T00:29:23.777197+0000","last_became_active":"2024-06-10T00:28:37.658574+0000","last_became_peered":"2024-06-10T00:28:37.658574+0000","last_unstale":"2024-06-10T00:29:23.777197+0000","last_undegraded":"2024-06-10T00:29:23.777197+0000","last_fullsized":"2024-06-10T00:29:23.777197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:16:47.437822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.775310+0000","last_change":"2024-06-10T00:28:48.706560+0000","last_active":"2024-06-10T00:29:23.775310+0000","last_peered":"2024-06-10T00:29:23.775310+0000","last_clean":"2024-06-10T00:29:23.775310+0000","last_became_active":"2024-06-10T00:28:48.705986+0000","last_became_peered":"2024-06-10T00:28:48.705986+0000","last_unstale":"2024-06-10T00:29:23.775310+0000","last_undegraded":"2024-06-10T00:29:23.775310+0000","last_fullsized":"2024-06-10T00:29:23.775310+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:21:26.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'3","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:43.872209+0000","last_change":"2024-06-10T00:28:34.394509+0000","last_active":"2024-06-10T00:29:43.872209+0000","last_peered":"2024-06-10T00:29:43.872209+0000","last_clean":"2024-06-10T00:29:43.872209+0000","last_became_active":"2024-06-10T00:28:34.394105+0000","last_became_peered":"2024-06-10T00:28:34.394105+0000","last_unstale":"2024-06-10T00:29:43.872209+0000","last_undegraded":"2024-06-10T00:29:43.872209+0000","last_fullsized":"2024-06-10T00:29:43.872209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:13.415544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780855+0000","last_change":"2024-06-10T00:28:37.653192+0000","last_active":"2024-06-10T00:29:23.780855+0000","last_peered":"2024-06-10T00:29:23.780855+0000","last_clean":"2024-06-10T00:29:23.780855+0000","last_became_active":"2024-06-10T00:28:37.652868+0000","last_became_peered":"2024-06-10T00:28:37.652868+0000","last_unstale":"2024-06-10T00:29:23.780855+0000","last_undegraded":"2024-06-10T00:29:23.780855+0000","last_fullsized":"2024-06-10T00:29:23.780855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:21.614915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.775035+0000","last_change":"2024-06-10T00:28:48.727615+0000","last_active":"2024-06-10T00:29:23.775035+0000","last_peered":"2024-06-10T00:29:23.775035+0000","last_clean":"2024-06-10T00:29:23.775035+0000","last_became_active":"2024-06-10T00:28:48.727190+0000","last_became_peered":"2024-06-10T00:28:48.727190+0000","last_unstale":"2024-06-10T00:29:23.775035+0000","last_undegraded":"2024-06-10T00:29:23.775035+0000","last_fullsized":"2024-06-10T00:29:23.775035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:57.498183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":55,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:41.853891+0000","last_change":"2024-06-10T00:29:23.784131+0000","last_active":"2024-06-10T00:29:41.853891+0000","last_peered":"2024-06-10T00:29:41.853891+0000","last_clean":"2024-06-10T00:29:41.853891+0000","last_became_active":"2024-06-10T00:29:23.780909+0000","last_became_peered":"2024-06-10T00:29:23.780909+0000","last_unstale":"2024-06-10T00:29:41.853891+0000","last_undegraded":"2024-06-10T00:29:41.853891+0000","last_fullsized":"2024-06-10T00:29:41.853891+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.774217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777897+0000","last_change":"2024-06-10T00:28:37.660399+0000","last_active":"2024-06-10T00:29:23.777897+0000","last_peered":"2024-06-10T00:29:23.777897+0000","last_clean":"2024-06-10T00:29:23.777897+0000","last_became_active":"2024-06-10T00:28:37.660069+0000","last_became_peered":"2024-06-10T00:28:37.660069+0000","last_unstale":"2024-06-10T00:29:23.777897+0000","last_undegraded":"2024-06-10T00:29:23.777897+0000","last_fullsized":"2024-06-10T00:29:23.777897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:59.727883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854444+0000","last_change":"2024-06-10T00:28:48.713267+0000","last_active":"2024-06-10T00:29:23.854444+0000","last_peered":"2024-06-10T00:29:23.854444+0000","last_clean":"2024-06-10T00:29:23.854444+0000","last_became_active":"2024-06-10T00:28:48.712870+0000","last_became_peered":"2024-06-10T00:28:48.712870+0000","last_unstale":"2024-06-10T00:29:23.854444+0000","last_undegraded":"2024-06-10T00:29:23.854444+0000","last_fullsized":"2024-06-10T00:29:23.854444+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:59.210101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'5","reported_seq":71,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:30:08.920048+0000","last_change":"2024-06-10T00:28:34.448652+0000","last_active":"2024-06-10T00:30:08.920048+0000","last_peered":"2024-06-10T00:30:08.920048+0000","last_clean":"2024-06-10T00:30:08.920048+0000","last_became_active":"2024-06-10T00:28:34.448243+0000","last_became_peered":"2024-06-10T00:28:34.448243+0000","last_unstale":"2024-06-10T00:30:08.920048+0000","last_undegraded":"2024-06-10T00:30:08.920048+0000","last_fullsized":"2024-06-10T00:30:08.920048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:51:40.003429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.778119+0000","last_change":"2024-06-10T00:28:37.662307+0000","last_active":"2024-06-10T00:29:23.778119+0000","last_peered":"2024-06-10T00:29:23.778119+0000","last_clean":"2024-06-10T00:29:23.778119+0000","last_became_active":"2024-06-10T00:28:37.661727+0000","last_became_peered":"2024-06-10T00:28:37.661727+0000","last_unstale":"2024-06-10T00:29:23.778119+0000","last_undegraded":"2024-06-10T00:29:23.778119+0000","last_fullsized":"2024-06-10T00:29:23.778119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:23.989429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.857626+0000","last_change":"2024-06-10T00:29:23.857626+0000","last_active":"2024-06-10T00:29:23.857626+0000","last_peered":"2024-06-10T00:29:23.857626+0000","last_clean":"2024-06-10T00:29:23.857626+0000","last_became_active":"2024-06-10T00:29:23.857057+0000","last_became_peered":"2024-06-10T00:29:23.857057+0000","last_unstale":"2024-06-10T00:29:23.857626+0000","last_undegraded":"2024-06-10T00:29:23.857626+0000","last_fullsized":"2024-06-10T00:29:23.857626+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:25:10.340053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"97'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.881073+0000","last_change":"2024-06-10T00:28:34.399253+0000","last_active":"2024-06-10T00:29:44.881073+0000","last_peered":"2024-06-10T00:29:44.881073+0000","last_clean":"2024-06-10T00:29:44.881073+0000","last_became_active":"2024-06-10T00:28:34.397763+0000","last_became_peered":"2024-06-10T00:28:34.397763+0000","last_unstale":"2024-06-10T00:29:44.881073+0000","last_undegraded":"2024-06-10T00:29:44.881073+0000","last_fullsized":"2024-06-10T00:29:44.881073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:05:57.371625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777468+0000","last_change":"2024-06-10T00:28:37.658027+0000","last_active":"2024-06-10T00:29:23.777468+0000","last_peered":"2024-06-10T00:29:23.777468+0000","last_clean":"2024-06-10T00:29:23.777468+0000","last_became_active":"2024-06-10T00:28:37.655513+0000","last_became_peered":"2024-06-10T00:28:37.655513+0000","last_unstale":"2024-06-10T00:29:23.777468+0000","last_undegraded":"2024-06-10T00:29:23.777468+0000","last_fullsized":"2024-06-10T00:29:23.777468+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:50:23.525855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854462+0000","last_change":"2024-06-10T00:28:48.706997+0000","last_active":"2024-06-10T00:29:23.854462+0000","last_peered":"2024-06-10T00:29:23.854462+0000","last_clean":"2024-06-10T00:29:23.854462+0000","last_became_active":"2024-06-10T00:28:48.706597+0000","last_became_peered":"2024-06-10T00:28:48.706597+0000","last_unstale":"2024-06-10T00:29:23.854462+0000","last_undegraded":"2024-06-10T00:29:23.854462+0000","last_fullsized":"2024-06-10T00:29:23.854462+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:16.385312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:44.880832+0000","last_change":"2024-06-10T00:29:23.785774+0000","last_active":"2024-06-10T00:29:44.880832+0000","last_peered":"2024-06-10T00:29:44.880832+0000","last_clean":"2024-06-10T00:29:44.880832+0000","last_became_active":"2024-06-10T00:29:23.781768+0000","last_became_peered":"2024-06-10T00:29:23.781768+0000","last_unstale":"2024-06-10T00:29:44.880832+0000","last_undegraded":"2024-06-10T00:29:44.880832+0000","last_fullsized":"2024-06-10T00:29:44.880832+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:27.348264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777755+0000","last_change":"2024-06-10T00:28:37.657820+0000","last_active":"2024-06-10T00:29:23.777755+0000","last_peered":"2024-06-10T00:29:23.777755+0000","last_clean":"2024-06-10T00:29:23.777755+0000","last_became_active":"2024-06-10T00:28:37.657487+0000","last_became_peered":"2024-06-10T00:28:37.657487+0000","last_unstale":"2024-06-10T00:29:23.777755+0000","last_undegraded":"2024-06-10T00:29:23.777755+0000","last_fullsized":"2024-06-10T00:29:23.777755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:44:35.930401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777344+0000","last_change":"2024-06-10T00:28:37.664543+0000","last_active":"2024-06-10T00:29:23.777344+0000","last_peered":"2024-06-10T00:29:23.777344+0000","last_clean":"2024-06-10T00:29:23.777344+0000","last_became_active":"2024-06-10T00:28:37.659015+0000","last_became_peered":"2024-06-10T00:28:37.659015+0000","last_unstale":"2024-06-10T00:29:23.777344+0000","last_undegraded":"2024-06-10T00:29:23.777344+0000","last_fullsized":"2024-06-10T00:29:23.777344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:06.555472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774751+0000","last_change":"2024-06-10T00:28:37.659389+0000","last_active":"2024-06-10T00:29:23.774751+0000","last_peered":"2024-06-10T00:29:23.774751+0000","last_clean":"2024-06-10T00:29:23.774751+0000","last_became_active":"2024-06-10T00:28:37.658990+0000","last_became_peered":"2024-06-10T00:28:37.658990+0000","last_unstale":"2024-06-10T00:29:23.774751+0000","last_undegraded":"2024-06-10T00:29:23.774751+0000","last_fullsized":"2024-06-10T00:29:23.774751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:29.532779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.774557+0000","last_change":"2024-06-10T00:28:37.655621+0000","last_active":"2024-06-10T00:29:23.774557+0000","last_peered":"2024-06-10T00:29:23.774557+0000","last_clean":"2024-06-10T00:29:23.774557+0000","last_became_active":"2024-06-10T00:28:37.653599+0000","last_became_peered":"2024-06-10T00:28:37.653599+0000","last_unstale":"2024-06-10T00:29:23.774557+0000","last_undegraded":"2024-06-10T00:29:23.774557+0000","last_fullsized":"2024-06-10T00:29:23.774557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:22.165851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.782727+0000","last_change":"2024-06-10T00:29:23.782727+0000","last_active":"2024-06-10T00:29:23.782727+0000","last_peered":"2024-06-10T00:29:23.782727+0000","last_clean":"2024-06-10T00:29:23.782727+0000","last_became_active":"2024-06-10T00:29:23.782309+0000","last_became_peered":"2024-06-10T00:29:23.782309+0000","last_unstale":"2024-06-10T00:29:23.782727+0000","last_undegraded":"2024-06-10T00:29:23.782727+0000","last_fullsized":"2024-06-10T00:29:23.782727+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:20.585955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777419+0000","last_change":"2024-06-10T00:28:37.646750+0000","last_active":"2024-06-10T00:29:23.777419+0000","last_peered":"2024-06-10T00:29:23.777419+0000","last_clean":"2024-06-10T00:29:23.777419+0000","last_became_active":"2024-06-10T00:28:37.646372+0000","last_became_peered":"2024-06-10T00:28:37.646372+0000","last_unstale":"2024-06-10T00:29:23.777419+0000","last_undegraded":"2024-06-10T00:29:23.777419+0000","last_fullsized":"2024-06-10T00:29:23.777419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:45.762956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779106+0000","last_change":"2024-06-10T00:28:37.662278+0000","last_active":"2024-06-10T00:29:23.779106+0000","last_peered":"2024-06-10T00:29:23.779106+0000","last_clean":"2024-06-10T00:29:23.779106+0000","last_became_active":"2024-06-10T00:28:37.661855+0000","last_became_peered":"2024-06-10T00:28:37.661855+0000","last_unstale":"2024-06-10T00:29:23.779106+0000","last_undegraded":"2024-06-10T00:29:23.779106+0000","last_fullsized":"2024-06-10T00:29:23.779106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:20.074208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.775489+0000","last_change":"2024-06-10T00:28:37.656738+0000","last_active":"2024-06-10T00:29:23.775489+0000","last_peered":"2024-06-10T00:29:23.775489+0000","last_clean":"2024-06-10T00:29:23.775489+0000","last_became_active":"2024-06-10T00:28:37.656375+0000","last_became_peered":"2024-06-10T00:28:37.656375+0000","last_unstale":"2024-06-10T00:29:23.775489+0000","last_undegraded":"2024-06-10T00:29:23.775489+0000","last_fullsized":"2024-06-10T00:29:23.775489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:49:17.779670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780920+0000","last_change":"2024-06-10T00:28:37.656849+0000","last_active":"2024-06-10T00:29:23.780920+0000","last_peered":"2024-06-10T00:29:23.780920+0000","last_clean":"2024-06-10T00:29:23.780920+0000","last_became_active":"2024-06-10T00:28:37.656261+0000","last_became_peered":"2024-06-10T00:28:37.656261+0000","last_unstale":"2024-06-10T00:29:23.780920+0000","last_undegraded":"2024-06-10T00:29:23.780920+0000","last_fullsized":"2024-06-10T00:29:23.780920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:13.152165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.854724+0000","last_change":"2024-06-10T00:28:37.648384+0000","last_active":"2024-06-10T00:29:23.854724+0000","last_peered":"2024-06-10T00:29:23.854724+0000","last_clean":"2024-06-10T00:29:23.854724+0000","last_became_active":"2024-06-10T00:28:37.648009+0000","last_became_peered":"2024-06-10T00:28:37.648009+0000","last_unstale":"2024-06-10T00:29:23.854724+0000","last_undegraded":"2024-06-10T00:29:23.854724+0000","last_fullsized":"2024-06-10T00:29:23.854724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:45:55.808932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.780879+0000","last_change":"2024-06-10T00:28:37.655828+0000","last_active":"2024-06-10T00:29:23.780879+0000","last_peered":"2024-06-10T00:29:23.780879+0000","last_clean":"2024-06-10T00:29:23.780879+0000","last_became_active":"2024-06-10T00:28:37.655269+0000","last_became_peered":"2024-06-10T00:28:37.655269+0000","last_unstale":"2024-06-10T00:29:23.780879+0000","last_undegraded":"2024-06-10T00:29:23.780879+0000","last_fullsized":"2024-06-10T00:29:23.780879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:56.618469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.855445+0000","last_change":"2024-06-10T00:28:37.663756+0000","last_active":"2024-06-10T00:29:23.855445+0000","last_peered":"2024-06-10T00:29:23.855445+0000","last_clean":"2024-06-10T00:29:23.855445+0000","last_became_active":"2024-06-10T00:28:37.663380+0000","last_became_peered":"2024-06-10T00:28:37.663380+0000","last_unstale":"2024-06-10T00:29:23.855445+0000","last_undegraded":"2024-06-10T00:29:23.855445+0000","last_fullsized":"2024-06-10T00:29:23.855445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:45.534648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":32,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.777983+0000","last_change":"2024-06-10T00:28:37.655246+0000","last_active":"2024-06-10T00:29:23.777983+0000","last_peered":"2024-06-10T00:29:23.777983+0000","last_clean":"2024-06-10T00:29:23.777983+0000","last_became_active":"2024-06-10T00:28:37.654605+0000","last_became_peered":"2024-06-10T00:28:37.654605+0000","last_unstale":"2024-06-10T00:29:23.777983+0000","last_undegraded":"2024-06-10T00:29:23.777983+0000","last_fullsized":"2024-06-10T00:29:23.777983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:15.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-06-10T00:29:23.779394+0000","last_change":"2024-06-10T00:28:37.662485+0000","last_active":"2024-06-10T00:29:23.779394+0000","last_peered":"2024-06-10T00:29:23.779394+0000","last_clean":"2024-06-10T00:29:23.779394+0000","last_became_active":"2024-06-10T00:28:37.661900+0000","last_became_peered":"2024-06-10T00:28:37.661900+0000","last_unstale":"2024-06-10T00:29:23.779394+0000","last_undegraded":"2024-06-10T00:29:23.779394+0000","last_fullsized":"2024-06-10T00:29:23.779394+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:30:03.210634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69857,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":311,"num_read_kb":236,"num_write":248,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":205306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1126400,"data_stored":551803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28784,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716368,"statfs":{"total":95995035648,"available":95965560832,"internally_reserved":0,"allocated":1810432,"data_stored":1201462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":555627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":551125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1732608,"data_stored":1148795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":547035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36500,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708652,"statfs":{"total":95995035648,"available":95957659648,"internally_reserved":0,"allocated":1191936,"data_stored":611392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":548491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36508,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708644,"statfs":{"total":95995035648,"available":95957651456,"internally_reserved":0,"allocated":1200128,"data_stored":607065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":546743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":551499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28712,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716440,"statfs":{"total":95995035648,"available":95965634560,"internally_reserved":0,"allocated":1736704,"data_stored":1143897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:30:15.599 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-10T00:30:15.600 INFO:tasks.cephadm:Setting up client nodes... 2024-06-10T00:30:15.600 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-10T00:30:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:17.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:17 smithi062 ceph-mon[32390]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-10T00:30:17.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:17 smithi138 ceph-mon[36549]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-10T00:30:17.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:17 smithi105 ceph-mon[36321]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-06-10T00:30:17.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.633+0000 7f02a8db9700 1 -- 172.21.15.62:0/1126269168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a4102510 msgr2=0x7f02a4102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.633+0000 7f02a8db9700 1 --2- 172.21.15.62:0/1126269168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a4102510 0x7f02a4102930 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f02940099c0 tx=0x7f0294009cd0 comp rx=0 tx=0).stop 2024-06-10T00:30:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.633+0000 7f02a8db9700 1 -- 172.21.15.62:0/1126269168 shutdown_connections 2024-06-10T00:30:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.633+0000 7f02a8db9700 1 --2- 172.21.15.62:0/1126269168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a4102510 0x7f02a4102930 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.633+0000 7f02a8db9700 1 --2- 172.21.15.62:0/1126269168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a41040b0 0x7f02a410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.633+0000 7f02a8db9700 1 --2- 172.21.15.62:0/1126269168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a4103710 0x7f02a4103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.633+0000 7f02a8db9700 1 -- 172.21.15.62:0/1126269168 >> 172.21.15.62:0/1126269168 conn(0x7f02a40fdab0 msgr2=0x7f02a40ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.633+0000 7f02a8db9700 1 -- 172.21.15.62:0/1126269168 shutdown_connections 2024-06-10T00:30:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.633+0000 7f02a8db9700 1 -- 172.21.15.62:0/1126269168 wait complete. 2024-06-10T00:30:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a8db9700 1 Processor -- start 2024-06-10T00:30:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a8db9700 1 -- start start 2024-06-10T00:30:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a8db9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a4102510 0x7f02a4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a8db9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a4103710 0x7f02a4198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a8db9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a41040b0 0x7f02a419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a8db9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f02a410e150 con 0x7f02a4103710 2024-06-10T00:30:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a8db9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f02a410dfd0 con 0x7f02a41040b0 2024-06-10T00:30:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a8db9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02a410e2d0 con 0x7f02a4102510 2024-06-10T00:30:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a4103710 0x7f02a4198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a4103710 0x7f02a4198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46534/0 (socket says 172.21.15.62:46534) 2024-06-10T00:30:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a1d9b700 1 -- 172.21.15.62:0/1641803024 learned_addr learned my addr 172.21.15.62:0/1641803024 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a259c700 1 --2- 172.21.15.62:0/1641803024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a4102510 0x7f02a4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a2d9d700 1 --2- 172.21.15.62:0/1641803024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a41040b0 0x7f02a419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a1d9b700 1 -- 172.21.15.62:0/1641803024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a4102510 msgr2=0x7f02a4198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a1d9b700 1 --2- 172.21.15.62:0/1641803024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a4102510 0x7f02a4198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a1d9b700 1 -- 172.21.15.62:0/1641803024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a41040b0 msgr2=0x7f02a419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a1d9b700 1 --2- 172.21.15.62:0/1641803024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a41040b0 0x7f02a419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.634+0000 7f02a1d9b700 1 -- 172.21.15.62:0/1641803024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a41a0360 con 0x7f02a4103710 2024-06-10T00:30:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.635+0000 7f02a1d9b700 1 --2- 172.21.15.62:0/1641803024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a4103710 0x7f02a4198c40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f029800b560 tx=0x7f029800b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.635+0000 7f02937fe700 1 -- 172.21.15.62:0/1641803024 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298015020 con 0x7f02a4103710 2024-06-10T00:30:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.635+0000 7f02a8db9700 1 -- 172.21.15.62:0/1641803024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a41a0650 con 0x7f02a4103710 2024-06-10T00:30:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.635+0000 7f02937fe700 1 -- 172.21.15.62:0/1641803024 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0298011d10 con 0x7f02a4103710 2024-06-10T00:30:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.635+0000 7f02937fe700 1 -- 172.21.15.62:0/1641803024 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029800f2e0 con 0x7f02a4103710 2024-06-10T00:30:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.635+0000 7f02a8db9700 1 -- 172.21.15.62:0/1641803024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a41a0be0 con 0x7f02a4103710 2024-06-10T00:30:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.636+0000 7f02a8db9700 1 -- 172.21.15.62:0/1641803024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a4050a00 con 0x7f02a4103710 2024-06-10T00:30:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.638+0000 7f02937fe700 1 -- 172.21.15.62:0/1641803024 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0298018020 con 0x7f02a4103710 2024-06-10T00:30:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.638+0000 7f02937fe700 1 --2- 172.21.15.62:0/1641803024 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f028c0a2d60 0x7f028c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.639+0000 7f02a259c700 1 --2- 172.21.15.62:0/1641803024 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f028c0a2d60 0x7f028c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.639+0000 7f02937fe700 1 -- 172.21.15.62:0/1641803024 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f02980cf1a0 con 0x7f02a4103710 2024-06-10T00:30:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.639+0000 7f02a259c700 1 --2- 172.21.15.62:0/1641803024 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f028c0a2d60 0x7f028c0a5220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0294003890 tx=0x7f0294005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:17.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.642+0000 7f02937fe700 1 -- 172.21.15.62:0/1641803024 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f029801f020 con 0x7f02a4103710 2024-06-10T00:30:17.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.934+0000 7f02a8db9700 1 -- 172.21.15.62:0/1641803024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f02a402dca0 con 0x7f02a4103710 2024-06-10T00:30:17.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.936+0000 7f02937fe700 1 -- 172.21.15.62:0/1641803024 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f029809bd50 con 0x7f02a4103710 2024-06-10T00:30:17.936 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-06-10T00:30:17.936 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAZSWZmU6bBNxAAXnC5odJpqFGPcWCh2Ok8MA== 2024-06-10T00:30:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.938+0000 7f02a8db9700 1 -- 172.21.15.62:0/1641803024 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f028c0a2d60 msgr2=0x7f028c0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.938+0000 7f02a8db9700 1 --2- 172.21.15.62:0/1641803024 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f028c0a2d60 0x7f028c0a5220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0294003890 tx=0x7f0294005fb0 comp rx=0 tx=0).stop 2024-06-10T00:30:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.938+0000 7f02a8db9700 1 -- 172.21.15.62:0/1641803024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a4103710 msgr2=0x7f02a4198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.938+0000 7f02a8db9700 1 --2- 172.21.15.62:0/1641803024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a4103710 0x7f02a4198c40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f029800b560 tx=0x7f029800b920 comp rx=0 tx=0).stop 2024-06-10T00:30:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.938+0000 7f02a8db9700 1 -- 172.21.15.62:0/1641803024 shutdown_connections 2024-06-10T00:30:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.938+0000 7f02a8db9700 1 --2- 172.21.15.62:0/1641803024 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f028c0a2d60 0x7f028c0a5220 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.938+0000 7f02a8db9700 1 --2- 172.21.15.62:0/1641803024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a41040b0 0x7f02a419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.938+0000 7f02a8db9700 1 --2- 172.21.15.62:0/1641803024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a4102510 0x7f02a4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.938+0000 7f02a8db9700 1 --2- 172.21.15.62:0/1641803024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02a4103710 0x7f02a4198c40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.938+0000 7f02a8db9700 1 -- 172.21.15.62:0/1641803024 >> 172.21.15.62:0/1641803024 conn(0x7f02a40fdab0 msgr2=0x7f02a41069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.939+0000 7f02a8db9700 1 -- 172.21.15.62:0/1641803024 shutdown_connections 2024-06-10T00:30:17.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:17.939+0000 7f02a8db9700 1 -- 172.21.15.62:0/1641803024 wait complete. 2024-06-10T00:30:18.264 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:18 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1641803024' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-10T00:30:18.264 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:18 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1641803024' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-10T00:30:18.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:18 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1641803024' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-10T00:30:18.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:18 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1641803024' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-10T00:30:18.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:18 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1641803024' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-10T00:30:18.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:18 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1641803024' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-10T00:30:18.711 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:30:18.711 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-10T00:30:18.711 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-10T00:30:18.751 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-10T00:30:18.751 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-10T00:30:18.751 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-10T00:30:19.040 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:19.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:19 smithi062 ceph-mon[32390]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-06-10T00:30:19.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:19 smithi138 ceph-mon[36549]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-06-10T00:30:19.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:19 smithi105 ceph-mon[36321]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-06-10T00:30:20.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.744+0000 7f474c28f700 1 -- 172.21.15.62:0/279176004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47441066f0 msgr2=0x7f474410e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.744+0000 7f474c28f700 1 --2- 172.21.15.62:0/279176004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47441066f0 0x7f474410e2c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f473c0099c0 tx=0x7f473c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:30:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.744+0000 7f474c28f700 1 -- 172.21.15.62:0/279176004 shutdown_connections 2024-06-10T00:30:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.744+0000 7f474c28f700 1 --2- 172.21.15.62:0/279176004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47441066f0 0x7f474410e2c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.744+0000 7f474c28f700 1 --2- 172.21.15.62:0/279176004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4744101610 0x7f47441019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.744+0000 7f474c28f700 1 --2- 172.21.15.62:0/279176004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4744101fc0 0x7f4744106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.744+0000 7f474c28f700 1 -- 172.21.15.62:0/279176004 >> 172.21.15.62:0/279176004 conn(0x7f4744078430 msgr2=0x7f4744078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.744+0000 7f474c28f700 1 -- 172.21.15.62:0/279176004 shutdown_connections 2024-06-10T00:30:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.744+0000 7f474c28f700 1 -- 172.21.15.62:0/279176004 wait complete. 2024-06-10T00:30:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.745+0000 7f474c28f700 1 Processor -- start 2024-06-10T00:30:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.745+0000 7f474c28f700 1 -- start start 2024-06-10T00:30:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474c28f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4744101610 0x7f474419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:20.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474c28f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4744101fc0 0x7f474419dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:20.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474c28f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47441066f0 0x7f4744197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:20.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474c28f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4744111700 con 0x7f4744101610 2024-06-10T00:30:20.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474c28f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4744111580 con 0x7f4744101fc0 2024-06-10T00:30:20.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474c28f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4744111880 con 0x7f47441066f0 2024-06-10T00:30:20.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474a02b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4744101610 0x7f474419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:20.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474a02b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4744101610 0x7f474419d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37280/0 (socket says 172.21.15.62:37280) 2024-06-10T00:30:20.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474a02b700 1 -- 172.21.15.62:0/2019345965 learned_addr learned my addr 172.21.15.62:0/2019345965 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:20.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474982a700 1 --2- 172.21.15.62:0/2019345965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4744101fc0 0x7f474419dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:20.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474a82c700 1 --2- 172.21.15.62:0/2019345965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47441066f0 0x7f4744197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:20.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474a02b700 1 -- 172.21.15.62:0/2019345965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47441066f0 msgr2=0x7f4744197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:20.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474a02b700 1 --2- 172.21.15.62:0/2019345965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47441066f0 0x7f4744197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:20.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474a02b700 1 -- 172.21.15.62:0/2019345965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4744101fc0 msgr2=0x7f474419dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:20.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474a02b700 1 --2- 172.21.15.62:0/2019345965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4744101fc0 0x7f474419dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:20.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474a02b700 1 -- 172.21.15.62:0/2019345965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47441980c0 con 0x7f4744101610 2024-06-10T00:30:20.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.746+0000 7f474a02b700 1 --2- 172.21.15.62:0/2019345965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4744101610 0x7f474419d7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f474000ada0 tx=0x7f474000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:20.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.747+0000 7f473b7fe700 1 -- 172.21.15.62:0/2019345965 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740017020 con 0x7f4744101610 2024-06-10T00:30:20.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.747+0000 7f473b7fe700 1 -- 172.21.15.62:0/2019345965 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4740004bb0 con 0x7f4744101610 2024-06-10T00:30:20.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.747+0000 7f474c28f700 1 -- 172.21.15.62:0/2019345965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47441983b0 con 0x7f4744101610 2024-06-10T00:30:20.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.747+0000 7f473b7fe700 1 -- 172.21.15.62:0/2019345965 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740005560 con 0x7f4744101610 2024-06-10T00:30:20.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.747+0000 7f474c28f700 1 -- 172.21.15.62:0/2019345965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f474406abc0 con 0x7f4744101610 2024-06-10T00:30:20.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.748+0000 7f474c28f700 1 -- 172.21.15.62:0/2019345965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4744069180 con 0x7f4744101610 2024-06-10T00:30:20.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.749+0000 7f473b7fe700 1 -- 172.21.15.62:0/2019345965 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4740004d60 con 0x7f4744101610 2024-06-10T00:30:20.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.750+0000 7f473b7fe700 1 --2- 172.21.15.62:0/2019345965 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f47300a2d30 0x7f47300a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:20.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.750+0000 7f474982a700 1 --2- 172.21.15.62:0/2019345965 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f47300a2d30 0x7f47300a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:20.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.750+0000 7f473b7fe700 1 -- 172.21.15.62:0/2019345965 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f47400cf2f0 con 0x7f4744101610 2024-06-10T00:30:20.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.750+0000 7f474982a700 1 --2- 172.21.15.62:0/2019345965 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f47300a2d30 0x7f47300a51f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4734005fd0 tx=0x7f4734005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:20.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:20.754+0000 7f473b7fe700 1 -- 172.21.15.62:0/2019345965 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4740021020 con 0x7f4744101610 2024-06-10T00:30:21.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:21.021+0000 7f474c28f700 1 -- 172.21.15.62:0/2019345965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f4744198e90 con 0x7f4744101610 2024-06-10T00:30:21.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:21.026+0000 7f473b7fe700 1 -- 172.21.15.62:0/2019345965 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263201 (secure 0 0 0) 0x7f474009be20 con 0x7f4744101610 2024-06-10T00:30:21.030 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:30:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:21.029+0000 7f474c28f700 1 -- 172.21.15.62:0/2019345965 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f47300a2d30 msgr2=0x7f47300a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:21.029+0000 7f474c28f700 1 --2- 172.21.15.62:0/2019345965 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f47300a2d30 0x7f47300a51f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4734005fd0 tx=0x7f4734005d90 comp rx=0 tx=0).stop 2024-06-10T00:30:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:21.029+0000 7f474c28f700 1 -- 172.21.15.62:0/2019345965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4744101610 msgr2=0x7f474419d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:21.029+0000 7f474c28f700 1 --2- 172.21.15.62:0/2019345965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4744101610 0x7f474419d7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f474000ada0 tx=0x7f474000c330 comp rx=0 tx=0).stop 2024-06-10T00:30:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:21.030+0000 7f474c28f700 1 -- 172.21.15.62:0/2019345965 shutdown_connections 2024-06-10T00:30:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:21.030+0000 7f474c28f700 1 --2- 172.21.15.62:0/2019345965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4744101610 0x7f474419d7a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:21.030+0000 7f474c28f700 1 --2- 172.21.15.62:0/2019345965 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f47300a2d30 0x7f47300a51f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:21.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:21.030+0000 7f474c28f700 1 --2- 172.21.15.62:0/2019345965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47441066f0 0x7f4744197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:21.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:21.030+0000 7f474c28f700 1 --2- 172.21.15.62:0/2019345965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4744101fc0 0x7f474419dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:21.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:21.030+0000 7f474c28f700 1 -- 172.21.15.62:0/2019345965 >> 172.21.15.62:0/2019345965 conn(0x7f4744078430 msgr2=0x7f47440ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:21.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:21.030+0000 7f474c28f700 1 -- 172.21.15.62:0/2019345965 shutdown_connections 2024-06-10T00:30:21.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:21.030+0000 7f474c28f700 1 -- 172.21.15.62:0/2019345965 wait complete. 2024-06-10T00:30:21.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:21 smithi062 ceph-mon[32390]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-06-10T00:30:21.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:21 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2019345965' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-10T00:30:21.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:21 smithi138 ceph-mon[36549]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-06-10T00:30:21.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:21 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2019345965' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-10T00:30:21.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:21 smithi105 ceph-mon[36321]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-06-10T00:30:21.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:21 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2019345965' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-10T00:30:21.905 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":3602410963},{"type":"v1","addr":"172.21.15.62:6801","nonce":3602410963}]},"active_addr":"172.21.15.62:6801/3602410963","active_change":"2024-06-10T00:21:21.851651+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.62:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":1557827118}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":1086776708}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":2350700567}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":2528347688}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":3139301897}]}]} 2024-06-10T00:30:21.911 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-10T00:30:21.911 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-10T00:30:21.911 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:30:22.169 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:23.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:23 smithi062 ceph-mon[32390]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-10T00:30:23.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-06-10T00:30:23.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-06-10T00:30:23.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-06-10T00:30:23.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:23 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-06-10T00:30:23.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:23 smithi138 ceph-mon[36549]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-10T00:30:23.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-06-10T00:30:23.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-06-10T00:30:23.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-06-10T00:30:23.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:23 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-06-10T00:30:23.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:23 smithi105 ceph-mon[36321]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-10T00:30:23.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-06-10T00:30:23.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-06-10T00:30:23.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-06-10T00:30:23.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:23 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-06-10T00:30:24.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.031+0000 7fd8c1098700 1 -- 172.21.15.62:0/235338396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8bc101610 msgr2=0x7fd8bc1019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:24.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.031+0000 7fd8c1098700 1 --2- 172.21.15.62:0/235338396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8bc101610 0x7fd8bc1019f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0099c0 tx=0x7fd8ac009cd0 comp rx=0 tx=0).stop 2024-06-10T00:30:24.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.031+0000 7fd8c1098700 1 -- 172.21.15.62:0/235338396 shutdown_connections 2024-06-10T00:30:24.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.031+0000 7fd8c1098700 1 --2- 172.21.15.62:0/235338396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8bc101610 0x7fd8bc1019f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:24.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.031+0000 7fd8c1098700 1 --2- 172.21.15.62:0/235338396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8bc1066f0 0x7fd8bc10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:24.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.031+0000 7fd8c1098700 1 --2- 172.21.15.62:0/235338396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8bc101fc0 0x7fd8bc106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:24.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.032+0000 7fd8c1098700 1 -- 172.21.15.62:0/235338396 >> 172.21.15.62:0/235338396 conn(0x7fd8bc078430 msgr2=0x7fd8bc078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:24.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.032+0000 7fd8c1098700 1 -- 172.21.15.62:0/235338396 shutdown_connections 2024-06-10T00:30:24.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.032+0000 7fd8c1098700 1 -- 172.21.15.62:0/235338396 wait complete. 2024-06-10T00:30:24.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.033+0000 7fd8c1098700 1 Processor -- start 2024-06-10T00:30:24.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.033+0000 7fd8c1098700 1 -- start start 2024-06-10T00:30:24.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.033+0000 7fd8c1098700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8bc101610 0x7fd8bc1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:24.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.033+0000 7fd8c1098700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8bc101fc0 0x7fd8bc1035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:24.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.033+0000 7fd8c1098700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8bc1066f0 0x7fd8bc103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:24.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.033+0000 7fd8c1098700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc111750 con 0x7fd8bc101fc0 2024-06-10T00:30:24.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.033+0000 7fd8c1098700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1115d0 con 0x7fd8bc101610 2024-06-10T00:30:24.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.033+0000 7fd8c1098700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc1118d0 con 0x7fd8bc1066f0 2024-06-10T00:30:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.033+0000 7fd8ba59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8bc101fc0 0x7fd8bc1035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.033+0000 7fd8ba59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8bc101fc0 0x7fd8bc1035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37302/0 (socket says 172.21.15.62:37302) 2024-06-10T00:30:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.033+0000 7fd8ba59c700 1 -- 172.21.15.62:0/49889647 learned_addr learned my addr 172.21.15.62:0/49889647 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.033+0000 7fd8bad9d700 1 --2- 172.21.15.62:0/49889647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8bc101610 0x7fd8bc1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.033+0000 7fd8bb59e700 1 --2- 172.21.15.62:0/49889647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8bc1066f0 0x7fd8bc103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.034+0000 7fd8ba59c700 1 -- 172.21.15.62:0/49889647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8bc1066f0 msgr2=0x7fd8bc103b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.034+0000 7fd8ba59c700 1 --2- 172.21.15.62:0/49889647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8bc1066f0 0x7fd8bc103b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.034+0000 7fd8ba59c700 1 -- 172.21.15.62:0/49889647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8bc101610 msgr2=0x7fd8bc1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.034+0000 7fd8ba59c700 1 --2- 172.21.15.62:0/49889647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8bc101610 0x7fd8bc1030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.034+0000 7fd8ba59c700 1 -- 172.21.15.62:0/49889647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8bc1a7220 con 0x7fd8bc101fc0 2024-06-10T00:30:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.034+0000 7fd8ba59c700 1 --2- 172.21.15.62:0/49889647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8bc101fc0 0x7fd8bc1035f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000b500 tx=0x7fd8b000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.034+0000 7fd8a3fff700 1 -- 172.21.15.62:0/49889647 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0015020 con 0x7fd8bc101fc0 2024-06-10T00:30:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.034+0000 7fd8a3fff700 1 -- 172.21.15.62:0/49889647 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8b0011d10 con 0x7fd8bc101fc0 2024-06-10T00:30:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.034+0000 7fd8c1098700 1 -- 172.21.15.62:0/49889647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8bc1a7510 con 0x7fd8bc101fc0 2024-06-10T00:30:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.034+0000 7fd8a3fff700 1 -- 172.21.15.62:0/49889647 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b000f2e0 con 0x7fd8bc101fc0 2024-06-10T00:30:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.034+0000 7fd8c1098700 1 -- 172.21.15.62:0/49889647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8bc1a7aa0 con 0x7fd8bc101fc0 2024-06-10T00:30:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.035+0000 7fd8c1098700 1 -- 172.21.15.62:0/49889647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8bc069180 con 0x7fd8bc101fc0 2024-06-10T00:30:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.036+0000 7fd8a3fff700 1 -- 172.21.15.62:0/49889647 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd8b0018020 con 0x7fd8bc101fc0 2024-06-10T00:30:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.037+0000 7fd8a3fff700 1 --2- 172.21.15.62:0/49889647 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd8a40a2d30 0x7fd8a40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.037+0000 7fd8bad9d700 1 --2- 172.21.15.62:0/49889647 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd8a40a2d30 0x7fd8a40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.037+0000 7fd8a3fff700 1 -- 172.21.15.62:0/49889647 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9208+0+0 (secure 0 0 0) 0x7fd8b00ceb70 con 0x7fd8bc101fc0 2024-06-10T00:30:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.037+0000 7fd8bad9d700 1 --2- 172.21.15.62:0/49889647 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd8a40a2d30 0x7fd8a40a51f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009fd0 tx=0x7fd8ac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:24.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.041+0000 7fd8a3fff700 1 -- 172.21.15.62:0/49889647 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd8b001f020 con 0x7fd8bc101fc0 2024-06-10T00:30:24.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.257+0000 7fd8c1098700 1 -- 172.21.15.62:0/49889647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8bc1045e0 con 0x7fd8bc101fc0 2024-06-10T00:30:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.259+0000 7fd8a3fff700 1 -- 172.21.15.62:0/49889647 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24839 (secure 0 0 0) 0x7fd8b0079e20 con 0x7fd8bc101fc0 2024-06-10T00:30:24.259 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:30:24.259 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":99,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:30:24.104116+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:30:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.261+0000 7fd8c1098700 1 -- 172.21.15.62:0/49889647 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd8a40a2d30 msgr2=0x7fd8a40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.261+0000 7fd8c1098700 1 --2- 172.21.15.62:0/49889647 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd8a40a2d30 0x7fd8a40a51f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009fd0 tx=0x7fd8ac005fb0 comp rx=0 tx=0).stop 2024-06-10T00:30:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.261+0000 7fd8c1098700 1 -- 172.21.15.62:0/49889647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8bc101fc0 msgr2=0x7fd8bc1035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.261+0000 7fd8c1098700 1 --2- 172.21.15.62:0/49889647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8bc101fc0 0x7fd8bc1035f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000b500 tx=0x7fd8b000b810 comp rx=0 tx=0).stop 2024-06-10T00:30:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.261+0000 7fd8c1098700 1 -- 172.21.15.62:0/49889647 shutdown_connections 2024-06-10T00:30:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.261+0000 7fd8c1098700 1 --2- 172.21.15.62:0/49889647 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd8a40a2d30 0x7fd8a40a51f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.261+0000 7fd8c1098700 1 --2- 172.21.15.62:0/49889647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8bc1066f0 0x7fd8bc103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.261+0000 7fd8c1098700 1 --2- 172.21.15.62:0/49889647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8bc101610 0x7fd8bc1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.261+0000 7fd8c1098700 1 --2- 172.21.15.62:0/49889647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8bc101fc0 0x7fd8bc1035f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.261+0000 7fd8c1098700 1 -- 172.21.15.62:0/49889647 >> 172.21.15.62:0/49889647 conn(0x7fd8bc078430 msgr2=0x7fd8bc1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.261+0000 7fd8c1098700 1 -- 172.21.15.62:0/49889647 shutdown_connections 2024-06-10T00:30:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:24.261+0000 7fd8c1098700 1 -- 172.21.15.62:0/49889647 wait complete. 2024-06-10T00:30:24.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-06-10T00:30:24.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-06-10T00:30:24.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-06-10T00:30:24.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:24 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-06-10T00:30:24.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:24 smithi062 ceph-mon[32390]: osdmap e98: 12 total, 12 up, 12 in 2024-06-10T00:30:24.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-06-10T00:30:24.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-06-10T00:30:24.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-06-10T00:30:24.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:24 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-06-10T00:30:24.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:24 smithi138 ceph-mon[36549]: osdmap e98: 12 total, 12 up, 12 in 2024-06-10T00:30:24.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-06-10T00:30:24.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-06-10T00:30:24.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-06-10T00:30:24.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:24 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-06-10T00:30:24.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:24 smithi105 ceph-mon[36321]: osdmap e98: 12 total, 12 up, 12 in 2024-06-10T00:30:24.976 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-10T00:30:24.976 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:30:25.229 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:25.351 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:25 smithi062 ceph-mon[32390]: pgmap v359: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-10T00:30:25.351 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:25 smithi062 ceph-mon[32390]: osdmap e99: 12 total, 12 up, 12 in 2024-06-10T00:30:25.351 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:25 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/49889647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:30:25.351 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:25 smithi062 ceph-mon[32390]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-10T00:30:25.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:25 smithi138 ceph-mon[36549]: pgmap v359: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-10T00:30:25.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:25 smithi138 ceph-mon[36549]: osdmap e99: 12 total, 12 up, 12 in 2024-06-10T00:30:25.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:25 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/49889647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:30:25.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:25 smithi138 ceph-mon[36549]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-10T00:30:25.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:25 smithi105 ceph-mon[36321]: pgmap v359: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-10T00:30:25.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:25 smithi105 ceph-mon[36321]: osdmap e99: 12 total, 12 up, 12 in 2024-06-10T00:30:25.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:25 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/49889647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:30:25.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:25 smithi105 ceph-mon[36321]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-10T00:30:27.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.033+0000 7fcd8635b700 1 -- 172.21.15.62:0/3020577491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd8006b520 msgr2=0x7fcd8010f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:27.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.033+0000 7fcd8635b700 1 --2- 172.21.15.62:0/3020577491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd8006b520 0x7fcd8010f340 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800b0a0 tx=0x7fcd7800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:30:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.033+0000 7fcd8635b700 1 -- 172.21.15.62:0/3020577491 shutdown_connections 2024-06-10T00:30:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.033+0000 7fcd8635b700 1 --2- 172.21.15.62:0/3020577491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd8006b520 0x7fcd8010f340 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.033+0000 7fcd8635b700 1 --2- 172.21.15.62:0/3020577491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd80108520 0x7fcd80108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.033+0000 7fcd8635b700 1 --2- 172.21.15.62:0/3020577491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd8006ab60 0x7fcd8006afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.033+0000 7fcd8635b700 1 -- 172.21.15.62:0/3020577491 >> 172.21.15.62:0/3020577491 conn(0x7fcd80078340 msgr2=0x7fcd80078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.033+0000 7fcd8635b700 1 -- 172.21.15.62:0/3020577491 shutdown_connections 2024-06-10T00:30:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.033+0000 7fcd8635b700 1 -- 172.21.15.62:0/3020577491 wait complete. 2024-06-10T00:30:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.034+0000 7fcd8635b700 1 Processor -- start 2024-06-10T00:30:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.034+0000 7fcd8635b700 1 -- start start 2024-06-10T00:30:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.034+0000 7fcd8635b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd8006ab60 0x7fcd8019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:27.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.034+0000 7fcd8635b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd8006b520 0x7fcd8019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.034+0000 7fcd8635b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd80108520 0x7fcd80198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.034+0000 7fcd8635b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcd80112800 con 0x7fcd8006b520 2024-06-10T00:30:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.034+0000 7fcd8635b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd80112680 con 0x7fcd8006ab60 2024-06-10T00:30:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.034+0000 7fcd8635b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd80112980 con 0x7fcd80108520 2024-06-10T00:30:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.034+0000 7fcd7f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd8006b520 0x7fcd8019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.034+0000 7fcd7f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd8006b520 0x7fcd8019edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37332/0 (socket says 172.21.15.62:37332) 2024-06-10T00:30:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.034+0000 7fcd7f7fe700 1 -- 172.21.15.62:0/4226334254 learned_addr learned my addr 172.21.15.62:0/4226334254 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.034+0000 7fcd7ffff700 1 --2- 172.21.15.62:0/4226334254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd8006ab60 0x7fcd8019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.034+0000 7fcd848f8700 1 --2- 172.21.15.62:0/4226334254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd80108520 0x7fcd80198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.035+0000 7fcd7f7fe700 1 -- 172.21.15.62:0/4226334254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd80108520 msgr2=0x7fcd80198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.035+0000 7fcd7f7fe700 1 --2- 172.21.15.62:0/4226334254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd80108520 0x7fcd80198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.035+0000 7fcd7f7fe700 1 -- 172.21.15.62:0/4226334254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd8006ab60 msgr2=0x7fcd8019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:27.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.035+0000 7fcd7f7fe700 1 --2- 172.21.15.62:0/4226334254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd8006ab60 0x7fcd8019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:27.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.035+0000 7fcd7f7fe700 1 -- 172.21.15.62:0/4226334254 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd801991a0 con 0x7fcd8006b520 2024-06-10T00:30:27.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.035+0000 7fcd7f7fe700 1 --2- 172.21.15.62:0/4226334254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd8006b520 0x7fcd8019edc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400b530 tx=0x7fcd7400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:27.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.035+0000 7fcd7d7fa700 1 -- 172.21.15.62:0/4226334254 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74015020 con 0x7fcd8006b520 2024-06-10T00:30:27.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.035+0000 7fcd7d7fa700 1 -- 172.21.15.62:0/4226334254 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd74011d10 con 0x7fcd8006b520 2024-06-10T00:30:27.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.035+0000 7fcd8635b700 1 -- 172.21.15.62:0/4226334254 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd80199490 con 0x7fcd8006b520 2024-06-10T00:30:27.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.035+0000 7fcd8635b700 1 -- 172.21.15.62:0/4226334254 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd801a5750 con 0x7fcd8006b520 2024-06-10T00:30:27.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.035+0000 7fcd7d7fa700 1 -- 172.21.15.62:0/4226334254 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7400f2e0 con 0x7fcd8006b520 2024-06-10T00:30:27.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.036+0000 7fcd8635b700 1 -- 172.21.15.62:0/4226334254 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd80103be0 con 0x7fcd8006b520 2024-06-10T00:30:27.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.037+0000 7fcd7d7fa700 1 -- 172.21.15.62:0/4226334254 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcd74018020 con 0x7fcd8006b520 2024-06-10T00:30:27.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.038+0000 7fcd7d7fa700 1 --2- 172.21.15.62:0/4226334254 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcd680a2cb0 0x7fcd680a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:27.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.039+0000 7fcd7ffff700 1 --2- 172.21.15.62:0/4226334254 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcd680a2cb0 0x7fcd680a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:27.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.039+0000 7fcd7d7fa700 1 -- 172.21.15.62:0/4226334254 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fcd740cf110 con 0x7fcd8006b520 2024-06-10T00:30:27.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.039+0000 7fcd7ffff700 1 --2- 172.21.15.62:0/4226334254 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcd680a2cb0 0x7fcd680a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005950 tx=0x7fcd7000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:27.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.044+0000 7fcd7d7fa700 1 -- 172.21.15.62:0/4226334254 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcd7401f020 con 0x7fcd8006b520 2024-06-10T00:30:27.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.251+0000 7fcd8635b700 1 -- 172.21.15.62:0/4226334254 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd80199fa0 con 0x7fcd8006b520 2024-06-10T00:30:27.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.252+0000 7fcd7d7fa700 1 -- 172.21.15.62:0/4226334254 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24839 (secure 0 0 0) 0x7fcd80199fa0 con 0x7fcd8006b520 2024-06-10T00:30:27.253 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:30:27.253 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":99,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:30:24.104116+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:30:27.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.255+0000 7fcd8635b700 1 -- 172.21.15.62:0/4226334254 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcd680a2cb0 msgr2=0x7fcd680a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:27.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.255+0000 7fcd8635b700 1 --2- 172.21.15.62:0/4226334254 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcd680a2cb0 0x7fcd680a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005950 tx=0x7fcd7000a300 comp rx=0 tx=0).stop 2024-06-10T00:30:27.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.255+0000 7fcd8635b700 1 -- 172.21.15.62:0/4226334254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd8006b520 msgr2=0x7fcd8019edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:27.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.255+0000 7fcd8635b700 1 --2- 172.21.15.62:0/4226334254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd8006b520 0x7fcd8019edc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400b530 tx=0x7fcd7400b840 comp rx=0 tx=0).stop 2024-06-10T00:30:27.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.255+0000 7fcd8635b700 1 -- 172.21.15.62:0/4226334254 shutdown_connections 2024-06-10T00:30:27.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.255+0000 7fcd8635b700 1 --2- 172.21.15.62:0/4226334254 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcd680a2cb0 0x7fcd680a5170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:27.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.255+0000 7fcd8635b700 1 --2- 172.21.15.62:0/4226334254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd80108520 0x7fcd80198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:27.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.255+0000 7fcd8635b700 1 --2- 172.21.15.62:0/4226334254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd8006ab60 0x7fcd8019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:27.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.255+0000 7fcd8635b700 1 --2- 172.21.15.62:0/4226334254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd8006b520 0x7fcd8019edc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:27.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.255+0000 7fcd8635b700 1 -- 172.21.15.62:0/4226334254 >> 172.21.15.62:0/4226334254 conn(0x7fcd80078340 msgr2=0x7fcd801003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:27.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.255+0000 7fcd8635b700 1 -- 172.21.15.62:0/4226334254 shutdown_connections 2024-06-10T00:30:27.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:27.255+0000 7fcd8635b700 1 -- 172.21.15.62:0/4226334254 wait complete. 2024-06-10T00:30:27.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:27 smithi062 ceph-mon[32390]: pgmap v361: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-10T00:30:27.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:27 smithi138 ceph-mon[36549]: pgmap v361: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-10T00:30:27.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:27 smithi105 ceph-mon[36321]: pgmap v361: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-10T00:30:27.929 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-10T00:30:27.930 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-10T00:30:27.930 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-10T00:30:27.931 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-10T00:30:27.931 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-10T00:30:27.932 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-10T00:30:27.932 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-10T00:30:27.932 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-10T00:30:27.933 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-10T00:30:27.933 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-10T00:30:27.934 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-10T00:30:27.934 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-10T00:30:28.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:28 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/4226334254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:30:28.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:28 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/4226334254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:30:28.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:28 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/4226334254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:30:28.579 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:28.718 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:29.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:29 smithi062 ceph-mon[32390]: pgmap v362: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-10T00:30:29.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:29 smithi105 ceph-mon[36321]: pgmap v362: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-10T00:30:29.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:29 smithi138 ceph-mon[36549]: pgmap v362: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-10T00:30:29.850 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:29.864 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:29.865 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:29.869 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:29.869 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:29.870 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:29.871 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:29.878 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:29.885 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:30.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:30 smithi062 ceph-mon[32390]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-10T00:30:30.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:30 smithi062 ceph-mon[32390]: Cluster is now healthy 2024-06-10T00:30:30.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:30 smithi105 ceph-mon[36321]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-10T00:30:30.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:30 smithi105 ceph-mon[36321]: Cluster is now healthy 2024-06-10T00:30:30.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:30 smithi138 ceph-mon[36549]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-10T00:30:30.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:30 smithi138 ceph-mon[36549]: Cluster is now healthy 2024-06-10T00:30:31.349 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:31 smithi062 ceph-mon[32390]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-06-10T00:30:31.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:31 smithi138 ceph-mon[36549]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-06-10T00:30:31.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:31 smithi105 ceph-mon[36321]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-06-10T00:30:33.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:33 smithi062 ceph-mon[32390]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-06-10T00:30:33.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:33 smithi105 ceph-mon[36321]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-06-10T00:30:33.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:33 smithi138 ceph-mon[36549]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-06-10T00:30:35.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:35 smithi062 ceph-mon[32390]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-10T00:30:35.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:35 smithi105 ceph-mon[36321]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-10T00:30:35.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:35 smithi138 ceph-mon[36549]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-10T00:30:36.211 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:36 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:30:36.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:36 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:30:36.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:36 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:30:37.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:37 smithi062 ceph-mon[32390]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:30:37.424 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:37.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:37 smithi105 ceph-mon[36321]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:30:37.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:37 smithi138 ceph-mon[36549]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:30:37.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.843+0000 7fdd50793700 1 -- 172.21.15.62:0/1601510157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4806ab30 msgr2=0x7fdd4810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:37.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.843+0000 7fdd50793700 1 --2- 172.21.15.62:0/1601510157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4806ab30 0x7fdd4810f320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000b0a0 tx=0x7fdd4000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:30:37.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.843+0000 7fdd50793700 1 -- 172.21.15.62:0/1601510157 shutdown_connections 2024-06-10T00:30:37.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.843+0000 7fdd50793700 1 --2- 172.21.15.62:0/1601510157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4806ab30 0x7fdd4810f320 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:37.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.843+0000 7fdd50793700 1 --2- 172.21.15.62:0/1601510157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd48108500 0x7fdd481088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:37.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.843+0000 7fdd50793700 1 --2- 172.21.15.62:0/1601510157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd4806a170 0x7fdd4806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:37.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.843+0000 7fdd50793700 1 -- 172.21.15.62:0/1601510157 >> 172.21.15.62:0/1601510157 conn(0x7fdd48078340 msgr2=0x7fdd48078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:37.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.843+0000 7fdd50793700 1 -- 172.21.15.62:0/1601510157 shutdown_connections 2024-06-10T00:30:37.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.843+0000 7fdd50793700 1 -- 172.21.15.62:0/1601510157 wait complete. 2024-06-10T00:30:37.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd50793700 1 Processor -- start 2024-06-10T00:30:37.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd50793700 1 -- start start 2024-06-10T00:30:37.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd50793700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd4806a170 0x7fdd48101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:37.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd50793700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd4806ab30 0x7fdd481019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:37.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd50793700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd48108500 0x7fdd48101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:37.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd50793700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdd48112760 con 0x7fdd48108500 2024-06-10T00:30:37.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd50793700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdd481125e0 con 0x7fdd4806ab30 2024-06-10T00:30:37.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd50793700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd481128e0 con 0x7fdd4806a170 2024-06-10T00:30:37.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd4ed30700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd48108500 0x7fdd48101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:37.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd4ed30700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd48108500 0x7fdd48101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36984/0 (socket says 172.21.15.62:36984) 2024-06-10T00:30:37.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd4ed30700 1 -- 172.21.15.62:0/3007715435 learned_addr learned my addr 172.21.15.62:0/3007715435 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd4e52f700 1 --2- 172.21.15.62:0/3007715435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd4806a170 0x7fdd48101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd4dd2e700 1 --2- 172.21.15.62:0/3007715435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd4806ab30 0x7fdd481019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd4ed30700 1 -- 172.21.15.62:0/3007715435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd4806a170 msgr2=0x7fdd48101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd4ed30700 1 --2- 172.21.15.62:0/3007715435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd4806a170 0x7fdd48101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd4ed30700 1 -- 172.21.15.62:0/3007715435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd4806ab30 msgr2=0x7fdd481019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd4ed30700 1 --2- 172.21.15.62:0/3007715435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd4806ab30 0x7fdd481019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.844+0000 7fdd4ed30700 1 -- 172.21.15.62:0/3007715435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd481a7140 con 0x7fdd48108500 2024-06-10T00:30:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.845+0000 7fdd4ed30700 1 --2- 172.21.15.62:0/3007715435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd48108500 0x7fdd48101ee0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdd400062a0 tx=0x7fdd40004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.845+0000 7fdd3f7fe700 1 -- 172.21.15.62:0/3007715435 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40021070 con 0x7fdd48108500 2024-06-10T00:30:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.845+0000 7fdd3f7fe700 1 -- 172.21.15.62:0/3007715435 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd40003c00 con 0x7fdd48108500 2024-06-10T00:30:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.845+0000 7fdd3f7fe700 1 -- 172.21.15.62:0/3007715435 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40004880 con 0x7fdd48108500 2024-06-10T00:30:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.845+0000 7fdd50793700 1 -- 172.21.15.62:0/3007715435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd481a73d0 con 0x7fdd48108500 2024-06-10T00:30:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.845+0000 7fdd50793700 1 -- 172.21.15.62:0/3007715435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd481a7830 con 0x7fdd48108500 2024-06-10T00:30:37.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.846+0000 7fdd50793700 1 -- 172.21.15.62:0/3007715435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdd48062ba0 con 0x7fdd48108500 2024-06-10T00:30:37.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.847+0000 7fdd3f7fe700 1 -- 172.21.15.62:0/3007715435 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdd40007980 con 0x7fdd48108500 2024-06-10T00:30:37.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.848+0000 7fdd3f7fe700 1 --2- 172.21.15.62:0/3007715435 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdd340a2d30 0x7fdd340a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:37.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.848+0000 7fdd4e52f700 1 --2- 172.21.15.62:0/3007715435 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdd340a2d30 0x7fdd340a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:37.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.848+0000 7fdd3f7fe700 1 -- 172.21.15.62:0/3007715435 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fdd4000f020 con 0x7fdd48108500 2024-06-10T00:30:37.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.848+0000 7fdd3f7fe700 1 --2- 172.21.15.62:0/3007715435 >> [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087] conn(0x7fdd340ae090 0x7fdd340b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:37.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.848+0000 7fdd3f7fe700 1 -- 172.21.15.62:0/3007715435 --> [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd40008ea0 con 0x7fdd340ae090 2024-06-10T00:30:37.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.848+0000 7fdd3f7fe700 1 -- 172.21.15.62:0/3007715435 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fdd400d10e0 con 0x7fdd48108500 2024-06-10T00:30:37.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.848+0000 7fdd4e52f700 1 --2- 172.21.15.62:0/3007715435 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdd340a2d30 0x7fdd340a51f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd44007b60 tx=0x7fdd440073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:37.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.848+0000 7fdd4dd2e700 1 --2- 172.21.15.62:0/3007715435 >> [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087] conn(0x7fdd340ae090 0x7fdd340b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:37.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.849+0000 7fdd4dd2e700 1 --2- 172.21.15.62:0/3007715435 >> [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087] conn(0x7fdd340ae090 0x7fdd340b0530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:37.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.849+0000 7fdd3f7fe700 1 -- 172.21.15.62:0/3007715435 <== osd.7 v2:172.21.15.105:6816/3912053087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdd40008ea0 con 0x7fdd340ae090 2024-06-10T00:30:37.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.866+0000 7fdd50793700 1 -- 172.21.15.62:0/3007715435 --> [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdd48062ff0 con 0x7fdd340ae090 2024-06-10T00:30:37.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.866+0000 7fdd3f7fe700 1 -- 172.21.15.62:0/3007715435 <== osd.7 v2:172.21.15.105:6816/3912053087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdd48062ff0 con 0x7fdd340ae090 2024-06-10T00:30:37.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.867+0000 7fdd50793700 1 -- 172.21.15.62:0/3007715435 >> [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087] conn(0x7fdd340ae090 msgr2=0x7fdd340b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:37.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.867+0000 7fdd50793700 1 --2- 172.21.15.62:0/3007715435 >> [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087] conn(0x7fdd340ae090 0x7fdd340b0530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:37.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.867+0000 7fdd50793700 1 -- 172.21.15.62:0/3007715435 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdd340a2d30 msgr2=0x7fdd340a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:37.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.867+0000 7fdd50793700 1 --2- 172.21.15.62:0/3007715435 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fdd340a2d30 0x7fdd340a51f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd44007b60 tx=0x7fdd440073d0 comp rx=0 tx=0).stop 2024-06-10T00:30:37.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.867+0000 7fdd50793700 1 -- 172.21.15.62:0/3007715435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd48108500 msgr2=0x7fdd48101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:37.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.867+0000 7fdd50793700 1 --2- 172.21.15.62:0/3007715435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd48108500 0x7fdd48101ee0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdd400062a0 tx=0x7fdd40004d70 comp rx=0 tx=0).stop 2024-06-10T00:30:37.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.867+0000 7fdd4ed30700 1 -- 172.21.15.62:0/3007715435 reap_dead start 2024-06-10T00:30:37.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.867+0000 7fdd50793700 1 -- 172.21.15.62:0/3007715435 shutdown_connections 2024-06-10T00:30:37.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.867+0000 7fdd50793700 1 -- 172.21.15.62:0/3007715435 >> 172.21.15.62:0/3007715435 conn(0x7fdd48078340 msgr2=0x7fdd480fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:37.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.867+0000 7fdd50793700 1 -- 172.21.15.62:0/3007715435 shutdown_connections 2024-06-10T00:30:37.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:37.867+0000 7fdd50793700 1 -- 172.21.15.62:0/3007715435 wait complete. 2024-06-10T00:30:38.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.027+0000 7f81c8d52700 1 -- 172.21.15.62:0/4012648616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81c4107400 msgr2=0x7f81c41077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:38.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.027+0000 7f81c8d52700 1 --2- 172.21.15.62:0/4012648616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81c4107400 0x7f81c41077e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009990 tx=0x7f81b4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:30:38.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.027+0000 7f81c8d52700 1 -- 172.21.15.62:0/4012648616 shutdown_connections 2024-06-10T00:30:38.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.027+0000 7f81c8d52700 1 --2- 172.21.15.62:0/4012648616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81c4107400 0x7f81c41077e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:38.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.027+0000 7f81c8d52700 1 --2- 172.21.15.62:0/4012648616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81c4101d50 0x7f81c410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:38.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.027+0000 7f81c8d52700 1 --2- 172.21.15.62:0/4012648616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c4101390 0x7f81c4101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:38.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.027+0000 7f81c8d52700 1 -- 172.21.15.62:0/4012648616 >> 172.21.15.62:0/4012648616 conn(0x7f81c4078340 msgr2=0x7f81c4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:38.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.027+0000 7f81c8d52700 1 -- 172.21.15.62:0/4012648616 shutdown_connections 2024-06-10T00:30:38.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.027+0000 7f81c8d52700 1 -- 172.21.15.62:0/4012648616 wait complete. 2024-06-10T00:30:38.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.028+0000 7f81c8d52700 1 Processor -- start 2024-06-10T00:30:38.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.028+0000 7f81c8d52700 1 -- start start 2024-06-10T00:30:38.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.028+0000 7f81c8d52700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c4101390 0x7f81c419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:38.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.028+0000 7f81c8d52700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81c4101d50 0x7f81c419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:38.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.028+0000 7f81c8d52700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81c4107400 0x7f81c4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:38.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.028+0000 7f81c8d52700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41116a0 con 0x7f81c4101d50 2024-06-10T00:30:38.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.028+0000 7f81c8d52700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f81c4111520 con 0x7f81c4101390 2024-06-10T00:30:38.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.028+0000 7f81c8d52700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81c4111820 con 0x7f81c4107400 2024-06-10T00:30:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.028+0000 7f81c1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81c4101d50 0x7f81c419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81c259c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c4101390 0x7f81c419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81c1d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81c4101d50 0x7f81c419edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36996/0 (socket says 172.21.15.62:36996) 2024-06-10T00:30:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81c259c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c4101390 0x7f81c419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:35186/0 (socket says 172.21.15.62:35186) 2024-06-10T00:30:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81c1d9b700 1 -- 172.21.15.62:0/2225814445 learned_addr learned my addr 172.21.15.62:0/2225814445 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81c2d9d700 1 --2- 172.21.15.62:0/2225814445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81c4107400 0x7f81c4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81c1d9b700 1 -- 172.21.15.62:0/2225814445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81c4107400 msgr2=0x7f81c4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81c1d9b700 1 --2- 172.21.15.62:0/2225814445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81c4107400 0x7f81c4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81c1d9b700 1 -- 172.21.15.62:0/2225814445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c4101390 msgr2=0x7f81c419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81c1d9b700 1 --2- 172.21.15.62:0/2225814445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c4101390 0x7f81c419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81c1d9b700 1 -- 172.21.15.62:0/2225814445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c41991a0 con 0x7f81c4101d50 2024-06-10T00:30:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81c259c700 1 --2- 172.21.15.62:0/2225814445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c4101390 0x7f81c419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:30:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81c1d9b700 1 --2- 172.21.15.62:0/2225814445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81c4101d50 0x7f81c419edc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f81b800e9a0 tx=0x7f81b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81b37fe700 1 -- 172.21.15.62:0/2225814445 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b800ca10 con 0x7f81c4101d50 2024-06-10T00:30:38.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81b37fe700 1 -- 172.21.15.62:0/2225814445 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81b800cbb0 con 0x7f81c4101d50 2024-06-10T00:30:38.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81c8d52700 1 -- 172.21.15.62:0/2225814445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c4199490 con 0x7f81c4101d50 2024-06-10T00:30:38.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81b37fe700 1 -- 172.21.15.62:0/2225814445 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8018680 con 0x7f81c4101d50 2024-06-10T00:30:38.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.029+0000 7f81c8d52700 1 -- 172.21.15.62:0/2225814445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c41a5750 con 0x7f81c4101d50 2024-06-10T00:30:38.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.030+0000 7f81c8d52700 1 -- 172.21.15.62:0/2225814445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f81c41054c0 con 0x7f81c4101d50 2024-06-10T00:30:38.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.031+0000 7f81b37fe700 1 -- 172.21.15.62:0/2225814445 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f81b8010430 con 0x7f81c4101d50 2024-06-10T00:30:38.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.032+0000 7f81b37fe700 1 --2- 172.21.15.62:0/2225814445 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f81ac0a2cb0 0x7f81ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:38.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.032+0000 7f81c259c700 1 --2- 172.21.15.62:0/2225814445 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f81ac0a2cb0 0x7f81ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:38.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.032+0000 7f81b37fe700 1 -- 172.21.15.62:0/2225814445 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f81b80cffc0 con 0x7f81c4101d50 2024-06-10T00:30:38.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.033+0000 7f81b37fe700 1 --2- 172.21.15.62:0/2225814445 >> [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246] conn(0x7f81ac0ae010 0x7f81ac0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:38.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.033+0000 7f81b37fe700 1 -- 172.21.15.62:0/2225814445 --> [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81b801bec0 con 0x7f81ac0ae010 2024-06-10T00:30:38.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.033+0000 7f81b37fe700 1 -- 172.21.15.62:0/2225814445 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f81b80d0410 con 0x7f81c4101d50 2024-06-10T00:30:38.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.033+0000 7f81c2d9d700 1 --2- 172.21.15.62:0/2225814445 >> [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246] conn(0x7f81ac0ae010 0x7f81ac0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:38.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.033+0000 7f81c259c700 1 --2- 172.21.15.62:0/2225814445 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f81ac0a2cb0 0x7f81ac0a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f81b400b2e0 tx=0x7f81b4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:38.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.033+0000 7f81c2d9d700 1 --2- 172.21.15.62:0/2225814445 >> [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246] conn(0x7f81ac0ae010 0x7f81ac0b04b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:38.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.034+0000 7f81b37fe700 1 -- 172.21.15.62:0/2225814445 <== osd.0 v2:172.21.15.62:6802/2865393246 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f81b801bec0 con 0x7f81ac0ae010 2024-06-10T00:30:38.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.050+0000 7f81c8d52700 1 -- 172.21.15.62:0/2225814445 --> [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f81c4102ab0 con 0x7f81ac0ae010 2024-06-10T00:30:38.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.051+0000 7f81b37fe700 1 -- 172.21.15.62:0/2225814445 <== osd.0 v2:172.21.15.62:6802/2865393246 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f81c4102ab0 con 0x7f81ac0ae010 2024-06-10T00:30:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.051+0000 7f81c8d52700 1 -- 172.21.15.62:0/2225814445 >> [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246] conn(0x7f81ac0ae010 msgr2=0x7f81ac0b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.051+0000 7f81c8d52700 1 --2- 172.21.15.62:0/2225814445 >> [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246] conn(0x7f81ac0ae010 0x7f81ac0b04b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.051+0000 7f81c8d52700 1 -- 172.21.15.62:0/2225814445 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f81ac0a2cb0 msgr2=0x7f81ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.051+0000 7f81c8d52700 1 --2- 172.21.15.62:0/2225814445 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f81ac0a2cb0 0x7f81ac0a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f81b400b2e0 tx=0x7f81b4005960 comp rx=0 tx=0).stop 2024-06-10T00:30:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.052+0000 7f81c8d52700 1 -- 172.21.15.62:0/2225814445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81c4101d50 msgr2=0x7f81c419edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.052+0000 7f81c8d52700 1 --2- 172.21.15.62:0/2225814445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81c4101d50 0x7f81c419edc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f81b800e9a0 tx=0x7f81b800ecb0 comp rx=0 tx=0).stop 2024-06-10T00:30:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.052+0000 7f81c2d9d700 1 -- 172.21.15.62:0/2225814445 reap_dead start 2024-06-10T00:30:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.052+0000 7f81c8d52700 1 -- 172.21.15.62:0/2225814445 shutdown_connections 2024-06-10T00:30:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.052+0000 7f81c8d52700 1 -- 172.21.15.62:0/2225814445 >> 172.21.15.62:0/2225814445 conn(0x7f81c4078340 msgr2=0x7f81c4100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.052+0000 7f81c8d52700 1 -- 172.21.15.62:0/2225814445 shutdown_connections 2024-06-10T00:30:38.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:38.052+0000 7f81c8d52700 1 -- 172.21.15.62:0/2225814445 wait complete. 2024-06-10T00:30:38.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:38 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:30:38.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:38 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:30:38.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:38 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:30:38.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:38 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:30:38.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:38 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:30:38.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:38 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:30:38.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:38 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:30:38.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:38 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:30:39.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:38 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:30:39.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:38 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:30:39.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:38 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:30:39.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:38 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:30:39.970 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:39 smithi062 ceph-mon[32390]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:30:40.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:39 smithi105 ceph-mon[36321]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:30:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:39 smithi138 ceph-mon[36549]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:30:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.473+0000 7ffad42c0700 1 -- 172.21.15.62:0/3118087192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffacc107840 msgr2=0x7ffacc10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.473+0000 7ffad42c0700 1 --2- 172.21.15.62:0/3118087192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffacc107840 0x7ffacc10f3c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffac400b0d0 tx=0x7ffac400b3e0 comp rx=0 tx=0).stop 2024-06-10T00:30:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.473+0000 7ffad42c0700 1 -- 172.21.15.62:0/3118087192 shutdown_connections 2024-06-10T00:30:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.473+0000 7ffad42c0700 1 --2- 172.21.15.62:0/3118087192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffacc107840 0x7ffacc10f3c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.473+0000 7ffad42c0700 1 --2- 172.21.15.62:0/3118087192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffacc102740 0x7ffacc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.473+0000 7ffad42c0700 1 --2- 172.21.15.62:0/3118087192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffacc1030f0 0x7ffacc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:40.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.473+0000 7ffad42c0700 1 -- 172.21.15.62:0/3118087192 >> 172.21.15.62:0/3118087192 conn(0x7ffacc0795f0 msgr2=0x7ffacc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:40.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.473+0000 7ffad42c0700 1 -- 172.21.15.62:0/3118087192 shutdown_connections 2024-06-10T00:30:40.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.473+0000 7ffad42c0700 1 -- 172.21.15.62:0/3118087192 wait complete. 2024-06-10T00:30:40.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.473+0000 7ffad42c0700 1 Processor -- start 2024-06-10T00:30:40.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.473+0000 7ffad42c0700 1 -- start start 2024-06-10T00:30:40.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad42c0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffacc102740 0x7ffacc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:40.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad42c0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffacc1030f0 0x7ffacc074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:40.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad42c0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffacc107840 0x7ffacc075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:40.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad42c0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc112780 con 0x7ffacc107840 2024-06-10T00:30:40.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad42c0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc112600 con 0x7ffacc1030f0 2024-06-10T00:30:40.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad42c0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc112900 con 0x7ffacc102740 2024-06-10T00:30:40.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad285d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffacc107840 0x7ffacc075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:40.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad285d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffacc107840 0x7ffacc075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37010/0 (socket says 172.21.15.62:37010) 2024-06-10T00:30:40.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad285d700 1 -- 172.21.15.62:0/1618030008 learned_addr learned my addr 172.21.15.62:0/1618030008 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:40.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad205c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffacc102740 0x7ffacc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:40.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad185b700 1 --2- 172.21.15.62:0/1618030008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffacc1030f0 0x7ffacc074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:40.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad285d700 1 -- 172.21.15.62:0/1618030008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffacc102740 msgr2=0x7ffacc074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:40.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad285d700 1 --2- 172.21.15.62:0/1618030008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffacc102740 0x7ffacc074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:40.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad285d700 1 -- 172.21.15.62:0/1618030008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffacc1030f0 msgr2=0x7ffacc074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:40.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad285d700 1 --2- 172.21.15.62:0/1618030008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffacc1030f0 0x7ffacc074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:40.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad285d700 1 -- 172.21.15.62:0/1618030008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffacc1a70c0 con 0x7ffacc107840 2024-06-10T00:30:40.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.474+0000 7ffad285d700 1 --2- 172.21.15.62:0/1618030008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffacc107840 0x7ffacc075210 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffac4006070 tx=0x7ffac4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.475+0000 7ffac37fe700 1 -- 172.21.15.62:0/1618030008 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4021070 con 0x7ffacc107840 2024-06-10T00:30:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.475+0000 7ffac37fe700 1 -- 172.21.15.62:0/1618030008 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac4007b80 con 0x7ffacc107840 2024-06-10T00:30:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.475+0000 7ffad42c0700 1 -- 172.21.15.62:0/1618030008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffacc1a7350 con 0x7ffacc107840 2024-06-10T00:30:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.475+0000 7ffac37fe700 1 -- 172.21.15.62:0/1618030008 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac40052a0 con 0x7ffacc107840 2024-06-10T00:30:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.475+0000 7ffad42c0700 1 -- 172.21.15.62:0/1618030008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffacc1a7800 con 0x7ffacc107840 2024-06-10T00:30:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.475+0000 7ffad42c0700 1 -- 172.21.15.62:0/1618030008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffacc062c20 con 0x7ffacc107840 2024-06-10T00:30:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.477+0000 7ffac37fe700 1 -- 172.21.15.62:0/1618030008 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffac4024070 con 0x7ffacc107840 2024-06-10T00:30:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.477+0000 7ffac37fe700 1 --2- 172.21.15.62:0/1618030008 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffab80a2d30 0x7ffab80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.478+0000 7ffad205c700 1 --2- 172.21.15.62:0/1618030008 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffab80a2d30 0x7ffab80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.478+0000 7ffac37fe700 1 -- 172.21.15.62:0/1618030008 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7ffac40d0a40 con 0x7ffacc107840 2024-06-10T00:30:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.478+0000 7ffac37fe700 1 --2- 172.21.15.62:0/1618030008 >> [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738] conn(0x7ffab80ae090 0x7ffab80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.478+0000 7ffac37fe700 1 -- 172.21.15.62:0/1618030008 --> [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffac400cbc0 con 0x7ffab80ae090 2024-06-10T00:30:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.478+0000 7ffac37fe700 1 -- 172.21.15.62:0/1618030008 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7ffac40d0e90 con 0x7ffacc107840 2024-06-10T00:30:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.478+0000 7ffad205c700 1 --2- 172.21.15.62:0/1618030008 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffab80a2d30 0x7ffab80a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffac8005950 tx=0x7ffac800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.478+0000 7ffad185b700 1 --2- 172.21.15.62:0/1618030008 >> [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738] conn(0x7ffab80ae090 0x7ffab80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.479+0000 7ffad185b700 1 --2- 172.21.15.62:0/1618030008 >> [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738] conn(0x7ffab80ae090 0x7ffab80b0530 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.480+0000 7ffac37fe700 1 -- 172.21.15.62:0/1618030008 <== osd.4 v2:172.21.15.105:6808/1897328738 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffac400cbc0 con 0x7ffab80ae090 2024-06-10T00:30:40.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.496+0000 7ffad42c0700 1 -- 172.21.15.62:0/1618030008 --> [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffacc0630a0 con 0x7ffab80ae090 2024-06-10T00:30:40.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.496+0000 7ffac37fe700 1 -- 172.21.15.62:0/1618030008 <== osd.4 v2:172.21.15.105:6808/1897328738 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffacc0630a0 con 0x7ffab80ae090 2024-06-10T00:30:40.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.497+0000 7ffad42c0700 1 -- 172.21.15.62:0/1618030008 >> [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738] conn(0x7ffab80ae090 msgr2=0x7ffab80b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:40.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.497+0000 7ffad42c0700 1 --2- 172.21.15.62:0/1618030008 >> [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738] conn(0x7ffab80ae090 0x7ffab80b0530 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:40.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.497+0000 7ffad42c0700 1 -- 172.21.15.62:0/1618030008 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffab80a2d30 msgr2=0x7ffab80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:40.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.497+0000 7ffad42c0700 1 --2- 172.21.15.62:0/1618030008 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ffab80a2d30 0x7ffab80a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffac8005950 tx=0x7ffac800b410 comp rx=0 tx=0).stop 2024-06-10T00:30:40.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.497+0000 7ffad42c0700 1 -- 172.21.15.62:0/1618030008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffacc107840 msgr2=0x7ffacc075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:40.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.497+0000 7ffad42c0700 1 --2- 172.21.15.62:0/1618030008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffacc107840 0x7ffacc075210 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffac4006070 tx=0x7ffac4004d00 comp rx=0 tx=0).stop 2024-06-10T00:30:40.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.497+0000 7ffad285d700 1 -- 172.21.15.62:0/1618030008 reap_dead start 2024-06-10T00:30:40.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.497+0000 7ffad42c0700 1 -- 172.21.15.62:0/1618030008 shutdown_connections 2024-06-10T00:30:40.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.497+0000 7ffad42c0700 1 -- 172.21.15.62:0/1618030008 >> 172.21.15.62:0/1618030008 conn(0x7ffacc0795f0 msgr2=0x7ffacc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:40.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.497+0000 7ffad42c0700 1 -- 172.21.15.62:0/1618030008 shutdown_connections 2024-06-10T00:30:40.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:40.497+0000 7ffad42c0700 1 -- 172.21.15.62:0/1618030008 wait complete. 2024-06-10T00:30:42.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:41 smithi105 ceph-mon[36321]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-10T00:30:42.082 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:41 smithi062 ceph-mon[32390]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-10T00:30:42.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:41 smithi138 ceph-mon[36549]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-10T00:30:43.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:43 smithi062 ceph-mon[32390]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-10T00:30:43.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.973+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2262433330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda4102ea0 msgr2=0x7fbda410f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:43.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.973+0000 7fbdaa27f700 1 --2- 172.21.15.62:0/2262433330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda4102ea0 0x7fbda410f380 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00b0a0 tx=0x7fbd9c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:30:43.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.973+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2262433330 shutdown_connections 2024-06-10T00:30:43.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.973+0000 7fbdaa27f700 1 --2- 172.21.15.62:0/2262433330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda4102ea0 0x7fbda410f380 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:43.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.973+0000 7fbdaa27f700 1 --2- 172.21.15.62:0/2262433330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbda41085f0 0x7fbda41089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:43.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.973+0000 7fbdaa27f700 1 --2- 172.21.15.62:0/2262433330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda41024e0 0x7fbda4102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:43.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.973+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2262433330 >> 172.21.15.62:0/2262433330 conn(0x7fbda4079640 msgr2=0x7fbda4079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.973+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2262433330 shutdown_connections 2024-06-10T00:30:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.973+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2262433330 wait complete. 2024-06-10T00:30:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.974+0000 7fbdaa27f700 1 Processor -- start 2024-06-10T00:30:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.974+0000 7fbdaa27f700 1 -- start start 2024-06-10T00:30:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.974+0000 7fbdaa27f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda41024e0 0x7fbda4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.974+0000 7fbdaa27f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda4102ea0 0x7fbda4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbdaa27f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbda41085f0 0x7fbda4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbda3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda41024e0 0x7fbda4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:43.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbda3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda41024e0 0x7fbda4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37036/0 (socket says 172.21.15.62:37036) 2024-06-10T00:30:43.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbda3fff700 1 -- 172.21.15.62:0/2035327865 learned_addr learned my addr 172.21.15.62:0/2035327865 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:43.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2035327865 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbda4112800 con 0x7fbda41024e0 2024-06-10T00:30:43.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2035327865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbda4112680 con 0x7fbda41085f0 2024-06-10T00:30:43.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbda881c700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbda41085f0 0x7fbda4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:43.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2035327865 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbda4112980 con 0x7fbda4102ea0 2024-06-10T00:30:43.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbd9bfff700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda4102ea0 0x7fbda4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:43.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbda881c700 1 -- 172.21.15.62:0/2035327865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda4102ea0 msgr2=0x7fbda4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:43.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbda881c700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda4102ea0 0x7fbda4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:43.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbda881c700 1 -- 172.21.15.62:0/2035327865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda41024e0 msgr2=0x7fbda4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:43.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbda881c700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbda41024e0 0x7fbda4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:43.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbda881c700 1 -- 172.21.15.62:0/2035327865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda41a7180 con 0x7fbda41085f0 2024-06-10T00:30:43.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.975+0000 7fbd9bfff700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda4102ea0 0x7fbda4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:30:43.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.976+0000 7fbda881c700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbda41085f0 0x7fbda4075210 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0062a0 tx=0x7fbd9c004d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:43.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.976+0000 7fbda1ffb700 1 -- 172.21.15.62:0/2035327865 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c021070 con 0x7fbda41085f0 2024-06-10T00:30:43.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.976+0000 7fbda1ffb700 1 -- 172.21.15.62:0/2035327865 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd9c007c00 con 0x7fbda41085f0 2024-06-10T00:30:43.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.977+0000 7fbda1ffb700 1 -- 172.21.15.62:0/2035327865 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0053b0 con 0x7fbda41085f0 2024-06-10T00:30:43.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.977+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2035327865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda41a7410 con 0x7fbda41085f0 2024-06-10T00:30:43.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.977+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2035327865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda41a7870 con 0x7fbda41085f0 2024-06-10T00:30:43.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.978+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2035327865 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbda4062ba0 con 0x7fbda41085f0 2024-06-10T00:30:43.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.979+0000 7fbda1ffb700 1 -- 172.21.15.62:0/2035327865 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbd9c005550 con 0x7fbda41085f0 2024-06-10T00:30:43.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.980+0000 7fbda1ffb700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbd900a2d30 0x7fbd900a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:43.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.981+0000 7fbda3fff700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbd900a2d30 0x7fbd900a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:43.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.981+0000 7fbda1ffb700 1 -- 172.21.15.62:0/2035327865 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fbd9c00f020 con 0x7fbda41085f0 2024-06-10T00:30:43.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.981+0000 7fbda1ffb700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663] conn(0x7fbd900ae090 0x7fbd900b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:43.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.981+0000 7fbda1ffb700 1 -- 172.21.15.62:0/2035327865 --> [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd9c0020b0 con 0x7fbd900ae090 2024-06-10T00:30:43.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.981+0000 7fbda1ffb700 1 -- 172.21.15.62:0/2035327865 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fbd9c0d0cb0 con 0x7fbda41085f0 2024-06-10T00:30:43.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.981+0000 7fbda3fff700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbd900a2d30 0x7fbd900a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbd94005e90 tx=0x7fbd94005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:43.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.981+0000 7fbd9bfff700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663] conn(0x7fbd900ae090 0x7fbd900b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:43.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.982+0000 7fbd9bfff700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663] conn(0x7fbd900ae090 0x7fbd900b0530 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:43.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.983+0000 7fbda1ffb700 1 -- 172.21.15.62:0/2035327865 <== osd.2 v2:172.21.15.138:6800/4170554663 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd9c0020b0 con 0x7fbd900ae090 2024-06-10T00:30:43.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:43.999+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2035327865 --> [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbda4063070 con 0x7fbd900ae090 2024-06-10T00:30:44.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.000+0000 7fbda1ffb700 1 -- 172.21.15.62:0/2035327865 <== osd.2 v2:172.21.15.138:6800/4170554663 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbda4063070 con 0x7fbd900ae090 2024-06-10T00:30:44.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.001+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2035327865 >> [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663] conn(0x7fbd900ae090 msgr2=0x7fbd900b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:44.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.001+0000 7fbdaa27f700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663] conn(0x7fbd900ae090 0x7fbd900b0530 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:44.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.001+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2035327865 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbd900a2d30 msgr2=0x7fbd900a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:44.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.001+0000 7fbdaa27f700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fbd900a2d30 0x7fbd900a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbd94005e90 tx=0x7fbd94005d20 comp rx=0 tx=0).stop 2024-06-10T00:30:44.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.001+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2035327865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbda41085f0 msgr2=0x7fbda4075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:44.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.001+0000 7fbdaa27f700 1 --2- 172.21.15.62:0/2035327865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbda41085f0 0x7fbda4075210 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0062a0 tx=0x7fbd9c004d30 comp rx=0 tx=0).stop 2024-06-10T00:30:44.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.001+0000 7fbda881c700 1 -- 172.21.15.62:0/2035327865 reap_dead start 2024-06-10T00:30:44.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.001+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2035327865 shutdown_connections 2024-06-10T00:30:44.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.001+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2035327865 >> 172.21.15.62:0/2035327865 conn(0x7fbda4079640 msgr2=0x7fbda4100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:44.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.001+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2035327865 shutdown_connections 2024-06-10T00:30:44.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.001+0000 7fbdaa27f700 1 -- 172.21.15.62:0/2035327865 wait complete. 2024-06-10T00:30:44.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:43 smithi105 ceph-mon[36321]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-10T00:30:44.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:43 smithi138 ceph-mon[36549]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-10T00:30:44.859 INFO:teuthology.orchestra.run.smithi062.stdout:141733920844 2024-06-10T00:30:44.859 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-10T00:30:45.308 INFO:teuthology.orchestra.run.smithi062.stdout:34359738463 2024-06-10T00:30:45.308 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-10T00:30:45.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:45 smithi062 ceph-mon[32390]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:30:46.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:45 smithi105 ceph-mon[36321]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:30:46.093 INFO:teuthology.orchestra.run.smithi062.stdout:219043332156 2024-06-10T00:30:46.094 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-10T00:30:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:45 smithi138 ceph-mon[36549]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:30:46.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.914+0000 7f861f9e5700 1 -- 172.21.15.62:0/3321708929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f861806a170 msgr2=0x7f861806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:46.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.914+0000 7f861f9e5700 1 --2- 172.21.15.62:0/3321708929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f861806a170 0x7f861806a5f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8608009990 tx=0x7f8608009ca0 comp rx=0 tx=0).stop 2024-06-10T00:30:46.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.914+0000 7f861f9e5700 1 -- 172.21.15.62:0/3321708929 shutdown_connections 2024-06-10T00:30:46.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.914+0000 7f861f9e5700 1 --2- 172.21.15.62:0/3321708929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f861806ab30 0x7f861810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:46.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.914+0000 7f861f9e5700 1 --2- 172.21.15.62:0/3321708929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8618108500 0x7f86181088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:46.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.914+0000 7f861f9e5700 1 --2- 172.21.15.62:0/3321708929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f861806a170 0x7f861806a5f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:46.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.914+0000 7f861f9e5700 1 -- 172.21.15.62:0/3321708929 >> 172.21.15.62:0/3321708929 conn(0x7f8618078340 msgr2=0x7f8618078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:46.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.914+0000 7f861f9e5700 1 -- 172.21.15.62:0/3321708929 shutdown_connections 2024-06-10T00:30:46.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.914+0000 7f861f9e5700 1 -- 172.21.15.62:0/3321708929 wait complete. 2024-06-10T00:30:46.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861f9e5700 1 Processor -- start 2024-06-10T00:30:46.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861f9e5700 1 -- start start 2024-06-10T00:30:46.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861f9e5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f861806a170 0x7f861819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:46.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861f9e5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f861806ab30 0x7f861819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:46.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861f9e5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8618108500 0x7f8618198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:46.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861f9e5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8618112780 con 0x7f861806ab30 2024-06-10T00:30:46.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861f9e5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8618112600 con 0x7f8618108500 2024-06-10T00:30:46.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861f9e5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8618112900 con 0x7f861806a170 2024-06-10T00:30:46.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861cf80700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f861806ab30 0x7f861819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:46.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861cf80700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f861806ab30 0x7f861819ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37086/0 (socket says 172.21.15.62:37086) 2024-06-10T00:30:46.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861cf80700 1 -- 172.21.15.62:0/2627457424 learned_addr learned my addr 172.21.15.62:0/2627457424 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:46.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861d781700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f861806a170 0x7f861819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:46.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861df82700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8618108500 0x7f8618198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:46.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861cf80700 1 -- 172.21.15.62:0/2627457424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f861806a170 msgr2=0x7f861819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:46.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861cf80700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f861806a170 0x7f861819e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:46.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861cf80700 1 -- 172.21.15.62:0/2627457424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8618108500 msgr2=0x7f8618198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:46.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.915+0000 7f861cf80700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8618108500 0x7f8618198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:46.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.916+0000 7f861cf80700 1 -- 172.21.15.62:0/2627457424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8618199200 con 0x7f861806ab30 2024-06-10T00:30:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.916+0000 7f861df82700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8618108500 0x7f8618198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:30:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.916+0000 7f861cf80700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f861806ab30 0x7f861819ee20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8608000c00 tx=0x7f86080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.916+0000 7f860e7fc700 1 -- 172.21.15.62:0/2627457424 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608027070 con 0x7f861806ab30 2024-06-10T00:30:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.916+0000 7f860e7fc700 1 -- 172.21.15.62:0/2627457424 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8608003780 con 0x7f861806ab30 2024-06-10T00:30:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.916+0000 7f860e7fc700 1 -- 172.21.15.62:0/2627457424 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608022dd0 con 0x7f861806ab30 2024-06-10T00:30:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.916+0000 7f861f9e5700 1 -- 172.21.15.62:0/2627457424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8618199490 con 0x7f861806ab30 2024-06-10T00:30:46.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.916+0000 7f861f9e5700 1 -- 172.21.15.62:0/2627457424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86181a57b0 con 0x7f861806ab30 2024-06-10T00:30:46.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.917+0000 7f861f9e5700 1 -- 172.21.15.62:0/2627457424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8618062ba0 con 0x7f861806ab30 2024-06-10T00:30:46.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.918+0000 7f860e7fc700 1 -- 172.21.15.62:0/2627457424 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f860802a070 con 0x7f861806ab30 2024-06-10T00:30:46.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.919+0000 7f860e7fc700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f86040a2d30 0x7f86040a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:46.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.919+0000 7f861d781700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f86040a2d30 0x7f86040a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:46.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.919+0000 7f860e7fc700 1 -- 172.21.15.62:0/2627457424 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f86080d07e0 con 0x7f861806ab30 2024-06-10T00:30:46.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.919+0000 7f860e7fc700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531] conn(0x7f86040ae090 0x7f86040b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:46.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.919+0000 7f860e7fc700 1 -- 172.21.15.62:0/2627457424 --> [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86080043d0 con 0x7f86040ae090 2024-06-10T00:30:46.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.919+0000 7f860e7fc700 1 -- 172.21.15.62:0/2627457424 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f86080d0c30 con 0x7f861806ab30 2024-06-10T00:30:46.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.920+0000 7f861d781700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f86040a2d30 0x7f86040a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8618199be0 tx=0x7f861400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:46.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.920+0000 7f861df82700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531] conn(0x7f86040ae090 0x7f86040b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:46.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.920+0000 7f861df82700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531] conn(0x7f86040ae090 0x7f86040b0530 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:46.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.921+0000 7f860e7fc700 1 -- 172.21.15.62:0/2627457424 <== osd.8 v2:172.21.15.138:6816/3579467531 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86080043d0 con 0x7f86040ae090 2024-06-10T00:30:46.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.938+0000 7f861f9e5700 1 -- 172.21.15.62:0/2627457424 --> [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8618063070 con 0x7f86040ae090 2024-06-10T00:30:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.938+0000 7f860e7fc700 1 -- 172.21.15.62:0/2627457424 <== osd.8 v2:172.21.15.138:6816/3579467531 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8618063070 con 0x7f86040ae090 2024-06-10T00:30:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.938+0000 7f861f9e5700 1 -- 172.21.15.62:0/2627457424 >> [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531] conn(0x7f86040ae090 msgr2=0x7f86040b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.938+0000 7f861f9e5700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531] conn(0x7f86040ae090 0x7f86040b0530 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.938+0000 7f861f9e5700 1 -- 172.21.15.62:0/2627457424 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f86040a2d30 msgr2=0x7f86040a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.938+0000 7f861f9e5700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f86040a2d30 0x7f86040a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8618199be0 tx=0x7f861400b410 comp rx=0 tx=0).stop 2024-06-10T00:30:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.938+0000 7f861f9e5700 1 -- 172.21.15.62:0/2627457424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f861806ab30 msgr2=0x7f861819ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.938+0000 7f861f9e5700 1 --2- 172.21.15.62:0/2627457424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f861806ab30 0x7f861819ee20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8608000c00 tx=0x7f86080047b0 comp rx=0 tx=0).stop 2024-06-10T00:30:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.939+0000 7f861df82700 1 -- 172.21.15.62:0/2627457424 reap_dead start 2024-06-10T00:30:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.939+0000 7f861f9e5700 1 -- 172.21.15.62:0/2627457424 shutdown_connections 2024-06-10T00:30:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.939+0000 7f861f9e5700 1 -- 172.21.15.62:0/2627457424 >> 172.21.15.62:0/2627457424 conn(0x7f8618078340 msgr2=0x7f86181000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:46.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.939+0000 7f861f9e5700 1 -- 172.21.15.62:0/2627457424 shutdown_connections 2024-06-10T00:30:46.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:46.939+0000 7f861f9e5700 1 -- 172.21.15.62:0/2627457424 wait complete. 2024-06-10T00:30:47.592 INFO:teuthology.orchestra.run.smithi062.stdout:77309411415 2024-06-10T00:30:47.592 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-10T00:30:47.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:47 smithi062 ceph-mon[32390]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-10T00:30:48.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:47 smithi105 ceph-mon[36321]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-10T00:30:48.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:47 smithi138 ceph-mon[36549]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-10T00:30:48.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.300+0000 7f759a85b700 1 -- 172.21.15.62:0/4093225316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7594102eb0 msgr2=0x7f759410f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:48.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.300+0000 7f759a85b700 1 --2- 172.21.15.62:0/4093225316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7594102eb0 0x7f759410f390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f758c00b0a0 tx=0x7f758c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:30:48.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.300+0000 7f759a85b700 1 -- 172.21.15.62:0/4093225316 shutdown_connections 2024-06-10T00:30:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.300+0000 7f759a85b700 1 --2- 172.21.15.62:0/4093225316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7594102eb0 0x7f759410f390 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.300+0000 7f759a85b700 1 --2- 172.21.15.62:0/4093225316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7594107420 0x7f7594107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.300+0000 7f759a85b700 1 --2- 172.21.15.62:0/4093225316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7594102510 0x7f7594102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.300+0000 7f759a85b700 1 -- 172.21.15.62:0/4093225316 >> 172.21.15.62:0/4093225316 conn(0x7f75940795f0 msgr2=0x7f7594079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.300+0000 7f759a85b700 1 -- 172.21.15.62:0/4093225316 shutdown_connections 2024-06-10T00:30:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.300+0000 7f759a85b700 1 -- 172.21.15.62:0/4093225316 wait complete. 2024-06-10T00:30:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.301+0000 7f759a85b700 1 Processor -- start 2024-06-10T00:30:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.301+0000 7f759a85b700 1 -- start start 2024-06-10T00:30:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.301+0000 7f759a85b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7594102510 0x7f7594074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.301+0000 7f759a85b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7594102eb0 0x7f7594074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.301+0000 7f759a85b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7594107420 0x7f7594075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.301+0000 7f759a85b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f75941127d0 con 0x7f7594102eb0 2024-06-10T00:30:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.301+0000 7f759a85b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7594112650 con 0x7f7594107420 2024-06-10T00:30:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.301+0000 7f759a85b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7594112950 con 0x7f7594102510 2024-06-10T00:30:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.302+0000 7f75937fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7594102eb0 0x7f7594074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.302+0000 7f75937fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7594102eb0 0x7f7594074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37058/0 (socket says 172.21.15.62:37058) 2024-06-10T00:30:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.302+0000 7f75937fe700 1 -- 172.21.15.62:0/3582441065 learned_addr learned my addr 172.21.15.62:0/3582441065 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.302+0000 7f7598df8700 1 --2- 172.21.15.62:0/3582441065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7594107420 0x7f7594075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.302+0000 7f7593fff700 1 --2- 172.21.15.62:0/3582441065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7594102510 0x7f7594074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.302+0000 7f75937fe700 1 -- 172.21.15.62:0/3582441065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7594102510 msgr2=0x7f7594074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.302+0000 7f75937fe700 1 --2- 172.21.15.62:0/3582441065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7594102510 0x7f7594074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:48.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.302+0000 7f75937fe700 1 -- 172.21.15.62:0/3582441065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7594107420 msgr2=0x7f7594075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:48.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.302+0000 7f75937fe700 1 --2- 172.21.15.62:0/3582441065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7594107420 0x7f7594075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:48.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.302+0000 7f75937fe700 1 -- 172.21.15.62:0/3582441065 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75941a7160 con 0x7f7594102eb0 2024-06-10T00:30:48.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.302+0000 7f75937fe700 1 --2- 172.21.15.62:0/3582441065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7594102eb0 0x7f7594074cd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f758800d950 tx=0x7f758800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:48.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.302+0000 7f75917fa700 1 -- 172.21.15.62:0/3582441065 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588016070 con 0x7f7594102eb0 2024-06-10T00:30:48.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.302+0000 7f75917fa700 1 -- 172.21.15.62:0/3582441065 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758800fd70 con 0x7f7594102eb0 2024-06-10T00:30:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.303+0000 7f759a85b700 1 -- 172.21.15.62:0/3582441065 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75941a7450 con 0x7f7594102eb0 2024-06-10T00:30:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.303+0000 7f75917fa700 1 -- 172.21.15.62:0/3582441065 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758800e610 con 0x7f7594102eb0 2024-06-10T00:30:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.303+0000 7f759a85b700 1 -- 172.21.15.62:0/3582441065 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75941a7990 con 0x7f7594102eb0 2024-06-10T00:30:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.304+0000 7f759a85b700 1 -- 172.21.15.62:0/3582441065 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7594062b60 con 0x7f7594102eb0 2024-06-10T00:30:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.305+0000 7f75917fa700 1 -- 172.21.15.62:0/3582441065 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7588019070 con 0x7f7594102eb0 2024-06-10T00:30:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.305+0000 7f75917fa700 1 --2- 172.21.15.62:0/3582441065 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f757c0a2cb0 0x7f757c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.306+0000 7f7593fff700 1 --2- 172.21.15.62:0/3582441065 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f757c0a2cb0 0x7f757c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.306+0000 7f75917fa700 1 -- 172.21.15.62:0/3582441065 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f75880cf050 con 0x7f7594102eb0 2024-06-10T00:30:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.306+0000 7f75917fa700 1 --2- 172.21.15.62:0/3582441065 >> [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679] conn(0x7f757c0ae010 0x7f757c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.306+0000 7f75917fa700 1 -- 172.21.15.62:0/3582441065 --> [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7588019e50 con 0x7f757c0ae010 2024-06-10T00:30:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.306+0000 7f75917fa700 1 -- 172.21.15.62:0/3582441065 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f75880d6050 con 0x7f7594102eb0 2024-06-10T00:30:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.306+0000 7f7593fff700 1 --2- 172.21.15.62:0/3582441065 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f757c0a2cb0 0x7f757c0a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f75941036f0 tx=0x7f758400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.306+0000 7f7598df8700 1 --2- 172.21.15.62:0/3582441065 >> [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679] conn(0x7f757c0ae010 0x7f757c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.307+0000 7f7598df8700 1 --2- 172.21.15.62:0/3582441065 >> [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679] conn(0x7f757c0ae010 0x7f757c0b04b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.307+0000 7f75917fa700 1 -- 172.21.15.62:0/3582441065 <== osd.10 v2:172.21.15.105:6824/576155679 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7588019e50 con 0x7f757c0ae010 2024-06-10T00:30:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.336+0000 7f759a85b700 1 -- 172.21.15.62:0/3582441065 --> [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7594062f60 con 0x7f757c0ae010 2024-06-10T00:30:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.337+0000 7f75917fa700 1 -- 172.21.15.62:0/3582441065 <== osd.10 v2:172.21.15.105:6824/576155679 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7594062f60 con 0x7f757c0ae010 2024-06-10T00:30:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.337+0000 7f759a85b700 1 -- 172.21.15.62:0/3582441065 >> [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679] conn(0x7f757c0ae010 msgr2=0x7f757c0b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.337+0000 7f759a85b700 1 --2- 172.21.15.62:0/3582441065 >> [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679] conn(0x7f757c0ae010 0x7f757c0b04b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.337+0000 7f759a85b700 1 -- 172.21.15.62:0/3582441065 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f757c0a2cb0 msgr2=0x7f757c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.337+0000 7f759a85b700 1 --2- 172.21.15.62:0/3582441065 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f757c0a2cb0 0x7f757c0a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f75941036f0 tx=0x7f758400b410 comp rx=0 tx=0).stop 2024-06-10T00:30:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.337+0000 7f759a85b700 1 -- 172.21.15.62:0/3582441065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7594102eb0 msgr2=0x7f7594074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.337+0000 7f759a85b700 1 --2- 172.21.15.62:0/3582441065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7594102eb0 0x7f7594074cd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f758800d950 tx=0x7f758800dc60 comp rx=0 tx=0).stop 2024-06-10T00:30:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.337+0000 7f7598df8700 1 -- 172.21.15.62:0/3582441065 reap_dead start 2024-06-10T00:30:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.338+0000 7f759a85b700 1 -- 172.21.15.62:0/3582441065 shutdown_connections 2024-06-10T00:30:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.338+0000 7f759a85b700 1 -- 172.21.15.62:0/3582441065 >> 172.21.15.62:0/3582441065 conn(0x7f75940795f0 msgr2=0x7f75941004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.338+0000 7f759a85b700 1 -- 172.21.15.62:0/3582441065 shutdown_connections 2024-06-10T00:30:48.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:44.338+0000 7f759a85b700 1 -- 172.21.15.62:0/3582441065 wait complete. 2024-06-10T00:30:48.809 INFO:teuthology.orchestra.run.smithi062.stdout:309237645358 2024-06-10T00:30:48.809 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-10T00:30:49.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.010+0000 7f1a22e34700 1 -- 172.21.15.62:0/2650672540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a1c101440 msgr2=0x7f1a1c10f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.010+0000 7f1a22e34700 1 --2- 172.21.15.62:0/2650672540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a1c101440 0x7f1a1c10f330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400b0a0 tx=0x7f1a1400b3b0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.010+0000 7f1a22e34700 1 -- 172.21.15.62:0/2650672540 shutdown_connections 2024-06-10T00:30:49.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.010+0000 7f1a22e34700 1 --2- 172.21.15.62:0/2650672540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a1c101440 0x7f1a1c10f330 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.010+0000 7f1a22e34700 1 --2- 172.21.15.62:0/2650672540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a1c108510 0x7f1a1c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.010+0000 7f1a22e34700 1 --2- 172.21.15.62:0/2650672540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c100a80 0x7f1a1c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.010+0000 7f1a22e34700 1 -- 172.21.15.62:0/2650672540 >> 172.21.15.62:0/2650672540 conn(0x7f1a1c077a00 msgr2=0x7f1a1c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:49.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.011+0000 7f1a22e34700 1 -- 172.21.15.62:0/2650672540 shutdown_connections 2024-06-10T00:30:49.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.011+0000 7f1a22e34700 1 -- 172.21.15.62:0/2650672540 wait complete. 2024-06-10T00:30:49.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.011+0000 7f1a22e34700 1 Processor -- start 2024-06-10T00:30:49.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.011+0000 7f1a22e34700 1 -- start start 2024-06-10T00:30:49.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a22e34700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a1c100a80 0x7f1a1c101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a22e34700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c101440 0x7f1a1c1021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a22e34700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a1c108510 0x7f1a1c1026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a22e34700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c112790 con 0x7f1a1c100a80 2024-06-10T00:30:49.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a22e34700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c112610 con 0x7f1a1c101440 2024-06-10T00:30:49.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a22e34700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c112910 con 0x7f1a1c108510 2024-06-10T00:30:49.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a1bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c101440 0x7f1a1c1021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a20bd0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a1c100a80 0x7f1a1c101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a20bd0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a1c100a80 0x7f1a1c101c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36292/0 (socket says 172.21.15.62:36292) 2024-06-10T00:30:49.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a1bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c101440 0x7f1a1c1021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:56280/0 (socket says 172.21.15.62:56280) 2024-06-10T00:30:49.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a20bd0700 1 -- 172.21.15.62:0/662645932 learned_addr learned my addr 172.21.15.62:0/662645932 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:49.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a213d1700 1 --2- 172.21.15.62:0/662645932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a1c108510 0x7f1a1c1026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a20bd0700 1 -- 172.21.15.62:0/662645932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a1c108510 msgr2=0x7f1a1c1026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a20bd0700 1 --2- 172.21.15.62:0/662645932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a1c108510 0x7f1a1c1026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a20bd0700 1 -- 172.21.15.62:0/662645932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c101440 msgr2=0x7f1a1c1021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a20bd0700 1 --2- 172.21.15.62:0/662645932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c101440 0x7f1a1c1021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.012+0000 7f1a20bd0700 1 -- 172.21.15.62:0/662645932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1c1a70d0 con 0x7f1a1c100a80 2024-06-10T00:30:49.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.013+0000 7f1a20bd0700 1 --2- 172.21.15.62:0/662645932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a1c100a80 0x7f1a1c101c60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000e9c0 tx=0x7f1a1000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:49.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.013+0000 7f1a19ffb700 1 -- 172.21.15.62:0/662645932 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1000cbd0 con 0x7f1a1c100a80 2024-06-10T00:30:49.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.013+0000 7f1a22e34700 1 -- 172.21.15.62:0/662645932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1c1a73c0 con 0x7f1a1c100a80 2024-06-10T00:30:49.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.013+0000 7f1a19ffb700 1 -- 172.21.15.62:0/662645932 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1000ee60 con 0x7f1a1c100a80 2024-06-10T00:30:49.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.013+0000 7f1a19ffb700 1 -- 172.21.15.62:0/662645932 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10005630 con 0x7f1a1c100a80 2024-06-10T00:30:49.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.013+0000 7f1a22e34700 1 -- 172.21.15.62:0/662645932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a1c1a7900 con 0x7f1a1c100a80 2024-06-10T00:30:49.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.014+0000 7f1a22e34700 1 -- 172.21.15.62:0/662645932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a1c062ba0 con 0x7f1a1c100a80 2024-06-10T00:30:49.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.015+0000 7f1a19ffb700 1 -- 172.21.15.62:0/662645932 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1a10004750 con 0x7f1a1c100a80 2024-06-10T00:30:49.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.016+0000 7f1a19ffb700 1 --2- 172.21.15.62:0/662645932 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f1a0c0a2d30 0x7f1a0c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.016+0000 7f1a19ffb700 1 -- 172.21.15.62:0/662645932 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f1a100cf720 con 0x7f1a1c100a80 2024-06-10T00:30:49.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.016+0000 7f1a19ffb700 1 --2- 172.21.15.62:0/662645932 >> [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835] conn(0x7f1a0c0ae090 0x7f1a0c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.016+0000 7f1a19ffb700 1 -- 172.21.15.62:0/662645932 --> [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a0c0b0c00 con 0x7f1a0c0ae090 2024-06-10T00:30:49.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.016+0000 7f1a19ffb700 1 -- 172.21.15.62:0/662645932 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f1a100d6050 con 0x7f1a1c100a80 2024-06-10T00:30:49.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.016+0000 7f1a1bfff700 1 --2- 172.21.15.62:0/662645932 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f1a0c0a2d30 0x7f1a0c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.017+0000 7f1a213d1700 1 --2- 172.21.15.62:0/662645932 >> [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835] conn(0x7f1a0c0ae090 0x7f1a0c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.017+0000 7f1a213d1700 1 --2- 172.21.15.62:0/662645932 >> [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835] conn(0x7f1a0c0ae090 0x7f1a0c0b0530 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:49.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.017+0000 7f1a1bfff700 1 --2- 172.21.15.62:0/662645932 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f1a0c0a2d30 0x7f1a0c0a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c107b60 tx=0x7f1a08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:49.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.018+0000 7f1a19ffb700 1 -- 172.21.15.62:0/662645932 <== osd.3 v2:172.21.15.62:6810/851860835 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a0c0b0c00 con 0x7f1a0c0ae090 2024-06-10T00:30:49.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.038+0000 7f1a22e34700 1 -- 172.21.15.62:0/662645932 --> [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a1c062fa0 con 0x7f1a0c0ae090 2024-06-10T00:30:49.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.039+0000 7f1a19ffb700 1 -- 172.21.15.62:0/662645932 <== osd.3 v2:172.21.15.62:6810/851860835 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1a1c062fa0 con 0x7f1a0c0ae090 2024-06-10T00:30:49.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.039+0000 7f1a22e34700 1 -- 172.21.15.62:0/662645932 >> [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835] conn(0x7f1a0c0ae090 msgr2=0x7f1a0c0b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.039+0000 7f1a22e34700 1 --2- 172.21.15.62:0/662645932 >> [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835] conn(0x7f1a0c0ae090 0x7f1a0c0b0530 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.039+0000 7f1a22e34700 1 -- 172.21.15.62:0/662645932 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f1a0c0a2d30 msgr2=0x7f1a0c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.039+0000 7f1a22e34700 1 --2- 172.21.15.62:0/662645932 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f1a0c0a2d30 0x7f1a0c0a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c107b60 tx=0x7f1a08009260 comp rx=0 tx=0).stop 2024-06-10T00:30:49.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.039+0000 7f1a22e34700 1 -- 172.21.15.62:0/662645932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a1c100a80 msgr2=0x7f1a1c101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.039+0000 7f1a22e34700 1 --2- 172.21.15.62:0/662645932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a1c100a80 0x7f1a1c101c60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000e9c0 tx=0x7f1a1000ecd0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.039+0000 7f1a213d1700 1 -- 172.21.15.62:0/662645932 reap_dead start 2024-06-10T00:30:49.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.040+0000 7f1a22e34700 1 -- 172.21.15.62:0/662645932 shutdown_connections 2024-06-10T00:30:49.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.040+0000 7f1a22e34700 1 -- 172.21.15.62:0/662645932 >> 172.21.15.62:0/662645932 conn(0x7f1a1c077a00 msgr2=0x7f1a1c0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:49.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.040+0000 7f1a22e34700 1 -- 172.21.15.62:0/662645932 shutdown_connections 2024-06-10T00:30:49.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.040+0000 7f1a22e34700 1 -- 172.21.15.62:0/662645932 wait complete. 2024-06-10T00:30:49.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.467+0000 7f9867439700 1 -- 172.21.15.62:0/1105079719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9860102740 msgr2=0x7f9860102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.467+0000 7f9867439700 1 --2- 172.21.15.62:0/1105079719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9860102740 0x7f9860102b20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f985c0099c0 tx=0x7f985c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.467+0000 7f9867439700 1 -- 172.21.15.62:0/1105079719 shutdown_connections 2024-06-10T00:30:49.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.467+0000 7f9867439700 1 --2- 172.21.15.62:0/1105079719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9860102740 0x7f9860102b20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.467+0000 7f9867439700 1 --2- 172.21.15.62:0/1105079719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9860107840 0x7f986010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.467+0000 7f9867439700 1 --2- 172.21.15.62:0/1105079719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98601030f0 0x7f9860107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.467+0000 7f9867439700 1 -- 172.21.15.62:0/1105079719 >> 172.21.15.62:0/1105079719 conn(0x7f98600795f0 msgr2=0x7f9860079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:49.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.467+0000 7f9867439700 1 -- 172.21.15.62:0/1105079719 shutdown_connections 2024-06-10T00:30:49.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.467+0000 7f9867439700 1 -- 172.21.15.62:0/1105079719 wait complete. 2024-06-10T00:30:49.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.468+0000 7f9867439700 1 Processor -- start 2024-06-10T00:30:49.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.468+0000 7f9867439700 1 -- start start 2024-06-10T00:30:49.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.468+0000 7f9867439700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9860102740 0x7f9860074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.468+0000 7f9867439700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98601030f0 0x7f9860074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.468+0000 7f9867439700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9860107840 0x7f9860075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.468+0000 7f9867439700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9860112760 con 0x7f98601030f0 2024-06-10T00:30:49.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.468+0000 7f9867439700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f98601125e0 con 0x7f9860107840 2024-06-10T00:30:49.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.468+0000 7f9867439700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98601128e0 con 0x7f9860102740 2024-06-10T00:30:49.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.468+0000 7f98649d4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98601030f0 0x7f9860074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.468+0000 7f98649d4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98601030f0 0x7f9860074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36306/0 (socket says 172.21.15.62:36306) 2024-06-10T00:30:49.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.468+0000 7f98649d4700 1 -- 172.21.15.62:0/3323697533 learned_addr learned my addr 172.21.15.62:0/3323697533 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:49.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.468+0000 7f98659d6700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9860107840 0x7f9860075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.468+0000 7f98651d5700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9860102740 0x7f9860074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.469+0000 7f98649d4700 1 -- 172.21.15.62:0/3323697533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9860102740 msgr2=0x7f9860074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.469+0000 7f98649d4700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9860102740 0x7f9860074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.469+0000 7f98649d4700 1 -- 172.21.15.62:0/3323697533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9860107840 msgr2=0x7f9860075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.469+0000 7f98649d4700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9860107840 0x7f9860075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.469+0000 7f98649d4700 1 -- 172.21.15.62:0/3323697533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98601a7080 con 0x7f98601030f0 2024-06-10T00:30:49.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.469+0000 7f98659d6700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9860107840 0x7f9860075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:30:49.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.469+0000 7f98649d4700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98601030f0 0x7f9860074cd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f985000d950 tx=0x7f985000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:49.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.469+0000 7f98567fc700 1 -- 172.21.15.62:0/3323697533 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850016070 con 0x7f98601030f0 2024-06-10T00:30:49.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.469+0000 7f98567fc700 1 -- 172.21.15.62:0/3323697533 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f985000fd70 con 0x7f98601030f0 2024-06-10T00:30:49.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.469+0000 7f9867439700 1 -- 172.21.15.62:0/3323697533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98601a7370 con 0x7f98601030f0 2024-06-10T00:30:49.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.469+0000 7f98567fc700 1 -- 172.21.15.62:0/3323697533 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985000e610 con 0x7f98601030f0 2024-06-10T00:30:49.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.469+0000 7f9867439700 1 -- 172.21.15.62:0/3323697533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98601a78b0 con 0x7f98601030f0 2024-06-10T00:30:49.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.470+0000 7f9867439700 1 -- 172.21.15.62:0/3323697533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9860100eb0 con 0x7f98601030f0 2024-06-10T00:30:49.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.472+0000 7f98567fc700 1 -- 172.21.15.62:0/3323697533 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9850019070 con 0x7f98601030f0 2024-06-10T00:30:49.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.472+0000 7f98567fc700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f984c0a2cb0 0x7f984c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.472+0000 7f98651d5700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f984c0a2cb0 0x7f984c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.472+0000 7f98567fc700 1 -- 172.21.15.62:0/3323697533 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f98500cf550 con 0x7f98601030f0 2024-06-10T00:30:49.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.472+0000 7f98567fc700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519] conn(0x7f984c0ae010 0x7f984c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.472+0000 7f98567fc700 1 -- 172.21.15.62:0/3323697533 --> [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f984c0b0b80 con 0x7f984c0ae010 2024-06-10T00:30:49.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.473+0000 7f98567fc700 1 -- 172.21.15.62:0/3323697533 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f98500cf9a0 con 0x7f98601030f0 2024-06-10T00:30:49.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.473+0000 7f98651d5700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f984c0a2cb0 0x7f984c0a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f985c000c00 tx=0x7f985c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:49.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.473+0000 7f98659d6700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519] conn(0x7f984c0ae010 0x7f984c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.473+0000 7f98659d6700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519] conn(0x7f984c0ae010 0x7f984c0b04b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:49.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.474+0000 7f98567fc700 1 -- 172.21.15.62:0/3323697533 <== osd.5 v2:172.21.15.138:6808/3331484519 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f984c0b0b80 con 0x7f984c0ae010 2024-06-10T00:30:49.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.490+0000 7f9867439700 1 -- 172.21.15.62:0/3323697533 --> [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9860103bd0 con 0x7f984c0ae010 2024-06-10T00:30:49.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.491+0000 7f98567fc700 1 -- 172.21.15.62:0/3323697533 <== osd.5 v2:172.21.15.138:6808/3331484519 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9860103bd0 con 0x7f984c0ae010 2024-06-10T00:30:49.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.492+0000 7f9867439700 1 -- 172.21.15.62:0/3323697533 >> [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519] conn(0x7f984c0ae010 msgr2=0x7f984c0b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.492+0000 7f9867439700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519] conn(0x7f984c0ae010 0x7f984c0b04b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.492+0000 7f9867439700 1 -- 172.21.15.62:0/3323697533 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f984c0a2cb0 msgr2=0x7f984c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.492+0000 7f9867439700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f984c0a2cb0 0x7f984c0a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f985c000c00 tx=0x7f985c005fb0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.492+0000 7f9867439700 1 -- 172.21.15.62:0/3323697533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98601030f0 msgr2=0x7f9860074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.492+0000 7f9867439700 1 --2- 172.21.15.62:0/3323697533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98601030f0 0x7f9860074cd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f985000d950 tx=0x7f985000dc60 comp rx=0 tx=0).stop 2024-06-10T00:30:49.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.492+0000 7f98659d6700 1 -- 172.21.15.62:0/3323697533 reap_dead start 2024-06-10T00:30:49.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.492+0000 7f9867439700 1 -- 172.21.15.62:0/3323697533 shutdown_connections 2024-06-10T00:30:49.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.492+0000 7f9867439700 1 -- 172.21.15.62:0/3323697533 >> 172.21.15.62:0/3323697533 conn(0x7f98600795f0 msgr2=0x7f9860100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:49.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.492+0000 7f9867439700 1 -- 172.21.15.62:0/3323697533 shutdown_connections 2024-06-10T00:30:49.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.492+0000 7f9867439700 1 -- 172.21.15.62:0/3323697533 wait complete. 2024-06-10T00:30:49.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.653+0000 7f7c218bd700 1 -- 172.21.15.62:0/2100179788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c1c107400 msgr2=0x7f7c1c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.653+0000 7f7c218bd700 1 --2- 172.21.15.62:0/2100179788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c1c107400 0x7f7c1c1077e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00b0a0 tx=0x7f7c0c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.653+0000 7f7c218bd700 1 -- 172.21.15.62:0/2100179788 shutdown_connections 2024-06-10T00:30:49.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.653+0000 7f7c218bd700 1 --2- 172.21.15.62:0/2100179788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c1c107400 0x7f7c1c1077e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.653+0000 7f7c218bd700 1 --2- 172.21.15.62:0/2100179788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c101d50 0x7f7c1c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.653+0000 7f7c218bd700 1 --2- 172.21.15.62:0/2100179788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c1c101390 0x7f7c1c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.653+0000 7f7c218bd700 1 -- 172.21.15.62:0/2100179788 >> 172.21.15.62:0/2100179788 conn(0x7f7c1c078340 msgr2=0x7f7c1c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:49.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.653+0000 7f7c218bd700 1 -- 172.21.15.62:0/2100179788 shutdown_connections 2024-06-10T00:30:49.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.653+0000 7f7c218bd700 1 -- 172.21.15.62:0/2100179788 wait complete. 2024-06-10T00:30:49.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.653+0000 7f7c218bd700 1 Processor -- start 2024-06-10T00:30:49.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c218bd700 1 -- start start 2024-06-10T00:30:49.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c218bd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c1c101390 0x7f7c1c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c218bd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c101d50 0x7f7c1c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c218bd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c1c107400 0x7f7c1c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c218bd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1116a0 con 0x7f7c1c101390 2024-06-10T00:30:49.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c218bd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c111520 con 0x7f7c1c107400 2024-06-10T00:30:49.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c218bd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c111820 con 0x7f7c1c101d50 2024-06-10T00:30:49.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c1affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c1c101390 0x7f7c1c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c1affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c1c101390 0x7f7c1c19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36314/0 (socket says 172.21.15.62:36314) 2024-06-10T00:30:49.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c1affd700 1 -- 172.21.15.62:0/2244457029 learned_addr learned my addr 172.21.15.62:0/2244457029 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:49.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c1a7fc700 1 --2- 172.21.15.62:0/2244457029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c101d50 0x7f7c1c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c1b7fe700 1 --2- 172.21.15.62:0/2244457029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c1c107400 0x7f7c1c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c1affd700 1 -- 172.21.15.62:0/2244457029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c101d50 msgr2=0x7f7c1c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c1affd700 1 --2- 172.21.15.62:0/2244457029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c101d50 0x7f7c1c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c1affd700 1 -- 172.21.15.62:0/2244457029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c1c107400 msgr2=0x7f7c1c198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c1affd700 1 --2- 172.21.15.62:0/2244457029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c1c107400 0x7f7c1c198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c1affd700 1 -- 172.21.15.62:0/2244457029 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1c199180 con 0x7f7c1c101390 2024-06-10T00:30:49.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c1affd700 1 --2- 172.21.15.62:0/2244457029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c1c101390 0x7f7c1c19e860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00b070 tx=0x7f7c0c004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:49.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c208bb700 1 -- 172.21.15.62:0/2244457029 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c021070 con 0x7f7c1c101390 2024-06-10T00:30:49.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.654+0000 7f7c218bd700 1 -- 172.21.15.62:0/2244457029 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1c199410 con 0x7f7c1c101390 2024-06-10T00:30:49.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.655+0000 7f7c208bb700 1 -- 172.21.15.62:0/2244457029 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c0c01d930 con 0x7f7c1c101390 2024-06-10T00:30:49.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.655+0000 7f7c208bb700 1 -- 172.21.15.62:0/2244457029 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c005440 con 0x7f7c1c101390 2024-06-10T00:30:49.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.655+0000 7f7c218bd700 1 -- 172.21.15.62:0/2244457029 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c1c1a5730 con 0x7f7c1c101390 2024-06-10T00:30:49.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.655+0000 7f7c218bd700 1 -- 172.21.15.62:0/2244457029 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7c1c1054c0 con 0x7f7c1c101390 2024-06-10T00:30:49.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.657+0000 7f7c208bb700 1 -- 172.21.15.62:0/2244457029 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7c0c024070 con 0x7f7c1c101390 2024-06-10T00:30:49.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.658+0000 7f7c208bb700 1 --2- 172.21.15.62:0/2244457029 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7c080a2cb0 0x7f7c080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.659+0000 7f7c208bb700 1 -- 172.21.15.62:0/2244457029 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f7c0c00f020 con 0x7f7c1c101390 2024-06-10T00:30:49.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.659+0000 7f7c1a7fc700 1 --2- 172.21.15.62:0/2244457029 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7c080a2cb0 0x7f7c080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.659+0000 7f7c208bb700 1 --2- 172.21.15.62:0/2244457029 >> [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325] conn(0x7f7c080adee0 0x7f7c080b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:49.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.659+0000 7f7c208bb700 1 -- 172.21.15.62:0/2244457029 --> [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c0c0020b0 con 0x7f7c080adee0 2024-06-10T00:30:49.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.659+0000 7f7c208bb700 1 -- 172.21.15.62:0/2244457029 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f7c0c0d10e0 con 0x7f7c1c101390 2024-06-10T00:30:49.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.659+0000 7f7c1b7fe700 1 --2- 172.21.15.62:0/2244457029 >> [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325] conn(0x7f7c080adee0 0x7f7c080b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:49.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.659+0000 7f7c1a7fc700 1 --2- 172.21.15.62:0/2244457029 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7c080a2cb0 0x7f7c080a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7c04005950 tx=0x7f7c040058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:49.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.660+0000 7f7c1b7fe700 1 --2- 172.21.15.62:0/2244457029 >> [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325] conn(0x7f7c080adee0 0x7f7c080b0380 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:49.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.661+0000 7f7c208bb700 1 -- 172.21.15.62:0/2244457029 <== osd.6 v2:172.21.15.62:6818/1649680325 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c0c0020b0 con 0x7f7c080adee0 2024-06-10T00:30:49.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.678+0000 7f7c218bd700 1 -- 172.21.15.62:0/2244457029 --> [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7c1c102ab0 con 0x7f7c080adee0 2024-06-10T00:30:49.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.679+0000 7f7c208bb700 1 -- 172.21.15.62:0/2244457029 <== osd.6 v2:172.21.15.62:6818/1649680325 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7c1c102ab0 con 0x7f7c080adee0 2024-06-10T00:30:49.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.679+0000 7f7c218bd700 1 -- 172.21.15.62:0/2244457029 >> [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325] conn(0x7f7c080adee0 msgr2=0x7f7c080b0380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.679+0000 7f7c218bd700 1 --2- 172.21.15.62:0/2244457029 >> [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325] conn(0x7f7c080adee0 0x7f7c080b0380 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.679+0000 7f7c218bd700 1 -- 172.21.15.62:0/2244457029 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7c080a2cb0 msgr2=0x7f7c080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.679+0000 7f7c218bd700 1 --2- 172.21.15.62:0/2244457029 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7c080a2cb0 0x7f7c080a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7c04005950 tx=0x7f7c040058e0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.679+0000 7f7c218bd700 1 -- 172.21.15.62:0/2244457029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c1c101390 msgr2=0x7f7c1c19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:49.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.679+0000 7f7c218bd700 1 --2- 172.21.15.62:0/2244457029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c1c101390 0x7f7c1c19e860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00b070 tx=0x7f7c0c004ef0 comp rx=0 tx=0).stop 2024-06-10T00:30:49.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.679+0000 7f7c1b7fe700 1 -- 172.21.15.62:0/2244457029 reap_dead start 2024-06-10T00:30:49.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.679+0000 7f7c218bd700 1 -- 172.21.15.62:0/2244457029 shutdown_connections 2024-06-10T00:30:49.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.679+0000 7f7c218bd700 1 -- 172.21.15.62:0/2244457029 >> 172.21.15.62:0/2244457029 conn(0x7f7c1c078340 msgr2=0x7f7c1c1006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:49.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.680+0000 7f7c218bd700 1 -- 172.21.15.62:0/2244457029 shutdown_connections 2024-06-10T00:30:49.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:49.680+0000 7f7c218bd700 1 -- 172.21.15.62:0/2244457029 wait complete. 2024-06-10T00:30:49.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:49 smithi062 ceph-mon[32390]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-06-10T00:30:50.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:49 smithi105 ceph-mon[36321]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-06-10T00:30:50.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:49 smithi138 ceph-mon[36549]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-06-10T00:30:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.443+0000 7f48c96d5700 1 -- 172.21.15.62:0/1711960782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4102ea0 msgr2=0x7f48c410f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.443+0000 7f48c96d5700 1 --2- 172.21.15.62:0/1711960782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4102ea0 0x7f48c410f380 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00b0a0 tx=0x7f48bc00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.443+0000 7f48c96d5700 1 -- 172.21.15.62:0/1711960782 shutdown_connections 2024-06-10T00:30:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.443+0000 7f48c96d5700 1 --2- 172.21.15.62:0/1711960782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c4102ea0 0x7f48c410f380 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.443+0000 7f48c96d5700 1 --2- 172.21.15.62:0/1711960782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48c41085f0 0x7f48c41089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.443+0000 7f48c96d5700 1 --2- 172.21.15.62:0/1711960782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48c41024e0 0x7f48c4102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.443+0000 7f48c96d5700 1 -- 172.21.15.62:0/1711960782 >> 172.21.15.62:0/1711960782 conn(0x7f48c4079640 msgr2=0x7f48c4079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.443+0000 7f48c96d5700 1 -- 172.21.15.62:0/1711960782 shutdown_connections 2024-06-10T00:30:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.443+0000 7f48c96d5700 1 -- 172.21.15.62:0/1711960782 wait complete. 2024-06-10T00:30:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.444+0000 7f48c96d5700 1 Processor -- start 2024-06-10T00:30:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.444+0000 7f48c96d5700 1 -- start start 2024-06-10T00:30:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.444+0000 7f48c96d5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48c41024e0 0x7f48c419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.444+0000 7f48c96d5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48c4102ea0 0x7f48c419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.444+0000 7f48c96d5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c41085f0 0x7f48c4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.444+0000 7f48c96d5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f48c4112890 con 0x7f48c41085f0 2024-06-10T00:30:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.444+0000 7f48c96d5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f48c4112710 con 0x7f48c4102ea0 2024-06-10T00:30:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.444+0000 7f48c96d5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48c4112a10 con 0x7f48c41024e0 2024-06-10T00:30:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.444+0000 7f48c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c41085f0 0x7f48c4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.444+0000 7f48c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c41085f0 0x7f48c4198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36324/0 (socket says 172.21.15.62:36324) 2024-06-10T00:30:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.444+0000 7f48c37fe700 1 -- 172.21.15.62:0/757786604 learned_addr learned my addr 172.21.15.62:0/757786604 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.445+0000 7f48c2ffd700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48c41024e0 0x7f48c419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.445+0000 7f48c27fc700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48c4102ea0 0x7f48c419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.445+0000 7f48c37fe700 1 -- 172.21.15.62:0/757786604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48c41024e0 msgr2=0x7f48c419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.445+0000 7f48c37fe700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48c41024e0 0x7f48c419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.445+0000 7f48c37fe700 1 -- 172.21.15.62:0/757786604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48c4102ea0 msgr2=0x7f48c419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.445+0000 7f48c37fe700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48c4102ea0 0x7f48c419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.445+0000 7f48c37fe700 1 -- 172.21.15.62:0/757786604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c4199210 con 0x7f48c41085f0 2024-06-10T00:30:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.445+0000 7f48c27fc700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48c4102ea0 0x7f48c419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:30:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.445+0000 7f48c37fe700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c41085f0 0x7f48c4198ab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0062a0 tx=0x7f48bc004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.445+0000 7f48abfff700 1 -- 172.21.15.62:0/757786604 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc021070 con 0x7f48c41085f0 2024-06-10T00:30:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.445+0000 7f48c96d5700 1 -- 172.21.15.62:0/757786604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c41994a0 con 0x7f48c41085f0 2024-06-10T00:30:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.445+0000 7f48abfff700 1 -- 172.21.15.62:0/757786604 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48bc007df0 con 0x7f48c41085f0 2024-06-10T00:30:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.445+0000 7f48abfff700 1 -- 172.21.15.62:0/757786604 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc005430 con 0x7f48c41085f0 2024-06-10T00:30:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.445+0000 7f48c96d5700 1 -- 172.21.15.62:0/757786604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c41a57c0 con 0x7f48c41085f0 2024-06-10T00:30:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.446+0000 7f48c96d5700 1 -- 172.21.15.62:0/757786604 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f48c4101890 con 0x7f48c41085f0 2024-06-10T00:30:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.447+0000 7f48abfff700 1 -- 172.21.15.62:0/757786604 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f48bc024070 con 0x7f48c41085f0 2024-06-10T00:30:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.448+0000 7f48abfff700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f48ac0a2cb0 0x7f48ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.448+0000 7f48c2ffd700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f48ac0a2cb0 0x7f48ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.448+0000 7f48abfff700 1 -- 172.21.15.62:0/757786604 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f48bc00f020 con 0x7f48c41085f0 2024-06-10T00:30:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.448+0000 7f48abfff700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019] conn(0x7f48ac0ae010 0x7f48ac0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.448+0000 7f48abfff700 1 -- 172.21.15.62:0/757786604 --> [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48bc008ea0 con 0x7f48ac0ae010 2024-06-10T00:30:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.448+0000 7f48abfff700 1 -- 172.21.15.62:0/757786604 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f48bc0d13b0 con 0x7f48c41085f0 2024-06-10T00:30:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.449+0000 7f48c2ffd700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f48ac0a2cb0 0x7f48ac0a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f48b4006fd0 tx=0x7f48b4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:50.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.449+0000 7f48c27fc700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019] conn(0x7f48ac0ae010 0x7f48ac0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:50.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.449+0000 7f48c27fc700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019] conn(0x7f48ac0ae010 0x7f48ac0b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:50.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.450+0000 7f48abfff700 1 -- 172.21.15.62:0/757786604 <== osd.1 v2:172.21.15.105:6800/1473053019 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48bc008ea0 con 0x7f48ac0ae010 2024-06-10T00:30:50.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.466+0000 7f48c96d5700 1 -- 172.21.15.62:0/757786604 --> [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f48c4103c00 con 0x7f48ac0ae010 2024-06-10T00:30:50.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.467+0000 7f48abfff700 1 -- 172.21.15.62:0/757786604 <== osd.1 v2:172.21.15.105:6800/1473053019 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f48c4103c00 con 0x7f48ac0ae010 2024-06-10T00:30:50.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.467+0000 7f48c96d5700 1 -- 172.21.15.62:0/757786604 >> [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019] conn(0x7f48ac0ae010 msgr2=0x7f48ac0b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:50.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.467+0000 7f48c96d5700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019] conn(0x7f48ac0ae010 0x7f48ac0b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.467+0000 7f48c96d5700 1 -- 172.21.15.62:0/757786604 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f48ac0a2cb0 msgr2=0x7f48ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.467+0000 7f48c96d5700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f48ac0a2cb0 0x7f48ac0a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f48b4006fd0 tx=0x7f48b4008040 comp rx=0 tx=0).stop 2024-06-10T00:30:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.467+0000 7f48c96d5700 1 -- 172.21.15.62:0/757786604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c41085f0 msgr2=0x7f48c4198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.467+0000 7f48c96d5700 1 --2- 172.21.15.62:0/757786604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48c41085f0 0x7f48c4198ab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0062a0 tx=0x7f48bc004ef0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.467+0000 7f48c37fe700 1 -- 172.21.15.62:0/757786604 reap_dead start 2024-06-10T00:30:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.468+0000 7f48c96d5700 1 -- 172.21.15.62:0/757786604 shutdown_connections 2024-06-10T00:30:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.468+0000 7f48c96d5700 1 -- 172.21.15.62:0/757786604 >> 172.21.15.62:0/757786604 conn(0x7f48c4079640 msgr2=0x7f48c4100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.468+0000 7f48c96d5700 1 -- 172.21.15.62:0/757786604 shutdown_connections 2024-06-10T00:30:50.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.468+0000 7f48c96d5700 1 -- 172.21.15.62:0/757786604 wait complete. 2024-06-10T00:30:50.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.686+0000 7fd253232700 1 -- 172.21.15.62:0/4048445812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd24c105330 msgr2=0x7fd24c105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:50.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.686+0000 7fd253232700 1 --2- 172.21.15.62:0/4048445812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd24c105330 0x7fd24c105710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd2340099c0 tx=0x7fd234009cd0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.686+0000 7fd253232700 1 -- 172.21.15.62:0/4048445812 shutdown_connections 2024-06-10T00:30:50.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.686+0000 7fd253232700 1 --2- 172.21.15.62:0/4048445812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd24c105330 0x7fd24c105710 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.686+0000 7fd253232700 1 --2- 172.21.15.62:0/4048445812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c107b60 0x7fd24c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.686+0000 7fd253232700 1 --2- 172.21.15.62:0/4048445812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd24c1071a0 0x7fd24c107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.686+0000 7fd253232700 1 -- 172.21.15.62:0/4048445812 >> 172.21.15.62:0/4048445812 conn(0x7fd24c0766b0 msgr2=0x7fd24c076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.687+0000 7fd253232700 1 -- 172.21.15.62:0/4048445812 shutdown_connections 2024-06-10T00:30:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.687+0000 7fd253232700 1 -- 172.21.15.62:0/4048445812 wait complete. 2024-06-10T00:30:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.687+0000 7fd253232700 1 Processor -- start 2024-06-10T00:30:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.687+0000 7fd253232700 1 -- start start 2024-06-10T00:30:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.687+0000 7fd253232700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c105330 0x7fd24c102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.687+0000 7fd253232700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd24c1071a0 0x7fd24c102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.687+0000 7fd253232700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd24c107b60 0x7fd24c102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.687+0000 7fd253232700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c111730 con 0x7fd24c107b60 2024-06-10T00:30:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.687+0000 7fd253232700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c1115b0 con 0x7fd24c1071a0 2024-06-10T00:30:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.687+0000 7fd253232700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c1118b0 con 0x7fd24c105330 2024-06-10T00:30:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd2517cf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd24c107b60 0x7fd24c102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd2517cf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd24c107b60 0x7fd24c102ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36338/0 (socket says 172.21.15.62:36338) 2024-06-10T00:30:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd2517cf700 1 -- 172.21.15.62:0/2358214178 learned_addr learned my addr 172.21.15.62:0/2358214178 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd24bfff700 1 --2- 172.21.15.62:0/2358214178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd24c1071a0 0x7fd24c102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd250fce700 1 --2- 172.21.15.62:0/2358214178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c105330 0x7fd24c102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd2517cf700 1 -- 172.21.15.62:0/2358214178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c105330 msgr2=0x7fd24c102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd2517cf700 1 --2- 172.21.15.62:0/2358214178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd24c105330 0x7fd24c102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd2517cf700 1 -- 172.21.15.62:0/2358214178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd24c1071a0 msgr2=0x7fd24c102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd2517cf700 1 --2- 172.21.15.62:0/2358214178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd24c1071a0 0x7fd24c102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd2517cf700 1 -- 172.21.15.62:0/2358214178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24c1a7110 con 0x7fd24c107b60 2024-06-10T00:30:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd2517cf700 1 --2- 172.21.15.62:0/2358214178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd24c107b60 0x7fd24c102ab0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd24400d980 tx=0x7fd24400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd249ffb700 1 -- 172.21.15.62:0/2358214178 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244016070 con 0x7fd24c107b60 2024-06-10T00:30:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd249ffb700 1 -- 172.21.15.62:0/2358214178 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd244004d10 con 0x7fd24c107b60 2024-06-10T00:30:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd253232700 1 -- 172.21.15.62:0/2358214178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24c1a7400 con 0x7fd24c107b60 2024-06-10T00:30:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd249ffb700 1 -- 172.21.15.62:0/2358214178 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2440055e0 con 0x7fd24c107b60 2024-06-10T00:30:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.688+0000 7fd253232700 1 -- 172.21.15.62:0/2358214178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24c1a7940 con 0x7fd24c107b60 2024-06-10T00:30:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.689+0000 7fd253232700 1 -- 172.21.15.62:0/2358214178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd24c101890 con 0x7fd24c107b60 2024-06-10T00:30:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.691+0000 7fd249ffb700 1 -- 172.21.15.62:0/2358214178 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd244004750 con 0x7fd24c107b60 2024-06-10T00:30:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.691+0000 7fd249ffb700 1 --2- 172.21.15.62:0/2358214178 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd2380a2cb0 0x7fd2380a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.691+0000 7fd250fce700 1 --2- 172.21.15.62:0/2358214178 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd2380a2cb0 0x7fd2380a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.691+0000 7fd249ffb700 1 -- 172.21.15.62:0/2358214178 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fd2440cf960 con 0x7fd24c107b60 2024-06-10T00:30:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.692+0000 7fd249ffb700 1 --2- 172.21.15.62:0/2358214178 >> [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685] conn(0x7fd2380ae010 0x7fd2380b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.692+0000 7fd249ffb700 1 -- 172.21.15.62:0/2358214178 --> [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd244015ec0 con 0x7fd2380ae010 2024-06-10T00:30:50.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.692+0000 7fd249ffb700 1 -- 172.21.15.62:0/2358214178 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7fd2440cfdb0 con 0x7fd24c107b60 2024-06-10T00:30:50.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.692+0000 7fd250fce700 1 --2- 172.21.15.62:0/2358214178 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd2380a2cb0 0x7fd2380a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd2340059d0 tx=0x7fd234005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:50.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.692+0000 7fd24bfff700 1 --2- 172.21.15.62:0/2358214178 >> [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685] conn(0x7fd2380ae010 0x7fd2380b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:50.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.692+0000 7fd24bfff700 1 --2- 172.21.15.62:0/2358214178 >> [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685] conn(0x7fd2380ae010 0x7fd2380b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:50.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.693+0000 7fd249ffb700 1 -- 172.21.15.62:0/2358214178 <== osd.11 v2:172.21.15.138:6824/364141685 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd244015ec0 con 0x7fd2380ae010 2024-06-10T00:30:50.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.710+0000 7fd253232700 1 -- 172.21.15.62:0/2358214178 --> [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd24c06a780 con 0x7fd2380ae010 2024-06-10T00:30:50.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.711+0000 7fd249ffb700 1 -- 172.21.15.62:0/2358214178 <== osd.11 v2:172.21.15.138:6824/364141685 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd24c06a780 con 0x7fd2380ae010 2024-06-10T00:30:50.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.711+0000 7fd253232700 1 -- 172.21.15.62:0/2358214178 >> [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685] conn(0x7fd2380ae010 msgr2=0x7fd2380b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:50.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.711+0000 7fd253232700 1 --2- 172.21.15.62:0/2358214178 >> [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685] conn(0x7fd2380ae010 0x7fd2380b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:50.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.711+0000 7fd253232700 1 -- 172.21.15.62:0/2358214178 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd2380a2cb0 msgr2=0x7fd2380a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:50.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.711+0000 7fd253232700 1 --2- 172.21.15.62:0/2358214178 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd2380a2cb0 0x7fd2380a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd2340059d0 tx=0x7fd234005940 comp rx=0 tx=0).stop 2024-06-10T00:30:50.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.711+0000 7fd253232700 1 -- 172.21.15.62:0/2358214178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd24c107b60 msgr2=0x7fd24c102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:50.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.711+0000 7fd253232700 1 --2- 172.21.15.62:0/2358214178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd24c107b60 0x7fd24c102ab0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd24400d980 tx=0x7fd24400dc90 comp rx=0 tx=0).stop 2024-06-10T00:30:50.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.711+0000 7fd2517cf700 1 -- 172.21.15.62:0/2358214178 reap_dead start 2024-06-10T00:30:50.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.711+0000 7fd253232700 1 -- 172.21.15.62:0/2358214178 shutdown_connections 2024-06-10T00:30:50.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.711+0000 7fd253232700 1 -- 172.21.15.62:0/2358214178 >> 172.21.15.62:0/2358214178 conn(0x7fd24c0766b0 msgr2=0x7fd24c100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:50.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.711+0000 7fd253232700 1 -- 172.21.15.62:0/2358214178 shutdown_connections 2024-06-10T00:30:50.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:50.711+0000 7fd253232700 1 -- 172.21.15.62:0/2358214178 wait complete. 2024-06-10T00:30:51.766 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:51 smithi062 ceph-mon[32390]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-06-10T00:30:52.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:51 smithi105 ceph-mon[36321]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-06-10T00:30:52.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:51 smithi138 ceph-mon[36549]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-06-10T00:30:52.399 INFO:teuthology.orchestra.run.smithi062.stdout:107374182483 2024-06-10T00:30:52.399 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-10T00:30:52.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.806+0000 7f894d0bb700 1 -- 172.21.15.62:0/4127513738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8948101610 msgr2=0x7f89481019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:52.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.806+0000 7f894d0bb700 1 --2- 172.21.15.62:0/4127513738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8948101610 0x7f89481019f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f89380099c0 tx=0x7f8938009cd0 comp rx=0 tx=0).stop 2024-06-10T00:30:52.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.806+0000 7f894d0bb700 1 -- 172.21.15.62:0/4127513738 shutdown_connections 2024-06-10T00:30:52.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.806+0000 7f894d0bb700 1 --2- 172.21.15.62:0/4127513738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8948101610 0x7f89481019f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:52.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.806+0000 7f894d0bb700 1 --2- 172.21.15.62:0/4127513738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89481066f0 0x7f894810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:52.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.806+0000 7f894d0bb700 1 --2- 172.21.15.62:0/4127513738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8948101fc0 0x7f8948106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:52.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.806+0000 7f894d0bb700 1 -- 172.21.15.62:0/4127513738 >> 172.21.15.62:0/4127513738 conn(0x7f8948078430 msgr2=0x7f8948078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:52.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.806+0000 7f894d0bb700 1 -- 172.21.15.62:0/4127513738 shutdown_connections 2024-06-10T00:30:52.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.806+0000 7f894d0bb700 1 -- 172.21.15.62:0/4127513738 wait complete. 2024-06-10T00:30:52.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.807+0000 7f894d0bb700 1 Processor -- start 2024-06-10T00:30:52.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.807+0000 7f894d0bb700 1 -- start start 2024-06-10T00:30:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894d0bb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8948101610 0x7f894819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894d0bb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8948101fc0 0x7f894819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894d0bb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89481066f0 0x7f8948198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894d0bb700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89481116d0 con 0x7f89481066f0 2024-06-10T00:30:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894d0bb700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8948111550 con 0x7f8948101fc0 2024-06-10T00:30:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894d0bb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8948111850 con 0x7f8948101610 2024-06-10T00:30:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89481066f0 0x7f8948198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f8946d9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8948101610 0x7f894819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89481066f0 0x7f8948198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36346/0 (socket says 172.21.15.62:36346) 2024-06-10T00:30:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894759e700 1 -- 172.21.15.62:0/3289593038 learned_addr learned my addr 172.21.15.62:0/3289593038 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:30:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894659c700 1 --2- 172.21.15.62:0/3289593038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8948101fc0 0x7f894819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894759e700 1 -- 172.21.15.62:0/3289593038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8948101610 msgr2=0x7f894819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894759e700 1 --2- 172.21.15.62:0/3289593038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8948101610 0x7f894819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894759e700 1 -- 172.21.15.62:0/3289593038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8948101fc0 msgr2=0x7f894819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894759e700 1 --2- 172.21.15.62:0/3289593038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8948101fc0 0x7f894819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894759e700 1 -- 172.21.15.62:0/3289593038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89481991e0 con 0x7f89481066f0 2024-06-10T00:30:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f894759e700 1 --2- 172.21.15.62:0/3289593038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89481066f0 0x7f8948198a80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8940007ee0 tx=0x7f894000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.808+0000 7f892ffff700 1 -- 172.21.15.62:0/3289593038 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940004bb0 con 0x7f89481066f0 2024-06-10T00:30:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.809+0000 7f894d0bb700 1 -- 172.21.15.62:0/3289593038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89481994d0 con 0x7f89481066f0 2024-06-10T00:30:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.809+0000 7f894d0bb700 1 -- 172.21.15.62:0/3289593038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89481a5790 con 0x7f89481066f0 2024-06-10T00:30:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.809+0000 7f892ffff700 1 -- 172.21.15.62:0/3289593038 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8940004d50 con 0x7f89481066f0 2024-06-10T00:30:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.809+0000 7f892ffff700 1 -- 172.21.15.62:0/3289593038 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89400058d0 con 0x7f89481066f0 2024-06-10T00:30:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.809+0000 7f894d0bb700 1 -- 172.21.15.62:0/3289593038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8934001030 con 0x7f89481066f0 2024-06-10T00:30:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.811+0000 7f892ffff700 1 -- 172.21.15.62:0/3289593038 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f894001a020 con 0x7f89481066f0 2024-06-10T00:30:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.812+0000 7f892ffff700 1 --2- 172.21.15.62:0/3289593038 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f89300a2cb0 0x7f89300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.812+0000 7f8946d9d700 1 --2- 172.21.15.62:0/3289593038 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f89300a2cb0 0x7f89300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.812+0000 7f892ffff700 1 -- 172.21.15.62:0/3289593038 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f89400cf710 con 0x7f89481066f0 2024-06-10T00:30:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.812+0000 7f892ffff700 1 --2- 172.21.15.62:0/3289593038 >> [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101] conn(0x7f89300ae000 0x7f89300b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:30:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.812+0000 7f892ffff700 1 -- 172.21.15.62:0/3289593038 --> [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8940009390 con 0x7f89300ae000 2024-06-10T00:30:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.812+0000 7f892ffff700 1 -- 172.21.15.62:0/3289593038 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f89400cfac0 con 0x7f89481066f0 2024-06-10T00:30:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.812+0000 7f894659c700 1 --2- 172.21.15.62:0/3289593038 >> [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101] conn(0x7f89300ae000 0x7f89300b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:30:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.812+0000 7f8946d9d700 1 --2- 172.21.15.62:0/3289593038 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f89300a2cb0 0x7f89300a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8938000c00 tx=0x7f8938005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.812+0000 7f894659c700 1 --2- 172.21.15.62:0/3289593038 >> [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101] conn(0x7f89300ae000 0x7f89300b04a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:30:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.813+0000 7f892ffff700 1 -- 172.21.15.62:0/3289593038 <== osd.9 v2:172.21.15.62:6826/1109246101 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8940009390 con 0x7f89300ae000 2024-06-10T00:30:52.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.830+0000 7f894d0bb700 1 -- 172.21.15.62:0/3289593038 --> [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8934002e60 con 0x7f89300ae000 2024-06-10T00:30:52.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.830+0000 7f892ffff700 1 -- 172.21.15.62:0/3289593038 <== osd.9 v2:172.21.15.62:6826/1109246101 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8934002e60 con 0x7f89300ae000 2024-06-10T00:30:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.830+0000 7f894d0bb700 1 -- 172.21.15.62:0/3289593038 >> [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101] conn(0x7f89300ae000 msgr2=0x7f89300b04a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.830+0000 7f894d0bb700 1 --2- 172.21.15.62:0/3289593038 >> [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101] conn(0x7f89300ae000 0x7f89300b04a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:30:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.830+0000 7f894d0bb700 1 -- 172.21.15.62:0/3289593038 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f89300a2cb0 msgr2=0x7f89300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.830+0000 7f894d0bb700 1 --2- 172.21.15.62:0/3289593038 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f89300a2cb0 0x7f89300a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8938000c00 tx=0x7f8938005fb0 comp rx=0 tx=0).stop 2024-06-10T00:30:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.830+0000 7f894d0bb700 1 -- 172.21.15.62:0/3289593038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89481066f0 msgr2=0x7f8948198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:30:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.830+0000 7f894d0bb700 1 --2- 172.21.15.62:0/3289593038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89481066f0 0x7f8948198a80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8940007ee0 tx=0x7f894000d3b0 comp rx=0 tx=0).stop 2024-06-10T00:30:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.830+0000 7f894759e700 1 -- 172.21.15.62:0/3289593038 reap_dead start 2024-06-10T00:30:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.830+0000 7f894d0bb700 1 -- 172.21.15.62:0/3289593038 shutdown_connections 2024-06-10T00:30:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.830+0000 7f894d0bb700 1 -- 172.21.15.62:0/3289593038 >> 172.21.15.62:0/3289593038 conn(0x7f8948078430 msgr2=0x7f894806aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:30:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.830+0000 7f894d0bb700 1 -- 172.21.15.62:0/3289593038 shutdown_connections 2024-06-10T00:30:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:30:52.831+0000 7f894d0bb700 1 -- 172.21.15.62:0/3289593038 wait complete. 2024-06-10T00:30:53.419 INFO:teuthology.orchestra.run.smithi062.stdout:163208757320 2024-06-10T00:30:53.420 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-10T00:30:54.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:53 smithi062 ceph-mon[32390]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:30:54.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:53 smithi105 ceph-mon[36321]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:30:54.049 INFO:teuthology.orchestra.run.smithi062.stdout:287762808884 2024-06-10T00:30:54.050 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-10T00:30:54.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:53 smithi138 ceph-mon[36549]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:30:54.441 INFO:teuthology.orchestra.run.smithi062.stdout:343597383723 2024-06-10T00:30:54.441 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-10T00:30:54.969 INFO:teuthology.orchestra.run.smithi062.stdout:197568495683 2024-06-10T00:30:54.970 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-10T00:30:55.387 INFO:teuthology.orchestra.run.smithi062.stdout:55834574941 2024-06-10T00:30:55.387 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-10T00:30:55.950 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:55 smithi062 ceph-mon[32390]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:30:55.990 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:55.991 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:56.011 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:56.015 INFO:teuthology.orchestra.run.smithi062.stdout:253403070521 2024-06-10T00:30:56.016 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-10T00:30:56.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:55 smithi105 ceph-mon[36321]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:30:56.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:55 smithi138 ceph-mon[36549]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:30:56.322 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:57.823 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:57 smithi062 ceph-mon[32390]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:30:58.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:57 smithi105 ceph-mon[36321]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:30:58.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:57 smithi138 ceph-mon[36549]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:30:58.263 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:59.100 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:59.104 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:59.105 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:59.108 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:30:59.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:30:59 smithi062 ceph-mon[32390]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-06-10T00:31:00.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:30:59 smithi105 ceph-mon[36321]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-06-10T00:31:00.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:30:59 smithi138 ceph-mon[36549]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-06-10T00:31:01.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.460+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/1751046202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9806b3c0 msgr2=0x7f9c9810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:01.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.460+0000 7f9c9e3aa700 1 --2- 172.21.15.62:0/1751046202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9806b3c0 0x7f9c9810f310 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000b0a0 tx=0x7f9c9000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:31:01.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.460+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/1751046202 shutdown_connections 2024-06-10T00:31:01.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.460+0000 7f9c9e3aa700 1 --2- 172.21.15.62:0/1751046202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9806b3c0 0x7f9c9810f310 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:01.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.460+0000 7f9c9e3aa700 1 --2- 172.21.15.62:0/1751046202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c981084f0 0x7f9c981088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.460+0000 7f9c9e3aa700 1 --2- 172.21.15.62:0/1751046202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c9806aa00 0x7f9c9806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.460+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/1751046202 >> 172.21.15.62:0/1751046202 conn(0x7f9c98078300 msgr2=0x7f9c98078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.460+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/1751046202 shutdown_connections 2024-06-10T00:31:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.460+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/1751046202 wait complete. 2024-06-10T00:31:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.461+0000 7f9c9e3aa700 1 Processor -- start 2024-06-10T00:31:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.461+0000 7f9c9e3aa700 1 -- start start 2024-06-10T00:31:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.462+0000 7f9c9e3aa700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c9806aa00 0x7f9c9819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.462+0000 7f9c9e3aa700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c9806b3c0 0x7f9c9819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.462+0000 7f9c9e3aa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c981084f0 0x7f9c981967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.462+0000 7f9c9e3aa700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9c981105c0 con 0x7f9c981084f0 2024-06-10T00:31:01.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.462+0000 7f9c9e3aa700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98110440 con 0x7f9c9806aa00 2024-06-10T00:31:01.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.462+0000 7f9c9c947700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c981084f0 0x7f9c981967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:01.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.462+0000 7f9c9e3aa700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98110740 con 0x7f9c9806b3c0 2024-06-10T00:31:01.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.462+0000 7f9c977fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c9806b3c0 0x7f9c9819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:01.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.462+0000 7f9c9c947700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c981084f0 0x7f9c981967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51402/0 (socket says 172.21.15.62:51402) 2024-06-10T00:31:01.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.462+0000 7f9c9c947700 1 -- 172.21.15.62:0/2902027765 learned_addr learned my addr 172.21.15.62:0/2902027765 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.462+0000 7f9c97fff700 1 --2- 172.21.15.62:0/2902027765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c9806aa00 0x7f9c9819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.463+0000 7f9c9c947700 1 -- 172.21.15.62:0/2902027765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c9806b3c0 msgr2=0x7f9c9819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.463+0000 7f9c9c947700 1 --2- 172.21.15.62:0/2902027765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c9806b3c0 0x7f9c9819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.463+0000 7f9c9c947700 1 -- 172.21.15.62:0/2902027765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c9806aa00 msgr2=0x7f9c9819c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.463+0000 7f9c9c947700 1 --2- 172.21.15.62:0/2902027765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c9806aa00 0x7f9c9819c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.463+0000 7f9c9c947700 1 -- 172.21.15.62:0/2902027765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c98196fa0 con 0x7f9c981084f0 2024-06-10T00:31:01.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.463+0000 7f9c9c947700 1 --2- 172.21.15.62:0/2902027765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c981084f0 0x7f9c981967c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000b780 tx=0x7f9c90004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:01.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.463+0000 7f9c957fa700 1 -- 172.21.15.62:0/2902027765 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90021070 con 0x7f9c981084f0 2024-06-10T00:31:01.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.463+0000 7f9c957fa700 1 -- 172.21.15.62:0/2902027765 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c90007b80 con 0x7f9c981084f0 2024-06-10T00:31:01.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.463+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/2902027765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c98197230 con 0x7f9c981084f0 2024-06-10T00:31:01.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.464+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/2902027765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c981a34a0 con 0x7f9c981084f0 2024-06-10T00:31:01.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.464+0000 7f9c957fa700 1 -- 172.21.15.62:0/2902027765 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c900052a0 con 0x7f9c981084f0 2024-06-10T00:31:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.466+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/2902027765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c84005330 con 0x7f9c981084f0 2024-06-10T00:31:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.466+0000 7f9c957fa700 1 -- 172.21.15.62:0/2902027765 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9c90024070 con 0x7f9c981084f0 2024-06-10T00:31:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.467+0000 7f9c957fa700 1 --2- 172.21.15.62:0/2902027765 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9c800a2cb0 0x7f9c800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.467+0000 7f9c97fff700 1 --2- 172.21.15.62:0/2902027765 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9c800a2cb0 0x7f9c800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.467+0000 7f9c957fa700 1 -- 172.21.15.62:0/2902027765 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f9c900d03d0 con 0x7f9c981084f0 2024-06-10T00:31:01.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.468+0000 7f9c97fff700 1 --2- 172.21.15.62:0/2902027765 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9c800a2cb0 0x7f9c800a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800d640 tx=0x7f9c88018040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:01.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.473+0000 7f9c957fa700 1 -- 172.21.15.62:0/2902027765 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9c9009ce90 con 0x7f9c981084f0 2024-06-10T00:31:01.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.704+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/2902027765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f9c840058f0 con 0x7f9c981084f0 2024-06-10T00:31:01.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.705+0000 7f9c957fa700 1 -- 172.21.15.62:0/2902027765 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f9c900a0b40 con 0x7f9c981084f0 2024-06-10T00:31:01.706 INFO:teuthology.orchestra.run.smithi062.stdout:309237645361 2024-06-10T00:31:01.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.707+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/2902027765 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9c800a2cb0 msgr2=0x7f9c800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:01.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.707+0000 7f9c9e3aa700 1 --2- 172.21.15.62:0/2902027765 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9c800a2cb0 0x7f9c800a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800d640 tx=0x7f9c88018040 comp rx=0 tx=0).stop 2024-06-10T00:31:01.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.707+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/2902027765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c981084f0 msgr2=0x7f9c981967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:01.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.707+0000 7f9c9e3aa700 1 --2- 172.21.15.62:0/2902027765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c981084f0 0x7f9c981967c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000b780 tx=0x7f9c90004d00 comp rx=0 tx=0).stop 2024-06-10T00:31:01.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.707+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/2902027765 shutdown_connections 2024-06-10T00:31:01.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.707+0000 7f9c9e3aa700 1 --2- 172.21.15.62:0/2902027765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c981084f0 0x7f9c981967c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:01.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.707+0000 7f9c9e3aa700 1 --2- 172.21.15.62:0/2902027765 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f9c800a2cb0 0x7f9c800a5170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:01.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.707+0000 7f9c9e3aa700 1 --2- 172.21.15.62:0/2902027765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c9806aa00 0x7f9c9819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:01.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.707+0000 7f9c9e3aa700 1 --2- 172.21.15.62:0/2902027765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c9806b3c0 0x7f9c9819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.707+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/2902027765 >> 172.21.15.62:0/2902027765 conn(0x7f9c98078300 msgr2=0x7f9c980ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.707+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/2902027765 shutdown_connections 2024-06-10T00:31:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:01.707+0000 7f9c9e3aa700 1 -- 172.21.15.62:0/2902027765 wait complete. 2024-06-10T00:31:01.832 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:01 smithi062 ceph-mon[32390]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-06-10T00:31:01.832 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:01 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2902027765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-10T00:31:02.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:01 smithi105 ceph-mon[36321]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-06-10T00:31:02.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:01 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2902027765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-10T00:31:02.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:01 smithi138 ceph-mon[36549]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-06-10T00:31:02.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:01 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2902027765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-10T00:31:03.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:03 smithi062 ceph-mon[32390]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-06-10T00:31:04.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:03 smithi105 ceph-mon[36321]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-06-10T00:31:04.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.082+0000 7f7d057cf700 1 -- 172.21.15.62:0/57623616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d001030f0 msgr2=0x7f7d00107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.082+0000 7f7d057cf700 1 --2- 172.21.15.62:0/57623616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d001030f0 0x7f7d00107110 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40099c0 tx=0x7f7cf4009cd0 comp rx=0 tx=0).stop 2024-06-10T00:31:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.082+0000 7f7d057cf700 1 -- 172.21.15.62:0/57623616 shutdown_connections 2024-06-10T00:31:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.082+0000 7f7d057cf700 1 --2- 172.21.15.62:0/57623616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d00107840 0x7f7d0010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.082+0000 7f7d057cf700 1 --2- 172.21.15.62:0/57623616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00102740 0x7f7d00102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.082+0000 7f7d057cf700 1 --2- 172.21.15.62:0/57623616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d001030f0 0x7f7d00107110 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.082+0000 7f7d057cf700 1 -- 172.21.15.62:0/57623616 >> 172.21.15.62:0/57623616 conn(0x7f7d000795f0 msgr2=0x7f7d00079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:04.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.082+0000 7f7d057cf700 1 -- 172.21.15.62:0/57623616 shutdown_connections 2024-06-10T00:31:04.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.082+0000 7f7d057cf700 1 -- 172.21.15.62:0/57623616 wait complete. 2024-06-10T00:31:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.083+0000 7f7d057cf700 1 Processor -- start 2024-06-10T00:31:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.083+0000 7f7d057cf700 1 -- start start 2024-06-10T00:31:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.083+0000 7f7d057cf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00102740 0x7f7d00074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.083+0000 7f7d057cf700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d001030f0 0x7f7d00074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.083+0000 7f7d057cf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d00107840 0x7f7d00075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:04.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.083+0000 7f7d057cf700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7d001127d0 con 0x7f7d00107840 2024-06-10T00:31:04.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.083+0000 7f7d057cf700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7d00112650 con 0x7f7d001030f0 2024-06-10T00:31:04.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.083+0000 7f7d057cf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d00112950 con 0x7f7d00102740 2024-06-10T00:31:04.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7cff7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d00107840 0x7f7d00075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:04.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7cff7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d00107840 0x7f7d00075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51428/0 (socket says 172.21.15.62:51428) 2024-06-10T00:31:04.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7cff7fe700 1 -- 172.21.15.62:0/2433689474 learned_addr learned my addr 172.21.15.62:0/2433689474 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:04.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7cfeffd700 1 --2- 172.21.15.62:0/2433689474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00102740 0x7f7d00074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:04.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7cfe7fc700 1 --2- 172.21.15.62:0/2433689474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d001030f0 0x7f7d00074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:04.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7cff7fe700 1 -- 172.21.15.62:0/2433689474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00102740 msgr2=0x7f7d00074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:04.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7cff7fe700 1 --2- 172.21.15.62:0/2433689474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00102740 0x7f7d00074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:04.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7cff7fe700 1 -- 172.21.15.62:0/2433689474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d001030f0 msgr2=0x7f7d00074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:04.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7cff7fe700 1 --2- 172.21.15.62:0/2433689474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d001030f0 0x7f7d00074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7cff7fe700 1 -- 172.21.15.62:0/2433689474 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d001a7110 con 0x7f7d00107840 2024-06-10T00:31:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7cff7fe700 1 --2- 172.21.15.62:0/2433689474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d00107840 0x7f7d00075210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8007d70 tx=0x7f7cf800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7ce7fff700 1 -- 172.21.15.62:0/2433689474 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8016020 con 0x7f7d00107840 2024-06-10T00:31:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7ce7fff700 1 -- 172.21.15.62:0/2433689474 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cf800ad40 con 0x7f7d00107840 2024-06-10T00:31:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7d057cf700 1 -- 172.21.15.62:0/2433689474 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d001a7400 con 0x7f7d00107840 2024-06-10T00:31:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7ce7fff700 1 -- 172.21.15.62:0/2433689474 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf80116f0 con 0x7f7d00107840 2024-06-10T00:31:04.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.084+0000 7f7d057cf700 1 -- 172.21.15.62:0/2433689474 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d001a7910 con 0x7f7d00107840 2024-06-10T00:31:04.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.085+0000 7f7d057cf700 1 -- 172.21.15.62:0/2433689474 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d00103bd0 con 0x7f7d00107840 2024-06-10T00:31:04.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.087+0000 7f7ce7fff700 1 -- 172.21.15.62:0/2433689474 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7cf8019020 con 0x7f7d00107840 2024-06-10T00:31:04.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.088+0000 7f7ce7fff700 1 --2- 172.21.15.62:0/2433689474 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7ce80a2cb0 0x7f7ce80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:04.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.088+0000 7f7cfeffd700 1 --2- 172.21.15.62:0/2433689474 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7ce80a2cb0 0x7f7ce80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:04.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.088+0000 7f7ce7fff700 1 -- 172.21.15.62:0/2433689474 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f7cf80cfab0 con 0x7f7d00107840 2024-06-10T00:31:04.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.088+0000 7f7cfeffd700 1 --2- 172.21.15.62:0/2433689474 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7ce80a2cb0 0x7f7ce80a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0006fd0 tx=0x7f7cf0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:04.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.091+0000 7f7ce7fff700 1 -- 172.21.15.62:0/2433689474 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7cf8020020 con 0x7f7d00107840 2024-06-10T00:31:04.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:03 smithi138 ceph-mon[36549]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-06-10T00:31:04.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.301+0000 7f7d057cf700 1 -- 172.21.15.62:0/2433689474 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7d0002dca0 con 0x7f7d00107840 2024-06-10T00:31:04.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.301+0000 7f7ce7fff700 1 -- 172.21.15.62:0/2433689474 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7cf8022300 con 0x7f7d00107840 2024-06-10T00:31:04.302 INFO:teuthology.orchestra.run.smithi062.stdout:77309411419 2024-06-10T00:31:04.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.303+0000 7f7d057cf700 1 -- 172.21.15.62:0/2433689474 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7ce80a2cb0 msgr2=0x7f7ce80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:04.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.303+0000 7f7d057cf700 1 --2- 172.21.15.62:0/2433689474 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7ce80a2cb0 0x7f7ce80a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0006fd0 tx=0x7f7cf0008040 comp rx=0 tx=0).stop 2024-06-10T00:31:04.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.303+0000 7f7d057cf700 1 -- 172.21.15.62:0/2433689474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d00107840 msgr2=0x7f7d00075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:04.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.303+0000 7f7d057cf700 1 --2- 172.21.15.62:0/2433689474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d00107840 0x7f7d00075210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8007d70 tx=0x7f7cf800a340 comp rx=0 tx=0).stop 2024-06-10T00:31:04.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.303+0000 7f7d057cf700 1 -- 172.21.15.62:0/2433689474 shutdown_connections 2024-06-10T00:31:04.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.303+0000 7f7d057cf700 1 --2- 172.21.15.62:0/2433689474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d00107840 0x7f7d00075210 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:04.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.303+0000 7f7d057cf700 1 --2- 172.21.15.62:0/2433689474 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f7ce80a2cb0 0x7f7ce80a5170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:04.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.303+0000 7f7d057cf700 1 --2- 172.21.15.62:0/2433689474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00102740 0x7f7d00074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:04.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.303+0000 7f7d057cf700 1 --2- 172.21.15.62:0/2433689474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d001030f0 0x7f7d00074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:04.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.303+0000 7f7d057cf700 1 -- 172.21.15.62:0/2433689474 >> 172.21.15.62:0/2433689474 conn(0x7f7d000795f0 msgr2=0x7f7d001004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:04.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.303+0000 7f7d057cf700 1 -- 172.21.15.62:0/2433689474 shutdown_connections 2024-06-10T00:31:04.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:04.303+0000 7f7d057cf700 1 -- 172.21.15.62:0/2433689474 wait complete. 2024-06-10T00:31:05.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:04 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2433689474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-10T00:31:05.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:04 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2433689474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-10T00:31:05.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:04 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2433689474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-10T00:31:05.148 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:05.148 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:05.150 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:05.771 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645358 got 309237645361 for osd.10 2024-06-10T00:31:05.771 DEBUG:teuthology.parallel:result is None 2024-06-10T00:31:06.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:05 smithi105 ceph-mon[36321]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:31:06.038 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:05 smithi062 ceph-mon[32390]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:31:06.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:05 smithi138 ceph-mon[36549]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:31:06.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.144+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3981170192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac108500 msgr2=0x7f43ac1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:06.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.144+0000 7f43b0a3a700 1 --2- 172.21.15.62:0/3981170192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac108500 0x7f43ac1088e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f439c009990 tx=0x7f439c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:31:06.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.144+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3981170192 shutdown_connections 2024-06-10T00:31:06.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.144+0000 7f43b0a3a700 1 --2- 172.21.15.62:0/3981170192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac108500 0x7f43ac1088e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:06.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.144+0000 7f43b0a3a700 1 --2- 172.21.15.62:0/3981170192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43ac06ab30 0x7f43ac10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:06.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.144+0000 7f43b0a3a700 1 --2- 172.21.15.62:0/3981170192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43ac06a170 0x7f43ac06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:06.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.144+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3981170192 >> 172.21.15.62:0/3981170192 conn(0x7f43ac078340 msgr2=0x7f43ac078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.144+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3981170192 shutdown_connections 2024-06-10T00:31:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.144+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3981170192 wait complete. 2024-06-10T00:31:06.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.145+0000 7f43b0a3a700 1 Processor -- start 2024-06-10T00:31:06.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.145+0000 7f43b0a3a700 1 -- start start 2024-06-10T00:31:06.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.145+0000 7f43b0a3a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43ac06a170 0x7f43ac19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:06.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.145+0000 7f43b0a3a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43ac06ab30 0x7f43ac19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:06.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.145+0000 7f43b0a3a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac108500 0x7f43ac198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:06.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.145+0000 7f43b0a3a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac112780 con 0x7f43ac108500 2024-06-10T00:31:06.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.145+0000 7f43b0a3a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac112600 con 0x7f43ac06ab30 2024-06-10T00:31:06.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.145+0000 7f43b0a3a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac112900 con 0x7f43ac06a170 2024-06-10T00:31:06.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.146+0000 7f43aad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac108500 0x7f43ac198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:06.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.146+0000 7f43a9d9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43ac06ab30 0x7f43ac19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:06.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.146+0000 7f43aad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac108500 0x7f43ac198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51448/0 (socket says 172.21.15.62:51448) 2024-06-10T00:31:06.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.146+0000 7f43aad9d700 1 -- 172.21.15.62:0/3472483224 learned_addr learned my addr 172.21.15.62:0/3472483224 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:06.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.146+0000 7f43aa59c700 1 --2- 172.21.15.62:0/3472483224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43ac06a170 0x7f43ac19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:06.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.146+0000 7f43aad9d700 1 -- 172.21.15.62:0/3472483224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43ac06a170 msgr2=0x7f43ac19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:06.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.146+0000 7f43aad9d700 1 --2- 172.21.15.62:0/3472483224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43ac06a170 0x7f43ac19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:06.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.146+0000 7f43aad9d700 1 -- 172.21.15.62:0/3472483224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43ac06ab30 msgr2=0x7f43ac19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:06.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.146+0000 7f43aad9d700 1 --2- 172.21.15.62:0/3472483224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43ac06ab30 0x7f43ac19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:06.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.146+0000 7f43aad9d700 1 -- 172.21.15.62:0/3472483224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ac199240 con 0x7f43ac108500 2024-06-10T00:31:06.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.146+0000 7f43a9d9b700 1 --2- 172.21.15.62:0/3472483224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43ac06ab30 0x7f43ac19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:31:06.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.146+0000 7f43aad9d700 1 --2- 172.21.15.62:0/3472483224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac108500 0x7f43ac198ae0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f43a4007ee0 tx=0x7f43a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:06.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.146+0000 7f43a37fe700 1 -- 172.21.15.62:0/3472483224 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4004bb0 con 0x7f43ac108500 2024-06-10T00:31:06.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.147+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3472483224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ac199530 con 0x7f43ac108500 2024-06-10T00:31:06.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.147+0000 7f43a37fe700 1 -- 172.21.15.62:0/3472483224 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43a4004d50 con 0x7f43ac108500 2024-06-10T00:31:06.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.147+0000 7f43a37fe700 1 -- 172.21.15.62:0/3472483224 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4005960 con 0x7f43ac108500 2024-06-10T00:31:06.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.147+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3472483224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ac1a57f0 con 0x7f43ac108500 2024-06-10T00:31:06.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.148+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3472483224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43ac10a9e0 con 0x7f43ac108500 2024-06-10T00:31:06.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.149+0000 7f43a37fe700 1 -- 172.21.15.62:0/3472483224 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f43a401a020 con 0x7f43ac108500 2024-06-10T00:31:06.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.150+0000 7f43a37fe700 1 --2- 172.21.15.62:0/3472483224 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f43980a2d30 0x7f43980a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:06.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.150+0000 7f43a37fe700 1 -- 172.21.15.62:0/3472483224 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f43a40cecd0 con 0x7f43ac108500 2024-06-10T00:31:06.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.150+0000 7f43aa59c700 1 --2- 172.21.15.62:0/3472483224 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f43980a2d30 0x7f43980a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:06.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.151+0000 7f43aa59c700 1 --2- 172.21.15.62:0/3472483224 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f43980a2d30 0x7f43980a51f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f439c00b5c0 tx=0x7f439c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:06.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.155+0000 7f43a37fe700 1 -- 172.21.15.62:0/3472483224 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f43a40d4020 con 0x7f43ac108500 2024-06-10T00:31:06.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.368+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3472483224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f43ac199fd0 con 0x7f43ac108500 2024-06-10T00:31:06.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.368+0000 7f43a37fe700 1 -- 172.21.15.62:0/3472483224 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f43a4010070 con 0x7f43ac108500 2024-06-10T00:31:06.369 INFO:teuthology.orchestra.run.smithi062.stdout:34359738468 2024-06-10T00:31:06.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.371+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3472483224 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f43980a2d30 msgr2=0x7f43980a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:06.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.371+0000 7f43b0a3a700 1 --2- 172.21.15.62:0/3472483224 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f43980a2d30 0x7f43980a51f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f439c00b5c0 tx=0x7f439c005960 comp rx=0 tx=0).stop 2024-06-10T00:31:06.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.371+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3472483224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac108500 msgr2=0x7f43ac198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:06.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.371+0000 7f43b0a3a700 1 --2- 172.21.15.62:0/3472483224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac108500 0x7f43ac198ae0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f43a4007ee0 tx=0x7f43a400d3b0 comp rx=0 tx=0).stop 2024-06-10T00:31:06.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.371+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3472483224 shutdown_connections 2024-06-10T00:31:06.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.371+0000 7f43b0a3a700 1 --2- 172.21.15.62:0/3472483224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f43ac108500 0x7f43ac198ae0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:06.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.371+0000 7f43b0a3a700 1 --2- 172.21.15.62:0/3472483224 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f43980a2d30 0x7f43980a51f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:06.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.371+0000 7f43b0a3a700 1 --2- 172.21.15.62:0/3472483224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43ac06a170 0x7f43ac19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:06.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.371+0000 7f43b0a3a700 1 --2- 172.21.15.62:0/3472483224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43ac06ab30 0x7f43ac19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:06.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.371+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3472483224 >> 172.21.15.62:0/3472483224 conn(0x7f43ac078340 msgr2=0x7f43ac100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:06.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.371+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3472483224 shutdown_connections 2024-06-10T00:31:06.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:06.371+0000 7f43b0a3a700 1 -- 172.21.15.62:0/3472483224 wait complete. 2024-06-10T00:31:07.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:06 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3472483224' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-10T00:31:07.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:06 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3472483224' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-10T00:31:07.060 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411415 got 77309411419 for osd.2 2024-06-10T00:31:07.060 DEBUG:teuthology.parallel:result is None 2024-06-10T00:31:07.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:06 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3472483224' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-10T00:31:07.503 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738463 got 34359738468 for osd.0 2024-06-10T00:31:07.504 DEBUG:teuthology.parallel:result is None 2024-06-10T00:31:07.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.687+0000 7fa1f79a2700 1 -- 172.21.15.62:0/4254329612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f0108500 msgr2=0x7fa1f01088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:07.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.687+0000 7fa1f79a2700 1 --2- 172.21.15.62:0/4254329612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f0108500 0x7fa1f01088e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009990 tx=0x7fa1ec009ca0 comp rx=0 tx=0).stop 2024-06-10T00:31:07.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.687+0000 7fa1f79a2700 1 -- 172.21.15.62:0/4254329612 shutdown_connections 2024-06-10T00:31:07.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.687+0000 7fa1f79a2700 1 --2- 172.21.15.62:0/4254329612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f0108500 0x7fa1f01088e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:07.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.687+0000 7fa1f79a2700 1 --2- 172.21.15.62:0/4254329612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f006ab30 0x7fa1f010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:07.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.687+0000 7fa1f79a2700 1 --2- 172.21.15.62:0/4254329612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f006a170 0x7fa1f006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:07.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.687+0000 7fa1f79a2700 1 -- 172.21.15.62:0/4254329612 >> 172.21.15.62:0/4254329612 conn(0x7fa1f0078340 msgr2=0x7fa1f0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:07.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.687+0000 7fa1f79a2700 1 -- 172.21.15.62:0/4254329612 shutdown_connections 2024-06-10T00:31:07.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.687+0000 7fa1f79a2700 1 -- 172.21.15.62:0/4254329612 wait complete. 2024-06-10T00:31:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.688+0000 7fa1f79a2700 1 Processor -- start 2024-06-10T00:31:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.688+0000 7fa1f79a2700 1 -- start start 2024-06-10T00:31:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.688+0000 7fa1f79a2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f006a170 0x7fa1f019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:07.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.688+0000 7fa1f79a2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f006ab30 0x7fa1f019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:07.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.688+0000 7fa1f79a2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f0108500 0x7fa1f019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:07.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.688+0000 7fa1f79a2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0114980 con 0x7fa1f006ab30 2024-06-10T00:31:07.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.688+0000 7fa1f79a2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0114800 con 0x7fa1f006a170 2024-06-10T00:31:07.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.688+0000 7fa1f79a2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0114b00 con 0x7fa1f0108500 2024-06-10T00:31:07.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.688+0000 7fa1f4f3d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f006ab30 0x7fa1f019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:07.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.688+0000 7fa1f573e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f006a170 0x7fa1f019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:07.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.688+0000 7fa1f4f3d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f006ab30 0x7fa1f019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51474/0 (socket says 172.21.15.62:51474) 2024-06-10T00:31:07.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.688+0000 7fa1f4f3d700 1 -- 172.21.15.62:0/3643539101 learned_addr learned my addr 172.21.15.62:0/3643539101 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:07.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.689+0000 7fa1f5f3f700 1 --2- 172.21.15.62:0/3643539101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f0108500 0x7fa1f019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:07.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.689+0000 7fa1f573e700 1 -- 172.21.15.62:0/3643539101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f0108500 msgr2=0x7fa1f019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:07.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.689+0000 7fa1f573e700 1 --2- 172.21.15.62:0/3643539101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f0108500 0x7fa1f019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:07.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.689+0000 7fa1f573e700 1 -- 172.21.15.62:0/3643539101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f006ab30 msgr2=0x7fa1f019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:07.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.689+0000 7fa1f573e700 1 --2- 172.21.15.62:0/3643539101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f006ab30 0x7fa1f019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:07.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.689+0000 7fa1f573e700 1 -- 172.21.15.62:0/3643539101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f019fe40 con 0x7fa1f006a170 2024-06-10T00:31:07.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.689+0000 7fa1f573e700 1 --2- 172.21.15.62:0/3643539101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f006a170 0x7fa1f019ae30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec006010 tx=0x7fa1ec00bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:07.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.689+0000 7fa1e67fc700 1 -- 172.21.15.62:0/3643539101 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec027070 con 0x7fa1f006a170 2024-06-10T00:31:07.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.689+0000 7fa1e67fc700 1 -- 172.21.15.62:0/3643539101 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ec003d70 con 0x7fa1f006a170 2024-06-10T00:31:07.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.689+0000 7fa1f79a2700 1 -- 172.21.15.62:0/3643539101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f01a00d0 con 0x7fa1f006a170 2024-06-10T00:31:07.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.689+0000 7fa1e67fc700 1 -- 172.21.15.62:0/3643539101 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec022dc0 con 0x7fa1f006a170 2024-06-10T00:31:07.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.689+0000 7fa1f79a2700 1 -- 172.21.15.62:0/3643539101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f01a79b0 con 0x7fa1f006a170 2024-06-10T00:31:07.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.690+0000 7fa1f79a2700 1 -- 172.21.15.62:0/3643539101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f010a9e0 con 0x7fa1f006a170 2024-06-10T00:31:07.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.692+0000 7fa1e67fc700 1 -- 172.21.15.62:0/3643539101 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa1ec02a070 con 0x7fa1f006a170 2024-06-10T00:31:07.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.692+0000 7fa1e67fc700 1 --2- 172.21.15.62:0/3643539101 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa1dc0a2cb0 0x7fa1dc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:07.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.692+0000 7fa1f4f3d700 1 --2- 172.21.15.62:0/3643539101 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa1dc0a2cb0 0x7fa1dc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:07.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.692+0000 7fa1e67fc700 1 -- 172.21.15.62:0/3643539101 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fa1ec0d0cf0 con 0x7fa1f006a170 2024-06-10T00:31:07.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.693+0000 7fa1f4f3d700 1 --2- 172.21.15.62:0/3643539101 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa1dc0a2cb0 0x7fa1dc0a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00789b0 tx=0x7fa1e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:07.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.696+0000 7fa1e67fc700 1 -- 172.21.15.62:0/3643539101 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa1ec013070 con 0x7fa1f006a170 2024-06-10T00:31:07.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.911+0000 7fa1f79a2700 1 -- 172.21.15.62:0/3643539101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fa1f019c190 con 0x7fa1f006a170 2024-06-10T00:31:07.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.912+0000 7fa1e67fc700 1 -- 172.21.15.62:0/3643539101 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa1ec09d750 con 0x7fa1f006a170 2024-06-10T00:31:07.912 INFO:teuthology.orchestra.run.smithi062.stdout:219043332161 2024-06-10T00:31:07.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.915+0000 7fa1f79a2700 1 -- 172.21.15.62:0/3643539101 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa1dc0a2cb0 msgr2=0x7fa1dc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:07.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.915+0000 7fa1f79a2700 1 --2- 172.21.15.62:0/3643539101 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa1dc0a2cb0 0x7fa1dc0a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00789b0 tx=0x7fa1e0009260 comp rx=0 tx=0).stop 2024-06-10T00:31:07.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.915+0000 7fa1f79a2700 1 -- 172.21.15.62:0/3643539101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f006a170 msgr2=0x7fa1f019ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:07.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.915+0000 7fa1f79a2700 1 --2- 172.21.15.62:0/3643539101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f006a170 0x7fa1f019ae30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec006010 tx=0x7fa1ec00bab0 comp rx=0 tx=0).stop 2024-06-10T00:31:07.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.915+0000 7fa1f79a2700 1 -- 172.21.15.62:0/3643539101 shutdown_connections 2024-06-10T00:31:07.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.915+0000 7fa1f79a2700 1 --2- 172.21.15.62:0/3643539101 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fa1dc0a2cb0 0x7fa1dc0a5170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:07.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.915+0000 7fa1f79a2700 1 --2- 172.21.15.62:0/3643539101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f0108500 0x7fa1f019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:07.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.915+0000 7fa1f79a2700 1 --2- 172.21.15.62:0/3643539101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f006a170 0x7fa1f019ae30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:07.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.915+0000 7fa1f79a2700 1 --2- 172.21.15.62:0/3643539101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1f006ab30 0x7fa1f019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:07.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.915+0000 7fa1f79a2700 1 -- 172.21.15.62:0/3643539101 >> 172.21.15.62:0/3643539101 conn(0x7fa1f0078340 msgr2=0x7fa1f00fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:07.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.915+0000 7fa1f79a2700 1 -- 172.21.15.62:0/3643539101 shutdown_connections 2024-06-10T00:31:07.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:07.915+0000 7fa1f79a2700 1 -- 172.21.15.62:0/3643539101 wait complete. 2024-06-10T00:31:07.947 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:07 smithi062 ceph-mon[32390]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:31:08.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:07 smithi105 ceph-mon[36321]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:31:08.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:07 smithi138 ceph-mon[36549]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:31:08.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.719+0000 7f6e26d09700 1 -- 172.21.15.62:0/1447977064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e20100e30 msgr2=0x7f6e201012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:08.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.720+0000 7f6e26d09700 1 --2- 172.21.15.62:0/1447977064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e20100e30 0x7f6e201012b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009990 tx=0x7f6e0c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:31:08.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.720+0000 7f6e26d09700 1 -- 172.21.15.62:0/1447977064 shutdown_connections 2024-06-10T00:31:08.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.720+0000 7f6e26d09700 1 --2- 172.21.15.62:0/1447977064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e201017f0 0x7f6e2010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:08.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.720+0000 7f6e26d09700 1 --2- 172.21.15.62:0/1447977064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e20107550 0x7f6e20107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:08.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.720+0000 7f6e26d09700 1 --2- 172.21.15.62:0/1447977064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e20100e30 0x7f6e201012b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:08.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.720+0000 7f6e26d09700 1 -- 172.21.15.62:0/1447977064 >> 172.21.15.62:0/1447977064 conn(0x7f6e200766b0 msgr2=0x7f6e20076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:08.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.720+0000 7f6e26d09700 1 -- 172.21.15.62:0/1447977064 shutdown_connections 2024-06-10T00:31:08.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.720+0000 7f6e26d09700 1 -- 172.21.15.62:0/1447977064 wait complete. 2024-06-10T00:31:08.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.720+0000 7f6e26d09700 1 Processor -- start 2024-06-10T00:31:08.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.720+0000 7f6e26d09700 1 -- start start 2024-06-10T00:31:08.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e26d09700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e20100e30 0x7f6e2019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:08.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e26d09700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e201017f0 0x7f6e2019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:08.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e26d09700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e20107550 0x7f6e20198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:08.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e26d09700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6e20112790 con 0x7f6e201017f0 2024-06-10T00:31:08.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e26d09700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6e20112610 con 0x7f6e20107550 2024-06-10T00:31:08.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e26d09700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e20112910 con 0x7f6e20100e30 2024-06-10T00:31:08.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e201017f0 0x7f6e2019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:08.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e201017f0 0x7f6e2019ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58614/0 (socket says 172.21.15.62:58614) 2024-06-10T00:31:08.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e1ffff700 1 -- 172.21.15.62:0/3571436469 learned_addr learned my addr 172.21.15.62:0/3571436469 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:08.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e24aa5700 1 --2- 172.21.15.62:0/3571436469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e20100e30 0x7f6e2019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:08.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e252a6700 1 --2- 172.21.15.62:0/3571436469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e20107550 0x7f6e20198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:08.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e1ffff700 1 -- 172.21.15.62:0/3571436469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e20100e30 msgr2=0x7f6e2019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:08.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e1ffff700 1 --2- 172.21.15.62:0/3571436469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e20100e30 0x7f6e2019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:08.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e1ffff700 1 -- 172.21.15.62:0/3571436469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e20107550 msgr2=0x7f6e20198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:08.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e1ffff700 1 --2- 172.21.15.62:0/3571436469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e20107550 0x7f6e20198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:08.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e1ffff700 1 -- 172.21.15.62:0/3571436469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e20199250 con 0x7f6e201017f0 2024-06-10T00:31:08.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e1ffff700 1 --2- 172.21.15.62:0/3571436469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e201017f0 0x7f6e2019ee20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c000c00 tx=0x7f6e0c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:08.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.721+0000 7f6e1dffb700 1 -- 172.21.15.62:0/3571436469 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c027070 con 0x7f6e201017f0 2024-06-10T00:31:08.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.722+0000 7f6e26d09700 1 -- 172.21.15.62:0/3571436469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e201994e0 con 0x7f6e201017f0 2024-06-10T00:31:08.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.722+0000 7f6e1dffb700 1 -- 172.21.15.62:0/3571436469 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0c003e40 con 0x7f6e201017f0 2024-06-10T00:31:08.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.722+0000 7f6e1dffb700 1 -- 172.21.15.62:0/3571436469 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c022d10 con 0x7f6e201017f0 2024-06-10T00:31:08.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.722+0000 7f6e26d09700 1 -- 172.21.15.62:0/3571436469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e201a57b0 con 0x7f6e201017f0 2024-06-10T00:31:08.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.723+0000 7f6e26d09700 1 -- 172.21.15.62:0/3571436469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2006a6e0 con 0x7f6e201017f0 2024-06-10T00:31:08.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.724+0000 7f6e1dffb700 1 -- 172.21.15.62:0/3571436469 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6e0c02a070 con 0x7f6e201017f0 2024-06-10T00:31:08.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.725+0000 7f6e1dffb700 1 --2- 172.21.15.62:0/3571436469 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6e100a2cb0 0x7f6e100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:08.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.725+0000 7f6e24aa5700 1 --2- 172.21.15.62:0/3571436469 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6e100a2cb0 0x7f6e100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:08.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.725+0000 7f6e1dffb700 1 -- 172.21.15.62:0/3571436469 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f6e0c0d0ae0 con 0x7f6e201017f0 2024-06-10T00:31:08.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.725+0000 7f6e24aa5700 1 --2- 172.21.15.62:0/3571436469 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6e100a2cb0 0x7f6e100a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e200ff5f0 tx=0x7f6e1400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:08.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.729+0000 7f6e1dffb700 1 -- 172.21.15.62:0/3571436469 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6e0c031070 con 0x7f6e201017f0 2024-06-10T00:31:08.921 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:08 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3643539101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-10T00:31:08.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.942+0000 7f6e26d09700 1 -- 172.21.15.62:0/3571436469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6e2002dca0 con 0x7f6e201017f0 2024-06-10T00:31:08.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.942+0000 7f6e1dffb700 1 -- 172.21.15.62:0/3571436469 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6e0c013070 con 0x7f6e201017f0 2024-06-10T00:31:08.942 INFO:teuthology.orchestra.run.smithi062.stdout:163208757324 2024-06-10T00:31:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.944+0000 7f6e26d09700 1 -- 172.21.15.62:0/3571436469 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6e100a2cb0 msgr2=0x7f6e100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.944+0000 7f6e26d09700 1 --2- 172.21.15.62:0/3571436469 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6e100a2cb0 0x7f6e100a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e200ff5f0 tx=0x7f6e1400b410 comp rx=0 tx=0).stop 2024-06-10T00:31:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.944+0000 7f6e26d09700 1 -- 172.21.15.62:0/3571436469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e201017f0 msgr2=0x7f6e2019ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.944+0000 7f6e26d09700 1 --2- 172.21.15.62:0/3571436469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e201017f0 0x7f6e2019ee20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c000c00 tx=0x7f6e0c0047b0 comp rx=0 tx=0).stop 2024-06-10T00:31:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.944+0000 7f6e26d09700 1 -- 172.21.15.62:0/3571436469 shutdown_connections 2024-06-10T00:31:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.944+0000 7f6e26d09700 1 --2- 172.21.15.62:0/3571436469 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6e100a2cb0 0x7f6e100a5170 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.944+0000 7f6e26d09700 1 --2- 172.21.15.62:0/3571436469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e20107550 0x7f6e20198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.944+0000 7f6e26d09700 1 --2- 172.21.15.62:0/3571436469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e20100e30 0x7f6e2019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.944+0000 7f6e26d09700 1 --2- 172.21.15.62:0/3571436469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e201017f0 0x7f6e2019ee20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.944+0000 7f6e26d09700 1 -- 172.21.15.62:0/3571436469 >> 172.21.15.62:0/3571436469 conn(0x7f6e200766b0 msgr2=0x7f6e200ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.944+0000 7f6e26d09700 1 -- 172.21.15.62:0/3571436469 shutdown_connections 2024-06-10T00:31:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:08.945+0000 7f6e26d09700 1 -- 172.21.15.62:0/3571436469 wait complete. 2024-06-10T00:31:09.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:08 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3643539101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-10T00:31:09.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:08 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3643539101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-10T00:31:10.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:09 smithi105 ceph-mon[36321]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:31:10.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:09 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3571436469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-10T00:31:10.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:09 smithi062 ceph-mon[32390]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:31:10.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:09 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3571436469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-10T00:31:10.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:09 smithi138 ceph-mon[36549]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:31:10.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:09 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3571436469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-10T00:31:10.682 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332156 got 219043332161 for osd.7 2024-06-10T00:31:10.682 DEBUG:teuthology.parallel:result is None 2024-06-10T00:31:12.009 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:11 smithi062 ceph-mon[32390]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:31:12.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:11 smithi105 ceph-mon[36321]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:31:12.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:11 smithi138 ceph-mon[36549]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:31:13.420 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757320 got 163208757324 for osd.5 2024-06-10T00:31:13.420 DEBUG:teuthology.parallel:result is None 2024-06-10T00:31:14.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:13 smithi105 ceph-mon[36321]: pgmap v384: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:31:14.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:13 smithi062 ceph-mon[32390]: pgmap v384: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:31:14.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:13 smithi138 ceph-mon[36549]: pgmap v384: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:31:14.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.993+0000 7fd33d1a2700 1 -- 172.21.15.62:0/4088570426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3381073c0 msgr2=0x7fd33806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:14.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.993+0000 7fd33d1a2700 1 --2- 172.21.15.62:0/4088570426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3381073c0 0x7fd33806a1a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd328009990 tx=0x7fd328009ca0 comp rx=0 tx=0).stop 2024-06-10T00:31:14.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.993+0000 7fd33d1a2700 1 -- 172.21.15.62:0/4088570426 shutdown_connections 2024-06-10T00:31:14.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.993+0000 7fd33d1a2700 1 --2- 172.21.15.62:0/4088570426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd338104ed0 0x7fd338105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:14.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.993+0000 7fd33d1a2700 1 --2- 172.21.15.62:0/4088570426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3381073c0 0x7fd33806a1a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:14.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.993+0000 7fd33d1a2700 1 --2- 172.21.15.62:0/4088570426 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd33806a6e0 0x7fd33806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:14.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.993+0000 7fd33d1a2700 1 -- 172.21.15.62:0/4088570426 >> 172.21.15.62:0/4088570426 conn(0x7fd338078340 msgr2=0x7fd338078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:14.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.993+0000 7fd33d1a2700 1 -- 172.21.15.62:0/4088570426 shutdown_connections 2024-06-10T00:31:14.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.993+0000 7fd33d1a2700 1 -- 172.21.15.62:0/4088570426 wait complete. 2024-06-10T00:31:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.994+0000 7fd33d1a2700 1 Processor -- start 2024-06-10T00:31:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.994+0000 7fd33d1a2700 1 -- start start 2024-06-10T00:31:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.994+0000 7fd33d1a2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd33806a6e0 0x7fd33819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.994+0000 7fd33d1a2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd338104ed0 0x7fd33819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.994+0000 7fd33d1a2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3381073c0 0x7fd3381a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.994+0000 7fd33d1a2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd338118d70 con 0x7fd3381073c0 2024-06-10T00:31:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.994+0000 7fd33d1a2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd338118bf0 con 0x7fd33806a6e0 2024-06-10T00:31:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.994+0000 7fd33d1a2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd338118ef0 con 0x7fd338104ed0 2024-06-10T00:31:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.994+0000 7fd33759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3381073c0 0x7fd3381a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.994+0000 7fd33759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3381073c0 0x7fd3381a3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58622/0 (socket says 172.21.15.62:58622) 2024-06-10T00:31:14.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.994+0000 7fd33759e700 1 -- 172.21.15.62:0/927517518 learned_addr learned my addr 172.21.15.62:0/927517518 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.995+0000 7fd336d9d700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd33806a6e0 0x7fd33819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.995+0000 7fd33659c700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd338104ed0 0x7fd33819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.995+0000 7fd33759e700 1 -- 172.21.15.62:0/927517518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd338104ed0 msgr2=0x7fd33819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.995+0000 7fd33759e700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd338104ed0 0x7fd33819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.995+0000 7fd33759e700 1 -- 172.21.15.62:0/927517518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd33806a6e0 msgr2=0x7fd33819f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.995+0000 7fd33759e700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd33806a6e0 0x7fd33819f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.995+0000 7fd33759e700 1 -- 172.21.15.62:0/927517518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3381a4250 con 0x7fd3381073c0 2024-06-10T00:31:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.995+0000 7fd33659c700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd338104ed0 0x7fd33819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:31:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.995+0000 7fd33759e700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3381073c0 0x7fd3381a3b10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd33000ef30 tx=0x7fd33000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.995+0000 7fd31ffff700 1 -- 172.21.15.62:0/927517518 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3300092e0 con 0x7fd3381073c0 2024-06-10T00:31:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.995+0000 7fd31ffff700 1 -- 172.21.15.62:0/927517518 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd330004bb0 con 0x7fd3381073c0 2024-06-10T00:31:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.995+0000 7fd33d1a2700 1 -- 172.21.15.62:0/927517518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3381a44e0 con 0x7fd3381073c0 2024-06-10T00:31:14.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.995+0000 7fd31ffff700 1 -- 172.21.15.62:0/927517518 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330004280 con 0x7fd3381073c0 2024-06-10T00:31:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.995+0000 7fd33d1a2700 1 -- 172.21.15.62:0/927517518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3381abdc0 con 0x7fd3381073c0 2024-06-10T00:31:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.997+0000 7fd33d1a2700 1 -- 172.21.15.62:0/927517518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd338069180 con 0x7fd3381073c0 2024-06-10T00:31:14.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.999+0000 7fd31ffff700 1 -- 172.21.15.62:0/927517518 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd33001b030 con 0x7fd3381073c0 2024-06-10T00:31:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.999+0000 7fd31ffff700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd3200a2cb0 0x7fd3200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.999+0000 7fd31ffff700 1 -- 172.21.15.62:0/927517518 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fd3300cfa00 con 0x7fd3381073c0 2024-06-10T00:31:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:14.999+0000 7fd336d9d700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd3200a2cb0 0x7fd3200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:15.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.000+0000 7fd336d9d700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd3200a2cb0 0x7fd3200a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd32800b2e0 tx=0x7fd328005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:15.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.003+0000 7fd31ffff700 1 -- 172.21.15.62:0/927517518 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd330022020 con 0x7fd3381073c0 2024-06-10T00:31:15.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.216+0000 7fd33d1a2700 1 -- 172.21.15.62:0/927517518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd3381abf90 con 0x7fd3381073c0 2024-06-10T00:31:15.217 INFO:teuthology.orchestra.run.smithi062.stdout:197568495688 2024-06-10T00:31:15.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.216+0000 7fd31ffff700 1 -- 172.21.15.62:0/927517518 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd330010040 con 0x7fd3381073c0 2024-06-10T00:31:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.219+0000 7fd33d1a2700 1 -- 172.21.15.62:0/927517518 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd3200a2cb0 msgr2=0x7fd3200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.219+0000 7fd33d1a2700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd3200a2cb0 0x7fd3200a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd32800b2e0 tx=0x7fd328005960 comp rx=0 tx=0).stop 2024-06-10T00:31:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.219+0000 7fd33d1a2700 1 -- 172.21.15.62:0/927517518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3381073c0 msgr2=0x7fd3381a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.219+0000 7fd33d1a2700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3381073c0 0x7fd3381a3b10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd33000ef30 tx=0x7fd33000c3a0 comp rx=0 tx=0).stop 2024-06-10T00:31:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.219+0000 7fd33d1a2700 1 -- 172.21.15.62:0/927517518 shutdown_connections 2024-06-10T00:31:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.219+0000 7fd33d1a2700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd3381073c0 0x7fd3381a3b10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.219+0000 7fd33d1a2700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd3200a2cb0 0x7fd3200a5170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.219+0000 7fd33d1a2700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd33806a6e0 0x7fd33819f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:15.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.219+0000 7fd33d1a2700 1 --2- 172.21.15.62:0/927517518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd338104ed0 0x7fd33819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:15.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.219+0000 7fd33d1a2700 1 -- 172.21.15.62:0/927517518 >> 172.21.15.62:0/927517518 conn(0x7fd338078340 msgr2=0x7fd338100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:15.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.220+0000 7fd33d1a2700 1 -- 172.21.15.62:0/927517518 shutdown_connections 2024-06-10T00:31:15.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:15.220+0000 7fd33d1a2700 1 -- 172.21.15.62:0/927517518 wait complete. 2024-06-10T00:31:16.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:15 smithi105 ceph-mon[36321]: pgmap v385: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:31:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:15 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/927517518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-10T00:31:16.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:15 smithi062 ceph-mon[32390]: pgmap v385: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:31:16.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:15 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/927517518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-10T00:31:16.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:15 smithi138 ceph-mon[36549]: pgmap v385: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:31:16.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:15 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/927517518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-10T00:31:16.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.511+0000 7f3d76daf700 1 -- 172.21.15.62:0/2352214054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d70101d50 msgr2=0x7f3d7010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:16.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.511+0000 7f3d76daf700 1 --2- 172.21.15.62:0/2352214054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d70101d50 0x7f3d7010e240 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800b0a0 tx=0x7f3d6800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.511+0000 7f3d76daf700 1 -- 172.21.15.62:0/2352214054 shutdown_connections 2024-06-10T00:31:16.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.511+0000 7f3d76daf700 1 --2- 172.21.15.62:0/2352214054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d70101d50 0x7f3d7010e240 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.511+0000 7f3d76daf700 1 --2- 172.21.15.62:0/2352214054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d70107400 0x7f3d701077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.511+0000 7f3d76daf700 1 --2- 172.21.15.62:0/2352214054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d70101390 0x7f3d70101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.511+0000 7f3d76daf700 1 -- 172.21.15.62:0/2352214054 >> 172.21.15.62:0/2352214054 conn(0x7f3d70078340 msgr2=0x7f3d70078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:16.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.512+0000 7f3d76daf700 1 -- 172.21.15.62:0/2352214054 shutdown_connections 2024-06-10T00:31:16.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.512+0000 7f3d76daf700 1 -- 172.21.15.62:0/2352214054 wait complete. 2024-06-10T00:31:16.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.512+0000 7f3d76daf700 1 Processor -- start 2024-06-10T00:31:16.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.512+0000 7f3d76daf700 1 -- start start 2024-06-10T00:31:16.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.512+0000 7f3d76daf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d70101390 0x7f3d7019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:16.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.512+0000 7f3d76daf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d70101d50 0x7f3d7019dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:16.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.512+0000 7f3d76daf700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d70107400 0x7f3d70197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:16.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.512+0000 7f3d76daf700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3d70111670 con 0x7f3d70101390 2024-06-10T00:31:16.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.512+0000 7f3d76daf700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3d701114f0 con 0x7f3d70107400 2024-06-10T00:31:16.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.512+0000 7f3d76daf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d701117f0 con 0x7f3d70101d50 2024-06-10T00:31:16.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d74b4b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d70101390 0x7f3d7019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:16.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d74b4b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d70101390 0x7f3d7019d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58650/0 (socket says 172.21.15.62:58650) 2024-06-10T00:31:16.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d74b4b700 1 -- 172.21.15.62:0/1054885681 learned_addr learned my addr 172.21.15.62:0/1054885681 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:16.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d7534c700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d70107400 0x7f3d70197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:16.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d74b4b700 1 -- 172.21.15.62:0/1054885681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d70101d50 msgr2=0x7f3d7019dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:16.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d6ffff700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d70101d50 0x7f3d7019dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:16.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d74b4b700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d70101d50 0x7f3d7019dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d74b4b700 1 -- 172.21.15.62:0/1054885681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d70107400 msgr2=0x7f3d70197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:16.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d74b4b700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d70107400 0x7f3d70197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d74b4b700 1 -- 172.21.15.62:0/1054885681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d70198090 con 0x7f3d70101390 2024-06-10T00:31:16.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d6ffff700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d70101d50 0x7f3d7019dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:31:16.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d7534c700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d70107400 0x7f3d70197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:31:16.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d74b4b700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d70101390 0x7f3d7019d770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009970 tx=0x7f3d64009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:16.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d6dffb700 1 -- 172.21.15.62:0/1054885681 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6401c070 con 0x7f3d70101390 2024-06-10T00:31:16.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d76daf700 1 -- 172.21.15.62:0/1054885681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d70198380 con 0x7f3d70101390 2024-06-10T00:31:16.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d6dffb700 1 -- 172.21.15.62:0/1054885681 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d6400d260 con 0x7f3d70101390 2024-06-10T00:31:16.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d6dffb700 1 -- 172.21.15.62:0/1054885681 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64017be0 con 0x7f3d70101390 2024-06-10T00:31:16.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.513+0000 7f3d76daf700 1 -- 172.21.15.62:0/1054885681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7006a170 con 0x7f3d70101390 2024-06-10T00:31:16.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.514+0000 7f3d76daf700 1 -- 172.21.15.62:0/1054885681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d70069180 con 0x7f3d70101390 2024-06-10T00:31:16.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.516+0000 7f3d6dffb700 1 -- 172.21.15.62:0/1054885681 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3d6401f070 con 0x7f3d70101390 2024-06-10T00:31:16.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.516+0000 7f3d6dffb700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3d600a2cb0 0x7f3d600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:16.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.516+0000 7f3d6ffff700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3d600a2cb0 0x7f3d600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:16.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.516+0000 7f3d6dffb700 1 -- 172.21.15.62:0/1054885681 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f3d640d09b0 con 0x7f3d70101390 2024-06-10T00:31:16.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.517+0000 7f3d6ffff700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3d600a2cb0 0x7f3d600a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c005fd0 tx=0x7f3d5c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:16.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.520+0000 7f3d6dffb700 1 -- 172.21.15.62:0/1054885681 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3d6409d3e0 con 0x7f3d70101390 2024-06-10T00:31:16.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.744+0000 7f3d76daf700 1 -- 172.21.15.62:0/1054885681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3d7006ac00 con 0x7f3d70101390 2024-06-10T00:31:16.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.745+0000 7f3d6dffb700 1 -- 172.21.15.62:0/1054885681 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3d6409cce0 con 0x7f3d70101390 2024-06-10T00:31:16.753 INFO:teuthology.orchestra.run.smithi062.stdout:107374182488 2024-06-10T00:31:16.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.747+0000 7f3d76daf700 1 -- 172.21.15.62:0/1054885681 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3d600a2cb0 msgr2=0x7f3d600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:16.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.747+0000 7f3d76daf700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3d600a2cb0 0x7f3d600a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c005fd0 tx=0x7f3d5c005ee0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.747+0000 7f3d76daf700 1 -- 172.21.15.62:0/1054885681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d70101390 msgr2=0x7f3d7019d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:16.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.747+0000 7f3d76daf700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d70101390 0x7f3d7019d770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009970 tx=0x7f3d64009d30 comp rx=0 tx=0).stop 2024-06-10T00:31:16.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.747+0000 7f3d76daf700 1 -- 172.21.15.62:0/1054885681 shutdown_connections 2024-06-10T00:31:16.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.747+0000 7f3d76daf700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d70101390 0x7f3d7019d770 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.747+0000 7f3d76daf700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3d600a2cb0 0x7f3d600a5170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.747+0000 7f3d76daf700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d70107400 0x7f3d70197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.747+0000 7f3d76daf700 1 --2- 172.21.15.62:0/1054885681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d70101d50 0x7f3d7019dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.747+0000 7f3d76daf700 1 -- 172.21.15.62:0/1054885681 >> 172.21.15.62:0/1054885681 conn(0x7f3d70078340 msgr2=0x7f3d700ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:16.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.747+0000 7f3d76daf700 1 -- 172.21.15.62:0/1054885681 shutdown_connections 2024-06-10T00:31:16.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.747+0000 7f3d76daf700 1 -- 172.21.15.62:0/1054885681 wait complete. 2024-06-10T00:31:16.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.880+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/3993828560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b806abc0 msgr2=0x7fb9b806b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:16.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.880+0000 7fb9bdbc0700 1 --2- 172.21.15.62:0/3993828560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b806abc0 0x7fb9b806b040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009990 tx=0x7fb9ac009ca0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.880+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/3993828560 shutdown_connections 2024-06-10T00:31:16.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.880+0000 7fb9bdbc0700 1 --2- 172.21.15.62:0/3993828560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b806b580 0x7fb9b810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.880+0000 7fb9bdbc0700 1 --2- 172.21.15.62:0/3993828560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9b8108500 0x7fb9b81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.880+0000 7fb9bdbc0700 1 --2- 172.21.15.62:0/3993828560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b806abc0 0x7fb9b806b040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.880+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/3993828560 >> 172.21.15.62:0/3993828560 conn(0x7fb9b8078340 msgr2=0x7fb9b8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:16.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.880+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/3993828560 shutdown_connections 2024-06-10T00:31:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.880+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/3993828560 wait complete. 2024-06-10T00:31:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.880+0000 7fb9bdbc0700 1 Processor -- start 2024-06-10T00:31:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9bdbc0700 1 -- start start 2024-06-10T00:31:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9bdbc0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b806abc0 0x7fb9b8101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9bdbc0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b806b580 0x7fb9b81019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9bdbc0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9b8108500 0x7fb9b8101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9bdbc0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b8112760 con 0x7fb9b806abc0 2024-06-10T00:31:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9bdbc0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81125e0 con 0x7fb9b8108500 2024-06-10T00:31:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9bdbc0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81128e0 con 0x7fb9b806b580 2024-06-10T00:31:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9b77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b806abc0 0x7fb9b8101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9b77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b806abc0 0x7fb9b8101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58670/0 (socket says 172.21.15.62:58670) 2024-06-10T00:31:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9b77fe700 1 -- 172.21.15.62:0/1724991429 learned_addr learned my addr 172.21.15.62:0/1724991429 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9b7fff700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9b8108500 0x7fb9b8101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9b6ffd700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b806b580 0x7fb9b81019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9b77fe700 1 -- 172.21.15.62:0/1724991429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b806b580 msgr2=0x7fb9b81019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9b77fe700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b806b580 0x7fb9b81019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9b77fe700 1 -- 172.21.15.62:0/1724991429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9b8108500 msgr2=0x7fb9b8101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9b77fe700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9b8108500 0x7fb9b8101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9b77fe700 1 -- 172.21.15.62:0/1724991429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b81a71a0 con 0x7fb9b806abc0 2024-06-10T00:31:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.881+0000 7fb9b7fff700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9b8108500 0x7fb9b8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:31:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.882+0000 7fb9b77fe700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b806abc0 0x7fb9b8101490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800c920 tx=0x7fb9a800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.882+0000 7fb9b4ff9700 1 -- 172.21.15.62:0/1724991429 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8007b00 con 0x7fb9b806abc0 2024-06-10T00:31:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.882+0000 7fb9b4ff9700 1 -- 172.21.15.62:0/1724991429 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a8004d10 con 0x7fb9b806abc0 2024-06-10T00:31:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.882+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/1724991429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b81a7490 con 0x7fb9b806abc0 2024-06-10T00:31:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.882+0000 7fb9b4ff9700 1 -- 172.21.15.62:0/1724991429 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a800f450 con 0x7fb9b806abc0 2024-06-10T00:31:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.882+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/1724991429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b81a79d0 con 0x7fb9b806abc0 2024-06-10T00:31:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.883+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/1724991429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b8069180 con 0x7fb9b806abc0 2024-06-10T00:31:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.884+0000 7fb9b4ff9700 1 -- 172.21.15.62:0/1724991429 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb9a8004750 con 0x7fb9b806abc0 2024-06-10T00:31:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.885+0000 7fb9b4ff9700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb9a00a2cb0 0x7fb9a00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.885+0000 7fb9b6ffd700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb9a00a2cb0 0x7fb9a00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.885+0000 7fb9b4ff9700 1 -- 172.21.15.62:0/1724991429 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fb9a80cf5b0 con 0x7fb9b806abc0 2024-06-10T00:31:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.885+0000 7fb9b6ffd700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb9a00a2cb0 0x7fb9a00a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac006010 tx=0x7fb9ac005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:16.889+0000 7fb9b4ff9700 1 -- 172.21.15.62:0/1724991429 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb9a8014030 con 0x7fb9b806abc0 2024-06-10T00:31:17.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:16 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1054885681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-10T00:31:17.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:17.097+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/1724991429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb9b802dca0 con 0x7fb9b806abc0 2024-06-10T00:31:17.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:16 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1054885681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-10T00:31:17.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:17.097+0000 7fb9b4ff9700 1 -- 172.21.15.62:0/1724991429 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb9a809bfe0 con 0x7fb9b806abc0 2024-06-10T00:31:17.098 INFO:teuthology.orchestra.run.smithi062.stdout:55834574946 2024-06-10T00:31:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:17.100+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/1724991429 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb9a00a2cb0 msgr2=0x7fb9a00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:17.100+0000 7fb9bdbc0700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb9a00a2cb0 0x7fb9a00a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac006010 tx=0x7fb9ac005c40 comp rx=0 tx=0).stop 2024-06-10T00:31:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:17.100+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/1724991429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b806abc0 msgr2=0x7fb9b8101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:17.100+0000 7fb9bdbc0700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b806abc0 0x7fb9b8101490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800c920 tx=0x7fb9a800cce0 comp rx=0 tx=0).stop 2024-06-10T00:31:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:17.100+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/1724991429 shutdown_connections 2024-06-10T00:31:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:17.100+0000 7fb9bdbc0700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9b806abc0 0x7fb9b8101490 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:17.100+0000 7fb9bdbc0700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb9a00a2cb0 0x7fb9a00a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:17.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:17.100+0000 7fb9bdbc0700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9b8108500 0x7fb9b8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:17.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:17.100+0000 7fb9bdbc0700 1 --2- 172.21.15.62:0/1724991429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b806b580 0x7fb9b81019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:17.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:17.100+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/1724991429 >> 172.21.15.62:0/1724991429 conn(0x7fb9b8078340 msgr2=0x7fb9b81001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:17.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:17.100+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/1724991429 shutdown_connections 2024-06-10T00:31:17.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:17.100+0000 7fb9bdbc0700 1 -- 172.21.15.62:0/1724991429 wait complete. 2024-06-10T00:31:17.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:16 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1054885681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-10T00:31:17.937 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574941 got 55834574946 for osd.1 2024-06-10T00:31:17.938 DEBUG:teuthology.parallel:result is None 2024-06-10T00:31:18.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:17 smithi105 ceph-mon[36321]: pgmap v386: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:31:18.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:17 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1724991429' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-10T00:31:18.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:17 smithi062 ceph-mon[32390]: pgmap v386: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:31:18.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:17 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1724991429' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-10T00:31:18.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:17 smithi138 ceph-mon[36549]: pgmap v386: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:31:18.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:17 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1724991429' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-10T00:31:19.187 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182483 got 107374182488 for osd.3 2024-06-10T00:31:19.188 DEBUG:teuthology.parallel:result is None 2024-06-10T00:31:19.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:19 smithi062 ceph-mon[32390]: pgmap v387: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:31:19.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:31:19.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:31:19.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:31:19.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:31:19.733 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:31:19.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.742+0000 7feaf1ca7700 1 -- 172.21.15.62:0/455518213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec101390 msgr2=0x7feaec101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:19.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.742+0000 7feaf1ca7700 1 --2- 172.21.15.62:0/455518213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec101390 0x7feaec101810 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7feae0009990 tx=0x7feae0009ca0 comp rx=0 tx=0).stop 2024-06-10T00:31:19.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.742+0000 7feaf1ca7700 1 -- 172.21.15.62:0/455518213 shutdown_connections 2024-06-10T00:31:19.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.742+0000 7feaf1ca7700 1 --2- 172.21.15.62:0/455518213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaec101d50 0x7feaec10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:19.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.742+0000 7feaf1ca7700 1 --2- 172.21.15.62:0/455518213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaec107400 0x7feaec1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:19.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.742+0000 7feaf1ca7700 1 --2- 172.21.15.62:0/455518213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec101390 0x7feaec101810 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:19.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.742+0000 7feaf1ca7700 1 -- 172.21.15.62:0/455518213 >> 172.21.15.62:0/455518213 conn(0x7feaec078340 msgr2=0x7feaec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:19.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.742+0000 7feaf1ca7700 1 -- 172.21.15.62:0/455518213 shutdown_connections 2024-06-10T00:31:19.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.742+0000 7feaf1ca7700 1 -- 172.21.15.62:0/455518213 wait complete. 2024-06-10T00:31:19.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.743+0000 7feaf1ca7700 1 Processor -- start 2024-06-10T00:31:19.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.743+0000 7feaf1ca7700 1 -- start start 2024-06-10T00:31:19.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaf1ca7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec101390 0x7feaec199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:19.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaf1ca7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaec101d50 0x7feaec199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:19.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaf1ca7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaec107400 0x7feaec1a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaf1ca7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feaec111670 con 0x7feaec101390 2024-06-10T00:31:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaf1ca7700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feaec1114f0 con 0x7feaec107400 2024-06-10T00:31:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaf1ca7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feaec1117f0 con 0x7feaec101d50 2024-06-10T00:31:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaeb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec101390 0x7feaec199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaeb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec101390 0x7feaec199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42338/0 (socket says 172.21.15.62:42338) 2024-06-10T00:31:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaeb7fe700 1 -- 172.21.15.62:0/2741695786 learned_addr learned my addr 172.21.15.62:0/2741695786 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaeaffd700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaec101d50 0x7feaec199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaebfff700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaec107400 0x7feaec1a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaeb7fe700 1 -- 172.21.15.62:0/2741695786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaec101d50 msgr2=0x7feaec199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaeb7fe700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaec101d50 0x7feaec199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaeb7fe700 1 -- 172.21.15.62:0/2741695786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaec107400 msgr2=0x7feaec1a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaeb7fe700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaec107400 0x7feaec1a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:19.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaeb7fe700 1 -- 172.21.15.62:0/2741695786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaec1a5030 con 0x7feaec101390 2024-06-10T00:31:19.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaeaffd700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaec101d50 0x7feaec199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:31:19.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.744+0000 7feaeb7fe700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec101390 0x7feaec199a70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feadc00d950 tx=0x7feadc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:19.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.745+0000 7feae8ff9700 1 -- 172.21.15.62:0/2741695786 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc016070 con 0x7feaec101390 2024-06-10T00:31:19.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.745+0000 7feaf1ca7700 1 -- 172.21.15.62:0/2741695786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaec1a5260 con 0x7feaec101390 2024-06-10T00:31:19.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.745+0000 7feae8ff9700 1 -- 172.21.15.62:0/2741695786 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feadc00fd70 con 0x7feaec101390 2024-06-10T00:31:19.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.745+0000 7feae8ff9700 1 -- 172.21.15.62:0/2741695786 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc00e610 con 0x7feaec101390 2024-06-10T00:31:19.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.745+0000 7feaf1ca7700 1 -- 172.21.15.62:0/2741695786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaec1a57c0 con 0x7feaec101390 2024-06-10T00:31:19.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.746+0000 7feaf1ca7700 1 -- 172.21.15.62:0/2741695786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaec069180 con 0x7feaec101390 2024-06-10T00:31:19.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.747+0000 7feae8ff9700 1 -- 172.21.15.62:0/2741695786 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7feadc019070 con 0x7feaec101390 2024-06-10T00:31:19.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.748+0000 7feae8ff9700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fead40a2d30 0x7fead40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:19.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.748+0000 7feaeaffd700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fead40a2d30 0x7fead40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:19.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.748+0000 7feae8ff9700 1 -- 172.21.15.62:0/2741695786 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7feadc0cf310 con 0x7feaec101390 2024-06-10T00:31:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.749+0000 7feaeaffd700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fead40a2d30 0x7fead40a51f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feae00098e0 tx=0x7feae0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.752+0000 7feae8ff9700 1 -- 172.21.15.62:0/2741695786 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feadc09bce0 con 0x7feaec101390 2024-06-10T00:31:19.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.966+0000 7feaf1ca7700 1 -- 172.21.15.62:0/2741695786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7feaec02dca0 con 0x7feaec101390 2024-06-10T00:31:19.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.966+0000 7feae8ff9700 1 -- 172.21.15.62:0/2741695786 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7feadc09b5e0 con 0x7feaec101390 2024-06-10T00:31:19.966 INFO:teuthology.orchestra.run.smithi062.stdout:141733920852 2024-06-10T00:31:19.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.968+0000 7feaf1ca7700 1 -- 172.21.15.62:0/2741695786 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fead40a2d30 msgr2=0x7fead40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:19.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.968+0000 7feaf1ca7700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fead40a2d30 0x7fead40a51f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feae00098e0 tx=0x7feae0005960 comp rx=0 tx=0).stop 2024-06-10T00:31:19.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.968+0000 7feaf1ca7700 1 -- 172.21.15.62:0/2741695786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec101390 msgr2=0x7feaec199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:19.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.968+0000 7feaf1ca7700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec101390 0x7feaec199a70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feadc00d950 tx=0x7feadc00dc60 comp rx=0 tx=0).stop 2024-06-10T00:31:19.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.968+0000 7feaf1ca7700 1 -- 172.21.15.62:0/2741695786 shutdown_connections 2024-06-10T00:31:19.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.968+0000 7feaf1ca7700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec101390 0x7feaec199a70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:19.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.968+0000 7feaf1ca7700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fead40a2d30 0x7fead40a51f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:19.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.968+0000 7feaf1ca7700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaec107400 0x7feaec1a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:19.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.968+0000 7feaf1ca7700 1 --2- 172.21.15.62:0/2741695786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaec101d50 0x7feaec199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:19.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.968+0000 7feaf1ca7700 1 -- 172.21.15.62:0/2741695786 >> 172.21.15.62:0/2741695786 conn(0x7feaec078340 msgr2=0x7feaec0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:19.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.969+0000 7feaf1ca7700 1 -- 172.21.15.62:0/2741695786 shutdown_connections 2024-06-10T00:31:19.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.969+0000 7feaf1ca7700 1 -- 172.21.15.62:0/2741695786 wait complete. 2024-06-10T00:31:20.034 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:19 smithi105 ceph-mon[36321]: pgmap v387: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:31:20.034 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:31:20.034 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:31:20.034 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:31:20.034 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:31:20.034 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:31:20.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:19 smithi138 ceph-mon[36549]: pgmap v387: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:31:20.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:31:20.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:31:20.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:31:20.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:31:20.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:31:20.147 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495683 got 197568495688 for osd.6 2024-06-10T00:31:20.147 DEBUG:teuthology.parallel:result is None 2024-06-10T00:31:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.359+0000 7f770f5f2700 1 -- 172.21.15.62:0/2002494578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7708101d50 msgr2=0x7f770810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.359+0000 7f770f5f2700 1 --2- 172.21.15.62:0/2002494578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7708101d50 0x7f770810e240 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7700009990 tx=0x7f7700009ca0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.359+0000 7f770f5f2700 1 -- 172.21.15.62:0/2002494578 shutdown_connections 2024-06-10T00:31:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.359+0000 7f770f5f2700 1 --2- 172.21.15.62:0/2002494578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7708101d50 0x7f770810e240 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.359+0000 7f770f5f2700 1 --2- 172.21.15.62:0/2002494578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7708107400 0x7f77081077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.359+0000 7f770f5f2700 1 --2- 172.21.15.62:0/2002494578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7708101390 0x7f7708101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.359+0000 7f770f5f2700 1 -- 172.21.15.62:0/2002494578 >> 172.21.15.62:0/2002494578 conn(0x7f7708078340 msgr2=0x7f7708078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.359+0000 7f770f5f2700 1 -- 172.21.15.62:0/2002494578 shutdown_connections 2024-06-10T00:31:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.360+0000 7f770f5f2700 1 -- 172.21.15.62:0/2002494578 wait complete. 2024-06-10T00:31:20.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.360+0000 7f770f5f2700 1 Processor -- start 2024-06-10T00:31:20.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.360+0000 7f770f5f2700 1 -- start start 2024-06-10T00:31:20.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770f5f2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7708101390 0x7f7708199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:20.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770f5f2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7708101d50 0x7f7708199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:20.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770f5f2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7708107400 0x7f77081a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:20.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770f5f2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7708111690 con 0x7f7708107400 2024-06-10T00:31:20.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770f5f2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7708111510 con 0x7f7708101d50 2024-06-10T00:31:20.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770f5f2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7708111810 con 0x7f7708101390 2024-06-10T00:31:20.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770db8f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7708107400 0x7f77081a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:20.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770db8f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7708107400 0x7f77081a4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42352/0 (socket says 172.21.15.62:42352) 2024-06-10T00:31:20.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770db8f700 1 -- 172.21.15.62:0/1989075047 learned_addr learned my addr 172.21.15.62:0/1989075047 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:20.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770d38e700 1 --2- 172.21.15.62:0/1989075047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7708101390 0x7f7708199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:20.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770cb8d700 1 --2- 172.21.15.62:0/1989075047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7708101d50 0x7f7708199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:20.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770db8f700 1 -- 172.21.15.62:0/1989075047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7708101390 msgr2=0x7f7708199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:20.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770db8f700 1 --2- 172.21.15.62:0/1989075047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7708101390 0x7f7708199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770db8f700 1 -- 172.21.15.62:0/1989075047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7708101d50 msgr2=0x7f7708199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:20.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770db8f700 1 --2- 172.21.15.62:0/1989075047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7708101d50 0x7f7708199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770db8f700 1 -- 172.21.15.62:0/1989075047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77081a5030 con 0x7f7708107400 2024-06-10T00:31:20.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.361+0000 7f770db8f700 1 --2- 172.21.15.62:0/1989075047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7708107400 0x7f77081a4af0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f77000047b0 tx=0x7f7700005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:20.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.362+0000 7f76fe7fc700 1 -- 172.21.15.62:0/1989075047 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700026070 con 0x7f7708107400 2024-06-10T00:31:20.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.362+0000 7f76fe7fc700 1 -- 172.21.15.62:0/1989075047 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7700004cc0 con 0x7f7708107400 2024-06-10T00:31:20.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.362+0000 7f770f5f2700 1 -- 172.21.15.62:0/1989075047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77081a5200 con 0x7f7708107400 2024-06-10T00:31:20.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.362+0000 7f76fe7fc700 1 -- 172.21.15.62:0/1989075047 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700003ba0 con 0x7f7708107400 2024-06-10T00:31:20.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.362+0000 7f770f5f2700 1 -- 172.21.15.62:0/1989075047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77081a5650 con 0x7f7708107400 2024-06-10T00:31:20.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.363+0000 7f770f5f2700 1 -- 172.21.15.62:0/1989075047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7708069180 con 0x7f7708107400 2024-06-10T00:31:20.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.364+0000 7f76fe7fc700 1 -- 172.21.15.62:0/1989075047 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7700029070 con 0x7f7708107400 2024-06-10T00:31:20.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.365+0000 7f76fe7fc700 1 --2- 172.21.15.62:0/1989075047 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f76f40a2cb0 0x7f76f40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:20.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.365+0000 7f770d38e700 1 --2- 172.21.15.62:0/1989075047 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f76f40a2cb0 0x7f76f40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:20.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.365+0000 7f76fe7fc700 1 -- 172.21.15.62:0/1989075047 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f77000d0200 con 0x7f7708107400 2024-06-10T00:31:20.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.365+0000 7f770d38e700 1 --2- 172.21.15.62:0/1989075047 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f76f40a2cb0 0x7f76f40a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7704007fd0 tx=0x7f770400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:20.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.369+0000 7f76fe7fc700 1 -- 172.21.15.62:0/1989075047 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7700030020 con 0x7f7708107400 2024-06-10T00:31:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.657+0000 7f2330933700 1 -- 172.21.15.62:0/3709626001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f232c101d50 msgr2=0x7f232c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.657+0000 7f2330933700 1 --2- 172.21.15.62:0/3709626001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f232c101d50 0x7f232c10e240 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f232400b0a0 tx=0x7f232400b3b0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.657+0000 7f2330933700 1 -- 172.21.15.62:0/3709626001 shutdown_connections 2024-06-10T00:31:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.657+0000 7f2330933700 1 --2- 172.21.15.62:0/3709626001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f232c101d50 0x7f232c10e240 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.657+0000 7f2330933700 1 --2- 172.21.15.62:0/3709626001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232c107400 0x7f232c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.657+0000 7f2330933700 1 --2- 172.21.15.62:0/3709626001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c101390 0x7f232c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.657+0000 7f2330933700 1 -- 172.21.15.62:0/3709626001 >> 172.21.15.62:0/3709626001 conn(0x7f232c078340 msgr2=0x7f232c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.657+0000 7f2330933700 1 -- 172.21.15.62:0/3709626001 shutdown_connections 2024-06-10T00:31:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.657+0000 7f2330933700 1 -- 172.21.15.62:0/3709626001 wait complete. 2024-06-10T00:31:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.658+0000 7f2330933700 1 Processor -- start 2024-06-10T00:31:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.658+0000 7f2330933700 1 -- start start 2024-06-10T00:31:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.658+0000 7f2330933700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232c101390 0x7f232c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.658+0000 7f2330933700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f232c101d50 0x7f232c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.658+0000 7f2330933700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c107400 0x7f232c1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.658+0000 7f2330933700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1116f0 con 0x7f232c101d50 2024-06-10T00:31:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.658+0000 7f2330933700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f232c111570 con 0x7f232c107400 2024-06-10T00:31:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.658+0000 7f2330933700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f232c111870 con 0x7f232c101390 2024-06-10T00:31:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.658+0000 7f2329d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f232c101d50 0x7f232c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f2329d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f232c101d50 0x7f232c19a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42324/0 (socket says 172.21.15.62:42324) 2024-06-10T00:31:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f2329d9b700 1 -- 172.21.15.62:0/2043950315 learned_addr learned my addr 172.21.15.62:0/2043950315 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f232a59c700 1 --2- 172.21.15.62:0/2043950315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232c101390 0x7f232c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f232ad9d700 1 --2- 172.21.15.62:0/2043950315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c107400 0x7f232c1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f2329d9b700 1 -- 172.21.15.62:0/2043950315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232c101390 msgr2=0x7f232c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f2329d9b700 1 --2- 172.21.15.62:0/2043950315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232c101390 0x7f232c199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f2329d9b700 1 -- 172.21.15.62:0/2043950315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c107400 msgr2=0x7f232c1a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f2329d9b700 1 --2- 172.21.15.62:0/2043950315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c107400 0x7f232c1a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f2329d9b700 1 -- 172.21.15.62:0/2043950315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c1a50b0 con 0x7f232c101d50 2024-06-10T00:31:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f2329d9b700 1 --2- 172.21.15.62:0/2043950315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f232c101d50 0x7f232c19a030 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f232000d980 tx=0x7f232000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f231b7fe700 1 -- 172.21.15.62:0/2043950315 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320016070 con 0x7f232c101d50 2024-06-10T00:31:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f2330933700 1 -- 172.21.15.62:0/2043950315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c1a5280 con 0x7f232c101d50 2024-06-10T00:31:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f231b7fe700 1 -- 172.21.15.62:0/2043950315 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2320004d10 con 0x7f232c101d50 2024-06-10T00:31:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f231b7fe700 1 -- 172.21.15.62:0/2043950315 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23200055f0 con 0x7f232c101d50 2024-06-10T00:31:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.659+0000 7f2330933700 1 -- 172.21.15.62:0/2043950315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c1a5790 con 0x7f232c101d50 2024-06-10T00:31:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.660+0000 7f2330933700 1 -- 172.21.15.62:0/2043950315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c109970 con 0x7f232c101d50 2024-06-10T00:31:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.661+0000 7f231b7fe700 1 -- 172.21.15.62:0/2043950315 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2320004750 con 0x7f232c101d50 2024-06-10T00:31:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.662+0000 7f231b7fe700 1 --2- 172.21.15.62:0/2043950315 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f23140a2cb0 0x7f23140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.662+0000 7f232a59c700 1 --2- 172.21.15.62:0/2043950315 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f23140a2cb0 0x7f23140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.662+0000 7f231b7fe700 1 -- 172.21.15.62:0/2043950315 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f23200cf3d0 con 0x7f232c101d50 2024-06-10T00:31:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.663+0000 7f232a59c700 1 --2- 172.21.15.62:0/2043950315 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f23140a2cb0 0x7f23140a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f231c009a10 tx=0x7f231c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.667+0000 7f231b7fe700 1 -- 172.21.15.62:0/2043950315 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f232001f020 con 0x7f232c101d50 2024-06-10T00:31:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.882+0000 7f2330933700 1 -- 172.21.15.62:0/2043950315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f232c109350 con 0x7f232c101d50 2024-06-10T00:31:20.484 INFO:teuthology.orchestra.run.smithi062.stdout:343597383728 2024-06-10T00:31:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.882+0000 7f231b7fe700 1 -- 172.21.15.62:0/2043950315 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f232009be00 con 0x7f232c101d50 2024-06-10T00:31:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.884+0000 7f2330933700 1 -- 172.21.15.62:0/2043950315 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f23140a2cb0 msgr2=0x7f23140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.884+0000 7f2330933700 1 --2- 172.21.15.62:0/2043950315 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f23140a2cb0 0x7f23140a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f231c009a10 tx=0x7f231c009190 comp rx=0 tx=0).stop 2024-06-10T00:31:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.884+0000 7f2330933700 1 -- 172.21.15.62:0/2043950315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f232c101d50 msgr2=0x7f232c19a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.884+0000 7f2330933700 1 --2- 172.21.15.62:0/2043950315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f232c101d50 0x7f232c19a030 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f232000d980 tx=0x7f232000dc90 comp rx=0 tx=0).stop 2024-06-10T00:31:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.884+0000 7f2330933700 1 -- 172.21.15.62:0/2043950315 shutdown_connections 2024-06-10T00:31:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.884+0000 7f2330933700 1 --2- 172.21.15.62:0/2043950315 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f23140a2cb0 0x7f23140a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.884+0000 7f2330933700 1 --2- 172.21.15.62:0/2043950315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c107400 0x7f232c1a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.884+0000 7f2330933700 1 --2- 172.21.15.62:0/2043950315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232c101390 0x7f232c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.884+0000 7f2330933700 1 --2- 172.21.15.62:0/2043950315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f232c101d50 0x7f232c19a030 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.884+0000 7f2330933700 1 -- 172.21.15.62:0/2043950315 >> 172.21.15.62:0/2043950315 conn(0x7f232c078340 msgr2=0x7f232c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:20.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.885+0000 7f2330933700 1 -- 172.21.15.62:0/2043950315 shutdown_connections 2024-06-10T00:31:20.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:19.885+0000 7f2330933700 1 -- 172.21.15.62:0/2043950315 wait complete. 2024-06-10T00:31:20.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.578+0000 7f770f5f2700 1 -- 172.21.15.62:0/1989075047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f770802de60 con 0x7f7708107400 2024-06-10T00:31:20.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.578+0000 7f76fe7fc700 1 -- 172.21.15.62:0/1989075047 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7700013070 con 0x7f7708107400 2024-06-10T00:31:20.579 INFO:teuthology.orchestra.run.smithi062.stdout:253403070528 2024-06-10T00:31:20.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.581+0000 7f770f5f2700 1 -- 172.21.15.62:0/1989075047 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f76f40a2cb0 msgr2=0x7f76f40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:20.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.581+0000 7f770f5f2700 1 --2- 172.21.15.62:0/1989075047 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f76f40a2cb0 0x7f76f40a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7704007fd0 tx=0x7f770400c040 comp rx=0 tx=0).stop 2024-06-10T00:31:20.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.581+0000 7f770f5f2700 1 -- 172.21.15.62:0/1989075047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7708107400 msgr2=0x7f77081a4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:20.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.581+0000 7f770f5f2700 1 --2- 172.21.15.62:0/1989075047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7708107400 0x7f77081a4af0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f77000047b0 tx=0x7f7700005e80 comp rx=0 tx=0).stop 2024-06-10T00:31:20.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.581+0000 7f770f5f2700 1 -- 172.21.15.62:0/1989075047 shutdown_connections 2024-06-10T00:31:20.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.581+0000 7f770f5f2700 1 --2- 172.21.15.62:0/1989075047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7708107400 0x7f77081a4af0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.581+0000 7f770f5f2700 1 --2- 172.21.15.62:0/1989075047 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f76f40a2cb0 0x7f76f40a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.581+0000 7f770f5f2700 1 --2- 172.21.15.62:0/1989075047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7708101390 0x7f7708199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.581+0000 7f770f5f2700 1 --2- 172.21.15.62:0/1989075047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7708101d50 0x7f7708199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:20.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.581+0000 7f770f5f2700 1 -- 172.21.15.62:0/1989075047 >> 172.21.15.62:0/1989075047 conn(0x7f7708078340 msgr2=0x7f77080ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:20.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.581+0000 7f770f5f2700 1 -- 172.21.15.62:0/1989075047 shutdown_connections 2024-06-10T00:31:20.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:20.581+0000 7f770f5f2700 1 -- 172.21.15.62:0/1989075047 wait complete. 2024-06-10T00:31:20.801 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:20 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2043950315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-10T00:31:20.802 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:20 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2741695786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-10T00:31:20.802 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:20 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1989075047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-10T00:31:21.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:20 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2043950315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-10T00:31:21.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:20 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2741695786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-10T00:31:21.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:20 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1989075047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-10T00:31:21.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:20 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2043950315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-10T00:31:21.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:20 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2741695786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-10T00:31:21.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:20 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1989075047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-10T00:31:21.230 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383723 got 343597383728 for osd.11 2024-06-10T00:31:21.231 DEBUG:teuthology.parallel:result is None 2024-06-10T00:31:21.808 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:21 smithi062 ceph-mon[32390]: pgmap v388: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:31:21.841 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920844 got 141733920852 for osd.4 2024-06-10T00:31:21.841 DEBUG:teuthology.parallel:result is None 2024-06-10T00:31:22.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:21 smithi105 ceph-mon[36321]: pgmap v388: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:31:22.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:21 smithi138 ceph-mon[36549]: pgmap v388: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:31:22.482 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070521 got 253403070528 for osd.8 2024-06-10T00:31:22.482 DEBUG:teuthology.parallel:result is None 2024-06-10T00:31:22.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.916+0000 7f9483f3e700 1 -- 172.21.15.62:0/1645957348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947c107400 msgr2=0x7f947c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:22.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.916+0000 7f9483f3e700 1 --2- 172.21.15.62:0/1645957348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947c107400 0x7f947c1077e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9478009990 tx=0x7f9478009ca0 comp rx=0 tx=0).stop 2024-06-10T00:31:22.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.916+0000 7f9483f3e700 1 -- 172.21.15.62:0/1645957348 shutdown_connections 2024-06-10T00:31:22.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.916+0000 7f9483f3e700 1 --2- 172.21.15.62:0/1645957348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947c107400 0x7f947c1077e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:22.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.916+0000 7f9483f3e700 1 --2- 172.21.15.62:0/1645957348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947c101d50 0x7f947c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:22.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.916+0000 7f9483f3e700 1 --2- 172.21.15.62:0/1645957348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f947c101390 0x7f947c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:22.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.916+0000 7f9483f3e700 1 -- 172.21.15.62:0/1645957348 >> 172.21.15.62:0/1645957348 conn(0x7f947c078340 msgr2=0x7f947c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:22.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.916+0000 7f9483f3e700 1 -- 172.21.15.62:0/1645957348 shutdown_connections 2024-06-10T00:31:22.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.916+0000 7f9483f3e700 1 -- 172.21.15.62:0/1645957348 wait complete. 2024-06-10T00:31:22.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.917+0000 7f9483f3e700 1 Processor -- start 2024-06-10T00:31:22.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.917+0000 7f9483f3e700 1 -- start start 2024-06-10T00:31:22.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.917+0000 7f9483f3e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f947c101390 0x7f947c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:22.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.917+0000 7f9483f3e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947c101d50 0x7f947c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:22.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.917+0000 7f9483f3e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947c107400 0x7f947c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:22.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.917+0000 7f9483f3e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1116f0 con 0x7f947c101d50 2024-06-10T00:31:22.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.917+0000 7f9483f3e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f947c111570 con 0x7f947c107400 2024-06-10T00:31:22.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.917+0000 7f9483f3e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f947c111870 con 0x7f947c101390 2024-06-10T00:31:22.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.917+0000 7f94814d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947c101d50 0x7f947c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:22.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f94814d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947c101d50 0x7f947c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42376/0 (socket says 172.21.15.62:42376) 2024-06-10T00:31:22.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f94814d9700 1 -- 172.21.15.62:0/712493902 learned_addr learned my addr 172.21.15.62:0/712493902 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:22.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f9481cda700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f947c101390 0x7f947c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:22.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f94824db700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947c107400 0x7f947c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:22.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f94814d9700 1 -- 172.21.15.62:0/712493902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f947c101390 msgr2=0x7f947c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:22.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f94814d9700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f947c101390 0x7f947c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:22.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f94814d9700 1 -- 172.21.15.62:0/712493902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947c107400 msgr2=0x7f947c198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:22.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f94814d9700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947c107400 0x7f947c198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:22.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f94814d9700 1 -- 172.21.15.62:0/712493902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947c199400 con 0x7f947c101d50 2024-06-10T00:31:22.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f94824db700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947c107400 0x7f947c198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:31:22.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f94814d9700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947c101d50 0x7f947c19ef90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f946c00c970 tx=0x7f946c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:22.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f9472ffd700 1 -- 172.21.15.62:0/712493902 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c007aa0 con 0x7f947c101d50 2024-06-10T00:31:22.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f9472ffd700 1 -- 172.21.15.62:0/712493902 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f946c004d10 con 0x7f947c101d50 2024-06-10T00:31:22.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f9472ffd700 1 -- 172.21.15.62:0/712493902 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c00f450 con 0x7f947c101d50 2024-06-10T00:31:22.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f9483f3e700 1 -- 172.21.15.62:0/712493902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947c1996f0 con 0x7f947c101d50 2024-06-10T00:31:22.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.918+0000 7f9483f3e700 1 -- 172.21.15.62:0/712493902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947c1a59d0 con 0x7f947c101d50 2024-06-10T00:31:22.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.919+0000 7f9483f3e700 1 -- 172.21.15.62:0/712493902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f947c102ab0 con 0x7f947c101d50 2024-06-10T00:31:22.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.921+0000 7f9472ffd700 1 -- 172.21.15.62:0/712493902 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f946c004750 con 0x7f947c101d50 2024-06-10T00:31:22.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.921+0000 7f9472ffd700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f94680a2c20 0x7f94680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:22.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.921+0000 7f9481cda700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f94680a2c20 0x7f94680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:22.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.921+0000 7f9472ffd700 1 -- 172.21.15.62:0/712493902 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f946c0cf4b0 con 0x7f947c101d50 2024-06-10T00:31:22.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.922+0000 7f9481cda700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f94680a2c20 0x7f94680a50e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f947800b5c0 tx=0x7f9478005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:22.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:22.925+0000 7f9472ffd700 1 -- 172.21.15.62:0/712493902 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f946c09c160 con 0x7f947c101d50 2024-06-10T00:31:23.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:23.134+0000 7f9483f3e700 1 -- 172.21.15.62:0/712493902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f947c199fe0 con 0x7f947c101d50 2024-06-10T00:31:23.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:23.134+0000 7f9472ffd700 1 -- 172.21.15.62:0/712493902 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f946c005700 con 0x7f947c101d50 2024-06-10T00:31:23.135 INFO:teuthology.orchestra.run.smithi062.stdout:287762808890 2024-06-10T00:31:23.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:23.137+0000 7f9483f3e700 1 -- 172.21.15.62:0/712493902 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f94680a2c20 msgr2=0x7f94680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:23.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:23.137+0000 7f9483f3e700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f94680a2c20 0x7f94680a50e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f947800b5c0 tx=0x7f9478005fd0 comp rx=0 tx=0).stop 2024-06-10T00:31:23.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:23.137+0000 7f9483f3e700 1 -- 172.21.15.62:0/712493902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947c101d50 msgr2=0x7f947c19ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:23.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:23.137+0000 7f9483f3e700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947c101d50 0x7f947c19ef90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f946c00c970 tx=0x7f946c00cc80 comp rx=0 tx=0).stop 2024-06-10T00:31:23.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:23.137+0000 7f9483f3e700 1 -- 172.21.15.62:0/712493902 shutdown_connections 2024-06-10T00:31:23.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:23.137+0000 7f9483f3e700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f94680a2c20 0x7f94680a50e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:23.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:23.137+0000 7f9483f3e700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f947c107400 0x7f947c198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:23.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:23.137+0000 7f9483f3e700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f947c101390 0x7f947c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:23.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:23.137+0000 7f9483f3e700 1 --2- 172.21.15.62:0/712493902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f947c101d50 0x7f947c19ef90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:23.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:23.137+0000 7f9483f3e700 1 -- 172.21.15.62:0/712493902 >> 172.21.15.62:0/712493902 conn(0x7f947c078340 msgr2=0x7f947c100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:23.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:23.137+0000 7f9483f3e700 1 -- 172.21.15.62:0/712493902 shutdown_connections 2024-06-10T00:31:23.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:23.137+0000 7f9483f3e700 1 -- 172.21.15.62:0/712493902 wait complete. 2024-06-10T00:31:23.849 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:23 smithi062 ceph-mon[32390]: pgmap v389: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:31:23.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:23 smithi062 ceph-mon[32390]: mgrmap e18: x(active, since 10m), standbys: y, z 2024-06-10T00:31:23.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:23 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/712493902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-10T00:31:23.851 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808884 got 287762808890 for osd.9 2024-06-10T00:31:23.852 DEBUG:teuthology.parallel:result is None 2024-06-10T00:31:23.852 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-10T00:31:23.852 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T00:31:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:23 smithi105 ceph-mon[36321]: pgmap v389: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:31:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:23 smithi105 ceph-mon[36321]: mgrmap e18: x(active, since 10m), standbys: y, z 2024-06-10T00:31:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:23 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/712493902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-10T00:31:24.105 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:24.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:23 smithi138 ceph-mon[36549]: pgmap v389: 129 pgs: 129 active+clean; 645 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:31:24.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:23 smithi138 ceph-mon[36549]: mgrmap e18: x(active, since 10m), standbys: y, z 2024-06-10T00:31:24.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:23 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/712493902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-10T00:31:25.799 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:25 smithi062 ceph-mon[32390]: pgmap v390: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-10T00:31:25.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.901+0000 7fe8db023700 1 -- 172.21.15.62:0/1244243812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8d4101720 msgr2=0x7fe8d4101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:25.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.901+0000 7fe8db023700 1 --2- 172.21.15.62:0/1244243812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8d4101720 0x7fe8d4101b00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80099c0 tx=0x7fe8c8009cd0 comp rx=0 tx=0).stop 2024-06-10T00:31:25.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 -- 172.21.15.62:0/1244243812 shutdown_connections 2024-06-10T00:31:25.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 --2- 172.21.15.62:0/1244243812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8d4101720 0x7fe8d4101b00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:25.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 --2- 172.21.15.62:0/1244243812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8d41066f0 0x7fe8d410e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:25.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 --2- 172.21.15.62:0/1244243812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d41020d0 0x7fe8d4106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:25.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 -- 172.21.15.62:0/1244243812 >> 172.21.15.62:0/1244243812 conn(0x7fe8d4077270 msgr2=0x7fe8d4077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:25.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 -- 172.21.15.62:0/1244243812 shutdown_connections 2024-06-10T00:31:25.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 -- 172.21.15.62:0/1244243812 wait complete. 2024-06-10T00:31:25.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 Processor -- start 2024-06-10T00:31:25.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 -- start start 2024-06-10T00:31:25.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8d4101720 0x7fe8d419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:25.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8d41020d0 0x7fe8d419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:25.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d41066f0 0x7fe8d4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:25.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41116d0 con 0x7fe8d41020d0 2024-06-10T00:31:25.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4111550 con 0x7fe8d4101720 2024-06-10T00:31:25.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.902+0000 7fe8db023700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4111850 con 0x7fe8d41066f0 2024-06-10T00:31:25.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8d41020d0 0x7fe8d419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:25.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8d41020d0 0x7fe8d419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42394/0 (socket says 172.21.15.62:42394) 2024-06-10T00:31:25.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d3fff700 1 -- 172.21.15.62:0/2408094795 learned_addr learned my addr 172.21.15.62:0/2408094795 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:25.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d8dbf700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8d4101720 0x7fe8d419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:25.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d95c0700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d41066f0 0x7fe8d4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:25.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d3fff700 1 -- 172.21.15.62:0/2408094795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d41066f0 msgr2=0x7fe8d4198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:25.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d3fff700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d41066f0 0x7fe8d4198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:25.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d3fff700 1 -- 172.21.15.62:0/2408094795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8d4101720 msgr2=0x7fe8d419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:25.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d3fff700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8d4101720 0x7fe8d419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:25.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d3fff700 1 -- 172.21.15.62:0/2408094795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d4199180 con 0x7fe8d41020d0 2024-06-10T00:31:25.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d95c0700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d41066f0 0x7fe8d4198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:31:25.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d3fff700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8d41020d0 0x7fe8d419eda0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000c9e0 tx=0x7fe8c000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:25.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d1ffb700 1 -- 172.21.15.62:0/2408094795 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0007aa0 con 0x7fe8d41020d0 2024-06-10T00:31:25.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d1ffb700 1 -- 172.21.15.62:0/2408094795 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8c0007c40 con 0x7fe8d41020d0 2024-06-10T00:31:25.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8db023700 1 -- 172.21.15.62:0/2408094795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d4199470 con 0x7fe8d41020d0 2024-06-10T00:31:25.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8d1ffb700 1 -- 172.21.15.62:0/2408094795 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0018780 con 0x7fe8d41020d0 2024-06-10T00:31:25.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.903+0000 7fe8db023700 1 -- 172.21.15.62:0/2408094795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d41a5730 con 0x7fe8d41020d0 2024-06-10T00:31:25.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.904+0000 7fe8db023700 1 -- 172.21.15.62:0/2408094795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d4069180 con 0x7fe8d41020d0 2024-06-10T00:31:25.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.905+0000 7fe8d1ffb700 1 -- 172.21.15.62:0/2408094795 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe8c000f450 con 0x7fe8d41020d0 2024-06-10T00:31:25.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.906+0000 7fe8d1ffb700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe8c40a2ca0 0x7fe8c40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:25.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.907+0000 7fe8d8dbf700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe8c40a2ca0 0x7fe8c40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:25.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.907+0000 7fe8d1ffb700 1 -- 172.21.15.62:0/2408094795 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fe8c00cf350 con 0x7fe8d41020d0 2024-06-10T00:31:25.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.907+0000 7fe8d8dbf700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe8c40a2ca0 0x7fe8c40a5160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8000c00 tx=0x7fe8c8005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:25.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:25.911+0000 7fe8d1ffb700 1 -- 172.21.15.62:0/2408094795 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe8c0014030 con 0x7fe8d41020d0 2024-06-10T00:31:26.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:25 smithi105 ceph-mon[36321]: pgmap v390: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-10T00:31:26.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:26.113+0000 7fe8db023700 1 -- 172.21.15.62:0/2408094795 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8d40628d0 con 0x7fe8c40a2ca0 2024-06-10T00:31:26.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:26.123+0000 7fe8d1ffb700 1 -- 172.21.15.62:0/2408094795 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340629 (secure 0 0 0) 0x7fe8d40628d0 con 0x7fe8c40a2ca0 2024-06-10T00:31:26.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:25 smithi138 ceph-mon[36549]: pgmap v390: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-10T00:31:26.129 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:31:26.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:26.126+0000 7fe8db023700 1 -- 172.21.15.62:0/2408094795 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe8c40a2ca0 msgr2=0x7fe8c40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:26.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:26.126+0000 7fe8db023700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe8c40a2ca0 0x7fe8c40a5160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8000c00 tx=0x7fe8c8005ab0 comp rx=0 tx=0).stop 2024-06-10T00:31:26.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:26.126+0000 7fe8db023700 1 -- 172.21.15.62:0/2408094795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8d41020d0 msgr2=0x7fe8d419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:26.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:26.126+0000 7fe8db023700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8d41020d0 0x7fe8d419eda0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000c9e0 tx=0x7fe8c000cda0 comp rx=0 tx=0).stop 2024-06-10T00:31:26.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:26.126+0000 7fe8db023700 1 -- 172.21.15.62:0/2408094795 shutdown_connections 2024-06-10T00:31:26.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:26.126+0000 7fe8db023700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe8c40a2ca0 0x7fe8c40a5160 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:26.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:26.126+0000 7fe8db023700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d41066f0 0x7fe8d4198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:26.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:26.126+0000 7fe8db023700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8d4101720 0x7fe8d419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:26.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:26.126+0000 7fe8db023700 1 --2- 172.21.15.62:0/2408094795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8d41020d0 0x7fe8d419eda0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:26.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:26.126+0000 7fe8db023700 1 -- 172.21.15.62:0/2408094795 >> 172.21.15.62:0/2408094795 conn(0x7fe8d4077270 msgr2=0x7fe8d406af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:26.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:26.126+0000 7fe8db023700 1 -- 172.21.15.62:0/2408094795 shutdown_connections 2024-06-10T00:31:26.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:26.126+0000 7fe8db023700 1 -- 172.21.15.62:0/2408094795 wait complete. 2024-06-10T00:31:26.133 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-06-10T00:31:26.834 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-06-10T00:31:25.982753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660849,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1153,"num_read_kb":706,"num_write":438,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":716,"ondisk_log_size":716,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381400,"kb_used_data":15584,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560424,"statfs":{"total":1151940427776,"available":1151549874176,"internally_reserved":0,"allocated":15958016,"data_stored":8797841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":52,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003724"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.978168+0000","last_change":"2024-06-10T00:28:37.657234+0000","last_active":"2024-06-10T00:31:21.978168+0000","last_peered":"2024-06-10T00:31:21.978168+0000","last_clean":"2024-06-10T00:31:21.978168+0000","last_became_active":"2024-06-10T00:28:37.656879+0000","last_became_peered":"2024-06-10T00:28:37.656879+0000","last_unstale":"2024-06-10T00:31:21.978168+0000","last_undegraded":"2024-06-10T00:31:21.978168+0000","last_fullsized":"2024-06-10T00:31:21.978168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:01:02.907597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.973472+0000","last_change":"2024-06-10T00:28:37.656440+0000","last_active":"2024-06-10T00:31:21.973472+0000","last_peered":"2024-06-10T00:31:21.973472+0000","last_clean":"2024-06-10T00:31:21.973472+0000","last_became_active":"2024-06-10T00:28:37.656113+0000","last_became_peered":"2024-06-10T00:28:37.656113+0000","last_unstale":"2024-06-10T00:31:21.973472+0000","last_undegraded":"2024-06-10T00:31:21.973472+0000","last_fullsized":"2024-06-10T00:31:21.973472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:49:31.447951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.977252+0000","last_change":"2024-06-10T00:28:37.659114+0000","last_active":"2024-06-10T00:31:21.977252+0000","last_peered":"2024-06-10T00:31:21.977252+0000","last_clean":"2024-06-10T00:31:21.977252+0000","last_became_active":"2024-06-10T00:28:37.658763+0000","last_became_peered":"2024-06-10T00:28:37.658763+0000","last_unstale":"2024-06-10T00:31:21.977252+0000","last_undegraded":"2024-06-10T00:31:21.977252+0000","last_fullsized":"2024-06-10T00:31:21.977252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:39:54.963590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.961574+0000","last_change":"2024-06-10T00:28:37.656786+0000","last_active":"2024-06-10T00:31:21.961574+0000","last_peered":"2024-06-10T00:31:21.961574+0000","last_clean":"2024-06-10T00:31:21.961574+0000","last_became_active":"2024-06-10T00:28:37.656073+0000","last_became_peered":"2024-06-10T00:28:37.656073+0000","last_unstale":"2024-06-10T00:31:21.961574+0000","last_undegraded":"2024-06-10T00:31:21.961574+0000","last_fullsized":"2024-06-10T00:31:21.961574+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:19.176608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.960169+0000","last_change":"2024-06-10T00:28:37.650250+0000","last_active":"2024-06-10T00:31:21.960169+0000","last_peered":"2024-06-10T00:31:21.960169+0000","last_clean":"2024-06-10T00:31:21.960169+0000","last_became_active":"2024-06-10T00:28:37.649852+0000","last_became_peered":"2024-06-10T00:28:37.649852+0000","last_unstale":"2024-06-10T00:31:21.960169+0000","last_undegraded":"2024-06-10T00:31:21.960169+0000","last_fullsized":"2024-06-10T00:31:21.960169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:39.492884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.971523+0000","last_change":"2024-06-10T00:28:37.646042+0000","last_active":"2024-06-10T00:31:21.971523+0000","last_peered":"2024-06-10T00:31:21.971523+0000","last_clean":"2024-06-10T00:31:21.971523+0000","last_became_active":"2024-06-10T00:28:37.644591+0000","last_became_peered":"2024-06-10T00:28:37.644591+0000","last_unstale":"2024-06-10T00:31:21.971523+0000","last_undegraded":"2024-06-10T00:31:21.971523+0000","last_fullsized":"2024-06-10T00:31:21.971523+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:29.932303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111103+0000","last_change":"2024-06-10T00:28:37.664076+0000","last_active":"2024-06-10T00:30:24.111103+0000","last_peered":"2024-06-10T00:30:24.111103+0000","last_clean":"2024-06-10T00:30:24.111103+0000","last_became_active":"2024-06-10T00:28:37.663741+0000","last_became_peered":"2024-06-10T00:28:37.663741+0000","last_unstale":"2024-06-10T00:30:24.111103+0000","last_undegraded":"2024-06-10T00:30:24.111103+0000","last_fullsized":"2024-06-10T00:30:24.111103+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:14.581409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.969207+0000","last_change":"2024-06-10T00:28:37.648758+0000","last_active":"2024-06-10T00:31:21.969207+0000","last_peered":"2024-06-10T00:31:21.969207+0000","last_clean":"2024-06-10T00:31:21.969207+0000","last_became_active":"2024-06-10T00:28:37.648387+0000","last_became_peered":"2024-06-10T00:28:37.648387+0000","last_unstale":"2024-06-10T00:31:21.969207+0000","last_undegraded":"2024-06-10T00:31:21.969207+0000","last_fullsized":"2024-06-10T00:31:21.969207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:17.187265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.973868+0000","last_change":"2024-06-10T00:28:37.664650+0000","last_active":"2024-06-10T00:31:21.973868+0000","last_peered":"2024-06-10T00:31:21.973868+0000","last_clean":"2024-06-10T00:31:21.973868+0000","last_became_active":"2024-06-10T00:28:37.663178+0000","last_became_peered":"2024-06-10T00:28:37.663178+0000","last_unstale":"2024-06-10T00:31:21.973868+0000","last_undegraded":"2024-06-10T00:31:21.973868+0000","last_fullsized":"2024-06-10T00:31:21.973868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:46.032078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.970373+0000","last_change":"2024-06-10T00:28:37.658353+0000","last_active":"2024-06-10T00:31:21.970373+0000","last_peered":"2024-06-10T00:31:21.970373+0000","last_clean":"2024-06-10T00:31:21.970373+0000","last_became_active":"2024-06-10T00:28:37.657961+0000","last_became_peered":"2024-06-10T00:28:37.657961+0000","last_unstale":"2024-06-10T00:31:21.970373+0000","last_undegraded":"2024-06-10T00:31:21.970373+0000","last_fullsized":"2024-06-10T00:31:21.970373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:22:27.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111643+0000","last_change":"2024-06-10T00:28:37.663823+0000","last_active":"2024-06-10T00:30:24.111643+0000","last_peered":"2024-06-10T00:30:24.111643+0000","last_clean":"2024-06-10T00:30:24.111643+0000","last_became_active":"2024-06-10T00:28:37.663499+0000","last_became_peered":"2024-06-10T00:28:37.663499+0000","last_unstale":"2024-06-10T00:30:24.111643+0000","last_undegraded":"2024-06-10T00:30:24.111643+0000","last_fullsized":"2024-06-10T00:30:24.111643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.100923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":18,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.968137+0000","last_change":"2024-06-10T00:30:24.112722+0000","last_active":"2024-06-10T00:31:21.968137+0000","last_peered":"2024-06-10T00:31:21.968137+0000","last_clean":"2024-06-10T00:31:21.968137+0000","last_became_active":"2024-06-10T00:30:24.112325+0000","last_became_peered":"2024-06-10T00:30:24.112325+0000","last_unstale":"2024-06-10T00:31:21.968137+0000","last_undegraded":"2024-06-10T00:31:21.968137+0000","last_fullsized":"2024-06-10T00:31:21.968137+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:26.671583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228490+0000","last_change":"2024-06-10T00:28:37.649997+0000","last_active":"2024-06-10T00:30:24.228490+0000","last_peered":"2024-06-10T00:30:24.228490+0000","last_clean":"2024-06-10T00:30:24.228490+0000","last_became_active":"2024-06-10T00:28:37.649494+0000","last_became_peered":"2024-06-10T00:28:37.649494+0000","last_unstale":"2024-06-10T00:30:24.228490+0000","last_undegraded":"2024-06-10T00:30:24.228490+0000","last_fullsized":"2024-06-10T00:30:24.228490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:44:07.467206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.976587+0000","last_change":"2024-06-10T00:28:37.653064+0000","last_active":"2024-06-10T00:31:21.976587+0000","last_peered":"2024-06-10T00:31:21.976587+0000","last_clean":"2024-06-10T00:31:21.976587+0000","last_became_active":"2024-06-10T00:28:37.652684+0000","last_became_peered":"2024-06-10T00:28:37.652684+0000","last_unstale":"2024-06-10T00:31:21.976587+0000","last_undegraded":"2024-06-10T00:31:21.976587+0000","last_fullsized":"2024-06-10T00:31:21.976587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:30.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.977758+0000","last_change":"2024-06-10T00:28:37.655570+0000","last_active":"2024-06-10T00:31:21.977758+0000","last_peered":"2024-06-10T00:31:21.977758+0000","last_clean":"2024-06-10T00:31:21.977758+0000","last_became_active":"2024-06-10T00:28:37.655241+0000","last_became_peered":"2024-06-10T00:28:37.655241+0000","last_unstale":"2024-06-10T00:31:21.977758+0000","last_undegraded":"2024-06-10T00:31:21.977758+0000","last_fullsized":"2024-06-10T00:31:21.977758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:07:03.543064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.977899+0000","last_change":"2024-06-10T00:28:37.665607+0000","last_active":"2024-06-10T00:31:21.977899+0000","last_peered":"2024-06-10T00:31:21.977899+0000","last_clean":"2024-06-10T00:31:21.977899+0000","last_became_active":"2024-06-10T00:28:37.664943+0000","last_became_peered":"2024-06-10T00:28:37.664943+0000","last_unstale":"2024-06-10T00:31:21.977899+0000","last_undegraded":"2024-06-10T00:31:21.977899+0000","last_fullsized":"2024-06-10T00:31:21.977899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:43:02.940298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.978293+0000","last_change":"2024-06-10T00:28:37.658214+0000","last_active":"2024-06-10T00:31:21.978293+0000","last_peered":"2024-06-10T00:31:21.978293+0000","last_clean":"2024-06-10T00:31:21.978293+0000","last_became_active":"2024-06-10T00:28:37.657295+0000","last_became_peered":"2024-06-10T00:28:37.657295+0000","last_unstale":"2024-06-10T00:31:21.978293+0000","last_undegraded":"2024-06-10T00:31:21.978293+0000","last_fullsized":"2024-06-10T00:31:21.978293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:20.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.979031+0000","last_change":"2024-06-10T00:28:37.656645+0000","last_active":"2024-06-10T00:31:21.979031+0000","last_peered":"2024-06-10T00:31:21.979031+0000","last_clean":"2024-06-10T00:31:21.979031+0000","last_became_active":"2024-06-10T00:28:37.656157+0000","last_became_peered":"2024-06-10T00:28:37.656157+0000","last_unstale":"2024-06-10T00:31:21.979031+0000","last_undegraded":"2024-06-10T00:31:21.979031+0000","last_fullsized":"2024-06-10T00:31:21.979031+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:00.953804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.968382+0000","last_change":"2024-06-10T00:28:37.664336+0000","last_active":"2024-06-10T00:31:21.968382+0000","last_peered":"2024-06-10T00:31:21.968382+0000","last_clean":"2024-06-10T00:31:21.968382+0000","last_became_active":"2024-06-10T00:28:37.663960+0000","last_became_peered":"2024-06-10T00:28:37.663960+0000","last_unstale":"2024-06-10T00:31:21.968382+0000","last_undegraded":"2024-06-10T00:31:21.968382+0000","last_fullsized":"2024-06-10T00:31:21.968382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:59:32.774442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650441+0000","last_change":"2024-06-10T00:28:48.705930+0000","last_active":"2024-06-10T00:30:24.650441+0000","last_peered":"2024-06-10T00:30:24.650441+0000","last_clean":"2024-06-10T00:30:24.650441+0000","last_became_active":"2024-06-10T00:28:48.705558+0000","last_became_peered":"2024-06-10T00:28:48.705558+0000","last_unstale":"2024-06-10T00:30:24.650441+0000","last_undegraded":"2024-06-10T00:30:24.650441+0000","last_fullsized":"2024-06-10T00:30:24.650441+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:41:44.914378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'13","reported_seq":168,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.019649+0000","last_change":"2024-06-10T00:29:23.862141+0000","last_active":"2024-06-10T00:31:21.019649+0000","last_peered":"2024-06-10T00:31:21.019649+0000","last_clean":"2024-06-10T00:31:21.019649+0000","last_became_active":"2024-06-10T00:29:23.857051+0000","last_became_peered":"2024-06-10T00:29:23.857051+0000","last_unstale":"2024-06-10T00:31:21.019649+0000","last_undegraded":"2024-06-10T00:31:21.019649+0000","last_fullsized":"2024-06-10T00:31:21.019649+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:41:41.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.968693+0000","last_change":"2024-06-10T00:28:37.653938+0000","last_active":"2024-06-10T00:31:21.968693+0000","last_peered":"2024-06-10T00:31:21.968693+0000","last_clean":"2024-06-10T00:31:21.968693+0000","last_became_active":"2024-06-10T00:28:37.653629+0000","last_became_peered":"2024-06-10T00:28:37.653629+0000","last_unstale":"2024-06-10T00:31:21.968693+0000","last_undegraded":"2024-06-10T00:31:21.968693+0000","last_fullsized":"2024-06-10T00:31:21.968693+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:21:39.285961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650408+0000","last_change":"2024-06-10T00:28:48.704413+0000","last_active":"2024-06-10T00:30:24.650408+0000","last_peered":"2024-06-10T00:30:24.650408+0000","last_clean":"2024-06-10T00:30:24.650408+0000","last_became_active":"2024-06-10T00:28:48.703972+0000","last_became_peered":"2024-06-10T00:28:48.703972+0000","last_unstale":"2024-06-10T00:30:24.650408+0000","last_undegraded":"2024-06-10T00:30:24.650408+0000","last_fullsized":"2024-06-10T00:30:24.650408+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:51.927435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.226323+0000","last_change":"2024-06-10T00:28:34.445413+0000","last_active":"2024-06-10T00:30:24.226323+0000","last_peered":"2024-06-10T00:30:24.226323+0000","last_clean":"2024-06-10T00:30:24.226323+0000","last_became_active":"2024-06-10T00:28:34.445077+0000","last_became_peered":"2024-06-10T00:28:34.445077+0000","last_unstale":"2024-06-10T00:30:24.226323+0000","last_undegraded":"2024-06-10T00:30:24.226323+0000","last_fullsized":"2024-06-10T00:30:24.226323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:19:19.535976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.976040+0000","last_change":"2024-06-10T00:28:37.657123+0000","last_active":"2024-06-10T00:31:21.976040+0000","last_peered":"2024-06-10T00:31:21.976040+0000","last_clean":"2024-06-10T00:31:21.976040+0000","last_became_active":"2024-06-10T00:28:37.656791+0000","last_became_peered":"2024-06-10T00:28:37.656791+0000","last_unstale":"2024-06-10T00:31:21.976040+0000","last_undegraded":"2024-06-10T00:31:21.976040+0000","last_fullsized":"2024-06-10T00:31:21.976040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:11.102478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.651391+0000","last_change":"2024-06-10T00:28:48.705989+0000","last_active":"2024-06-10T00:30:24.651391+0000","last_peered":"2024-06-10T00:30:24.651391+0000","last_clean":"2024-06-10T00:30:24.651391+0000","last_became_active":"2024-06-10T00:28:48.705555+0000","last_became_peered":"2024-06-10T00:28:48.705555+0000","last_unstale":"2024-06-10T00:30:24.651391+0000","last_undegraded":"2024-06-10T00:30:24.651391+0000","last_fullsized":"2024-06-10T00:30:24.651391+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:36.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'12","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.109393+0000","last_change":"2024-06-10T00:28:34.400117+0000","last_active":"2024-06-10T00:30:24.109393+0000","last_peered":"2024-06-10T00:30:24.109393+0000","last_clean":"2024-06-10T00:30:24.109393+0000","last_became_active":"2024-06-10T00:28:34.399743+0000","last_became_peered":"2024-06-10T00:28:34.399743+0000","last_unstale":"2024-06-10T00:30:24.109393+0000","last_undegraded":"2024-06-10T00:30:24.109393+0000","last_fullsized":"2024-06-10T00:30:24.109393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:20:57.580146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.978895+0000","last_change":"2024-06-10T00:28:37.664005+0000","last_active":"2024-06-10T00:31:21.978895+0000","last_peered":"2024-06-10T00:31:21.978895+0000","last_clean":"2024-06-10T00:31:21.978895+0000","last_became_active":"2024-06-10T00:28:37.661297+0000","last_became_peered":"2024-06-10T00:28:37.661297+0000","last_unstale":"2024-06-10T00:31:21.978895+0000","last_undegraded":"2024-06-10T00:31:21.978895+0000","last_fullsized":"2024-06-10T00:31:21.978895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:57.795080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650379+0000","last_change":"2024-06-10T00:29:23.859091+0000","last_active":"2024-06-10T00:30:24.650379+0000","last_peered":"2024-06-10T00:30:24.650379+0000","last_clean":"2024-06-10T00:30:24.650379+0000","last_became_active":"2024-06-10T00:29:23.858485+0000","last_became_peered":"2024-06-10T00:29:23.858485+0000","last_unstale":"2024-06-10T00:30:24.650379+0000","last_undegraded":"2024-06-10T00:30:24.650379+0000","last_fullsized":"2024-06-10T00:30:24.650379+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:07:10.925323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.109290+0000","last_change":"2024-06-10T00:28:34.400588+0000","last_active":"2024-06-10T00:30:24.109290+0000","last_peered":"2024-06-10T00:30:24.109290+0000","last_clean":"2024-06-10T00:30:24.109290+0000","last_became_active":"2024-06-10T00:28:34.400027+0000","last_became_peered":"2024-06-10T00:28:34.400027+0000","last_unstale":"2024-06-10T00:30:24.109290+0000","last_undegraded":"2024-06-10T00:30:24.109290+0000","last_fullsized":"2024-06-10T00:30:24.109290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:25:59.890076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.112319+0000","last_change":"2024-06-10T00:28:37.659401+0000","last_active":"2024-06-10T00:30:24.112319+0000","last_peered":"2024-06-10T00:30:24.112319+0000","last_clean":"2024-06-10T00:30:24.112319+0000","last_became_active":"2024-06-10T00:28:37.659056+0000","last_became_peered":"2024-06-10T00:28:37.659056+0000","last_unstale":"2024-06-10T00:30:24.112319+0000","last_undegraded":"2024-06-10T00:30:24.112319+0000","last_fullsized":"2024-06-10T00:30:24.112319+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:35:01.415630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650705+0000","last_change":"2024-06-10T00:28:48.702961+0000","last_active":"2024-06-10T00:30:24.650705+0000","last_peered":"2024-06-10T00:30:24.650705+0000","last_clean":"2024-06-10T00:30:24.650705+0000","last_became_active":"2024-06-10T00:28:48.702502+0000","last_became_peered":"2024-06-10T00:28:48.702502+0000","last_unstale":"2024-06-10T00:30:24.650705+0000","last_undegraded":"2024-06-10T00:30:24.650705+0000","last_fullsized":"2024-06-10T00:30:24.650705+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:54.881595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"97'6","reported_seq":149,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:20.015937+0000","last_change":"2024-06-10T00:28:34.396310+0000","last_active":"2024-06-10T00:31:20.015937+0000","last_peered":"2024-06-10T00:31:20.015937+0000","last_clean":"2024-06-10T00:31:20.015937+0000","last_became_active":"2024-06-10T00:28:34.395401+0000","last_became_peered":"2024-06-10T00:28:34.395401+0000","last_unstale":"2024-06-10T00:31:20.015937+0000","last_undegraded":"2024-06-10T00:31:20.015937+0000","last_fullsized":"2024-06-10T00:31:20.015937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:33:54.300308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.962137+0000","last_change":"2024-06-10T00:28:37.651427+0000","last_active":"2024-06-10T00:31:21.962137+0000","last_peered":"2024-06-10T00:31:21.962137+0000","last_clean":"2024-06-10T00:31:21.962137+0000","last_became_active":"2024-06-10T00:28:37.650510+0000","last_became_peered":"2024-06-10T00:28:37.650510+0000","last_unstale":"2024-06-10T00:31:21.962137+0000","last_undegraded":"2024-06-10T00:31:21.962137+0000","last_fullsized":"2024-06-10T00:31:21.962137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:42.953919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650743+0000","last_change":"2024-06-10T00:28:48.705918+0000","last_active":"2024-06-10T00:30:24.650743+0000","last_peered":"2024-06-10T00:30:24.650743+0000","last_clean":"2024-06-10T00:30:24.650743+0000","last_became_active":"2024-06-10T00:28:48.705462+0000","last_became_peered":"2024-06-10T00:28:48.705462+0000","last_unstale":"2024-06-10T00:30:24.650743+0000","last_undegraded":"2024-06-10T00:30:24.650743+0000","last_fullsized":"2024-06-10T00:30:24.650743+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:59:23.176511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111642+0000","last_change":"2024-06-10T00:28:34.393199+0000","last_active":"2024-06-10T00:30:24.111642+0000","last_peered":"2024-06-10T00:30:24.111642+0000","last_clean":"2024-06-10T00:30:24.111642+0000","last_became_active":"2024-06-10T00:28:34.392777+0000","last_became_peered":"2024-06-10T00:28:34.392777+0000","last_unstale":"2024-06-10T00:30:24.111642+0000","last_undegraded":"2024-06-10T00:30:24.111642+0000","last_fullsized":"2024-06-10T00:30:24.111642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:29.614242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.230125+0000","last_change":"2024-06-10T00:30:24.229721+0000","last_active":"2024-06-10T00:30:24.230125+0000","last_peered":"2024-06-10T00:30:24.230125+0000","last_clean":"2024-06-10T00:30:24.230125+0000","last_became_active":"2024-06-10T00:30:24.229407+0000","last_became_peered":"2024-06-10T00:30:24.229407+0000","last_unstale":"2024-06-10T00:30:24.230125+0000","last_undegraded":"2024-06-10T00:30:24.230125+0000","last_fullsized":"2024-06-10T00:30:24.230125+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:39.523197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.247459+0000","last_change":"2024-06-10T00:29:23.780971+0000","last_active":"2024-06-10T00:30:24.247459+0000","last_peered":"2024-06-10T00:30:24.247459+0000","last_clean":"2024-06-10T00:30:24.247459+0000","last_became_active":"2024-06-10T00:29:23.780034+0000","last_became_peered":"2024-06-10T00:29:23.780034+0000","last_unstale":"2024-06-10T00:30:24.247459+0000","last_undegraded":"2024-06-10T00:30:24.247459+0000","last_fullsized":"2024-06-10T00:30:24.247459+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:47:06.444181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228111+0000","last_change":"2024-06-10T00:28:34.392486+0000","last_active":"2024-06-10T00:30:24.228111+0000","last_peered":"2024-06-10T00:30:24.228111+0000","last_clean":"2024-06-10T00:30:24.228111+0000","last_became_active":"2024-06-10T00:28:34.392141+0000","last_became_peered":"2024-06-10T00:28:34.392141+0000","last_unstale":"2024-06-10T00:30:24.228111+0000","last_undegraded":"2024-06-10T00:30:24.228111+0000","last_fullsized":"2024-06-10T00:30:24.228111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:10:01.534195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.966446+0000","last_change":"2024-06-10T00:28:37.651033+0000","last_active":"2024-06-10T00:31:21.966446+0000","last_peered":"2024-06-10T00:31:21.966446+0000","last_clean":"2024-06-10T00:31:21.966446+0000","last_became_active":"2024-06-10T00:28:37.650505+0000","last_became_peered":"2024-06-10T00:28:37.650505+0000","last_unstale":"2024-06-10T00:31:21.966446+0000","last_undegraded":"2024-06-10T00:31:21.966446+0000","last_fullsized":"2024-06-10T00:31:21.966446+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:54.965865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650007+0000","last_change":"2024-06-10T00:28:48.710748+0000","last_active":"2024-06-10T00:30:24.650007+0000","last_peered":"2024-06-10T00:30:24.650007+0000","last_clean":"2024-06-10T00:30:24.650007+0000","last_became_active":"2024-06-10T00:28:48.710353+0000","last_became_peered":"2024-06-10T00:28:48.710353+0000","last_unstale":"2024-06-10T00:30:24.650007+0000","last_undegraded":"2024-06-10T00:30:24.650007+0000","last_fullsized":"2024-06-10T00:30:24.650007+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:30:47.546650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.112085+0000","last_change":"2024-06-10T00:28:34.398911+0000","last_active":"2024-06-10T00:30:24.112085+0000","last_peered":"2024-06-10T00:30:24.112085+0000","last_clean":"2024-06-10T00:30:24.112085+0000","last_became_active":"2024-06-10T00:28:34.398373+0000","last_became_peered":"2024-06-10T00:28:34.398373+0000","last_unstale":"2024-06-10T00:30:24.112085+0000","last_undegraded":"2024-06-10T00:30:24.112085+0000","last_fullsized":"2024-06-10T00:30:24.112085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:05:34.352825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.971261+0000","last_change":"2024-06-10T00:28:37.647492+0000","last_active":"2024-06-10T00:31:21.971261+0000","last_peered":"2024-06-10T00:31:21.971261+0000","last_clean":"2024-06-10T00:31:21.971261+0000","last_became_active":"2024-06-10T00:28:37.647103+0000","last_became_peered":"2024-06-10T00:28:37.647103+0000","last_unstale":"2024-06-10T00:31:21.971261+0000","last_undegraded":"2024-06-10T00:31:21.971261+0000","last_fullsized":"2024-06-10T00:31:21.971261+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:10.429256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.225577+0000","last_change":"2024-06-10T00:28:48.703095+0000","last_active":"2024-06-10T00:30:24.225577+0000","last_peered":"2024-06-10T00:30:24.225577+0000","last_clean":"2024-06-10T00:30:24.225577+0000","last_became_active":"2024-06-10T00:28:48.702687+0000","last_became_peered":"2024-06-10T00:28:48.702687+0000","last_unstale":"2024-06-10T00:30:24.225577+0000","last_undegraded":"2024-06-10T00:30:24.225577+0000","last_fullsized":"2024-06-10T00:30:24.225577+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:10:20.074173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'5","reported_seq":245,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:23.019819+0000","last_change":"2024-06-10T00:28:34.394807+0000","last_active":"2024-06-10T00:31:23.019819+0000","last_peered":"2024-06-10T00:31:23.019819+0000","last_clean":"2024-06-10T00:31:23.019819+0000","last_became_active":"2024-06-10T00:28:34.394022+0000","last_became_peered":"2024-06-10T00:28:34.394022+0000","last_unstale":"2024-06-10T00:31:23.019819+0000","last_undegraded":"2024-06-10T00:31:23.019819+0000","last_fullsized":"2024-06-10T00:31:23.019819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:21.626079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":196,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.978903+0000","last_change":"2024-06-10T00:28:37.662231+0000","last_active":"2024-06-10T00:31:21.978903+0000","last_peered":"2024-06-10T00:31:21.978903+0000","last_clean":"2024-06-10T00:31:21.978903+0000","last_became_active":"2024-06-10T00:28:37.661861+0000","last_became_peered":"2024-06-10T00:28:37.661861+0000","last_unstale":"2024-06-10T00:31:21.978903+0000","last_undegraded":"2024-06-10T00:31:21.978903+0000","last_fullsized":"2024-06-10T00:31:21.978903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:09.705349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.107608+0000","last_change":"2024-06-10T00:28:48.706643+0000","last_active":"2024-06-10T00:30:24.107608+0000","last_peered":"2024-06-10T00:30:24.107608+0000","last_clean":"2024-06-10T00:30:24.107608+0000","last_became_active":"2024-06-10T00:28:48.706127+0000","last_became_peered":"2024-06-10T00:28:48.706127+0000","last_unstale":"2024-06-10T00:30:24.107608+0000","last_undegraded":"2024-06-10T00:30:24.107608+0000","last_fullsized":"2024-06-10T00:30:24.107608+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:55.031303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111496+0000","last_change":"2024-06-10T00:28:34.392808+0000","last_active":"2024-06-10T00:30:24.111496+0000","last_peered":"2024-06-10T00:30:24.111496+0000","last_clean":"2024-06-10T00:30:24.111496+0000","last_became_active":"2024-06-10T00:28:34.392450+0000","last_became_peered":"2024-06-10T00:28:34.392450+0000","last_unstale":"2024-06-10T00:30:24.111496+0000","last_undegraded":"2024-06-10T00:30:24.111496+0000","last_fullsized":"2024-06-10T00:30:24.111496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:06:00.493316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.978033+0000","last_change":"2024-06-10T00:28:37.656099+0000","last_active":"2024-06-10T00:31:21.978033+0000","last_peered":"2024-06-10T00:31:21.978033+0000","last_clean":"2024-06-10T00:31:21.978033+0000","last_became_active":"2024-06-10T00:28:37.655640+0000","last_became_peered":"2024-06-10T00:28:37.655640+0000","last_unstale":"2024-06-10T00:31:21.978033+0000","last_undegraded":"2024-06-10T00:31:21.978033+0000","last_fullsized":"2024-06-10T00:31:21.978033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:59:26.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.651501+0000","last_change":"2024-06-10T00:28:48.702628+0000","last_active":"2024-06-10T00:30:24.651501+0000","last_peered":"2024-06-10T00:30:24.651501+0000","last_clean":"2024-06-10T00:30:24.651501+0000","last_became_active":"2024-06-10T00:28:48.702109+0000","last_became_peered":"2024-06-10T00:28:48.702109+0000","last_unstale":"2024-06-10T00:30:24.651501+0000","last_undegraded":"2024-06-10T00:30:24.651501+0000","last_fullsized":"2024-06-10T00:30:24.651501+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:25.262348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.112201+0000","last_change":"2024-06-10T00:28:34.446788+0000","last_active":"2024-06-10T00:30:24.112201+0000","last_peered":"2024-06-10T00:30:24.112201+0000","last_clean":"2024-06-10T00:30:24.112201+0000","last_became_active":"2024-06-10T00:28:34.446377+0000","last_became_peered":"2024-06-10T00:28:34.446377+0000","last_unstale":"2024-06-10T00:30:24.112201+0000","last_undegraded":"2024-06-10T00:30:24.112201+0000","last_fullsized":"2024-06-10T00:30:24.112201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:16.914244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.974064+0000","last_change":"2024-06-10T00:28:37.666159+0000","last_active":"2024-06-10T00:31:21.974064+0000","last_peered":"2024-06-10T00:31:21.974064+0000","last_clean":"2024-06-10T00:31:21.974064+0000","last_became_active":"2024-06-10T00:28:37.665711+0000","last_became_peered":"2024-06-10T00:28:37.665711+0000","last_unstale":"2024-06-10T00:31:21.974064+0000","last_undegraded":"2024-06-10T00:31:21.974064+0000","last_fullsized":"2024-06-10T00:31:21.974064+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:03.982375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.107717+0000","last_change":"2024-06-10T00:28:48.705071+0000","last_active":"2024-06-10T00:30:24.107717+0000","last_peered":"2024-06-10T00:30:24.107717+0000","last_clean":"2024-06-10T00:30:24.107717+0000","last_became_active":"2024-06-10T00:28:48.702211+0000","last_became_peered":"2024-06-10T00:28:48.702211+0000","last_unstale":"2024-06-10T00:30:24.107717+0000","last_undegraded":"2024-06-10T00:30:24.107717+0000","last_fullsized":"2024-06-10T00:30:24.107717+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:13.361985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'37","reported_seq":287,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:23.019803+0000","last_change":"2024-06-10T00:28:34.389784+0000","last_active":"2024-06-10T00:31:23.019803+0000","last_peered":"2024-06-10T00:31:23.019803+0000","last_clean":"2024-06-10T00:31:23.019803+0000","last_became_active":"2024-06-10T00:28:34.389416+0000","last_became_peered":"2024-06-10T00:28:34.389416+0000","last_unstale":"2024-06-10T00:31:23.019803+0000","last_undegraded":"2024-06-10T00:31:23.019803+0000","last_fullsized":"2024-06-10T00:31:23.019803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:23:05.803503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62904,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":206,"num_read_kb":165,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.958683+0000","last_change":"2024-06-10T00:28:37.656174+0000","last_active":"2024-06-10T00:31:21.958683+0000","last_peered":"2024-06-10T00:31:21.958683+0000","last_clean":"2024-06-10T00:31:21.958683+0000","last_became_active":"2024-06-10T00:28:37.655794+0000","last_became_peered":"2024-06-10T00:28:37.655794+0000","last_unstale":"2024-06-10T00:31:21.958683+0000","last_undegraded":"2024-06-10T00:31:21.958683+0000","last_fullsized":"2024-06-10T00:31:21.958683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:22:00.310309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.112614+0000","last_change":"2024-06-10T00:28:48.709915+0000","last_active":"2024-06-10T00:30:24.112614+0000","last_peered":"2024-06-10T00:30:24.112614+0000","last_clean":"2024-06-10T00:30:24.112614+0000","last_became_active":"2024-06-10T00:28:48.709426+0000","last_became_peered":"2024-06-10T00:28:48.709426+0000","last_unstale":"2024-06-10T00:30:24.112614+0000","last_undegraded":"2024-06-10T00:30:24.112614+0000","last_fullsized":"2024-06-10T00:30:24.112614+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:55.150617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650583+0000","last_change":"2024-06-10T00:28:34.389136+0000","last_active":"2024-06-10T00:30:24.650583+0000","last_peered":"2024-06-10T00:30:24.650583+0000","last_clean":"2024-06-10T00:30:24.650583+0000","last_became_active":"2024-06-10T00:28:34.388701+0000","last_became_peered":"2024-06-10T00:28:34.388701+0000","last_unstale":"2024-06-10T00:30:24.650583+0000","last_undegraded":"2024-06-10T00:30:24.650583+0000","last_fullsized":"2024-06-10T00:30:24.650583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:13:24.907824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228190+0000","last_change":"2024-06-10T00:28:37.659048+0000","last_active":"2024-06-10T00:30:24.228190+0000","last_peered":"2024-06-10T00:30:24.228190+0000","last_clean":"2024-06-10T00:30:24.228190+0000","last_became_active":"2024-06-10T00:28:37.658710+0000","last_became_peered":"2024-06-10T00:28:37.658710+0000","last_unstale":"2024-06-10T00:30:24.228190+0000","last_undegraded":"2024-06-10T00:30:24.228190+0000","last_fullsized":"2024-06-10T00:30:24.228190+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:44.314559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.247202+0000","last_change":"2024-06-10T00:28:48.709683+0000","last_active":"2024-06-10T00:30:24.247202+0000","last_peered":"2024-06-10T00:30:24.247202+0000","last_clean":"2024-06-10T00:30:24.247202+0000","last_became_active":"2024-06-10T00:28:48.709241+0000","last_became_peered":"2024-06-10T00:28:48.709241+0000","last_unstale":"2024-06-10T00:30:24.247202+0000","last_undegraded":"2024-06-10T00:30:24.247202+0000","last_fullsized":"2024-06-10T00:30:24.247202+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:25.190043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":21,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.247060+0000","last_change":"2024-06-10T00:29:23.782029+0000","last_active":"2024-06-10T00:30:24.247060+0000","last_peered":"2024-06-10T00:30:24.247060+0000","last_clean":"2024-06-10T00:30:24.247060+0000","last_became_active":"2024-06-10T00:29:23.781565+0000","last_became_peered":"2024-06-10T00:29:23.781565+0000","last_unstale":"2024-06-10T00:30:24.247060+0000","last_undegraded":"2024-06-10T00:30:24.247060+0000","last_fullsized":"2024-06-10T00:30:24.247060+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:53.258760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111206+0000","last_change":"2024-06-10T00:28:37.653385+0000","last_active":"2024-06-10T00:30:24.111206+0000","last_peered":"2024-06-10T00:30:24.111206+0000","last_clean":"2024-06-10T00:30:24.111206+0000","last_became_active":"2024-06-10T00:28:37.653020+0000","last_became_peered":"2024-06-10T00:28:37.653020+0000","last_unstale":"2024-06-10T00:30:24.111206+0000","last_undegraded":"2024-06-10T00:30:24.111206+0000","last_fullsized":"2024-06-10T00:30:24.111206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:57:54.469544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.108292+0000","last_change":"2024-06-10T00:28:48.707540+0000","last_active":"2024-06-10T00:30:24.108292+0000","last_peered":"2024-06-10T00:30:24.108292+0000","last_clean":"2024-06-10T00:30:24.108292+0000","last_became_active":"2024-06-10T00:28:48.707143+0000","last_became_peered":"2024-06-10T00:28:48.707143+0000","last_unstale":"2024-06-10T00:30:24.108292+0000","last_undegraded":"2024-06-10T00:30:24.108292+0000","last_fullsized":"2024-06-10T00:30:24.108292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:20:29.344808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'7","reported_seq":153,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:22.018397+0000","last_change":"2024-06-10T00:28:34.397496+0000","last_active":"2024-06-10T00:31:22.018397+0000","last_peered":"2024-06-10T00:31:22.018397+0000","last_clean":"2024-06-10T00:31:22.018397+0000","last_became_active":"2024-06-10T00:28:34.397124+0000","last_became_peered":"2024-06-10T00:28:34.397124+0000","last_unstale":"2024-06-10T00:31:22.018397+0000","last_undegraded":"2024-06-10T00:31:22.018397+0000","last_fullsized":"2024-06-10T00:31:22.018397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:41:48.609354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111104+0000","last_change":"2024-06-10T00:28:37.658338+0000","last_active":"2024-06-10T00:30:24.111104+0000","last_peered":"2024-06-10T00:30:24.111104+0000","last_clean":"2024-06-10T00:30:24.111104+0000","last_became_active":"2024-06-10T00:28:37.655910+0000","last_became_peered":"2024-06-10T00:28:37.655910+0000","last_unstale":"2024-06-10T00:30:24.111104+0000","last_undegraded":"2024-06-10T00:30:24.111104+0000","last_fullsized":"2024-06-10T00:30:24.111104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:38.501335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228758+0000","last_change":"2024-06-10T00:28:48.711151+0000","last_active":"2024-06-10T00:30:24.228758+0000","last_peered":"2024-06-10T00:30:24.228758+0000","last_clean":"2024-06-10T00:30:24.228758+0000","last_became_active":"2024-06-10T00:28:48.710768+0000","last_became_peered":"2024-06-10T00:28:48.710768+0000","last_unstale":"2024-06-10T00:30:24.228758+0000","last_undegraded":"2024-06-10T00:30:24.228758+0000","last_fullsized":"2024-06-10T00:30:24.228758+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:15:22.791839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":63,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.108802+0000","last_change":"2024-06-10T00:28:34.445812+0000","last_active":"2024-06-10T00:30:24.108802+0000","last_peered":"2024-06-10T00:30:24.108802+0000","last_clean":"2024-06-10T00:30:24.108802+0000","last_became_active":"2024-06-10T00:28:34.445472+0000","last_became_peered":"2024-06-10T00:28:34.445472+0000","last_unstale":"2024-06-10T00:30:24.108802+0000","last_undegraded":"2024-06-10T00:30:24.108802+0000","last_fullsized":"2024-06-10T00:30:24.108802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:00.912050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.112042+0000","last_change":"2024-06-10T00:28:37.655345+0000","last_active":"2024-06-10T00:30:24.112042+0000","last_peered":"2024-06-10T00:30:24.112042+0000","last_clean":"2024-06-10T00:30:24.112042+0000","last_became_active":"2024-06-10T00:28:37.654954+0000","last_became_peered":"2024-06-10T00:28:37.654954+0000","last_unstale":"2024-06-10T00:30:24.112042+0000","last_undegraded":"2024-06-10T00:30:24.112042+0000","last_fullsized":"2024-06-10T00:30:24.112042+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:39:27.403006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111912+0000","last_change":"2024-06-10T00:28:48.705247+0000","last_active":"2024-06-10T00:30:24.111912+0000","last_peered":"2024-06-10T00:30:24.111912+0000","last_clean":"2024-06-10T00:30:24.111912+0000","last_became_active":"2024-06-10T00:28:48.704822+0000","last_became_peered":"2024-06-10T00:28:48.704822+0000","last_unstale":"2024-06-10T00:30:24.111912+0000","last_undegraded":"2024-06-10T00:30:24.111912+0000","last_fullsized":"2024-06-10T00:30:24.111912+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:24:59.991491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.109540+0000","last_change":"2024-06-10T00:28:34.396792+0000","last_active":"2024-06-10T00:30:24.109540+0000","last_peered":"2024-06-10T00:30:24.109540+0000","last_clean":"2024-06-10T00:30:24.109540+0000","last_became_active":"2024-06-10T00:28:34.396442+0000","last_became_peered":"2024-06-10T00:28:34.396442+0000","last_unstale":"2024-06-10T00:30:24.109540+0000","last_undegraded":"2024-06-10T00:30:24.109540+0000","last_fullsized":"2024-06-10T00:30:24.109540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:46.436198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.967706+0000","last_change":"2024-06-10T00:28:37.657698+0000","last_active":"2024-06-10T00:31:21.967706+0000","last_peered":"2024-06-10T00:31:21.967706+0000","last_clean":"2024-06-10T00:31:21.967706+0000","last_became_active":"2024-06-10T00:28:37.657364+0000","last_became_peered":"2024-06-10T00:28:37.657364+0000","last_unstale":"2024-06-10T00:31:21.967706+0000","last_undegraded":"2024-06-10T00:31:21.967706+0000","last_fullsized":"2024-06-10T00:31:21.967706+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:23.097015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.108917+0000","last_change":"2024-06-10T00:28:48.727567+0000","last_active":"2024-06-10T00:30:24.108917+0000","last_peered":"2024-06-10T00:30:24.108917+0000","last_clean":"2024-06-10T00:30:24.108917+0000","last_became_active":"2024-06-10T00:28:48.727108+0000","last_became_peered":"2024-06-10T00:28:48.727108+0000","last_unstale":"2024-06-10T00:30:24.108917+0000","last_undegraded":"2024-06-10T00:30:24.108917+0000","last_fullsized":"2024-06-10T00:30:24.108917+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:35:00.824177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.108030+0000","last_change":"2024-06-10T00:28:34.446832+0000","last_active":"2024-06-10T00:30:24.108030+0000","last_peered":"2024-06-10T00:30:24.108030+0000","last_clean":"2024-06-10T00:30:24.108030+0000","last_became_active":"2024-06-10T00:28:34.446435+0000","last_became_peered":"2024-06-10T00:28:34.446435+0000","last_unstale":"2024-06-10T00:30:24.108030+0000","last_undegraded":"2024-06-10T00:30:24.108030+0000","last_fullsized":"2024-06-10T00:30:24.108030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:58.296674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.971403+0000","last_change":"2024-06-10T00:30:24.229514+0000","last_active":"2024-06-10T00:31:21.971403+0000","last_peered":"2024-06-10T00:31:21.971403+0000","last_clean":"2024-06-10T00:31:21.971403+0000","last_became_active":"2024-06-10T00:30:24.229163+0000","last_became_peered":"2024-06-10T00:30:24.229163+0000","last_unstale":"2024-06-10T00:31:21.971403+0000","last_undegraded":"2024-06-10T00:31:21.971403+0000","last_fullsized":"2024-06-10T00:31:21.971403+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:52.914144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228507+0000","last_change":"2024-06-10T00:28:48.704683+0000","last_active":"2024-06-10T00:30:24.228507+0000","last_peered":"2024-06-10T00:30:24.228507+0000","last_clean":"2024-06-10T00:30:24.228507+0000","last_became_active":"2024-06-10T00:28:48.704297+0000","last_became_peered":"2024-06-10T00:28:48.704297+0000","last_unstale":"2024-06-10T00:30:24.228507+0000","last_undegraded":"2024-06-10T00:30:24.228507+0000","last_fullsized":"2024-06-10T00:30:24.228507+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:44.980544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'5","reported_seq":146,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:20.018839+0000","last_change":"2024-06-10T00:28:34.393328+0000","last_active":"2024-06-10T00:31:20.018839+0000","last_peered":"2024-06-10T00:31:20.018839+0000","last_clean":"2024-06-10T00:31:20.018839+0000","last_became_active":"2024-06-10T00:28:34.391968+0000","last_became_peered":"2024-06-10T00:28:34.391968+0000","last_unstale":"2024-06-10T00:31:20.018839+0000","last_undegraded":"2024-06-10T00:31:20.018839+0000","last_fullsized":"2024-06-10T00:31:20.018839+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:18.898180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.969828+0000","last_change":"2024-06-10T00:28:37.652555+0000","last_active":"2024-06-10T00:31:21.969828+0000","last_peered":"2024-06-10T00:31:21.969828+0000","last_clean":"2024-06-10T00:31:21.969828+0000","last_became_active":"2024-06-10T00:28:37.651476+0000","last_became_peered":"2024-06-10T00:28:37.651476+0000","last_unstale":"2024-06-10T00:31:21.969828+0000","last_undegraded":"2024-06-10T00:31:21.969828+0000","last_fullsized":"2024-06-10T00:31:21.969828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:04:57.823224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228715+0000","last_change":"2024-06-10T00:28:48.727568+0000","last_active":"2024-06-10T00:30:24.228715+0000","last_peered":"2024-06-10T00:30:24.228715+0000","last_clean":"2024-06-10T00:30:24.228715+0000","last_became_active":"2024-06-10T00:28:48.727018+0000","last_became_peered":"2024-06-10T00:28:48.727018+0000","last_unstale":"2024-06-10T00:30:24.228715+0000","last_undegraded":"2024-06-10T00:30:24.228715+0000","last_fullsized":"2024-06-10T00:30:24.228715+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:02.025212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":19,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.958300+0000","last_change":"2024-06-10T00:30:24.651947+0000","last_active":"2024-06-10T00:31:21.958300+0000","last_peered":"2024-06-10T00:31:21.958300+0000","last_clean":"2024-06-10T00:31:21.958300+0000","last_became_active":"2024-06-10T00:30:24.651640+0000","last_became_peered":"2024-06-10T00:30:24.651640+0000","last_unstale":"2024-06-10T00:31:21.958300+0000","last_undegraded":"2024-06-10T00:31:21.958300+0000","last_fullsized":"2024-06-10T00:31:21.958300+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:47.070568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"97'5","reported_seq":146,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:20.015054+0000","last_change":"2024-06-10T00:28:34.445501+0000","last_active":"2024-06-10T00:31:20.015054+0000","last_peered":"2024-06-10T00:31:20.015054+0000","last_clean":"2024-06-10T00:31:20.015054+0000","last_became_active":"2024-06-10T00:28:34.445128+0000","last_became_peered":"2024-06-10T00:28:34.445128+0000","last_unstale":"2024-06-10T00:31:20.015054+0000","last_undegraded":"2024-06-10T00:31:20.015054+0000","last_fullsized":"2024-06-10T00:31:20.015054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:59.231585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111390+0000","last_change":"2024-06-10T00:28:48.709852+0000","last_active":"2024-06-10T00:30:24.111390+0000","last_peered":"2024-06-10T00:30:24.111390+0000","last_clean":"2024-06-10T00:30:24.111390+0000","last_became_active":"2024-06-10T00:28:48.709423+0000","last_became_peered":"2024-06-10T00:28:48.709423+0000","last_unstale":"2024-06-10T00:30:24.111390+0000","last_undegraded":"2024-06-10T00:30:24.111390+0000","last_fullsized":"2024-06-10T00:30:24.111390+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:42.123563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.969342+0000","last_change":"2024-06-10T00:28:37.657423+0000","last_active":"2024-06-10T00:31:21.969342+0000","last_peered":"2024-06-10T00:31:21.969342+0000","last_clean":"2024-06-10T00:31:21.969342+0000","last_became_active":"2024-06-10T00:28:37.657140+0000","last_became_peered":"2024-06-10T00:28:37.657140+0000","last_unstale":"2024-06-10T00:31:21.969342+0000","last_undegraded":"2024-06-10T00:31:21.969342+0000","last_fullsized":"2024-06-10T00:31:21.969342+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:20:08.943066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.651508+0000","last_change":"2024-06-10T00:28:34.392585+0000","last_active":"2024-06-10T00:30:24.651508+0000","last_peered":"2024-06-10T00:30:24.651508+0000","last_clean":"2024-06-10T00:30:24.651508+0000","last_became_active":"2024-06-10T00:28:34.392229+0000","last_became_peered":"2024-06-10T00:28:34.392229+0000","last_unstale":"2024-06-10T00:30:24.651508+0000","last_undegraded":"2024-06-10T00:30:24.651508+0000","last_fullsized":"2024-06-10T00:30:24.651508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:11.275761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":377,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.110595+0000","last_change":"2024-06-10T00:27:12.072577+0000","last_active":"2024-06-10T00:30:24.110595+0000","last_peered":"2024-06-10T00:30:24.110595+0000","last_clean":"2024-06-10T00:30:24.110595+0000","last_became_active":"2024-06-10T00:27:12.072193+0000","last_became_peered":"2024-06-10T00:27:12.072193+0000","last_unstale":"2024-06-10T00:30:24.110595+0000","last_undegraded":"2024-06-10T00:30:24.110595+0000","last_fullsized":"2024-06-10T00:30:24.110595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_clean_scrub_stamp":"2024-06-10T00:23:46.826438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:57.782124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.112299+0000","last_change":"2024-06-10T00:28:48.727913+0000","last_active":"2024-06-10T00:30:24.112299+0000","last_peered":"2024-06-10T00:30:24.112299+0000","last_clean":"2024-06-10T00:30:24.112299+0000","last_became_active":"2024-06-10T00:28:48.727402+0000","last_became_peered":"2024-06-10T00:28:48.727402+0000","last_unstale":"2024-06-10T00:30:24.112299+0000","last_undegraded":"2024-06-10T00:30:24.112299+0000","last_fullsized":"2024-06-10T00:30:24.112299+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:10:25.051296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.227978+0000","last_change":"2024-06-10T00:28:37.659901+0000","last_active":"2024-06-10T00:30:24.227978+0000","last_peered":"2024-06-10T00:30:24.227978+0000","last_clean":"2024-06-10T00:30:24.227978+0000","last_became_active":"2024-06-10T00:28:37.659553+0000","last_became_peered":"2024-06-10T00:28:37.659553+0000","last_unstale":"2024-06-10T00:30:24.227978+0000","last_undegraded":"2024-06-10T00:30:24.227978+0000","last_fullsized":"2024-06-10T00:30:24.227978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:25.694823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111061+0000","last_change":"2024-06-10T00:28:34.398155+0000","last_active":"2024-06-10T00:30:24.111061+0000","last_peered":"2024-06-10T00:30:24.111061+0000","last_clean":"2024-06-10T00:30:24.111061+0000","last_became_active":"2024-06-10T00:28:34.397793+0000","last_became_peered":"2024-06-10T00:28:34.397793+0000","last_unstale":"2024-06-10T00:30:24.111061+0000","last_undegraded":"2024-06-10T00:30:24.111061+0000","last_fullsized":"2024-06-10T00:30:24.111061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:21.188139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.247169+0000","last_change":"2024-06-10T00:28:48.709736+0000","last_active":"2024-06-10T00:30:24.247169+0000","last_peered":"2024-06-10T00:30:24.247169+0000","last_clean":"2024-06-10T00:30:24.247169+0000","last_became_active":"2024-06-10T00:28:48.709293+0000","last_became_peered":"2024-06-10T00:28:48.709293+0000","last_unstale":"2024-06-10T00:30:24.247169+0000","last_undegraded":"2024-06-10T00:30:24.247169+0000","last_fullsized":"2024-06-10T00:30:24.247169+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:30:04.840656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":20,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.247174+0000","last_change":"2024-06-10T00:29:23.780030+0000","last_active":"2024-06-10T00:30:24.247174+0000","last_peered":"2024-06-10T00:30:24.247174+0000","last_clean":"2024-06-10T00:30:24.247174+0000","last_became_active":"2024-06-10T00:29:23.779589+0000","last_became_peered":"2024-06-10T00:29:23.779589+0000","last_unstale":"2024-06-10T00:30:24.247174+0000","last_undegraded":"2024-06-10T00:30:24.247174+0000","last_fullsized":"2024-06-10T00:30:24.247174+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:53.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228632+0000","last_change":"2024-06-10T00:28:37.658330+0000","last_active":"2024-06-10T00:30:24.228632+0000","last_peered":"2024-06-10T00:30:24.228632+0000","last_clean":"2024-06-10T00:30:24.228632+0000","last_became_active":"2024-06-10T00:28:37.657343+0000","last_became_peered":"2024-06-10T00:28:37.657343+0000","last_unstale":"2024-06-10T00:30:24.228632+0000","last_undegraded":"2024-06-10T00:30:24.228632+0000","last_fullsized":"2024-06-10T00:30:24.228632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:31:52.167402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.225501+0000","last_change":"2024-06-10T00:28:48.727506+0000","last_active":"2024-06-10T00:30:24.225501+0000","last_peered":"2024-06-10T00:30:24.225501+0000","last_clean":"2024-06-10T00:30:24.225501+0000","last_became_active":"2024-06-10T00:28:48.727068+0000","last_became_peered":"2024-06-10T00:28:48.727068+0000","last_unstale":"2024-06-10T00:30:24.225501+0000","last_undegraded":"2024-06-10T00:30:24.225501+0000","last_fullsized":"2024-06-10T00:30:24.225501+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:42.147908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'16","reported_seq":61,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:47.932155+0000","last_change":"2024-06-10T00:28:34.399872+0000","last_active":"2024-06-10T00:30:47.932155+0000","last_peered":"2024-06-10T00:30:47.932155+0000","last_clean":"2024-06-10T00:30:47.932155+0000","last_became_active":"2024-06-10T00:28:34.399493+0000","last_became_peered":"2024-06-10T00:28:34.399493+0000","last_unstale":"2024-06-10T00:30:47.932155+0000","last_undegraded":"2024-06-10T00:30:47.932155+0000","last_fullsized":"2024-06-10T00:30:47.932155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:24:10.018110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.970161+0000","last_change":"2024-06-10T00:28:37.656597+0000","last_active":"2024-06-10T00:31:21.970161+0000","last_peered":"2024-06-10T00:31:21.970161+0000","last_clean":"2024-06-10T00:31:21.970161+0000","last_became_active":"2024-06-10T00:28:37.654356+0000","last_became_peered":"2024-06-10T00:28:37.654356+0000","last_unstale":"2024-06-10T00:31:21.970161+0000","last_undegraded":"2024-06-10T00:31:21.970161+0000","last_fullsized":"2024-06-10T00:31:21.970161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:07:12.950683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650311+0000","last_change":"2024-06-10T00:28:48.702686+0000","last_active":"2024-06-10T00:30:24.650311+0000","last_peered":"2024-06-10T00:30:24.650311+0000","last_clean":"2024-06-10T00:30:24.650311+0000","last_became_active":"2024-06-10T00:28:48.702273+0000","last_became_peered":"2024-06-10T00:28:48.702273+0000","last_unstale":"2024-06-10T00:30:24.650311+0000","last_undegraded":"2024-06-10T00:30:24.650311+0000","last_fullsized":"2024-06-10T00:30:24.650311+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:00.078232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650742+0000","last_change":"2024-06-10T00:28:34.447549+0000","last_active":"2024-06-10T00:30:24.650742+0000","last_peered":"2024-06-10T00:30:24.650742+0000","last_clean":"2024-06-10T00:30:24.650742+0000","last_became_active":"2024-06-10T00:28:34.447197+0000","last_became_peered":"2024-06-10T00:28:34.447197+0000","last_unstale":"2024-06-10T00:30:24.650742+0000","last_undegraded":"2024-06-10T00:30:24.650742+0000","last_fullsized":"2024-06-10T00:30:24.650742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:47.525153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.977843+0000","last_change":"2024-06-10T00:28:37.655585+0000","last_active":"2024-06-10T00:31:21.977843+0000","last_peered":"2024-06-10T00:31:21.977843+0000","last_clean":"2024-06-10T00:31:21.977843+0000","last_became_active":"2024-06-10T00:28:37.655225+0000","last_became_peered":"2024-06-10T00:28:37.655225+0000","last_unstale":"2024-06-10T00:31:21.977843+0000","last_undegraded":"2024-06-10T00:31:21.977843+0000","last_fullsized":"2024-06-10T00:31:21.977843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:57:25.184924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650847+0000","last_change":"2024-06-10T00:29:23.857447+0000","last_active":"2024-06-10T00:30:24.650847+0000","last_peered":"2024-06-10T00:30:24.650847+0000","last_clean":"2024-06-10T00:30:24.650847+0000","last_became_active":"2024-06-10T00:29:23.856943+0000","last_became_peered":"2024-06-10T00:29:23.856943+0000","last_unstale":"2024-06-10T00:30:24.650847+0000","last_undegraded":"2024-06-10T00:30:24.650847+0000","last_fullsized":"2024-06-10T00:30:24.650847+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:58:44.548818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650239+0000","last_change":"2024-06-10T00:28:34.445637+0000","last_active":"2024-06-10T00:30:24.650239+0000","last_peered":"2024-06-10T00:30:24.650239+0000","last_clean":"2024-06-10T00:30:24.650239+0000","last_became_active":"2024-06-10T00:28:34.445324+0000","last_became_peered":"2024-06-10T00:28:34.445324+0000","last_unstale":"2024-06-10T00:30:24.650239+0000","last_undegraded":"2024-06-10T00:30:24.650239+0000","last_fullsized":"2024-06-10T00:30:24.650239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:05.713557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.927333+0000","last_change":"2024-06-10T00:28:37.659037+0000","last_active":"2024-06-10T00:31:21.927333+0000","last_peered":"2024-06-10T00:31:21.927333+0000","last_clean":"2024-06-10T00:31:21.927333+0000","last_became_active":"2024-06-10T00:28:37.658705+0000","last_became_peered":"2024-06-10T00:28:37.658705+0000","last_unstale":"2024-06-10T00:31:21.927333+0000","last_undegraded":"2024-06-10T00:31:21.927333+0000","last_fullsized":"2024-06-10T00:31:21.927333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:20:52.797237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.108350+0000","last_change":"2024-06-10T00:28:48.704130+0000","last_active":"2024-06-10T00:30:24.108350+0000","last_peered":"2024-06-10T00:30:24.108350+0000","last_clean":"2024-06-10T00:30:24.108350+0000","last_became_active":"2024-06-10T00:28:48.703738+0000","last_became_peered":"2024-06-10T00:28:48.703738+0000","last_unstale":"2024-06-10T00:30:24.108350+0000","last_undegraded":"2024-06-10T00:30:24.108350+0000","last_fullsized":"2024-06-10T00:30:24.108350+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:01:40.436710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.225578+0000","last_change":"2024-06-10T00:28:34.386667+0000","last_active":"2024-06-10T00:30:24.225578+0000","last_peered":"2024-06-10T00:30:24.225578+0000","last_clean":"2024-06-10T00:30:24.225578+0000","last_became_active":"2024-06-10T00:28:34.386363+0000","last_became_peered":"2024-06-10T00:28:34.386363+0000","last_unstale":"2024-06-10T00:30:24.225578+0000","last_undegraded":"2024-06-10T00:30:24.225578+0000","last_fullsized":"2024-06-10T00:30:24.225578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:11.850162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.974176+0000","last_change":"2024-06-10T00:28:37.664754+0000","last_active":"2024-06-10T00:31:21.974176+0000","last_peered":"2024-06-10T00:31:21.974176+0000","last_clean":"2024-06-10T00:31:21.974176+0000","last_became_active":"2024-06-10T00:28:37.658574+0000","last_became_peered":"2024-06-10T00:28:37.658574+0000","last_unstale":"2024-06-10T00:31:21.974176+0000","last_undegraded":"2024-06-10T00:31:21.974176+0000","last_fullsized":"2024-06-10T00:31:21.974176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:16:47.437822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650338+0000","last_change":"2024-06-10T00:28:48.706560+0000","last_active":"2024-06-10T00:30:24.650338+0000","last_peered":"2024-06-10T00:30:24.650338+0000","last_clean":"2024-06-10T00:30:24.650338+0000","last_became_active":"2024-06-10T00:28:48.705986+0000","last_became_peered":"2024-06-10T00:28:48.705986+0000","last_unstale":"2024-06-10T00:30:24.650338+0000","last_undegraded":"2024-06-10T00:30:24.650338+0000","last_fullsized":"2024-06-10T00:30:24.650338+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:21:26.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650575+0000","last_change":"2024-06-10T00:28:34.394509+0000","last_active":"2024-06-10T00:30:24.650575+0000","last_peered":"2024-06-10T00:30:24.650575+0000","last_clean":"2024-06-10T00:30:24.650575+0000","last_became_active":"2024-06-10T00:28:34.394105+0000","last_became_peered":"2024-06-10T00:28:34.394105+0000","last_unstale":"2024-06-10T00:30:24.650575+0000","last_undegraded":"2024-06-10T00:30:24.650575+0000","last_fullsized":"2024-06-10T00:30:24.650575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:13.415544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.950432+0000","last_change":"2024-06-10T00:28:37.653192+0000","last_active":"2024-06-10T00:31:21.950432+0000","last_peered":"2024-06-10T00:31:21.950432+0000","last_clean":"2024-06-10T00:31:21.950432+0000","last_became_active":"2024-06-10T00:28:37.652868+0000","last_became_peered":"2024-06-10T00:28:37.652868+0000","last_unstale":"2024-06-10T00:31:21.950432+0000","last_undegraded":"2024-06-10T00:31:21.950432+0000","last_fullsized":"2024-06-10T00:31:21.950432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:21.614915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650624+0000","last_change":"2024-06-10T00:28:48.727615+0000","last_active":"2024-06-10T00:30:24.650624+0000","last_peered":"2024-06-10T00:30:24.650624+0000","last_clean":"2024-06-10T00:30:24.650624+0000","last_became_active":"2024-06-10T00:28:48.727190+0000","last_became_peered":"2024-06-10T00:28:48.727190+0000","last_unstale":"2024-06-10T00:30:24.650624+0000","last_undegraded":"2024-06-10T00:30:24.650624+0000","last_fullsized":"2024-06-10T00:30:24.650624+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:57.498183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":60,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.109216+0000","last_change":"2024-06-10T00:29:23.784131+0000","last_active":"2024-06-10T00:30:24.109216+0000","last_peered":"2024-06-10T00:30:24.109216+0000","last_clean":"2024-06-10T00:30:24.109216+0000","last_became_active":"2024-06-10T00:29:23.780909+0000","last_became_peered":"2024-06-10T00:29:23.780909+0000","last_unstale":"2024-06-10T00:30:24.109216+0000","last_undegraded":"2024-06-10T00:30:24.109216+0000","last_fullsized":"2024-06-10T00:30:24.109216+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.774217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111015+0000","last_change":"2024-06-10T00:28:37.660399+0000","last_active":"2024-06-10T00:30:24.111015+0000","last_peered":"2024-06-10T00:30:24.111015+0000","last_clean":"2024-06-10T00:30:24.111015+0000","last_became_active":"2024-06-10T00:28:37.660069+0000","last_became_peered":"2024-06-10T00:28:37.660069+0000","last_unstale":"2024-06-10T00:30:24.111015+0000","last_undegraded":"2024-06-10T00:30:24.111015+0000","last_fullsized":"2024-06-10T00:30:24.111015+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:59.727883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650474+0000","last_change":"2024-06-10T00:28:48.713267+0000","last_active":"2024-06-10T00:30:24.650474+0000","last_peered":"2024-06-10T00:30:24.650474+0000","last_clean":"2024-06-10T00:30:24.650474+0000","last_became_active":"2024-06-10T00:28:48.712870+0000","last_became_peered":"2024-06-10T00:28:48.712870+0000","last_unstale":"2024-06-10T00:30:24.650474+0000","last_undegraded":"2024-06-10T00:30:24.650474+0000","last_fullsized":"2024-06-10T00:30:24.650474+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:59.210101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'5","reported_seq":151,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:24.022115+0000","last_change":"2024-06-10T00:28:34.448652+0000","last_active":"2024-06-10T00:31:24.022115+0000","last_peered":"2024-06-10T00:31:24.022115+0000","last_clean":"2024-06-10T00:31:24.022115+0000","last_became_active":"2024-06-10T00:28:34.448243+0000","last_became_peered":"2024-06-10T00:28:34.448243+0000","last_unstale":"2024-06-10T00:31:24.022115+0000","last_undegraded":"2024-06-10T00:31:24.022115+0000","last_fullsized":"2024-06-10T00:31:24.022115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:51:40.003429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111841+0000","last_change":"2024-06-10T00:28:37.662307+0000","last_active":"2024-06-10T00:30:24.111841+0000","last_peered":"2024-06-10T00:30:24.111841+0000","last_clean":"2024-06-10T00:30:24.111841+0000","last_became_active":"2024-06-10T00:28:37.661727+0000","last_became_peered":"2024-06-10T00:28:37.661727+0000","last_unstale":"2024-06-10T00:30:24.111841+0000","last_undegraded":"2024-06-10T00:30:24.111841+0000","last_fullsized":"2024-06-10T00:30:24.111841+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:23.989429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.649668+0000","last_change":"2024-06-10T00:29:23.857626+0000","last_active":"2024-06-10T00:30:24.649668+0000","last_peered":"2024-06-10T00:30:24.649668+0000","last_clean":"2024-06-10T00:30:24.649668+0000","last_became_active":"2024-06-10T00:29:23.857057+0000","last_became_peered":"2024-06-10T00:29:23.857057+0000","last_unstale":"2024-06-10T00:30:24.649668+0000","last_undegraded":"2024-06-10T00:30:24.649668+0000","last_fullsized":"2024-06-10T00:30:24.649668+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:25:10.340053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.112465+0000","last_change":"2024-06-10T00:28:34.399253+0000","last_active":"2024-06-10T00:30:24.112465+0000","last_peered":"2024-06-10T00:30:24.112465+0000","last_clean":"2024-06-10T00:30:24.112465+0000","last_became_active":"2024-06-10T00:28:34.397763+0000","last_became_peered":"2024-06-10T00:28:34.397763+0000","last_unstale":"2024-06-10T00:30:24.112465+0000","last_undegraded":"2024-06-10T00:30:24.112465+0000","last_fullsized":"2024-06-10T00:30:24.112465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:05:57.371625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111289+0000","last_change":"2024-06-10T00:28:37.658027+0000","last_active":"2024-06-10T00:30:24.111289+0000","last_peered":"2024-06-10T00:30:24.111289+0000","last_clean":"2024-06-10T00:30:24.111289+0000","last_became_active":"2024-06-10T00:28:37.655513+0000","last_became_peered":"2024-06-10T00:28:37.655513+0000","last_unstale":"2024-06-10T00:30:24.111289+0000","last_undegraded":"2024-06-10T00:30:24.111289+0000","last_fullsized":"2024-06-10T00:30:24.111289+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:50:23.525855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111709+0000","last_change":"2024-06-10T00:28:48.706997+0000","last_active":"2024-06-10T00:30:24.111709+0000","last_peered":"2024-06-10T00:30:24.111709+0000","last_clean":"2024-06-10T00:30:24.111709+0000","last_became_active":"2024-06-10T00:28:48.706597+0000","last_became_peered":"2024-06-10T00:28:48.706597+0000","last_unstale":"2024-06-10T00:30:24.111709+0000","last_undegraded":"2024-06-10T00:30:24.111709+0000","last_fullsized":"2024-06-10T00:30:24.111709+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:16.385312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":59,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228755+0000","last_change":"2024-06-10T00:29:23.785774+0000","last_active":"2024-06-10T00:30:24.228755+0000","last_peered":"2024-06-10T00:30:24.228755+0000","last_clean":"2024-06-10T00:30:24.228755+0000","last_became_active":"2024-06-10T00:29:23.781768+0000","last_became_peered":"2024-06-10T00:29:23.781768+0000","last_unstale":"2024-06-10T00:30:24.228755+0000","last_undegraded":"2024-06-10T00:30:24.228755+0000","last_fullsized":"2024-06-10T00:30:24.228755+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:27.348264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111826+0000","last_change":"2024-06-10T00:28:37.657820+0000","last_active":"2024-06-10T00:30:24.111826+0000","last_peered":"2024-06-10T00:30:24.111826+0000","last_clean":"2024-06-10T00:30:24.111826+0000","last_became_active":"2024-06-10T00:28:37.657487+0000","last_became_peered":"2024-06-10T00:28:37.657487+0000","last_unstale":"2024-06-10T00:30:24.111826+0000","last_undegraded":"2024-06-10T00:30:24.111826+0000","last_fullsized":"2024-06-10T00:30:24.111826+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:44:35.930401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.967849+0000","last_change":"2024-06-10T00:28:37.664543+0000","last_active":"2024-06-10T00:31:21.967849+0000","last_peered":"2024-06-10T00:31:21.967849+0000","last_clean":"2024-06-10T00:31:21.967849+0000","last_became_active":"2024-06-10T00:28:37.659015+0000","last_became_peered":"2024-06-10T00:28:37.659015+0000","last_unstale":"2024-06-10T00:31:21.967849+0000","last_undegraded":"2024-06-10T00:31:21.967849+0000","last_fullsized":"2024-06-10T00:31:21.967849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:06.555472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.963605+0000","last_change":"2024-06-10T00:28:37.659389+0000","last_active":"2024-06-10T00:31:21.963605+0000","last_peered":"2024-06-10T00:31:21.963605+0000","last_clean":"2024-06-10T00:31:21.963605+0000","last_became_active":"2024-06-10T00:28:37.658990+0000","last_became_peered":"2024-06-10T00:28:37.658990+0000","last_unstale":"2024-06-10T00:31:21.963605+0000","last_undegraded":"2024-06-10T00:31:21.963605+0000","last_fullsized":"2024-06-10T00:31:21.963605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:29.532779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.973288+0000","last_change":"2024-06-10T00:28:37.655621+0000","last_active":"2024-06-10T00:31:21.973288+0000","last_peered":"2024-06-10T00:31:21.973288+0000","last_clean":"2024-06-10T00:31:21.973288+0000","last_became_active":"2024-06-10T00:28:37.653599+0000","last_became_peered":"2024-06-10T00:28:37.653599+0000","last_unstale":"2024-06-10T00:31:21.973288+0000","last_undegraded":"2024-06-10T00:31:21.973288+0000","last_fullsized":"2024-06-10T00:31:21.973288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:22.165851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.976702+0000","last_change":"2024-06-10T00:29:23.782727+0000","last_active":"2024-06-10T00:31:21.976702+0000","last_peered":"2024-06-10T00:31:21.976702+0000","last_clean":"2024-06-10T00:31:21.976702+0000","last_became_active":"2024-06-10T00:29:23.782309+0000","last_became_peered":"2024-06-10T00:29:23.782309+0000","last_unstale":"2024-06-10T00:31:21.976702+0000","last_undegraded":"2024-06-10T00:31:21.976702+0000","last_fullsized":"2024-06-10T00:31:21.976702+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:20.585955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228614+0000","last_change":"2024-06-10T00:28:37.646750+0000","last_active":"2024-06-10T00:30:24.228614+0000","last_peered":"2024-06-10T00:30:24.228614+0000","last_clean":"2024-06-10T00:30:24.228614+0000","last_became_active":"2024-06-10T00:28:37.646372+0000","last_became_peered":"2024-06-10T00:28:37.646372+0000","last_unstale":"2024-06-10T00:30:24.228614+0000","last_undegraded":"2024-06-10T00:30:24.228614+0000","last_fullsized":"2024-06-10T00:30:24.228614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:45.762956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.968092+0000","last_change":"2024-06-10T00:28:37.662278+0000","last_active":"2024-06-10T00:31:21.968092+0000","last_peered":"2024-06-10T00:31:21.968092+0000","last_clean":"2024-06-10T00:31:21.968092+0000","last_became_active":"2024-06-10T00:28:37.661855+0000","last_became_peered":"2024-06-10T00:28:37.661855+0000","last_unstale":"2024-06-10T00:31:21.968092+0000","last_undegraded":"2024-06-10T00:31:21.968092+0000","last_fullsized":"2024-06-10T00:31:21.968092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:20.074208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.962431+0000","last_change":"2024-06-10T00:28:37.656738+0000","last_active":"2024-06-10T00:31:21.962431+0000","last_peered":"2024-06-10T00:31:21.962431+0000","last_clean":"2024-06-10T00:31:21.962431+0000","last_became_active":"2024-06-10T00:28:37.656375+0000","last_became_peered":"2024-06-10T00:28:37.656375+0000","last_unstale":"2024-06-10T00:31:21.962431+0000","last_undegraded":"2024-06-10T00:31:21.962431+0000","last_fullsized":"2024-06-10T00:31:21.962431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:49:17.779670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.963881+0000","last_change":"2024-06-10T00:28:37.656849+0000","last_active":"2024-06-10T00:31:21.963881+0000","last_peered":"2024-06-10T00:31:21.963881+0000","last_clean":"2024-06-10T00:31:21.963881+0000","last_became_active":"2024-06-10T00:28:37.656261+0000","last_became_peered":"2024-06-10T00:28:37.656261+0000","last_unstale":"2024-06-10T00:31:21.963881+0000","last_undegraded":"2024-06-10T00:31:21.963881+0000","last_fullsized":"2024-06-10T00:31:21.963881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:13.152165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111998+0000","last_change":"2024-06-10T00:28:37.648384+0000","last_active":"2024-06-10T00:30:24.111998+0000","last_peered":"2024-06-10T00:30:24.111998+0000","last_clean":"2024-06-10T00:30:24.111998+0000","last_became_active":"2024-06-10T00:28:37.648009+0000","last_became_peered":"2024-06-10T00:28:37.648009+0000","last_unstale":"2024-06-10T00:30:24.111998+0000","last_undegraded":"2024-06-10T00:30:24.111998+0000","last_fullsized":"2024-06-10T00:30:24.111998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:45:55.808932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.964661+0000","last_change":"2024-06-10T00:28:37.655828+0000","last_active":"2024-06-10T00:31:21.964661+0000","last_peered":"2024-06-10T00:31:21.964661+0000","last_clean":"2024-06-10T00:31:21.964661+0000","last_became_active":"2024-06-10T00:28:37.655269+0000","last_became_peered":"2024-06-10T00:28:37.655269+0000","last_unstale":"2024-06-10T00:31:21.964661+0000","last_undegraded":"2024-06-10T00:31:21.964661+0000","last_fullsized":"2024-06-10T00:31:21.964661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:56.618469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.977532+0000","last_change":"2024-06-10T00:28:37.663756+0000","last_active":"2024-06-10T00:31:21.977532+0000","last_peered":"2024-06-10T00:31:21.977532+0000","last_clean":"2024-06-10T00:31:21.977532+0000","last_became_active":"2024-06-10T00:28:37.663380+0000","last_became_peered":"2024-06-10T00:28:37.663380+0000","last_unstale":"2024-06-10T00:31:21.977532+0000","last_undegraded":"2024-06-10T00:31:21.977532+0000","last_fullsized":"2024-06-10T00:31:21.977532+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:45.534648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228378+0000","last_change":"2024-06-10T00:28:37.655246+0000","last_active":"2024-06-10T00:30:24.228378+0000","last_peered":"2024-06-10T00:30:24.228378+0000","last_clean":"2024-06-10T00:30:24.228378+0000","last_became_active":"2024-06-10T00:28:37.654605+0000","last_became_peered":"2024-06-10T00:28:37.654605+0000","last_unstale":"2024-06-10T00:30:24.228378+0000","last_undegraded":"2024-06-10T00:30:24.228378+0000","last_fullsized":"2024-06-10T00:30:24.228378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:15.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.959365+0000","last_change":"2024-06-10T00:28:37.662485+0000","last_active":"2024-06-10T00:31:21.959365+0000","last_peered":"2024-06-10T00:31:21.959365+0000","last_clean":"2024-06-10T00:31:21.959365+0000","last_became_active":"2024-06-10T00:28:37.661900+0000","last_became_peered":"2024-06-10T00:28:37.661900+0000","last_unstale":"2024-06-10T00:31:21.959365+0000","last_undegraded":"2024-06-10T00:31:21.959365+0000","last_fullsized":"2024-06-10T00:31:21.959365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:30:03.210634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":358,"ondisk_log_size":358,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70247,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1017,"num_read_kb":587,"num_write":255,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":210741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383730,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32308,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712844,"statfs":{"total":95995035648,"available":95961952256,"internally_reserved":0,"allocated":1159168,"data_stored":570759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645366,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28816,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716336,"statfs":{"total":95995035648,"available":95965528064,"internally_reserved":0,"allocated":1843200,"data_stored":1220808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808891,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":574583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070529,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":570081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332165,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28748,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716404,"statfs":{"total":95995035648,"available":95965597696,"internally_reserved":0,"allocated":1765376,"data_stored":1167751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495690,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1146880,"data_stored":565991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36532,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708620,"statfs":{"total":95995035648,"available":95957626880,"internally_reserved":0,"allocated":1224704,"data_stored":630597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1171456,"data_stored":567447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36544,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708608,"statfs":{"total":95995035648,"available":95957614592,"internally_reserved":0,"allocated":1236992,"data_stored":630676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1159168,"data_stored":565699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920853,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1155072,"data_stored":570596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757327,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28748,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716404,"statfs":{"total":95995035648,"available":95965597696,"internally_reserved":0,"allocated":1769472,"data_stored":1162853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:31:26.841 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T00:31:27.094 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:27.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:27 smithi062 ceph-mon[32390]: pgmap v391: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-10T00:31:27.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:27 smithi062 ceph-mon[32390]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:31:28.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:27 smithi105 ceph-mon[36321]: pgmap v391: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-10T00:31:28.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:27 smithi105 ceph-mon[36321]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:31:28.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:27 smithi138 ceph-mon[36549]: pgmap v391: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-10T00:31:28.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:27 smithi138 ceph-mon[36549]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:31:28.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.869+0000 7f72a28ed700 1 -- 172.21.15.62:0/3810611219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f729c102f10 msgr2=0x7f729c10f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:28.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.869+0000 7f72a28ed700 1 --2- 172.21.15.62:0/3810611219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f729c102f10 0x7f729c10f330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f729400b0a0 tx=0x7f729400b3b0 comp rx=0 tx=0).stop 2024-06-10T00:31:28.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.869+0000 7f72a28ed700 1 -- 172.21.15.62:0/3810611219 shutdown_connections 2024-06-10T00:31:28.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.869+0000 7f72a28ed700 1 --2- 172.21.15.62:0/3810611219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f729c102f10 0x7f729c10f330 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:28.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.869+0000 7f72a28ed700 1 --2- 172.21.15.62:0/3810611219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f729c1084f0 0x7f729c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:28.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.869+0000 7f72a28ed700 1 --2- 172.21.15.62:0/3810611219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f729c102550 0x7f729c1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:28.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.869+0000 7f72a28ed700 1 -- 172.21.15.62:0/3810611219 >> 172.21.15.62:0/3810611219 conn(0x7f729c079650 msgr2=0x7f729c079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:28.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.869+0000 7f72a28ed700 1 -- 172.21.15.62:0/3810611219 shutdown_connections 2024-06-10T00:31:28.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.869+0000 7f72a28ed700 1 -- 172.21.15.62:0/3810611219 wait complete. 2024-06-10T00:31:28.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.869+0000 7f72a28ed700 1 Processor -- start 2024-06-10T00:31:28.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.869+0000 7f72a28ed700 1 -- start start 2024-06-10T00:31:28.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f72a28ed700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f729c102550 0x7f729c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:28.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f72a28ed700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f729c102f10 0x7f729c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:28.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f72a28ed700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f729c1084f0 0x7f729c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:28.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f72a28ed700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1127b0 con 0x7f729c102f10 2024-06-10T00:31:28.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f72a28ed700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f729c112630 con 0x7f729c102550 2024-06-10T00:31:28.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f72a28ed700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f729c112930 con 0x7f729c1084f0 2024-06-10T00:31:28.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f729b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f729c102f10 0x7f729c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:28.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f729b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f729c102f10 0x7f729c19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40038/0 (socket says 172.21.15.62:40038) 2024-06-10T00:31:28.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f729b7fe700 1 -- 172.21.15.62:0/1247354667 learned_addr learned my addr 172.21.15.62:0/1247354667 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:28.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f729bfff700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f729c102550 0x7f729c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:28.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f72a0e8a700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f729c1084f0 0x7f729c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:28.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f729b7fe700 1 -- 172.21.15.62:0/1247354667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f729c1084f0 msgr2=0x7f729c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:28.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f729b7fe700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f729c1084f0 0x7f729c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:28.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f729b7fe700 1 -- 172.21.15.62:0/1247354667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f729c102550 msgr2=0x7f729c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:28.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f729b7fe700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f729c102550 0x7f729c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:28.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f729b7fe700 1 -- 172.21.15.62:0/1247354667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c199150 con 0x7f729c102f10 2024-06-10T00:31:28.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f729bfff700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f729c102550 0x7f729c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:31:28.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.870+0000 7f729b7fe700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f729c102f10 0x7f729c19ed20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f729000d980 tx=0x7f729000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:28.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.871+0000 7f72997fa700 1 -- 172.21.15.62:0/1247354667 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290016070 con 0x7f729c102f10 2024-06-10T00:31:28.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.871+0000 7f72a28ed700 1 -- 172.21.15.62:0/1247354667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729c199440 con 0x7f729c102f10 2024-06-10T00:31:28.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.871+0000 7f72997fa700 1 -- 172.21.15.62:0/1247354667 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7290004d10 con 0x7f729c102f10 2024-06-10T00:31:28.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.871+0000 7f72997fa700 1 -- 172.21.15.62:0/1247354667 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72900055f0 con 0x7f729c102f10 2024-06-10T00:31:28.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.871+0000 7f72a28ed700 1 -- 172.21.15.62:0/1247354667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f729c1a56b0 con 0x7f729c102f10 2024-06-10T00:31:28.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.872+0000 7f72a28ed700 1 -- 172.21.15.62:0/1247354667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f729c069180 con 0x7f729c102f10 2024-06-10T00:31:28.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.873+0000 7f72997fa700 1 -- 172.21.15.62:0/1247354667 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7290004750 con 0x7f729c102f10 2024-06-10T00:31:28.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.874+0000 7f72997fa700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f72840a2c20 0x7f72840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:28.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.874+0000 7f729bfff700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f72840a2c20 0x7f72840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:28.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.874+0000 7f72997fa700 1 -- 172.21.15.62:0/1247354667 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f72900cf240 con 0x7f729c102f10 2024-06-10T00:31:28.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.874+0000 7f729bfff700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f72840a2c20 0x7f72840a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f728c005ea0 tx=0x7f728c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:28.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:28.878+0000 7f72997fa700 1 -- 172.21.15.62:0/1247354667 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f729001f020 con 0x7f729c102f10 2024-06-10T00:31:29.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:29.082+0000 7f72a28ed700 1 -- 172.21.15.62:0/1247354667 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f729c0628d0 con 0x7f72840a2c20 2024-06-10T00:31:29.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:29.092+0000 7f72997fa700 1 -- 172.21.15.62:0/1247354667 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340641 (secure 0 0 0) 0x7f729c0628d0 con 0x7f72840a2c20 2024-06-10T00:31:29.097 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:31:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:29.096+0000 7f72a28ed700 1 -- 172.21.15.62:0/1247354667 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f72840a2c20 msgr2=0x7f72840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:29.096+0000 7f72a28ed700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f72840a2c20 0x7f72840a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f728c005ea0 tx=0x7f728c005e30 comp rx=0 tx=0).stop 2024-06-10T00:31:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:29.096+0000 7f72a28ed700 1 -- 172.21.15.62:0/1247354667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f729c102f10 msgr2=0x7f729c19ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:29.096+0000 7f72a28ed700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f729c102f10 0x7f729c19ed20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f729000d980 tx=0x7f729000dc90 comp rx=0 tx=0).stop 2024-06-10T00:31:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:29.096+0000 7f72a28ed700 1 -- 172.21.15.62:0/1247354667 shutdown_connections 2024-06-10T00:31:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:29.096+0000 7f72a28ed700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f72840a2c20 0x7f72840a50e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:29.096+0000 7f72a28ed700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f729c1084f0 0x7f729c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:29.096+0000 7f72a28ed700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f729c102550 0x7f729c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:29.096+0000 7f72a28ed700 1 --2- 172.21.15.62:0/1247354667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f729c102f10 0x7f729c19ed20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:29.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:29.096+0000 7f72a28ed700 1 -- 172.21.15.62:0/1247354667 >> 172.21.15.62:0/1247354667 conn(0x7f729c079650 msgr2=0x7f729c100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:29.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:29.096+0000 7f72a28ed700 1 -- 172.21.15.62:0/1247354667 shutdown_connections 2024-06-10T00:31:29.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:29.096+0000 7f72a28ed700 1 -- 172.21.15.62:0/1247354667 wait complete. 2024-06-10T00:31:29.101 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-06-10T00:31:29.783 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-06-10T00:31:27.983303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660990,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1178,"num_read_kb":717,"num_write":440,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":860,"ondisk_log_size":860,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381376,"kb_used_data":15584,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560448,"statfs":{"total":1151940427776,"available":1151549898752,"internally_reserved":0,"allocated":15958016,"data_stored":8798264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":63,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003702"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.978168+0000","last_change":"2024-06-10T00:28:37.657234+0000","last_active":"2024-06-10T00:31:21.978168+0000","last_peered":"2024-06-10T00:31:21.978168+0000","last_clean":"2024-06-10T00:31:21.978168+0000","last_became_active":"2024-06-10T00:28:37.656879+0000","last_became_peered":"2024-06-10T00:28:37.656879+0000","last_unstale":"2024-06-10T00:31:21.978168+0000","last_undegraded":"2024-06-10T00:31:21.978168+0000","last_fullsized":"2024-06-10T00:31:21.978168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:01:02.907597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.973472+0000","last_change":"2024-06-10T00:28:37.656440+0000","last_active":"2024-06-10T00:31:21.973472+0000","last_peered":"2024-06-10T00:31:21.973472+0000","last_clean":"2024-06-10T00:31:21.973472+0000","last_became_active":"2024-06-10T00:28:37.656113+0000","last_became_peered":"2024-06-10T00:28:37.656113+0000","last_unstale":"2024-06-10T00:31:21.973472+0000","last_undegraded":"2024-06-10T00:31:21.973472+0000","last_fullsized":"2024-06-10T00:31:21.973472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:49:31.447951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.977252+0000","last_change":"2024-06-10T00:28:37.659114+0000","last_active":"2024-06-10T00:31:21.977252+0000","last_peered":"2024-06-10T00:31:21.977252+0000","last_clean":"2024-06-10T00:31:21.977252+0000","last_became_active":"2024-06-10T00:28:37.658763+0000","last_became_peered":"2024-06-10T00:28:37.658763+0000","last_unstale":"2024-06-10T00:31:21.977252+0000","last_undegraded":"2024-06-10T00:31:21.977252+0000","last_fullsized":"2024-06-10T00:31:21.977252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:39:54.963590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.961574+0000","last_change":"2024-06-10T00:28:37.656786+0000","last_active":"2024-06-10T00:31:21.961574+0000","last_peered":"2024-06-10T00:31:21.961574+0000","last_clean":"2024-06-10T00:31:21.961574+0000","last_became_active":"2024-06-10T00:28:37.656073+0000","last_became_peered":"2024-06-10T00:28:37.656073+0000","last_unstale":"2024-06-10T00:31:21.961574+0000","last_undegraded":"2024-06-10T00:31:21.961574+0000","last_fullsized":"2024-06-10T00:31:21.961574+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:19.176608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.960169+0000","last_change":"2024-06-10T00:28:37.650250+0000","last_active":"2024-06-10T00:31:21.960169+0000","last_peered":"2024-06-10T00:31:21.960169+0000","last_clean":"2024-06-10T00:31:21.960169+0000","last_became_active":"2024-06-10T00:28:37.649852+0000","last_became_peered":"2024-06-10T00:28:37.649852+0000","last_unstale":"2024-06-10T00:31:21.960169+0000","last_undegraded":"2024-06-10T00:31:21.960169+0000","last_fullsized":"2024-06-10T00:31:21.960169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:39.492884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.971523+0000","last_change":"2024-06-10T00:28:37.646042+0000","last_active":"2024-06-10T00:31:21.971523+0000","last_peered":"2024-06-10T00:31:21.971523+0000","last_clean":"2024-06-10T00:31:21.971523+0000","last_became_active":"2024-06-10T00:28:37.644591+0000","last_became_peered":"2024-06-10T00:28:37.644591+0000","last_unstale":"2024-06-10T00:31:21.971523+0000","last_undegraded":"2024-06-10T00:31:21.971523+0000","last_fullsized":"2024-06-10T00:31:21.971523+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:29.932303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.974438+0000","last_change":"2024-06-10T00:28:37.664076+0000","last_active":"2024-06-10T00:31:21.974438+0000","last_peered":"2024-06-10T00:31:21.974438+0000","last_clean":"2024-06-10T00:31:21.974438+0000","last_became_active":"2024-06-10T00:28:37.663741+0000","last_became_peered":"2024-06-10T00:28:37.663741+0000","last_unstale":"2024-06-10T00:31:21.974438+0000","last_undegraded":"2024-06-10T00:31:21.974438+0000","last_fullsized":"2024-06-10T00:31:21.974438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:14.581409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.969207+0000","last_change":"2024-06-10T00:28:37.648758+0000","last_active":"2024-06-10T00:31:21.969207+0000","last_peered":"2024-06-10T00:31:21.969207+0000","last_clean":"2024-06-10T00:31:21.969207+0000","last_became_active":"2024-06-10T00:28:37.648387+0000","last_became_peered":"2024-06-10T00:28:37.648387+0000","last_unstale":"2024-06-10T00:31:21.969207+0000","last_undegraded":"2024-06-10T00:31:21.969207+0000","last_fullsized":"2024-06-10T00:31:21.969207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:17.187265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.973868+0000","last_change":"2024-06-10T00:28:37.664650+0000","last_active":"2024-06-10T00:31:21.973868+0000","last_peered":"2024-06-10T00:31:21.973868+0000","last_clean":"2024-06-10T00:31:21.973868+0000","last_became_active":"2024-06-10T00:28:37.663178+0000","last_became_peered":"2024-06-10T00:28:37.663178+0000","last_unstale":"2024-06-10T00:31:21.973868+0000","last_undegraded":"2024-06-10T00:31:21.973868+0000","last_fullsized":"2024-06-10T00:31:21.973868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:46.032078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.970373+0000","last_change":"2024-06-10T00:28:37.658353+0000","last_active":"2024-06-10T00:31:21.970373+0000","last_peered":"2024-06-10T00:31:21.970373+0000","last_clean":"2024-06-10T00:31:21.970373+0000","last_became_active":"2024-06-10T00:28:37.657961+0000","last_became_peered":"2024-06-10T00:28:37.657961+0000","last_unstale":"2024-06-10T00:31:21.970373+0000","last_undegraded":"2024-06-10T00:31:21.970373+0000","last_fullsized":"2024-06-10T00:31:21.970373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:22:27.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.970817+0000","last_change":"2024-06-10T00:28:37.663823+0000","last_active":"2024-06-10T00:31:21.970817+0000","last_peered":"2024-06-10T00:31:21.970817+0000","last_clean":"2024-06-10T00:31:21.970817+0000","last_became_active":"2024-06-10T00:28:37.663499+0000","last_became_peered":"2024-06-10T00:28:37.663499+0000","last_unstale":"2024-06-10T00:31:21.970817+0000","last_undegraded":"2024-06-10T00:31:21.970817+0000","last_fullsized":"2024-06-10T00:31:21.970817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.100923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":18,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.968137+0000","last_change":"2024-06-10T00:30:24.112722+0000","last_active":"2024-06-10T00:31:21.968137+0000","last_peered":"2024-06-10T00:31:21.968137+0000","last_clean":"2024-06-10T00:31:21.968137+0000","last_became_active":"2024-06-10T00:30:24.112325+0000","last_became_peered":"2024-06-10T00:30:24.112325+0000","last_unstale":"2024-06-10T00:31:21.968137+0000","last_undegraded":"2024-06-10T00:31:21.968137+0000","last_fullsized":"2024-06-10T00:31:21.968137+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:26.671583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.969656+0000","last_change":"2024-06-10T00:28:37.649997+0000","last_active":"2024-06-10T00:31:21.969656+0000","last_peered":"2024-06-10T00:31:21.969656+0000","last_clean":"2024-06-10T00:31:21.969656+0000","last_became_active":"2024-06-10T00:28:37.649494+0000","last_became_peered":"2024-06-10T00:28:37.649494+0000","last_unstale":"2024-06-10T00:31:21.969656+0000","last_undegraded":"2024-06-10T00:31:21.969656+0000","last_fullsized":"2024-06-10T00:31:21.969656+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:44:07.467206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.976587+0000","last_change":"2024-06-10T00:28:37.653064+0000","last_active":"2024-06-10T00:31:21.976587+0000","last_peered":"2024-06-10T00:31:21.976587+0000","last_clean":"2024-06-10T00:31:21.976587+0000","last_became_active":"2024-06-10T00:28:37.652684+0000","last_became_peered":"2024-06-10T00:28:37.652684+0000","last_unstale":"2024-06-10T00:31:21.976587+0000","last_undegraded":"2024-06-10T00:31:21.976587+0000","last_fullsized":"2024-06-10T00:31:21.976587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:30.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.977758+0000","last_change":"2024-06-10T00:28:37.655570+0000","last_active":"2024-06-10T00:31:21.977758+0000","last_peered":"2024-06-10T00:31:21.977758+0000","last_clean":"2024-06-10T00:31:21.977758+0000","last_became_active":"2024-06-10T00:28:37.655241+0000","last_became_peered":"2024-06-10T00:28:37.655241+0000","last_unstale":"2024-06-10T00:31:21.977758+0000","last_undegraded":"2024-06-10T00:31:21.977758+0000","last_fullsized":"2024-06-10T00:31:21.977758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:07:03.543064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.977899+0000","last_change":"2024-06-10T00:28:37.665607+0000","last_active":"2024-06-10T00:31:21.977899+0000","last_peered":"2024-06-10T00:31:21.977899+0000","last_clean":"2024-06-10T00:31:21.977899+0000","last_became_active":"2024-06-10T00:28:37.664943+0000","last_became_peered":"2024-06-10T00:28:37.664943+0000","last_unstale":"2024-06-10T00:31:21.977899+0000","last_undegraded":"2024-06-10T00:31:21.977899+0000","last_fullsized":"2024-06-10T00:31:21.977899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:43:02.940298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.978293+0000","last_change":"2024-06-10T00:28:37.658214+0000","last_active":"2024-06-10T00:31:21.978293+0000","last_peered":"2024-06-10T00:31:21.978293+0000","last_clean":"2024-06-10T00:31:21.978293+0000","last_became_active":"2024-06-10T00:28:37.657295+0000","last_became_peered":"2024-06-10T00:28:37.657295+0000","last_unstale":"2024-06-10T00:31:21.978293+0000","last_undegraded":"2024-06-10T00:31:21.978293+0000","last_fullsized":"2024-06-10T00:31:21.978293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:20.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.979031+0000","last_change":"2024-06-10T00:28:37.656645+0000","last_active":"2024-06-10T00:31:21.979031+0000","last_peered":"2024-06-10T00:31:21.979031+0000","last_clean":"2024-06-10T00:31:21.979031+0000","last_became_active":"2024-06-10T00:28:37.656157+0000","last_became_peered":"2024-06-10T00:28:37.656157+0000","last_unstale":"2024-06-10T00:31:21.979031+0000","last_undegraded":"2024-06-10T00:31:21.979031+0000","last_fullsized":"2024-06-10T00:31:21.979031+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:00.953804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.968382+0000","last_change":"2024-06-10T00:28:37.664336+0000","last_active":"2024-06-10T00:31:21.968382+0000","last_peered":"2024-06-10T00:31:21.968382+0000","last_clean":"2024-06-10T00:31:21.968382+0000","last_became_active":"2024-06-10T00:28:37.663960+0000","last_became_peered":"2024-06-10T00:28:37.663960+0000","last_unstale":"2024-06-10T00:31:21.968382+0000","last_undegraded":"2024-06-10T00:31:21.968382+0000","last_fullsized":"2024-06-10T00:31:21.968382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:59:32.774442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650441+0000","last_change":"2024-06-10T00:28:48.705930+0000","last_active":"2024-06-10T00:30:24.650441+0000","last_peered":"2024-06-10T00:30:24.650441+0000","last_clean":"2024-06-10T00:30:24.650441+0000","last_became_active":"2024-06-10T00:28:48.705558+0000","last_became_peered":"2024-06-10T00:28:48.705558+0000","last_unstale":"2024-06-10T00:30:24.650441+0000","last_undegraded":"2024-06-10T00:30:24.650441+0000","last_fullsized":"2024-06-10T00:30:24.650441+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:41:44.914378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"97'13","reported_seq":173,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:26.024676+0000","last_change":"2024-06-10T00:29:23.862141+0000","last_active":"2024-06-10T00:31:26.024676+0000","last_peered":"2024-06-10T00:31:26.024676+0000","last_clean":"2024-06-10T00:31:26.024676+0000","last_became_active":"2024-06-10T00:29:23.857051+0000","last_became_peered":"2024-06-10T00:29:23.857051+0000","last_unstale":"2024-06-10T00:31:26.024676+0000","last_undegraded":"2024-06-10T00:31:26.024676+0000","last_fullsized":"2024-06-10T00:31:26.024676+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:41:41.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.968693+0000","last_change":"2024-06-10T00:28:37.653938+0000","last_active":"2024-06-10T00:31:21.968693+0000","last_peered":"2024-06-10T00:31:21.968693+0000","last_clean":"2024-06-10T00:31:21.968693+0000","last_became_active":"2024-06-10T00:28:37.653629+0000","last_became_peered":"2024-06-10T00:28:37.653629+0000","last_unstale":"2024-06-10T00:31:21.968693+0000","last_undegraded":"2024-06-10T00:31:21.968693+0000","last_fullsized":"2024-06-10T00:31:21.968693+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:21:39.285961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650408+0000","last_change":"2024-06-10T00:28:48.704413+0000","last_active":"2024-06-10T00:30:24.650408+0000","last_peered":"2024-06-10T00:30:24.650408+0000","last_clean":"2024-06-10T00:30:24.650408+0000","last_became_active":"2024-06-10T00:28:48.703972+0000","last_became_peered":"2024-06-10T00:28:48.703972+0000","last_unstale":"2024-06-10T00:30:24.650408+0000","last_undegraded":"2024-06-10T00:30:24.650408+0000","last_fullsized":"2024-06-10T00:30:24.650408+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:51.927435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.226323+0000","last_change":"2024-06-10T00:28:34.445413+0000","last_active":"2024-06-10T00:30:24.226323+0000","last_peered":"2024-06-10T00:30:24.226323+0000","last_clean":"2024-06-10T00:30:24.226323+0000","last_became_active":"2024-06-10T00:28:34.445077+0000","last_became_peered":"2024-06-10T00:28:34.445077+0000","last_unstale":"2024-06-10T00:30:24.226323+0000","last_undegraded":"2024-06-10T00:30:24.226323+0000","last_fullsized":"2024-06-10T00:30:24.226323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:19:19.535976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.976040+0000","last_change":"2024-06-10T00:28:37.657123+0000","last_active":"2024-06-10T00:31:21.976040+0000","last_peered":"2024-06-10T00:31:21.976040+0000","last_clean":"2024-06-10T00:31:21.976040+0000","last_became_active":"2024-06-10T00:28:37.656791+0000","last_became_peered":"2024-06-10T00:28:37.656791+0000","last_unstale":"2024-06-10T00:31:21.976040+0000","last_undegraded":"2024-06-10T00:31:21.976040+0000","last_fullsized":"2024-06-10T00:31:21.976040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:11.102478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.651391+0000","last_change":"2024-06-10T00:28:48.705989+0000","last_active":"2024-06-10T00:30:24.651391+0000","last_peered":"2024-06-10T00:30:24.651391+0000","last_clean":"2024-06-10T00:30:24.651391+0000","last_became_active":"2024-06-10T00:28:48.705555+0000","last_became_peered":"2024-06-10T00:28:48.705555+0000","last_unstale":"2024-06-10T00:30:24.651391+0000","last_undegraded":"2024-06-10T00:30:24.651391+0000","last_fullsized":"2024-06-10T00:30:24.651391+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:36.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"97'12","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.109393+0000","last_change":"2024-06-10T00:28:34.400117+0000","last_active":"2024-06-10T00:30:24.109393+0000","last_peered":"2024-06-10T00:30:24.109393+0000","last_clean":"2024-06-10T00:30:24.109393+0000","last_became_active":"2024-06-10T00:28:34.399743+0000","last_became_peered":"2024-06-10T00:28:34.399743+0000","last_unstale":"2024-06-10T00:30:24.109393+0000","last_undegraded":"2024-06-10T00:30:24.109393+0000","last_fullsized":"2024-06-10T00:30:24.109393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:20:57.580146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.978895+0000","last_change":"2024-06-10T00:28:37.664005+0000","last_active":"2024-06-10T00:31:21.978895+0000","last_peered":"2024-06-10T00:31:21.978895+0000","last_clean":"2024-06-10T00:31:21.978895+0000","last_became_active":"2024-06-10T00:28:37.661297+0000","last_became_peered":"2024-06-10T00:28:37.661297+0000","last_unstale":"2024-06-10T00:31:21.978895+0000","last_undegraded":"2024-06-10T00:31:21.978895+0000","last_fullsized":"2024-06-10T00:31:21.978895+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:57.795080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650379+0000","last_change":"2024-06-10T00:29:23.859091+0000","last_active":"2024-06-10T00:30:24.650379+0000","last_peered":"2024-06-10T00:30:24.650379+0000","last_clean":"2024-06-10T00:30:24.650379+0000","last_became_active":"2024-06-10T00:29:23.858485+0000","last_became_peered":"2024-06-10T00:29:23.858485+0000","last_unstale":"2024-06-10T00:30:24.650379+0000","last_undegraded":"2024-06-10T00:30:24.650379+0000","last_fullsized":"2024-06-10T00:30:24.650379+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:07:10.925323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.109290+0000","last_change":"2024-06-10T00:28:34.400588+0000","last_active":"2024-06-10T00:30:24.109290+0000","last_peered":"2024-06-10T00:30:24.109290+0000","last_clean":"2024-06-10T00:30:24.109290+0000","last_became_active":"2024-06-10T00:28:34.400027+0000","last_became_peered":"2024-06-10T00:28:34.400027+0000","last_unstale":"2024-06-10T00:30:24.109290+0000","last_undegraded":"2024-06-10T00:30:24.109290+0000","last_fullsized":"2024-06-10T00:30:24.109290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:25:59.890076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.995195+0000","last_change":"2024-06-10T00:28:37.659401+0000","last_active":"2024-06-10T00:31:21.995195+0000","last_peered":"2024-06-10T00:31:21.995195+0000","last_clean":"2024-06-10T00:31:21.995195+0000","last_became_active":"2024-06-10T00:28:37.659056+0000","last_became_peered":"2024-06-10T00:28:37.659056+0000","last_unstale":"2024-06-10T00:31:21.995195+0000","last_undegraded":"2024-06-10T00:31:21.995195+0000","last_fullsized":"2024-06-10T00:31:21.995195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:35:01.415630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650705+0000","last_change":"2024-06-10T00:28:48.702961+0000","last_active":"2024-06-10T00:30:24.650705+0000","last_peered":"2024-06-10T00:30:24.650705+0000","last_clean":"2024-06-10T00:30:24.650705+0000","last_became_active":"2024-06-10T00:28:48.702502+0000","last_became_peered":"2024-06-10T00:28:48.702502+0000","last_unstale":"2024-06-10T00:30:24.650705+0000","last_undegraded":"2024-06-10T00:30:24.650705+0000","last_fullsized":"2024-06-10T00:30:24.650705+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:54.881595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"99'7","reported_seq":155,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:24.022209+0000","last_change":"2024-06-10T00:28:34.396310+0000","last_active":"2024-06-10T00:31:24.022209+0000","last_peered":"2024-06-10T00:31:24.022209+0000","last_clean":"2024-06-10T00:31:24.022209+0000","last_became_active":"2024-06-10T00:28:34.395401+0000","last_became_peered":"2024-06-10T00:28:34.395401+0000","last_unstale":"2024-06-10T00:31:24.022209+0000","last_undegraded":"2024-06-10T00:31:24.022209+0000","last_fullsized":"2024-06-10T00:31:24.022209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:33:54.300308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.962137+0000","last_change":"2024-06-10T00:28:37.651427+0000","last_active":"2024-06-10T00:31:21.962137+0000","last_peered":"2024-06-10T00:31:21.962137+0000","last_clean":"2024-06-10T00:31:21.962137+0000","last_became_active":"2024-06-10T00:28:37.650510+0000","last_became_peered":"2024-06-10T00:28:37.650510+0000","last_unstale":"2024-06-10T00:31:21.962137+0000","last_undegraded":"2024-06-10T00:31:21.962137+0000","last_fullsized":"2024-06-10T00:31:21.962137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:42.953919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650743+0000","last_change":"2024-06-10T00:28:48.705918+0000","last_active":"2024-06-10T00:30:24.650743+0000","last_peered":"2024-06-10T00:30:24.650743+0000","last_clean":"2024-06-10T00:30:24.650743+0000","last_became_active":"2024-06-10T00:28:48.705462+0000","last_became_peered":"2024-06-10T00:28:48.705462+0000","last_unstale":"2024-06-10T00:30:24.650743+0000","last_undegraded":"2024-06-10T00:30:24.650743+0000","last_fullsized":"2024-06-10T00:30:24.650743+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:59:23.176511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111642+0000","last_change":"2024-06-10T00:28:34.393199+0000","last_active":"2024-06-10T00:30:24.111642+0000","last_peered":"2024-06-10T00:30:24.111642+0000","last_clean":"2024-06-10T00:30:24.111642+0000","last_became_active":"2024-06-10T00:28:34.392777+0000","last_became_peered":"2024-06-10T00:28:34.392777+0000","last_unstale":"2024-06-10T00:30:24.111642+0000","last_undegraded":"2024-06-10T00:30:24.111642+0000","last_fullsized":"2024-06-10T00:30:24.111642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:29.614242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.995069+0000","last_change":"2024-06-10T00:30:24.229721+0000","last_active":"2024-06-10T00:31:21.995069+0000","last_peered":"2024-06-10T00:31:21.995069+0000","last_clean":"2024-06-10T00:31:21.995069+0000","last_became_active":"2024-06-10T00:30:24.229407+0000","last_became_peered":"2024-06-10T00:30:24.229407+0000","last_unstale":"2024-06-10T00:31:21.995069+0000","last_undegraded":"2024-06-10T00:31:21.995069+0000","last_fullsized":"2024-06-10T00:31:21.995069+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:39.523197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.247459+0000","last_change":"2024-06-10T00:29:23.780971+0000","last_active":"2024-06-10T00:30:24.247459+0000","last_peered":"2024-06-10T00:30:24.247459+0000","last_clean":"2024-06-10T00:30:24.247459+0000","last_became_active":"2024-06-10T00:29:23.780034+0000","last_became_peered":"2024-06-10T00:29:23.780034+0000","last_unstale":"2024-06-10T00:30:24.247459+0000","last_undegraded":"2024-06-10T00:30:24.247459+0000","last_fullsized":"2024-06-10T00:30:24.247459+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:47:06.444181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228111+0000","last_change":"2024-06-10T00:28:34.392486+0000","last_active":"2024-06-10T00:30:24.228111+0000","last_peered":"2024-06-10T00:30:24.228111+0000","last_clean":"2024-06-10T00:30:24.228111+0000","last_became_active":"2024-06-10T00:28:34.392141+0000","last_became_peered":"2024-06-10T00:28:34.392141+0000","last_unstale":"2024-06-10T00:30:24.228111+0000","last_undegraded":"2024-06-10T00:30:24.228111+0000","last_fullsized":"2024-06-10T00:30:24.228111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:10:01.534195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.966446+0000","last_change":"2024-06-10T00:28:37.651033+0000","last_active":"2024-06-10T00:31:21.966446+0000","last_peered":"2024-06-10T00:31:21.966446+0000","last_clean":"2024-06-10T00:31:21.966446+0000","last_became_active":"2024-06-10T00:28:37.650505+0000","last_became_peered":"2024-06-10T00:28:37.650505+0000","last_unstale":"2024-06-10T00:31:21.966446+0000","last_undegraded":"2024-06-10T00:31:21.966446+0000","last_fullsized":"2024-06-10T00:31:21.966446+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:54.965865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650007+0000","last_change":"2024-06-10T00:28:48.710748+0000","last_active":"2024-06-10T00:30:24.650007+0000","last_peered":"2024-06-10T00:30:24.650007+0000","last_clean":"2024-06-10T00:30:24.650007+0000","last_became_active":"2024-06-10T00:28:48.710353+0000","last_became_peered":"2024-06-10T00:28:48.710353+0000","last_unstale":"2024-06-10T00:30:24.650007+0000","last_undegraded":"2024-06-10T00:30:24.650007+0000","last_fullsized":"2024-06-10T00:30:24.650007+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:30:47.546650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.112085+0000","last_change":"2024-06-10T00:28:34.398911+0000","last_active":"2024-06-10T00:30:24.112085+0000","last_peered":"2024-06-10T00:30:24.112085+0000","last_clean":"2024-06-10T00:30:24.112085+0000","last_became_active":"2024-06-10T00:28:34.398373+0000","last_became_peered":"2024-06-10T00:28:34.398373+0000","last_unstale":"2024-06-10T00:30:24.112085+0000","last_undegraded":"2024-06-10T00:30:24.112085+0000","last_fullsized":"2024-06-10T00:30:24.112085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:05:34.352825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.971261+0000","last_change":"2024-06-10T00:28:37.647492+0000","last_active":"2024-06-10T00:31:21.971261+0000","last_peered":"2024-06-10T00:31:21.971261+0000","last_clean":"2024-06-10T00:31:21.971261+0000","last_became_active":"2024-06-10T00:28:37.647103+0000","last_became_peered":"2024-06-10T00:28:37.647103+0000","last_unstale":"2024-06-10T00:31:21.971261+0000","last_undegraded":"2024-06-10T00:31:21.971261+0000","last_fullsized":"2024-06-10T00:31:21.971261+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:10.429256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.225577+0000","last_change":"2024-06-10T00:28:48.703095+0000","last_active":"2024-06-10T00:30:24.225577+0000","last_peered":"2024-06-10T00:30:24.225577+0000","last_clean":"2024-06-10T00:30:24.225577+0000","last_became_active":"2024-06-10T00:28:48.702687+0000","last_became_peered":"2024-06-10T00:28:48.702687+0000","last_unstale":"2024-06-10T00:30:24.225577+0000","last_undegraded":"2024-06-10T00:30:24.225577+0000","last_fullsized":"2024-06-10T00:30:24.225577+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:10:20.074173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"99'5","reported_seq":245,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:23.019819+0000","last_change":"2024-06-10T00:28:34.394807+0000","last_active":"2024-06-10T00:31:23.019819+0000","last_peered":"2024-06-10T00:31:23.019819+0000","last_clean":"2024-06-10T00:31:23.019819+0000","last_became_active":"2024-06-10T00:28:34.394022+0000","last_became_peered":"2024-06-10T00:28:34.394022+0000","last_unstale":"2024-06-10T00:31:23.019819+0000","last_undegraded":"2024-06-10T00:31:23.019819+0000","last_fullsized":"2024-06-10T00:31:23.019819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:21.626079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":196,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.978903+0000","last_change":"2024-06-10T00:28:37.662231+0000","last_active":"2024-06-10T00:31:21.978903+0000","last_peered":"2024-06-10T00:31:21.978903+0000","last_clean":"2024-06-10T00:31:21.978903+0000","last_became_active":"2024-06-10T00:28:37.661861+0000","last_became_peered":"2024-06-10T00:28:37.661861+0000","last_unstale":"2024-06-10T00:31:21.978903+0000","last_undegraded":"2024-06-10T00:31:21.978903+0000","last_fullsized":"2024-06-10T00:31:21.978903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:09.705349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.107608+0000","last_change":"2024-06-10T00:28:48.706643+0000","last_active":"2024-06-10T00:30:24.107608+0000","last_peered":"2024-06-10T00:30:24.107608+0000","last_clean":"2024-06-10T00:30:24.107608+0000","last_became_active":"2024-06-10T00:28:48.706127+0000","last_became_peered":"2024-06-10T00:28:48.706127+0000","last_unstale":"2024-06-10T00:30:24.107608+0000","last_undegraded":"2024-06-10T00:30:24.107608+0000","last_fullsized":"2024-06-10T00:30:24.107608+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:55.031303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111496+0000","last_change":"2024-06-10T00:28:34.392808+0000","last_active":"2024-06-10T00:30:24.111496+0000","last_peered":"2024-06-10T00:30:24.111496+0000","last_clean":"2024-06-10T00:30:24.111496+0000","last_became_active":"2024-06-10T00:28:34.392450+0000","last_became_peered":"2024-06-10T00:28:34.392450+0000","last_unstale":"2024-06-10T00:30:24.111496+0000","last_undegraded":"2024-06-10T00:30:24.111496+0000","last_fullsized":"2024-06-10T00:30:24.111496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:06:00.493316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.978033+0000","last_change":"2024-06-10T00:28:37.656099+0000","last_active":"2024-06-10T00:31:21.978033+0000","last_peered":"2024-06-10T00:31:21.978033+0000","last_clean":"2024-06-10T00:31:21.978033+0000","last_became_active":"2024-06-10T00:28:37.655640+0000","last_became_peered":"2024-06-10T00:28:37.655640+0000","last_unstale":"2024-06-10T00:31:21.978033+0000","last_undegraded":"2024-06-10T00:31:21.978033+0000","last_fullsized":"2024-06-10T00:31:21.978033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:59:26.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.651501+0000","last_change":"2024-06-10T00:28:48.702628+0000","last_active":"2024-06-10T00:30:24.651501+0000","last_peered":"2024-06-10T00:30:24.651501+0000","last_clean":"2024-06-10T00:30:24.651501+0000","last_became_active":"2024-06-10T00:28:48.702109+0000","last_became_peered":"2024-06-10T00:28:48.702109+0000","last_unstale":"2024-06-10T00:30:24.651501+0000","last_undegraded":"2024-06-10T00:30:24.651501+0000","last_fullsized":"2024-06-10T00:30:24.651501+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:25.262348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.112201+0000","last_change":"2024-06-10T00:28:34.446788+0000","last_active":"2024-06-10T00:30:24.112201+0000","last_peered":"2024-06-10T00:30:24.112201+0000","last_clean":"2024-06-10T00:30:24.112201+0000","last_became_active":"2024-06-10T00:28:34.446377+0000","last_became_peered":"2024-06-10T00:28:34.446377+0000","last_unstale":"2024-06-10T00:30:24.112201+0000","last_undegraded":"2024-06-10T00:30:24.112201+0000","last_fullsized":"2024-06-10T00:30:24.112201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:16.914244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.974064+0000","last_change":"2024-06-10T00:28:37.666159+0000","last_active":"2024-06-10T00:31:21.974064+0000","last_peered":"2024-06-10T00:31:21.974064+0000","last_clean":"2024-06-10T00:31:21.974064+0000","last_became_active":"2024-06-10T00:28:37.665711+0000","last_became_peered":"2024-06-10T00:28:37.665711+0000","last_unstale":"2024-06-10T00:31:21.974064+0000","last_undegraded":"2024-06-10T00:31:21.974064+0000","last_fullsized":"2024-06-10T00:31:21.974064+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:03.982375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.107717+0000","last_change":"2024-06-10T00:28:48.705071+0000","last_active":"2024-06-10T00:30:24.107717+0000","last_peered":"2024-06-10T00:30:24.107717+0000","last_clean":"2024-06-10T00:30:24.107717+0000","last_became_active":"2024-06-10T00:28:48.702211+0000","last_became_peered":"2024-06-10T00:28:48.702211+0000","last_unstale":"2024-06-10T00:30:24.107717+0000","last_undegraded":"2024-06-10T00:30:24.107717+0000","last_fullsized":"2024-06-10T00:30:24.107717+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:13.361985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"99'37","reported_seq":287,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:23.019803+0000","last_change":"2024-06-10T00:28:34.389784+0000","last_active":"2024-06-10T00:31:23.019803+0000","last_peered":"2024-06-10T00:31:23.019803+0000","last_clean":"2024-06-10T00:31:23.019803+0000","last_became_active":"2024-06-10T00:28:34.389416+0000","last_became_peered":"2024-06-10T00:28:34.389416+0000","last_unstale":"2024-06-10T00:31:23.019803+0000","last_undegraded":"2024-06-10T00:31:23.019803+0000","last_fullsized":"2024-06-10T00:31:23.019803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:23:05.803503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62904,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":206,"num_read_kb":165,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.958683+0000","last_change":"2024-06-10T00:28:37.656174+0000","last_active":"2024-06-10T00:31:21.958683+0000","last_peered":"2024-06-10T00:31:21.958683+0000","last_clean":"2024-06-10T00:31:21.958683+0000","last_became_active":"2024-06-10T00:28:37.655794+0000","last_became_peered":"2024-06-10T00:28:37.655794+0000","last_unstale":"2024-06-10T00:31:21.958683+0000","last_undegraded":"2024-06-10T00:31:21.958683+0000","last_fullsized":"2024-06-10T00:31:21.958683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:22:00.310309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.112614+0000","last_change":"2024-06-10T00:28:48.709915+0000","last_active":"2024-06-10T00:30:24.112614+0000","last_peered":"2024-06-10T00:30:24.112614+0000","last_clean":"2024-06-10T00:30:24.112614+0000","last_became_active":"2024-06-10T00:28:48.709426+0000","last_became_peered":"2024-06-10T00:28:48.709426+0000","last_unstale":"2024-06-10T00:30:24.112614+0000","last_undegraded":"2024-06-10T00:30:24.112614+0000","last_fullsized":"2024-06-10T00:30:24.112614+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:55.150617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650583+0000","last_change":"2024-06-10T00:28:34.389136+0000","last_active":"2024-06-10T00:30:24.650583+0000","last_peered":"2024-06-10T00:30:24.650583+0000","last_clean":"2024-06-10T00:30:24.650583+0000","last_became_active":"2024-06-10T00:28:34.388701+0000","last_became_peered":"2024-06-10T00:28:34.388701+0000","last_unstale":"2024-06-10T00:30:24.650583+0000","last_undegraded":"2024-06-10T00:30:24.650583+0000","last_fullsized":"2024-06-10T00:30:24.650583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:13:24.907824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.971592+0000","last_change":"2024-06-10T00:28:37.659048+0000","last_active":"2024-06-10T00:31:21.971592+0000","last_peered":"2024-06-10T00:31:21.971592+0000","last_clean":"2024-06-10T00:31:21.971592+0000","last_became_active":"2024-06-10T00:28:37.658710+0000","last_became_peered":"2024-06-10T00:28:37.658710+0000","last_unstale":"2024-06-10T00:31:21.971592+0000","last_undegraded":"2024-06-10T00:31:21.971592+0000","last_fullsized":"2024-06-10T00:31:21.971592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:44.314559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.247202+0000","last_change":"2024-06-10T00:28:48.709683+0000","last_active":"2024-06-10T00:30:24.247202+0000","last_peered":"2024-06-10T00:30:24.247202+0000","last_clean":"2024-06-10T00:30:24.247202+0000","last_became_active":"2024-06-10T00:28:48.709241+0000","last_became_peered":"2024-06-10T00:28:48.709241+0000","last_unstale":"2024-06-10T00:30:24.247202+0000","last_undegraded":"2024-06-10T00:30:24.247202+0000","last_fullsized":"2024-06-10T00:30:24.247202+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:25.190043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":21,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.247060+0000","last_change":"2024-06-10T00:29:23.782029+0000","last_active":"2024-06-10T00:30:24.247060+0000","last_peered":"2024-06-10T00:30:24.247060+0000","last_clean":"2024-06-10T00:30:24.247060+0000","last_became_active":"2024-06-10T00:29:23.781565+0000","last_became_peered":"2024-06-10T00:29:23.781565+0000","last_unstale":"2024-06-10T00:30:24.247060+0000","last_undegraded":"2024-06-10T00:30:24.247060+0000","last_fullsized":"2024-06-10T00:30:24.247060+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:53.258760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.973369+0000","last_change":"2024-06-10T00:28:37.653385+0000","last_active":"2024-06-10T00:31:21.973369+0000","last_peered":"2024-06-10T00:31:21.973369+0000","last_clean":"2024-06-10T00:31:21.973369+0000","last_became_active":"2024-06-10T00:28:37.653020+0000","last_became_peered":"2024-06-10T00:28:37.653020+0000","last_unstale":"2024-06-10T00:31:21.973369+0000","last_undegraded":"2024-06-10T00:31:21.973369+0000","last_fullsized":"2024-06-10T00:31:21.973369+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:57:54.469544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.108292+0000","last_change":"2024-06-10T00:28:48.707540+0000","last_active":"2024-06-10T00:30:24.108292+0000","last_peered":"2024-06-10T00:30:24.108292+0000","last_clean":"2024-06-10T00:30:24.108292+0000","last_became_active":"2024-06-10T00:28:48.707143+0000","last_became_peered":"2024-06-10T00:28:48.707143+0000","last_unstale":"2024-06-10T00:30:24.108292+0000","last_undegraded":"2024-06-10T00:30:24.108292+0000","last_fullsized":"2024-06-10T00:30:24.108292+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:20:29.344808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"99'7","reported_seq":158,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:26.024615+0000","last_change":"2024-06-10T00:28:34.397496+0000","last_active":"2024-06-10T00:31:26.024615+0000","last_peered":"2024-06-10T00:31:26.024615+0000","last_clean":"2024-06-10T00:31:26.024615+0000","last_became_active":"2024-06-10T00:28:34.397124+0000","last_became_peered":"2024-06-10T00:28:34.397124+0000","last_unstale":"2024-06-10T00:31:26.024615+0000","last_undegraded":"2024-06-10T00:31:26.024615+0000","last_fullsized":"2024-06-10T00:31:26.024615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:41:48.609354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.973689+0000","last_change":"2024-06-10T00:28:37.658338+0000","last_active":"2024-06-10T00:31:21.973689+0000","last_peered":"2024-06-10T00:31:21.973689+0000","last_clean":"2024-06-10T00:31:21.973689+0000","last_became_active":"2024-06-10T00:28:37.655910+0000","last_became_peered":"2024-06-10T00:28:37.655910+0000","last_unstale":"2024-06-10T00:31:21.973689+0000","last_undegraded":"2024-06-10T00:31:21.973689+0000","last_fullsized":"2024-06-10T00:31:21.973689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:38.501335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228758+0000","last_change":"2024-06-10T00:28:48.711151+0000","last_active":"2024-06-10T00:30:24.228758+0000","last_peered":"2024-06-10T00:30:24.228758+0000","last_clean":"2024-06-10T00:30:24.228758+0000","last_became_active":"2024-06-10T00:28:48.710768+0000","last_became_peered":"2024-06-10T00:28:48.710768+0000","last_unstale":"2024-06-10T00:30:24.228758+0000","last_undegraded":"2024-06-10T00:30:24.228758+0000","last_fullsized":"2024-06-10T00:30:24.228758+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:15:22.791839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":63,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.108802+0000","last_change":"2024-06-10T00:28:34.445812+0000","last_active":"2024-06-10T00:30:24.108802+0000","last_peered":"2024-06-10T00:30:24.108802+0000","last_clean":"2024-06-10T00:30:24.108802+0000","last_became_active":"2024-06-10T00:28:34.445472+0000","last_became_peered":"2024-06-10T00:28:34.445472+0000","last_unstale":"2024-06-10T00:30:24.108802+0000","last_undegraded":"2024-06-10T00:30:24.108802+0000","last_fullsized":"2024-06-10T00:30:24.108802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:00.912050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.969528+0000","last_change":"2024-06-10T00:28:37.655345+0000","last_active":"2024-06-10T00:31:21.969528+0000","last_peered":"2024-06-10T00:31:21.969528+0000","last_clean":"2024-06-10T00:31:21.969528+0000","last_became_active":"2024-06-10T00:28:37.654954+0000","last_became_peered":"2024-06-10T00:28:37.654954+0000","last_unstale":"2024-06-10T00:31:21.969528+0000","last_undegraded":"2024-06-10T00:31:21.969528+0000","last_fullsized":"2024-06-10T00:31:21.969528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:39:27.403006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111912+0000","last_change":"2024-06-10T00:28:48.705247+0000","last_active":"2024-06-10T00:30:24.111912+0000","last_peered":"2024-06-10T00:30:24.111912+0000","last_clean":"2024-06-10T00:30:24.111912+0000","last_became_active":"2024-06-10T00:28:48.704822+0000","last_became_peered":"2024-06-10T00:28:48.704822+0000","last_unstale":"2024-06-10T00:30:24.111912+0000","last_undegraded":"2024-06-10T00:30:24.111912+0000","last_fullsized":"2024-06-10T00:30:24.111912+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:24:59.991491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.109540+0000","last_change":"2024-06-10T00:28:34.396792+0000","last_active":"2024-06-10T00:30:24.109540+0000","last_peered":"2024-06-10T00:30:24.109540+0000","last_clean":"2024-06-10T00:30:24.109540+0000","last_became_active":"2024-06-10T00:28:34.396442+0000","last_became_peered":"2024-06-10T00:28:34.396442+0000","last_unstale":"2024-06-10T00:30:24.109540+0000","last_undegraded":"2024-06-10T00:30:24.109540+0000","last_fullsized":"2024-06-10T00:30:24.109540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:46.436198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.967706+0000","last_change":"2024-06-10T00:28:37.657698+0000","last_active":"2024-06-10T00:31:21.967706+0000","last_peered":"2024-06-10T00:31:21.967706+0000","last_clean":"2024-06-10T00:31:21.967706+0000","last_became_active":"2024-06-10T00:28:37.657364+0000","last_became_peered":"2024-06-10T00:28:37.657364+0000","last_unstale":"2024-06-10T00:31:21.967706+0000","last_undegraded":"2024-06-10T00:31:21.967706+0000","last_fullsized":"2024-06-10T00:31:21.967706+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:23.097015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.108917+0000","last_change":"2024-06-10T00:28:48.727567+0000","last_active":"2024-06-10T00:30:24.108917+0000","last_peered":"2024-06-10T00:30:24.108917+0000","last_clean":"2024-06-10T00:30:24.108917+0000","last_became_active":"2024-06-10T00:28:48.727108+0000","last_became_peered":"2024-06-10T00:28:48.727108+0000","last_unstale":"2024-06-10T00:30:24.108917+0000","last_undegraded":"2024-06-10T00:30:24.108917+0000","last_fullsized":"2024-06-10T00:30:24.108917+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:35:00.824177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.108030+0000","last_change":"2024-06-10T00:28:34.446832+0000","last_active":"2024-06-10T00:30:24.108030+0000","last_peered":"2024-06-10T00:30:24.108030+0000","last_clean":"2024-06-10T00:30:24.108030+0000","last_became_active":"2024-06-10T00:28:34.446435+0000","last_became_peered":"2024-06-10T00:28:34.446435+0000","last_unstale":"2024-06-10T00:30:24.108030+0000","last_undegraded":"2024-06-10T00:30:24.108030+0000","last_fullsized":"2024-06-10T00:30:24.108030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:58.296674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.971403+0000","last_change":"2024-06-10T00:30:24.229514+0000","last_active":"2024-06-10T00:31:21.971403+0000","last_peered":"2024-06-10T00:31:21.971403+0000","last_clean":"2024-06-10T00:31:21.971403+0000","last_became_active":"2024-06-10T00:30:24.229163+0000","last_became_peered":"2024-06-10T00:30:24.229163+0000","last_unstale":"2024-06-10T00:31:21.971403+0000","last_undegraded":"2024-06-10T00:31:21.971403+0000","last_fullsized":"2024-06-10T00:31:21.971403+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:52.914144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228507+0000","last_change":"2024-06-10T00:28:48.704683+0000","last_active":"2024-06-10T00:30:24.228507+0000","last_peered":"2024-06-10T00:30:24.228507+0000","last_clean":"2024-06-10T00:30:24.228507+0000","last_became_active":"2024-06-10T00:28:48.704297+0000","last_became_peered":"2024-06-10T00:28:48.704297+0000","last_unstale":"2024-06-10T00:30:24.228507+0000","last_undegraded":"2024-06-10T00:30:24.228507+0000","last_fullsized":"2024-06-10T00:30:24.228507+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:44.980544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"97'5","reported_seq":151,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:25.024120+0000","last_change":"2024-06-10T00:28:34.393328+0000","last_active":"2024-06-10T00:31:25.024120+0000","last_peered":"2024-06-10T00:31:25.024120+0000","last_clean":"2024-06-10T00:31:25.024120+0000","last_became_active":"2024-06-10T00:28:34.391968+0000","last_became_peered":"2024-06-10T00:28:34.391968+0000","last_unstale":"2024-06-10T00:31:25.024120+0000","last_undegraded":"2024-06-10T00:31:25.024120+0000","last_fullsized":"2024-06-10T00:31:25.024120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:18.898180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.969828+0000","last_change":"2024-06-10T00:28:37.652555+0000","last_active":"2024-06-10T00:31:21.969828+0000","last_peered":"2024-06-10T00:31:21.969828+0000","last_clean":"2024-06-10T00:31:21.969828+0000","last_became_active":"2024-06-10T00:28:37.651476+0000","last_became_peered":"2024-06-10T00:28:37.651476+0000","last_unstale":"2024-06-10T00:31:21.969828+0000","last_undegraded":"2024-06-10T00:31:21.969828+0000","last_fullsized":"2024-06-10T00:31:21.969828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:04:57.823224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228715+0000","last_change":"2024-06-10T00:28:48.727568+0000","last_active":"2024-06-10T00:30:24.228715+0000","last_peered":"2024-06-10T00:30:24.228715+0000","last_clean":"2024-06-10T00:30:24.228715+0000","last_became_active":"2024-06-10T00:28:48.727018+0000","last_became_peered":"2024-06-10T00:28:48.727018+0000","last_unstale":"2024-06-10T00:30:24.228715+0000","last_undegraded":"2024-06-10T00:30:24.228715+0000","last_fullsized":"2024-06-10T00:30:24.228715+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:02.025212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":19,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.958300+0000","last_change":"2024-06-10T00:30:24.651947+0000","last_active":"2024-06-10T00:31:21.958300+0000","last_peered":"2024-06-10T00:31:21.958300+0000","last_clean":"2024-06-10T00:31:21.958300+0000","last_became_active":"2024-06-10T00:30:24.651640+0000","last_became_peered":"2024-06-10T00:30:24.651640+0000","last_unstale":"2024-06-10T00:31:21.958300+0000","last_undegraded":"2024-06-10T00:31:21.958300+0000","last_fullsized":"2024-06-10T00:31:21.958300+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:47.070568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"99'6","reported_seq":152,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:25.022722+0000","last_change":"2024-06-10T00:28:34.445501+0000","last_active":"2024-06-10T00:31:25.022722+0000","last_peered":"2024-06-10T00:31:25.022722+0000","last_clean":"2024-06-10T00:31:25.022722+0000","last_became_active":"2024-06-10T00:28:34.445128+0000","last_became_peered":"2024-06-10T00:28:34.445128+0000","last_unstale":"2024-06-10T00:31:25.022722+0000","last_undegraded":"2024-06-10T00:31:25.022722+0000","last_fullsized":"2024-06-10T00:31:25.022722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:59.231585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111390+0000","last_change":"2024-06-10T00:28:48.709852+0000","last_active":"2024-06-10T00:30:24.111390+0000","last_peered":"2024-06-10T00:30:24.111390+0000","last_clean":"2024-06-10T00:30:24.111390+0000","last_became_active":"2024-06-10T00:28:48.709423+0000","last_became_peered":"2024-06-10T00:28:48.709423+0000","last_unstale":"2024-06-10T00:30:24.111390+0000","last_undegraded":"2024-06-10T00:30:24.111390+0000","last_fullsized":"2024-06-10T00:30:24.111390+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:42.123563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.969342+0000","last_change":"2024-06-10T00:28:37.657423+0000","last_active":"2024-06-10T00:31:21.969342+0000","last_peered":"2024-06-10T00:31:21.969342+0000","last_clean":"2024-06-10T00:31:21.969342+0000","last_became_active":"2024-06-10T00:28:37.657140+0000","last_became_peered":"2024-06-10T00:28:37.657140+0000","last_unstale":"2024-06-10T00:31:21.969342+0000","last_undegraded":"2024-06-10T00:31:21.969342+0000","last_fullsized":"2024-06-10T00:31:21.969342+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:20:08.943066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":54,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.651508+0000","last_change":"2024-06-10T00:28:34.392585+0000","last_active":"2024-06-10T00:30:24.651508+0000","last_peered":"2024-06-10T00:30:24.651508+0000","last_clean":"2024-06-10T00:30:24.651508+0000","last_became_active":"2024-06-10T00:28:34.392229+0000","last_became_peered":"2024-06-10T00:28:34.392229+0000","last_unstale":"2024-06-10T00:30:24.651508+0000","last_undegraded":"2024-06-10T00:30:24.651508+0000","last_fullsized":"2024-06-10T00:30:24.651508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:11.275761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":377,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.110595+0000","last_change":"2024-06-10T00:27:12.072577+0000","last_active":"2024-06-10T00:30:24.110595+0000","last_peered":"2024-06-10T00:30:24.110595+0000","last_clean":"2024-06-10T00:30:24.110595+0000","last_became_active":"2024-06-10T00:27:12.072193+0000","last_became_peered":"2024-06-10T00:27:12.072193+0000","last_unstale":"2024-06-10T00:30:24.110595+0000","last_undegraded":"2024-06-10T00:30:24.110595+0000","last_fullsized":"2024-06-10T00:30:24.110595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_clean_scrub_stamp":"2024-06-10T00:23:46.826438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:57.782124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.112299+0000","last_change":"2024-06-10T00:28:48.727913+0000","last_active":"2024-06-10T00:30:24.112299+0000","last_peered":"2024-06-10T00:30:24.112299+0000","last_clean":"2024-06-10T00:30:24.112299+0000","last_became_active":"2024-06-10T00:28:48.727402+0000","last_became_peered":"2024-06-10T00:28:48.727402+0000","last_unstale":"2024-06-10T00:30:24.112299+0000","last_undegraded":"2024-06-10T00:30:24.112299+0000","last_fullsized":"2024-06-10T00:30:24.112299+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:10:25.051296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.962356+0000","last_change":"2024-06-10T00:28:37.659901+0000","last_active":"2024-06-10T00:31:21.962356+0000","last_peered":"2024-06-10T00:31:21.962356+0000","last_clean":"2024-06-10T00:31:21.962356+0000","last_became_active":"2024-06-10T00:28:37.659553+0000","last_became_peered":"2024-06-10T00:28:37.659553+0000","last_unstale":"2024-06-10T00:31:21.962356+0000","last_undegraded":"2024-06-10T00:31:21.962356+0000","last_fullsized":"2024-06-10T00:31:21.962356+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:25.694823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111061+0000","last_change":"2024-06-10T00:28:34.398155+0000","last_active":"2024-06-10T00:30:24.111061+0000","last_peered":"2024-06-10T00:30:24.111061+0000","last_clean":"2024-06-10T00:30:24.111061+0000","last_became_active":"2024-06-10T00:28:34.397793+0000","last_became_peered":"2024-06-10T00:28:34.397793+0000","last_unstale":"2024-06-10T00:30:24.111061+0000","last_undegraded":"2024-06-10T00:30:24.111061+0000","last_fullsized":"2024-06-10T00:30:24.111061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:21.188139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.247169+0000","last_change":"2024-06-10T00:28:48.709736+0000","last_active":"2024-06-10T00:30:24.247169+0000","last_peered":"2024-06-10T00:30:24.247169+0000","last_clean":"2024-06-10T00:30:24.247169+0000","last_became_active":"2024-06-10T00:28:48.709293+0000","last_became_peered":"2024-06-10T00:28:48.709293+0000","last_unstale":"2024-06-10T00:30:24.247169+0000","last_undegraded":"2024-06-10T00:30:24.247169+0000","last_fullsized":"2024-06-10T00:30:24.247169+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:30:04.840656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":20,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.247174+0000","last_change":"2024-06-10T00:29:23.780030+0000","last_active":"2024-06-10T00:30:24.247174+0000","last_peered":"2024-06-10T00:30:24.247174+0000","last_clean":"2024-06-10T00:30:24.247174+0000","last_became_active":"2024-06-10T00:29:23.779589+0000","last_became_peered":"2024-06-10T00:29:23.779589+0000","last_unstale":"2024-06-10T00:30:24.247174+0000","last_undegraded":"2024-06-10T00:30:24.247174+0000","last_fullsized":"2024-06-10T00:30:24.247174+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:53.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.962338+0000","last_change":"2024-06-10T00:28:37.658330+0000","last_active":"2024-06-10T00:31:21.962338+0000","last_peered":"2024-06-10T00:31:21.962338+0000","last_clean":"2024-06-10T00:31:21.962338+0000","last_became_active":"2024-06-10T00:28:37.657343+0000","last_became_peered":"2024-06-10T00:28:37.657343+0000","last_unstale":"2024-06-10T00:31:21.962338+0000","last_undegraded":"2024-06-10T00:31:21.962338+0000","last_fullsized":"2024-06-10T00:31:21.962338+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:31:52.167402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.225501+0000","last_change":"2024-06-10T00:28:48.727506+0000","last_active":"2024-06-10T00:30:24.225501+0000","last_peered":"2024-06-10T00:30:24.225501+0000","last_clean":"2024-06-10T00:30:24.225501+0000","last_became_active":"2024-06-10T00:28:48.727068+0000","last_became_peered":"2024-06-10T00:28:48.727068+0000","last_unstale":"2024-06-10T00:30:24.225501+0000","last_undegraded":"2024-06-10T00:30:24.225501+0000","last_fullsized":"2024-06-10T00:30:24.225501+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:42.147908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"99'16","reported_seq":61,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:47.932155+0000","last_change":"2024-06-10T00:28:34.399872+0000","last_active":"2024-06-10T00:30:47.932155+0000","last_peered":"2024-06-10T00:30:47.932155+0000","last_clean":"2024-06-10T00:30:47.932155+0000","last_became_active":"2024-06-10T00:28:34.399493+0000","last_became_peered":"2024-06-10T00:28:34.399493+0000","last_unstale":"2024-06-10T00:30:47.932155+0000","last_undegraded":"2024-06-10T00:30:47.932155+0000","last_fullsized":"2024-06-10T00:30:47.932155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:24:10.018110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.970161+0000","last_change":"2024-06-10T00:28:37.656597+0000","last_active":"2024-06-10T00:31:21.970161+0000","last_peered":"2024-06-10T00:31:21.970161+0000","last_clean":"2024-06-10T00:31:21.970161+0000","last_became_active":"2024-06-10T00:28:37.654356+0000","last_became_peered":"2024-06-10T00:28:37.654356+0000","last_unstale":"2024-06-10T00:31:21.970161+0000","last_undegraded":"2024-06-10T00:31:21.970161+0000","last_fullsized":"2024-06-10T00:31:21.970161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:07:12.950683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650311+0000","last_change":"2024-06-10T00:28:48.702686+0000","last_active":"2024-06-10T00:30:24.650311+0000","last_peered":"2024-06-10T00:30:24.650311+0000","last_clean":"2024-06-10T00:30:24.650311+0000","last_became_active":"2024-06-10T00:28:48.702273+0000","last_became_peered":"2024-06-10T00:28:48.702273+0000","last_unstale":"2024-06-10T00:30:24.650311+0000","last_undegraded":"2024-06-10T00:30:24.650311+0000","last_fullsized":"2024-06-10T00:30:24.650311+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:00.078232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650742+0000","last_change":"2024-06-10T00:28:34.447549+0000","last_active":"2024-06-10T00:30:24.650742+0000","last_peered":"2024-06-10T00:30:24.650742+0000","last_clean":"2024-06-10T00:30:24.650742+0000","last_became_active":"2024-06-10T00:28:34.447197+0000","last_became_peered":"2024-06-10T00:28:34.447197+0000","last_unstale":"2024-06-10T00:30:24.650742+0000","last_undegraded":"2024-06-10T00:30:24.650742+0000","last_fullsized":"2024-06-10T00:30:24.650742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:47.525153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.977843+0000","last_change":"2024-06-10T00:28:37.655585+0000","last_active":"2024-06-10T00:31:21.977843+0000","last_peered":"2024-06-10T00:31:21.977843+0000","last_clean":"2024-06-10T00:31:21.977843+0000","last_became_active":"2024-06-10T00:28:37.655225+0000","last_became_peered":"2024-06-10T00:28:37.655225+0000","last_unstale":"2024-06-10T00:31:21.977843+0000","last_undegraded":"2024-06-10T00:31:21.977843+0000","last_fullsized":"2024-06-10T00:31:21.977843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:57:25.184924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650847+0000","last_change":"2024-06-10T00:29:23.857447+0000","last_active":"2024-06-10T00:30:24.650847+0000","last_peered":"2024-06-10T00:30:24.650847+0000","last_clean":"2024-06-10T00:30:24.650847+0000","last_became_active":"2024-06-10T00:29:23.856943+0000","last_became_peered":"2024-06-10T00:29:23.856943+0000","last_unstale":"2024-06-10T00:30:24.650847+0000","last_undegraded":"2024-06-10T00:30:24.650847+0000","last_fullsized":"2024-06-10T00:30:24.650847+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:58:44.548818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650239+0000","last_change":"2024-06-10T00:28:34.445637+0000","last_active":"2024-06-10T00:30:24.650239+0000","last_peered":"2024-06-10T00:30:24.650239+0000","last_clean":"2024-06-10T00:30:24.650239+0000","last_became_active":"2024-06-10T00:28:34.445324+0000","last_became_peered":"2024-06-10T00:28:34.445324+0000","last_unstale":"2024-06-10T00:30:24.650239+0000","last_undegraded":"2024-06-10T00:30:24.650239+0000","last_fullsized":"2024-06-10T00:30:24.650239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:05.713557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.927333+0000","last_change":"2024-06-10T00:28:37.659037+0000","last_active":"2024-06-10T00:31:21.927333+0000","last_peered":"2024-06-10T00:31:21.927333+0000","last_clean":"2024-06-10T00:31:21.927333+0000","last_became_active":"2024-06-10T00:28:37.658705+0000","last_became_peered":"2024-06-10T00:28:37.658705+0000","last_unstale":"2024-06-10T00:31:21.927333+0000","last_undegraded":"2024-06-10T00:31:21.927333+0000","last_fullsized":"2024-06-10T00:31:21.927333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:20:52.797237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.108350+0000","last_change":"2024-06-10T00:28:48.704130+0000","last_active":"2024-06-10T00:30:24.108350+0000","last_peered":"2024-06-10T00:30:24.108350+0000","last_clean":"2024-06-10T00:30:24.108350+0000","last_became_active":"2024-06-10T00:28:48.703738+0000","last_became_peered":"2024-06-10T00:28:48.703738+0000","last_unstale":"2024-06-10T00:30:24.108350+0000","last_undegraded":"2024-06-10T00:30:24.108350+0000","last_fullsized":"2024-06-10T00:30:24.108350+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:01:40.436710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.225578+0000","last_change":"2024-06-10T00:28:34.386667+0000","last_active":"2024-06-10T00:30:24.225578+0000","last_peered":"2024-06-10T00:30:24.225578+0000","last_clean":"2024-06-10T00:30:24.225578+0000","last_became_active":"2024-06-10T00:28:34.386363+0000","last_became_peered":"2024-06-10T00:28:34.386363+0000","last_unstale":"2024-06-10T00:30:24.225578+0000","last_undegraded":"2024-06-10T00:30:24.225578+0000","last_fullsized":"2024-06-10T00:30:24.225578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:11.850162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.974176+0000","last_change":"2024-06-10T00:28:37.664754+0000","last_active":"2024-06-10T00:31:21.974176+0000","last_peered":"2024-06-10T00:31:21.974176+0000","last_clean":"2024-06-10T00:31:21.974176+0000","last_became_active":"2024-06-10T00:28:37.658574+0000","last_became_peered":"2024-06-10T00:28:37.658574+0000","last_unstale":"2024-06-10T00:31:21.974176+0000","last_undegraded":"2024-06-10T00:31:21.974176+0000","last_fullsized":"2024-06-10T00:31:21.974176+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:16:47.437822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":30,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650338+0000","last_change":"2024-06-10T00:28:48.706560+0000","last_active":"2024-06-10T00:30:24.650338+0000","last_peered":"2024-06-10T00:30:24.650338+0000","last_clean":"2024-06-10T00:30:24.650338+0000","last_became_active":"2024-06-10T00:28:48.705986+0000","last_became_peered":"2024-06-10T00:28:48.705986+0000","last_unstale":"2024-06-10T00:30:24.650338+0000","last_undegraded":"2024-06-10T00:30:24.650338+0000","last_fullsized":"2024-06-10T00:30:24.650338+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:21:26.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"97'3","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650575+0000","last_change":"2024-06-10T00:28:34.394509+0000","last_active":"2024-06-10T00:30:24.650575+0000","last_peered":"2024-06-10T00:30:24.650575+0000","last_clean":"2024-06-10T00:30:24.650575+0000","last_became_active":"2024-06-10T00:28:34.394105+0000","last_became_peered":"2024-06-10T00:28:34.394105+0000","last_unstale":"2024-06-10T00:30:24.650575+0000","last_undegraded":"2024-06-10T00:30:24.650575+0000","last_fullsized":"2024-06-10T00:30:24.650575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:13.415544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.950432+0000","last_change":"2024-06-10T00:28:37.653192+0000","last_active":"2024-06-10T00:31:21.950432+0000","last_peered":"2024-06-10T00:31:21.950432+0000","last_clean":"2024-06-10T00:31:21.950432+0000","last_became_active":"2024-06-10T00:28:37.652868+0000","last_became_peered":"2024-06-10T00:28:37.652868+0000","last_unstale":"2024-06-10T00:31:21.950432+0000","last_undegraded":"2024-06-10T00:31:21.950432+0000","last_fullsized":"2024-06-10T00:31:21.950432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:21.614915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650624+0000","last_change":"2024-06-10T00:28:48.727615+0000","last_active":"2024-06-10T00:30:24.650624+0000","last_peered":"2024-06-10T00:30:24.650624+0000","last_clean":"2024-06-10T00:30:24.650624+0000","last_became_active":"2024-06-10T00:28:48.727190+0000","last_became_peered":"2024-06-10T00:28:48.727190+0000","last_unstale":"2024-06-10T00:30:24.650624+0000","last_undegraded":"2024-06-10T00:30:24.650624+0000","last_fullsized":"2024-06-10T00:30:24.650624+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:57.498183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":60,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.109216+0000","last_change":"2024-06-10T00:29:23.784131+0000","last_active":"2024-06-10T00:30:24.109216+0000","last_peered":"2024-06-10T00:30:24.109216+0000","last_clean":"2024-06-10T00:30:24.109216+0000","last_became_active":"2024-06-10T00:29:23.780909+0000","last_became_peered":"2024-06-10T00:29:23.780909+0000","last_unstale":"2024-06-10T00:30:24.109216+0000","last_undegraded":"2024-06-10T00:30:24.109216+0000","last_fullsized":"2024-06-10T00:30:24.109216+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.774217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":42,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.975256+0000","last_change":"2024-06-10T00:28:37.660399+0000","last_active":"2024-06-10T00:31:21.975256+0000","last_peered":"2024-06-10T00:31:21.975256+0000","last_clean":"2024-06-10T00:31:21.975256+0000","last_became_active":"2024-06-10T00:28:37.660069+0000","last_became_peered":"2024-06-10T00:28:37.660069+0000","last_unstale":"2024-06-10T00:31:21.975256+0000","last_undegraded":"2024-06-10T00:31:21.975256+0000","last_fullsized":"2024-06-10T00:31:21.975256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:59.727883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.650474+0000","last_change":"2024-06-10T00:28:48.713267+0000","last_active":"2024-06-10T00:30:24.650474+0000","last_peered":"2024-06-10T00:30:24.650474+0000","last_clean":"2024-06-10T00:30:24.650474+0000","last_became_active":"2024-06-10T00:28:48.712870+0000","last_became_peered":"2024-06-10T00:28:48.712870+0000","last_unstale":"2024-06-10T00:30:24.650474+0000","last_undegraded":"2024-06-10T00:30:24.650474+0000","last_fullsized":"2024-06-10T00:30:24.650474+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:59.210101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"97'5","reported_seq":151,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:24.022115+0000","last_change":"2024-06-10T00:28:34.448652+0000","last_active":"2024-06-10T00:31:24.022115+0000","last_peered":"2024-06-10T00:31:24.022115+0000","last_clean":"2024-06-10T00:31:24.022115+0000","last_became_active":"2024-06-10T00:28:34.448243+0000","last_became_peered":"2024-06-10T00:28:34.448243+0000","last_unstale":"2024-06-10T00:31:24.022115+0000","last_undegraded":"2024-06-10T00:31:24.022115+0000","last_fullsized":"2024-06-10T00:31:24.022115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:51:40.003429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.971162+0000","last_change":"2024-06-10T00:28:37.662307+0000","last_active":"2024-06-10T00:31:21.971162+0000","last_peered":"2024-06-10T00:31:21.971162+0000","last_clean":"2024-06-10T00:31:21.971162+0000","last_became_active":"2024-06-10T00:28:37.661727+0000","last_became_peered":"2024-06-10T00:28:37.661727+0000","last_unstale":"2024-06-10T00:31:21.971162+0000","last_undegraded":"2024-06-10T00:31:21.971162+0000","last_fullsized":"2024-06-10T00:31:21.971162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:23.989429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":36,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.649668+0000","last_change":"2024-06-10T00:29:23.857626+0000","last_active":"2024-06-10T00:30:24.649668+0000","last_peered":"2024-06-10T00:30:24.649668+0000","last_clean":"2024-06-10T00:30:24.649668+0000","last_became_active":"2024-06-10T00:29:23.857057+0000","last_became_peered":"2024-06-10T00:29:23.857057+0000","last_unstale":"2024-06-10T00:30:24.649668+0000","last_undegraded":"2024-06-10T00:30:24.649668+0000","last_fullsized":"2024-06-10T00:30:24.649668+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:25:10.340053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"97'3","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.112465+0000","last_change":"2024-06-10T00:28:34.399253+0000","last_active":"2024-06-10T00:30:24.112465+0000","last_peered":"2024-06-10T00:30:24.112465+0000","last_clean":"2024-06-10T00:30:24.112465+0000","last_became_active":"2024-06-10T00:28:34.397763+0000","last_became_peered":"2024-06-10T00:28:34.397763+0000","last_unstale":"2024-06-10T00:30:24.112465+0000","last_undegraded":"2024-06-10T00:30:24.112465+0000","last_fullsized":"2024-06-10T00:30:24.112465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:05:57.371625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.967303+0000","last_change":"2024-06-10T00:28:37.658027+0000","last_active":"2024-06-10T00:31:21.967303+0000","last_peered":"2024-06-10T00:31:21.967303+0000","last_clean":"2024-06-10T00:31:21.967303+0000","last_became_active":"2024-06-10T00:28:37.655513+0000","last_became_peered":"2024-06-10T00:28:37.655513+0000","last_unstale":"2024-06-10T00:31:21.967303+0000","last_undegraded":"2024-06-10T00:31:21.967303+0000","last_fullsized":"2024-06-10T00:31:21.967303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:50:23.525855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.111709+0000","last_change":"2024-06-10T00:28:48.706997+0000","last_active":"2024-06-10T00:30:24.111709+0000","last_peered":"2024-06-10T00:30:24.111709+0000","last_clean":"2024-06-10T00:30:24.111709+0000","last_became_active":"2024-06-10T00:28:48.706597+0000","last_became_peered":"2024-06-10T00:28:48.706597+0000","last_unstale":"2024-06-10T00:30:24.111709+0000","last_undegraded":"2024-06-10T00:30:24.111709+0000","last_fullsized":"2024-06-10T00:30:24.111709+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:16.385312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":59,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:30:24.228755+0000","last_change":"2024-06-10T00:29:23.785774+0000","last_active":"2024-06-10T00:30:24.228755+0000","last_peered":"2024-06-10T00:30:24.228755+0000","last_clean":"2024-06-10T00:30:24.228755+0000","last_became_active":"2024-06-10T00:29:23.781768+0000","last_became_peered":"2024-06-10T00:29:23.781768+0000","last_unstale":"2024-06-10T00:30:24.228755+0000","last_undegraded":"2024-06-10T00:30:24.228755+0000","last_fullsized":"2024-06-10T00:30:24.228755+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:27.348264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":46,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.966005+0000","last_change":"2024-06-10T00:28:37.657820+0000","last_active":"2024-06-10T00:31:21.966005+0000","last_peered":"2024-06-10T00:31:21.966005+0000","last_clean":"2024-06-10T00:31:21.966005+0000","last_became_active":"2024-06-10T00:28:37.657487+0000","last_became_peered":"2024-06-10T00:28:37.657487+0000","last_unstale":"2024-06-10T00:31:21.966005+0000","last_undegraded":"2024-06-10T00:31:21.966005+0000","last_fullsized":"2024-06-10T00:31:21.966005+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:44:35.930401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.967849+0000","last_change":"2024-06-10T00:28:37.664543+0000","last_active":"2024-06-10T00:31:21.967849+0000","last_peered":"2024-06-10T00:31:21.967849+0000","last_clean":"2024-06-10T00:31:21.967849+0000","last_became_active":"2024-06-10T00:28:37.659015+0000","last_became_peered":"2024-06-10T00:28:37.659015+0000","last_unstale":"2024-06-10T00:31:21.967849+0000","last_undegraded":"2024-06-10T00:31:21.967849+0000","last_fullsized":"2024-06-10T00:31:21.967849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:06.555472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.963605+0000","last_change":"2024-06-10T00:28:37.659389+0000","last_active":"2024-06-10T00:31:21.963605+0000","last_peered":"2024-06-10T00:31:21.963605+0000","last_clean":"2024-06-10T00:31:21.963605+0000","last_became_active":"2024-06-10T00:28:37.658990+0000","last_became_peered":"2024-06-10T00:28:37.658990+0000","last_unstale":"2024-06-10T00:31:21.963605+0000","last_undegraded":"2024-06-10T00:31:21.963605+0000","last_fullsized":"2024-06-10T00:31:21.963605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:29.532779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.973288+0000","last_change":"2024-06-10T00:28:37.655621+0000","last_active":"2024-06-10T00:31:21.973288+0000","last_peered":"2024-06-10T00:31:21.973288+0000","last_clean":"2024-06-10T00:31:21.973288+0000","last_became_active":"2024-06-10T00:28:37.653599+0000","last_became_peered":"2024-06-10T00:28:37.653599+0000","last_unstale":"2024-06-10T00:31:21.973288+0000","last_undegraded":"2024-06-10T00:31:21.973288+0000","last_fullsized":"2024-06-10T00:31:21.973288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:22.165851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.976702+0000","last_change":"2024-06-10T00:29:23.782727+0000","last_active":"2024-06-10T00:31:21.976702+0000","last_peered":"2024-06-10T00:31:21.976702+0000","last_clean":"2024-06-10T00:31:21.976702+0000","last_became_active":"2024-06-10T00:29:23.782309+0000","last_became_peered":"2024-06-10T00:29:23.782309+0000","last_unstale":"2024-06-10T00:31:21.976702+0000","last_undegraded":"2024-06-10T00:31:21.976702+0000","last_fullsized":"2024-06-10T00:31:21.976702+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:20.585955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":41,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.970575+0000","last_change":"2024-06-10T00:28:37.646750+0000","last_active":"2024-06-10T00:31:21.970575+0000","last_peered":"2024-06-10T00:31:21.970575+0000","last_clean":"2024-06-10T00:31:21.970575+0000","last_became_active":"2024-06-10T00:28:37.646372+0000","last_became_peered":"2024-06-10T00:28:37.646372+0000","last_unstale":"2024-06-10T00:31:21.970575+0000","last_undegraded":"2024-06-10T00:31:21.970575+0000","last_fullsized":"2024-06-10T00:31:21.970575+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:45.762956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.968092+0000","last_change":"2024-06-10T00:28:37.662278+0000","last_active":"2024-06-10T00:31:21.968092+0000","last_peered":"2024-06-10T00:31:21.968092+0000","last_clean":"2024-06-10T00:31:21.968092+0000","last_became_active":"2024-06-10T00:28:37.661855+0000","last_became_peered":"2024-06-10T00:28:37.661855+0000","last_unstale":"2024-06-10T00:31:21.968092+0000","last_undegraded":"2024-06-10T00:31:21.968092+0000","last_fullsized":"2024-06-10T00:31:21.968092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:20.074208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.962431+0000","last_change":"2024-06-10T00:28:37.656738+0000","last_active":"2024-06-10T00:31:21.962431+0000","last_peered":"2024-06-10T00:31:21.962431+0000","last_clean":"2024-06-10T00:31:21.962431+0000","last_became_active":"2024-06-10T00:28:37.656375+0000","last_became_peered":"2024-06-10T00:28:37.656375+0000","last_unstale":"2024-06-10T00:31:21.962431+0000","last_undegraded":"2024-06-10T00:31:21.962431+0000","last_fullsized":"2024-06-10T00:31:21.962431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:49:17.779670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":39,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.963881+0000","last_change":"2024-06-10T00:28:37.656849+0000","last_active":"2024-06-10T00:31:21.963881+0000","last_peered":"2024-06-10T00:31:21.963881+0000","last_clean":"2024-06-10T00:31:21.963881+0000","last_became_active":"2024-06-10T00:28:37.656261+0000","last_became_peered":"2024-06-10T00:28:37.656261+0000","last_unstale":"2024-06-10T00:31:21.963881+0000","last_undegraded":"2024-06-10T00:31:21.963881+0000","last_fullsized":"2024-06-10T00:31:21.963881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:13.152165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":43,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.994685+0000","last_change":"2024-06-10T00:28:37.648384+0000","last_active":"2024-06-10T00:31:21.994685+0000","last_peered":"2024-06-10T00:31:21.994685+0000","last_clean":"2024-06-10T00:31:21.994685+0000","last_became_active":"2024-06-10T00:28:37.648009+0000","last_became_peered":"2024-06-10T00:28:37.648009+0000","last_unstale":"2024-06-10T00:31:21.994685+0000","last_undegraded":"2024-06-10T00:31:21.994685+0000","last_fullsized":"2024-06-10T00:31:21.994685+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:45:55.808932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":45,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.964661+0000","last_change":"2024-06-10T00:28:37.655828+0000","last_active":"2024-06-10T00:31:21.964661+0000","last_peered":"2024-06-10T00:31:21.964661+0000","last_clean":"2024-06-10T00:31:21.964661+0000","last_became_active":"2024-06-10T00:28:37.655269+0000","last_became_peered":"2024-06-10T00:28:37.655269+0000","last_unstale":"2024-06-10T00:31:21.964661+0000","last_undegraded":"2024-06-10T00:31:21.964661+0000","last_fullsized":"2024-06-10T00:31:21.964661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:56.618469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.977532+0000","last_change":"2024-06-10T00:28:37.663756+0000","last_active":"2024-06-10T00:31:21.977532+0000","last_peered":"2024-06-10T00:31:21.977532+0000","last_clean":"2024-06-10T00:31:21.977532+0000","last_became_active":"2024-06-10T00:28:37.663380+0000","last_became_peered":"2024-06-10T00:28:37.663380+0000","last_unstale":"2024-06-10T00:31:21.977532+0000","last_undegraded":"2024-06-10T00:31:21.977532+0000","last_fullsized":"2024-06-10T00:31:21.977532+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:45.534648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":48,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.972204+0000","last_change":"2024-06-10T00:28:37.655246+0000","last_active":"2024-06-10T00:31:21.972204+0000","last_peered":"2024-06-10T00:31:21.972204+0000","last_clean":"2024-06-10T00:31:21.972204+0000","last_became_active":"2024-06-10T00:28:37.654605+0000","last_became_peered":"2024-06-10T00:28:37.654605+0000","last_unstale":"2024-06-10T00:31:21.972204+0000","last_undegraded":"2024-06-10T00:31:21.972204+0000","last_fullsized":"2024-06-10T00:31:21.972204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:15.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":47,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-06-10T00:31:21.959365+0000","last_change":"2024-06-10T00:28:37.662485+0000","last_active":"2024-06-10T00:31:21.959365+0000","last_peered":"2024-06-10T00:31:21.959365+0000","last_clean":"2024-06-10T00:31:21.959365+0000","last_became_active":"2024-06-10T00:28:37.661900+0000","last_became_peered":"2024-06-10T00:28:37.661900+0000","last_unstale":"2024-06-10T00:31:21.959365+0000","last_undegraded":"2024-06-10T00:31:21.959365+0000","last_fullsized":"2024-06-10T00:31:21.959365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:30:03.210634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70388,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1042,"num_read_kb":598,"num_write":257,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383730,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32308,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712844,"statfs":{"total":95995035648,"available":95961952256,"internally_reserved":0,"allocated":1159168,"data_stored":570759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645366,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28816,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716336,"statfs":{"total":95995035648,"available":95965528064,"internally_reserved":0,"allocated":1843200,"data_stored":1220808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808892,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":574583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070530,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":570081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332166,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28740,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716412,"statfs":{"total":95995035648,"available":95965605888,"internally_reserved":0,"allocated":1765376,"data_stored":1168033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495691,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":565991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36532,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708620,"statfs":{"total":95995035648,"available":95957626880,"internally_reserved":0,"allocated":1224704,"data_stored":630597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1171456,"data_stored":567447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36544,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708608,"statfs":{"total":95995035648,"available":95957614592,"internally_reserved":0,"allocated":1236992,"data_stored":630676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":565699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920854,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1155072,"data_stored":570596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757328,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28744,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716408,"statfs":{"total":95995035648,"available":95965601792,"internally_reserved":0,"allocated":1769472,"data_stored":1162994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:31:29.790 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-10T00:31:29.790 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-10T00:31:29.790 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-10T00:31:29.790 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-10T00:31:30.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:29 smithi105 ceph-mon[36321]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-06-10T00:31:30.043 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:30.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:29 smithi062 ceph-mon[32390]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-06-10T00:31:30.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:29 smithi138 ceph-mon[36549]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-06-10T00:31:30.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:30 smithi062 ceph-mon[32390]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:31:31.025 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:30 smithi138 ceph-mon[36549]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:31:31.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:30 smithi105 ceph-mon[36321]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:31:31.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.801+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2363536749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb8102740 msgr2=0x7f8fb8102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:31.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.801+0000 7f8fbdcd2700 1 --2- 172.21.15.62:0/2363536749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb8102740 0x7f8fb8102b20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80099c0 tx=0x7f8fa8009cd0 comp rx=0 tx=0).stop 2024-06-10T00:31:31.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.801+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2363536749 shutdown_connections 2024-06-10T00:31:31.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.801+0000 7f8fbdcd2700 1 --2- 172.21.15.62:0/2363536749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb8102740 0x7f8fb8102b20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:31.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.801+0000 7f8fbdcd2700 1 --2- 172.21.15.62:0/2363536749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fb8107840 0x7f8fb810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:31.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.801+0000 7f8fbdcd2700 1 --2- 172.21.15.62:0/2363536749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fb81030f0 0x7f8fb8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:31.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.801+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2363536749 >> 172.21.15.62:0/2363536749 conn(0x7f8fb80795f0 msgr2=0x7f8fb8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:31.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.801+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2363536749 shutdown_connections 2024-06-10T00:31:31.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.801+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2363536749 wait complete. 2024-06-10T00:31:31.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.801+0000 7f8fbdcd2700 1 Processor -- start 2024-06-10T00:31:31.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fbdcd2700 1 -- start start 2024-06-10T00:31:31.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fbdcd2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fb8102740 0x7f8fb8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:31.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fbdcd2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fb81030f0 0x7f8fb8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:31.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fbdcd2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb8107840 0x7f8fb8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:31.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fbdcd2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8112760 con 0x7f8fb8107840 2024-06-10T00:31:31.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fbdcd2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb81125e0 con 0x7f8fb81030f0 2024-06-10T00:31:31.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fbdcd2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb81128e0 con 0x7f8fb8102740 2024-06-10T00:31:31.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fb7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb8107840 0x7f8fb8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:31.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fb7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb8107840 0x7f8fb8075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40064/0 (socket says 172.21.15.62:40064) 2024-06-10T00:31:31.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fb7fff700 1 -- 172.21.15.62:0/2059860395 learned_addr learned my addr 172.21.15.62:0/2059860395 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:31.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fb77fe700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fb8102740 0x7f8fb8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:31.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8faeffd700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fb81030f0 0x7f8fb8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:31.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fb7fff700 1 -- 172.21.15.62:0/2059860395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fb8102740 msgr2=0x7f8fb8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:31.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fb7fff700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fb8102740 0x7f8fb8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:31.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fb7fff700 1 -- 172.21.15.62:0/2059860395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fb81030f0 msgr2=0x7f8fb8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:31.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fb7fff700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fb81030f0 0x7f8fb8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:31.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8fb7fff700 1 -- 172.21.15.62:0/2059860395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb81a7130 con 0x7f8fb8107840 2024-06-10T00:31:31.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.802+0000 7f8faeffd700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fb81030f0 0x7f8fb8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:31:31.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.803+0000 7f8fb7fff700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb8107840 0x7f8fb8075210 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0007ee0 tx=0x7f8fb000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:31.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.803+0000 7f8fb57fa700 1 -- 172.21.15.62:0/2059860395 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb000ddd0 con 0x7f8fb8107840 2024-06-10T00:31:31.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.803+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2059860395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb81a7420 con 0x7f8fb8107840 2024-06-10T00:31:31.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.803+0000 7f8fb57fa700 1 -- 172.21.15.62:0/2059860395 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb0004020 con 0x7f8fb8107840 2024-06-10T00:31:31.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.803+0000 7f8fb57fa700 1 -- 172.21.15.62:0/2059860395 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0015660 con 0x7f8fb8107840 2024-06-10T00:31:31.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.803+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2059860395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb81a7960 con 0x7f8fb8107840 2024-06-10T00:31:31.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.804+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2059860395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb810aa80 con 0x7f8fb8107840 2024-06-10T00:31:31.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.805+0000 7f8fb57fa700 1 -- 172.21.15.62:0/2059860395 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8fb00041d0 con 0x7f8fb8107840 2024-06-10T00:31:31.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.806+0000 7f8fb57fa700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8fa40a2c20 0x7f8fa40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:31.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.806+0000 7f8fb77fe700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8fa40a2c20 0x7f8fa40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:31.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.806+0000 7f8fb57fa700 1 -- 172.21.15.62:0/2059860395 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f8fb00cedd0 con 0x7f8fb8107840 2024-06-10T00:31:31.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.806+0000 7f8fb77fe700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8fa40a2c20 0x7f8fa40a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8000c00 tx=0x7f8fa8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:31.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:31.810+0000 7f8fb57fa700 1 -- 172.21.15.62:0/2059860395 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8fb0010070 con 0x7f8fb8107840 2024-06-10T00:31:32.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:31 smithi105 ceph-mon[36321]: pgmap v393: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-10T00:31:32.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:32.067+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2059860395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8fb810a460 con 0x7f8fb8107840 2024-06-10T00:31:32.067 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:31 smithi062 ceph-mon[32390]: pgmap v393: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-10T00:31:32.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:32.067+0000 7f8fb57fa700 1 -- 172.21.15.62:0/2059860395 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8fb009b960 con 0x7f8fb8107840 2024-06-10T00:31:32.068 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:31:32.068 INFO:teuthology.orchestra.run.smithi062.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-10T00:31:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:32.069+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2059860395 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8fa40a2c20 msgr2=0x7f8fa40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:32.069+0000 7f8fbdcd2700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8fa40a2c20 0x7f8fa40a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8000c00 tx=0x7f8fa8005fb0 comp rx=0 tx=0).stop 2024-06-10T00:31:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:32.069+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2059860395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb8107840 msgr2=0x7f8fb8075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:32.069+0000 7f8fbdcd2700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb8107840 0x7f8fb8075210 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0007ee0 tx=0x7f8fb000d3b0 comp rx=0 tx=0).stop 2024-06-10T00:31:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:32.069+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2059860395 shutdown_connections 2024-06-10T00:31:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:32.069+0000 7f8fbdcd2700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb8107840 0x7f8fb8075210 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:32.069+0000 7f8fbdcd2700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8fa40a2c20 0x7f8fa40a50e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:32.069+0000 7f8fbdcd2700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fb8102740 0x7f8fb8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:32.069+0000 7f8fbdcd2700 1 --2- 172.21.15.62:0/2059860395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fb81030f0 0x7f8fb8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:32.069+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2059860395 >> 172.21.15.62:0/2059860395 conn(0x7f8fb80795f0 msgr2=0x7f8fb81007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:32.070+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2059860395 shutdown_connections 2024-06-10T00:31:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:32.070+0000 7f8fbdcd2700 1 -- 172.21.15.62:0/2059860395 wait complete. 2024-06-10T00:31:32.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:31 smithi138 ceph-mon[36549]: pgmap v393: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-10T00:31:32.821 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-10T00:31:32.821 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:31:33.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:32 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2059860395' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-10T00:31:33.074 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:33.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:32 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2059860395' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-10T00:31:33.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:32 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2059860395' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-10T00:31:33.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:33 smithi062 ceph-mon[32390]: pgmap v394: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-10T00:31:34.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:33 smithi105 ceph-mon[36321]: pgmap v394: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-10T00:31:34.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:33 smithi138 ceph-mon[36549]: pgmap v394: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-10T00:31:34.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.857+0000 7f3e8c08f700 1 -- 172.21.15.62:0/2804266796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e8406b580 msgr2=0x7f3e8410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:34.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.857+0000 7f3e8c08f700 1 --2- 172.21.15.62:0/2804266796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e8406b580 0x7f3e8410f320 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c00b0a0 tx=0x7f3e7c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:31:34.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.857+0000 7f3e8c08f700 1 -- 172.21.15.62:0/2804266796 shutdown_connections 2024-06-10T00:31:34.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.857+0000 7f3e8c08f700 1 --2- 172.21.15.62:0/2804266796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e8406b580 0x7f3e8410f320 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:34.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.857+0000 7f3e8c08f700 1 --2- 172.21.15.62:0/2804266796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e84108500 0x7f3e841088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:34.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.857+0000 7f3e8c08f700 1 --2- 172.21.15.62:0/2804266796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e8406abc0 0x7f3e8406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:34.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.857+0000 7f3e8c08f700 1 -- 172.21.15.62:0/2804266796 >> 172.21.15.62:0/2804266796 conn(0x7f3e84078340 msgr2=0x7f3e84078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:34.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.857+0000 7f3e8c08f700 1 -- 172.21.15.62:0/2804266796 shutdown_connections 2024-06-10T00:31:34.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.857+0000 7f3e8c08f700 1 -- 172.21.15.62:0/2804266796 wait complete. 2024-06-10T00:31:34.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.858+0000 7f3e8c08f700 1 Processor -- start 2024-06-10T00:31:34.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.858+0000 7f3e8c08f700 1 -- start start 2024-06-10T00:31:34.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.858+0000 7f3e8c08f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e8406abc0 0x7f3e8419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:34.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.858+0000 7f3e8c08f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e8406b580 0x7f3e8419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:34.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.858+0000 7f3e8c08f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e84108500 0x7f3e84198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:34.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.858+0000 7f3e8c08f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841127e0 con 0x7f3e84108500 2024-06-10T00:31:34.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.858+0000 7f3e8c08f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3e84112660 con 0x7f3e8406abc0 2024-06-10T00:31:34.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.858+0000 7f3e8c08f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e84112960 con 0x7f3e8406b580 2024-06-10T00:31:34.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.858+0000 7f3e8a62c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e84108500 0x7f3e84198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:34.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.858+0000 7f3e8a62c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e84108500 0x7f3e84198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40076/0 (socket says 172.21.15.62:40076) 2024-06-10T00:31:34.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.858+0000 7f3e8a62c700 1 -- 172.21.15.62:0/1449533620 learned_addr learned my addr 172.21.15.62:0/1449533620 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:34.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.858+0000 7f3e8962a700 1 --2- 172.21.15.62:0/1449533620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e8406b580 0x7f3e8419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:34.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.858+0000 7f3e89e2b700 1 --2- 172.21.15.62:0/1449533620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e8406abc0 0x7f3e8419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:34.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.859+0000 7f3e8a62c700 1 -- 172.21.15.62:0/1449533620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e8406b580 msgr2=0x7f3e8419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:34.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.859+0000 7f3e8a62c700 1 --2- 172.21.15.62:0/1449533620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e8406b580 0x7f3e8419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:34.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.859+0000 7f3e8a62c700 1 -- 172.21.15.62:0/1449533620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e8406abc0 msgr2=0x7f3e8419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:34.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.859+0000 7f3e8a62c700 1 --2- 172.21.15.62:0/1449533620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e8406abc0 0x7f3e8419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:34.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.859+0000 7f3e8a62c700 1 -- 172.21.15.62:0/1449533620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e84199260 con 0x7f3e84108500 2024-06-10T00:31:34.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.859+0000 7f3e8a62c700 1 --2- 172.21.15.62:0/1449533620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e84108500 0x7f3e84198b00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c0062a0 tx=0x7f3e7c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:34.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.859+0000 7f3e7affd700 1 -- 172.21.15.62:0/1449533620 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c021070 con 0x7f3e84108500 2024-06-10T00:31:34.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.859+0000 7f3e7affd700 1 -- 172.21.15.62:0/1449533620 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e7c007c80 con 0x7f3e84108500 2024-06-10T00:31:34.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.859+0000 7f3e8c08f700 1 -- 172.21.15.62:0/1449533620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e841994f0 con 0x7f3e84108500 2024-06-10T00:31:34.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.859+0000 7f3e7affd700 1 -- 172.21.15.62:0/1449533620 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c005420 con 0x7f3e84108500 2024-06-10T00:31:34.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.859+0000 7f3e8c08f700 1 -- 172.21.15.62:0/1449533620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e841a57c0 con 0x7f3e84108500 2024-06-10T00:31:34.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.860+0000 7f3e8c08f700 1 -- 172.21.15.62:0/1449533620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e84069180 con 0x7f3e84108500 2024-06-10T00:31:34.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.861+0000 7f3e7affd700 1 -- 172.21.15.62:0/1449533620 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3e7c024070 con 0x7f3e84108500 2024-06-10T00:31:34.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.862+0000 7f3e7affd700 1 --2- 172.21.15.62:0/1449533620 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3e700a2c20 0x7f3e700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:34.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.862+0000 7f3e7affd700 1 -- 172.21.15.62:0/1449533620 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f3e7c00f020 con 0x7f3e84108500 2024-06-10T00:31:34.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.862+0000 7f3e89e2b700 1 --2- 172.21.15.62:0/1449533620 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3e700a2c20 0x7f3e700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:34.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.863+0000 7f3e89e2b700 1 --2- 172.21.15.62:0/1449533620 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3e700a2c20 0x7f3e700a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3e80005950 tx=0x7f3e8000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:34.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:34.867+0000 7f3e7affd700 1 -- 172.21.15.62:0/1449533620 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3e7c02b030 con 0x7f3e84108500 2024-06-10T00:31:35.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:35.121+0000 7f3e8c08f700 1 -- 172.21.15.62:0/1449533620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e84199fe0 con 0x7f3e84108500 2024-06-10T00:31:35.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:35.122+0000 7f3e7affd700 1 -- 172.21.15.62:0/1449533620 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7f3e7c09d350 con 0x7f3e84108500 2024-06-10T00:31:35.122 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:31:35.122 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:29:47.712789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24431,24424,24418,24433,24464]},"id":1}]} 2024-06-10T00:31:35.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:35.124+0000 7f3e8c08f700 1 -- 172.21.15.62:0/1449533620 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3e700a2c20 msgr2=0x7f3e700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:35.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:35.124+0000 7f3e8c08f700 1 --2- 172.21.15.62:0/1449533620 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3e700a2c20 0x7f3e700a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3e80005950 tx=0x7f3e8000a300 comp rx=0 tx=0).stop 2024-06-10T00:31:35.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:35.124+0000 7f3e8c08f700 1 -- 172.21.15.62:0/1449533620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e84108500 msgr2=0x7f3e84198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:35.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:35.124+0000 7f3e8c08f700 1 --2- 172.21.15.62:0/1449533620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e84108500 0x7f3e84198b00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c0062a0 tx=0x7f3e7c00be60 comp rx=0 tx=0).stop 2024-06-10T00:31:35.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:35.124+0000 7f3e8c08f700 1 -- 172.21.15.62:0/1449533620 shutdown_connections 2024-06-10T00:31:35.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:35.124+0000 7f3e8c08f700 1 --2- 172.21.15.62:0/1449533620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e84108500 0x7f3e84198b00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:35.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:35.124+0000 7f3e8c08f700 1 --2- 172.21.15.62:0/1449533620 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3e700a2c20 0x7f3e700a50e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:35.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:35.124+0000 7f3e8c08f700 1 --2- 172.21.15.62:0/1449533620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e8406abc0 0x7f3e8419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:35.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:35.124+0000 7f3e8c08f700 1 --2- 172.21.15.62:0/1449533620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e8406b580 0x7f3e8419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:35.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:35.124+0000 7f3e8c08f700 1 -- 172.21.15.62:0/1449533620 >> 172.21.15.62:0/1449533620 conn(0x7f3e84078340 msgr2=0x7f3e84100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:35.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:35.124+0000 7f3e8c08f700 1 -- 172.21.15.62:0/1449533620 shutdown_connections 2024-06-10T00:31:35.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:35.124+0000 7f3e8c08f700 1 -- 172.21.15.62:0/1449533620 wait complete. 2024-06-10T00:31:35.125 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 29 2024-06-10T00:31:35.738 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:31:35.990 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:36.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:35 smithi105 ceph-mon[36321]: pgmap v395: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-06-10T00:31:36.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:35 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1449533620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:31:36.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:35 smithi062 ceph-mon[32390]: pgmap v395: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-06-10T00:31:36.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:35 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1449533620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:31:36.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:35 smithi138 ceph-mon[36549]: pgmap v395: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-06-10T00:31:36.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:35 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1449533620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:31:37.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.746+0000 7f08e586d700 1 -- 172.21.15.62:0/687455899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e01030f0 msgr2=0x7f08e0107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:37.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.746+0000 7f08e586d700 1 --2- 172.21.15.62:0/687455899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e01030f0 0x7f08e0107110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f08d00099c0 tx=0x7f08d0009cd0 comp rx=0 tx=0).stop 2024-06-10T00:31:37.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.746+0000 7f08e586d700 1 -- 172.21.15.62:0/687455899 shutdown_connections 2024-06-10T00:31:37.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.746+0000 7f08e586d700 1 --2- 172.21.15.62:0/687455899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08e0107840 0x7f08e010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:37.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.746+0000 7f08e586d700 1 --2- 172.21.15.62:0/687455899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08e0102740 0x7f08e0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:37.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.746+0000 7f08e586d700 1 --2- 172.21.15.62:0/687455899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e01030f0 0x7f08e0107110 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:37.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.746+0000 7f08e586d700 1 -- 172.21.15.62:0/687455899 >> 172.21.15.62:0/687455899 conn(0x7f08e00795f0 msgr2=0x7f08e0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:37.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:37 smithi062 ceph-mon[32390]: pgmap v396: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-10T00:31:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.746+0000 7f08e586d700 1 -- 172.21.15.62:0/687455899 shutdown_connections 2024-06-10T00:31:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.746+0000 7f08e586d700 1 -- 172.21.15.62:0/687455899 wait complete. 2024-06-10T00:31:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.747+0000 7f08e586d700 1 Processor -- start 2024-06-10T00:31:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.747+0000 7f08e586d700 1 -- start start 2024-06-10T00:31:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.747+0000 7f08e586d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08e0102740 0x7f08e0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.747+0000 7f08e586d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08e01030f0 0x7f08e0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.747+0000 7f08e586d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e0107840 0x7f08e0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.747+0000 7f08e586d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f08e01127d0 con 0x7f08e0107840 2024-06-10T00:31:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.747+0000 7f08e586d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f08e0112650 con 0x7f08e01030f0 2024-06-10T00:31:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.747+0000 7f08e586d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08e0112950 con 0x7f08e0102740 2024-06-10T00:31:37.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.747+0000 7f08df7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e0107840 0x7f08e0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.747+0000 7f08df7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e0107840 0x7f08e0075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40096/0 (socket says 172.21.15.62:40096) 2024-06-10T00:31:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.747+0000 7f08df7fe700 1 -- 172.21.15.62:0/4201177673 learned_addr learned my addr 172.21.15.62:0/4201177673 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.747+0000 7f08deffd700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08e0102740 0x7f08e0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.748+0000 7f08de7fc700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08e01030f0 0x7f08e0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.748+0000 7f08df7fe700 1 -- 172.21.15.62:0/4201177673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08e0102740 msgr2=0x7f08e0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.748+0000 7f08df7fe700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08e0102740 0x7f08e0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.748+0000 7f08df7fe700 1 -- 172.21.15.62:0/4201177673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08e01030f0 msgr2=0x7f08e0074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.748+0000 7f08df7fe700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08e01030f0 0x7f08e0074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.748+0000 7f08df7fe700 1 -- 172.21.15.62:0/4201177673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e01a70b0 con 0x7f08e0107840 2024-06-10T00:31:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.748+0000 7f08de7fc700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08e01030f0 0x7f08e0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:31:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.748+0000 7f08df7fe700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e0107840 0x7f08e0075210 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f08d800b970 tx=0x7f08d800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.748+0000 7f08e486b700 1 -- 172.21.15.62:0/4201177673 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8019070 con 0x7f08e0107840 2024-06-10T00:31:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.748+0000 7f08e486b700 1 -- 172.21.15.62:0/4201177673 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d8014440 con 0x7f08e0107840 2024-06-10T00:31:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.748+0000 7f08e586d700 1 -- 172.21.15.62:0/4201177673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e01a73a0 con 0x7f08e0107840 2024-06-10T00:31:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.748+0000 7f08e486b700 1 -- 172.21.15.62:0/4201177673 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d80075d0 con 0x7f08e0107840 2024-06-10T00:31:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.748+0000 7f08e586d700 1 -- 172.21.15.62:0/4201177673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e01a7930 con 0x7f08e0107840 2024-06-10T00:31:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.749+0000 7f08e586d700 1 -- 172.21.15.62:0/4201177673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e0069180 con 0x7f08e0107840 2024-06-10T00:31:37.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.750+0000 7f08e486b700 1 -- 172.21.15.62:0/4201177673 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f08d801c070 con 0x7f08e0107840 2024-06-10T00:31:37.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.751+0000 7f08e486b700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f08c80a2ca0 0x7f08c80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:37.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.751+0000 7f08deffd700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f08c80a2ca0 0x7f08c80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:37.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.751+0000 7f08e486b700 1 -- 172.21.15.62:0/4201177673 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f08d80ce7a0 con 0x7f08e0107840 2024-06-10T00:31:37.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.752+0000 7f08deffd700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f08c80a2ca0 0x7f08c80a5160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f08d4005ea0 tx=0x7f08d4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:37.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:37.756+0000 7f08e486b700 1 -- 172.21.15.62:0/4201177673 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f08d809b330 con 0x7f08e0107840 2024-06-10T00:31:38.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:38.009+0000 7f08e586d700 1 -- 172.21.15.62:0/4201177673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08e01a7c40 con 0x7f08e0107840 2024-06-10T00:31:38.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:38.010+0000 7f08e486b700 1 -- 172.21.15.62:0/4201177673 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7f08d809ac30 con 0x7f08e0107840 2024-06-10T00:31:38.010 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:31:38.011 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:29:47.712789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24431,24424,24418,24433,24464]},"id":1}]} 2024-06-10T00:31:38.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:38.012+0000 7f08e586d700 1 -- 172.21.15.62:0/4201177673 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f08c80a2ca0 msgr2=0x7f08c80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:38.012+0000 7f08e586d700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f08c80a2ca0 0x7f08c80a5160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f08d4005ea0 tx=0x7f08d4005e30 comp rx=0 tx=0).stop 2024-06-10T00:31:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:38.012+0000 7f08e586d700 1 -- 172.21.15.62:0/4201177673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e0107840 msgr2=0x7f08e0075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:38.012+0000 7f08e586d700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e0107840 0x7f08e0075210 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f08d800b970 tx=0x7f08d800bd30 comp rx=0 tx=0).stop 2024-06-10T00:31:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:38.012+0000 7f08e586d700 1 -- 172.21.15.62:0/4201177673 shutdown_connections 2024-06-10T00:31:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:38.012+0000 7f08e586d700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e0107840 0x7f08e0075210 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:38.012+0000 7f08e586d700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f08c80a2ca0 0x7f08c80a5160 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:38.012+0000 7f08e586d700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08e0102740 0x7f08e0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:38.012+0000 7f08e586d700 1 --2- 172.21.15.62:0/4201177673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08e01030f0 0x7f08e0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:38.012+0000 7f08e586d700 1 -- 172.21.15.62:0/4201177673 >> 172.21.15.62:0/4201177673 conn(0x7f08e00795f0 msgr2=0x7f08e0100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:38.013+0000 7f08e586d700 1 -- 172.21.15.62:0/4201177673 shutdown_connections 2024-06-10T00:31:38.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:38.013+0000 7f08e586d700 1 -- 172.21.15.62:0/4201177673 wait complete. 2024-06-10T00:31:38.014 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 29 2024-06-10T00:31:38.034 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:37 smithi105 ceph-mon[36321]: pgmap v396: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-10T00:31:38.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:37 smithi138 ceph-mon[36549]: pgmap v396: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-10T00:31:38.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:31:39.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:38 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/4201177673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:31:39.077 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:39.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:38 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/4201177673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:31:39.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:38 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/4201177673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:31:40.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:39 smithi105 ceph-mon[36321]: pgmap v397: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-10T00:31:40.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:39 smithi062 ceph-mon[32390]: pgmap v397: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-10T00:31:40.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:39 smithi138 ceph-mon[36549]: pgmap v397: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-10T00:31:40.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.680+0000 7fae3a0d9700 1 -- 172.21.15.62:0/4160355043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34101d30 msgr2=0x7fae3410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:40.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.680+0000 7fae3a0d9700 1 --2- 172.21.15.62:0/4160355043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34101d30 0x7fae3410e210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00b0a0 tx=0x7fae2c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:31:40.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.680+0000 7fae3a0d9700 1 -- 172.21.15.62:0/4160355043 shutdown_connections 2024-06-10T00:31:40.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.680+0000 7fae3a0d9700 1 --2- 172.21.15.62:0/4160355043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34101d30 0x7fae3410e210 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:40.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.680+0000 7fae3a0d9700 1 --2- 172.21.15.62:0/4160355043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae341073d0 0x7fae341077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:40.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.680+0000 7fae3a0d9700 1 --2- 172.21.15.62:0/4160355043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae34101390 0x7fae341017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:40.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.680+0000 7fae3a0d9700 1 -- 172.21.15.62:0/4160355043 >> 172.21.15.62:0/4160355043 conn(0x7fae340782c0 msgr2=0x7fae340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:40.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.680+0000 7fae3a0d9700 1 -- 172.21.15.62:0/4160355043 shutdown_connections 2024-06-10T00:31:40.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.680+0000 7fae3a0d9700 1 -- 172.21.15.62:0/4160355043 wait complete. 2024-06-10T00:31:40.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.681+0000 7fae3a0d9700 1 Processor -- start 2024-06-10T00:31:40.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.681+0000 7fae3a0d9700 1 -- start start 2024-06-10T00:31:40.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae3a0d9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae34101390 0x7fae3419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:40.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae3a0d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34101d30 0x7fae3419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:40.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae3a0d9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae341073d0 0x7fae34197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:40.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae3a0d9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fae34111690 con 0x7fae34101d30 2024-06-10T00:31:40.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae3a0d9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fae34111510 con 0x7fae34101390 2024-06-10T00:31:40.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae3a0d9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae34111810 con 0x7fae341073d0 2024-06-10T00:31:40.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae2affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34101d30 0x7fae3419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:40.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae2affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34101d30 0x7fae3419dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41364/0 (socket says 172.21.15.62:41364) 2024-06-10T00:31:40.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae2affd700 1 -- 172.21.15.62:0/859181953 learned_addr learned my addr 172.21.15.62:0/859181953 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:40.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae33fff700 1 --2- 172.21.15.62:0/859181953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae341073d0 0x7fae34197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:40.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae337fe700 1 --2- 172.21.15.62:0/859181953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae34101390 0x7fae3419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:40.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae2affd700 1 -- 172.21.15.62:0/859181953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae341073d0 msgr2=0x7fae34197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:40.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae2affd700 1 --2- 172.21.15.62:0/859181953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae341073d0 0x7fae34197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:40.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae2affd700 1 -- 172.21.15.62:0/859181953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae34101390 msgr2=0x7fae3419d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:40.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae2affd700 1 --2- 172.21.15.62:0/859181953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae34101390 0x7fae3419d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:40.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae2affd700 1 -- 172.21.15.62:0/859181953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae341980f0 con 0x7fae34101d30 2024-06-10T00:31:40.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.682+0000 7fae2affd700 1 --2- 172.21.15.62:0/859181953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34101d30 0x7fae3419dcc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fae1c007b50 tx=0x7fae1c007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:40.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.683+0000 7fae317fa700 1 -- 172.21.15.62:0/859181953 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c01a070 con 0x7fae34101d30 2024-06-10T00:31:40.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.683+0000 7fae317fa700 1 -- 172.21.15.62:0/859181953 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae1c00eaa0 con 0x7fae34101d30 2024-06-10T00:31:40.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.683+0000 7fae3a0d9700 1 -- 172.21.15.62:0/859181953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae341983e0 con 0x7fae34101d30 2024-06-10T00:31:40.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.683+0000 7fae317fa700 1 -- 172.21.15.62:0/859181953 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c00ab40 con 0x7fae34101d30 2024-06-10T00:31:40.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.683+0000 7fae3a0d9700 1 -- 172.21.15.62:0/859181953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3406a170 con 0x7fae34101d30 2024-06-10T00:31:40.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.684+0000 7fae3a0d9700 1 -- 172.21.15.62:0/859181953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae34069180 con 0x7fae34101d30 2024-06-10T00:31:40.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.685+0000 7fae317fa700 1 -- 172.21.15.62:0/859181953 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fae1c00ace0 con 0x7fae34101d30 2024-06-10T00:31:40.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.686+0000 7fae317fa700 1 --2- 172.21.15.62:0/859181953 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fae200a2c20 0x7fae200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:40.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.686+0000 7fae337fe700 1 --2- 172.21.15.62:0/859181953 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fae200a2c20 0x7fae200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:40.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.686+0000 7fae317fa700 1 -- 172.21.15.62:0/859181953 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fae1c0d06a0 con 0x7fae34101d30 2024-06-10T00:31:40.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.686+0000 7fae337fe700 1 --2- 172.21.15.62:0/859181953 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fae200a2c20 0x7fae200a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fae24006fd0 tx=0x7fae24008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:40.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.689+0000 7fae317fa700 1 -- 172.21.15.62:0/859181953 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fae1c023020 con 0x7fae34101d30 2024-06-10T00:31:40.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.898+0000 7fae3a0d9700 1 -- 172.21.15.62:0/859181953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae3406ac60 con 0x7fae34101d30 2024-06-10T00:31:40.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.900+0000 7fae317fa700 1 -- 172.21.15.62:0/859181953 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24839 (secure 0 0 0) 0x7fae1c09d230 con 0x7fae34101d30 2024-06-10T00:31:40.900 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:31:40.900 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":99,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:30:24.104116+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:31:40.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.902+0000 7fae3a0d9700 1 -- 172.21.15.62:0/859181953 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fae200a2c20 msgr2=0x7fae200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:40.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.902+0000 7fae3a0d9700 1 --2- 172.21.15.62:0/859181953 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fae200a2c20 0x7fae200a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fae24006fd0 tx=0x7fae24008040 comp rx=0 tx=0).stop 2024-06-10T00:31:40.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.902+0000 7fae3a0d9700 1 -- 172.21.15.62:0/859181953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34101d30 msgr2=0x7fae3419dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:40.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.902+0000 7fae3a0d9700 1 --2- 172.21.15.62:0/859181953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34101d30 0x7fae3419dcc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fae1c007b50 tx=0x7fae1c007f10 comp rx=0 tx=0).stop 2024-06-10T00:31:40.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.902+0000 7fae3a0d9700 1 -- 172.21.15.62:0/859181953 shutdown_connections 2024-06-10T00:31:40.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.902+0000 7fae3a0d9700 1 --2- 172.21.15.62:0/859181953 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fae200a2c20 0x7fae200a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:40.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.902+0000 7fae3a0d9700 1 --2- 172.21.15.62:0/859181953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae341073d0 0x7fae34197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:40.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.902+0000 7fae3a0d9700 1 --2- 172.21.15.62:0/859181953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae34101390 0x7fae3419d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:40.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.902+0000 7fae3a0d9700 1 --2- 172.21.15.62:0/859181953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34101d30 0x7fae3419dcc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:40.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.902+0000 7fae3a0d9700 1 -- 172.21.15.62:0/859181953 >> 172.21.15.62:0/859181953 conn(0x7fae340782c0 msgr2=0x7fae340ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:40.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.903+0000 7fae3a0d9700 1 -- 172.21.15.62:0/859181953 shutdown_connections 2024-06-10T00:31:40.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:40.903+0000 7fae3a0d9700 1 -- 172.21.15.62:0/859181953 wait complete. 2024-06-10T00:31:41.850 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:31:42.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:41 smithi105 ceph-mon[36321]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:31:42.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:41 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/859181953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:31:42.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:41 smithi062 ceph-mon[32390]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:31:42.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:41 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/859181953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:31:42.109 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:42.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:41 smithi138 ceph-mon[36549]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:31:42.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:41 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/859181953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:31:43.802 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:43 smithi062 ceph-mon[32390]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:31:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.911+0000 7f9158dcf700 1 -- 172.21.15.62:0/2470324182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9154108650 msgr2=0x7f9154108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.911+0000 7f9158dcf700 1 --2- 172.21.15.62:0/2470324182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9154108650 0x7f9154108a30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9144009990 tx=0x7f9144009ca0 comp rx=0 tx=0).stop 2024-06-10T00:31:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.911+0000 7f9158dcf700 1 -- 172.21.15.62:0/2470324182 shutdown_connections 2024-06-10T00:31:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.911+0000 7f9158dcf700 1 --2- 172.21.15.62:0/2470324182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9154108650 0x7f9154108a30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:43.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.911+0000 7f9158dcf700 1 --2- 172.21.15.62:0/2470324182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9154102e80 0x7f915410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.911+0000 7f9158dcf700 1 --2- 172.21.15.62:0/2470324182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91541024e0 0x7f9154102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.911+0000 7f9158dcf700 1 -- 172.21.15.62:0/2470324182 >> 172.21.15.62:0/2470324182 conn(0x7f91540795f0 msgr2=0x7f9154079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.911+0000 7f9158dcf700 1 -- 172.21.15.62:0/2470324182 shutdown_connections 2024-06-10T00:31:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.911+0000 7f9158dcf700 1 -- 172.21.15.62:0/2470324182 wait complete. 2024-06-10T00:31:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.911+0000 7f9158dcf700 1 Processor -- start 2024-06-10T00:31:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9158dcf700 1 -- start start 2024-06-10T00:31:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9158dcf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91541024e0 0x7f915419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9158dcf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9154102e80 0x7f915419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9158dcf700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9154108650 0x7f9154198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9158dcf700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f91541127a0 con 0x7f9154102e80 2024-06-10T00:31:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9158dcf700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9154112620 con 0x7f9154108650 2024-06-10T00:31:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9158dcf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9154112920 con 0x7f91541024e0 2024-06-10T00:31:43.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9151d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9154102e80 0x7f915419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9151d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9154102e80 0x7f915419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41376/0 (socket says 172.21.15.62:41376) 2024-06-10T00:31:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f915259c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91541024e0 0x7f915419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9151d9b700 1 -- 172.21.15.62:0/2051021392 learned_addr learned my addr 172.21.15.62:0/2051021392 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9152d9d700 1 --2- 172.21.15.62:0/2051021392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9154108650 0x7f9154198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9151d9b700 1 -- 172.21.15.62:0/2051021392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91541024e0 msgr2=0x7f915419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9151d9b700 1 --2- 172.21.15.62:0/2051021392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91541024e0 0x7f915419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9151d9b700 1 -- 172.21.15.62:0/2051021392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9154108650 msgr2=0x7f9154198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9151d9b700 1 --2- 172.21.15.62:0/2051021392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9154108650 0x7f9154198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:43.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9151d9b700 1 -- 172.21.15.62:0/2051021392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91541991e0 con 0x7f9154102e80 2024-06-10T00:31:43.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.912+0000 7f9151d9b700 1 --2- 172.21.15.62:0/2051021392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9154102e80 0x7f915419edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f914800d950 tx=0x7f914800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:43.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.913+0000 7f91437fe700 1 -- 172.21.15.62:0/2051021392 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9148016070 con 0x7f9154102e80 2024-06-10T00:31:43.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.913+0000 7f9158dcf700 1 -- 172.21.15.62:0/2051021392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91541994d0 con 0x7f9154102e80 2024-06-10T00:31:43.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.913+0000 7f91437fe700 1 -- 172.21.15.62:0/2051021392 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f914800fd70 con 0x7f9154102e80 2024-06-10T00:31:43.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.913+0000 7f91437fe700 1 -- 172.21.15.62:0/2051021392 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914800e610 con 0x7f9154102e80 2024-06-10T00:31:43.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.913+0000 7f9158dcf700 1 -- 172.21.15.62:0/2051021392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91541a5740 con 0x7f9154102e80 2024-06-10T00:31:43.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.914+0000 7f9158dcf700 1 -- 172.21.15.62:0/2051021392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9154069180 con 0x7f9154102e80 2024-06-10T00:31:43.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.915+0000 7f91437fe700 1 -- 172.21.15.62:0/2051021392 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9148019070 con 0x7f9154102e80 2024-06-10T00:31:43.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.916+0000 7f91437fe700 1 --2- 172.21.15.62:0/2051021392 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f913c0a2ca0 0x7f913c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:43.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.916+0000 7f91437fe700 1 -- 172.21.15.62:0/2051021392 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f91480cee40 con 0x7f9154102e80 2024-06-10T00:31:43.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.916+0000 7f915259c700 1 --2- 172.21.15.62:0/2051021392 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f913c0a2ca0 0x7f913c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:43.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.916+0000 7f915259c700 1 --2- 172.21.15.62:0/2051021392 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f913c0a2ca0 0x7f913c0a5160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f914400b5c0 tx=0x7f9144005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:43.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:43.920+0000 7f91437fe700 1 -- 172.21.15.62:0/2051021392 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f914809b9d0 con 0x7f9154102e80 2024-06-10T00:31:44.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:43 smithi105 ceph-mon[36321]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:31:44.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:43 smithi138 ceph-mon[36549]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:31:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:44.175+0000 7f9158dcf700 1 -- 172.21.15.62:0/2051021392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f915402dca0 con 0x7f9154102e80 2024-06-10T00:31:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:44.175+0000 7f91437fe700 1 -- 172.21.15.62:0/2051021392 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7f914809b2d0 con 0x7f9154102e80 2024-06-10T00:31:44.175 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:31:44.176 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:29:47.712789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24431,24424,24418,24433,24464]},"id":1}]} 2024-06-10T00:31:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:44.177+0000 7f9158dcf700 1 -- 172.21.15.62:0/2051021392 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f913c0a2ca0 msgr2=0x7f913c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:44.177+0000 7f9158dcf700 1 --2- 172.21.15.62:0/2051021392 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f913c0a2ca0 0x7f913c0a5160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f914400b5c0 tx=0x7f9144005960 comp rx=0 tx=0).stop 2024-06-10T00:31:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:44.177+0000 7f9158dcf700 1 -- 172.21.15.62:0/2051021392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9154102e80 msgr2=0x7f915419edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:44.177+0000 7f9158dcf700 1 --2- 172.21.15.62:0/2051021392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9154102e80 0x7f915419edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f914800d950 tx=0x7f914800dc60 comp rx=0 tx=0).stop 2024-06-10T00:31:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:44.177+0000 7f9158dcf700 1 -- 172.21.15.62:0/2051021392 shutdown_connections 2024-06-10T00:31:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:44.177+0000 7f9158dcf700 1 --2- 172.21.15.62:0/2051021392 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f913c0a2ca0 0x7f913c0a5160 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:44.177+0000 7f9158dcf700 1 --2- 172.21.15.62:0/2051021392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9154108650 0x7f9154198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:44.177+0000 7f9158dcf700 1 --2- 172.21.15.62:0/2051021392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91541024e0 0x7f915419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:44.177+0000 7f9158dcf700 1 --2- 172.21.15.62:0/2051021392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9154102e80 0x7f915419edb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:44.177+0000 7f9158dcf700 1 -- 172.21.15.62:0/2051021392 >> 172.21.15.62:0/2051021392 conn(0x7f91540795f0 msgr2=0x7f91541011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:44.178+0000 7f9158dcf700 1 -- 172.21.15.62:0/2051021392 shutdown_connections 2024-06-10T00:31:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:44.178+0000 7f9158dcf700 1 -- 172.21.15.62:0/2051021392 wait complete. 2024-06-10T00:31:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 29 2024-06-10T00:31:45.008 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:44 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2051021392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:31:45.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:44 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2051021392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:31:45.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-10T00:28:39.478894+0000', 'modified': '2024-06-10T00:29:47.712789+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24464, 'mds_1': 24433, 'mds_2': 24431, 'mds_3': 24424, 'mds_4': 24418}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6839/4040406390', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 4040406390}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 4040406390}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6841/2460552589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6840', 'nonce': 2460552589}, {'type': 'v1', 'addr': '172.21.15.62:6841', 'nonce': 2460552589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24377': {'gid': 24377, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6833/1194987741', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6832', 'nonce': 1194987741}, {'type': 'v1', 'addr': '172.21.15.105:6833', 'nonce': 1194987741}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24382': {'gid': 24382, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6833/2673201423', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 2673201423}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 2673201423}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6835/588077925', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 588077925}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 588077925}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 27, 'addr': '172.21.15.138:6835/331298815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 331298815}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 331298815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.138:6837/3880084398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 3880084398}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 3880084398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.105:6837/4270590303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 4270590303}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 4270590303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.138:6839/339302206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 339302206}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 339302206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.105:6839/1632080154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 1632080154}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 1632080154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24464, 'qdb_cluster': [24431, 24424, 24418, 24433, 24464]} 2024-06-10T00:31:45.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-10T00:31:45.037 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-06-10T00:31:45.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:44 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2051021392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:31:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:46.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:45 smithi105 ceph-mon[36321]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:31:46.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:45 smithi062 ceph-mon[32390]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:31:46.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:45 smithi138 ceph-mon[36549]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:31:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.979+0000 7f08fca86700 1 -- 172.21.15.62:0/3099832205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f8102740 msgr2=0x7f08f8102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.979+0000 7f08fca86700 1 --2- 172.21.15.62:0/3099832205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f8102740 0x7f08f8102b20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f08e80099c0 tx=0x7f08e8009cd0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.979+0000 7f08fca86700 1 -- 172.21.15.62:0/3099832205 shutdown_connections 2024-06-10T00:31:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.979+0000 7f08fca86700 1 --2- 172.21.15.62:0/3099832205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f8102740 0x7f08f8102b20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.979+0000 7f08fca86700 1 --2- 172.21.15.62:0/3099832205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08f8107840 0x7f08f810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.979+0000 7f08fca86700 1 --2- 172.21.15.62:0/3099832205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08f81030f0 0x7f08f8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.979+0000 7f08fca86700 1 -- 172.21.15.62:0/3099832205 >> 172.21.15.62:0/3099832205 conn(0x7f08f80795f0 msgr2=0x7f08f8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:46.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.979+0000 7f08fca86700 1 -- 172.21.15.62:0/3099832205 shutdown_connections 2024-06-10T00:31:46.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.979+0000 7f08fca86700 1 -- 172.21.15.62:0/3099832205 wait complete. 2024-06-10T00:31:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.979+0000 7f08fca86700 1 Processor -- start 2024-06-10T00:31:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08fca86700 1 -- start start 2024-06-10T00:31:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08fca86700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f8102740 0x7f08f819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08fca86700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08f81030f0 0x7f08f819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08fca86700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08f8107840 0x7f08f8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08fca86700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f08f8112780 con 0x7f08f8102740 2024-06-10T00:31:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08fca86700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f08f8112600 con 0x7f08f81030f0 2024-06-10T00:31:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08fca86700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08f8112900 con 0x7f08f8107840 2024-06-10T00:31:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08f659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f8102740 0x7f08f819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08f659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f8102740 0x7f08f819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41392/0 (socket says 172.21.15.62:41392) 2024-06-10T00:31:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08f659c700 1 -- 172.21.15.62:0/1070208454 learned_addr learned my addr 172.21.15.62:0/1070208454 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08f6d9d700 1 --2- 172.21.15.62:0/1070208454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08f8107840 0x7f08f8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08f5d9b700 1 --2- 172.21.15.62:0/1070208454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08f81030f0 0x7f08f819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08f659c700 1 -- 172.21.15.62:0/1070208454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08f8107840 msgr2=0x7f08f8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:46.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08f659c700 1 --2- 172.21.15.62:0/1070208454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08f8107840 0x7f08f8198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08f659c700 1 -- 172.21.15.62:0/1070208454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08f81030f0 msgr2=0x7f08f819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:46.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08f659c700 1 --2- 172.21.15.62:0/1070208454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08f81030f0 0x7f08f819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08f659c700 1 -- 172.21.15.62:0/1070208454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f81990e0 con 0x7f08f8102740 2024-06-10T00:31:46.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.980+0000 7f08f659c700 1 --2- 172.21.15.62:0/1070208454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f8102740 0x7f08f819e7c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f08e8006010 tx=0x7f08e8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:46.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.981+0000 7f08e77fe700 1 -- 172.21.15.62:0/1070208454 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8026070 con 0x7f08f8102740 2024-06-10T00:31:46.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.981+0000 7f08e77fe700 1 -- 172.21.15.62:0/1070208454 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e8004cd0 con 0x7f08f8102740 2024-06-10T00:31:46.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.981+0000 7f08fca86700 1 -- 172.21.15.62:0/1070208454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f8199370 con 0x7f08f8102740 2024-06-10T00:31:46.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.981+0000 7f08e77fe700 1 -- 172.21.15.62:0/1070208454 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8003ba0 con 0x7f08f8102740 2024-06-10T00:31:46.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.981+0000 7f08fca86700 1 -- 172.21.15.62:0/1070208454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f81a5690 con 0x7f08f8102740 2024-06-10T00:31:46.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.983+0000 7f08e77fe700 1 -- 172.21.15.62:0/1070208454 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f08e8029070 con 0x7f08f8102740 2024-06-10T00:31:46.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.984+0000 7f08e77fe700 1 --2- 172.21.15.62:0/1070208454 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f08e00a2c20 0x7f08e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:46.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.984+0000 7f08f5d9b700 1 --2- 172.21.15.62:0/1070208454 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f08e00a2c20 0x7f08e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:46.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.984+0000 7f08e77fe700 1 -- 172.21.15.62:0/1070208454 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f08e80cfd40 con 0x7f08f8102740 2024-06-10T00:31:46.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.984+0000 7f08f5d9b700 1 --2- 172.21.15.62:0/1070208454 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f08e00a2c20 0x7f08e00a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f08f807a090 tx=0x7f08ec00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:46.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.984+0000 7f0907309b80 1 Processor -- start 2024-06-10T00:31:46.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f0907309b80 1 -- start start 2024-06-10T00:31:46.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f0907309b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b34420 0x559f10b34800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:46.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f0907309b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b2e620 0x559f10b2eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:46.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f0907309b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b3b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:46.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f0907309b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559f10a1cf10 con 0x559f10b2efe0 2024-06-10T00:31:46.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f0907309b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559f10a7b800 con 0x559f10b34420 2024-06-10T00:31:46.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f0907309b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f109e3410 con 0x559f10b2e620 2024-06-10T00:31:46.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f08f6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b3b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:46.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f08f659c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b34420 0x559f10b34800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:46.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f08f6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b3b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41394/0 (socket says 172.21.15.62:41394) 2024-06-10T00:31:46.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f08f659c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b34420 0x559f10b34800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:35992/0 (socket says 172.21.15.62:35992) 2024-06-10T00:31:46.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f08f6d9d700 1 -- 172.21.15.62:0/4137829654 learned_addr learned my addr 172.21.15.62:0/4137829654 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:46.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f08f5d9b700 1 --2- 172.21.15.62:0/4137829654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b2e620 0x559f10b2eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:46.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f08f6d9d700 1 -- 172.21.15.62:0/4137829654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b2e620 msgr2=0x559f10b2eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:46.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f08f6d9d700 1 --2- 172.21.15.62:0/4137829654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b2e620 0x559f10b2eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f08f6d9d700 1 -- 172.21.15.62:0/4137829654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b34420 msgr2=0x559f10b34800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:46.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f08f6d9d700 1 --2- 172.21.15.62:0/4137829654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b34420 0x559f10b34800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f08f6d9d700 1 -- 172.21.15.62:0/4137829654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f109ca880 con 0x559f10b2efe0 2024-06-10T00:31:46.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.985+0000 7f08f659c700 1 --2- 172.21.15.62:0/4137829654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b34420 0x559f10b34800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:31:46.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.986+0000 7f08f6d9d700 1 --2- 172.21.15.62:0/4137829654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b3b1c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f08f000f5a0 tx=0x7f08f000f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fbc5d4e70592ef server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:46.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.986+0000 7f08e4ff9700 1 -- 172.21.15.62:0/4137829654 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0018030 con 0x559f10b2efe0 2024-06-10T00:31:46.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.986+0000 7f08e4ff9700 1 -- 172.21.15.62:0/4137829654 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f0004bb0 con 0x559f10b2efe0 2024-06-10T00:31:46.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.986+0000 7f08e4ff9700 1 -- 172.21.15.62:0/4137829654 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0005020 con 0x559f10b2efe0 2024-06-10T00:31:46.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.986+0000 7f0907309b80 1 -- 172.21.15.62:0/4137829654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 msgr2=0x559f10b3b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:46.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.986+0000 7f0907309b80 1 --2- 172.21.15.62:0/4137829654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b3b1c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f08f000f5a0 tx=0x7f08f000f8b0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 -- 172.21.15.62:0/4137829654 shutdown_connections 2024-06-10T00:31:46.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 --2- 172.21.15.62:0/4137829654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b3b1c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 --2- 172.21.15.62:0/4137829654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b34420 0x559f10b34800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 --2- 172.21.15.62:0/4137829654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b2e620 0x559f10b2eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 -- 172.21.15.62:0/4137829654 >> 172.21.15.62:0/4137829654 conn(0x559f10a57030 msgr2=0x559f10b251d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:46.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 -- 172.21.15.62:0/4137829654 shutdown_connections 2024-06-10T00:31:46.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 -- 172.21.15.62:0/4137829654 wait complete. 2024-06-10T00:31:46.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 -- start start 2024-06-10T00:31:46.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 Processor -- start 2024-06-10T00:31:46.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2e620 0x559f10b3aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:46.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b34420 0x559f10b38700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559f10a1bf70 con 0x559f10b2efe0 2024-06-10T00:31:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559f10a777c0 con 0x559f10b2e620 2024-06-10T00:31:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.987+0000 7f0907309b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f10b3b700 con 0x559f10b34420 2024-06-10T00:31:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f0907309b80 10 client.0 _ll_register_callbacks cb 0x559f10a11560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:31:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f08f5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f08f5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b381c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41408/0 (socket says 172.21.15.62:41408) 2024-06-10T00:31:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f08f5d9b700 1 -- 172.21.15.62:0/4029563174 learned_addr learned my addr 172.21.15.62:0/4029563174 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f08f6d9d700 1 --2- 172.21.15.62:0/4029563174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b34420 0x559f10b38700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f08f659c700 1 --2- 172.21.15.62:0/4029563174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2e620 0x559f10b3aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f08f6d9d700 1 -- 172.21.15.62:0/4029563174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2e620 msgr2=0x559f10b3aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f08f6d9d700 1 --2- 172.21.15.62:0/4029563174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2e620 0x559f10b3aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f08f6d9d700 1 -- 172.21.15.62:0/4029563174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 msgr2=0x559f10b381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:46.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f08f6d9d700 1 --2- 172.21.15.62:0/4029563174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b381c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f08f6d9d700 1 -- 172.21.15.62:0/4029563174 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f10a41700 con 0x559f10b34420 2024-06-10T00:31:46.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f08f5d9b700 1 --2- 172.21.15.62:0/4029563174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:31:46.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f08f6d9d700 1 --2- 172.21.15.62:0/4029563174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b34420 0x559f10b38700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f08f0000f80 tx=0x7f08f0003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cdefbdab357a6a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:46.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f08ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:31:46.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.988+0000 7f08ddffb700 1 -- 172.21.15.62:0/4029563174 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0004450 con 0x559f10b34420 2024-06-10T00:31:46.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.989+0000 7f0907309b80 1 -- 172.21.15.62:0/4029563174 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f0004bb0 con 0x559f10b34420 2024-06-10T00:31:46.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.989+0000 7f08ddffb700 1 -- 172.21.15.62:0/4029563174 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f08f00045f0 con 0x559f10b34420 2024-06-10T00:31:46.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.989+0000 7f08ddffb700 1 -- 172.21.15.62:0/4029563174 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f08f002e050 con 0x559f10b34420 2024-06-10T00:31:46.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.989+0000 7f0907309b80 10 client.24757 fetch_fsmap learned FSMap version 29 2024-06-10T00:31:46.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.989+0000 7f0907309b80 1 -- 172.21.15.62:0/4029563174 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559f10b45df0 con 0x559f10b34420 2024-06-10T00:31:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.989+0000 7f08ddffb700 1 -- 172.21.15.62:0/4029563174 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f0004bb0 con 0x559f10b34420 2024-06-10T00:31:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.989+0000 7f08ddffb700 1 -- 172.21.15.62:0/4029563174 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f08f001db80 con 0x559f10b34420 2024-06-10T00:31:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.989+0000 7f0907309b80 10 client.24757 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:31:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.989+0000 7f0907309b80 10 client.24757 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-10T00:31:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.989+0000 7f0907309b80 20 client.24757 populate_metadata read hostname 'smithi062' 2024-06-10T00:31:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.989+0000 7f0907309b80 1 --2- 172.21.15.62:0/4029563174 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x559f10b47fe0 0x559f10b49450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.989+0000 7f0907309b80 4 client.24757 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:31:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.989+0000 7f0907309b80 1 -- 172.21.15.62:0/4029563174 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f10a88d00 con 0x559f10b47fe0 2024-06-10T00:31:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.990+0000 7f08f659c700 1 --2- 172.21.15.62:0/4029563174 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x559f10b47fe0 0x559f10b49450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.990+0000 7f08f659c700 1 --2- 172.21.15.62:0/4029563174 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x559f10b47fe0 0x559f10b49450 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3c4a9f279d043ab5 server_cookie=31becdc60474838f in_seq=0 out_seq=0 2024-06-10T00:31:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.990+0000 7f08ddffb700 10 client.24757 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:31:46.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f08ddffb700 1 -- 172.21.15.62:0/4029563174 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559f10a88d00 con 0x559f10b47fe0 2024-06-10T00:31:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f08ddffb700 10 client.24757 handle_command_reply: tid=0 2024-06-10T00:31:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f0907309b80 1 client.24757 shutdown 2024-06-10T00:31:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f0907309b80 1 -- 172.21.15.62:0/4029563174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b34420 msgr2=0x559f10b38700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f0907309b80 1 --2- 172.21.15.62:0/4029563174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b34420 0x559f10b38700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f08f0000f80 tx=0x7f08f0003c30 comp rx=0 tx=0).stop 2024-06-10T00:31:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f0907309b80 1 -- 172.21.15.62:0/4029563174 shutdown_connections 2024-06-10T00:31:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f0907309b80 1 --2- 172.21.15.62:0/4029563174 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x559f10b47fe0 0x559f10b49450 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f0907309b80 1 --2- 172.21.15.62:0/4029563174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b34420 0x559f10b38700 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f0907309b80 1 --2- 172.21.15.62:0/4029563174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2e620 0x559f10b3aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f0907309b80 1 --2- 172.21.15.62:0/4029563174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f08ddffb700 0 client.24757 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:31:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f0907309b80 1 -- 172.21.15.62:0/4029563174 >> 172.21.15.62:0/4029563174 conn(0x559f10a57030 msgr2=0x559f10b246b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f0907309b80 1 -- 172.21.15.62:0/4029563174 shutdown_connections 2024-06-10T00:31:46.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f0907309b80 1 -- 172.21.15.62:0/4029563174 wait complete. 2024-06-10T00:31:46.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:46.991+0000 7f0907309b80 20 client.24757 trim_cache size 0 max 16384 2024-06-10T00:31:47.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f0907309b80 1 Processor -- start 2024-06-10T00:31:47.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f0907309b80 1 -- start start 2024-06-10T00:31:47.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f0907309b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2e620 0x559f10b44510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:47.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f0907309b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b33980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:47.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f0907309b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b34420 0x559f10b24e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:47.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f0907309b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559f10b47790 con 0x559f10b2efe0 2024-06-10T00:31:47.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f0907309b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559f10b47e60 con 0x559f10b2e620 2024-06-10T00:31:47.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f0907309b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f10b48530 con 0x559f10b34420 2024-06-10T00:31:47.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f08f5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b33980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:47.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f08f5d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b33980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41424/0 (socket says 172.21.15.62:41424) 2024-06-10T00:31:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f08f5d9b700 1 -- 172.21.15.62:0/3858137571 learned_addr learned my addr 172.21.15.62:0/3858137571 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f08f659c700 1 --2- 172.21.15.62:0/3858137571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2e620 0x559f10b44510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f08f6d9d700 1 --2- 172.21.15.62:0/3858137571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b34420 0x559f10b24e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f08f5d9b700 1 -- 172.21.15.62:0/3858137571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b34420 msgr2=0x559f10b24e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f08f5d9b700 1 --2- 172.21.15.62:0/3858137571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b34420 0x559f10b24e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f08f5d9b700 1 -- 172.21.15.62:0/3858137571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2e620 msgr2=0x559f10b44510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f08f5d9b700 1 --2- 172.21.15.62:0/3858137571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2e620 0x559f10b44510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f08f5d9b700 1 -- 172.21.15.62:0/3858137571 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f10b44a50 con 0x559f10b2efe0 2024-06-10T00:31:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.013+0000 7f08f6d9d700 1 --2- 172.21.15.62:0/3858137571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b34420 0x559f10b24e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:31:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.014+0000 7f08f5d9b700 1 --2- 172.21.15.62:0/3858137571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b33980 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f08ec004410 tx=0x7f08ec013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3c2fa301c9f085 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.014+0000 7f08dffff700 1 -- 172.21.15.62:0/3858137571 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0114c0 con 0x559f10b2efe0 2024-06-10T00:31:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.014+0000 7f08dffff700 1 -- 172.21.15.62:0/3858137571 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ec011b40 con 0x559f10b2efe0 2024-06-10T00:31:47.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.014+0000 7f08dffff700 1 -- 172.21.15.62:0/3858137571 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec01fe30 con 0x559f10b2efe0 2024-06-10T00:31:47.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.014+0000 7f0907309b80 1 -- 172.21.15.62:0/3858137571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 msgr2=0x559f10b33980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:47.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.014+0000 7f0907309b80 1 --2- 172.21.15.62:0/3858137571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b33980 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f08ec004410 tx=0x7f08ec013000 comp rx=0 tx=0).stop 2024-06-10T00:31:47.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.014+0000 7f0907309b80 1 -- 172.21.15.62:0/3858137571 shutdown_connections 2024-06-10T00:31:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.014+0000 7f0907309b80 1 --2- 172.21.15.62:0/3858137571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b34420 0x559f10b24e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.014+0000 7f0907309b80 1 --2- 172.21.15.62:0/3858137571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2e620 0x559f10b44510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.014+0000 7f0907309b80 1 --2- 172.21.15.62:0/3858137571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b2efe0 0x559f10b33980 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.015+0000 7f0907309b80 1 -- 172.21.15.62:0/3858137571 >> 172.21.15.62:0/3858137571 conn(0x559f10a57030 msgr2=0x559f10b33270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.015+0000 7f0907309b80 1 -- 172.21.15.62:0/3858137571 shutdown_connections 2024-06-10T00:31:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.015+0000 7f0907309b80 1 -- 172.21.15.62:0/3858137571 wait complete. 2024-06-10T00:31:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.015+0000 7f0907309b80 1 -- start start 2024-06-10T00:31:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.015+0000 7f0907309b80 1 Processor -- start 2024-06-10T00:31:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.015+0000 7f0907309b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b2e620 0x559f10b24000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.015+0000 7f0907309b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2efe0 0x559f10b24540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.015+0000 7f0907309b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b34420 0x559f10b518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.015+0000 7f0907309b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559f10b492d0 con 0x559f10b34420 2024-06-10T00:31:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.015+0000 7f0907309b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559f10b48c00 con 0x559f10b2efe0 2024-06-10T00:31:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.015+0000 7f0907309b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f10b499a0 con 0x559f10b2e620 2024-06-10T00:31:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.015+0000 7f0907309b80 10 client.0 _ll_register_callbacks cb 0x559f10a11560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:31:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08f6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b34420 0x559f10b518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08f6d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b34420 0x559f10b518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41440/0 (socket says 172.21.15.62:41440) 2024-06-10T00:31:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08f6d9d700 1 -- 172.21.15.62:0/159210256 learned_addr learned my addr 172.21.15.62:0/159210256 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08f5d9b700 1 --2- 172.21.15.62:0/159210256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2efe0 0x559f10b24540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08f659c700 1 --2- 172.21.15.62:0/159210256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b2e620 0x559f10b24000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08f6d9d700 1 -- 172.21.15.62:0/159210256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b2e620 msgr2=0x559f10b24000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08f6d9d700 1 --2- 172.21.15.62:0/159210256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b2e620 0x559f10b24000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08f6d9d700 1 -- 172.21.15.62:0/159210256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2efe0 msgr2=0x559f10b24540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08f6d9d700 1 --2- 172.21.15.62:0/159210256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2efe0 0x559f10b24540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08f6d9d700 1 -- 172.21.15.62:0/159210256 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f10b51e30 con 0x559f10b34420 2024-06-10T00:31:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08f659c700 1 --2- 172.21.15.62:0/159210256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b2e620 0x559f10b24000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:31:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08f6d9d700 1 --2- 172.21.15.62:0/159210256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b34420 0x559f10b518c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f08f0025830 tx=0x7f08f000f550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad37db02273a1372 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:31:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08ddffb700 1 -- 172.21.15.62:0/159210256 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0018070 con 0x559f10b34420 2024-06-10T00:31:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08ddffb700 1 -- 172.21.15.62:0/159210256 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f08f0004d90 con 0x559f10b34420 2024-06-10T00:31:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f0907309b80 1 -- 172.21.15.62:0/159210256 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08ec011b40 con 0x559f10b34420 2024-06-10T00:31:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f08ddffb700 1 -- 172.21.15.62:0/159210256 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f08f0014bd0 con 0x559f10b34420 2024-06-10T00:31:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f0907309b80 10 client.15087 fetch_fsmap learned FSMap version 29 2024-06-10T00:31:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.016+0000 7f0907309b80 1 -- 172.21.15.62:0/159210256 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559f10b53110 con 0x559f10b34420 2024-06-10T00:31:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.017+0000 7f08ddffb700 1 -- 172.21.15.62:0/159210256 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ec011b40 con 0x559f10b34420 2024-06-10T00:31:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.017+0000 7f08ddffb700 1 -- 172.21.15.62:0/159210256 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f08f0004370 con 0x559f10b34420 2024-06-10T00:31:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.017+0000 7f0907309b80 10 client.15087 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:31:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.017+0000 7f0907309b80 10 client.15087 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-10T00:31:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.017+0000 7f0907309b80 20 client.15087 populate_metadata read hostname 'smithi062' 2024-06-10T00:31:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.017+0000 7f0907309b80 1 --2- 172.21.15.62:0/159210256 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x559f10bb7e50 0x559f10bb6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.017+0000 7f0907309b80 4 client.15087 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:31:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.017+0000 7f0907309b80 1 -- 172.21.15.62:0/159210256 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "status"}) -- 0x559f10a83280 con 0x559f10bb7e50 2024-06-10T00:31:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.017+0000 7f08f659c700 1 --2- 172.21.15.62:0/159210256 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x559f10bb7e50 0x559f10bb6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f08f659c700 1 --2- 172.21.15.62:0/159210256 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x559f10bb7e50 0x559f10bb6b60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e9fecf5e87c90ef9 server_cookie=3bd59df99b9758f1 in_seq=0 out_seq=0 2024-06-10T00:31:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f08ddffb700 10 client.15087 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:31:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f08ddffb700 1 -- 172.21.15.62:0/159210256 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x559f10a83280 con 0x559f10bb7e50 2024-06-10T00:31:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f08ddffb700 10 client.15087 handle_command_reply: tid=0 2024-06-10T00:31:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f0907309b80 1 client.15087 shutdown 2024-06-10T00:31:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f0907309b80 1 -- 172.21.15.62:0/159210256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b34420 msgr2=0x559f10b518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f0907309b80 1 --2- 172.21.15.62:0/159210256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b34420 0x559f10b518c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f08f0025830 tx=0x7f08f000f550 comp rx=0 tx=0).stop 2024-06-10T00:31:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f0907309b80 1 -- 172.21.15.62:0/159210256 shutdown_connections 2024-06-10T00:31:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f0907309b80 1 --2- 172.21.15.62:0/159210256 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x559f10bb7e50 0x559f10bb6b60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f0907309b80 1 --2- 172.21.15.62:0/159210256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559f10b34420 0x559f10b518c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f0907309b80 1 --2- 172.21.15.62:0/159210256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f10b2e620 0x559f10b24000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f08ddffb700 0 client.15087 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:31:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f0907309b80 1 --2- 172.21.15.62:0/159210256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f10b2efe0 0x559f10b24540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f0907309b80 1 -- 172.21.15.62:0/159210256 >> 172.21.15.62:0/159210256 conn(0x559f10a57030 msgr2=0x559f10b31e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f0907309b80 1 -- 172.21.15.62:0/159210256 shutdown_connections 2024-06-10T00:31:47.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f0907309b80 1 -- 172.21.15.62:0/159210256 wait complete. 2024-06-10T00:31:47.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.018+0000 7f0907309b80 20 client.15087 trim_cache size 0 max 16384 2024-06-10T00:31:47.025 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:31:47.025 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:31:47.025 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 4, 2024-06-10T00:31:47.025 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24418, 2024-06-10T00:31:47.025 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:31:47.025 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:31:47.025 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:31:47.026 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 122.14546792, 2024-06-10T00:31:47.026 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 29, 2024-06-10T00:31:47.026 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 99, 2024-06-10T00:31:47.026 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T00:31:47.026 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 224.603336558 2024-06-10T00:31:47.026 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:31:47.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.019+0000 7f08fca86700 1 -- 172.21.15.62:0/1070208454 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f08e00a2c20 msgr2=0x7f08e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:47.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.019+0000 7f08fca86700 1 --2- 172.21.15.62:0/1070208454 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f08e00a2c20 0x7f08e00a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f08f807a090 tx=0x7f08ec00b3c0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.019+0000 7f08fca86700 1 -- 172.21.15.62:0/1070208454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f8102740 msgr2=0x7f08f819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:47.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.019+0000 7f08fca86700 1 --2- 172.21.15.62:0/1070208454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f8102740 0x7f08f819e7c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f08e8006010 tx=0x7f08e8009f80 comp rx=0 tx=0).stop 2024-06-10T00:31:47.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.019+0000 7f08fca86700 1 -- 172.21.15.62:0/1070208454 shutdown_connections 2024-06-10T00:31:47.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.019+0000 7f08fca86700 1 --2- 172.21.15.62:0/1070208454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f8102740 0x7f08f819e7c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.019+0000 7f08fca86700 1 --2- 172.21.15.62:0/1070208454 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f08e00a2c20 0x7f08e00a50e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.019+0000 7f08fca86700 1 --2- 172.21.15.62:0/1070208454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08f8107840 0x7f08f8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.019+0000 7f08fca86700 1 --2- 172.21.15.62:0/1070208454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08f81030f0 0x7f08f819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:47.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.019+0000 7f08fca86700 1 -- 172.21.15.62:0/1070208454 >> 172.21.15.62:0/1070208454 conn(0x7f08f80795f0 msgr2=0x7f08f8100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:47.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.019+0000 7f08fca86700 1 -- 172.21.15.62:0/1070208454 shutdown_connections 2024-06-10T00:31:47.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:47.019+0000 7f08fca86700 1 -- 172.21.15.62:0/1070208454 wait complete. 2024-06-10T00:31:47.758 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-10T00:31:48.009 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:48.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:47 smithi105 ceph-mon[36321]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:31:48.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:47 smithi062 ceph-mon[32390]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:31:48.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:47 smithi138 ceph-mon[36549]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:31:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.658+0000 7f0f801fe700 1 -- 172.21.15.62:0/644615131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f78107b60 msgr2=0x7f0f7810e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.658+0000 7f0f801fe700 1 --2- 172.21.15.62:0/644615131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f78107b60 0x7f0f7810e320 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000b0d0 tx=0x7f0f7000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.658+0000 7f0f801fe700 1 -- 172.21.15.62:0/644615131 shutdown_connections 2024-06-10T00:31:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.658+0000 7f0f801fe700 1 --2- 172.21.15.62:0/644615131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f78107b60 0x7f0f7810e320 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.658+0000 7f0f801fe700 1 --2- 172.21.15.62:0/644615131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f78105330 0x7f0f78105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.658+0000 7f0f801fe700 1 --2- 172.21.15.62:0/644615131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f781071a0 0x7f0f78107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.658+0000 7f0f801fe700 1 -- 172.21.15.62:0/644615131 >> 172.21.15.62:0/644615131 conn(0x7f0f780766b0 msgr2=0x7f0f78076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.658+0000 7f0f801fe700 1 -- 172.21.15.62:0/644615131 shutdown_connections 2024-06-10T00:31:49.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.658+0000 7f0f801fe700 1 -- 172.21.15.62:0/644615131 wait complete. 2024-06-10T00:31:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.659+0000 7f0f801fe700 1 Processor -- start 2024-06-10T00:31:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.659+0000 7f0f801fe700 1 -- start start 2024-06-10T00:31:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f801fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f78105330 0x7f0f7819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f801fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f781071a0 0x7f0f7819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f801fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f78107b60 0x7f0f78196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f801fe700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0f781106f0 con 0x7f0f78105330 2024-06-10T00:31:49.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f801fe700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0f78110570 con 0x7f0f781071a0 2024-06-10T00:31:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f801fe700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f78110870 con 0x7f0f78107b60 2024-06-10T00:31:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f7df9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f78105330 0x7f0f7819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f7df9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f78105330 0x7f0f7819c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37190/0 (socket says 172.21.15.62:37190) 2024-06-10T00:31:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f7df9a700 1 -- 172.21.15.62:0/2662531640 learned_addr learned my addr 172.21.15.62:0/2662531640 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f7d799700 1 --2- 172.21.15.62:0/2662531640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f781071a0 0x7f0f7819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f7e79b700 1 --2- 172.21.15.62:0/2662531640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f78107b60 0x7f0f78196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f7df9a700 1 -- 172.21.15.62:0/2662531640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f78107b60 msgr2=0x7f0f78196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f7df9a700 1 --2- 172.21.15.62:0/2662531640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f78107b60 0x7f0f78196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f7df9a700 1 -- 172.21.15.62:0/2662531640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f781071a0 msgr2=0x7f0f7819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f7df9a700 1 --2- 172.21.15.62:0/2662531640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f781071a0 0x7f0f7819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f7df9a700 1 -- 172.21.15.62:0/2662531640 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f78197050 con 0x7f0f78105330 2024-06-10T00:31:49.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f7df9a700 1 --2- 172.21.15.62:0/2662531640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f78105330 0x7f0f7819c6a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400e9a0 tx=0x7f0f7400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:49.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.660+0000 7f0f6effd700 1 -- 172.21.15.62:0/2662531640 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7400ca10 con 0x7f0f78105330 2024-06-10T00:31:49.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.661+0000 7f0f801fe700 1 -- 172.21.15.62:0/2662531640 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f78197340 con 0x7f0f78105330 2024-06-10T00:31:49.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.661+0000 7f0f6effd700 1 -- 172.21.15.62:0/2662531640 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f7400cbb0 con 0x7f0f78105330 2024-06-10T00:31:49.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.661+0000 7f0f6effd700 1 -- 172.21.15.62:0/2662531640 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74018680 con 0x7f0f78105330 2024-06-10T00:31:49.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.661+0000 7f0f801fe700 1 -- 172.21.15.62:0/2662531640 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f781a3500 con 0x7f0f78105330 2024-06-10T00:31:49.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.663+0000 7f0f6effd700 1 -- 172.21.15.62:0/2662531640 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0f74010430 con 0x7f0f78105330 2024-06-10T00:31:49.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.664+0000 7f0f6effd700 1 --2- 172.21.15.62:0/2662531640 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0f640a4e30 0x7f0f640a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.664+0000 7f0f7d799700 1 --2- 172.21.15.62:0/2662531640 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0f640a4e30 0x7f0f640a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.664+0000 7f0f6effd700 1 -- 172.21.15.62:0/2662531640 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f0f740cf630 con 0x7f0f78105330 2024-06-10T00:31:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.665+0000 7f0f7d799700 1 --2- 172.21.15.62:0/2662531640 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0f640a4e30 0x7f0f640a72f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0f78076bd0 tx=0x7f0f68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.665+0000 7f0f8aa81b80 1 Processor -- start 2024-06-10T00:31:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.665+0000 7f0f8aa81b80 1 -- start start 2024-06-10T00:31:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.665+0000 7f0f8aa81b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a83f10 0x55af96a842f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.665+0000 7f0f8aa81b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a84830 0x55af96a7e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.665+0000 7f0f8aa81b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a7eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.665+0000 7f0f8aa81b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55af96970f10 con 0x55af96a7e750 2024-06-10T00:31:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.665+0000 7f0f8aa81b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55af969cf800 con 0x55af96a83f10 2024-06-10T00:31:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.665+0000 7f0f8aa81b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55af96937410 con 0x55af96a84830 2024-06-10T00:31:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.665+0000 7f0f7e79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a7eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.665+0000 7f0f7d799700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a84830 0x55af96a7e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.665+0000 7f0f7df9a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a83f10 0x55af96a842f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.665+0000 7f0f7e79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a7eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37200/0 (socket says 172.21.15.62:37200) 2024-06-10T00:31:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.665+0000 7f0f7e79b700 1 -- 172.21.15.62:0/3983251731 learned_addr learned my addr 172.21.15.62:0/3983251731 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:49.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.666+0000 7f0f7d799700 1 -- 172.21.15.62:0/3983251731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a83f10 msgr2=0x55af96a842f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:49.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.666+0000 7f0f7d799700 1 --2- 172.21.15.62:0/3983251731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a83f10 0x55af96a842f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.666+0000 7f0f7d799700 1 -- 172.21.15.62:0/3983251731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 msgr2=0x55af96a7eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:49.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.666+0000 7f0f7d799700 1 --2- 172.21.15.62:0/3983251731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a7eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.666+0000 7f0f7d799700 1 -- 172.21.15.62:0/3983251731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9691e880 con 0x55af96a84830 2024-06-10T00:31:49.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.666+0000 7f0f7d799700 1 --2- 172.21.15.62:0/3983251731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a84830 0x55af96a7e210 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f68011c90 tx=0x7f0f68014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=632a8b6a6fa5389a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:49.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.666+0000 7f0f63fff700 1 -- 172.21.15.62:0/3983251731 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68004230 con 0x55af96a84830 2024-06-10T00:31:49.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.666+0000 7f0f63fff700 1 -- 172.21.15.62:0/3983251731 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f6800ed20 con 0x55af96a84830 2024-06-10T00:31:49.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.666+0000 7f0f63fff700 1 -- 172.21.15.62:0/3983251731 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68026470 con 0x55af96a84830 2024-06-10T00:31:49.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.667+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3983251731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a84830 msgr2=0x55af96a7e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:49.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.667+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/3983251731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a84830 0x55af96a7e210 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f68011c90 tx=0x7f0f68014000 comp rx=0 tx=0).stop 2024-06-10T00:31:49.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.667+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3983251731 shutdown_connections 2024-06-10T00:31:49.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.667+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/3983251731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a7eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.667+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/3983251731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a83f10 0x55af96a842f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.667+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/3983251731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a84830 0x55af96a7e210 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.667+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3983251731 >> 172.21.15.62:0/3983251731 conn(0x55af969ab030 msgr2=0x55af96a791d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:49.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.667+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3983251731 shutdown_connections 2024-06-10T00:31:49.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.667+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3983251731 wait complete. 2024-06-10T00:31:49.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.667+0000 7f0f8aa81b80 1 -- start start 2024-06-10T00:31:49.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.667+0000 7f0f8aa81b80 1 Processor -- start 2024-06-10T00:31:49.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f8aa81b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a8a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f8aa81b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a83f10 0x55af96a87890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f8aa81b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a84830 0x55af96a87dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f8aa81b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55af9696ff70 con 0x55af96a7e750 2024-06-10T00:31:49.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f8aa81b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55af969cb7c0 con 0x55af96a83f10 2024-06-10T00:31:49.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f8aa81b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55af96a8b210 con 0x55af96a84830 2024-06-10T00:31:49.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f8aa81b80 10 client.0 _ll_register_callbacks cb 0x55af96965560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:31:49.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f7df9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a8a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f7df9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a8a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37210/0 (socket says 172.21.15.62:37210) 2024-06-10T00:31:49.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f7df9a700 1 -- 172.21.15.62:0/637677168 learned_addr learned my addr 172.21.15.62:0/637677168 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:49.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f7d799700 1 --2- 172.21.15.62:0/637677168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a83f10 0x55af96a87890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f7e79b700 1 --2- 172.21.15.62:0/637677168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a84830 0x55af96a87dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f7df9a700 1 -- 172.21.15.62:0/637677168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a84830 msgr2=0x55af96a87dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f7df9a700 1 --2- 172.21.15.62:0/637677168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a84830 0x55af96a87dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f7df9a700 1 -- 172.21.15.62:0/637677168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a83f10 msgr2=0x55af96a87890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f7df9a700 1 --2- 172.21.15.62:0/637677168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a83f10 0x55af96a87890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.668+0000 7f0f7df9a700 1 -- 172.21.15.62:0/637677168 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af96995700 con 0x55af96a7e750 2024-06-10T00:31:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f7df9a700 1 --2- 172.21.15.62:0/637677168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a8a590 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0f740ccd60 tx=0x7f0f740cf580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1446b6040bd3503 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f617fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:31:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f617fa700 1 -- 172.21.15.62:0/637677168 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f740a4730 con 0x55af96a7e750 2024-06-10T00:31:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f617fa700 1 -- 172.21.15.62:0/637677168 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f0f74099dc0 con 0x55af96a7e750 2024-06-10T00:31:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/637677168 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f6800ed20 con 0x55af96a7e750 2024-06-10T00:31:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f617fa700 1 -- 172.21.15.62:0/637677168 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f7400cd60 con 0x55af96a7e750 2024-06-10T00:31:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f8aa81b80 10 client.15099 fetch_fsmap learned FSMap version 29 2024-06-10T00:31:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/637677168 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55af96a95990 con 0x55af96a7e750 2024-06-10T00:31:49.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f617fa700 1 -- 172.21.15.62:0/637677168 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f6800ed20 con 0x55af96a7e750 2024-06-10T00:31:49.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f617fa700 1 -- 172.21.15.62:0/637677168 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0f7409af10 con 0x55af96a7e750 2024-06-10T00:31:49.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f8aa81b80 10 client.15099 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:31:49.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f8aa81b80 10 client.15099 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T00:31:49.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f8aa81b80 20 client.15099 populate_metadata read hostname 'smithi062' 2024-06-10T00:31:49.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/637677168 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55af96a97f30 0x55af96a993a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f8aa81b80 4 client.15099 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:31:49.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.669+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/637677168 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af969dcd00 con 0x55af96a97f30 2024-06-10T00:31:49.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.670+0000 7f0f7d799700 1 --2- 172.21.15.62:0/637677168 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55af96a97f30 0x55af96a993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.670+0000 7f0f7d799700 1 --2- 172.21.15.62:0/637677168 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55af96a97f30 0x55af96a993a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=517dadacde351532 server_cookie=b89a935476159e79 in_seq=0 out_seq=0 2024-06-10T00:31:49.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.670+0000 7f0f617fa700 10 client.15099 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:31:49.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f617fa700 1 -- 172.21.15.62:0/637677168 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55af969dcd00 con 0x55af96a97f30 2024-06-10T00:31:49.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f617fa700 10 client.15099 handle_command_reply: tid=0 2024-06-10T00:31:49.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f8aa81b80 1 client.15099 shutdown 2024-06-10T00:31:49.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/637677168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 msgr2=0x55af96a8a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:49.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/637677168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a8a590 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0f740ccd60 tx=0x7f0f740cf580 comp rx=0 tx=0).stop 2024-06-10T00:31:49.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/637677168 shutdown_connections 2024-06-10T00:31:49.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/637677168 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55af96a97f30 0x55af96a993a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/637677168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a8a590 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/637677168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a84830 0x55af96a87dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f617fa700 0 client.15099 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:31:49.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/637677168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a83f10 0x55af96a87890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/637677168 >> 172.21.15.62:0/637677168 conn(0x55af969ab030 msgr2=0x55af96a786b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:49.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/637677168 shutdown_connections 2024-06-10T00:31:49.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/637677168 wait complete. 2024-06-10T00:31:49.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.671+0000 7f0f8aa81b80 20 client.15099 trim_cache size 0 max 16384 2024-06-10T00:31:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.693+0000 7f0f8aa81b80 1 Processor -- start 2024-06-10T00:31:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.693+0000 7f0f8aa81b80 1 -- start start 2024-06-10T00:31:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.693+0000 7f0f8aa81b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a7e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.693+0000 7f0f8aa81b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a83f10 0x55af96a85320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f8aa81b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a84830 0x55af96a8a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f8aa81b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55af96a976e0 con 0x55af96a7e750 2024-06-10T00:31:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f8aa81b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55af96a97db0 con 0x55af96a84830 2024-06-10T00:31:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f8aa81b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55af96a98480 con 0x55af96a83f10 2024-06-10T00:31:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f7df9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a7e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f7df9a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a7e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37224/0 (socket says 172.21.15.62:37224) 2024-06-10T00:31:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f7df9a700 1 -- 172.21.15.62:0/3064465850 learned_addr learned my addr 172.21.15.62:0/3064465850 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f7d799700 1 --2- 172.21.15.62:0/3064465850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a83f10 0x55af96a85320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f7df9a700 1 -- 172.21.15.62:0/3064465850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a83f10 msgr2=0x55af96a85320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f7e79b700 1 --2- 172.21.15.62:0/3064465850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a84830 0x55af96a8a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f7df9a700 1 --2- 172.21.15.62:0/3064465850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a83f10 0x55af96a85320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f7df9a700 1 -- 172.21.15.62:0/3064465850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a84830 msgr2=0x55af96a8a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f7df9a700 1 --2- 172.21.15.62:0/3064465850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a84830 0x55af96a8a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f7df9a700 1 -- 172.21.15.62:0/3064465850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af96a85860 con 0x55af96a7e750 2024-06-10T00:31:49.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f7df9a700 1 --2- 172.21.15.62:0/3064465850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a7e210 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0f74099d70 tx=0x7f0f7400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6220a6c212c726fa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:49.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f637fe700 1 -- 172.21.15.62:0/3064465850 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7409b890 con 0x55af96a7e750 2024-06-10T00:31:49.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f637fe700 1 -- 172.21.15.62:0/3064465850 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f7409a980 con 0x55af96a7e750 2024-06-10T00:31:49.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.694+0000 7f0f637fe700 1 -- 172.21.15.62:0/3064465850 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7409b600 con 0x55af96a7e750 2024-06-10T00:31:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.695+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3064465850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 msgr2=0x55af96a7e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.695+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/3064465850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a7e210 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0f74099d70 tx=0x7f0f7400cf00 comp rx=0 tx=0).stop 2024-06-10T00:31:49.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.695+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3064465850 shutdown_connections 2024-06-10T00:31:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.695+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/3064465850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a7e750 0x55af96a7e210 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.695+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/3064465850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a84830 0x55af96a8a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.695+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/3064465850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a83f10 0x55af96a85320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.695+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3064465850 >> 172.21.15.62:0/3064465850 conn(0x55af969ab030 msgr2=0x55af96a84c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.695+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3064465850 shutdown_connections 2024-06-10T00:31:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.695+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3064465850 wait complete. 2024-06-10T00:31:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.695+0000 7f0f8aa81b80 1 -- start start 2024-06-10T00:31:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f8aa81b80 1 Processor -- start 2024-06-10T00:31:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f8aa81b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a7e750 0x55af96aa2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f8aa81b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a83f10 0x55af96aa2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f8aa81b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a84830 0x55af96b067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f8aa81b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55af96a99220 con 0x55af96a84830 2024-06-10T00:31:49.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f8aa81b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55af96a98b50 con 0x55af96a7e750 2024-06-10T00:31:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f8aa81b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55af96a998f0 con 0x55af96a83f10 2024-06-10T00:31:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f8aa81b80 10 client.0 _ll_register_callbacks cb 0x55af96965560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:31:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f7e79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a84830 0x55af96b067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f7e79b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a84830 0x55af96b067f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37240/0 (socket says 172.21.15.62:37240) 2024-06-10T00:31:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f7e79b700 1 -- 172.21.15.62:0/3555565298 learned_addr learned my addr 172.21.15.62:0/3555565298 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f7df9a700 1 --2- 172.21.15.62:0/3555565298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a7e750 0x55af96aa2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f7d799700 1 --2- 172.21.15.62:0/3555565298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a83f10 0x55af96aa2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f7df9a700 1 -- 172.21.15.62:0/3555565298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a83f10 msgr2=0x55af96aa2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f7df9a700 1 --2- 172.21.15.62:0/3555565298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a83f10 0x55af96aa2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.696+0000 7f0f7df9a700 1 -- 172.21.15.62:0/3555565298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a84830 msgr2=0x55af96b067f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.697+0000 7f0f7df9a700 1 --2- 172.21.15.62:0/3555565298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a84830 0x55af96b067f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.697+0000 7f0f7df9a700 1 -- 172.21.15.62:0/3555565298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af96b06d30 con 0x55af96a7e750 2024-06-10T00:31:49.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.697+0000 7f0f7e79b700 1 --2- 172.21.15.62:0/3555565298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a84830 0x55af96b067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:31:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.697+0000 7f0f7df9a700 1 --2- 172.21.15.62:0/3555565298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a7e750 0x55af96aa2420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0f74098490 tx=0x7f0f7409af10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35a66595a815980b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.697+0000 7f0f617fa700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:31:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.697+0000 7f0f617fa700 1 -- 172.21.15.62:0/3555565298 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f740a42a0 con 0x55af96a7e750 2024-06-10T00:31:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.697+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3555565298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f7409a980 con 0x55af96a7e750 2024-06-10T00:31:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.697+0000 7f0f617fa700 1 -- 172.21.15.62:0/3555565298 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f0f740a4440 con 0x55af96a7e750 2024-06-10T00:31:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.697+0000 7f0f617fa700 1 -- 172.21.15.62:0/3555565298 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f74010720 con 0x55af96a7e750 2024-06-10T00:31:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.697+0000 7f0f8aa81b80 10 client.24791 fetch_fsmap learned FSMap version 29 2024-06-10T00:31:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.697+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3555565298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55af96b07ee0 con 0x55af96a7e750 2024-06-10T00:31:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.697+0000 7f0f617fa700 1 -- 172.21.15.62:0/3555565298 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f7409a980 con 0x55af96a7e750 2024-06-10T00:31:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.697+0000 7f0f617fa700 1 -- 172.21.15.62:0/3555565298 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0f740a1000 con 0x55af96a7e750 2024-06-10T00:31:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.698+0000 7f0f8aa81b80 10 client.24791 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:31:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.698+0000 7f0f8aa81b80 10 client.24791 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T00:31:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.698+0000 7f0f8aa81b80 20 client.24791 populate_metadata read hostname 'smithi062' 2024-06-10T00:31:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.698+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/3555565298 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55af96b0c380 0x55af96b0b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.698+0000 7f0f8aa81b80 4 client.24791 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:31:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.698+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3555565298 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "status"}) -- 0x55af969d7280 con 0x55af96b0c380 2024-06-10T00:31:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.698+0000 7f0f7d799700 1 --2- 172.21.15.62:0/3555565298 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55af96b0c380 0x55af96b0b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.698+0000 7f0f7d799700 1 --2- 172.21.15.62:0/3555565298 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55af96b0c380 0x55af96b0b090 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=89e5ff4f8e23d479 server_cookie=a5c6b0ca17e92fa0 in_seq=0 out_seq=0 2024-06-10T00:31:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.698+0000 7f0f617fa700 10 client.24791 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:31:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.698+0000 7f0f617fa700 1 -- 172.21.15.62:0/3555565298 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55af969d7280 con 0x55af96b0c380 2024-06-10T00:31:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.698+0000 7f0f617fa700 10 client.24791 handle_command_reply: tid=0 2024-06-10T00:31:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.698+0000 7f0f8aa81b80 1 client.24791 shutdown 2024-06-10T00:31:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.699+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3555565298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a7e750 msgr2=0x55af96aa2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.699+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/3555565298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a7e750 0x55af96aa2420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0f74098490 tx=0x7f0f7409af10 comp rx=0 tx=0).stop 2024-06-10T00:31:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.699+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3555565298 shutdown_connections 2024-06-10T00:31:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.699+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/3555565298 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55af96b0c380 0x55af96b0b090 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.699+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/3555565298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55af96a84830 0x55af96b067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.699+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/3555565298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af96a7e750 0x55af96aa2420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.699+0000 7f0f617fa700 0 client.24791 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:31:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.699+0000 7f0f8aa81b80 1 --2- 172.21.15.62:0/3555565298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af96a83f10 0x55af96aa2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.699+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3555565298 >> 172.21.15.62:0/3555565298 conn(0x55af969ab030 msgr2=0x55af96a829f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.699+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3555565298 shutdown_connections 2024-06-10T00:31:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.699+0000 7f0f8aa81b80 1 -- 172.21.15.62:0/3555565298 wait complete. 2024-06-10T00:31:49.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.699+0000 7f0f8aa81b80 20 client.24791 trim_cache size 0 max 16384 2024-06-10T00:31:49.704 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:31:49.705 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:31:49.705 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 3, 2024-06-10T00:31:49.705 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24424, 2024-06-10T00:31:49.705 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:31:49.705 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:31:49.705 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:31:49.705 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 125.836240894, 2024-06-10T00:31:49.705 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 29, 2024-06-10T00:31:49.705 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 99, 2024-06-10T00:31:49.705 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T00:31:49.705 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 224.35242242800001 2024-06-10T00:31:49.705 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:31:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.700+0000 7f0f801fe700 1 -- 172.21.15.62:0/2662531640 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0f640a4e30 msgr2=0x7f0f640a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.700+0000 7f0f801fe700 1 --2- 172.21.15.62:0/2662531640 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0f640a4e30 0x7f0f640a72f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0f78076bd0 tx=0x7f0f68009260 comp rx=0 tx=0).stop 2024-06-10T00:31:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.700+0000 7f0f801fe700 1 -- 172.21.15.62:0/2662531640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f78105330 msgr2=0x7f0f7819c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.700+0000 7f0f801fe700 1 --2- 172.21.15.62:0/2662531640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f78105330 0x7f0f7819c6a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400e9a0 tx=0x7f0f7400ecb0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.700+0000 7f0f801fe700 1 -- 172.21.15.62:0/2662531640 shutdown_connections 2024-06-10T00:31:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.700+0000 7f0f801fe700 1 --2- 172.21.15.62:0/2662531640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f78105330 0x7f0f7819c6a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.700+0000 7f0f801fe700 1 --2- 172.21.15.62:0/2662531640 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0f640a4e30 0x7f0f640a72f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.700+0000 7f0f801fe700 1 --2- 172.21.15.62:0/2662531640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f78107b60 0x7f0f78196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.700+0000 7f0f801fe700 1 --2- 172.21.15.62:0/2662531640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f781071a0 0x7f0f7819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.700+0000 7f0f801fe700 1 -- 172.21.15.62:0/2662531640 >> 172.21.15.62:0/2662531640 conn(0x7f0f780766b0 msgr2=0x7f0f780fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.700+0000 7f0f801fe700 1 -- 172.21.15.62:0/2662531640 shutdown_connections 2024-06-10T00:31:49.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:49.700+0000 7f0f801fe700 1 -- 172.21.15.62:0/2662531640 wait complete. 2024-06-10T00:31:50.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:49 smithi105 ceph-mon[36321]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:31:50.072 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:49 smithi062 ceph-mon[32390]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:31:50.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:49 smithi138 ceph-mon[36549]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:31:50.387 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-10T00:31:50.640 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:52.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:51 smithi062 ceph-mon[32390]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:31:52.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:51 smithi105 ceph-mon[36321]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:31:52.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:51 smithi138 ceph-mon[36549]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:31:52.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.252+0000 7ff58b403700 1 -- 172.21.15.62:0/1913454393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff584102740 msgr2=0x7ff584102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:52.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.252+0000 7ff58b403700 1 --2- 172.21.15.62:0/1913454393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff584102740 0x7ff584102b20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff5800099c0 tx=0x7ff580009cd0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.253+0000 7ff58b403700 1 -- 172.21.15.62:0/1913454393 shutdown_connections 2024-06-10T00:31:52.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.253+0000 7ff58b403700 1 --2- 172.21.15.62:0/1913454393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff584102740 0x7ff584102b20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.253+0000 7ff58b403700 1 --2- 172.21.15.62:0/1913454393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff584107840 0x7ff58410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.253+0000 7ff58b403700 1 --2- 172.21.15.62:0/1913454393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5841030f0 0x7ff584107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.253+0000 7ff58b403700 1 -- 172.21.15.62:0/1913454393 >> 172.21.15.62:0/1913454393 conn(0x7ff5840795f0 msgr2=0x7ff584079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:52.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.253+0000 7ff58b403700 1 -- 172.21.15.62:0/1913454393 shutdown_connections 2024-06-10T00:31:52.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.253+0000 7ff58b403700 1 -- 172.21.15.62:0/1913454393 wait complete. 2024-06-10T00:31:52.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.253+0000 7ff58b403700 1 Processor -- start 2024-06-10T00:31:52.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.253+0000 7ff58b403700 1 -- start start 2024-06-10T00:31:52.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff58b403700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff584102740 0x7ff58419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff58b403700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5841030f0 0x7ff58419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff58b403700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff584107840 0x7ff584198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff58b403700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff584112850 con 0x7ff584107840 2024-06-10T00:31:52.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff58b403700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5841126d0 con 0x7ff584102740 2024-06-10T00:31:52.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff58b403700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5841129d0 con 0x7ff5841030f0 2024-06-10T00:31:52.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff5899a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff584107840 0x7ff584198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff5899a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff584107840 0x7ff584198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37258/0 (socket says 172.21.15.62:37258) 2024-06-10T00:31:52.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff5899a0700 1 -- 172.21.15.62:0/3424452156 learned_addr learned my addr 172.21.15.62:0/3424452156 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:52.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff58899e700 1 --2- 172.21.15.62:0/3424452156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5841030f0 0x7ff58419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff58919f700 1 --2- 172.21.15.62:0/3424452156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff584102740 0x7ff58419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff5899a0700 1 -- 172.21.15.62:0/3424452156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5841030f0 msgr2=0x7ff58419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:52.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff5899a0700 1 --2- 172.21.15.62:0/3424452156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5841030f0 0x7ff58419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff5899a0700 1 -- 172.21.15.62:0/3424452156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff584102740 msgr2=0x7ff58419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:52.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff5899a0700 1 --2- 172.21.15.62:0/3424452156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff584102740 0x7ff58419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.254+0000 7ff5899a0700 1 -- 172.21.15.62:0/3424452156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5841992b0 con 0x7ff584107840 2024-06-10T00:31:52.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.255+0000 7ff5899a0700 1 --2- 172.21.15.62:0/3424452156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff584107840 0x7ff584198b50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00b970 tx=0x7ff57c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:52.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.255+0000 7ff57a7fc700 1 -- 172.21.15.62:0/3424452156 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00d860 con 0x7ff584107840 2024-06-10T00:31:52.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.255+0000 7ff57a7fc700 1 -- 172.21.15.62:0/3424452156 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff57c00e430 con 0x7ff584107840 2024-06-10T00:31:52.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.255+0000 7ff58b403700 1 -- 172.21.15.62:0/3424452156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5841995a0 con 0x7ff584107840 2024-06-10T00:31:52.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.255+0000 7ff57a7fc700 1 -- 172.21.15.62:0/3424452156 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c017660 con 0x7ff584107840 2024-06-10T00:31:52.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.255+0000 7ff58b403700 1 -- 172.21.15.62:0/3424452156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5841a5810 con 0x7ff584107840 2024-06-10T00:31:52.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.257+0000 7ff57a7fc700 1 -- 172.21.15.62:0/3424452156 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff57c00e5e0 con 0x7ff584107840 2024-06-10T00:31:52.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.258+0000 7ff57a7fc700 1 --2- 172.21.15.62:0/3424452156 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff5700a2c20 0x7ff5700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.258+0000 7ff57a7fc700 1 -- 172.21.15.62:0/3424452156 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7ff57c0cefa0 con 0x7ff584107840 2024-06-10T00:31:52.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.258+0000 7ff58919f700 1 --2- 172.21.15.62:0/3424452156 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff5700a2c20 0x7ff5700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.258+0000 7ff58919f700 1 --2- 172.21.15.62:0/3424452156 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff5700a2c20 0x7ff5700a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff580000c00 tx=0x7ff580005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:52.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.258+0000 7ff595c86b80 1 Processor -- start 2024-06-10T00:31:52.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff595c86b80 1 -- start start 2024-06-10T00:31:52.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff595c86b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a6a20 0x5632892a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff595c86b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a7340 0x5632892a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff595c86b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a12e0 0x5632892a1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff595c86b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5632891ee7c0 con 0x5632892a6a20 2024-06-10T00:31:52.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff595c86b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5632891966d0 con 0x5632892a7340 2024-06-10T00:31:52.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff595c86b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5632891ea780 con 0x5632892a12e0 2024-06-10T00:31:52.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff58919f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a6a20 0x5632892a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff58919f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a6a20 0x5632892a6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37274/0 (socket says 172.21.15.62:37274) 2024-06-10T00:31:52.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff58919f700 1 -- 172.21.15.62:0/4240258819 learned_addr learned my addr 172.21.15.62:0/4240258819 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:52.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff5899a0700 1 --2- 172.21.15.62:0/4240258819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a12e0 0x5632892a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff58919f700 1 -- 172.21.15.62:0/4240258819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a12e0 msgr2=0x5632892a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:52.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff58919f700 1 --2- 172.21.15.62:0/4240258819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a12e0 0x5632892a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff58919f700 1 -- 172.21.15.62:0/4240258819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a7340 msgr2=0x5632892a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:52.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff58919f700 1 --2- 172.21.15.62:0/4240258819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a7340 0x5632892a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff58919f700 1 -- 172.21.15.62:0/4240258819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328918cf00 con 0x5632892a6a20 2024-06-10T00:31:52.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.259+0000 7ff58919f700 1 --2- 172.21.15.62:0/4240258819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a6a20 0x5632892a6e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff580015000 tx=0x7ff580020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c495a32b65c8a69 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:52.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.260+0000 7ff56f7fe700 1 -- 172.21.15.62:0/4240258819 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58000bdc0 con 0x5632892a6a20 2024-06-10T00:31:52.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.260+0000 7ff56f7fe700 1 -- 172.21.15.62:0/4240258819 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff58000fd50 con 0x5632892a6a20 2024-06-10T00:31:52.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.260+0000 7ff56f7fe700 1 -- 172.21.15.62:0/4240258819 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58002f400 con 0x5632892a6a20 2024-06-10T00:31:52.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.260+0000 7ff595c86b80 1 -- 172.21.15.62:0/4240258819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a6a20 msgr2=0x5632892a6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:52.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.260+0000 7ff595c86b80 1 --2- 172.21.15.62:0/4240258819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a6a20 0x5632892a6e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff580015000 tx=0x7ff580020000 comp rx=0 tx=0).stop 2024-06-10T00:31:52.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.261+0000 7ff595c86b80 1 -- 172.21.15.62:0/4240258819 shutdown_connections 2024-06-10T00:31:52.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.261+0000 7ff595c86b80 1 --2- 172.21.15.62:0/4240258819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a6a20 0x5632892a6e00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.261+0000 7ff595c86b80 1 --2- 172.21.15.62:0/4240258819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a12e0 0x5632892a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.261+0000 7ff595c86b80 1 --2- 172.21.15.62:0/4240258819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a7340 0x5632892a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.261+0000 7ff595c86b80 1 -- 172.21.15.62:0/4240258819 >> 172.21.15.62:0/4240258819 conn(0x5632891c7c70 msgr2=0x563289297800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:52.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.261+0000 7ff595c86b80 1 -- 172.21.15.62:0/4240258819 shutdown_connections 2024-06-10T00:31:52.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.261+0000 7ff595c86b80 1 -- 172.21.15.62:0/4240258819 wait complete. 2024-06-10T00:31:52.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.261+0000 7ff595c86b80 1 -- start start 2024-06-10T00:31:52.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.261+0000 7ff595c86b80 1 Processor -- start 2024-06-10T00:31:52.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.261+0000 7ff595c86b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a12e0 0x5632892ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.261+0000 7ff595c86b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a6a20 0x5632892aa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.261+0000 7ff595c86b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a7340 0x5632892aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff595c86b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5632892adcc0 con 0x5632892a7340 2024-06-10T00:31:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff595c86b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56328918eba0 con 0x5632892a12e0 2024-06-10T00:31:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff595c86b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5632892ade40 con 0x5632892a6a20 2024-06-10T00:31:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff595c86b80 10 client.0 _ll_register_callbacks cb 0x5632891888d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:31:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff5899a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a7340 0x5632892aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff5899a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a7340 0x5632892aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37278/0 (socket says 172.21.15.62:37278) 2024-06-10T00:31:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff5899a0700 1 -- 172.21.15.62:0/3981456411 learned_addr learned my addr 172.21.15.62:0/3981456411 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff58899e700 1 --2- 172.21.15.62:0/3981456411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a6a20 0x5632892aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff58919f700 1 --2- 172.21.15.62:0/3981456411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a12e0 0x5632892ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff5899a0700 1 -- 172.21.15.62:0/3981456411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a6a20 msgr2=0x5632892aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff5899a0700 1 --2- 172.21.15.62:0/3981456411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a6a20 0x5632892aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff5899a0700 1 -- 172.21.15.62:0/3981456411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a12e0 msgr2=0x5632892ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff5899a0700 1 --2- 172.21.15.62:0/3981456411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a12e0 0x5632892ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff5899a0700 1 -- 172.21.15.62:0/3981456411 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328913d880 con 0x5632892a7340 2024-06-10T00:31:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff5899a0700 1 --2- 172.21.15.62:0/3981456411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a7340 0x5632892aa8c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff57c09b3c0 tx=0x7ff57c09bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320aa53969a0e5c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff56cff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:31:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff56cff9700 1 -- 172.21.15.62:0/3981456411 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0b3070 con 0x5632892a7340 2024-06-10T00:31:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff595c86b80 1 -- 172.21.15.62:0/3981456411 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff58000fd50 con 0x5632892a7340 2024-06-10T00:31:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff56cff9700 1 -- 172.21.15.62:0/3981456411 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7ff57c0b4260 con 0x5632892a7340 2024-06-10T00:31:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff56cff9700 1 -- 172.21.15.62:0/3981456411 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff57c0a40a0 con 0x5632892a7340 2024-06-10T00:31:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff595c86b80 10 client.15117 fetch_fsmap learned FSMap version 29 2024-06-10T00:31:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.262+0000 7ff595c86b80 1 -- 172.21.15.62:0/3981456411 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5632892b8670 con 0x5632892a7340 2024-06-10T00:31:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.263+0000 7ff56cff9700 1 -- 172.21.15.62:0/3981456411 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff58000fd50 con 0x5632892a7340 2024-06-10T00:31:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.263+0000 7ff56cff9700 1 -- 172.21.15.62:0/3981456411 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff57c01f030 con 0x5632892a7340 2024-06-10T00:31:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.263+0000 7ff595c86b80 10 client.15117 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:31:52.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.263+0000 7ff595c86b80 10 client.15117 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-10T00:31:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.263+0000 7ff595c86b80 20 client.15117 populate_metadata read hostname 'smithi062' 2024-06-10T00:31:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.263+0000 7ff595c86b80 1 --2- 172.21.15.62:0/3981456411 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x5632892bac30 0x5632892bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.263+0000 7ff595c86b80 4 client.15117 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:31:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.263+0000 7ff595c86b80 1 -- 172.21.15.62:0/3981456411 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632891fbcc0 con 0x5632892bac30 2024-06-10T00:31:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.263+0000 7ff58919f700 1 --2- 172.21.15.62:0/3981456411 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x5632892bac30 0x5632892bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.264+0000 7ff58919f700 1 --2- 172.21.15.62:0/3981456411 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x5632892bac30 0x5632892bc0a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5584e2bdb872233e server_cookie=857a65f2b78ee40e in_seq=0 out_seq=0 2024-06-10T00:31:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.264+0000 7ff56cff9700 10 client.15117 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:31:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.264+0000 7ff56cff9700 1 -- 172.21.15.62:0/3981456411 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5632891fbcc0 con 0x5632892bac30 2024-06-10T00:31:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.264+0000 7ff56cff9700 10 client.15117 handle_command_reply: tid=0 2024-06-10T00:31:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.264+0000 7ff595c86b80 1 client.15117 shutdown 2024-06-10T00:31:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.265+0000 7ff595c86b80 1 -- 172.21.15.62:0/3981456411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a7340 msgr2=0x5632892aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.265+0000 7ff595c86b80 1 --2- 172.21.15.62:0/3981456411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a7340 0x5632892aa8c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff57c09b3c0 tx=0x7ff57c09bc90 comp rx=0 tx=0).stop 2024-06-10T00:31:52.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.265+0000 7ff595c86b80 1 -- 172.21.15.62:0/3981456411 shutdown_connections 2024-06-10T00:31:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.265+0000 7ff595c86b80 1 --2- 172.21.15.62:0/3981456411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a7340 0x5632892aa8c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.265+0000 7ff595c86b80 1 --2- 172.21.15.62:0/3981456411 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x5632892bac30 0x5632892bc0a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.265+0000 7ff56cff9700 0 client.15117 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:31:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.265+0000 7ff595c86b80 1 --2- 172.21.15.62:0/3981456411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a12e0 0x5632892ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.265+0000 7ff595c86b80 1 --2- 172.21.15.62:0/3981456411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a6a20 0x5632892aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.265+0000 7ff595c86b80 1 -- 172.21.15.62:0/3981456411 >> 172.21.15.62:0/3981456411 conn(0x5632891c7c70 msgr2=0x56328929b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.265+0000 7ff595c86b80 1 -- 172.21.15.62:0/3981456411 shutdown_connections 2024-06-10T00:31:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.265+0000 7ff595c86b80 1 -- 172.21.15.62:0/3981456411 wait complete. 2024-06-10T00:31:52.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.265+0000 7ff595c86b80 20 client.15117 trim_cache size 0 max 16384 2024-06-10T00:31:52.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.287+0000 7ff595c86b80 1 Processor -- start 2024-06-10T00:31:52.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.287+0000 7ff595c86b80 1 -- start start 2024-06-10T00:31:52.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.287+0000 7ff595c86b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a12e0 0x5632892a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.287+0000 7ff595c86b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a6a20 0x563289297330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.287+0000 7ff595c86b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a7340 0x563289297870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.287+0000 7ff595c86b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5632892ba3e0 con 0x5632892a7340 2024-06-10T00:31:52.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.287+0000 7ff595c86b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5632892baab0 con 0x5632892a6a20 2024-06-10T00:31:52.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.287+0000 7ff595c86b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5632892bb180 con 0x5632892a12e0 2024-06-10T00:31:52.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.287+0000 7ff5899a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a7340 0x563289297870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.287+0000 7ff5899a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a7340 0x563289297870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37290/0 (socket says 172.21.15.62:37290) 2024-06-10T00:31:52.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.287+0000 7ff5899a0700 1 -- 172.21.15.62:0/369091826 learned_addr learned my addr 172.21.15.62:0/369091826 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:52.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.287+0000 7ff58919f700 1 --2- 172.21.15.62:0/369091826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a12e0 0x5632892a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.287+0000 7ff58899e700 1 --2- 172.21.15.62:0/369091826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a6a20 0x563289297330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.288+0000 7ff58919f700 1 -- 172.21.15.62:0/369091826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a6a20 msgr2=0x563289297330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:52.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.288+0000 7ff58919f700 1 --2- 172.21.15.62:0/369091826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a6a20 0x563289297330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.288+0000 7ff58919f700 1 -- 172.21.15.62:0/369091826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a7340 msgr2=0x563289297870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:52.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.288+0000 7ff58919f700 1 --2- 172.21.15.62:0/369091826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a7340 0x563289297870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.288+0000 7ff58919f700 1 -- 172.21.15.62:0/369091826 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632892a88f0 con 0x5632892a12e0 2024-06-10T00:31:52.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.288+0000 7ff5899a0700 1 --2- 172.21.15.62:0/369091826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a7340 0x563289297870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:31:52.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.288+0000 7ff58919f700 1 --2- 172.21.15.62:0/369091826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a12e0 0x5632892a16c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff580031040 tx=0x7ff5800249b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bafcfb5581c389b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:52.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.288+0000 7ff56effd700 1 -- 172.21.15.62:0/369091826 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580024ac0 con 0x5632892a12e0 2024-06-10T00:31:52.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.288+0000 7ff56effd700 1 -- 172.21.15.62:0/369091826 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff58000fc20 con 0x5632892a12e0 2024-06-10T00:31:52.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.288+0000 7ff56effd700 1 -- 172.21.15.62:0/369091826 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580039730 con 0x5632892a12e0 2024-06-10T00:31:52.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.289+0000 7ff595c86b80 1 -- 172.21.15.62:0/369091826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a12e0 msgr2=0x5632892a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.289+0000 7ff595c86b80 1 --2- 172.21.15.62:0/369091826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a12e0 0x5632892a16c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff580031040 tx=0x7ff5800249b0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.289+0000 7ff595c86b80 1 -- 172.21.15.62:0/369091826 shutdown_connections 2024-06-10T00:31:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.289+0000 7ff595c86b80 1 --2- 172.21.15.62:0/369091826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a7340 0x563289297870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.289+0000 7ff595c86b80 1 --2- 172.21.15.62:0/369091826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a12e0 0x5632892a16c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.289+0000 7ff595c86b80 1 --2- 172.21.15.62:0/369091826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a6a20 0x563289297330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.289+0000 7ff595c86b80 1 -- 172.21.15.62:0/369091826 >> 172.21.15.62:0/369091826 conn(0x5632891c7c70 msgr2=0x5632892aa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.289+0000 7ff595c86b80 1 -- 172.21.15.62:0/369091826 shutdown_connections 2024-06-10T00:31:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.289+0000 7ff595c86b80 1 -- 172.21.15.62:0/369091826 wait complete. 2024-06-10T00:31:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.290+0000 7ff595c86b80 1 -- start start 2024-06-10T00:31:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.290+0000 7ff595c86b80 1 Processor -- start 2024-06-10T00:31:52.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.290+0000 7ff595c86b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a12e0 0x56328929db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.290+0000 7ff595c86b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a6a20 0x56328929e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.290+0000 7ff595c86b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a7340 0x5632892c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.290+0000 7ff595c86b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5632892bbf20 con 0x5632892a12e0 2024-06-10T00:31:52.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.290+0000 7ff595c86b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5632892bb850 con 0x5632892a7340 2024-06-10T00:31:52.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.290+0000 7ff595c86b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5632892bc5f0 con 0x5632892a6a20 2024-06-10T00:31:52.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.290+0000 7ff595c86b80 10 client.0 _ll_register_callbacks cb 0x5632891888d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:31:52.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff58919f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a12e0 0x56328929db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff58919f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a12e0 0x56328929db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37300/0 (socket says 172.21.15.62:37300) 2024-06-10T00:31:52.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff58919f700 1 -- 172.21.15.62:0/3276649088 learned_addr learned my addr 172.21.15.62:0/3276649088 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:52.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff5899a0700 1 --2- 172.21.15.62:0/3276649088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a7340 0x5632892c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff58899e700 1 --2- 172.21.15.62:0/3276649088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a6a20 0x56328929e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff58919f700 1 -- 172.21.15.62:0/3276649088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a6a20 msgr2=0x56328929e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:52.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff58919f700 1 --2- 172.21.15.62:0/3276649088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a6a20 0x56328929e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff58919f700 1 -- 172.21.15.62:0/3276649088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a7340 msgr2=0x5632892c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:52.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff58919f700 1 --2- 172.21.15.62:0/3276649088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a7340 0x5632892c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff58919f700 1 -- 172.21.15.62:0/3276649088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632892c5c10 con 0x5632892a12e0 2024-06-10T00:31:52.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff58919f700 1 --2- 172.21.15.62:0/3276649088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a12e0 0x56328929db70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff580024820 tx=0x7ff58000bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a4be782d1a8984 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:52.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff56cff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:31:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff56cff9700 1 -- 172.21.15.62:0/3276649088 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580024a20 con 0x5632892a12e0 2024-06-10T00:31:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff56cff9700 1 -- 172.21.15.62:0/3276649088 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7ff580024bc0 con 0x5632892a12e0 2024-06-10T00:31:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.291+0000 7ff595c86b80 1 -- 172.21.15.62:0/3276649088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff58000fc20 con 0x5632892a12e0 2024-06-10T00:31:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.292+0000 7ff56cff9700 1 -- 172.21.15.62:0/3276649088 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff580024dc0 con 0x5632892a12e0 2024-06-10T00:31:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.292+0000 7ff595c86b80 10 client.15123 fetch_fsmap learned FSMap version 29 2024-06-10T00:31:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.292+0000 7ff595c86b80 1 -- 172.21.15.62:0/3276649088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5632892c10a0 con 0x5632892a12e0 2024-06-10T00:31:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.292+0000 7ff56cff9700 1 -- 172.21.15.62:0/3276649088 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff58000fc20 con 0x5632892a12e0 2024-06-10T00:31:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.292+0000 7ff56cff9700 1 -- 172.21.15.62:0/3276649088 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff580039490 con 0x5632892a12e0 2024-06-10T00:31:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.292+0000 7ff595c86b80 10 client.15123 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:31:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.292+0000 7ff595c86b80 10 client.15123 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-10T00:31:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.292+0000 7ff595c86b80 20 client.15123 populate_metadata read hostname 'smithi062' 2024-06-10T00:31:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.292+0000 7ff595c86b80 1 --2- 172.21.15.62:0/3276649088 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x56328932eda0 0x56328932dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.292+0000 7ff595c86b80 4 client.15123 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:31:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.292+0000 7ff595c86b80 1 -- 172.21.15.62:0/3276649088 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "status"}) -- 0x5632891f6240 con 0x56328932eda0 2024-06-10T00:31:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.292+0000 7ff58899e700 1 --2- 172.21.15.62:0/3276649088 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x56328932eda0 0x56328932dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.293+0000 7ff58899e700 1 --2- 172.21.15.62:0/3276649088 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x56328932eda0 0x56328932dab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aba634d4ab16c02f server_cookie=68f70b38c228dbbe in_seq=0 out_seq=0 2024-06-10T00:31:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.293+0000 7ff56cff9700 10 client.15123 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:31:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.293+0000 7ff56cff9700 1 -- 172.21.15.62:0/3276649088 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5632891f6240 con 0x56328932eda0 2024-06-10T00:31:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.293+0000 7ff56cff9700 10 client.15123 handle_command_reply: tid=0 2024-06-10T00:31:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.293+0000 7ff595c86b80 1 client.15123 shutdown 2024-06-10T00:31:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.293+0000 7ff595c86b80 1 -- 172.21.15.62:0/3276649088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a12e0 msgr2=0x56328929db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.293+0000 7ff595c86b80 1 --2- 172.21.15.62:0/3276649088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a12e0 0x56328929db70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff580024820 tx=0x7ff58000bec0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.293+0000 7ff595c86b80 1 -- 172.21.15.62:0/3276649088 shutdown_connections 2024-06-10T00:31:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.293+0000 7ff595c86b80 1 --2- 172.21.15.62:0/3276649088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5632892a12e0 0x56328929db70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.293+0000 7ff595c86b80 1 --2- 172.21.15.62:0/3276649088 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x56328932eda0 0x56328932dab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.293+0000 7ff595c86b80 1 --2- 172.21.15.62:0/3276649088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632892a7340 0x5632892c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.293+0000 7ff56cff9700 0 client.15123 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:31:52.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.293+0000 7ff595c86b80 1 --2- 172.21.15.62:0/3276649088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632892a6a20 0x56328929e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.293+0000 7ff595c86b80 1 -- 172.21.15.62:0/3276649088 >> 172.21.15.62:0/3276649088 conn(0x5632891c7c70 msgr2=0x5632892aa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:52.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.294+0000 7ff595c86b80 1 -- 172.21.15.62:0/3276649088 shutdown_connections 2024-06-10T00:31:52.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.294+0000 7ff595c86b80 1 -- 172.21.15.62:0/3276649088 wait complete. 2024-06-10T00:31:52.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.294+0000 7ff595c86b80 20 client.15123 trim_cache size 0 max 16384 2024-06-10T00:31:52.298 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:31:52.298 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:31:52.298 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2024-06-10T00:31:52.298 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24431, 2024-06-10T00:31:52.299 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:31:52.299 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:31:52.299 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:31:52.299 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 129.44154334999999, 2024-06-10T00:31:52.299 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 29, 2024-06-10T00:31:52.299 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 99, 2024-06-10T00:31:52.299 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T00:31:52.299 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 224.07271843300001 2024-06-10T00:31:52.299 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:31:52.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.294+0000 7ff58b403700 1 -- 172.21.15.62:0/3424452156 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff5700a2c20 msgr2=0x7ff5700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:52.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.294+0000 7ff58b403700 1 --2- 172.21.15.62:0/3424452156 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff5700a2c20 0x7ff5700a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff580000c00 tx=0x7ff580005fb0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.294+0000 7ff58b403700 1 -- 172.21.15.62:0/3424452156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff584107840 msgr2=0x7ff584198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:52.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.294+0000 7ff58b403700 1 --2- 172.21.15.62:0/3424452156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff584107840 0x7ff584198b50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00b970 tx=0x7ff57c00bc80 comp rx=0 tx=0).stop 2024-06-10T00:31:52.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.294+0000 7ff58b403700 1 -- 172.21.15.62:0/3424452156 shutdown_connections 2024-06-10T00:31:52.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.294+0000 7ff58b403700 1 --2- 172.21.15.62:0/3424452156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff584107840 0x7ff584198b50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.294+0000 7ff58b403700 1 --2- 172.21.15.62:0/3424452156 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff5700a2c20 0x7ff5700a50e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.294+0000 7ff58b403700 1 --2- 172.21.15.62:0/3424452156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff584102740 0x7ff58419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.294+0000 7ff58b403700 1 --2- 172.21.15.62:0/3424452156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5841030f0 0x7ff58419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:52.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.294+0000 7ff58b403700 1 -- 172.21.15.62:0/3424452156 >> 172.21.15.62:0/3424452156 conn(0x7ff5840795f0 msgr2=0x7ff5841007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:52.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.295+0000 7ff58b403700 1 -- 172.21.15.62:0/3424452156 shutdown_connections 2024-06-10T00:31:52.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:52.295+0000 7ff58b403700 1 -- 172.21.15.62:0/3424452156 wait complete. 2024-06-10T00:31:52.899 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T00:31:53.155 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:53.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:53 smithi062 ceph-mon[32390]: pgmap v404: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:31:54.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:53 smithi105 ceph-mon[36321]: pgmap v404: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:31:54.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:53 smithi138 ceph-mon[36549]: pgmap v404: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:31:54.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.894+0000 7f3c0d08b700 1 -- 172.21.15.62:0/3603824872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c08102380 msgr2=0x7f3c08105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:54.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.894+0000 7f3c0d08b700 1 --2- 172.21.15.62:0/3603824872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c08102380 0x7f3c08105ff0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00099c0 tx=0x7f3bf0009cd0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.894+0000 7f3c0d08b700 1 -- 172.21.15.62:0/3603824872 shutdown_connections 2024-06-10T00:31:54.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.894+0000 7f3c0d08b700 1 --2- 172.21.15.62:0/3603824872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c08106720 0x7f3c0810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.894+0000 7f3c0d08b700 1 --2- 172.21.15.62:0/3603824872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c081019d0 0x7f3c08101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.894+0000 7f3c0d08b700 1 --2- 172.21.15.62:0/3603824872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c08102380 0x7f3c08105ff0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.894+0000 7f3c0d08b700 1 -- 172.21.15.62:0/3603824872 >> 172.21.15.62:0/3603824872 conn(0x7f3c08078340 msgr2=0x7f3c08078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:54.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.894+0000 7f3c0d08b700 1 -- 172.21.15.62:0/3603824872 shutdown_connections 2024-06-10T00:31:54.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.894+0000 7f3c0d08b700 1 -- 172.21.15.62:0/3603824872 wait complete. 2024-06-10T00:31:54.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.895+0000 7f3c0d08b700 1 Processor -- start 2024-06-10T00:31:54.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.895+0000 7f3c0d08b700 1 -- start start 2024-06-10T00:31:54.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.895+0000 7f3c0d08b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c081019d0 0x7f3c08103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.895+0000 7f3c0d08b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c08102380 0x7f3c081035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.895+0000 7f3c0d08b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c08106720 0x7f3c08103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.895+0000 7f3c0d08b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3c08111710 con 0x7f3c08106720 2024-06-10T00:31:54.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.895+0000 7f3c0d08b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3c08111590 con 0x7f3c08102380 2024-06-10T00:31:54.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.895+0000 7f3c0d08b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c08111890 con 0x7f3c081019d0 2024-06-10T00:31:54.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.895+0000 7f3c0759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c08106720 0x7f3c08103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.895+0000 7f3c0759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c08106720 0x7f3c08103b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37324/0 (socket says 172.21.15.62:37324) 2024-06-10T00:31:54.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.895+0000 7f3c0759e700 1 -- 172.21.15.62:0/2738510694 learned_addr learned my addr 172.21.15.62:0/2738510694 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:54.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.895+0000 7f3c06d9d700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c081019d0 0x7f3c08103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.895+0000 7f3c0659c700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c08102380 0x7f3c081035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.896+0000 7f3c0759e700 1 -- 172.21.15.62:0/2738510694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c081019d0 msgr2=0x7f3c08103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:54.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.896+0000 7f3c0759e700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c081019d0 0x7f3c08103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.896+0000 7f3c0759e700 1 -- 172.21.15.62:0/2738510694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c08102380 msgr2=0x7f3c081035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:54.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.896+0000 7f3c0759e700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c08102380 0x7f3c081035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.896+0000 7f3c0759e700 1 -- 172.21.15.62:0/2738510694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c081a7270 con 0x7f3c08106720 2024-06-10T00:31:54.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.896+0000 7f3c0659c700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c08102380 0x7f3c081035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:31:54.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.896+0000 7f3c0759e700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c08106720 0x7f3c08103b10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000c970 tx=0x7f3c0000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:54.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.896+0000 7f3bfffff700 1 -- 172.21.15.62:0/2738510694 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000ce60 con 0x7f3c08106720 2024-06-10T00:31:54.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.896+0000 7f3c0d08b700 1 -- 172.21.15.62:0/2738510694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c081a7560 con 0x7f3c08106720 2024-06-10T00:31:54.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.896+0000 7f3bfffff700 1 -- 172.21.15.62:0/2738510694 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c00004830 con 0x7f3c08106720 2024-06-10T00:31:54.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.896+0000 7f3bfffff700 1 -- 172.21.15.62:0/2738510694 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c000055a0 con 0x7f3c08106720 2024-06-10T00:31:54.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.896+0000 7f3c0d08b700 1 -- 172.21.15.62:0/2738510694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c081a7a70 con 0x7f3c08106720 2024-06-10T00:31:54.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.899+0000 7f3bfffff700 1 -- 172.21.15.62:0/2738510694 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3c000049e0 con 0x7f3c08106720 2024-06-10T00:31:54.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.900+0000 7f3bfffff700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3bf40a2c20 0x7f3bf40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.900+0000 7f3c06d9d700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3bf40a2c20 0x7f3bf40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.900+0000 7f3bfffff700 1 -- 172.21.15.62:0/2738510694 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f3c000ceff0 con 0x7f3c08106720 2024-06-10T00:31:54.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.900+0000 7f3c06d9d700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3bf40a2c20 0x7f3bf40a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3bf800a400 tx=0x7f3bf800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:54.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.900+0000 7f3c1790eb80 1 Processor -- start 2024-06-10T00:31:54.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.900+0000 7f3c1790eb80 1 -- start start 2024-06-10T00:31:54.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c1790eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 0x56389e1ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c1790eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1ab830 0x56389e1a5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c1790eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1a5750 0x56389e1a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c1790eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56389e097f10 con 0x56389e1a5750 2024-06-10T00:31:54.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c1790eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56389e0f6800 con 0x56389e1aaf10 2024-06-10T00:31:54.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c1790eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56389e05e410 con 0x56389e1ab830 2024-06-10T00:31:54.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c0759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1a5750 0x56389e1a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c0759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1a5750 0x56389e1a5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37332/0 (socket says 172.21.15.62:37332) 2024-06-10T00:31:54.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c0759e700 1 -- 172.21.15.62:0/628852848 learned_addr learned my addr 172.21.15.62:0/628852848 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:54.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c06d9d700 1 --2- 172.21.15.62:0/628852848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 0x56389e1ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c0659c700 1 --2- 172.21.15.62:0/628852848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1ab830 0x56389e1a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c0759e700 1 -- 172.21.15.62:0/628852848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1ab830 msgr2=0x56389e1a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:54.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c0759e700 1 --2- 172.21.15.62:0/628852848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1ab830 0x56389e1a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c0759e700 1 -- 172.21.15.62:0/628852848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 msgr2=0x56389e1ab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:54.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c0759e700 1 --2- 172.21.15.62:0/628852848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 0x56389e1ab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c0759e700 1 -- 172.21.15.62:0/628852848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56389e045880 con 0x56389e1a5750 2024-06-10T00:31:54.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.901+0000 7f3c06d9d700 1 --2- 172.21.15.62:0/628852848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 0x56389e1ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:31:54.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.902+0000 7f3c0759e700 1 --2- 172.21.15.62:0/628852848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1a5750 0x56389e1a5b70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3c000978c0 tx=0x7f3c0009ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5005d6001e3334ae server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:54.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.902+0000 7f3bfd7fa700 1 -- 172.21.15.62:0/628852848 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0001d040 con 0x56389e1a5750 2024-06-10T00:31:54.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.902+0000 7f3bfd7fa700 1 -- 172.21.15.62:0/628852848 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c0009b460 con 0x56389e1a5750 2024-06-10T00:31:54.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.902+0000 7f3bfd7fa700 1 -- 172.21.15.62:0/628852848 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c000a3c40 con 0x56389e1a5750 2024-06-10T00:31:54.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.902+0000 7f3c1790eb80 1 -- 172.21.15.62:0/628852848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1a5750 msgr2=0x56389e1a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:54.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.902+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/628852848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1a5750 0x56389e1a5b70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3c000978c0 tx=0x7f3c0009ae70 comp rx=0 tx=0).stop 2024-06-10T00:31:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 -- 172.21.15.62:0/628852848 shutdown_connections 2024-06-10T00:31:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/628852848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1a5750 0x56389e1a5b70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/628852848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 0x56389e1ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/628852848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1ab830 0x56389e1a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 -- 172.21.15.62:0/628852848 >> 172.21.15.62:0/628852848 conn(0x56389e0d2030 msgr2=0x56389e1a01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 -- 172.21.15.62:0/628852848 shutdown_connections 2024-06-10T00:31:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 -- 172.21.15.62:0/628852848 wait complete. 2024-06-10T00:31:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 -- start start 2024-06-10T00:31:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 Processor -- start 2024-06-10T00:31:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1a5750 0x56389e1b1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1aaf10 0x56389e1ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1ab830 0x56389e1aed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56389e096f70 con 0x56389e1a5750 2024-06-10T00:31:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56389e0f27c0 con 0x56389e1ab830 2024-06-10T00:31:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.903+0000 7f3c1790eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56389e1b2210 con 0x56389e1aaf10 2024-06-10T00:31:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3c1790eb80 10 client.0 _ll_register_callbacks cb 0x56389e08c560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:31:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3c06d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1a5750 0x56389e1b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3c06d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1a5750 0x56389e1b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37334/0 (socket says 172.21.15.62:37334) 2024-06-10T00:31:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3c06d9d700 1 -- 172.21.15.62:0/1057043913 learned_addr learned my addr 172.21.15.62:0/1057043913 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3c0659c700 1 --2- 172.21.15.62:0/1057043913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1aaf10 0x56389e1ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3c0759e700 1 --2- 172.21.15.62:0/1057043913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1ab830 0x56389e1aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3c06d9d700 1 -- 172.21.15.62:0/1057043913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1aaf10 msgr2=0x56389e1ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3c06d9d700 1 --2- 172.21.15.62:0/1057043913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1aaf10 0x56389e1ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3c06d9d700 1 -- 172.21.15.62:0/1057043913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1ab830 msgr2=0x56389e1aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3c06d9d700 1 --2- 172.21.15.62:0/1057043913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1ab830 0x56389e1aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3c06d9d700 1 -- 172.21.15.62:0/1057043913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56389e0bc700 con 0x56389e1a5750 2024-06-10T00:31:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3c0759e700 1 --2- 172.21.15.62:0/1057043913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1ab830 0x56389e1aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:31:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3c06d9d700 1 --2- 172.21.15.62:0/1057043913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1a5750 0x56389e1b1540 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8016f00 tx=0x7f3bf8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198c671a3b5b499 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3bee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:31:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3bee7fc700 1 -- 172.21.15.62:0/1057043913 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf80223f0 con 0x56389e1a5750 2024-06-10T00:31:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3c1790eb80 1 -- 172.21.15.62:0/1057043913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c0009b460 con 0x56389e1a5750 2024-06-10T00:31:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.904+0000 7f3bee7fc700 1 -- 172.21.15.62:0/1057043913 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f3bf8022590 con 0x56389e1a5750 2024-06-10T00:31:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.905+0000 7f3bee7fc700 1 -- 172.21.15.62:0/1057043913 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3bf8030050 con 0x56389e1a5750 2024-06-10T00:31:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.905+0000 7f3c1790eb80 10 client.15135 fetch_fsmap learned FSMap version 29 2024-06-10T00:31:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.905+0000 7f3c1790eb80 1 -- 172.21.15.62:0/1057043913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56389e1bc9a0 con 0x56389e1a5750 2024-06-10T00:31:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.905+0000 7f3bee7fc700 1 -- 172.21.15.62:0/1057043913 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c0009b460 con 0x56389e1a5750 2024-06-10T00:31:54.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.905+0000 7f3bee7fc700 1 -- 172.21.15.62:0/1057043913 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3bf802b8e0 con 0x56389e1a5750 2024-06-10T00:31:54.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.905+0000 7f3c1790eb80 10 client.15135 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:31:54.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.905+0000 7f3c1790eb80 10 client.15135 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T00:31:54.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.905+0000 7f3c1790eb80 20 client.15135 populate_metadata read hostname 'smithi062' 2024-06-10T00:31:54.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.905+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/1057043913 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56389e1beff0 0x56389e1c0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.905+0000 7f3c1790eb80 4 client.15135 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:31:54.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.905+0000 7f3c1790eb80 1 -- 172.21.15.62:0/1057043913 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56389e103d00 con 0x56389e1beff0 2024-06-10T00:31:54.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.906+0000 7f3c0659c700 1 --2- 172.21.15.62:0/1057043913 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56389e1beff0 0x56389e1c0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.906+0000 7f3c0659c700 1 --2- 172.21.15.62:0/1057043913 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56389e1beff0 0x56389e1c0460 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2191eaff5f6f108 server_cookie=87cb61a6d3c4c476 in_seq=0 out_seq=0 2024-06-10T00:31:54.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.906+0000 7f3bee7fc700 10 client.15135 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:31:54.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3bee7fc700 1 -- 172.21.15.62:0/1057043913 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56389e103d00 con 0x56389e1beff0 2024-06-10T00:31:54.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3bee7fc700 10 client.15135 handle_command_reply: tid=0 2024-06-10T00:31:54.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3c1790eb80 1 client.15135 shutdown 2024-06-10T00:31:54.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3c1790eb80 1 -- 172.21.15.62:0/1057043913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1a5750 msgr2=0x56389e1b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:54.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/1057043913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1a5750 0x56389e1b1540 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8016f00 tx=0x7f3bf8004ad0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3c1790eb80 1 -- 172.21.15.62:0/1057043913 shutdown_connections 2024-06-10T00:31:54.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/1057043913 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56389e1beff0 0x56389e1c0460 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/1057043913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1a5750 0x56389e1b1540 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/1057043913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1ab830 0x56389e1aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/1057043913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1aaf10 0x56389e1ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3c1790eb80 1 -- 172.21.15.62:0/1057043913 >> 172.21.15.62:0/1057043913 conn(0x56389e0d2030 msgr2=0x56389e19f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:54.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3bee7fc700 0 client.15135 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:31:54.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3c1790eb80 1 -- 172.21.15.62:0/1057043913 shutdown_connections 2024-06-10T00:31:54.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3c1790eb80 1 -- 172.21.15.62:0/1057043913 wait complete. 2024-06-10T00:31:54.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.907+0000 7f3c1790eb80 20 client.15135 trim_cache size 0 max 16384 2024-06-10T00:31:54.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.929+0000 7f3c1790eb80 1 Processor -- start 2024-06-10T00:31:54.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.929+0000 7f3c1790eb80 1 -- start start 2024-06-10T00:31:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.930+0000 7f3c1790eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1a5750 0x56389e1a5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.930+0000 7f3c1790eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 0x56389e19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.930+0000 7f3c1790eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1ab830 0x56389e1a0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.930+0000 7f3c1790eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56389e1be7a0 con 0x56389e1ab830 2024-06-10T00:31:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.930+0000 7f3c1790eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56389e1bee70 con 0x56389e1aaf10 2024-06-10T00:31:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.930+0000 7f3c1790eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56389e1bf540 con 0x56389e1a5750 2024-06-10T00:31:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.930+0000 7f3c0759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1ab830 0x56389e1a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.930+0000 7f3c0759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1ab830 0x56389e1a0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37344/0 (socket says 172.21.15.62:37344) 2024-06-10T00:31:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.930+0000 7f3c0659c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 0x56389e19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.930+0000 7f3c0759e700 1 -- 172.21.15.62:0/1361063290 learned_addr learned my addr 172.21.15.62:0/1361063290 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.930+0000 7f3c06d9d700 1 --2- 172.21.15.62:0/1361063290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1a5750 0x56389e1a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.930+0000 7f3c0759e700 1 -- 172.21.15.62:0/1361063290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1a5750 msgr2=0x56389e1a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.930+0000 7f3c0759e700 1 --2- 172.21.15.62:0/1361063290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1a5750 0x56389e1a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.931+0000 7f3c0759e700 1 -- 172.21.15.62:0/1361063290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 msgr2=0x56389e19fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.931+0000 7f3c0759e700 1 --2- 172.21.15.62:0/1361063290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 0x56389e19fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.931+0000 7f3c0759e700 1 -- 172.21.15.62:0/1361063290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56389e1acdd0 con 0x56389e1ab830 2024-06-10T00:31:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.931+0000 7f3c0659c700 1 --2- 172.21.15.62:0/1361063290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 0x56389e19fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:31:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.931+0000 7f3c0759e700 1 --2- 172.21.15.62:0/1361063290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1ab830 0x56389e1a0370 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3c00097e30 tx=0x7f3c00005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2d5db95213c41f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.931+0000 7f3bfcff9700 1 -- 172.21.15.62:0/1361063290 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0001d040 con 0x56389e1ab830 2024-06-10T00:31:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.931+0000 7f3bfcff9700 1 -- 172.21.15.62:0/1361063290 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c0009f810 con 0x56389e1ab830 2024-06-10T00:31:54.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.931+0000 7f3bfcff9700 1 -- 172.21.15.62:0/1361063290 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c000a3c40 con 0x56389e1ab830 2024-06-10T00:31:54.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.932+0000 7f3c1790eb80 1 -- 172.21.15.62:0/1361063290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1ab830 msgr2=0x56389e1a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:54.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.932+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/1361063290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1ab830 0x56389e1a0370 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3c00097e30 tx=0x7f3c00005760 comp rx=0 tx=0).stop 2024-06-10T00:31:54.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.932+0000 7f3c1790eb80 1 -- 172.21.15.62:0/1361063290 shutdown_connections 2024-06-10T00:31:54.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.932+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/1361063290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1ab830 0x56389e1a0370 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.932+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/1361063290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1a5750 0x56389e1a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.932+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/1361063290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 0x56389e19fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.932+0000 7f3c1790eb80 1 -- 172.21.15.62:0/1361063290 >> 172.21.15.62:0/1361063290 conn(0x56389e0d2030 msgr2=0x56389e1ad150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:54.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.932+0000 7f3c1790eb80 1 -- 172.21.15.62:0/1361063290 shutdown_connections 2024-06-10T00:31:54.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.932+0000 7f3c1790eb80 1 -- 172.21.15.62:0/1361063290 wait complete. 2024-06-10T00:31:54.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.933+0000 7f3c1790eb80 1 -- start start 2024-06-10T00:31:54.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.933+0000 7f3c1790eb80 1 Processor -- start 2024-06-10T00:31:54.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.933+0000 7f3c1790eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1a5750 0x56389e1c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.933+0000 7f3c1790eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 0x56389e19f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.933+0000 7f3c1790eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1ab830 0x56389e19f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.933+0000 7f3c1790eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56389e1c02e0 con 0x56389e1ab830 2024-06-10T00:31:54.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.933+0000 7f3c1790eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56389e1bfc10 con 0x56389e1aaf10 2024-06-10T00:31:54.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.933+0000 7f3c1790eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56389e1c09b0 con 0x56389e1a5750 2024-06-10T00:31:54.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.933+0000 7f3c1790eb80 10 client.0 _ll_register_callbacks cb 0x56389e08c560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:31:54.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.933+0000 7f3c0759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1ab830 0x56389e19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.933+0000 7f3c0759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1ab830 0x56389e19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37352/0 (socket says 172.21.15.62:37352) 2024-06-10T00:31:54.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.933+0000 7f3c0759e700 1 -- 172.21.15.62:0/552056418 learned_addr learned my addr 172.21.15.62:0/552056418 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:54.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.933+0000 7f3c06d9d700 1 --2- 172.21.15.62:0/552056418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1a5750 0x56389e1c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.933+0000 7f3c0659c700 1 --2- 172.21.15.62:0/552056418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 0x56389e19f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.934+0000 7f3c0759e700 1 -- 172.21.15.62:0/552056418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1a5750 msgr2=0x56389e1c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:54.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.934+0000 7f3c0759e700 1 --2- 172.21.15.62:0/552056418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1a5750 0x56389e1c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.934+0000 7f3c0759e700 1 -- 172.21.15.62:0/552056418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 msgr2=0x56389e19f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:54.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.934+0000 7f3c0759e700 1 --2- 172.21.15.62:0/552056418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 0x56389e19f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.934+0000 7f3c0759e700 1 -- 172.21.15.62:0/552056418 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56389e19fcb0 con 0x56389e1ab830 2024-06-10T00:31:54.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.934+0000 7f3c06d9d700 1 --2- 172.21.15.62:0/552056418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1a5750 0x56389e1c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:31:54.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.934+0000 7f3c0759e700 1 --2- 172.21.15.62:0/552056418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1ab830 0x56389e19f740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3c000058b0 tx=0x7f3c0009ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42962278930aeb3b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:54.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.934+0000 7f3bee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:31:54.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.934+0000 7f3bee7fc700 1 -- 172.21.15.62:0/552056418 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c000a3e40 con 0x56389e1ab830 2024-06-10T00:31:54.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.934+0000 7f3bee7fc700 1 -- 172.21.15.62:0/552056418 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f3c000ccbc0 con 0x56389e1ab830 2024-06-10T00:31:54.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.934+0000 7f3c1790eb80 1 -- 172.21.15.62:0/552056418 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c0009f810 con 0x56389e1ab830 2024-06-10T00:31:54.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.935+0000 7f3bee7fc700 1 -- 172.21.15.62:0/552056418 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3c00083050 con 0x56389e1ab830 2024-06-10T00:31:54.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.935+0000 7f3c1790eb80 10 client.15141 fetch_fsmap learned FSMap version 29 2024-06-10T00:31:54.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.935+0000 7f3c1790eb80 1 -- 172.21.15.62:0/552056418 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56389e1c6610 con 0x56389e1ab830 2024-06-10T00:31:54.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.935+0000 7f3bee7fc700 1 -- 172.21.15.62:0/552056418 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c00099820 con 0x56389e1ab830 2024-06-10T00:31:54.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.935+0000 7f3bee7fc700 1 -- 172.21.15.62:0/552056418 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3c0009a9c0 con 0x56389e1ab830 2024-06-10T00:31:54.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.935+0000 7f3c1790eb80 10 client.15141 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:31:54.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.935+0000 7f3c1790eb80 10 client.15141 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T00:31:54.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.935+0000 7f3c1790eb80 20 client.15141 populate_metadata read hostname 'smithi062' 2024-06-10T00:31:54.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.935+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/552056418 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56389e237170 0x56389e235e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:54.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.935+0000 7f3c1790eb80 4 client.15141 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:31:54.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.935+0000 7f3c1790eb80 1 -- 172.21.15.62:0/552056418 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "status"}) -- 0x7f3c0009f810 con 0x56389e237170 2024-06-10T00:31:54.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.936+0000 7f3c06d9d700 1 --2- 172.21.15.62:0/552056418 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56389e237170 0x56389e235e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:54.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.936+0000 7f3c06d9d700 1 --2- 172.21.15.62:0/552056418 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56389e237170 0x56389e235e80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22b68bc9724cea01 server_cookie=3825667dde42d750 in_seq=0 out_seq=0 2024-06-10T00:31:54.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.936+0000 7f3bee7fc700 10 client.15141 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:31:54.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.936+0000 7f3bee7fc700 1 -- 172.21.15.62:0/552056418 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x7f3c0009f810 con 0x56389e237170 2024-06-10T00:31:54.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.936+0000 7f3bee7fc700 10 client.15141 handle_command_reply: tid=0 2024-06-10T00:31:54.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.936+0000 7f3c1790eb80 1 client.15141 shutdown 2024-06-10T00:31:54.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.936+0000 7f3c1790eb80 1 -- 172.21.15.62:0/552056418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1ab830 msgr2=0x56389e19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:54.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.936+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/552056418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1ab830 0x56389e19f740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3c000058b0 tx=0x7f3c0009ae70 comp rx=0 tx=0).stop 2024-06-10T00:31:54.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.937+0000 7f3c1790eb80 1 -- 172.21.15.62:0/552056418 shutdown_connections 2024-06-10T00:31:54.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.937+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/552056418 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56389e237170 0x56389e235e80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.937+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/552056418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56389e1ab830 0x56389e19f740 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.937+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/552056418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56389e1a5750 0x56389e1c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.937+0000 7f3c1790eb80 1 --2- 172.21.15.62:0/552056418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56389e1aaf10 0x56389e19f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.937+0000 7f3c1790eb80 1 -- 172.21.15.62:0/552056418 >> 172.21.15.62:0/552056418 conn(0x56389e0d2030 msgr2=0x56389e1abc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:54.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.937+0000 7f3bee7fc700 0 client.15141 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:31:54.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.937+0000 7f3c1790eb80 1 -- 172.21.15.62:0/552056418 shutdown_connections 2024-06-10T00:31:54.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.937+0000 7f3c1790eb80 1 -- 172.21.15.62:0/552056418 wait complete. 2024-06-10T00:31:54.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.937+0000 7f3c1790eb80 20 client.15141 trim_cache size 0 max 16384 2024-06-10T00:31:54.942 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:31:54.942 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:31:54.942 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2024-06-10T00:31:54.942 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24433, 2024-06-10T00:31:54.942 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:31:54.942 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:31:54.942 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:31:54.942 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 133.09380609799999, 2024-06-10T00:31:54.943 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 29, 2024-06-10T00:31:54.943 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 99, 2024-06-10T00:31:54.943 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T00:31:54.943 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 223.980134776 2024-06-10T00:31:54.943 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:31:54.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.938+0000 7f3c0d08b700 1 -- 172.21.15.62:0/2738510694 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3bf40a2c20 msgr2=0x7f3bf40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:54.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.938+0000 7f3c0d08b700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3bf40a2c20 0x7f3bf40a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3bf800a400 tx=0x7f3bf800d040 comp rx=0 tx=0).stop 2024-06-10T00:31:54.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.938+0000 7f3c0d08b700 1 -- 172.21.15.62:0/2738510694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c08106720 msgr2=0x7f3c08103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:54.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.938+0000 7f3c0d08b700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c08106720 0x7f3c08103b10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000c970 tx=0x7f3c0000cc80 comp rx=0 tx=0).stop 2024-06-10T00:31:54.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.938+0000 7f3c0d08b700 1 -- 172.21.15.62:0/2738510694 shutdown_connections 2024-06-10T00:31:54.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.938+0000 7f3c0d08b700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c08106720 0x7f3c08103b10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.938+0000 7f3c0d08b700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f3bf40a2c20 0x7f3bf40a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.938+0000 7f3c0d08b700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c081019d0 0x7f3c08103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.938+0000 7f3c0d08b700 1 --2- 172.21.15.62:0/2738510694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c08102380 0x7f3c081035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:54.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.938+0000 7f3c0d08b700 1 -- 172.21.15.62:0/2738510694 >> 172.21.15.62:0/2738510694 conn(0x7f3c08078340 msgr2=0x7f3c081053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:54.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.938+0000 7f3c0d08b700 1 -- 172.21.15.62:0/2738510694 shutdown_connections 2024-06-10T00:31:54.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:54.938+0000 7f3c0d08b700 1 -- 172.21.15.62:0/2738510694 wait complete. 2024-06-10T00:31:55.596 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-10T00:31:55.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:55 smithi062 ceph-mon[32390]: pgmap v405: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:31:55.885 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:56.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:55 smithi105 ceph-mon[36321]: pgmap v405: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:31:56.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:55 smithi138 ceph-mon[36549]: pgmap v405: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:31:57.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.603+0000 7f989e5e2700 1 -- 172.21.15.62:0/2500752242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9898107840 msgr2=0x7f989810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:57.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.604+0000 7f989e5e2700 1 --2- 172.21.15.62:0/2500752242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9898107840 0x7f989810f3c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f989000b0d0 tx=0x7f989000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.604+0000 7f989e5e2700 1 -- 172.21.15.62:0/2500752242 shutdown_connections 2024-06-10T00:31:57.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.604+0000 7f989e5e2700 1 --2- 172.21.15.62:0/2500752242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9898107840 0x7f989810f3c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.604+0000 7f989e5e2700 1 --2- 172.21.15.62:0/2500752242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9898102740 0x7f9898102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.604+0000 7f989e5e2700 1 --2- 172.21.15.62:0/2500752242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98981030f0 0x7f9898107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.604+0000 7f989e5e2700 1 -- 172.21.15.62:0/2500752242 >> 172.21.15.62:0/2500752242 conn(0x7f98980795f0 msgr2=0x7f9898079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:57.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.604+0000 7f989e5e2700 1 -- 172.21.15.62:0/2500752242 shutdown_connections 2024-06-10T00:31:57.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.604+0000 7f989e5e2700 1 -- 172.21.15.62:0/2500752242 wait complete. 2024-06-10T00:31:57.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.604+0000 7f989e5e2700 1 Processor -- start 2024-06-10T00:31:57.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.605+0000 7f989e5e2700 1 -- start start 2024-06-10T00:31:57.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.605+0000 7f989e5e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9898102740 0x7f989819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.605+0000 7f989e5e2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98981030f0 0x7f989819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.605+0000 7f989e5e2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9898107840 0x7f98981989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.605+0000 7f989e5e2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9898112780 con 0x7f9898102740 2024-06-10T00:31:57.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.605+0000 7f989e5e2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9898112600 con 0x7f9898107840 2024-06-10T00:31:57.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.605+0000 7f989e5e2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9898112900 con 0x7f98981030f0 2024-06-10T00:31:57.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.605+0000 7f9897fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9898102740 0x7f989819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.605+0000 7f9897fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9898102740 0x7f989819e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37356/0 (socket says 172.21.15.62:37356) 2024-06-10T00:31:57.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.605+0000 7f9897fff700 1 -- 172.21.15.62:0/27336985 learned_addr learned my addr 172.21.15.62:0/27336985 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:57.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.605+0000 7f989cb7f700 1 --2- 172.21.15.62:0/27336985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9898107840 0x7f98981989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.605+0000 7f98977fe700 1 --2- 172.21.15.62:0/27336985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98981030f0 0x7f989819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.606+0000 7f9897fff700 1 -- 172.21.15.62:0/27336985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98981030f0 msgr2=0x7f989819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:57.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.606+0000 7f9897fff700 1 --2- 172.21.15.62:0/27336985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98981030f0 0x7f989819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.606+0000 7f9897fff700 1 -- 172.21.15.62:0/27336985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9898107840 msgr2=0x7f98981989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:57.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.606+0000 7f9897fff700 1 --2- 172.21.15.62:0/27336985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9898107840 0x7f98981989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.606+0000 7f9897fff700 1 -- 172.21.15.62:0/27336985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9898199100 con 0x7f9898102740 2024-06-10T00:31:57.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.606+0000 7f9897fff700 1 --2- 172.21.15.62:0/27336985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9898102740 0x7f989819e7e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f988800c920 tx=0x7f988800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:57.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.606+0000 7f98957fa700 1 -- 172.21.15.62:0/27336985 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888007b00 con 0x7f9898102740 2024-06-10T00:31:57.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.606+0000 7f989e5e2700 1 -- 172.21.15.62:0/27336985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98981993f0 con 0x7f9898102740 2024-06-10T00:31:57.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.606+0000 7f98957fa700 1 -- 172.21.15.62:0/27336985 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9888004d10 con 0x7f9898102740 2024-06-10T00:31:57.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.606+0000 7f98957fa700 1 -- 172.21.15.62:0/27336985 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888005630 con 0x7f9898102740 2024-06-10T00:31:57.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.606+0000 7f989e5e2700 1 -- 172.21.15.62:0/27336985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98981a56b0 con 0x7f9898102740 2024-06-10T00:31:57.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.608+0000 7f98957fa700 1 -- 172.21.15.62:0/27336985 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f988800f450 con 0x7f9898102740 2024-06-10T00:31:57.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.609+0000 7f98957fa700 1 --2- 172.21.15.62:0/27336985 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f98800a2c20 0x7f98800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.609+0000 7f98957fa700 1 -- 172.21.15.62:0/27336985 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f98880cefd0 con 0x7f9898102740 2024-06-10T00:31:57.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.609+0000 7f98977fe700 1 --2- 172.21.15.62:0/27336985 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f98800a2c20 0x7f98800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.610+0000 7f98977fe700 1 --2- 172.21.15.62:0/27336985 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f98800a2c20 0x7f98800a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f988c005950 tx=0x7f988c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:57.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.610+0000 7f98a8e65b80 1 Processor -- start 2024-06-10T00:31:57.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.610+0000 7f98a8e65b80 1 -- start start 2024-06-10T00:31:57.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.610+0000 7f98a8e65b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e05f0 0x5560d02e09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.610+0000 7f98a8e65b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02da990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.610+0000 7f98a8e65b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02daed0 0x5560d02db2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.610+0000 7f98a8e65b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5560d022c7c0 con 0x5560d02e0f10 2024-06-10T00:31:57.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.610+0000 7f98a8e65b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5560d01d46d0 con 0x5560d02daed0 2024-06-10T00:31:57.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.610+0000 7f98a8e65b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5560d0228780 con 0x5560d02e05f0 2024-06-10T00:31:57.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.610+0000 7f98977fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02da990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.610+0000 7f98977fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02da990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37358/0 (socket says 172.21.15.62:37358) 2024-06-10T00:31:57.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.610+0000 7f98977fe700 1 -- 172.21.15.62:0/2530099073 learned_addr learned my addr 172.21.15.62:0/2530099073 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:57.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.610+0000 7f9897fff700 1 --2- 172.21.15.62:0/2530099073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e05f0 0x5560d02e09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.610+0000 7f989cb7f700 1 --2- 172.21.15.62:0/2530099073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02daed0 0x5560d02db2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.611+0000 7f98977fe700 1 -- 172.21.15.62:0/2530099073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e05f0 msgr2=0x5560d02e09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:57.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.611+0000 7f98977fe700 1 --2- 172.21.15.62:0/2530099073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e05f0 0x5560d02e09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.611+0000 7f98977fe700 1 -- 172.21.15.62:0/2530099073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02daed0 msgr2=0x5560d02db2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:57.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.611+0000 7f98977fe700 1 --2- 172.21.15.62:0/2530099073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02daed0 0x5560d02db2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.611+0000 7f98977fe700 1 -- 172.21.15.62:0/2530099073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d01caf00 con 0x5560d02e0f10 2024-06-10T00:31:57.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.611+0000 7f9897fff700 1 --2- 172.21.15.62:0/2530099073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e05f0 0x5560d02e09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:31:57.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.611+0000 7f989cb7f700 1 --2- 172.21.15.62:0/2530099073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02daed0 0x5560d02db2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:31:57.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.611+0000 7f98977fe700 1 --2- 172.21.15.62:0/2530099073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02da990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f988c00fd40 tx=0x7f988c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a1fec786371761 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:57.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.611+0000 7f987e7fc700 1 -- 172.21.15.62:0/2530099073 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c022070 con 0x5560d02e0f10 2024-06-10T00:31:57.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.611+0000 7f987e7fc700 1 -- 172.21.15.62:0/2530099073 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f988c011c60 con 0x5560d02e0f10 2024-06-10T00:31:57.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.611+0000 7f987e7fc700 1 -- 172.21.15.62:0/2530099073 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c01d6e0 con 0x5560d02e0f10 2024-06-10T00:31:57.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.612+0000 7f98a8e65b80 1 -- 172.21.15.62:0/2530099073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 msgr2=0x5560d02da990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:57.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.612+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/2530099073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02da990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f988c00fd40 tx=0x7f988c012000 comp rx=0 tx=0).stop 2024-06-10T00:31:57.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.612+0000 7f98a8e65b80 1 -- 172.21.15.62:0/2530099073 shutdown_connections 2024-06-10T00:31:57.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.612+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/2530099073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02daed0 0x5560d02db2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.612+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/2530099073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e05f0 0x5560d02e09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.612+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/2530099073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02da990 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.612+0000 7f98a8e65b80 1 -- 172.21.15.62:0/2530099073 >> 172.21.15.62:0/2530099073 conn(0x5560d0205c70 msgr2=0x5560d02d5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:57.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.612+0000 7f98a8e65b80 1 -- 172.21.15.62:0/2530099073 shutdown_connections 2024-06-10T00:31:57.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.612+0000 7f98a8e65b80 1 -- 172.21.15.62:0/2530099073 wait complete. 2024-06-10T00:31:57.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.612+0000 7f98a8e65b80 1 -- start start 2024-06-10T00:31:57.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.612+0000 7f98a8e65b80 1 Processor -- start 2024-06-10T00:31:57.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f98a8e65b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02daed0 0x5560d02e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f98a8e65b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02e05f0 0x5560d02e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f98a8e65b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e0f10 0x5560d02e4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f98a8e65b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5560d02e7980 con 0x5560d02daed0 2024-06-10T00:31:57.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f98a8e65b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5560d01ccba0 con 0x5560d02e05f0 2024-06-10T00:31:57.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f98a8e65b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5560d02e7b00 con 0x5560d02e0f10 2024-06-10T00:31:57.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f98a8e65b80 10 client.0 _ll_register_callbacks cb 0x5560d01c68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:31:57.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f9897fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02daed0 0x5560d02e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f9897fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02daed0 0x5560d02e6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37372/0 (socket says 172.21.15.62:37372) 2024-06-10T00:31:57.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f9897fff700 1 -- 172.21.15.62:0/3083283560 learned_addr learned my addr 172.21.15.62:0/3083283560 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:57.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f989cb7f700 1 --2- 172.21.15.62:0/3083283560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e0f10 0x5560d02e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f98977fe700 1 --2- 172.21.15.62:0/3083283560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02e05f0 0x5560d02e3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f9897fff700 1 -- 172.21.15.62:0/3083283560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e0f10 msgr2=0x5560d02e4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:57.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f9897fff700 1 --2- 172.21.15.62:0/3083283560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e0f10 0x5560d02e4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f9897fff700 1 -- 172.21.15.62:0/3083283560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02e05f0 msgr2=0x5560d02e3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:57.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f9897fff700 1 --2- 172.21.15.62:0/3083283560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02e05f0 0x5560d02e3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f9897fff700 1 -- 172.21.15.62:0/3083283560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d017b880 con 0x5560d02daed0 2024-06-10T00:31:57.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f9897fff700 1 --2- 172.21.15.62:0/3083283560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02daed0 0x5560d02e6cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9888005950 tx=0x7f988809bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf3596645185f03 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:57.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.613+0000 7f98777fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:31:57.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.614+0000 7f98777fe700 1 -- 172.21.15.62:0/3083283560 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988801d070 con 0x5560d02daed0 2024-06-10T00:31:57.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.614+0000 7f98777fe700 1 -- 172.21.15.62:0/3083283560 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f98880ccbc0 con 0x5560d02daed0 2024-06-10T00:31:57.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.614+0000 7f98a8e65b80 1 -- 172.21.15.62:0/3083283560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f988c011c60 con 0x5560d02daed0 2024-06-10T00:31:57.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.614+0000 7f98777fe700 1 -- 172.21.15.62:0/3083283560 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f98880a41b0 con 0x5560d02daed0 2024-06-10T00:31:57.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.614+0000 7f98a8e65b80 10 client.15153 fetch_fsmap learned FSMap version 29 2024-06-10T00:31:57.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.614+0000 7f98a8e65b80 1 -- 172.21.15.62:0/3083283560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5560d02f22e0 con 0x5560d02daed0 2024-06-10T00:31:57.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.614+0000 7f98777fe700 1 -- 172.21.15.62:0/3083283560 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f988c011c60 con 0x5560d02daed0 2024-06-10T00:31:57.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.614+0000 7f98777fe700 1 -- 172.21.15.62:0/3083283560 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f988809b4a0 con 0x5560d02daed0 2024-06-10T00:31:57.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.614+0000 7f98a8e65b80 10 client.15153 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:31:57.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.614+0000 7f98a8e65b80 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T00:31:57.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.614+0000 7f98a8e65b80 20 client.15153 populate_metadata read hostname 'smithi062' 2024-06-10T00:31:57.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.614+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/3083283560 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5560d02f47f0 0x5560d02f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.614+0000 7f98a8e65b80 4 client.15153 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:31:57.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.614+0000 7f98a8e65b80 1 -- 172.21.15.62:0/3083283560 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560d0239cc0 con 0x5560d02f47f0 2024-06-10T00:31:57.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.615+0000 7f98977fe700 1 --2- 172.21.15.62:0/3083283560 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5560d02f47f0 0x5560d02f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.615+0000 7f98977fe700 1 --2- 172.21.15.62:0/3083283560 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5560d02f47f0 0x5560d02f5c60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db9a39dee6354935 server_cookie=ab7965332fe316d2 in_seq=0 out_seq=0 2024-06-10T00:31:57.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.615+0000 7f98777fe700 10 client.15153 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:31:57.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98777fe700 1 -- 172.21.15.62:0/3083283560 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5560d0239cc0 con 0x5560d02f47f0 2024-06-10T00:31:57.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98777fe700 10 client.15153 handle_command_reply: tid=0 2024-06-10T00:31:57.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98a8e65b80 1 client.15153 shutdown 2024-06-10T00:31:57.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98a8e65b80 1 -- 172.21.15.62:0/3083283560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02daed0 msgr2=0x5560d02e6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:57.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/3083283560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02daed0 0x5560d02e6cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9888005950 tx=0x7f988809bf20 comp rx=0 tx=0).stop 2024-06-10T00:31:57.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98a8e65b80 1 -- 172.21.15.62:0/3083283560 shutdown_connections 2024-06-10T00:31:57.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/3083283560 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5560d02f47f0 0x5560d02f5c60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/3083283560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02daed0 0x5560d02e6cc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/3083283560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e0f10 0x5560d02e4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/3083283560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02e05f0 0x5560d02e3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98a8e65b80 1 -- 172.21.15.62:0/3083283560 >> 172.21.15.62:0/3083283560 conn(0x5560d0205c70 msgr2=0x5560d02d9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:57.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98777fe700 0 client.15153 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:31:57.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98a8e65b80 1 -- 172.21.15.62:0/3083283560 shutdown_connections 2024-06-10T00:31:57.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98a8e65b80 1 -- 172.21.15.62:0/3083283560 wait complete. 2024-06-10T00:31:57.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.616+0000 7f98a8e65b80 20 client.15153 trim_cache size 0 max 16384 2024-06-10T00:31:57.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.638+0000 7f98a8e65b80 1 Processor -- start 2024-06-10T00:31:57.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.638+0000 7f98a8e65b80 1 -- start start 2024-06-10T00:31:57.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.638+0000 7f98a8e65b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02daed0 0x5560d02da990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.638+0000 7f98a8e65b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e05f0 0x5560d02cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.638+0000 7f98a8e65b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.638+0000 7f98a8e65b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5560d02f3fa0 con 0x5560d02e0f10 2024-06-10T00:31:57.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.638+0000 7f98a8e65b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5560d02f4670 con 0x5560d02daed0 2024-06-10T00:31:57.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.638+0000 7f98a8e65b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5560d02f4d40 con 0x5560d02e05f0 2024-06-10T00:31:57.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.639+0000 7f989cb7f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.639+0000 7f989cb7f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02ff0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37380/0 (socket says 172.21.15.62:37380) 2024-06-10T00:31:57.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.639+0000 7f989cb7f700 1 -- 172.21.15.62:0/3185173108 learned_addr learned my addr 172.21.15.62:0/3185173108 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:57.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.639+0000 7f9897fff700 1 --2- 172.21.15.62:0/3185173108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02daed0 0x5560d02da990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.639+0000 7f989cb7f700 1 -- 172.21.15.62:0/3185173108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e05f0 msgr2=0x5560d02cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:57.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.639+0000 7f989cb7f700 1 --2- 172.21.15.62:0/3185173108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e05f0 0x5560d02cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.639+0000 7f989cb7f700 1 -- 172.21.15.62:0/3185173108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02daed0 msgr2=0x5560d02da990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:57.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.639+0000 7f989cb7f700 1 --2- 172.21.15.62:0/3185173108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02daed0 0x5560d02da990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.639+0000 7f989cb7f700 1 -- 172.21.15.62:0/3185173108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d02d7310 con 0x5560d02e0f10 2024-06-10T00:31:57.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.639+0000 7f989cb7f700 1 --2- 172.21.15.62:0/3185173108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02ff0c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9890015000 tx=0x7f9890006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acacc570530f9e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:57.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.639+0000 7f987dffb700 1 -- 172.21.15.62:0/3185173108 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890003c00 con 0x5560d02e0f10 2024-06-10T00:31:57.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.639+0000 7f987dffb700 1 -- 172.21.15.62:0/3185173108 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9890003da0 con 0x5560d02e0f10 2024-06-10T00:31:57.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.639+0000 7f987dffb700 1 -- 172.21.15.62:0/3185173108 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890005500 con 0x5560d02e0f10 2024-06-10T00:31:57.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.640+0000 7f98a8e65b80 1 -- 172.21.15.62:0/3185173108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 msgr2=0x5560d02ff0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:57.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.640+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/3185173108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02ff0c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9890015000 tx=0x7f9890006210 comp rx=0 tx=0).stop 2024-06-10T00:31:57.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.640+0000 7f98a8e65b80 1 -- 172.21.15.62:0/3185173108 shutdown_connections 2024-06-10T00:31:57.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.640+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/3185173108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02ff0c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.640+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/3185173108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02daed0 0x5560d02da990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.640+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/3185173108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02e05f0 0x5560d02cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.640+0000 7f98a8e65b80 1 -- 172.21.15.62:0/3185173108 >> 172.21.15.62:0/3185173108 conn(0x5560d0205c70 msgr2=0x5560d02cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:57.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.640+0000 7f98a8e65b80 1 -- 172.21.15.62:0/3185173108 shutdown_connections 2024-06-10T00:31:57.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.640+0000 7f98a8e65b80 1 -- 172.21.15.62:0/3185173108 wait complete. 2024-06-10T00:31:57.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.640+0000 7f98a8e65b80 1 -- start start 2024-06-10T00:31:57.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98a8e65b80 1 Processor -- start 2024-06-10T00:31:57.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98a8e65b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02daed0 0x5560d02d9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98a8e65b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02e05f0 0x5560d02da2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98a8e65b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02da820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98a8e65b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5560d02f5ae0 con 0x5560d02e0f10 2024-06-10T00:31:57.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98a8e65b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5560d02f5410 con 0x5560d02e05f0 2024-06-10T00:31:57.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98a8e65b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5560d02f61b0 con 0x5560d02daed0 2024-06-10T00:31:57.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98a8e65b80 10 client.0 _ll_register_callbacks cb 0x5560d01c68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:31:57.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f989cb7f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02da820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98977fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02e05f0 0x5560d02da2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f989cb7f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02da820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37382/0 (socket says 172.21.15.62:37382) 2024-06-10T00:31:57.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f989cb7f700 1 -- 172.21.15.62:0/4259884788 learned_addr learned my addr 172.21.15.62:0/4259884788 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:31:57.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f9897fff700 1 --2- 172.21.15.62:0/4259884788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02daed0 0x5560d02d9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98977fe700 1 -- 172.21.15.62:0/4259884788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02daed0 msgr2=0x5560d02d9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:57.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98977fe700 1 --2- 172.21.15.62:0/4259884788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02daed0 0x5560d02d9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98977fe700 1 -- 172.21.15.62:0/4259884788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 msgr2=0x5560d02da820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:31:57.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98977fe700 1 --2- 172.21.15.62:0/4259884788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02da820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98977fe700 1 -- 172.21.15.62:0/4259884788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d02fd740 con 0x5560d02e05f0 2024-06-10T00:31:57.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98977fe700 1 --2- 172.21.15.62:0/4259884788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02e05f0 0x5560d02da2e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f988c00fd10 tx=0x7f988c011f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a136e159af30a2ed server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:31:57.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.641+0000 7f98777fe700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:31:57.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.642+0000 7f98777fe700 1 -- 172.21.15.62:0/4259884788 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c022050 con 0x5560d02e05f0 2024-06-10T00:31:57.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.642+0000 7f98777fe700 1 -- 172.21.15.62:0/4259884788 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f988c005240 con 0x5560d02e05f0 2024-06-10T00:31:57.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.642+0000 7f98a8e65b80 1 -- 172.21.15.62:0/4259884788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9890003da0 con 0x5560d02e05f0 2024-06-10T00:31:57.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.642+0000 7f98777fe700 1 -- 172.21.15.62:0/4259884788 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f988c042050 con 0x5560d02e05f0 2024-06-10T00:31:57.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.642+0000 7f98a8e65b80 10 client.24824 fetch_fsmap learned FSMap version 29 2024-06-10T00:31:57.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.642+0000 7f98a8e65b80 1 -- 172.21.15.62:0/4259884788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5560d03603c0 con 0x5560d02e05f0 2024-06-10T00:31:57.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.642+0000 7f98777fe700 1 -- 172.21.15.62:0/4259884788 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9890003da0 con 0x5560d02e05f0 2024-06-10T00:31:57.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.642+0000 7f98777fe700 1 -- 172.21.15.62:0/4259884788 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f988c01e4e0 con 0x5560d02e05f0 2024-06-10T00:31:57.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.642+0000 7f98a8e65b80 10 client.24824 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:31:57.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.642+0000 7f98a8e65b80 10 client.24824 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T00:31:57.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.642+0000 7f98a8e65b80 20 client.24824 populate_metadata read hostname 'smithi062' 2024-06-10T00:31:57.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.642+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/4259884788 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5560d0364c50 0x5560d0363960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:31:57.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.642+0000 7f98a8e65b80 4 client.24824 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:31:57.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.642+0000 7f98a8e65b80 1 -- 172.21.15.62:0/4259884788 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "status"}) -- 0x5560d0234240 con 0x5560d0364c50 2024-06-10T00:31:57.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.643+0000 7f9897fff700 1 --2- 172.21.15.62:0/4259884788 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5560d0364c50 0x5560d0363960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:31:57.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.643+0000 7f98777fe700 10 client.24824 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:31:57.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.643+0000 7f9897fff700 1 --2- 172.21.15.62:0/4259884788 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5560d0364c50 0x5560d0363960 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e189c3b6b0fcafbc server_cookie=65c9daa2b73c0b20 in_seq=0 out_seq=0 2024-06-10T00:31:57.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.643+0000 7f98777fe700 1 -- 172.21.15.62:0/4259884788 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5560d0234240 con 0x5560d0364c50 2024-06-10T00:31:57.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.643+0000 7f98777fe700 10 client.24824 handle_command_reply: tid=0 2024-06-10T00:31:57.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.643+0000 7f98a8e65b80 1 client.24824 shutdown 2024-06-10T00:31:57.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.643+0000 7f98a8e65b80 1 -- 172.21.15.62:0/4259884788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02e05f0 msgr2=0x5560d02da2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:57.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.643+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/4259884788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02e05f0 0x5560d02da2e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f988c00fd10 tx=0x7f988c011f80 comp rx=0 tx=0).stop 2024-06-10T00:31:57.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.643+0000 7f98a8e65b80 1 -- 172.21.15.62:0/4259884788 shutdown_connections 2024-06-10T00:31:57.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.643+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/4259884788 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5560d0364c50 0x5560d0363960 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.643+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/4259884788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5560d02e0f10 0x5560d02da820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.643+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/4259884788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5560d02daed0 0x5560d02d9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.643+0000 7f98777fe700 0 client.24824 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:31:57.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.643+0000 7f98a8e65b80 1 --2- 172.21.15.62:0/4259884788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560d02e05f0 0x5560d02da2e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f98a8e65b80 1 -- 172.21.15.62:0/4259884788 >> 172.21.15.62:0/4259884788 conn(0x5560d0205c70 msgr2=0x5560d02e12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:57.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f98a8e65b80 1 -- 172.21.15.62:0/4259884788 shutdown_connections 2024-06-10T00:31:57.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f98a8e65b80 1 -- 172.21.15.62:0/4259884788 wait complete. 2024-06-10T00:31:57.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f98a8e65b80 20 client.24824 trim_cache size 0 max 16384 2024-06-10T00:31:57.654 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:31:57.654 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:31:57.654 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-06-10T00:31:57.654 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24464, 2024-06-10T00:31:57.655 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:31:57.655 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:31:57.655 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:31:57.655 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 197.99992818300001, 2024-06-10T00:31:57.655 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 29, 2024-06-10T00:31:57.655 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 99, 2024-06-10T00:31:57.655 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T00:31:57.655 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 215.72506227900001 2024-06-10T00:31:57.656 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:31:57.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f989e5e2700 1 -- 172.21.15.62:0/27336985 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f98800a2c20 msgr2=0x7f98800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:57.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f989e5e2700 1 --2- 172.21.15.62:0/27336985 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f98800a2c20 0x7f98800a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f988c005950 tx=0x7f988c0058e0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f989e5e2700 1 -- 172.21.15.62:0/27336985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9898102740 msgr2=0x7f989819e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:31:57.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f989e5e2700 1 --2- 172.21.15.62:0/27336985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9898102740 0x7f989819e7e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f988800c920 tx=0x7f988800cc30 comp rx=0 tx=0).stop 2024-06-10T00:31:57.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f989e5e2700 1 -- 172.21.15.62:0/27336985 shutdown_connections 2024-06-10T00:31:57.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f989e5e2700 1 --2- 172.21.15.62:0/27336985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9898102740 0x7f989819e7e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f989e5e2700 1 --2- 172.21.15.62:0/27336985 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f98800a2c20 0x7f98800a50e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f989e5e2700 1 --2- 172.21.15.62:0/27336985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9898107840 0x7f98981989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f989e5e2700 1 --2- 172.21.15.62:0/27336985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98981030f0 0x7f989819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:31:57.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f989e5e2700 1 -- 172.21.15.62:0/27336985 >> 172.21.15.62:0/27336985 conn(0x7f98980795f0 msgr2=0x7f9898100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:31:57.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.644+0000 7f989e5e2700 1 -- 172.21.15.62:0/27336985 shutdown_connections 2024-06-10T00:31:57.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:31:57.645+0000 7f989e5e2700 1 -- 172.21.15.62:0/27336985 wait complete. 2024-06-10T00:31:57.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:57 smithi062 ceph-mon[32390]: pgmap v406: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:31:58.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:57 smithi105 ceph-mon[36321]: pgmap v406: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:31:58.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:57 smithi138 ceph-mon[36549]: pgmap v406: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:31:58.292 INFO:tasks.cephadm:Setup complete, yielding 2024-06-10T00:31:58.292 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-10T00:31:58.306 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-06-10T00:31:58.306 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-10T00:31:58.557 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:31:59.968 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:31:59 smithi062 ceph-mon[32390]: pgmap v407: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:00.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:31:59 smithi105 ceph-mon[36321]: pgmap v407: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:00.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:31:59 smithi138 ceph-mon[36549]: pgmap v407: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:00.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.237+0000 7f108454b700 1 -- 172.21.15.62:0/316243286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f107c104090 msgr2=0x7f107c10a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:00.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.237+0000 7f108454b700 1 --2- 172.21.15.62:0/316243286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f107c104090 0x7f107c10a940 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f107400b0d0 tx=0x7f107400b3e0 comp rx=0 tx=0).stop 2024-06-10T00:32:00.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.237+0000 7f108454b700 1 -- 172.21.15.62:0/316243286 shutdown_connections 2024-06-10T00:32:00.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.237+0000 7f108454b700 1 --2- 172.21.15.62:0/316243286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f107c104090 0x7f107c10a940 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:00.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.237+0000 7f108454b700 1 --2- 172.21.15.62:0/316243286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f107c1024f0 0x7f107c102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:00.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.237+0000 7f108454b700 1 --2- 172.21.15.62:0/316243286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f107c1036f0 0x7f107c103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:00.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.237+0000 7f108454b700 1 -- 172.21.15.62:0/316243286 >> 172.21.15.62:0/316243286 conn(0x7f107c0fda90 msgr2=0x7f107c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:00.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.237+0000 7f108454b700 1 -- 172.21.15.62:0/316243286 shutdown_connections 2024-06-10T00:32:00.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.237+0000 7f108454b700 1 -- 172.21.15.62:0/316243286 wait complete. 2024-06-10T00:32:00.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.237+0000 7f108454b700 1 Processor -- start 2024-06-10T00:32:00.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.237+0000 7f108454b700 1 -- start start 2024-06-10T00:32:00.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f108454b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f107c1024f0 0x7f107c076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f108454b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f107c1036f0 0x7f107c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f108454b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f107c104090 0x7f107c074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f108454b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f107c10e0c0 con 0x7f107c1036f0 2024-06-10T00:32:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f108454b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f107c10df40 con 0x7f107c104090 2024-06-10T00:32:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f108454b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f107c10e240 con 0x7f107c1024f0 2024-06-10T00:32:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f1081ae6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f107c1036f0 0x7f107c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f1081ae6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f107c1036f0 0x7f107c074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39610/0 (socket says 172.21.15.62:39610) 2024-06-10T00:32:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f1081ae6700 1 -- 172.21.15.62:0/3074276454 learned_addr learned my addr 172.21.15.62:0/3074276454 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f1082ae8700 1 --2- 172.21.15.62:0/3074276454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f107c104090 0x7f107c074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f10822e7700 1 --2- 172.21.15.62:0/3074276454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f107c1024f0 0x7f107c076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f1081ae6700 1 -- 172.21.15.62:0/3074276454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f107c1024f0 msgr2=0x7f107c076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f1081ae6700 1 --2- 172.21.15.62:0/3074276454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f107c1024f0 0x7f107c076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f1081ae6700 1 -- 172.21.15.62:0/3074276454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f107c104090 msgr2=0x7f107c074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f1081ae6700 1 --2- 172.21.15.62:0/3074276454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f107c104090 0x7f107c074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f1081ae6700 1 -- 172.21.15.62:0/3074276454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107c075670 con 0x7f107c1036f0 2024-06-10T00:32:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f1081ae6700 1 --2- 172.21.15.62:0/3074276454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f107c1036f0 0x7f107c074880 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f106c00c920 tx=0x7f106c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f10737fe700 1 -- 172.21.15.62:0/3074276454 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c007b00 con 0x7f107c1036f0 2024-06-10T00:32:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f108454b700 1 -- 172.21.15.62:0/3074276454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107c1a2770 con 0x7f107c1036f0 2024-06-10T00:32:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f10737fe700 1 -- 172.21.15.62:0/3074276454 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f106c004d10 con 0x7f107c1036f0 2024-06-10T00:32:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.238+0000 7f10737fe700 1 -- 172.21.15.62:0/3074276454 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c005630 con 0x7f107c1036f0 2024-06-10T00:32:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.239+0000 7f108454b700 1 -- 172.21.15.62:0/3074276454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f107c1a2c60 con 0x7f107c1036f0 2024-06-10T00:32:00.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.239+0000 7f108454b700 1 -- 172.21.15.62:0/3074276454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f107c069180 con 0x7f107c1036f0 2024-06-10T00:32:00.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.241+0000 7f10737fe700 1 -- 172.21.15.62:0/3074276454 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f106c004750 con 0x7f107c1036f0 2024-06-10T00:32:00.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.241+0000 7f10737fe700 1 --2- 172.21.15.62:0/3074276454 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f10680a2c20 0x7f10680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:00.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.242+0000 7f10822e7700 1 --2- 172.21.15.62:0/3074276454 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f10680a2c20 0x7f10680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:00.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.242+0000 7f10737fe700 1 -- 172.21.15.62:0/3074276454 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f106c0cf3e0 con 0x7f107c1036f0 2024-06-10T00:32:00.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.242+0000 7f10822e7700 1 --2- 172.21.15.62:0/3074276454 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f10680a2c20 0x7f10680a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1078005fd0 tx=0x7f1078005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:00.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.245+0000 7f10737fe700 1 -- 172.21.15.62:0/3074276454 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f106c014030 con 0x7f107c1036f0 2024-06-10T00:32:00.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.469+0000 7f108454b700 1 -- 172.21.15.62:0/3074276454 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f107c0628d0 con 0x7f10680a2c20 2024-06-10T00:32:00.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.471+0000 7f10737fe700 1 -- 172.21.15.62:0/3074276454 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f107c0628d0 con 0x7f10680a2c20 2024-06-10T00:32:00.472 INFO:teuthology.orchestra.run.smithi062.stdout:Backend: cephadm 2024-06-10T00:32:00.472 INFO:teuthology.orchestra.run.smithi062.stdout:Available: Yes 2024-06-10T00:32:00.472 INFO:teuthology.orchestra.run.smithi062.stdout:Paused: No 2024-06-10T00:32:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.473+0000 7f108454b700 1 -- 172.21.15.62:0/3074276454 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f10680a2c20 msgr2=0x7f10680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.473+0000 7f108454b700 1 --2- 172.21.15.62:0/3074276454 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f10680a2c20 0x7f10680a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1078005fd0 tx=0x7f1078005e50 comp rx=0 tx=0).stop 2024-06-10T00:32:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.473+0000 7f108454b700 1 -- 172.21.15.62:0/3074276454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f107c1036f0 msgr2=0x7f107c074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.473+0000 7f108454b700 1 --2- 172.21.15.62:0/3074276454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f107c1036f0 0x7f107c074880 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f106c00c920 tx=0x7f106c00cc30 comp rx=0 tx=0).stop 2024-06-10T00:32:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.473+0000 7f108454b700 1 -- 172.21.15.62:0/3074276454 shutdown_connections 2024-06-10T00:32:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.473+0000 7f108454b700 1 --2- 172.21.15.62:0/3074276454 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f10680a2c20 0x7f10680a50e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.473+0000 7f108454b700 1 --2- 172.21.15.62:0/3074276454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f107c104090 0x7f107c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.473+0000 7f108454b700 1 --2- 172.21.15.62:0/3074276454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f107c1024f0 0x7f107c076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.473+0000 7f108454b700 1 --2- 172.21.15.62:0/3074276454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f107c1036f0 0x7f107c074880 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.473+0000 7f108454b700 1 -- 172.21.15.62:0/3074276454 >> 172.21.15.62:0/3074276454 conn(0x7f107c0fda90 msgr2=0x7f107c106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.473+0000 7f108454b700 1 -- 172.21.15.62:0/3074276454 shutdown_connections 2024-06-10T00:32:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:00.473+0000 7f108454b700 1 -- 172.21.15.62:0/3074276454 wait complete. 2024-06-10T00:32:01.137 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-10T00:32:01.389 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:02.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:01 smithi105 ceph-mon[36321]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:02.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:01 smithi105 ceph-mon[36321]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:02.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:01 smithi062 ceph-mon[32390]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:02.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:01 smithi062 ceph-mon[32390]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:02.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:01 smithi138 ceph-mon[36549]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:02.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:01 smithi138 ceph-mon[36549]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.060+0000 7fee6ba2a700 1 -- 172.21.15.62:0/1311479089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee64101430 msgr2=0x7fee64101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.060+0000 7fee6ba2a700 1 --2- 172.21.15.62:0/1311479089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee64101430 0x7fee64101850 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fee600099c0 tx=0x7fee60009cd0 comp rx=0 tx=0).stop 2024-06-10T00:32:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.060+0000 7fee6ba2a700 1 -- 172.21.15.62:0/1311479089 shutdown_connections 2024-06-10T00:32:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.060+0000 7fee6ba2a700 1 --2- 172.21.15.62:0/1311479089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee64101430 0x7fee64101850 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.060+0000 7fee6ba2a700 1 --2- 172.21.15.62:0/1311479089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee64102fd0 0x7fee64109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.060+0000 7fee6ba2a700 1 --2- 172.21.15.62:0/1311479089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee64102630 0x7fee64102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.060+0000 7fee6ba2a700 1 -- 172.21.15.62:0/1311479089 >> 172.21.15.62:0/1311479089 conn(0x7fee640fc9f0 msgr2=0x7fee640fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.060+0000 7fee6ba2a700 1 -- 172.21.15.62:0/1311479089 shutdown_connections 2024-06-10T00:32:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.060+0000 7fee6ba2a700 1 -- 172.21.15.62:0/1311479089 wait complete. 2024-06-10T00:32:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.061+0000 7fee6ba2a700 1 Processor -- start 2024-06-10T00:32:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.061+0000 7fee6ba2a700 1 -- start start 2024-06-10T00:32:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.061+0000 7fee6ba2a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee64101430 0x7fee6406b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.061+0000 7fee6ba2a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee64102630 0x7fee6419d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:03.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.061+0000 7fee6ba2a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee64102fd0 0x7fee6419fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.061+0000 7fee6ba2a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fee6410d000 con 0x7fee64102fd0 2024-06-10T00:32:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.061+0000 7fee6ba2a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fee6410ce80 con 0x7fee64101430 2024-06-10T00:32:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.061+0000 7fee6ba2a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee6410d180 con 0x7fee64102630 2024-06-10T00:32:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.061+0000 7fee69fc7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee64102fd0 0x7fee6419fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.061+0000 7fee69fc7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee64102fd0 0x7fee6419fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39628/0 (socket says 172.21.15.62:39628) 2024-06-10T00:32:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.061+0000 7fee69fc7700 1 -- 172.21.15.62:0/2757873907 learned_addr learned my addr 172.21.15.62:0/2757873907 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.061+0000 7fee697c6700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee64101430 0x7fee6406b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.061+0000 7fee68fc5700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee64102630 0x7fee6419d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.062+0000 7fee69fc7700 1 -- 172.21.15.62:0/2757873907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee64102630 msgr2=0x7fee6419d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.062+0000 7fee69fc7700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee64102630 0x7fee6419d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:03.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.062+0000 7fee69fc7700 1 -- 172.21.15.62:0/2757873907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee64101430 msgr2=0x7fee6406b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.062+0000 7fee69fc7700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee64101430 0x7fee6406b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.062+0000 7fee69fc7700 1 -- 172.21.15.62:0/2757873907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee641a02f0 con 0x7fee64102fd0 2024-06-10T00:32:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.062+0000 7fee697c6700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee64101430 0x7fee6406b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:32:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.062+0000 7fee69fc7700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee64102fd0 0x7fee6419fa20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fee5c007d70 tx=0x7fee5c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.062+0000 7fee5a7fc700 1 -- 172.21.15.62:0/2757873907 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c016020 con 0x7fee64102fd0 2024-06-10T00:32:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.062+0000 7fee6ba2a700 1 -- 172.21.15.62:0/2757873907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee641a05e0 con 0x7fee64102fd0 2024-06-10T00:32:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.062+0000 7fee5a7fc700 1 -- 172.21.15.62:0/2757873907 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5c00ad40 con 0x7fee64102fd0 2024-06-10T00:32:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.062+0000 7fee5a7fc700 1 -- 172.21.15.62:0/2757873907 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c011600 con 0x7fee64102fd0 2024-06-10T00:32:03.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.062+0000 7fee6ba2a700 1 -- 172.21.15.62:0/2757873907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee641a0ad0 con 0x7fee64102fd0 2024-06-10T00:32:03.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.063+0000 7fee6ba2a700 1 -- 172.21.15.62:0/2757873907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee64069180 con 0x7fee64102fd0 2024-06-10T00:32:03.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.064+0000 7fee5a7fc700 1 -- 172.21.15.62:0/2757873907 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fee5c019020 con 0x7fee64102fd0 2024-06-10T00:32:03.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.065+0000 7fee5a7fc700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fee500a2c20 0x7fee500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:03.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.065+0000 7fee697c6700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fee500a2c20 0x7fee500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:03.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.065+0000 7fee5a7fc700 1 -- 172.21.15.62:0/2757873907 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fee5c0cf7d0 con 0x7fee64102fd0 2024-06-10T00:32:03.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.066+0000 7fee697c6700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fee500a2c20 0x7fee500a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee60003890 tx=0x7fee60005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:03.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.069+0000 7fee5a7fc700 1 -- 172.21.15.62:0/2757873907 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fee5c020020 con 0x7fee64102fd0 2024-06-10T00:32:03.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.288+0000 7fee6ba2a700 1 -- 172.21.15.62:0/2757873907 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fee640628d0 con 0x7fee500a2c20 2024-06-10T00:32:03.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.302+0000 7fee5a7fc700 1 -- 172.21.15.62:0/2757873907 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fee640628d0 con 0x7fee500a2c20 2024-06-10T00:32:03.302 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-10T00:32:03.302 INFO:teuthology.orchestra.run.smithi062.stdout:mds.a smithi062 running (3m) 44s ago 3m 18.9M - 19.0.0-2946-g3d89accf b045ac545f82 9267421457af 2024-06-10T00:32:03.302 INFO:teuthology.orchestra.run.smithi062.stdout:mds.b smithi105 running (3m) 85s ago 3m 25.4M - 19.0.0-2946-g3d89accf b045ac545f82 c7fd797fca0b 2024-06-10T00:32:03.302 INFO:teuthology.orchestra.run.smithi062.stdout:mds.c smithi138 running (3m) 85s ago 3m 20.6M - 19.0.0-2946-g3d89accf b045ac545f82 c3949f77fe3f 2024-06-10T00:32:03.302 INFO:teuthology.orchestra.run.smithi062.stdout:mds.d smithi062 running (4m) 44s ago 4m 15.9M - 19.0.0-2946-g3d89accf b045ac545f82 6b93b2f244cc 2024-06-10T00:32:03.302 INFO:teuthology.orchestra.run.smithi062.stdout:mds.e smithi105 running (4m) 85s ago 4m 18.3M - 19.0.0-2946-g3d89accf b045ac545f82 209bf3ad6388 2024-06-10T00:32:03.302 INFO:teuthology.orchestra.run.smithi062.stdout:mds.f smithi138 running (4m) 85s ago 4m 19.6M - 19.0.0-2946-g3d89accf b045ac545f82 41fcb2b3c9be 2024-06-10T00:32:03.302 INFO:teuthology.orchestra.run.smithi062.stdout:mds.g smithi062 running (3m) 44s ago 3m 20.1M - 19.0.0-2946-g3d89accf b045ac545f82 55f780cf7a11 2024-06-10T00:32:03.302 INFO:teuthology.orchestra.run.smithi062.stdout:mds.h smithi105 running (3m) 85s ago 3m 17.5M - 19.0.0-2946-g3d89accf b045ac545f82 abed1b6864ba 2024-06-10T00:32:03.302 INFO:teuthology.orchestra.run.smithi062.stdout:mds.i smithi138 running (3m) 85s ago 3m 20.0M - 19.0.0-2946-g3d89accf b045ac545f82 13a4245f0167 2024-06-10T00:32:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:mds.j smithi062 running (4m) 44s ago 4m 14.6M - 19.0.0-2946-g3d89accf b045ac545f82 06963e5804a5 2024-06-10T00:32:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:mds.k smithi105 running (4m) 85s ago 4m 16.3M - 19.0.0-2946-g3d89accf b045ac545f82 0ccb9711c5ca 2024-06-10T00:32:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:mds.l smithi138 running (4m) 85s ago 4m 17.3M - 19.0.0-2946-g3d89accf b045ac545f82 f5decf1d16c4 2024-06-10T00:32:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi062 *:9283,8765 running (11m) 44s ago 11m 518M - 19.0.0-2946-g3d89accf b045ac545f82 e454fa394545 2024-06-10T00:32:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi105 *:8443,8765 running (9m) 85s ago 9m 456M - 19.0.0-2946-g3d89accf b045ac545f82 313819fe3621 2024-06-10T00:32:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.z smithi138 *:8443,8765 running (9m) 85s ago 9m 454M - 19.0.0-2946-g3d89accf b045ac545f82 2cfb57db6f25 2024-06-10T00:32:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (11m) 44s ago 11m 53.2M 2048M 19.0.0-2946-g3d89accf b045ac545f82 4f3ad4cc1fad 2024-06-10T00:32:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi105 running (9m) 85s ago 9m 38.9M 2048M 19.0.0-2946-g3d89accf b045ac545f82 00abad0a9f83 2024-06-10T00:32:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi138 running (9m) 85s ago 9m 39.0M 2048M 19.0.0-2946-g3d89accf b045ac545f82 f0c99954ecf6 2024-06-10T00:32:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (9m) 44s ago 9m 53.4M 4272M 19.0.0-2946-g3d89accf b045ac545f82 b92386682311 2024-06-10T00:32:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi105 running (8m) 85s ago 8m 56.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 5e101094640a 2024-06-10T00:32:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi138 running (8m) 85s ago 8m 55.2M 4272M 19.0.0-2946-g3d89accf b045ac545f82 b2438d7ff1f1 2024-06-10T00:32:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (7m) 44s ago 7m 47.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 a57c79574a14 2024-06-10T00:32:03.303 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi105 running (7m) 85s ago 7m 51.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 c474be70c4d1 2024-06-10T00:32:03.304 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi138 running (7m) 85s ago 7m 45.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 e01cc1b8ed95 2024-06-10T00:32:03.304 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi062 running (6m) 44s ago 6m 45.1M 4272M 19.0.0-2946-g3d89accf b045ac545f82 22fcd5005064 2024-06-10T00:32:03.304 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi105 running (6m) 85s ago 6m 44.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 4581264367e0 2024-06-10T00:32:03.304 INFO:teuthology.orchestra.run.smithi062.stdout:osd.8 smithi138 running (5m) 85s ago 5m 53.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 a50e1e94001f 2024-06-10T00:32:03.304 INFO:teuthology.orchestra.run.smithi062.stdout:osd.9 smithi062 running (5m) 44s ago 5m 43.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 4959bb401506 2024-06-10T00:32:03.304 INFO:teuthology.orchestra.run.smithi062.stdout:osd.10 smithi105 running (5m) 85s ago 5m 46.4M 4272M 19.0.0-2946-g3d89accf b045ac545f82 0eae061b0238 2024-06-10T00:32:03.304 INFO:teuthology.orchestra.run.smithi062.stdout:osd.11 smithi138 running (4m) 85s ago 4m 52.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 b4da41759dae 2024-06-10T00:32:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.304+0000 7fee6ba2a700 1 -- 172.21.15.62:0/2757873907 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fee500a2c20 msgr2=0x7fee500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.304+0000 7fee6ba2a700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fee500a2c20 0x7fee500a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee60003890 tx=0x7fee60005fd0 comp rx=0 tx=0).stop 2024-06-10T00:32:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.304+0000 7fee6ba2a700 1 -- 172.21.15.62:0/2757873907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee64102fd0 msgr2=0x7fee6419fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.304+0000 7fee6ba2a700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee64102fd0 0x7fee6419fa20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fee5c007d70 tx=0x7fee5c00a340 comp rx=0 tx=0).stop 2024-06-10T00:32:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.304+0000 7fee6ba2a700 1 -- 172.21.15.62:0/2757873907 shutdown_connections 2024-06-10T00:32:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.304+0000 7fee6ba2a700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee64102fd0 0x7fee6419fa20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.304+0000 7fee6ba2a700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fee500a2c20 0x7fee500a50e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.304+0000 7fee6ba2a700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee64101430 0x7fee6406b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.304+0000 7fee6ba2a700 1 --2- 172.21.15.62:0/2757873907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee64102630 0x7fee6419d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.304+0000 7fee6ba2a700 1 -- 172.21.15.62:0/2757873907 >> 172.21.15.62:0/2757873907 conn(0x7fee640fc9f0 msgr2=0x7fee64105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.304+0000 7fee6ba2a700 1 -- 172.21.15.62:0/2757873907 shutdown_connections 2024-06-10T00:32:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:03.304+0000 7fee6ba2a700 1 -- 172.21.15.62:0/2757873907 wait complete. 2024-06-10T00:32:04.005 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-10T00:32:04.029 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:03 smithi062 ceph-mon[32390]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:32:04.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:03 smithi105 ceph-mon[36321]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:32:04.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:03 smithi138 ceph-mon[36549]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:32:04.254 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:05.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:04 smithi105 ceph-mon[36321]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:05.059 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:04 smithi062 ceph-mon[32390]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:05.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:04 smithi138 ceph-mon[36549]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:05.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.941+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/321383638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1f41025f0 msgr2=0x7fe1f4102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:05.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.941+0000 7fe1f9ba7700 1 --2- 172.21.15.62:0/321383638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1f41025f0 0x7fe1f4102a50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80099c0 tx=0x7fe1e8009cd0 comp rx=0 tx=0).stop 2024-06-10T00:32:05.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.941+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/321383638 shutdown_connections 2024-06-10T00:32:05.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.941+0000 7fe1f9ba7700 1 --2- 172.21.15.62:0/321383638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f4102f90 0x7fe1f4109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:05.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.941+0000 7fe1f9ba7700 1 --2- 172.21.15.62:0/321383638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f41013f0 0x7fe1f4101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:05.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.941+0000 7fe1f9ba7700 1 --2- 172.21.15.62:0/321383638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1f41025f0 0x7fe1f4102a50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:05.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.941+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/321383638 >> 172.21.15.62:0/321383638 conn(0x7fe1f40fc9b0 msgr2=0x7fe1f40fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:05.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.941+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/321383638 shutdown_connections 2024-06-10T00:32:05.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.942+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/321383638 wait complete. 2024-06-10T00:32:05.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.942+0000 7fe1f9ba7700 1 Processor -- start 2024-06-10T00:32:05.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.942+0000 7fe1f9ba7700 1 -- start start 2024-06-10T00:32:05.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.942+0000 7fe1f9ba7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1f41013f0 0x7fe1f4105380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:05.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.942+0000 7fe1f9ba7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f41025f0 0x7fe1f41039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:05.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.942+0000 7fe1f9ba7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f4102f90 0x7fe1f4103f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:05.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f9ba7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f410cfc0 con 0x7fe1f41013f0 2024-06-10T00:32:05.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f9ba7700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f410ce40 con 0x7fe1f4102f90 2024-06-10T00:32:05.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f9ba7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f410d140 con 0x7fe1f41025f0 2024-06-10T00:32:05.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1f41013f0 0x7fe1f4105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:05.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1f41013f0 0x7fe1f4105380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39638/0 (socket says 172.21.15.62:39638) 2024-06-10T00:32:05.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f37fe700 1 -- 172.21.15.62:0/3781133330 learned_addr learned my addr 172.21.15.62:0/3781133330 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:05.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f3fff700 1 --2- 172.21.15.62:0/3781133330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f4102f90 0x7fe1f4103f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:05.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f2ffd700 1 --2- 172.21.15.62:0/3781133330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f41025f0 0x7fe1f41039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:05.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f37fe700 1 -- 172.21.15.62:0/3781133330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f41025f0 msgr2=0x7fe1f41039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:05.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f37fe700 1 --2- 172.21.15.62:0/3781133330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f41025f0 0x7fe1f41039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:05.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f37fe700 1 -- 172.21.15.62:0/3781133330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f4102f90 msgr2=0x7fe1f4103f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:05.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f37fe700 1 --2- 172.21.15.62:0/3781133330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f4102f90 0x7fe1f4103f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:05.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f37fe700 1 -- 172.21.15.62:0/3781133330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f41047c0 con 0x7fe1f41013f0 2024-06-10T00:32:05.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f37fe700 1 --2- 172.21.15.62:0/3781133330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1f41013f0 0x7fe1f4105380 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400d980 tx=0x7fe1e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:05.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f0ff9700 1 -- 172.21.15.62:0/3781133330 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4016070 con 0x7fe1f41013f0 2024-06-10T00:32:05.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/3781133330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f41a2860 con 0x7fe1f41013f0 2024-06-10T00:32:05.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f0ff9700 1 -- 172.21.15.62:0/3781133330 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1e400be30 con 0x7fe1f41013f0 2024-06-10T00:32:05.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.943+0000 7fe1f0ff9700 1 -- 172.21.15.62:0/3781133330 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e400f3c0 con 0x7fe1f41013f0 2024-06-10T00:32:05.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.944+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/3781133330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f41a2da0 con 0x7fe1f41013f0 2024-06-10T00:32:05.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.944+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/3781133330 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f4069180 con 0x7fe1f41013f0 2024-06-10T00:32:05.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.946+0000 7fe1f0ff9700 1 -- 172.21.15.62:0/3781133330 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe1e4019070 con 0x7fe1f41013f0 2024-06-10T00:32:05.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.946+0000 7fe1f0ff9700 1 --2- 172.21.15.62:0/3781133330 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe1dc0a2c20 0x7fe1dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:05.947 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:05 smithi062 ceph-mon[32390]: pgmap v410: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:32:05.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.947+0000 7fe1f2ffd700 1 --2- 172.21.15.62:0/3781133330 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe1dc0a2c20 0x7fe1dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:05.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.947+0000 7fe1f0ff9700 1 -- 172.21.15.62:0/3781133330 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fe1e40cefc0 con 0x7fe1f41013f0 2024-06-10T00:32:05.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.947+0000 7fe1f2ffd700 1 --2- 172.21.15.62:0/3781133330 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe1dc0a2c20 0x7fe1dc0a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8000c00 tx=0x7fe1e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:05.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:05.951+0000 7fe1f0ff9700 1 -- 172.21.15.62:0/3781133330 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe1e4020030 con 0x7fe1f41013f0 2024-06-10T00:32:06.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:05 smithi105 ceph-mon[36321]: pgmap v410: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:32:06.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:05 smithi138 ceph-mon[36549]: pgmap v410: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:32:06.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:06.169+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/3781133330 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fe1f40feaf0 con 0x7fe1dc0a2c20 2024-06-10T00:32:06.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:06.175+0000 7fe1f0ff9700 1 -- 172.21.15.62:0/3781133330 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fe1f40feaf0 con 0x7fe1dc0a2c20 2024-06-10T00:32:06.175 INFO:teuthology.orchestra.run.smithi062.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-10T00:32:06.175 INFO:teuthology.orchestra.run.smithi062.stdout:mds.all 12/12 88s ago 4m smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l;count:12 2024-06-10T00:32:06.175 INFO:teuthology.orchestra.run.smithi062.stdout:mgr 3/3 88s ago 9m smithi062=x;smithi105=y;smithi138=z;count:3 2024-06-10T00:32:06.175 INFO:teuthology.orchestra.run.smithi062.stdout:mon 3/3 88s ago 10m smithi062:172.21.15.62=a;smithi105:172.21.15.105=b;smithi138:172.21.15.138=c;count:3 2024-06-10T00:32:06.175 INFO:teuthology.orchestra.run.smithi062.stdout:osd 12 88s ago - 2024-06-10T00:32:06.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:06.177+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/3781133330 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe1dc0a2c20 msgr2=0x7fe1dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:06.177+0000 7fe1f9ba7700 1 --2- 172.21.15.62:0/3781133330 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe1dc0a2c20 0x7fe1dc0a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8000c00 tx=0x7fe1e8005fd0 comp rx=0 tx=0).stop 2024-06-10T00:32:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:06.177+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/3781133330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1f41013f0 msgr2=0x7fe1f4105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:06.177+0000 7fe1f9ba7700 1 --2- 172.21.15.62:0/3781133330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1f41013f0 0x7fe1f4105380 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400d980 tx=0x7fe1e400dc90 comp rx=0 tx=0).stop 2024-06-10T00:32:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:06.177+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/3781133330 shutdown_connections 2024-06-10T00:32:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:06.177+0000 7fe1f9ba7700 1 --2- 172.21.15.62:0/3781133330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1f41013f0 0x7fe1f4105380 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:06.177+0000 7fe1f9ba7700 1 --2- 172.21.15.62:0/3781133330 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fe1dc0a2c20 0x7fe1dc0a50e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:06.177+0000 7fe1f9ba7700 1 --2- 172.21.15.62:0/3781133330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f4102f90 0x7fe1f4103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:06.177+0000 7fe1f9ba7700 1 --2- 172.21.15.62:0/3781133330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f41025f0 0x7fe1f41039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:06.177+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/3781133330 >> 172.21.15.62:0/3781133330 conn(0x7fe1f40fc9b0 msgr2=0x7fe1f40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:06.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:06.177+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/3781133330 shutdown_connections 2024-06-10T00:32:06.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:06.177+0000 7fe1f9ba7700 1 -- 172.21.15.62:0/3781133330 wait complete. 2024-06-10T00:32:06.882 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-10T00:32:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:07.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:07 smithi062 ceph-mon[32390]: pgmap v411: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:32:07.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:07 smithi062 ceph-mon[32390]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:08.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:07 smithi105 ceph-mon[36321]: pgmap v411: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:32:08.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:07 smithi105 ceph-mon[36321]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:08.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:07 smithi138 ceph-mon[36549]: pgmap v411: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:32:08.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:07 smithi138 ceph-mon[36549]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:08.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.898+0000 7f4af67bf700 1 -- 172.21.15.62:0/2754487096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af01005c0 msgr2=0x7f4af0100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:08.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.898+0000 7f4af67bf700 1 --2- 172.21.15.62:0/2754487096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af01005c0 0x7f4af0100a40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4ad80099c0 tx=0x7f4ad8009cd0 comp rx=0 tx=0).stop 2024-06-10T00:32:08.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.898+0000 7f4af67bf700 1 -- 172.21.15.62:0/2754487096 shutdown_connections 2024-06-10T00:32:08.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.898+0000 7f4af67bf700 1 --2- 172.21.15.62:0/2754487096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af0100f80 0x7f4af010ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.898+0000 7f4af67bf700 1 --2- 172.21.15.62:0/2754487096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4af0103b40 0x7f4af0103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.898+0000 7f4af67bf700 1 --2- 172.21.15.62:0/2754487096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af01005c0 0x7f4af0100a40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.898+0000 7f4af67bf700 1 -- 172.21.15.62:0/2754487096 >> 172.21.15.62:0/2754487096 conn(0x7f4af00fc160 msgr2=0x7f4af00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.898+0000 7f4af67bf700 1 -- 172.21.15.62:0/2754487096 shutdown_connections 2024-06-10T00:32:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.898+0000 7f4af67bf700 1 -- 172.21.15.62:0/2754487096 wait complete. 2024-06-10T00:32:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4af67bf700 1 Processor -- start 2024-06-10T00:32:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4af67bf700 1 -- start start 2024-06-10T00:32:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4af67bf700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af01005c0 0x7f4af0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4af67bf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af0100f80 0x7f4af0198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4af67bf700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4af0103b40 0x7f4af019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4af67bf700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4af010e1f0 con 0x7f4af01005c0 2024-06-10T00:32:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4af67bf700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4af010e070 con 0x7f4af0103b40 2024-06-10T00:32:08.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4af67bf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4af010e370 con 0x7f4af0100f80 2024-06-10T00:32:08.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af01005c0 0x7f4af0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:08.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af01005c0 0x7f4af0198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44650/0 (socket says 172.21.15.62:44650) 2024-06-10T00:32:08.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4aeffff700 1 -- 172.21.15.62:0/689474521 learned_addr learned my addr 172.21.15.62:0/689474521 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:08.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4ae7fff700 1 --2- 172.21.15.62:0/689474521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af0100f80 0x7f4af0198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:08.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4af4d5c700 1 --2- 172.21.15.62:0/689474521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4af0103b40 0x7f4af019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:08.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4aeffff700 1 -- 172.21.15.62:0/689474521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af0100f80 msgr2=0x7f4af0198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:08.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4aeffff700 1 --2- 172.21.15.62:0/689474521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af0100f80 0x7f4af0198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:08.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4aeffff700 1 -- 172.21.15.62:0/689474521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4af0103b40 msgr2=0x7f4af019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:08.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.899+0000 7f4aeffff700 1 --2- 172.21.15.62:0/689474521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4af0103b40 0x7f4af019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:08.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.900+0000 7f4aeffff700 1 -- 172.21.15.62:0/689474521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af01a03d0 con 0x7f4af01005c0 2024-06-10T00:32:08.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.900+0000 7f4aeffff700 1 --2- 172.21.15.62:0/689474521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af01005c0 0x7f4af0198770 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000b530 tx=0x7f4ae000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:08.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.900+0000 7f4aedffb700 1 -- 172.21.15.62:0/689474521 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0015020 con 0x7f4af01005c0 2024-06-10T00:32:08.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.900+0000 7f4aedffb700 1 -- 172.21.15.62:0/689474521 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae0011d10 con 0x7f4af01005c0 2024-06-10T00:32:08.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.900+0000 7f4af67bf700 1 -- 172.21.15.62:0/689474521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af01a06c0 con 0x7f4af01005c0 2024-06-10T00:32:08.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.900+0000 7f4aedffb700 1 -- 172.21.15.62:0/689474521 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000f2e0 con 0x7f4af01005c0 2024-06-10T00:32:08.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.900+0000 7f4af67bf700 1 -- 172.21.15.62:0/689474521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af01a0c80 con 0x7f4af01005c0 2024-06-10T00:32:08.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.901+0000 7f4af67bf700 1 -- 172.21.15.62:0/689474521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af0069180 con 0x7f4af01005c0 2024-06-10T00:32:08.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.902+0000 7f4aedffb700 1 -- 172.21.15.62:0/689474521 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4ae0018020 con 0x7f4af01005c0 2024-06-10T00:32:08.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.903+0000 7f4aedffb700 1 --2- 172.21.15.62:0/689474521 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f4adc0a2c20 0x7f4adc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:08.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.903+0000 7f4ae7fff700 1 --2- 172.21.15.62:0/689474521 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f4adc0a2c20 0x7f4adc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:08.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.903+0000 7f4aedffb700 1 -- 172.21.15.62:0/689474521 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f4ae00ceb50 con 0x7f4af01005c0 2024-06-10T00:32:08.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.904+0000 7f4ae7fff700 1 --2- 172.21.15.62:0/689474521 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f4adc0a2c20 0x7f4adc0a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800b580 tx=0x7f4ad8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:08.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:08.908+0000 7f4aedffb700 1 -- 172.21.15.62:0/689474521 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4ae001f020 con 0x7f4af01005c0 2024-06-10T00:32:09.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:09.126+0000 7f4af67bf700 1 -- 172.21.15.62:0/689474521 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f4af00628d0 con 0x7f4adc0a2c20 2024-06-10T00:32:09.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:09.127+0000 7f4aedffb700 1 -- 172.21.15.62:0/689474521 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f4af00628d0 con 0x7f4adc0a2c20 2024-06-10T00:32:09.128 INFO:teuthology.orchestra.run.smithi062.stdout:HOST ADDR LABELS STATUS 2024-06-10T00:32:09.128 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062 172.21.15.62 2024-06-10T00:32:09.128 INFO:teuthology.orchestra.run.smithi062.stdout:smithi105 172.21.15.105 2024-06-10T00:32:09.128 INFO:teuthology.orchestra.run.smithi062.stdout:smithi138 172.21.15.138 2024-06-10T00:32:09.128 INFO:teuthology.orchestra.run.smithi062.stdout:3 hosts in cluster 2024-06-10T00:32:09.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:09.130+0000 7f4af67bf700 1 -- 172.21.15.62:0/689474521 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f4adc0a2c20 msgr2=0x7f4adc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:09.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:09.130+0000 7f4af67bf700 1 --2- 172.21.15.62:0/689474521 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f4adc0a2c20 0x7f4adc0a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800b580 tx=0x7f4ad8005fb0 comp rx=0 tx=0).stop 2024-06-10T00:32:09.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:09.130+0000 7f4af67bf700 1 -- 172.21.15.62:0/689474521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af01005c0 msgr2=0x7f4af0198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:09.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:09.130+0000 7f4af67bf700 1 --2- 172.21.15.62:0/689474521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af01005c0 0x7f4af0198770 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000b530 tx=0x7f4ae000b840 comp rx=0 tx=0).stop 2024-06-10T00:32:09.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:09.130+0000 7f4af67bf700 1 -- 172.21.15.62:0/689474521 shutdown_connections 2024-06-10T00:32:09.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:09.130+0000 7f4af67bf700 1 --2- 172.21.15.62:0/689474521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af01005c0 0x7f4af0198770 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:09.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:09.130+0000 7f4af67bf700 1 --2- 172.21.15.62:0/689474521 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f4adc0a2c20 0x7f4adc0a50e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:09.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:09.130+0000 7f4af67bf700 1 --2- 172.21.15.62:0/689474521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4af0103b40 0x7f4af019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:09.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:09.130+0000 7f4af67bf700 1 --2- 172.21.15.62:0/689474521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af0100f80 0x7f4af0198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:09.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:09.130+0000 7f4af67bf700 1 -- 172.21.15.62:0/689474521 >> 172.21.15.62:0/689474521 conn(0x7f4af00fc160 msgr2=0x7f4af0106b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:09.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:09.130+0000 7f4af67bf700 1 -- 172.21.15.62:0/689474521 shutdown_connections 2024-06-10T00:32:09.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:09.130+0000 7f4af67bf700 1 -- 172.21.15.62:0/689474521 wait complete. 2024-06-10T00:32:09.848 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:09 smithi062 ceph-mon[32390]: pgmap v412: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:09.850 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-10T00:32:10.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:09 smithi105 ceph-mon[36321]: pgmap v412: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:10.101 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:10.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:09 smithi138 ceph-mon[36549]: pgmap v412: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:10.789 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:10 smithi062 ceph-mon[32390]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:11.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:10 smithi138 ceph-mon[36549]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:11.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:10 smithi105 ceph-mon[36321]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.796+0000 7ff705c95700 1 -- 172.21.15.62:0/3849637128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7001013f0 msgr2=0x7ff700101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.796+0000 7ff705c95700 1 --2- 172.21.15.62:0/3849637128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7001013f0 0x7ff700101810 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00099c0 tx=0x7ff6f0009cd0 comp rx=0 tx=0).stop 2024-06-10T00:32:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.796+0000 7ff705c95700 1 -- 172.21.15.62:0/3849637128 shutdown_connections 2024-06-10T00:32:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.796+0000 7ff705c95700 1 --2- 172.21.15.62:0/3849637128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7001013f0 0x7ff700101810 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.796+0000 7ff705c95700 1 --2- 172.21.15.62:0/3849637128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff700102f90 0x7ff700109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.796+0000 7ff705c95700 1 --2- 172.21.15.62:0/3849637128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7001025f0 0x7ff700102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.796+0000 7ff705c95700 1 -- 172.21.15.62:0/3849637128 >> 172.21.15.62:0/3849637128 conn(0x7ff7000fc990 msgr2=0x7ff7000fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.796+0000 7ff705c95700 1 -- 172.21.15.62:0/3849637128 shutdown_connections 2024-06-10T00:32:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.796+0000 7ff705c95700 1 -- 172.21.15.62:0/3849637128 wait complete. 2024-06-10T00:32:11.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.796+0000 7ff705c95700 1 Processor -- start 2024-06-10T00:32:11.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff705c95700 1 -- start start 2024-06-10T00:32:11.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff705c95700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7001013f0 0x7ff700198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:11.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff705c95700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7001025f0 0x7ff700198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff705c95700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff700102f90 0x7ff70019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff705c95700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff70010e110 con 0x7ff7001025f0 2024-06-10T00:32:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff705c95700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff70010df90 con 0x7ff700102f90 2024-06-10T00:32:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff705c95700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff70010e290 con 0x7ff7001013f0 2024-06-10T00:32:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff6feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7001025f0 0x7ff700198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff6feffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7001025f0 0x7ff700198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44674/0 (socket says 172.21.15.62:44674) 2024-06-10T00:32:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff6feffd700 1 -- 172.21.15.62:0/2988351907 learned_addr learned my addr 172.21.15.62:0/2988351907 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff6fffff700 1 --2- 172.21.15.62:0/2988351907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff700102f90 0x7ff70019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff6ff7fe700 1 --2- 172.21.15.62:0/2988351907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7001013f0 0x7ff700198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff6feffd700 1 -- 172.21.15.62:0/2988351907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7001013f0 msgr2=0x7ff700198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff6feffd700 1 --2- 172.21.15.62:0/2988351907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7001013f0 0x7ff700198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff6feffd700 1 -- 172.21.15.62:0/2988351907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff700102f90 msgr2=0x7ff70019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff6feffd700 1 --2- 172.21.15.62:0/2988351907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff700102f90 0x7ff70019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.797+0000 7ff6feffd700 1 -- 172.21.15.62:0/2988351907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7001a03b0 con 0x7ff7001025f0 2024-06-10T00:32:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.798+0000 7ff6feffd700 1 --2- 172.21.15.62:0/2988351907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7001025f0 0x7ff700198c90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400e9d0 tx=0x7ff6f400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.798+0000 7ff6fcff9700 1 -- 172.21.15.62:0/2988351907 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f400cbd0 con 0x7ff7001025f0 2024-06-10T00:32:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.798+0000 7ff6fcff9700 1 -- 172.21.15.62:0/2988351907 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6f400cd70 con 0x7ff7001025f0 2024-06-10T00:32:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.798+0000 7ff6fcff9700 1 -- 172.21.15.62:0/2988351907 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4018660 con 0x7ff7001025f0 2024-06-10T00:32:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.798+0000 7ff705c95700 1 -- 172.21.15.62:0/2988351907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7001a06a0 con 0x7ff7001025f0 2024-06-10T00:32:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.798+0000 7ff705c95700 1 -- 172.21.15.62:0/2988351907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7001a0be0 con 0x7ff7001025f0 2024-06-10T00:32:11.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.799+0000 7ff705c95700 1 -- 172.21.15.62:0/2988351907 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff700069180 con 0x7ff7001025f0 2024-06-10T00:32:11.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.800+0000 7ff6fcff9700 1 -- 172.21.15.62:0/2988351907 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff6f4018840 con 0x7ff7001025f0 2024-06-10T00:32:11.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.801+0000 7ff6fcff9700 1 --2- 172.21.15.62:0/2988351907 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff6e80a2c20 0x7ff6e80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:11.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.801+0000 7ff6ff7fe700 1 --2- 172.21.15.62:0/2988351907 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff6e80a2c20 0x7ff6e80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:11.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.801+0000 7ff6fcff9700 1 -- 172.21.15.62:0/2988351907 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7ff6f40cf6c0 con 0x7ff7001025f0 2024-06-10T00:32:11.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.801+0000 7ff6ff7fe700 1 --2- 172.21.15.62:0/2988351907 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff6e80a2c20 0x7ff6e80a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0000c00 tx=0x7ff6f0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:11.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:11.805+0000 7ff6fcff9700 1 -- 172.21.15.62:0/2988351907 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff6f409c250 con 0x7ff7001025f0 2024-06-10T00:32:12.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:12.023+0000 7ff705c95700 1 -- 172.21.15.62:0/2988351907 --> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7ff7000628d0 con 0x7ff6e80a2c20 2024-06-10T00:32:12.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:12.024+0000 7ff6fcff9700 1 -- 172.21.15.62:0/2988351907 <== mgr.14152 v2:172.21.15.62:6800/3602410963 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7ff7000628d0 con 0x7ff6e80a2c20 2024-06-10T00:32:12.024 INFO:teuthology.orchestra.run.smithi062.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-10T00:32:12.025 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440094400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-06-10T00:32:12.025 INFO:teuthology.orchestra.run.smithi062.stdout:smithi105 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400TR400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-06-10T00:32:12.025 INFO:teuthology.orchestra.run.smithi062.stdout:smithi138 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6161003C400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-06-10T00:32:12.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:12.027+0000 7ff705c95700 1 -- 172.21.15.62:0/2988351907 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff6e80a2c20 msgr2=0x7ff6e80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:12.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:12.027+0000 7ff705c95700 1 --2- 172.21.15.62:0/2988351907 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff6e80a2c20 0x7ff6e80a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0000c00 tx=0x7ff6f0005fd0 comp rx=0 tx=0).stop 2024-06-10T00:32:12.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:12.027+0000 7ff705c95700 1 -- 172.21.15.62:0/2988351907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7001025f0 msgr2=0x7ff700198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:12.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:12.027+0000 7ff705c95700 1 --2- 172.21.15.62:0/2988351907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7001025f0 0x7ff700198c90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400e9d0 tx=0x7ff6f400ece0 comp rx=0 tx=0).stop 2024-06-10T00:32:12.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:12.027+0000 7ff705c95700 1 -- 172.21.15.62:0/2988351907 shutdown_connections 2024-06-10T00:32:12.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:12.027+0000 7ff705c95700 1 --2- 172.21.15.62:0/2988351907 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff6e80a2c20 0x7ff6e80a50e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:12.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:12.027+0000 7ff705c95700 1 --2- 172.21.15.62:0/2988351907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff700102f90 0x7ff70019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:12.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:12.027+0000 7ff705c95700 1 --2- 172.21.15.62:0/2988351907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7001013f0 0x7ff700198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:12.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:12.027+0000 7ff705c95700 1 --2- 172.21.15.62:0/2988351907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7001025f0 0x7ff700198c90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:12.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:12.027+0000 7ff705c95700 1 -- 172.21.15.62:0/2988351907 >> 172.21.15.62:0/2988351907 conn(0x7ff7000fc990 msgr2=0x7ff700105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:12.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:12.027+0000 7ff705c95700 1 -- 172.21.15.62:0/2988351907 shutdown_connections 2024-06-10T00:32:12.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:12.027+0000 7ff705c95700 1 -- 172.21.15.62:0/2988351907 wait complete. 2024-06-10T00:32:12.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:11 smithi062 ceph-mon[32390]: pgmap v413: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:12.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:11 smithi138 ceph-mon[36549]: pgmap v413: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:12.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:11 smithi105 ceph-mon[36321]: pgmap v413: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:12.651 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-10T00:32:12.659 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-06-10T00:32:12.659 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-10T00:32:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:14.024 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:13 smithi062 ceph-mon[32390]: pgmap v414: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:32:14.024 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:13 smithi062 ceph-mon[32390]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:14.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:13 smithi138 ceph-mon[36549]: pgmap v414: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:32:14.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:13 smithi138 ceph-mon[36549]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:14.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:13 smithi105 ceph-mon[36321]: pgmap v414: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:32:14.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:13 smithi105 ceph-mon[36321]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:32:14.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.545+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/576743514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b18100df0 msgr2=0x7f8b18101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:14.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.545+0000 7f8b1f5c5700 1 --2- 172.21.15.62:0/576743514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b18100df0 0x7f8b18101210 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009990 tx=0x7f8b14009ca0 comp rx=0 tx=0).stop 2024-06-10T00:32:14.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.545+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/576743514 shutdown_connections 2024-06-10T00:32:14.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.545+0000 7f8b1f5c5700 1 --2- 172.21.15.62:0/576743514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b18100df0 0x7f8b18101210 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:14.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.545+0000 7f8b1f5c5700 1 --2- 172.21.15.62:0/576743514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b18102110 0x7f8b18102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:14.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.545+0000 7f8b1f5c5700 1 --2- 172.21.15.62:0/576743514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b18101750 0x7f8b18101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:14.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.545+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/576743514 >> 172.21.15.62:0/576743514 conn(0x7f8b180fc990 msgr2=0x7f8b180fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:14.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.545+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/576743514 shutdown_connections 2024-06-10T00:32:14.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.545+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/576743514 wait complete. 2024-06-10T00:32:14.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.545+0000 7f8b1f5c5700 1 Processor -- start 2024-06-10T00:32:14.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1f5c5700 1 -- start start 2024-06-10T00:32:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1f5c5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b18100df0 0x7f8b1819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1f5c5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b18101750 0x7f8b1819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1f5c5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b18102110 0x7f8b181a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1f5c5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8b181102b0 con 0x7f8b18102110 2024-06-10T00:32:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1f5c5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8b18110130 con 0x7f8b18101750 2024-06-10T00:32:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1f5c5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b18110430 con 0x7f8b18100df0 2024-06-10T00:32:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1d361700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b18100df0 0x7f8b1819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1d361700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b18100df0 0x7f8b1819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:35072/0 (socket says 172.21.15.62:35072) 2024-06-10T00:32:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1d361700 1 -- 172.21.15.62:0/924767456 learned_addr learned my addr 172.21.15.62:0/924767456 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1db62700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b18102110 0x7f8b181a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1cb60700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b18101750 0x7f8b1819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1d361700 1 -- 172.21.15.62:0/924767456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b18101750 msgr2=0x7f8b1819aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1d361700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b18101750 0x7f8b1819aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1d361700 1 -- 172.21.15.62:0/924767456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b18102110 msgr2=0x7f8b181a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1d361700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b18102110 0x7f8b181a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1d361700 1 -- 172.21.15.62:0/924767456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b181a25c0 con 0x7f8b18100df0 2024-06-10T00:32:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.546+0000 7f8b1db62700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b18102110 0x7f8b181a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:32:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.547+0000 7f8b1d361700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b18100df0 0x7f8b1819a960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8b14000c00 tx=0x7f8b14005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.547+0000 7f8b0e7fc700 1 -- 172.21.15.62:0/924767456 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14027070 con 0x7f8b18100df0 2024-06-10T00:32:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.547+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/924767456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b181a2850 con 0x7f8b18100df0 2024-06-10T00:32:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.547+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/924767456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b181a2d00 con 0x7f8b18100df0 2024-06-10T00:32:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.547+0000 7f8b0e7fc700 1 -- 172.21.15.62:0/924767456 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b14003f40 con 0x7f8b18100df0 2024-06-10T00:32:14.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.547+0000 7f8b0e7fc700 1 -- 172.21.15.62:0/924767456 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14022de0 con 0x7f8b18100df0 2024-06-10T00:32:14.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.548+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/924767456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8afc005360 con 0x7f8b18100df0 2024-06-10T00:32:14.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.550+0000 7f8b0e7fc700 1 -- 172.21.15.62:0/924767456 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8b1402a070 con 0x7f8b18100df0 2024-06-10T00:32:14.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.551+0000 7f8b0e7fc700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8b040a2c20 0x7f8b040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:14.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.551+0000 7f8b1cb60700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8b040a2c20 0x7f8b040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:14.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.551+0000 7f8b0e7fc700 1 -- 172.21.15.62:0/924767456 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f8b140d0380 con 0x7f8b18100df0 2024-06-10T00:32:14.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.551+0000 7f8b1cb60700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8b040a2c20 0x7f8b040a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8b08004630 tx=0x7f8b0800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:14.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.554+0000 7f8b0e7fc700 1 -- 172.21.15.62:0/924767456 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8b14013070 con 0x7f8b18100df0 2024-06-10T00:32:14.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.807+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/924767456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8afc005920 con 0x7f8b18100df0 2024-06-10T00:32:14.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.808+0000 7f8b0e7fc700 1 -- 172.21.15.62:0/924767456 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3096 (secure 0 0 0) 0x7f8b1409cfa0 con 0x7f8b18100df0 2024-06-10T00:32:14.808 INFO:teuthology.orchestra.run.smithi062.stdout:e29 2024-06-10T00:32:14.808 INFO:teuthology.orchestra.run.smithi062.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-10T00:32:14.808 INFO:teuthology.orchestra.run.smithi062.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T00:32:14.808 INFO:teuthology.orchestra.run.smithi062.stdout:legacy client fscid: 1 2024-06-10T00:32:14.808 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:32:14.808 INFO:teuthology.orchestra.run.smithi062.stdout:Filesystem 'cephfs' (1) 2024-06-10T00:32:14.808 INFO:teuthology.orchestra.run.smithi062.stdout:fs_name cephfs 2024-06-10T00:32:14.808 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 29 2024-06-10T00:32:14.809 INFO:teuthology.orchestra.run.smithi062.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-10T00:32:14.809 INFO:teuthology.orchestra.run.smithi062.stdout:created 2024-06-10T00:28:39.478894+0000 2024-06-10T00:32:14.809 INFO:teuthology.orchestra.run.smithi062.stdout:modified 2024-06-10T00:29:47.712789+0000 2024-06-10T00:32:14.809 INFO:teuthology.orchestra.run.smithi062.stdout:tableserver 0 2024-06-10T00:32:14.809 INFO:teuthology.orchestra.run.smithi062.stdout:root 0 2024-06-10T00:32:14.809 INFO:teuthology.orchestra.run.smithi062.stdout:session_timeout 300 2024-06-10T00:32:14.809 INFO:teuthology.orchestra.run.smithi062.stdout:session_autoclose 300 2024-06-10T00:32:14.809 INFO:teuthology.orchestra.run.smithi062.stdout:max_file_size 1099511627776 2024-06-10T00:32:14.809 INFO:teuthology.orchestra.run.smithi062.stdout:max_xattr_size 65536 2024-06-10T00:32:14.809 INFO:teuthology.orchestra.run.smithi062.stdout:required_client_features {} 2024-06-10T00:32:14.809 INFO:teuthology.orchestra.run.smithi062.stdout:last_failure 0 2024-06-10T00:32:14.809 INFO:teuthology.orchestra.run.smithi062.stdout:last_failure_osd_epoch 0 2024-06-10T00:32:14.810 INFO:teuthology.orchestra.run.smithi062.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T00:32:14.810 INFO:teuthology.orchestra.run.smithi062.stdout:max_mds 5 2024-06-10T00:32:14.810 INFO:teuthology.orchestra.run.smithi062.stdout:in 0,1,2,3,4 2024-06-10T00:32:14.810 INFO:teuthology.orchestra.run.smithi062.stdout:up {0=24464,1=24433,2=24431,3=24424,4=24418} 2024-06-10T00:32:14.810 INFO:teuthology.orchestra.run.smithi062.stdout:failed 2024-06-10T00:32:14.810 INFO:teuthology.orchestra.run.smithi062.stdout:damaged 2024-06-10T00:32:14.810 INFO:teuthology.orchestra.run.smithi062.stdout:stopped 2024-06-10T00:32:14.810 INFO:teuthology.orchestra.run.smithi062.stdout:data_pools [3,4] 2024-06-10T00:32:14.810 INFO:teuthology.orchestra.run.smithi062.stdout:metadata_pool 2 2024-06-10T00:32:14.810 INFO:teuthology.orchestra.run.smithi062.stdout:inline_data disabled 2024-06-10T00:32:14.810 INFO:teuthology.orchestra.run.smithi062.stdout:balancer 2024-06-10T00:32:14.810 INFO:teuthology.orchestra.run.smithi062.stdout:bal_rank_mask -1 2024-06-10T00:32:14.810 INFO:teuthology.orchestra.run.smithi062.stdout:standby_count_wanted 0 2024-06-10T00:32:14.811 INFO:teuthology.orchestra.run.smithi062.stdout:qdb_cluster leader: 24464 members: 24431,24424,24418,24433,24464 2024-06-10T00:32:14.811 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b{0:24464} state up:active seq 6 addr [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:32:14.811 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.e{0:24404} state up:standby-replay seq 1 addr [v2:172.21.15.105:6834/588077925,v1:172.21.15.105:6835/588077925] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:32:14.811 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.i{1:24433} state up:active seq 24 addr [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:32:14.811 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.l{1:24382} state up:standby-replay seq 1 addr [v2:172.21.15.138:6832/2673201423,v1:172.21.15.138:6833/2673201423] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:32:14.811 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.h{2:24431} state up:active seq 25 addr [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:32:14.811 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.k{2:24377} state up:standby-replay seq 1 addr [v2:172.21.15.105:6832/1194987741,v1:172.21.15.105:6833/1194987741] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:32:14.811 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c{3:24424} state up:active seq 26 addr [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:32:14.811 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a{3:14625} state up:standby-replay seq 1 addr [v2:172.21.15.62:6840/2460552589,v1:172.21.15.62:6841/2460552589] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:32:14.811 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.f{4:24418} state up:active seq 27 addr [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:32:14.811 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.g{4:14613} state up:standby-replay seq 1 addr [v2:172.21.15.62:6838/4040406390,v1:172.21.15.62:6839/4040406390] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:32:14.811 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:32:14.811 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:32:14.811 INFO:teuthology.orchestra.run.smithi062.stdout:Standby daemons: 2024-06-10T00:32:14.812 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:32:14.812 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.62:6834/3839480602,v1:172.21.15.62:6835/3839480602] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:32:14.812 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.62:6836/147630858,v1:172.21.15.62:6837/147630858] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:32:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.810+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/924767456 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8b040a2c20 msgr2=0x7f8b040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.810+0000 7f8b1f5c5700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8b040a2c20 0x7f8b040a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8b08004630 tx=0x7f8b0800b3c0 comp rx=0 tx=0).stop 2024-06-10T00:32:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.810+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/924767456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b18100df0 msgr2=0x7f8b1819a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.810+0000 7f8b1f5c5700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b18100df0 0x7f8b1819a960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8b14000c00 tx=0x7f8b14005e80 comp rx=0 tx=0).stop 2024-06-10T00:32:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.810+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/924767456 shutdown_connections 2024-06-10T00:32:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.810+0000 7f8b1f5c5700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b18102110 0x7f8b181a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.810+0000 7f8b1f5c5700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8b040a2c20 0x7f8b040a50e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.810+0000 7f8b1f5c5700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b18100df0 0x7f8b1819a960 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.810+0000 7f8b1f5c5700 1 --2- 172.21.15.62:0/924767456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b18101750 0x7f8b1819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.810+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/924767456 >> 172.21.15.62:0/924767456 conn(0x7f8b180fc990 msgr2=0x7f8b180fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.810+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/924767456 shutdown_connections 2024-06-10T00:32:14.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:14.810+0000 7f8b1f5c5700 1 -- 172.21.15.62:0/924767456 wait complete. 2024-06-10T00:32:14.814 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 29 2024-06-10T00:32:15.637 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-10T00:32:15.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:15 smithi062 ceph-mon[32390]: pgmap v415: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:32:15.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:15 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/924767456' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-10T00:32:15.890 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:16.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:15 smithi138 ceph-mon[36549]: pgmap v415: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:32:16.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:15 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/924767456' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-10T00:32:16.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:15 smithi105 ceph-mon[36321]: pgmap v415: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:32:16.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:15 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/924767456' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-10T00:32:17.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.552+0000 7f63bf0ec700 1 -- 172.21.15.62:0/2704603413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b81013f0 msgr2=0x7f63b8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:17.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.552+0000 7f63bf0ec700 1 --2- 172.21.15.62:0/2704603413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b81013f0 0x7f63b8101810 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0099c0 tx=0x7f63ac009cd0 comp rx=0 tx=0).stop 2024-06-10T00:32:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.552+0000 7f63bf0ec700 1 -- 172.21.15.62:0/2704603413 shutdown_connections 2024-06-10T00:32:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.552+0000 7f63bf0ec700 1 --2- 172.21.15.62:0/2704603413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b81013f0 0x7f63b8101810 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.552+0000 7f63bf0ec700 1 --2- 172.21.15.62:0/2704603413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63b8102f90 0x7f63b8109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.552+0000 7f63bf0ec700 1 --2- 172.21.15.62:0/2704603413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63b81025f0 0x7f63b8102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.552+0000 7f63bf0ec700 1 -- 172.21.15.62:0/2704603413 >> 172.21.15.62:0/2704603413 conn(0x7f63b80fc990 msgr2=0x7f63b80fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.552+0000 7f63bf0ec700 1 -- 172.21.15.62:0/2704603413 shutdown_connections 2024-06-10T00:32:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.552+0000 7f63bf0ec700 1 -- 172.21.15.62:0/2704603413 wait complete. 2024-06-10T00:32:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.553+0000 7f63bf0ec700 1 Processor -- start 2024-06-10T00:32:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.553+0000 7f63bf0ec700 1 -- start start 2024-06-10T00:32:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.553+0000 7f63bf0ec700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63b81013f0 0x7f63b81054d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.553+0000 7f63bf0ec700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63b81025f0 0x7f63b8103b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.553+0000 7f63bf0ec700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8102f90 0x7f63b8104060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.553+0000 7f63bf0ec700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f63b810cfd0 con 0x7f63b8102f90 2024-06-10T00:32:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.553+0000 7f63bf0ec700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f63b810ce50 con 0x7f63b81025f0 2024-06-10T00:32:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.553+0000 7f63bf0ec700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f63b810d150 con 0x7f63b81013f0 2024-06-10T00:32:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.553+0000 7f63bd689700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8102f90 0x7f63b8104060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.553+0000 7f63bd689700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8102f90 0x7f63b8104060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44716/0 (socket says 172.21.15.62:44716) 2024-06-10T00:32:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.553+0000 7f63bd689700 1 -- 172.21.15.62:0/4247707568 learned_addr learned my addr 172.21.15.62:0/4247707568 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.553+0000 7f63b7fff700 1 --2- 172.21.15.62:0/4247707568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63b81025f0 0x7f63b8103b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.554+0000 7f63bce88700 1 --2- 172.21.15.62:0/4247707568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63b81013f0 0x7f63b81054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.554+0000 7f63bd689700 1 -- 172.21.15.62:0/4247707568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63b81013f0 msgr2=0x7f63b81054d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.554+0000 7f63bd689700 1 --2- 172.21.15.62:0/4247707568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63b81013f0 0x7f63b81054d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.554+0000 7f63bd689700 1 -- 172.21.15.62:0/4247707568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63b81025f0 msgr2=0x7f63b8103b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.554+0000 7f63bd689700 1 --2- 172.21.15.62:0/4247707568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63b81025f0 0x7f63b8103b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:17.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.554+0000 7f63bd689700 1 -- 172.21.15.62:0/4247707568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b8104910 con 0x7f63b8102f90 2024-06-10T00:32:17.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.554+0000 7f63bd689700 1 --2- 172.21.15.62:0/4247707568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8102f90 0x7f63b8104060 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f63b0007ee0 tx=0x7f63b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:17.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.554+0000 7f63b5ffb700 1 -- 172.21.15.62:0/4247707568 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b000dbd0 con 0x7f63b8102f90 2024-06-10T00:32:17.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.554+0000 7f63bf0ec700 1 -- 172.21.15.62:0/4247707568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b81a2900 con 0x7f63b8102f90 2024-06-10T00:32:17.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.554+0000 7f63b5ffb700 1 -- 172.21.15.62:0/4247707568 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b000dd70 con 0x7f63b8102f90 2024-06-10T00:32:17.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.554+0000 7f63b5ffb700 1 -- 172.21.15.62:0/4247707568 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0005a60 con 0x7f63b8102f90 2024-06-10T00:32:17.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.554+0000 7f63bf0ec700 1 -- 172.21.15.62:0/4247707568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b81a2dc0 con 0x7f63b8102f90 2024-06-10T00:32:17.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.555+0000 7f63bf0ec700 1 -- 172.21.15.62:0/4247707568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b8069180 con 0x7f63b8102f90 2024-06-10T00:32:17.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.556+0000 7f63b5ffb700 1 -- 172.21.15.62:0/4247707568 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f63b001a020 con 0x7f63b8102f90 2024-06-10T00:32:17.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.557+0000 7f63b5ffb700 1 --2- 172.21.15.62:0/4247707568 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f63a80a2c20 0x7f63a80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:17.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.557+0000 7f63bce88700 1 --2- 172.21.15.62:0/4247707568 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f63a80a2c20 0x7f63a80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:17.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.557+0000 7f63b5ffb700 1 -- 172.21.15.62:0/4247707568 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f63b00ceb40 con 0x7f63b8102f90 2024-06-10T00:32:17.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.557+0000 7f63bce88700 1 --2- 172.21.15.62:0/4247707568 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f63a80a2c20 0x7f63a80a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f63ac000c00 tx=0x7f63ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:17.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.562+0000 7f63b5ffb700 1 -- 172.21.15.62:0/4247707568 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f63b0010070 con 0x7f63b8102f90 2024-06-10T00:32:17.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.768+0000 7f63bf0ec700 1 -- 172.21.15.62:0/4247707568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f63b81a3020 con 0x7f63b8102f90 2024-06-10T00:32:17.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.769+0000 7f63b5ffb700 1 -- 172.21.15.62:0/4247707568 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v99) ==== 56+0+5702 (secure 0 0 0) 0x7f63b009b6d0 con 0x7f63b8102f90 2024-06-10T00:32:17.769 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 99 2024-06-10T00:32:17.769 INFO:teuthology.orchestra.run.smithi062.stdout:fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:32:17.769 INFO:teuthology.orchestra.run.smithi062.stdout:created 2024-06-10T00:20:27.981753+0000 2024-06-10T00:32:17.769 INFO:teuthology.orchestra.run.smithi062.stdout:modified 2024-06-10T00:30:24.104116+0000 2024-06-10T00:32:17.769 INFO:teuthology.orchestra.run.smithi062.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-10T00:32:17.769 INFO:teuthology.orchestra.run.smithi062.stdout:crush_version 27 2024-06-10T00:32:17.769 INFO:teuthology.orchestra.run.smithi062.stdout:full_ratio 0.95 2024-06-10T00:32:17.769 INFO:teuthology.orchestra.run.smithi062.stdout:backfillfull_ratio 0.9 2024-06-10T00:32:17.770 INFO:teuthology.orchestra.run.smithi062.stdout:nearfull_ratio 0.85 2024-06-10T00:32:17.770 INFO:teuthology.orchestra.run.smithi062.stdout:require_min_compat_client luminous 2024-06-10T00:32:17.770 INFO:teuthology.orchestra.run.smithi062.stdout:min_compat_client luminous 2024-06-10T00:32:17.770 INFO:teuthology.orchestra.run.smithi062.stdout:require_osd_release squid 2024-06-10T00:32:17.770 INFO:teuthology.orchestra.run.smithi062.stdout:stretch_mode_enabled false 2024-06-10T00:32:17.770 INFO:teuthology.orchestra.run.smithi062.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-10T00:32:17.770 INFO:teuthology.orchestra.run.smithi062.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-06-10T00:32:17.770 INFO:teuthology.orchestra.run.smithi062.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-06-10T00:32:17.770 INFO:teuthology.orchestra.run.smithi062.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 95 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-06-10T00:32:17.770 INFO:teuthology.orchestra.run.smithi062.stdout:max_osd 12 2024-06-10T00:32:17.770 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 up in weight 1 up_from 8 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6802/2865393246,v1:172.21.15.62:6803/2865393246] [v2:172.21.15.62:6804/2865393246,v1:172.21.15.62:6805/2865393246] exists,up a148b909-d252-4277-b842-365cb9f169d1 2024-06-10T00:32:17.770 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6800/1473053019,v1:172.21.15.105:6801/1473053019] [v2:172.21.15.105:6802/1473053019,v1:172.21.15.105:6803/1473053019] exists,up d83d356c-ece6-401e-8f1e-156d5076c817 2024-06-10T00:32:17.771 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6800/4170554663,v1:172.21.15.138:6801/4170554663] [v2:172.21.15.138:6802/4170554663,v1:172.21.15.138:6803/4170554663] exists,up 863db396-689a-4e13-8b18-534f0e38a86f 2024-06-10T00:32:17.771 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 up in weight 1 up_from 25 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6810/851860835,v1:172.21.15.62:6811/851860835] [v2:172.21.15.62:6812/851860835,v1:172.21.15.62:6813/851860835] exists,up 756165fb-7807-41e8-a200-d23686c300a8 2024-06-10T00:32:17.771 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 up in weight 1 up_from 33 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6808/1897328738,v1:172.21.15.105:6809/1897328738] [v2:172.21.15.105:6810/1897328738,v1:172.21.15.105:6811/1897328738] exists,up 73ed9b45-1a37-4078-83e4-e34607aa2835 2024-06-10T00:32:17.771 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 up in weight 1 up_from 38 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6808/3331484519,v1:172.21.15.138:6809/3331484519] [v2:172.21.15.138:6810/3331484519,v1:172.21.15.138:6811/3331484519] exists,up 8c0fe663-9fd5-465e-808f-210bfd7ac201 2024-06-10T00:32:17.771 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 up in weight 1 up_from 46 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6818/1649680325,v1:172.21.15.62:6819/1649680325] [v2:172.21.15.62:6820/1649680325,v1:172.21.15.62:6821/1649680325] exists,up 2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e 2024-06-10T00:32:17.771 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 up in weight 1 up_from 51 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6816/3912053087,v1:172.21.15.105:6817/3912053087] [v2:172.21.15.105:6818/3912053087,v1:172.21.15.105:6819/3912053087] exists,up 59bacdeb-ed9d-4fdb-a1e9-f49732af81df 2024-06-10T00:32:17.771 INFO:teuthology.orchestra.run.smithi062.stdout:osd.8 up in weight 1 up_from 59 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6816/3579467531,v1:172.21.15.138:6817/3579467531] [v2:172.21.15.138:6818/3579467531,v1:172.21.15.138:6819/3579467531] exists,up b5f449fd-35c6-4885-827a-ae7f5b104404 2024-06-10T00:32:17.771 INFO:teuthology.orchestra.run.smithi062.stdout:osd.9 up in weight 1 up_from 67 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6826/1109246101,v1:172.21.15.62:6827/1109246101] [v2:172.21.15.62:6828/1109246101,v1:172.21.15.62:6829/1109246101] exists,up f02b059a-ffcd-4dd2-b8a1-5782cc47a378 2024-06-10T00:32:17.771 INFO:teuthology.orchestra.run.smithi062.stdout:osd.10 up in weight 1 up_from 72 up_thru 98 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6824/576155679,v1:172.21.15.105:6825/576155679] [v2:172.21.15.105:6826/576155679,v1:172.21.15.105:6827/576155679] exists,up 55659c31-fcfc-4881-9533-8a1d6de76bc5 2024-06-10T00:32:17.771 INFO:teuthology.orchestra.run.smithi062.stdout:osd.11 up in weight 1 up_from 80 up_thru 96 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6824/364141685,v1:172.21.15.138:6825/364141685] [v2:172.21.15.138:6826/364141685,v1:172.21.15.138:6827/364141685] exists,up c01832bc-be75-400a-a707-5fe11d996b54 2024-06-10T00:32:17.771 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.5 [8,11] 2024-06-10T00:32:17.771 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.b [2,4] 2024-06-10T00:32:17.771 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.e [2,4] 2024-06-10T00:32:17.772 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.12 [5,11] 2024-06-10T00:32:17.772 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.1f [8,1] 2024-06-10T00:32:17.772 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.0 [5,9] 2024-06-10T00:32:17.772 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.5 [3,9] 2024-06-10T00:32:17.772 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.1b [4,6] 2024-06-10T00:32:17.772 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.23 [4,8] 2024-06-10T00:32:17.772 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.34 [4,9] 2024-06-10T00:32:17.772 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 4.b [9,8] 2024-06-10T00:32:17.772 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 4.e [9,11] 2024-06-10T00:32:17.772 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 4.1a [5,10] 2024-06-10T00:32:17.772 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 4.1f [9,8] 2024-06-10T00:32:17.772 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/3856763121 expires 2024-06-11T00:21:21.851532+0000 2024-06-10T00:32:17.772 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/381111141 expires 2024-06-11T00:21:21.851532+0000 2024-06-10T00:32:17.773 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/1704528283 expires 2024-06-11T00:21:21.851532+0000 2024-06-10T00:32:17.773 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/3912577307 expires 2024-06-11T00:20:55.088039+0000 2024-06-10T00:32:17.773 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/178507377 expires 2024-06-11T00:20:55.088039+0000 2024-06-10T00:32:17.773 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/4137526131 expires 2024-06-11T00:20:55.088039+0000 2024-06-10T00:32:17.773 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6801/2327526412 expires 2024-06-11T00:21:21.851532+0000 2024-06-10T00:32:17.773 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6800/594215923 expires 2024-06-11T00:20:55.088039+0000 2024-06-10T00:32:17.773 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6801/594215923 expires 2024-06-11T00:20:55.088039+0000 2024-06-10T00:32:17.773 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6800/2327526412 expires 2024-06-11T00:21:21.851532+0000 2024-06-10T00:32:17.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.772+0000 7f63bf0ec700 1 -- 172.21.15.62:0/4247707568 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f63a80a2c20 msgr2=0x7f63a80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:17.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.772+0000 7f63bf0ec700 1 --2- 172.21.15.62:0/4247707568 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f63a80a2c20 0x7f63a80a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f63ac000c00 tx=0x7f63ac005fd0 comp rx=0 tx=0).stop 2024-06-10T00:32:17.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.772+0000 7f63bf0ec700 1 -- 172.21.15.62:0/4247707568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8102f90 msgr2=0x7f63b8104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:17.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.772+0000 7f63bf0ec700 1 --2- 172.21.15.62:0/4247707568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8102f90 0x7f63b8104060 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f63b0007ee0 tx=0x7f63b000d3b0 comp rx=0 tx=0).stop 2024-06-10T00:32:17.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.772+0000 7f63bf0ec700 1 -- 172.21.15.62:0/4247707568 shutdown_connections 2024-06-10T00:32:17.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.772+0000 7f63bf0ec700 1 --2- 172.21.15.62:0/4247707568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63b8102f90 0x7f63b8104060 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:17.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.772+0000 7f63bf0ec700 1 --2- 172.21.15.62:0/4247707568 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f63a80a2c20 0x7f63a80a50e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:17.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.772+0000 7f63bf0ec700 1 --2- 172.21.15.62:0/4247707568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63b81013f0 0x7f63b81054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:17.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.772+0000 7f63bf0ec700 1 --2- 172.21.15.62:0/4247707568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63b81025f0 0x7f63b8103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:17.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.772+0000 7f63bf0ec700 1 -- 172.21.15.62:0/4247707568 >> 172.21.15.62:0/4247707568 conn(0x7f63b80fc990 msgr2=0x7f63b806a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:17.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.772+0000 7f63bf0ec700 1 -- 172.21.15.62:0/4247707568 shutdown_connections 2024-06-10T00:32:17.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:17.772+0000 7f63bf0ec700 1 -- 172.21.15.62:0/4247707568 wait complete. 2024-06-10T00:32:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:17 smithi062 ceph-mon[32390]: pgmap v416: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:32:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:17 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/4247707568' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-10T00:32:18.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:17 smithi138 ceph-mon[36549]: pgmap v416: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:32:18.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:17 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/4247707568' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-10T00:32:18.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:17 smithi105 ceph-mon[36321]: pgmap v416: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:32:18.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:17 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/4247707568' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-10T00:32:18.433 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-10T00:32:18.444 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:32:18.697 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:20.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:19 smithi062 ceph-mon[32390]: pgmap v417: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:20.098 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:19 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:32:20.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:19 smithi138 ceph-mon[36549]: pgmap v417: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:20.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:19 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:32:20.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:19 smithi105 ceph-mon[36321]: pgmap v417: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:20.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:19 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:32:20.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.350+0000 7f04ff83d700 1 -- 172.21.15.62:0/781909069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04f8101d50 msgr2=0x7f04f810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:20.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.350+0000 7f04ff83d700 1 --2- 172.21.15.62:0/781909069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04f8101d50 0x7f04f810e240 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f04f000b0a0 tx=0x7f04f000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:32:20.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.350+0000 7f04ff83d700 1 -- 172.21.15.62:0/781909069 shutdown_connections 2024-06-10T00:32:20.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.350+0000 7f04ff83d700 1 --2- 172.21.15.62:0/781909069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04f8101d50 0x7f04f810e240 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:20.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.350+0000 7f04ff83d700 1 --2- 172.21.15.62:0/781909069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107400 0x7f04f81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:20.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.350+0000 7f04ff83d700 1 --2- 172.21.15.62:0/781909069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f8101390 0x7f04f8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:20.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.350+0000 7f04ff83d700 1 -- 172.21.15.62:0/781909069 >> 172.21.15.62:0/781909069 conn(0x7f04f8078340 msgr2=0x7f04f8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:20.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.350+0000 7f04ff83d700 1 -- 172.21.15.62:0/781909069 shutdown_connections 2024-06-10T00:32:20.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.350+0000 7f04ff83d700 1 -- 172.21.15.62:0/781909069 wait complete. 2024-06-10T00:32:20.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.351+0000 7f04ff83d700 1 Processor -- start 2024-06-10T00:32:20.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.351+0000 7f04ff83d700 1 -- start start 2024-06-10T00:32:20.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.351+0000 7f04ff83d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04f8101390 0x7f04f819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:20.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04ff83d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f8101d50 0x7f04f819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:20.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04ff83d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107400 0x7f04f8196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:20.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04ff83d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8111670 con 0x7f04f8101390 2024-06-10T00:32:20.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04ff83d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04f81114f0 con 0x7f04f8107400 2024-06-10T00:32:20.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04ff83d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04f81117f0 con 0x7f04f8101d50 2024-06-10T00:32:20.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04fd5d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04f8101390 0x7f04f819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:20.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04fddda700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107400 0x7f04f8196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:20.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04fd5d9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04f8101390 0x7f04f819e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54610/0 (socket says 172.21.15.62:54610) 2024-06-10T00:32:20.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04fd5d9700 1 -- 172.21.15.62:0/447783777 learned_addr learned my addr 172.21.15.62:0/447783777 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:20.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04fcdd8700 1 --2- 172.21.15.62:0/447783777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f8101d50 0x7f04f819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04fddda700 1 -- 172.21.15.62:0/447783777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f8101d50 msgr2=0x7f04f819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04fddda700 1 --2- 172.21.15.62:0/447783777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f8101d50 0x7f04f819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04fddda700 1 -- 172.21.15.62:0/447783777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04f8101390 msgr2=0x7f04f819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04fddda700 1 --2- 172.21.15.62:0/447783777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04f8101390 0x7f04f819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04fddda700 1 -- 172.21.15.62:0/447783777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f81974a0 con 0x7f04f8107400 2024-06-10T00:32:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.352+0000 7f04fddda700 1 --2- 172.21.15.62:0/447783777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107400 0x7f04f8196ba0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f04f000d010 tx=0x7f04f0004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.353+0000 7f04ee7fc700 1 -- 172.21.15.62:0/447783777 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0021070 con 0x7f04f8107400 2024-06-10T00:32:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.353+0000 7f04ee7fc700 1 -- 172.21.15.62:0/447783777 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f0007b80 con 0x7f04f8107400 2024-06-10T00:32:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.353+0000 7f04ff83d700 1 -- 172.21.15.62:0/447783777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f8197730 con 0x7f04f8107400 2024-06-10T00:32:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.353+0000 7f04ee7fc700 1 -- 172.21.15.62:0/447783777 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f00052a0 con 0x7f04f8107400 2024-06-10T00:32:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.353+0000 7f04ff83d700 1 -- 172.21.15.62:0/447783777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f81b4550 con 0x7f04f8107400 2024-06-10T00:32:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.354+0000 7f04ff83d700 1 -- 172.21.15.62:0/447783777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f8102ab0 con 0x7f04f8107400 2024-06-10T00:32:20.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.355+0000 7f04ee7fc700 1 -- 172.21.15.62:0/447783777 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f04f0024070 con 0x7f04f8107400 2024-06-10T00:32:20.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.355+0000 7f04ee7fc700 1 --2- 172.21.15.62:0/447783777 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04e40a2c20 0x7f04e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:20.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.356+0000 7f04fd5d9700 1 --2- 172.21.15.62:0/447783777 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04e40a2c20 0x7f04e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:20.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.356+0000 7f04ee7fc700 1 -- 172.21.15.62:0/447783777 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f04f000f020 con 0x7f04f8107400 2024-06-10T00:32:20.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.356+0000 7f04fd5d9700 1 --2- 172.21.15.62:0/447783777 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04e40a2c20 0x7f04e40a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f04f4007c10 tx=0x7f04f4007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:20.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.359+0000 7f04ee7fc700 1 -- 172.21.15.62:0/447783777 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f04f002b020 con 0x7f04f8107400 2024-06-10T00:32:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.616+0000 7f04ff83d700 1 -- 172.21.15.62:0/447783777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04f81b4720 con 0x7f04f8107400 2024-06-10T00:32:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.617+0000 7f04ee7fc700 1 -- 172.21.15.62:0/447783777 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7f04f009d5e0 con 0x7f04f8107400 2024-06-10T00:32:20.617 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:32:20.617 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:29:47.712789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24431,24424,24418,24433,24464]},"id":1}]} 2024-06-10T00:32:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.619+0000 7f04ff83d700 1 -- 172.21.15.62:0/447783777 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04e40a2c20 msgr2=0x7f04e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.619+0000 7f04ff83d700 1 --2- 172.21.15.62:0/447783777 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04e40a2c20 0x7f04e40a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f04f4007c10 tx=0x7f04f4007480 comp rx=0 tx=0).stop 2024-06-10T00:32:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.619+0000 7f04ff83d700 1 -- 172.21.15.62:0/447783777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107400 msgr2=0x7f04f8196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.619+0000 7f04ff83d700 1 --2- 172.21.15.62:0/447783777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107400 0x7f04f8196ba0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f04f000d010 tx=0x7f04f0004d50 comp rx=0 tx=0).stop 2024-06-10T00:32:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.619+0000 7f04ff83d700 1 -- 172.21.15.62:0/447783777 shutdown_connections 2024-06-10T00:32:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.619+0000 7f04ff83d700 1 --2- 172.21.15.62:0/447783777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04f8101390 0x7f04f819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.619+0000 7f04ff83d700 1 --2- 172.21.15.62:0/447783777 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04e40a2c20 0x7f04e40a50e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.619+0000 7f04ff83d700 1 --2- 172.21.15.62:0/447783777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f8107400 0x7f04f8196ba0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.619+0000 7f04ff83d700 1 --2- 172.21.15.62:0/447783777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f8101d50 0x7f04f819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.619+0000 7f04ff83d700 1 -- 172.21.15.62:0/447783777 >> 172.21.15.62:0/447783777 conn(0x7f04f8078340 msgr2=0x7f04f806a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.619+0000 7f04ff83d700 1 -- 172.21.15.62:0/447783777 shutdown_connections 2024-06-10T00:32:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:20.619+0000 7f04ff83d700 1 -- 172.21.15.62:0/447783777 wait complete. 2024-06-10T00:32:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 29 2024-06-10T00:32:21.263 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:21 smithi062 ceph-mon[32390]: pgmap v418: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:21.263 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.263 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.264 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.264 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.264 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.264 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.264 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:21 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/447783777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:32:21.264 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:32:21.264 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:32:21.264 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:21 smithi062 ceph-mon[32390]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.292 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:32:21.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:21 smithi138 ceph-mon[36549]: pgmap v418: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:21.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:21 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:21 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:21 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:21 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:21 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:21 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:21 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/447783777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:32:21.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:21 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:32:21.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:21 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:32:21.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:21 smithi138 ceph-mon[36549]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:21 smithi105 ceph-mon[36321]: pgmap v418: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:21.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:21 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:21 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:21 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:21 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:21 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:21 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:21 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/447783777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:32:21.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:21 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:32:21.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:21 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:32:21.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:21 smithi105 ceph-mon[36321]: from='mgr.14152 172.21.15.62:0/139204246' entity='mgr.x' 2024-06-10T00:32:21.542 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:23.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.249+0000 7f78e6eca700 1 -- 172.21.15.62:0/1097429189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e006b580 msgr2=0x7f78e010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:23.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.249+0000 7f78e6eca700 1 --2- 172.21.15.62:0/1097429189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e006b580 0x7f78e010f320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f78d800b0a0 tx=0x7f78d800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:32:23.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.250+0000 7f78e6eca700 1 -- 172.21.15.62:0/1097429189 shutdown_connections 2024-06-10T00:32:23.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.250+0000 7f78e6eca700 1 --2- 172.21.15.62:0/1097429189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e006b580 0x7f78e010f320 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:23.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.250+0000 7f78e6eca700 1 --2- 172.21.15.62:0/1097429189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78e0108500 0x7f78e01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:23.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.250+0000 7f78e6eca700 1 --2- 172.21.15.62:0/1097429189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e006abc0 0x7f78e006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:23.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.250+0000 7f78e6eca700 1 -- 172.21.15.62:0/1097429189 >> 172.21.15.62:0/1097429189 conn(0x7f78e0078340 msgr2=0x7f78e0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:23.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.250+0000 7f78e6eca700 1 -- 172.21.15.62:0/1097429189 shutdown_connections 2024-06-10T00:32:23.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.250+0000 7f78e6eca700 1 -- 172.21.15.62:0/1097429189 wait complete. 2024-06-10T00:32:23.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.250+0000 7f78e6eca700 1 Processor -- start 2024-06-10T00:32:23.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.250+0000 7f78e6eca700 1 -- start start 2024-06-10T00:32:23.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.250+0000 7f78e6eca700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e006abc0 0x7f78e019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:23.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.250+0000 7f78e6eca700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78e006b580 0x7f78e019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:23.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.250+0000 7f78e6eca700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e0108500 0x7f78e0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:23.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78e6eca700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f78e0112780 con 0x7f78e0108500 2024-06-10T00:32:23.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78e6eca700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f78e0112600 con 0x7f78e006abc0 2024-06-10T00:32:23.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78e6eca700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78e0112900 con 0x7f78e006b580 2024-06-10T00:32:23.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78e5467700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e0108500 0x7f78e0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:23.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78e5467700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e0108500 0x7f78e0198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54634/0 (socket says 172.21.15.62:54634) 2024-06-10T00:32:23.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78e5467700 1 -- 172.21.15.62:0/3187085191 learned_addr learned my addr 172.21.15.62:0/3187085191 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:23.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78e4c66700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e006abc0 0x7f78e019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:23.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78dffff700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78e006b580 0x7f78e019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:23.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78e5467700 1 -- 172.21.15.62:0/3187085191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78e006b580 msgr2=0x7f78e019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:23.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78e5467700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78e006b580 0x7f78e019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:23.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78e5467700 1 -- 172.21.15.62:0/3187085191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e006abc0 msgr2=0x7f78e019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:23.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78e5467700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e006abc0 0x7f78e019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:23.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78e5467700 1 -- 172.21.15.62:0/3187085191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e0199160 con 0x7f78e0108500 2024-06-10T00:32:23.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78dffff700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78e006b580 0x7f78e019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:32:23.255 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:23 smithi062 ceph-mon[32390]: pgmap v419: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:32:23.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78e5467700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e0108500 0x7f78e0198a00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f78d80062a0 tx=0x7f78d8004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:23.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78ddffb700 1 -- 172.21.15.62:0/3187085191 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8021070 con 0x7f78e0108500 2024-06-10T00:32:23.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.251+0000 7f78ddffb700 1 -- 172.21.15.62:0/3187085191 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d8007c00 con 0x7f78e0108500 2024-06-10T00:32:23.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.252+0000 7f78e6eca700 1 -- 172.21.15.62:0/3187085191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e01993f0 con 0x7f78e0108500 2024-06-10T00:32:23.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.252+0000 7f78ddffb700 1 -- 172.21.15.62:0/3187085191 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d801d450 con 0x7f78e0108500 2024-06-10T00:32:23.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.252+0000 7f78e6eca700 1 -- 172.21.15.62:0/3187085191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e01a5710 con 0x7f78e0108500 2024-06-10T00:32:23.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.253+0000 7f78e6eca700 1 -- 172.21.15.62:0/3187085191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e010a9e0 con 0x7f78e0108500 2024-06-10T00:32:23.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.255+0000 7f78ddffb700 1 -- 172.21.15.62:0/3187085191 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f78d8004000 con 0x7f78e0108500 2024-06-10T00:32:23.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.255+0000 7f78ddffb700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f78cc0a2ca0 0x7f78cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:23.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.256+0000 7f78ddffb700 1 -- 172.21.15.62:0/3187085191 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f78d800f020 con 0x7f78e0108500 2024-06-10T00:32:23.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.256+0000 7f78e4c66700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f78cc0a2ca0 0x7f78cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:23.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.256+0000 7f78e4c66700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f78cc0a2ca0 0x7f78cc0a5160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f78e0199b40 tx=0x7f78d000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:23.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.259+0000 7f78ddffb700 1 -- 172.21.15.62:0/3187085191 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f78d802a020 con 0x7f78e0108500 2024-06-10T00:32:23.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:23 smithi138 ceph-mon[36549]: pgmap v419: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:32:23.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.513+0000 7f78e6eca700 1 -- 172.21.15.62:0/3187085191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78e0199f30 con 0x7f78e0108500 2024-06-10T00:32:23.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.514+0000 7f78ddffb700 1 -- 172.21.15.62:0/3187085191 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7f78d809d570 con 0x7f78e0108500 2024-06-10T00:32:23.514 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:32:23.514 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:29:47.712789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24431,24424,24418,24433,24464]},"id":1}]} 2024-06-10T00:32:23.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.516+0000 7f78e6eca700 1 -- 172.21.15.62:0/3187085191 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f78cc0a2ca0 msgr2=0x7f78cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:23.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.516+0000 7f78e6eca700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f78cc0a2ca0 0x7f78cc0a5160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f78e0199b40 tx=0x7f78d000b3c0 comp rx=0 tx=0).stop 2024-06-10T00:32:23.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.516+0000 7f78e6eca700 1 -- 172.21.15.62:0/3187085191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e0108500 msgr2=0x7f78e0198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:23.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.516+0000 7f78e6eca700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e0108500 0x7f78e0198a00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f78d80062a0 tx=0x7f78d8004d70 comp rx=0 tx=0).stop 2024-06-10T00:32:23.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.516+0000 7f78e6eca700 1 -- 172.21.15.62:0/3187085191 shutdown_connections 2024-06-10T00:32:23.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.516+0000 7f78e6eca700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78e0108500 0x7f78e0198a00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:23.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.516+0000 7f78e6eca700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f78cc0a2ca0 0x7f78cc0a5160 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:23.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.516+0000 7f78e6eca700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e006abc0 0x7f78e019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:23.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.516+0000 7f78e6eca700 1 --2- 172.21.15.62:0/3187085191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78e006b580 0x7f78e019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:23.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.516+0000 7f78e6eca700 1 -- 172.21.15.62:0/3187085191 >> 172.21.15.62:0/3187085191 conn(0x7f78e0078340 msgr2=0x7f78e0100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:23.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.516+0000 7f78e6eca700 1 -- 172.21.15.62:0/3187085191 shutdown_connections 2024-06-10T00:32:23.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:23.516+0000 7f78e6eca700 1 -- 172.21.15.62:0/3187085191 wait complete. 2024-06-10T00:32:23.517 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 29 2024-06-10T00:32:23.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:23 smithi105 ceph-mon[36321]: pgmap v419: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:32:24.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:24 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3187085191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:32:24.160 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:32:24.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:24 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3187085191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:32:24.410 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:24.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:24 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3187085191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:32:25.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:25 smithi062 ceph-mon[32390]: pgmap v420: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:32:25.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:25 smithi138 ceph-mon[36549]: pgmap v420: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:32:25.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:25 smithi105 ceph-mon[36321]: pgmap v420: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:32:26.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.142+0000 7fd79b63f700 1 -- 172.21.15.62:0/3424118738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7941066b0 msgr2=0x7fd79410e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:26.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.142+0000 7fd79b63f700 1 --2- 172.21.15.62:0/3424118738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7941066b0 0x7fd79410e280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00b0d0 tx=0x7fd78c00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:32:26.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.142+0000 7fd79b63f700 1 -- 172.21.15.62:0/3424118738 shutdown_connections 2024-06-10T00:32:26.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.142+0000 7fd79b63f700 1 --2- 172.21.15.62:0/3424118738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7941066b0 0x7fd79410e280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:26.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.142+0000 7fd79b63f700 1 --2- 172.21.15.62:0/3424118738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7941015d0 0x7fd7941019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:26.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.142+0000 7fd79b63f700 1 --2- 172.21.15.62:0/3424118738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd794101f80 0x7fd794105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:26.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.142+0000 7fd79b63f700 1 -- 172.21.15.62:0/3424118738 >> 172.21.15.62:0/3424118738 conn(0x7fd794078430 msgr2=0x7fd794078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:26.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.142+0000 7fd79b63f700 1 -- 172.21.15.62:0/3424118738 shutdown_connections 2024-06-10T00:32:26.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.142+0000 7fd79b63f700 1 -- 172.21.15.62:0/3424118738 wait complete. 2024-06-10T00:32:26.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd79b63f700 1 Processor -- start 2024-06-10T00:32:26.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd79b63f700 1 -- start start 2024-06-10T00:32:26.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd79b63f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7941015d0 0x7fd79419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:26.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd79b63f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd794101f80 0x7fd79419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:26.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd79b63f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7941066b0 0x7fd794198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:26.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd79b63f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941116b0 con 0x7fd7941066b0 2024-06-10T00:32:26.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd79b63f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd794111530 con 0x7fd794101f80 2024-06-10T00:32:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd79b63f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd794111830 con 0x7fd7941015d0 2024-06-10T00:32:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd799bdc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7941066b0 0x7fd794198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd799bdc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7941066b0 0x7fd794198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54658/0 (socket says 172.21.15.62:54658) 2024-06-10T00:32:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd799bdc700 1 -- 172.21.15.62:0/2037708631 learned_addr learned my addr 172.21.15.62:0/2037708631 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd7993db700 1 --2- 172.21.15.62:0/2037708631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7941015d0 0x7fd79419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd798bda700 1 --2- 172.21.15.62:0/2037708631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd794101f80 0x7fd79419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd799bdc700 1 -- 172.21.15.62:0/2037708631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7941015d0 msgr2=0x7fd79419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd799bdc700 1 --2- 172.21.15.62:0/2037708631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7941015d0 0x7fd79419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd799bdc700 1 -- 172.21.15.62:0/2037708631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd794101f80 msgr2=0x7fd79419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd799bdc700 1 --2- 172.21.15.62:0/2037708631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd794101f80 0x7fd79419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:26.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.143+0000 7fd799bdc700 1 -- 172.21.15.62:0/2037708631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7941990a0 con 0x7fd7941066b0 2024-06-10T00:32:26.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.144+0000 7fd799bdc700 1 --2- 172.21.15.62:0/2037708631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7941066b0 0x7fd794198940 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd78c006070 tx=0x7fd78c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:26.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.144+0000 7fd7867fc700 1 -- 172.21.15.62:0/2037708631 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c021070 con 0x7fd7941066b0 2024-06-10T00:32:26.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.144+0000 7fd7867fc700 1 -- 172.21.15.62:0/2037708631 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78c007b80 con 0x7fd7941066b0 2024-06-10T00:32:26.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.144+0000 7fd79b63f700 1 -- 172.21.15.62:0/2037708631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd794199330 con 0x7fd7941066b0 2024-06-10T00:32:26.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.144+0000 7fd7867fc700 1 -- 172.21.15.62:0/2037708631 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c005330 con 0x7fd7941066b0 2024-06-10T00:32:26.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.144+0000 7fd79b63f700 1 -- 172.21.15.62:0/2037708631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7941a5650 con 0x7fd7941066b0 2024-06-10T00:32:26.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.145+0000 7fd79b63f700 1 -- 172.21.15.62:0/2037708631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd794069180 con 0x7fd7941066b0 2024-06-10T00:32:26.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.146+0000 7fd7867fc700 1 -- 172.21.15.62:0/2037708631 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd78c024070 con 0x7fd7941066b0 2024-06-10T00:32:26.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.147+0000 7fd7867fc700 1 --2- 172.21.15.62:0/2037708631 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd7800a2ca0 0x7fd7800a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:26.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.147+0000 7fd7993db700 1 --2- 172.21.15.62:0/2037708631 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd7800a2ca0 0x7fd7800a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:26.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.147+0000 7fd7867fc700 1 -- 172.21.15.62:0/2037708631 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fd78c0d0470 con 0x7fd7941066b0 2024-06-10T00:32:26.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.148+0000 7fd7993db700 1 --2- 172.21.15.62:0/2037708631 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd7800a2ca0 0x7fd7800a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd794199a80 tx=0x7fd79000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:26.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.151+0000 7fd7867fc700 1 -- 172.21.15.62:0/2037708631 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd78c09d090 con 0x7fd7941066b0 2024-06-10T00:32:26.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.357+0000 7fd79b63f700 1 -- 172.21.15.62:0/2037708631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd79402dca0 con 0x7fd7941066b0 2024-06-10T00:32:26.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.359+0000 7fd7867fc700 1 -- 172.21.15.62:0/2037708631 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24839 (secure 0 0 0) 0x7fd78c09c990 con 0x7fd7941066b0 2024-06-10T00:32:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:32:26.360 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":99,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:30:24.104116+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:32:26.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.362+0000 7fd79b63f700 1 -- 172.21.15.62:0/2037708631 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd7800a2ca0 msgr2=0x7fd7800a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:26.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.362+0000 7fd79b63f700 1 --2- 172.21.15.62:0/2037708631 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd7800a2ca0 0x7fd7800a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd794199a80 tx=0x7fd79000b410 comp rx=0 tx=0).stop 2024-06-10T00:32:26.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.362+0000 7fd79b63f700 1 -- 172.21.15.62:0/2037708631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7941066b0 msgr2=0x7fd794198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:26.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.362+0000 7fd79b63f700 1 --2- 172.21.15.62:0/2037708631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7941066b0 0x7fd794198940 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd78c006070 tx=0x7fd78c004d00 comp rx=0 tx=0).stop 2024-06-10T00:32:26.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.362+0000 7fd79b63f700 1 -- 172.21.15.62:0/2037708631 shutdown_connections 2024-06-10T00:32:26.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.362+0000 7fd79b63f700 1 --2- 172.21.15.62:0/2037708631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7941066b0 0x7fd794198940 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:26.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.362+0000 7fd79b63f700 1 --2- 172.21.15.62:0/2037708631 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd7800a2ca0 0x7fd7800a5160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:26.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.362+0000 7fd79b63f700 1 --2- 172.21.15.62:0/2037708631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7941015d0 0x7fd79419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:26.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.362+0000 7fd79b63f700 1 --2- 172.21.15.62:0/2037708631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd794101f80 0x7fd79419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:26.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.362+0000 7fd79b63f700 1 -- 172.21.15.62:0/2037708631 >> 172.21.15.62:0/2037708631 conn(0x7fd794078430 msgr2=0x7fd79406ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:26.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.362+0000 7fd79b63f700 1 -- 172.21.15.62:0/2037708631 shutdown_connections 2024-06-10T00:32:26.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:26.362+0000 7fd79b63f700 1 -- 172.21.15.62:0/2037708631 wait complete. 2024-06-10T00:32:26.996 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-10T00:28:39.478894+0000', 'modified': '2024-06-10T00:29:47.712789+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24464, 'mds_1': 24433, 'mds_2': 24431, 'mds_3': 24424, 'mds_4': 24418}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6839/4040406390', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 4040406390}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 4040406390}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6841/2460552589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6840', 'nonce': 2460552589}, {'type': 'v1', 'addr': '172.21.15.62:6841', 'nonce': 2460552589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24377': {'gid': 24377, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6833/1194987741', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6832', 'nonce': 1194987741}, {'type': 'v1', 'addr': '172.21.15.105:6833', 'nonce': 1194987741}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24382': {'gid': 24382, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6833/2673201423', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 2673201423}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 2673201423}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6835/588077925', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 588077925}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 588077925}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 27, 'addr': '172.21.15.138:6835/331298815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 331298815}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 331298815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.138:6837/3880084398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 3880084398}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 3880084398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.105:6837/4270590303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 4270590303}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 4270590303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.138:6839/339302206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 339302206}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 339302206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.105:6839/1632080154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 1632080154}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 1632080154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24464, 'qdb_cluster': [24431, 24424, 24418, 24433, 24464]} 2024-06-10T00:32:26.996 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-10T00:32:26.996 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-06-10T00:32:27.250 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:27.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:27 smithi062 ceph-mon[32390]: pgmap v421: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:32:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:27 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2037708631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:32:27.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:27 smithi138 ceph-mon[36549]: pgmap v421: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:32:27.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:27 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2037708631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:32:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:27 smithi105 ceph-mon[36321]: pgmap v421: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:32:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:27 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2037708631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:32:28.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.962+0000 7fce0af9c700 1 -- 172.21.15.62:0/2153756366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce04107860 msgr2=0x7fce0410f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:28.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.962+0000 7fce0af9c700 1 --2- 172.21.15.62:0/2153756366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce04107860 0x7fce0410f3e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00b0d0 tx=0x7fcdfc00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.962+0000 7fce0af9c700 1 -- 172.21.15.62:0/2153756366 shutdown_connections 2024-06-10T00:32:28.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.962+0000 7fce0af9c700 1 --2- 172.21.15.62:0/2153756366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce04107860 0x7fce0410f3e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.962+0000 7fce0af9c700 1 --2- 172.21.15.62:0/2153756366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce04102770 0x7fce04102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.962+0000 7fce0af9c700 1 --2- 172.21.15.62:0/2153756366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce04103120 0x7fce04107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.962+0000 7fce0af9c700 1 -- 172.21.15.62:0/2153756366 >> 172.21.15.62:0/2153756366 conn(0x7fce04079600 msgr2=0x7fce04079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:28.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.962+0000 7fce0af9c700 1 -- 172.21.15.62:0/2153756366 shutdown_connections 2024-06-10T00:32:28.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.962+0000 7fce0af9c700 1 -- 172.21.15.62:0/2153756366 wait complete. 2024-06-10T00:32:28.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce0af9c700 1 Processor -- start 2024-06-10T00:32:28.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce0af9c700 1 -- start start 2024-06-10T00:32:28.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce0af9c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce04102770 0x7fce0419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:28.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce0af9c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce04103120 0x7fce0419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:28.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce0af9c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce04107860 0x7fce04198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:28.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce0af9c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fce041127a0 con 0x7fce04107860 2024-06-10T00:32:28.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce0af9c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fce04112620 con 0x7fce04102770 2024-06-10T00:32:28.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce0af9c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce04112920 con 0x7fce04103120 2024-06-10T00:32:28.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce09539700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce04107860 0x7fce04198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:28.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce09539700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce04107860 0x7fce04198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45434/0 (socket says 172.21.15.62:45434) 2024-06-10T00:32:28.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce09539700 1 -- 172.21.15.62:0/3423556966 learned_addr learned my addr 172.21.15.62:0/3423556966 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:28.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce08d38700 1 --2- 172.21.15.62:0/3423556966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce04102770 0x7fce0419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:28.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce03fff700 1 --2- 172.21.15.62:0/3423556966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce04103120 0x7fce0419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:28.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce09539700 1 -- 172.21.15.62:0/3423556966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce04103120 msgr2=0x7fce0419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:28.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce09539700 1 --2- 172.21.15.62:0/3423556966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce04103120 0x7fce0419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce09539700 1 -- 172.21.15.62:0/3423556966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce04102770 msgr2=0x7fce0419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:28.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce09539700 1 --2- 172.21.15.62:0/3423556966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce04102770 0x7fce0419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce09539700 1 -- 172.21.15.62:0/3423556966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce041990a0 con 0x7fce04107860 2024-06-10T00:32:28.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.963+0000 7fce09539700 1 --2- 172.21.15.62:0/3423556966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce04107860 0x7fce04198940 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc006070 tx=0x7fcdfc004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:28.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.964+0000 7fce01ffb700 1 -- 172.21.15.62:0/3423556966 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc021070 con 0x7fce04107860 2024-06-10T00:32:28.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.964+0000 7fce01ffb700 1 -- 172.21.15.62:0/3423556966 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdfc007b80 con 0x7fce04107860 2024-06-10T00:32:28.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.964+0000 7fce01ffb700 1 -- 172.21.15.62:0/3423556966 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0052a0 con 0x7fce04107860 2024-06-10T00:32:28.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.964+0000 7fce0af9c700 1 -- 172.21.15.62:0/3423556966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce04199330 con 0x7fce04107860 2024-06-10T00:32:28.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.964+0000 7fce0af9c700 1 -- 172.21.15.62:0/3423556966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce041a5650 con 0x7fce04107860 2024-06-10T00:32:28.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.966+0000 7fce01ffb700 1 -- 172.21.15.62:0/3423556966 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcdfc024070 con 0x7fce04107860 2024-06-10T00:32:28.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.967+0000 7fce01ffb700 1 --2- 172.21.15.62:0/3423556966 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcdf40a2c20 0x7fcdf40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:28.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.967+0000 7fce08d38700 1 --2- 172.21.15.62:0/3423556966 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcdf40a2c20 0x7fcdf40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:28.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.967+0000 7fce01ffb700 1 -- 172.21.15.62:0/3423556966 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fcdfc00f020 con 0x7fce04107860 2024-06-10T00:32:28.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.967+0000 7fce08d38700 1 --2- 172.21.15.62:0/3423556966 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcdf40a2c20 0x7fcdf40a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8005950 tx=0x7fcdf800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:28.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce1581fb80 1 Processor -- start 2024-06-10T00:32:28.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce1581fb80 1 -- start start 2024-06-10T00:32:28.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce1581fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd8122420 0x55cbd8122800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:28.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce1581fb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd811c620 0x55cbd811caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:28.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce1581fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811cfe0 0x55cbd81291c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:28.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce1581fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cbd800af10 con 0x55cbd8122420 2024-06-10T00:32:28.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce1581fb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cbd8069800 con 0x55cbd811cfe0 2024-06-10T00:32:28.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce1581fb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbd7fd1410 con 0x55cbd811c620 2024-06-10T00:32:28.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce08d38700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd8122420 0x55cbd8122800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:28.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce08d38700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd8122420 0x55cbd8122800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45438/0 (socket says 172.21.15.62:45438) 2024-06-10T00:32:28.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce08d38700 1 -- 172.21.15.62:0/3515664977 learned_addr learned my addr 172.21.15.62:0/3515664977 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:28.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce03fff700 1 --2- 172.21.15.62:0/3515664977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd811c620 0x55cbd811caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:28.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce09539700 1 --2- 172.21.15.62:0/3515664977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811cfe0 0x55cbd81291c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:28.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce08d38700 1 -- 172.21.15.62:0/3515664977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd811c620 msgr2=0x55cbd811caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:28.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce08d38700 1 --2- 172.21.15.62:0/3515664977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd811c620 0x55cbd811caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce08d38700 1 -- 172.21.15.62:0/3515664977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811cfe0 msgr2=0x55cbd81291c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:28.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce08d38700 1 --2- 172.21.15.62:0/3515664977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811cfe0 0x55cbd81291c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce08d38700 1 -- 172.21.15.62:0/3515664977 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd7fb8880 con 0x55cbd8122420 2024-06-10T00:32:28.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.968+0000 7fce03fff700 1 --2- 172.21.15.62:0/3515664977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd811c620 0x55cbd811caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:32:28.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.969+0000 7fce08d38700 1 --2- 172.21.15.62:0/3515664977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd8122420 0x55cbd8122800 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800fda0 tx=0x7fcdf8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c480c74eb51030 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:28.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.969+0000 7fcdeeffd700 1 -- 172.21.15.62:0/3515664977 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8016070 con 0x55cbd8122420 2024-06-10T00:32:28.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.969+0000 7fcdeeffd700 1 -- 172.21.15.62:0/3515664977 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdf8010da0 con 0x55cbd8122420 2024-06-10T00:32:28.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.969+0000 7fcdeeffd700 1 -- 172.21.15.62:0/3515664977 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf801b420 con 0x55cbd8122420 2024-06-10T00:32:28.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.969+0000 7fce1581fb80 1 -- 172.21.15.62:0/3515664977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd8122420 msgr2=0x55cbd8122800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:28.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.969+0000 7fce1581fb80 1 --2- 172.21.15.62:0/3515664977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd8122420 0x55cbd8122800 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800fda0 tx=0x7fcdf8012000 comp rx=0 tx=0).stop 2024-06-10T00:32:28.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.970+0000 7fce1581fb80 1 -- 172.21.15.62:0/3515664977 shutdown_connections 2024-06-10T00:32:28.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.970+0000 7fce1581fb80 1 --2- 172.21.15.62:0/3515664977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd8122420 0x55cbd8122800 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.970+0000 7fce1581fb80 1 --2- 172.21.15.62:0/3515664977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811cfe0 0x55cbd81291c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.970+0000 7fce1581fb80 1 --2- 172.21.15.62:0/3515664977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd811c620 0x55cbd811caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.970+0000 7fce1581fb80 1 -- 172.21.15.62:0/3515664977 >> 172.21.15.62:0/3515664977 conn(0x55cbd8045030 msgr2=0x55cbd81131d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.970+0000 7fce1581fb80 1 -- 172.21.15.62:0/3515664977 shutdown_connections 2024-06-10T00:32:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.970+0000 7fce1581fb80 1 -- 172.21.15.62:0/3515664977 wait complete. 2024-06-10T00:32:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.970+0000 7fce1581fb80 1 -- start start 2024-06-10T00:32:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.970+0000 7fce1581fb80 1 Processor -- start 2024-06-10T00:32:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce1581fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811c620 0x55cbd8128ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce1581fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811cfe0 0x55cbd81261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce1581fb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd8122420 0x55cbd8126720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce1581fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cbd8009f70 con 0x55cbd811cfe0 2024-06-10T00:32:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce1581fb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cbd80657c0 con 0x55cbd811c620 2024-06-10T00:32:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce1581fb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbd8129700 con 0x55cbd8122420 2024-06-10T00:32:28.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce1581fb80 10 client.0 _ll_register_callbacks cb 0x55cbd7fff560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:32:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce03fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811cfe0 0x55cbd81261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce03fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811cfe0 0x55cbd81261e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45444/0 (socket says 172.21.15.62:45444) 2024-06-10T00:32:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce03fff700 1 -- 172.21.15.62:0/209174733 learned_addr learned my addr 172.21.15.62:0/209174733 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce09539700 1 --2- 172.21.15.62:0/209174733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd8122420 0x55cbd8126720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce08d38700 1 --2- 172.21.15.62:0/209174733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811c620 0x55cbd8128ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce03fff700 1 -- 172.21.15.62:0/209174733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd8122420 msgr2=0x55cbd8126720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce03fff700 1 --2- 172.21.15.62:0/209174733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd8122420 0x55cbd8126720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce03fff700 1 -- 172.21.15.62:0/209174733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811c620 msgr2=0x55cbd8128ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce03fff700 1 --2- 172.21.15.62:0/209174733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811c620 0x55cbd8128ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce03fff700 1 -- 172.21.15.62:0/209174733 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd802f700 con 0x55cbd811cfe0 2024-06-10T00:32:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce08d38700 1 --2- 172.21.15.62:0/209174733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811c620 0x55cbd8128ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:32:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fce03fff700 1 --2- 172.21.15.62:0/209174733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811cfe0 0x55cbd81261e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000bb40 tx=0x7fcdf000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac60759d301ed9a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.971+0000 7fcde3fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.972+0000 7fcde3fff700 1 -- 172.21.15.62:0/209174733 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf001b440 con 0x55cbd811cfe0 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.972+0000 7fce1581fb80 1 -- 172.21.15.62:0/209174733 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf8010da0 con 0x55cbd811cfe0 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.972+0000 7fcde3fff700 1 -- 172.21.15.62:0/209174733 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fcdf0011440 con 0x55cbd811cfe0 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.972+0000 7fcde3fff700 1 -- 172.21.15.62:0/209174733 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcdf0011640 con 0x55cbd811cfe0 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.972+0000 7fce1581fb80 10 client.15231 fetch_fsmap learned FSMap version 29 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.972+0000 7fce1581fb80 1 -- 172.21.15.62:0/209174733 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cbd8133eb0 con 0x55cbd811cfe0 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.972+0000 7fcde3fff700 1 -- 172.21.15.62:0/209174733 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdf8010da0 con 0x55cbd811cfe0 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.972+0000 7fcde3fff700 1 -- 172.21.15.62:0/209174733 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcdf0019020 con 0x55cbd811cfe0 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.972+0000 7fce1581fb80 10 client.15231 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.972+0000 7fce1581fb80 10 client.15231 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.972+0000 7fce1581fb80 20 client.15231 populate_metadata read hostname 'smithi062' 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.972+0000 7fce1581fb80 1 --2- 172.21.15.62:0/209174733 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55cbd81364b0 0x55cbd8137920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.972+0000 7fce1581fb80 4 client.15231 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.972+0000 7fce1581fb80 1 -- 172.21.15.62:0/209174733 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbd8076d00 con 0x55cbd81364b0 2024-06-10T00:32:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.973+0000 7fce08d38700 1 --2- 172.21.15.62:0/209174733 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55cbd81364b0 0x55cbd8137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:28.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.973+0000 7fce08d38700 1 --2- 172.21.15.62:0/209174733 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55cbd81364b0 0x55cbd8137920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=29b9d26f26664f2f server_cookie=39daa072194db73e in_seq=0 out_seq=0 2024-06-10T00:32:28.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.973+0000 7fcde3fff700 10 client.15231 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:32:28.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fcde3fff700 1 -- 172.21.15.62:0/209174733 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cbd8076d00 con 0x55cbd81364b0 2024-06-10T00:32:28.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fcde3fff700 10 client.15231 handle_command_reply: tid=0 2024-06-10T00:32:28.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fce1581fb80 1 client.15231 shutdown 2024-06-10T00:32:28.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fce1581fb80 1 -- 172.21.15.62:0/209174733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811cfe0 msgr2=0x55cbd81261e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:28.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fce1581fb80 1 --2- 172.21.15.62:0/209174733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811cfe0 0x55cbd81261e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000bb40 tx=0x7fcdf000be50 comp rx=0 tx=0).stop 2024-06-10T00:32:28.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fce1581fb80 1 -- 172.21.15.62:0/209174733 shutdown_connections 2024-06-10T00:32:28.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fce1581fb80 1 --2- 172.21.15.62:0/209174733 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55cbd81364b0 0x55cbd8137920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fce1581fb80 1 --2- 172.21.15.62:0/209174733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd8122420 0x55cbd8126720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fce1581fb80 1 --2- 172.21.15.62:0/209174733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811c620 0x55cbd8128ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fcde3fff700 0 client.15231 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:32:28.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fce1581fb80 1 --2- 172.21.15.62:0/209174733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811cfe0 0x55cbd81261e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fce1581fb80 1 -- 172.21.15.62:0/209174733 >> 172.21.15.62:0/209174733 conn(0x55cbd8045030 msgr2=0x55cbd81126b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:28.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fce1581fb80 1 -- 172.21.15.62:0/209174733 shutdown_connections 2024-06-10T00:32:28.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fce1581fb80 1 -- 172.21.15.62:0/209174733 wait complete. 2024-06-10T00:32:28.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.974+0000 7fce1581fb80 20 client.15231 trim_cache size 0 max 16384 2024-06-10T00:32:28.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.996+0000 7fce1581fb80 1 Processor -- start 2024-06-10T00:32:28.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.996+0000 7fce1581fb80 1 -- start start 2024-06-10T00:32:28.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.996+0000 7fce1581fb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd811c620 0x55cbd8121900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:28.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.996+0000 7fce1581fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811cfe0 0x55cbd8112e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:28.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.996+0000 7fce1581fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd8122420 0x55cbd8113370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.996+0000 7fce1581fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cbd8135c60 con 0x55cbd811cfe0 2024-06-10T00:32:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.996+0000 7fce1581fb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cbd8136330 con 0x55cbd8122420 2024-06-10T00:32:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.996+0000 7fce1581fb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbd8136a00 con 0x55cbd811c620 2024-06-10T00:32:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.996+0000 7fce03fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811cfe0 0x55cbd8112e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.996+0000 7fce03fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811cfe0 0x55cbd8112e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45454/0 (socket says 172.21.15.62:45454) 2024-06-10T00:32:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.996+0000 7fce03fff700 1 -- 172.21.15.62:0/3652130665 learned_addr learned my addr 172.21.15.62:0/3652130665 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.997+0000 7fce08d38700 1 --2- 172.21.15.62:0/3652130665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd811c620 0x55cbd8121900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.997+0000 7fce09539700 1 --2- 172.21.15.62:0/3652130665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd8122420 0x55cbd8113370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.997+0000 7fce03fff700 1 -- 172.21.15.62:0/3652130665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd811c620 msgr2=0x55cbd8121900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.997+0000 7fce03fff700 1 --2- 172.21.15.62:0/3652130665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd811c620 0x55cbd8121900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.997+0000 7fce03fff700 1 -- 172.21.15.62:0/3652130665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd8122420 msgr2=0x55cbd8113370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.997+0000 7fce03fff700 1 --2- 172.21.15.62:0/3652130665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd8122420 0x55cbd8113370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.997+0000 7fce03fff700 1 -- 172.21.15.62:0/3652130665 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd811d610 con 0x55cbd811cfe0 2024-06-10T00:32:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.997+0000 7fce03fff700 1 --2- 172.21.15.62:0/3652130665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811cfe0 0x55cbd8112e30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005950 tx=0x7fcdf0004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f9b46c0984372c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.997+0000 7fcdee7fc700 1 -- 172.21.15.62:0/3652130665 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf001b440 con 0x55cbd811cfe0 2024-06-10T00:32:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.997+0000 7fcdee7fc700 1 -- 172.21.15.62:0/3652130665 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdf002d070 con 0x55cbd811cfe0 2024-06-10T00:32:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.997+0000 7fcdee7fc700 1 -- 172.21.15.62:0/3652130665 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0011a40 con 0x55cbd811cfe0 2024-06-10T00:32:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.998+0000 7fce1581fb80 1 -- 172.21.15.62:0/3652130665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811cfe0 msgr2=0x55cbd8112e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.998+0000 7fce1581fb80 1 --2- 172.21.15.62:0/3652130665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811cfe0 0x55cbd8112e30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005950 tx=0x7fcdf0004170 comp rx=0 tx=0).stop 2024-06-10T00:32:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.998+0000 7fce1581fb80 1 -- 172.21.15.62:0/3652130665 shutdown_connections 2024-06-10T00:32:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.998+0000 7fce1581fb80 1 --2- 172.21.15.62:0/3652130665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd8122420 0x55cbd8113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.998+0000 7fce1581fb80 1 --2- 172.21.15.62:0/3652130665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd811c620 0x55cbd8121900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.998+0000 7fce1581fb80 1 --2- 172.21.15.62:0/3652130665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811cfe0 0x55cbd8112e30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.998+0000 7fce1581fb80 1 -- 172.21.15.62:0/3652130665 >> 172.21.15.62:0/3652130665 conn(0x55cbd8045030 msgr2=0x55cbd81206a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.998+0000 7fce1581fb80 1 -- 172.21.15.62:0/3652130665 shutdown_connections 2024-06-10T00:32:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.998+0000 7fce1581fb80 1 -- 172.21.15.62:0/3652130665 wait complete. 2024-06-10T00:32:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.998+0000 7fce1581fb80 1 -- start start 2024-06-10T00:32:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.998+0000 7fce1581fb80 1 Processor -- start 2024-06-10T00:32:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce1581fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811c620 0x55cbd8128380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce1581fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811cfe0 0x55cbd81288c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce1581fb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd8122420 0x55cbd8128e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce1581fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cbd81377a0 con 0x55cbd811c620 2024-06-10T00:32:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce1581fb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cbd81370d0 con 0x55cbd811cfe0 2024-06-10T00:32:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce1581fb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbd8137e70 con 0x55cbd8122420 2024-06-10T00:32:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce1581fb80 10 client.0 _ll_register_callbacks cb 0x55cbd7fff560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:32:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce08d38700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811c620 0x55cbd8128380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce08d38700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811c620 0x55cbd8128380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45456/0 (socket says 172.21.15.62:45456) 2024-06-10T00:32:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce08d38700 1 -- 172.21.15.62:0/1055716131 learned_addr learned my addr 172.21.15.62:0/1055716131 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce03fff700 1 --2- 172.21.15.62:0/1055716131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811cfe0 0x55cbd81288c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:29.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce09539700 1 --2- 172.21.15.62:0/1055716131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd8122420 0x55cbd8128e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:29.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce08d38700 1 -- 172.21.15.62:0/1055716131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd8122420 msgr2=0x55cbd8128e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:29.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce08d38700 1 --2- 172.21.15.62:0/1055716131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd8122420 0x55cbd8128e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:29.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce08d38700 1 -- 172.21.15.62:0/1055716131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811cfe0 msgr2=0x55cbd81288c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:29.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce08d38700 1 --2- 172.21.15.62:0/1055716131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811cfe0 0x55cbd81288c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:29.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce08d38700 1 -- 172.21.15.62:0/1055716131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd813f220 con 0x55cbd811c620 2024-06-10T00:32:29.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce08d38700 1 --2- 172.21.15.62:0/1055716131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811c620 0x55cbd8128380 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800a710 tx=0x7fcdf8005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e22a98b52c9c22a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:29.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fcde3fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:32:29.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fcde3fff700 1 -- 172.21.15.62:0/1055716131 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8016040 con 0x55cbd811c620 2024-06-10T00:32:29.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fce1581fb80 1 -- 172.21.15.62:0/1055716131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf002d070 con 0x55cbd811c620 2024-06-10T00:32:29.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:28.999+0000 7fcde3fff700 1 -- 172.21.15.62:0/1055716131 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fcdf8025670 con 0x55cbd811c620 2024-06-10T00:32:29.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.000+0000 7fcde3fff700 1 -- 172.21.15.62:0/1055716131 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcdf803f050 con 0x55cbd811c620 2024-06-10T00:32:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.000+0000 7fce1581fb80 10 client.15237 fetch_fsmap learned FSMap version 29 2024-06-10T00:32:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.000+0000 7fce1581fb80 1 -- 172.21.15.62:0/1055716131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cbd81a2000 con 0x55cbd811c620 2024-06-10T00:32:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.000+0000 7fcde3fff700 1 -- 172.21.15.62:0/1055716131 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdf002d070 con 0x55cbd811c620 2024-06-10T00:32:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.000+0000 7fcde3fff700 1 -- 172.21.15.62:0/1055716131 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcdf80306b0 con 0x55cbd811c620 2024-06-10T00:32:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.000+0000 7fce1581fb80 10 client.15237 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:32:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.000+0000 7fce1581fb80 10 client.15237 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-10T00:32:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.000+0000 7fce1581fb80 20 client.15237 populate_metadata read hostname 'smithi062' 2024-06-10T00:32:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.000+0000 7fce1581fb80 1 --2- 172.21.15.62:0/1055716131 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55cbd81364b0 0x55cbd81a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.000+0000 7fce1581fb80 4 client.15237 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:32:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.000+0000 7fce1581fb80 1 -- 172.21.15.62:0/1055716131 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "status"}) -- 0x55cbd8071280 con 0x55cbd81364b0 2024-06-10T00:32:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.000+0000 7fce03fff700 1 --2- 172.21.15.62:0/1055716131 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55cbd81364b0 0x55cbd81a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fce03fff700 1 --2- 172.21.15.62:0/1055716131 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55cbd81364b0 0x55cbd81a5660 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9214ec886f189a58 server_cookie=9c6718eaf78fe0e7 in_seq=0 out_seq=0 2024-06-10T00:32:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fcde3fff700 10 client.15237 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:32:29.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fcde3fff700 1 -- 172.21.15.62:0/1055716131 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cbd8071280 con 0x55cbd81364b0 2024-06-10T00:32:29.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fcde3fff700 10 client.15237 handle_command_reply: tid=0 2024-06-10T00:32:29.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fce1581fb80 1 client.15237 shutdown 2024-06-10T00:32:29.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fce1581fb80 1 -- 172.21.15.62:0/1055716131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811c620 msgr2=0x55cbd8128380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:29.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fce1581fb80 1 --2- 172.21.15.62:0/1055716131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811c620 0x55cbd8128380 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800a710 tx=0x7fcdf8005d70 comp rx=0 tx=0).stop 2024-06-10T00:32:29.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fce1581fb80 1 -- 172.21.15.62:0/1055716131 shutdown_connections 2024-06-10T00:32:29.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fce1581fb80 1 --2- 172.21.15.62:0/1055716131 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55cbd81364b0 0x55cbd81a5660 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:29.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fce1581fb80 1 --2- 172.21.15.62:0/1055716131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbd811c620 0x55cbd8128380 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:29.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fce1581fb80 1 --2- 172.21.15.62:0/1055716131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbd8122420 0x55cbd8128e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:29.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fce1581fb80 1 --2- 172.21.15.62:0/1055716131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbd811cfe0 0x55cbd81288c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:29.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fce1581fb80 1 -- 172.21.15.62:0/1055716131 >> 172.21.15.62:0/1055716131 conn(0x55cbd8045030 msgr2=0x55cbd8117200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:29.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fcde3fff700 0 client.15237 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:32:29.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fce1581fb80 1 -- 172.21.15.62:0/1055716131 shutdown_connections 2024-06-10T00:32:29.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fce1581fb80 1 -- 172.21.15.62:0/1055716131 wait complete. 2024-06-10T00:32:29.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.001+0000 7fce1581fb80 20 client.15237 trim_cache size 0 max 16384 2024-06-10T00:32:29.008 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:32:29.008 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:32:29.008 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 4, 2024-06-10T00:32:29.008 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24418, 2024-06-10T00:32:29.008 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:32:29.008 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:32:29.009 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:32:29.009 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 164.12867866100001, 2024-06-10T00:32:29.009 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 29, 2024-06-10T00:32:29.009 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 99, 2024-06-10T00:32:29.009 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T00:32:29.009 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 266.586547521 2024-06-10T00:32:29.009 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:32:29.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.002+0000 7fce0af9c700 1 -- 172.21.15.62:0/3423556966 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcdf40a2c20 msgr2=0x7fcdf40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:29.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.002+0000 7fce0af9c700 1 --2- 172.21.15.62:0/3423556966 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcdf40a2c20 0x7fcdf40a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8005950 tx=0x7fcdf800a300 comp rx=0 tx=0).stop 2024-06-10T00:32:29.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.002+0000 7fce0af9c700 1 -- 172.21.15.62:0/3423556966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce04107860 msgr2=0x7fce04198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:29.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.002+0000 7fce0af9c700 1 --2- 172.21.15.62:0/3423556966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce04107860 0x7fce04198940 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc006070 tx=0x7fcdfc004d00 comp rx=0 tx=0).stop 2024-06-10T00:32:29.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.002+0000 7fce0af9c700 1 -- 172.21.15.62:0/3423556966 shutdown_connections 2024-06-10T00:32:29.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.002+0000 7fce0af9c700 1 --2- 172.21.15.62:0/3423556966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce04107860 0x7fce04198940 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:29.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.002+0000 7fce0af9c700 1 --2- 172.21.15.62:0/3423556966 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fcdf40a2c20 0x7fcdf40a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:29.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.002+0000 7fce0af9c700 1 --2- 172.21.15.62:0/3423556966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce04102770 0x7fce0419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:29.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.002+0000 7fce0af9c700 1 --2- 172.21.15.62:0/3423556966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce04103120 0x7fce0419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:29.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.002+0000 7fce0af9c700 1 -- 172.21.15.62:0/3423556966 >> 172.21.15.62:0/3423556966 conn(0x7fce04079600 msgr2=0x7fce04100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:29.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.002+0000 7fce0af9c700 1 -- 172.21.15.62:0/3423556966 shutdown_connections 2024-06-10T00:32:29.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:29.002+0000 7fce0af9c700 1 -- 172.21.15.62:0/3423556966 wait complete. 2024-06-10T00:32:29.667 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:29 smithi062 ceph-mon[32390]: pgmap v422: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:29.694 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-10T00:32:29.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:29 smithi105 ceph-mon[36321]: pgmap v422: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:29.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:29 smithi138 ceph-mon[36549]: pgmap v422: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:29.944 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:31.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.654+0000 7ff3431a6700 1 -- 172.21.15.62:0/1323485496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff33c103120 msgr2=0x7ff33c107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.654+0000 7ff3431a6700 1 --2- 172.21.15.62:0/1323485496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff33c103120 0x7ff33c107130 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff3240099c0 tx=0x7ff324009cd0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.654+0000 7ff3431a6700 1 -- 172.21.15.62:0/1323485496 shutdown_connections 2024-06-10T00:32:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.654+0000 7ff3431a6700 1 --2- 172.21.15.62:0/1323485496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff33c107860 0x7ff33c10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.654+0000 7ff3431a6700 1 --2- 172.21.15.62:0/1323485496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff33c102770 0x7ff33c102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.654+0000 7ff3431a6700 1 --2- 172.21.15.62:0/1323485496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff33c103120 0x7ff33c107130 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.654+0000 7ff3431a6700 1 -- 172.21.15.62:0/1323485496 >> 172.21.15.62:0/1323485496 conn(0x7ff33c079600 msgr2=0x7ff33c079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:31.656 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:31 smithi062 ceph-mon[32390]: pgmap v423: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:31.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.654+0000 7ff3431a6700 1 -- 172.21.15.62:0/1323485496 shutdown_connections 2024-06-10T00:32:31.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.654+0000 7ff3431a6700 1 -- 172.21.15.62:0/1323485496 wait complete. 2024-06-10T00:32:31.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.655+0000 7ff3431a6700 1 Processor -- start 2024-06-10T00:32:31.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.655+0000 7ff3431a6700 1 -- start start 2024-06-10T00:32:31.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.655+0000 7ff3431a6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff33c102770 0x7ff33c19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.655+0000 7ff3431a6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff33c103120 0x7ff33c19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.655+0000 7ff3431a6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff33c107860 0x7ff33c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.655+0000 7ff3431a6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c112770 con 0x7ff33c107860 2024-06-10T00:32:31.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.655+0000 7ff3431a6700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1125f0 con 0x7ff33c103120 2024-06-10T00:32:31.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.655+0000 7ff3431a6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1128f0 con 0x7ff33c102770 2024-06-10T00:32:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.655+0000 7ff341743700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff33c107860 0x7ff33c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.655+0000 7ff341743700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff33c107860 0x7ff33c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45468/0 (socket says 172.21.15.62:45468) 2024-06-10T00:32:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.655+0000 7ff341743700 1 -- 172.21.15.62:0/625891692 learned_addr learned my addr 172.21.15.62:0/625891692 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.655+0000 7ff340f42700 1 --2- 172.21.15.62:0/625891692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff33c102770 0x7ff33c19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.655+0000 7ff33bfff700 1 --2- 172.21.15.62:0/625891692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff33c103120 0x7ff33c19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.656+0000 7ff341743700 1 -- 172.21.15.62:0/625891692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff33c102770 msgr2=0x7ff33c19e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.656+0000 7ff341743700 1 --2- 172.21.15.62:0/625891692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff33c102770 0x7ff33c19e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.656+0000 7ff341743700 1 -- 172.21.15.62:0/625891692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff33c103120 msgr2=0x7ff33c19ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.656+0000 7ff341743700 1 --2- 172.21.15.62:0/625891692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff33c103120 0x7ff33c19ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.656+0000 7ff341743700 1 -- 172.21.15.62:0/625891692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c199090 con 0x7ff33c107860 2024-06-10T00:32:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.656+0000 7ff341743700 1 --2- 172.21.15.62:0/625891692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff33c107860 0x7ff33c198930 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff33400c900 tx=0x7ff33400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:31.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.656+0000 7ff339ffb700 1 -- 172.21.15.62:0/625891692 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3340078d0 con 0x7ff33c107860 2024-06-10T00:32:31.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.656+0000 7ff339ffb700 1 -- 172.21.15.62:0/625891692 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff33400f450 con 0x7ff33c107860 2024-06-10T00:32:31.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.656+0000 7ff3431a6700 1 -- 172.21.15.62:0/625891692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33c199380 con 0x7ff33c107860 2024-06-10T00:32:31.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.656+0000 7ff339ffb700 1 -- 172.21.15.62:0/625891692 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334018660 con 0x7ff33c107860 2024-06-10T00:32:31.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.656+0000 7ff3431a6700 1 -- 172.21.15.62:0/625891692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33c1a5640 con 0x7ff33c107860 2024-06-10T00:32:31.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.658+0000 7ff339ffb700 1 -- 172.21.15.62:0/625891692 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff334007a70 con 0x7ff33c107860 2024-06-10T00:32:31.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.659+0000 7ff339ffb700 1 --2- 172.21.15.62:0/625891692 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff3280a2c20 0x7ff3280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.659+0000 7ff340f42700 1 --2- 172.21.15.62:0/625891692 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff3280a2c20 0x7ff3280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.659+0000 7ff339ffb700 1 -- 172.21.15.62:0/625891692 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7ff3340cf230 con 0x7ff33c107860 2024-06-10T00:32:31.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.660+0000 7ff340f42700 1 --2- 172.21.15.62:0/625891692 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff3280a2c20 0x7ff3280a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0060b0 tx=0x7ff32c00c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:31.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.660+0000 7ff34da29b80 1 Processor -- start 2024-06-10T00:32:31.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.660+0000 7ff34da29b80 1 -- start start 2024-06-10T00:32:31.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.660+0000 7ff34da29b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 0x562aac2bf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.660+0000 7ff34da29b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 0x562aac2b9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.660+0000 7ff34da29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2b9b60 0x562aac2b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.660+0000 7ff34da29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562aac1a7f10 con 0x562aac2b9b60 2024-06-10T00:32:31.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.660+0000 7ff34da29b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562aac206800 con 0x562aac2bf320 2024-06-10T00:32:31.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.660+0000 7ff34da29b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562aac16e410 con 0x562aac2bfc40 2024-06-10T00:32:31.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.661+0000 7ff341743700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2b9b60 0x562aac2b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.661+0000 7ff341743700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2b9b60 0x562aac2b9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45470/0 (socket says 172.21.15.62:45470) 2024-06-10T00:32:31.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.661+0000 7ff341743700 1 -- 172.21.15.62:0/1870139799 learned_addr learned my addr 172.21.15.62:0/1870139799 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.661+0000 7ff340f42700 1 --2- 172.21.15.62:0/1870139799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 0x562aac2bf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.661+0000 7ff33bfff700 1 --2- 172.21.15.62:0/1870139799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 0x562aac2b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.661+0000 7ff341743700 1 -- 172.21.15.62:0/1870139799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 msgr2=0x562aac2b9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.661+0000 7ff341743700 1 --2- 172.21.15.62:0/1870139799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 0x562aac2b9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.661+0000 7ff341743700 1 -- 172.21.15.62:0/1870139799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 msgr2=0x562aac2bf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.661+0000 7ff341743700 1 --2- 172.21.15.62:0/1870139799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 0x562aac2bf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.661+0000 7ff341743700 1 -- 172.21.15.62:0/1870139799 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aac155880 con 0x562aac2b9b60 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.661+0000 7ff341743700 1 --2- 172.21.15.62:0/1870139799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2b9b60 0x562aac2b9fa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff33409b650 tx=0x7ff33409b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd2149d73485962 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.661+0000 7ff332ffd700 1 -- 172.21.15.62:0/1870139799 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3340b4070 con 0x562aac2b9b60 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.661+0000 7ff332ffd700 1 -- 172.21.15.62:0/1870139799 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff33409b6c0 con 0x562aac2b9b60 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.661+0000 7ff332ffd700 1 -- 172.21.15.62:0/1870139799 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33401fdd0 con 0x562aac2b9b60 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.662+0000 7ff34da29b80 1 -- 172.21.15.62:0/1870139799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2b9b60 msgr2=0x562aac2b9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.662+0000 7ff34da29b80 1 --2- 172.21.15.62:0/1870139799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2b9b60 0x562aac2b9fa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff33409b650 tx=0x7ff33409b2d0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.662+0000 7ff34da29b80 1 -- 172.21.15.62:0/1870139799 shutdown_connections 2024-06-10T00:32:31.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.662+0000 7ff34da29b80 1 --2- 172.21.15.62:0/1870139799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2b9b60 0x562aac2b9fa0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.662+0000 7ff34da29b80 1 --2- 172.21.15.62:0/1870139799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 0x562aac2bf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.662+0000 7ff34da29b80 1 --2- 172.21.15.62:0/1870139799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 0x562aac2b9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.662+0000 7ff34da29b80 1 -- 172.21.15.62:0/1870139799 >> 172.21.15.62:0/1870139799 conn(0x562aac1e2030 msgr2=0x562aac2b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:31.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.662+0000 7ff34da29b80 1 -- 172.21.15.62:0/1870139799 shutdown_connections 2024-06-10T00:32:31.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.662+0000 7ff34da29b80 1 -- 172.21.15.62:0/1870139799 wait complete. 2024-06-10T00:32:31.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.662+0000 7ff34da29b80 1 -- start start 2024-06-10T00:32:31.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff34da29b80 1 Processor -- start 2024-06-10T00:32:31.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff34da29b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2b9b60 0x562aac2c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff34da29b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 0x562aac2c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff34da29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2bfc40 0x562aac2c3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff34da29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562aac1a6f70 con 0x562aac2bfc40 2024-06-10T00:32:31.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff34da29b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562aac2027c0 con 0x562aac2bf320 2024-06-10T00:32:31.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff34da29b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562aac2c6630 con 0x562aac2b9b60 2024-06-10T00:32:31.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff34da29b80 10 client.0 _ll_register_callbacks cb 0x562aac19c560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:32:31.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff341743700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2bfc40 0x562aac2c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff341743700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2bfc40 0x562aac2c3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45476/0 (socket says 172.21.15.62:45476) 2024-06-10T00:32:31.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff341743700 1 -- 172.21.15.62:0/2231385512 learned_addr learned my addr 172.21.15.62:0/2231385512 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:31.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff33bfff700 1 --2- 172.21.15.62:0/2231385512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 0x562aac2c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff340f42700 1 --2- 172.21.15.62:0/2231385512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2b9b60 0x562aac2c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff341743700 1 -- 172.21.15.62:0/2231385512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2b9b60 msgr2=0x562aac2c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:31.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff341743700 1 --2- 172.21.15.62:0/2231385512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2b9b60 0x562aac2c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.663+0000 7ff341743700 1 -- 172.21.15.62:0/2231385512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 msgr2=0x562aac2c2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:31.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.664+0000 7ff341743700 1 --2- 172.21.15.62:0/2231385512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 0x562aac2c2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.664+0000 7ff341743700 1 -- 172.21.15.62:0/2231385512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aac1cc700 con 0x562aac2bfc40 2024-06-10T00:32:31.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.664+0000 7ff341743700 1 --2- 172.21.15.62:0/2231385512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2bfc40 0x562aac2c3230 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff3340cccd0 tx=0x7ff33400a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0ff46be4b1b5cc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:31.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.664+0000 7ff31bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:32:31.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.664+0000 7ff31bfff700 1 -- 172.21.15.62:0/2231385512 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3340b9070 con 0x562aac2bfc40 2024-06-10T00:32:31.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.664+0000 7ff31bfff700 1 -- 172.21.15.62:0/2231385512 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7ff3340999c0 con 0x562aac2bfc40 2024-06-10T00:32:31.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.664+0000 7ff34da29b80 1 -- 172.21.15.62:0/2231385512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff33409b6c0 con 0x562aac2bfc40 2024-06-10T00:32:31.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.664+0000 7ff31bfff700 1 -- 172.21.15.62:0/2231385512 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff3340b4040 con 0x562aac2bfc40 2024-06-10T00:32:31.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.664+0000 7ff34da29b80 10 client.15249 fetch_fsmap learned FSMap version 29 2024-06-10T00:32:31.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.664+0000 7ff34da29b80 1 -- 172.21.15.62:0/2231385512 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562aac2d0e30 con 0x562aac2bfc40 2024-06-10T00:32:31.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.664+0000 7ff31bfff700 1 -- 172.21.15.62:0/2231385512 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff33409b6c0 con 0x562aac2bfc40 2024-06-10T00:32:31.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.664+0000 7ff31bfff700 1 -- 172.21.15.62:0/2231385512 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff334020030 con 0x562aac2bfc40 2024-06-10T00:32:31.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.664+0000 7ff34da29b80 10 client.15249 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:32:31.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.665+0000 7ff34da29b80 10 client.15249 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T00:32:31.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.665+0000 7ff34da29b80 20 client.15249 populate_metadata read hostname 'smithi062' 2024-06-10T00:32:31.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.665+0000 7ff34da29b80 1 --2- 172.21.15.62:0/2231385512 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x562aac2d2180 0x562aac2d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.665+0000 7ff34da29b80 4 client.15249 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:32:31.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.665+0000 7ff34da29b80 1 -- 172.21.15.62:0/2231385512 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aac213d00 con 0x562aac2d2180 2024-06-10T00:32:31.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.665+0000 7ff340f42700 1 --2- 172.21.15.62:0/2231385512 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x562aac2d2180 0x562aac2d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.665+0000 7ff340f42700 1 --2- 172.21.15.62:0/2231385512 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x562aac2d2180 0x562aac2d4260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d57ec61296042454 server_cookie=64bb2d236635a4af in_seq=0 out_seq=0 2024-06-10T00:32:31.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.665+0000 7ff31bfff700 10 client.15249 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:32:31.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.666+0000 7ff31bfff700 1 -- 172.21.15.62:0/2231385512 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562aac213d00 con 0x562aac2d2180 2024-06-10T00:32:31.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.666+0000 7ff31bfff700 10 client.15249 handle_command_reply: tid=0 2024-06-10T00:32:31.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.666+0000 7ff34da29b80 1 client.15249 shutdown 2024-06-10T00:32:31.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.666+0000 7ff34da29b80 1 -- 172.21.15.62:0/2231385512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2bfc40 msgr2=0x562aac2c3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:31.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.666+0000 7ff34da29b80 1 --2- 172.21.15.62:0/2231385512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2bfc40 0x562aac2c3230 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff3340cccd0 tx=0x7ff33400a5e0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.666+0000 7ff34da29b80 1 -- 172.21.15.62:0/2231385512 shutdown_connections 2024-06-10T00:32:31.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.666+0000 7ff34da29b80 1 --2- 172.21.15.62:0/2231385512 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x562aac2d2180 0x562aac2d4260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.666+0000 7ff31bfff700 0 client.15249 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:32:31.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.666+0000 7ff34da29b80 1 --2- 172.21.15.62:0/2231385512 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2bfc40 0x562aac2c3230 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.666+0000 7ff34da29b80 1 --2- 172.21.15.62:0/2231385512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2b9b60 0x562aac2c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.666+0000 7ff34da29b80 1 --2- 172.21.15.62:0/2231385512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 0x562aac2c2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.666+0000 7ff34da29b80 1 -- 172.21.15.62:0/2231385512 >> 172.21.15.62:0/2231385512 conn(0x562aac1e2030 msgr2=0x562aac2af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:31.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.667+0000 7ff34da29b80 1 -- 172.21.15.62:0/2231385512 shutdown_connections 2024-06-10T00:32:31.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.667+0000 7ff34da29b80 1 -- 172.21.15.62:0/2231385512 wait complete. 2024-06-10T00:32:31.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.667+0000 7ff34da29b80 20 client.15249 trim_cache size 0 max 16384 2024-06-10T00:32:31.675 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:31 smithi138 ceph-mon[36549]: pgmap v423: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:31.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.688+0000 7ff34da29b80 1 Processor -- start 2024-06-10T00:32:31.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.688+0000 7ff34da29b80 1 -- start start 2024-06-10T00:32:31.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.688+0000 7ff34da29b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2b9b60 0x562aac2b9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.688+0000 7ff34da29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2bf320 0x562aac2c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.688+0000 7ff34da29b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 0x562aac2b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.688+0000 7ff34da29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562aac2d25a0 con 0x562aac2bf320 2024-06-10T00:32:31.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.688+0000 7ff34da29b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562aac2d2c70 con 0x562aac2b9b60 2024-06-10T00:32:31.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.688+0000 7ff34da29b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562aac2d3340 con 0x562aac2bfc40 2024-06-10T00:32:31.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.688+0000 7ff33bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2bf320 0x562aac2c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.688+0000 7ff33bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2bf320 0x562aac2c08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45486/0 (socket says 172.21.15.62:45486) 2024-06-10T00:32:31.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.688+0000 7ff33bfff700 1 -- 172.21.15.62:0/4229123648 learned_addr learned my addr 172.21.15.62:0/4229123648 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:31.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.688+0000 7ff340f42700 1 --2- 172.21.15.62:0/4229123648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2b9b60 0x562aac2b9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.689+0000 7ff33bfff700 1 -- 172.21.15.62:0/4229123648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 msgr2=0x562aac2b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:31.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.689+0000 7ff33bfff700 1 --2- 172.21.15.62:0/4229123648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 0x562aac2b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.689+0000 7ff33bfff700 1 -- 172.21.15.62:0/4229123648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2b9b60 msgr2=0x562aac2b9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:31.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.689+0000 7ff33bfff700 1 --2- 172.21.15.62:0/4229123648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2b9b60 0x562aac2b9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.689+0000 7ff33bfff700 1 -- 172.21.15.62:0/4229123648 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aac2bd760 con 0x562aac2bf320 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.689+0000 7ff33bfff700 1 --2- 172.21.15.62:0/4229123648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2bf320 0x562aac2c08b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff3240099c0 tx=0x7ff324005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143d4e6438d66cf7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.689+0000 7ff3327fc700 1 -- 172.21.15.62:0/4229123648 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324003c20 con 0x562aac2bf320 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.689+0000 7ff3327fc700 1 -- 172.21.15.62:0/4229123648 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff324003dc0 con 0x562aac2bf320 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.689+0000 7ff3327fc700 1 -- 172.21.15.62:0/4229123648 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32400f460 con 0x562aac2bf320 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.690+0000 7ff34da29b80 1 -- 172.21.15.62:0/4229123648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2bf320 msgr2=0x562aac2c08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.690+0000 7ff34da29b80 1 --2- 172.21.15.62:0/4229123648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2bf320 0x562aac2c08b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff3240099c0 tx=0x7ff324005040 comp rx=0 tx=0).stop 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.690+0000 7ff34da29b80 1 -- 172.21.15.62:0/4229123648 shutdown_connections 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.690+0000 7ff34da29b80 1 --2- 172.21.15.62:0/4229123648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 0x562aac2b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.690+0000 7ff34da29b80 1 --2- 172.21.15.62:0/4229123648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2b9b60 0x562aac2b9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.690+0000 7ff34da29b80 1 --2- 172.21.15.62:0/4229123648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2bf320 0x562aac2c08b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.690+0000 7ff34da29b80 1 -- 172.21.15.62:0/4229123648 >> 172.21.15.62:0/4229123648 conn(0x562aac1e2030 msgr2=0x562aac2b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.690+0000 7ff34da29b80 1 -- 172.21.15.62:0/4229123648 shutdown_connections 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.690+0000 7ff34da29b80 1 -- 172.21.15.62:0/4229123648 wait complete. 2024-06-10T00:32:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.690+0000 7ff34da29b80 1 -- start start 2024-06-10T00:32:31.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.690+0000 7ff34da29b80 1 Processor -- start 2024-06-10T00:32:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff34da29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2b9b60 0x562aac2d9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff34da29b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 0x562aac2da110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff34da29b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 0x562aac33d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff34da29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562aac2d40e0 con 0x562aac2b9b60 2024-06-10T00:32:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff34da29b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562aac2d3a10 con 0x562aac2bf320 2024-06-10T00:32:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff34da29b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562aac2d47b0 con 0x562aac2bfc40 2024-06-10T00:32:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff34da29b80 10 client.0 _ll_register_callbacks cb 0x562aac19c560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:32:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff340f42700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2b9b60 0x562aac2d9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff340f42700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2b9b60 0x562aac2d9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45500/0 (socket says 172.21.15.62:45500) 2024-06-10T00:32:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff340f42700 1 -- 172.21.15.62:0/774032638 learned_addr learned my addr 172.21.15.62:0/774032638 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff33bfff700 1 --2- 172.21.15.62:0/774032638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 0x562aac2da110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff341743700 1 --2- 172.21.15.62:0/774032638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 0x562aac33d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff340f42700 1 -- 172.21.15.62:0/774032638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 msgr2=0x562aac33d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff340f42700 1 --2- 172.21.15.62:0/774032638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 0x562aac33d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff340f42700 1 -- 172.21.15.62:0/774032638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 msgr2=0x562aac2da110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff340f42700 1 --2- 172.21.15.62:0/774032638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 0x562aac2da110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff340f42700 1 -- 172.21.15.62:0/774032638 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aac33dc40 con 0x562aac2b9b60 2024-06-10T00:32:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff341743700 1 --2- 172.21.15.62:0/774032638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 0x562aac33d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:32:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff340f42700 1 --2- 172.21.15.62:0/774032638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2b9b60 0x562aac2d9bd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x562aac2d4a90 tx=0x7ff32c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e173388724f4be server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff31bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:32:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff31bfff700 1 -- 172.21.15.62:0/774032638 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c02e470 con 0x562aac2b9b60 2024-06-10T00:32:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff34da29b80 1 -- 172.21.15.62:0/774032638 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff324003dc0 con 0x562aac2b9b60 2024-06-10T00:32:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.691+0000 7ff31bfff700 1 -- 172.21.15.62:0/774032638 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7ff32c02e610 con 0x562aac2b9b60 2024-06-10T00:32:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.692+0000 7ff31bfff700 1 -- 172.21.15.62:0/774032638 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff32c03c050 con 0x562aac2b9b60 2024-06-10T00:32:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.692+0000 7ff34da29b80 10 client.15255 fetch_fsmap learned FSMap version 29 2024-06-10T00:32:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.692+0000 7ff34da29b80 1 -- 172.21.15.62:0/774032638 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562aac33ef80 con 0x562aac2b9b60 2024-06-10T00:32:31.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.692+0000 7ff31bfff700 1 -- 172.21.15.62:0/774032638 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff324003dc0 con 0x562aac2b9b60 2024-06-10T00:32:31.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.692+0000 7ff31bfff700 1 -- 172.21.15.62:0/774032638 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff32c025860 con 0x562aac2b9b60 2024-06-10T00:32:31.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.692+0000 7ff34da29b80 10 client.15255 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:32:31.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.692+0000 7ff34da29b80 10 client.15255 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T00:32:31.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.692+0000 7ff34da29b80 20 client.15255 populate_metadata read hostname 'smithi062' 2024-06-10T00:32:31.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.692+0000 7ff34da29b80 1 --2- 172.21.15.62:0/774032638 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x562aac343c30 0x562aac342940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:31.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.692+0000 7ff34da29b80 4 client.15255 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:32:31.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.692+0000 7ff34da29b80 1 -- 172.21.15.62:0/774032638 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "status"}) -- 0x562aac20e280 con 0x562aac343c30 2024-06-10T00:32:31.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.692+0000 7ff33bfff700 1 --2- 172.21.15.62:0/774032638 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x562aac343c30 0x562aac342940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:31.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.693+0000 7ff31bfff700 10 client.15255 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:32:31.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.693+0000 7ff33bfff700 1 --2- 172.21.15.62:0/774032638 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x562aac343c30 0x562aac342940 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca491676dfa2c362 server_cookie=86e9e442af6c58a7 in_seq=0 out_seq=0 2024-06-10T00:32:31.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.693+0000 7ff31bfff700 1 -- 172.21.15.62:0/774032638 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562aac20e280 con 0x562aac343c30 2024-06-10T00:32:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.693+0000 7ff31bfff700 10 client.15255 handle_command_reply: tid=0 2024-06-10T00:32:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.693+0000 7ff34da29b80 1 client.15255 shutdown 2024-06-10T00:32:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.693+0000 7ff34da29b80 1 -- 172.21.15.62:0/774032638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2b9b60 msgr2=0x562aac2d9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.693+0000 7ff34da29b80 1 --2- 172.21.15.62:0/774032638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2b9b60 0x562aac2d9bd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x562aac2d4a90 tx=0x7ff32c004ad0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.693+0000 7ff34da29b80 1 -- 172.21.15.62:0/774032638 shutdown_connections 2024-06-10T00:32:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.693+0000 7ff34da29b80 1 --2- 172.21.15.62:0/774032638 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x562aac343c30 0x562aac342940 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.693+0000 7ff34da29b80 1 --2- 172.21.15.62:0/774032638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562aac2b9b60 0x562aac2d9bd0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.693+0000 7ff34da29b80 1 --2- 172.21.15.62:0/774032638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562aac2bfc40 0x562aac33d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.693+0000 7ff34da29b80 1 --2- 172.21.15.62:0/774032638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562aac2bf320 0x562aac2da110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.693+0000 7ff34da29b80 1 -- 172.21.15.62:0/774032638 >> 172.21.15.62:0/774032638 conn(0x562aac1e2030 msgr2=0x562aac2b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.693+0000 7ff31bfff700 0 client.15255 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:32:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff34da29b80 1 -- 172.21.15.62:0/774032638 shutdown_connections 2024-06-10T00:32:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff34da29b80 1 -- 172.21.15.62:0/774032638 wait complete. 2024-06-10T00:32:31.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff34da29b80 20 client.15255 trim_cache size 0 max 16384 2024-06-10T00:32:31.700 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:32:31.700 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:32:31.700 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 3, 2024-06-10T00:32:31.700 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24424, 2024-06-10T00:32:31.700 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:32:31.700 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:32:31.700 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:32:31.701 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 167.830921821, 2024-06-10T00:32:31.701 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 29, 2024-06-10T00:32:31.701 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 99, 2024-06-10T00:32:31.701 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T00:32:31.701 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 266.34710275899999 2024-06-10T00:32:31.701 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:32:31.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff3431a6700 1 -- 172.21.15.62:0/625891692 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff3280a2c20 msgr2=0x7ff3280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:31.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff3431a6700 1 --2- 172.21.15.62:0/625891692 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff3280a2c20 0x7ff3280a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0060b0 tx=0x7ff32c00c040 comp rx=0 tx=0).stop 2024-06-10T00:32:31.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff3431a6700 1 -- 172.21.15.62:0/625891692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff33c107860 msgr2=0x7ff33c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:31.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff3431a6700 1 --2- 172.21.15.62:0/625891692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff33c107860 0x7ff33c198930 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff33400c900 tx=0x7ff33400cc10 comp rx=0 tx=0).stop 2024-06-10T00:32:31.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff3431a6700 1 -- 172.21.15.62:0/625891692 shutdown_connections 2024-06-10T00:32:31.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff3431a6700 1 --2- 172.21.15.62:0/625891692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff33c107860 0x7ff33c198930 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff3431a6700 1 --2- 172.21.15.62:0/625891692 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7ff3280a2c20 0x7ff3280a50e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff3431a6700 1 --2- 172.21.15.62:0/625891692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff33c102770 0x7ff33c19e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff3431a6700 1 --2- 172.21.15.62:0/625891692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff33c103120 0x7ff33c19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:31.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff3431a6700 1 -- 172.21.15.62:0/625891692 >> 172.21.15.62:0/625891692 conn(0x7ff33c079600 msgr2=0x7ff33c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:31.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff3431a6700 1 -- 172.21.15.62:0/625891692 shutdown_connections 2024-06-10T00:32:31.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:31.694+0000 7ff3431a6700 1 -- 172.21.15.62:0/625891692 wait complete. 2024-06-10T00:32:31.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:31 smithi105 ceph-mon[36321]: pgmap v423: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:32.546 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-10T00:32:32.803 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:33.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:33 smithi062 ceph-mon[32390]: pgmap v424: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:32:33.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:33 smithi105 ceph-mon[36321]: pgmap v424: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:32:33.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:33 smithi138 ceph-mon[36549]: pgmap v424: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-10T00:32:34.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.471+0000 7f628dfc2700 1 -- 172.21.15.62:0/2061658723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6288107550 msgr2=0x7f6288107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:34.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.471+0000 7f628dfc2700 1 --2- 172.21.15.62:0/2061658723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6288107550 0x7f6288107930 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6278009990 tx=0x7f6278009ca0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.471+0000 7f628dfc2700 1 -- 172.21.15.62:0/2061658723 shutdown_connections 2024-06-10T00:32:34.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.471+0000 7f628dfc2700 1 --2- 172.21.15.62:0/2061658723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6288107550 0x7f6288107930 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.471+0000 7f628dfc2700 1 --2- 172.21.15.62:0/2061658723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62881017f0 0x7f628810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.471+0000 7f628dfc2700 1 --2- 172.21.15.62:0/2061658723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6288100e30 0x7f62881012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.471+0000 7f628dfc2700 1 -- 172.21.15.62:0/2061658723 >> 172.21.15.62:0/2061658723 conn(0x7f62880766b0 msgr2=0x7f6288076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.471+0000 7f628dfc2700 1 -- 172.21.15.62:0/2061658723 shutdown_connections 2024-06-10T00:32:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.471+0000 7f628dfc2700 1 -- 172.21.15.62:0/2061658723 wait complete. 2024-06-10T00:32:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.472+0000 7f628dfc2700 1 Processor -- start 2024-06-10T00:32:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.472+0000 7f628dfc2700 1 -- start start 2024-06-10T00:32:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.472+0000 7f628dfc2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6288100e30 0x7f6288102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.472+0000 7f628dfc2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62881017f0 0x7f6288102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.472+0000 7f628dfc2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6288107550 0x7f6288102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.472+0000 7f628dfc2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6288112890 con 0x7f6288107550 2024-06-10T00:32:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.472+0000 7f628dfc2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6288112710 con 0x7f62881017f0 2024-06-10T00:32:34.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.472+0000 7f628dfc2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6288112a10 con 0x7f6288100e30 2024-06-10T00:32:34.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.472+0000 7f6287fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6288107550 0x7f6288102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.472+0000 7f6287fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6288107550 0x7f6288102a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45516/0 (socket says 172.21.15.62:45516) 2024-06-10T00:32:34.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.472+0000 7f6287fff700 1 -- 172.21.15.62:0/2442558896 learned_addr learned my addr 172.21.15.62:0/2442558896 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:34.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.472+0000 7f62877fe700 1 --2- 172.21.15.62:0/2442558896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6288100e30 0x7f6288102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.472+0000 7f6286ffd700 1 --2- 172.21.15.62:0/2442558896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62881017f0 0x7f6288102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.473+0000 7f6287fff700 1 -- 172.21.15.62:0/2442558896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6288100e30 msgr2=0x7f6288102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:34.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.473+0000 7f6287fff700 1 --2- 172.21.15.62:0/2442558896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6288100e30 0x7f6288102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.473+0000 7f6287fff700 1 -- 172.21.15.62:0/2442558896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62881017f0 msgr2=0x7f6288102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:34.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.473+0000 7f6287fff700 1 --2- 172.21.15.62:0/2442558896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62881017f0 0x7f6288102550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.473+0000 7f6287fff700 1 -- 172.21.15.62:0/2442558896 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62881a72b0 con 0x7f6288107550 2024-06-10T00:32:34.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.473+0000 7f6287fff700 1 --2- 172.21.15.62:0/2442558896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6288107550 0x7f6288102a90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f628000b970 tx=0x7f628000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:34.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.473+0000 7f6284ff9700 1 -- 172.21.15.62:0/2442558896 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6280019070 con 0x7f6288107550 2024-06-10T00:32:34.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.473+0000 7f6284ff9700 1 -- 172.21.15.62:0/2442558896 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6280014440 con 0x7f6288107550 2024-06-10T00:32:34.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.473+0000 7f628dfc2700 1 -- 172.21.15.62:0/2442558896 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62881a75a0 con 0x7f6288107550 2024-06-10T00:32:34.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.473+0000 7f6284ff9700 1 -- 172.21.15.62:0/2442558896 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800075d0 con 0x7f6288107550 2024-06-10T00:32:34.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.473+0000 7f628dfc2700 1 -- 172.21.15.62:0/2442558896 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62881a7a90 con 0x7f6288107550 2024-06-10T00:32:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.475+0000 7f6284ff9700 1 -- 172.21.15.62:0/2442558896 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f628001c070 con 0x7f6288107550 2024-06-10T00:32:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.476+0000 7f6284ff9700 1 --2- 172.21.15.62:0/2442558896 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f62700a2c20 0x7f62700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.476+0000 7f62877fe700 1 --2- 172.21.15.62:0/2442558896 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f62700a2c20 0x7f62700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.476+0000 7f6284ff9700 1 -- 172.21.15.62:0/2442558896 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f62800ce8a0 con 0x7f6288107550 2024-06-10T00:32:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.477+0000 7f62877fe700 1 --2- 172.21.15.62:0/2442558896 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f62700a2c20 0x7f62700a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6278009750 tx=0x7f627800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.477+0000 7f6298845b80 1 Processor -- start 2024-06-10T00:32:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.477+0000 7f6298845b80 1 -- start start 2024-06-10T00:32:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.477+0000 7f6298845b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb3829d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.477+0000 7f6298845b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb382f10 0x5647bb37c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.477+0000 7f6298845b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb37ced0 0x5647bb37d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.477+0000 7f6298845b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5647bb2ce7c0 con 0x5647bb37ced0 2024-06-10T00:32:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.477+0000 7f6298845b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5647bb2766d0 con 0x5647bb382f10 2024-06-10T00:32:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.477+0000 7f6298845b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5647bb2ca780 con 0x5647bb3825f0 2024-06-10T00:32:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.477+0000 7f6287fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb37ced0 0x5647bb37d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.477+0000 7f6287fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb37ced0 0x5647bb37d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45528/0 (socket says 172.21.15.62:45528) 2024-06-10T00:32:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.477+0000 7f6287fff700 1 -- 172.21.15.62:0/2670800387 learned_addr learned my addr 172.21.15.62:0/2670800387 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.478+0000 7f62877fe700 1 --2- 172.21.15.62:0/2670800387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb3829d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.478+0000 7f6286ffd700 1 --2- 172.21.15.62:0/2670800387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb382f10 0x5647bb37c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.478+0000 7f6287fff700 1 -- 172.21.15.62:0/2670800387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 msgr2=0x5647bb3829d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:34.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.478+0000 7f6287fff700 1 --2- 172.21.15.62:0/2670800387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb3829d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.478+0000 7f6287fff700 1 -- 172.21.15.62:0/2670800387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb382f10 msgr2=0x5647bb37c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:34.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.478+0000 7f6287fff700 1 --2- 172.21.15.62:0/2670800387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb382f10 0x5647bb37c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.478+0000 7f6287fff700 1 -- 172.21.15.62:0/2670800387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647bb26cf00 con 0x5647bb37ced0 2024-06-10T00:32:34.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.478+0000 7f6287fff700 1 --2- 172.21.15.62:0/2670800387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb37ced0 0x5647bb37d2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6280097700 tx=0x7f6280004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d319df4344e725 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:34.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.478+0000 7f626dffb700 1 -- 172.21.15.62:0/2670800387 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800a39a0 con 0x5647bb37ced0 2024-06-10T00:32:34.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.478+0000 7f626dffb700 1 -- 172.21.15.62:0/2670800387 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6280099030 con 0x5647bb37ced0 2024-06-10T00:32:34.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.478+0000 7f626dffb700 1 -- 172.21.15.62:0/2670800387 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800a37a0 con 0x5647bb37ced0 2024-06-10T00:32:34.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.479+0000 7f6298845b80 1 -- 172.21.15.62:0/2670800387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb37ced0 msgr2=0x5647bb37d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:34.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.479+0000 7f6298845b80 1 --2- 172.21.15.62:0/2670800387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb37ced0 0x5647bb37d2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6280097700 tx=0x7f6280004be0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.479+0000 7f6298845b80 1 -- 172.21.15.62:0/2670800387 shutdown_connections 2024-06-10T00:32:34.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.479+0000 7f6298845b80 1 --2- 172.21.15.62:0/2670800387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb37ced0 0x5647bb37d2f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.479+0000 7f6298845b80 1 --2- 172.21.15.62:0/2670800387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb3829d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.479+0000 7f6298845b80 1 --2- 172.21.15.62:0/2670800387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb382f10 0x5647bb37c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.479+0000 7f6298845b80 1 -- 172.21.15.62:0/2670800387 >> 172.21.15.62:0/2670800387 conn(0x5647bb2a7c70 msgr2=0x5647bb377800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:34.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.479+0000 7f6298845b80 1 -- 172.21.15.62:0/2670800387 shutdown_connections 2024-06-10T00:32:34.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.479+0000 7f6298845b80 1 -- 172.21.15.62:0/2670800387 wait complete. 2024-06-10T00:32:34.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.479+0000 7f6298845b80 1 -- start start 2024-06-10T00:32:34.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6298845b80 1 Processor -- start 2024-06-10T00:32:34.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6298845b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 0x5647bb388c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6298845b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb385f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6298845b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb3864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6298845b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5647bb389980 con 0x5647bb382f10 2024-06-10T00:32:34.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6298845b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5647bb26eba0 con 0x5647bb37ced0 2024-06-10T00:32:34.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6298845b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5647bb389b00 con 0x5647bb3825f0 2024-06-10T00:32:34.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6298845b80 10 client.0 _ll_register_callbacks cb 0x5647bb2688d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:32:34.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6287fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb3864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6287fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb3864b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45530/0 (socket says 172.21.15.62:45530) 2024-06-10T00:32:34.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6287fff700 1 -- 172.21.15.62:0/564875651 learned_addr learned my addr 172.21.15.62:0/564875651 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:34.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f62877fe700 1 --2- 172.21.15.62:0/564875651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 0x5647bb388c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6286ffd700 1 --2- 172.21.15.62:0/564875651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb385f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6287fff700 1 -- 172.21.15.62:0/564875651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 msgr2=0x5647bb385f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:34.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6287fff700 1 --2- 172.21.15.62:0/564875651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb385f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6287fff700 1 -- 172.21.15.62:0/564875651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 msgr2=0x5647bb388c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:34.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6287fff700 1 --2- 172.21.15.62:0/564875651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 0x5647bb388c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.480+0000 7f6287fff700 1 -- 172.21.15.62:0/564875651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647bb21d880 con 0x5647bb382f10 2024-06-10T00:32:34.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6287fff700 1 --2- 172.21.15.62:0/564875651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb3864b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6280098fe0 tx=0x7f62800cbf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936d7e56254ee9a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:34.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6266ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:32:34.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6266ffd700 1 -- 172.21.15.62:0/564875651 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800b9070 con 0x5647bb382f10 2024-06-10T00:32:34.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6266ffd700 1 -- 172.21.15.62:0/564875651 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f628009b190 con 0x5647bb382f10 2024-06-10T00:32:34.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6298845b80 1 -- 172.21.15.62:0/564875651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6280099030 con 0x5647bb382f10 2024-06-10T00:32:34.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6266ffd700 1 -- 172.21.15.62:0/564875651 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6280079ba0 con 0x5647bb382f10 2024-06-10T00:32:34.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6298845b80 10 client.15267 fetch_fsmap learned FSMap version 29 2024-06-10T00:32:34.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6298845b80 1 -- 172.21.15.62:0/564875651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5647bb394280 con 0x5647bb382f10 2024-06-10T00:32:34.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6266ffd700 1 -- 172.21.15.62:0/564875651 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6280099030 con 0x5647bb382f10 2024-06-10T00:32:34.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6266ffd700 1 -- 172.21.15.62:0/564875651 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6280099bf0 con 0x5647bb382f10 2024-06-10T00:32:34.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6298845b80 10 client.15267 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:32:34.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6298845b80 10 client.15267 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-10T00:32:34.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6298845b80 20 client.15267 populate_metadata read hostname 'smithi062' 2024-06-10T00:32:34.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6298845b80 1 --2- 172.21.15.62:0/564875651 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x5647bb396450 0x5647bb3978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6298845b80 4 client.15267 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:32:34.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.481+0000 7f6298845b80 1 -- 172.21.15.62:0/564875651 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647bb2dbcc0 con 0x5647bb396450 2024-06-10T00:32:34.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.482+0000 7f62877fe700 1 --2- 172.21.15.62:0/564875651 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x5647bb396450 0x5647bb3978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.482+0000 7f6266ffd700 10 client.15267 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:32:34.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.482+0000 7f62877fe700 1 --2- 172.21.15.62:0/564875651 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x5647bb396450 0x5647bb3978c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a5f7da11ce568e5 server_cookie=eb7110586a0fc6b9 in_seq=0 out_seq=0 2024-06-10T00:32:34.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6266ffd700 1 -- 172.21.15.62:0/564875651 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5647bb2dbcc0 con 0x5647bb396450 2024-06-10T00:32:34.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6266ffd700 10 client.15267 handle_command_reply: tid=0 2024-06-10T00:32:34.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6298845b80 1 client.15267 shutdown 2024-06-10T00:32:34.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6298845b80 1 -- 172.21.15.62:0/564875651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 msgr2=0x5647bb3864b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:34.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6298845b80 1 --2- 172.21.15.62:0/564875651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb3864b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6280098fe0 tx=0x7f62800cbf40 comp rx=0 tx=0).stop 2024-06-10T00:32:34.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6298845b80 1 -- 172.21.15.62:0/564875651 shutdown_connections 2024-06-10T00:32:34.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6298845b80 1 --2- 172.21.15.62:0/564875651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb3864b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6298845b80 1 --2- 172.21.15.62:0/564875651 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x5647bb396450 0x5647bb3978c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6298845b80 1 --2- 172.21.15.62:0/564875651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 0x5647bb388c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6298845b80 1 --2- 172.21.15.62:0/564875651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb385f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6298845b80 1 -- 172.21.15.62:0/564875651 >> 172.21.15.62:0/564875651 conn(0x5647bb2a7c70 msgr2=0x5647bb37b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:34.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6266ffd700 0 client.15267 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:32:34.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6298845b80 1 -- 172.21.15.62:0/564875651 shutdown_connections 2024-06-10T00:32:34.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6298845b80 1 -- 172.21.15.62:0/564875651 wait complete. 2024-06-10T00:32:34.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.483+0000 7f6298845b80 20 client.15267 trim_cache size 0 max 16384 2024-06-10T00:32:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6298845b80 1 Processor -- start 2024-06-10T00:32:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6298845b80 1 -- start start 2024-06-10T00:32:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6298845b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 0x5647bb37c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6298845b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb3955d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6298845b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb383a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6298845b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5647bb395c00 con 0x5647bb382f10 2024-06-10T00:32:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6298845b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5647bb3962d0 con 0x5647bb37ced0 2024-06-10T00:32:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6298845b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5647bb3969a0 con 0x5647bb3825f0 2024-06-10T00:32:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6287fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb383a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6287fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb383a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45544/0 (socket says 172.21.15.62:45544) 2024-06-10T00:32:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6287fff700 1 -- 172.21.15.62:0/3212544344 learned_addr learned my addr 172.21.15.62:0/3212544344 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6286ffd700 1 --2- 172.21.15.62:0/3212544344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb3955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f62877fe700 1 --2- 172.21.15.62:0/3212544344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 0x5647bb37c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6287fff700 1 -- 172.21.15.62:0/3212544344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 msgr2=0x5647bb3955d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6287fff700 1 --2- 172.21.15.62:0/3212544344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb3955d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6287fff700 1 -- 172.21.15.62:0/3212544344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 msgr2=0x5647bb37c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6287fff700 1 --2- 172.21.15.62:0/3212544344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 0x5647bb37c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6287fff700 1 -- 172.21.15.62:0/3212544344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647bb3784d0 con 0x5647bb382f10 2024-06-10T00:32:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f62877fe700 1 --2- 172.21.15.62:0/3212544344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 0x5647bb37c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:32:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.506+0000 7f6287fff700 1 --2- 172.21.15.62:0/3212544344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb383a00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f62800cc340 tx=0x7f6280014b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb83c5a8423f7bce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.507+0000 7f626d7fa700 1 -- 172.21.15.62:0/3212544344 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800b9040 con 0x5647bb382f10 2024-06-10T00:32:34.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.507+0000 7f626d7fa700 1 -- 172.21.15.62:0/3212544344 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62800b7070 con 0x5647bb382f10 2024-06-10T00:32:34.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.507+0000 7f626d7fa700 1 -- 172.21.15.62:0/3212544344 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6280009040 con 0x5647bb382f10 2024-06-10T00:32:34.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.507+0000 7f6298845b80 1 -- 172.21.15.62:0/3212544344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 msgr2=0x5647bb383a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:34.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.507+0000 7f6298845b80 1 --2- 172.21.15.62:0/3212544344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb383a00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f62800cc340 tx=0x7f6280014b70 comp rx=0 tx=0).stop 2024-06-10T00:32:34.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.507+0000 7f6298845b80 1 -- 172.21.15.62:0/3212544344 shutdown_connections 2024-06-10T00:32:34.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.507+0000 7f6298845b80 1 --2- 172.21.15.62:0/3212544344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb383a00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.507+0000 7f6298845b80 1 --2- 172.21.15.62:0/3212544344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 0x5647bb37c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.507+0000 7f6298845b80 1 --2- 172.21.15.62:0/3212544344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb3955d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.507+0000 7f6298845b80 1 -- 172.21.15.62:0/3212544344 >> 172.21.15.62:0/3212544344 conn(0x5647bb2a7c70 msgr2=0x5647bb3832f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:34.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.507+0000 7f6298845b80 1 -- 172.21.15.62:0/3212544344 shutdown_connections 2024-06-10T00:32:34.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.508+0000 7f6298845b80 1 -- 172.21.15.62:0/3212544344 wait complete. 2024-06-10T00:32:34.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.508+0000 7f6298845b80 1 -- start start 2024-06-10T00:32:34.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.508+0000 7f6298845b80 1 Processor -- start 2024-06-10T00:32:34.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.508+0000 7f6298845b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 0x5647bb3a1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.508+0000 7f6298845b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb39c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.508+0000 7f6298845b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb39cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.508+0000 7f6298845b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5647bb397740 con 0x5647bb382f10 2024-06-10T00:32:34.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.508+0000 7f6298845b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5647bb397070 con 0x5647bb37ced0 2024-06-10T00:32:34.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.508+0000 7f6298845b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5647bb397e10 con 0x5647bb3825f0 2024-06-10T00:32:34.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.508+0000 7f6298845b80 10 client.0 _ll_register_callbacks cb 0x5647bb2688d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:32:34.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6287fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb39cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6287fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb39cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45558/0 (socket says 172.21.15.62:45558) 2024-06-10T00:32:34.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6287fff700 1 -- 172.21.15.62:0/1771060698 learned_addr learned my addr 172.21.15.62:0/1771060698 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:34.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6286ffd700 1 --2- 172.21.15.62:0/1771060698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb39c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f62877fe700 1 --2- 172.21.15.62:0/1771060698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 0x5647bb3a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6287fff700 1 -- 172.21.15.62:0/1771060698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 msgr2=0x5647bb39c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:34.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6287fff700 1 --2- 172.21.15.62:0/1771060698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb39c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6287fff700 1 -- 172.21.15.62:0/1771060698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 msgr2=0x5647bb3a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:34.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6287fff700 1 --2- 172.21.15.62:0/1771060698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 0x5647bb3a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6287fff700 1 -- 172.21.15.62:0/1771060698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647bb39d1e0 con 0x5647bb382f10 2024-06-10T00:32:34.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f62877fe700 1 --2- 172.21.15.62:0/1771060698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 0x5647bb3a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:32:34.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6287fff700 1 --2- 172.21.15.62:0/1771060698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb39cba0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f628009acc0 tx=0x7f628009ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe57a6c45fb9d4b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:34.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6266ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:32:34.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6266ffd700 1 -- 172.21.15.62:0/1771060698 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800b9040 con 0x5647bb382f10 2024-06-10T00:32:34.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6266ffd700 1 -- 172.21.15.62:0/1771060698 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f628009af90 con 0x5647bb382f10 2024-06-10T00:32:34.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6298845b80 1 -- 172.21.15.62:0/1771060698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62800b7070 con 0x5647bb382f10 2024-06-10T00:32:34.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6266ffd700 1 -- 172.21.15.62:0/1771060698 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f62800ad050 con 0x5647bb382f10 2024-06-10T00:32:34.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6298845b80 10 client.15273 fetch_fsmap learned FSMap version 29 2024-06-10T00:32:34.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.509+0000 7f6298845b80 1 -- 172.21.15.62:0/1771060698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5647bb39d1e0 con 0x5647bb382f10 2024-06-10T00:32:34.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.510+0000 7f6266ffd700 1 -- 172.21.15.62:0/1771060698 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62800b7070 con 0x5647bb382f10 2024-06-10T00:32:34.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.510+0000 7f6266ffd700 1 -- 172.21.15.62:0/1771060698 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6280085330 con 0x5647bb382f10 2024-06-10T00:32:34.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.510+0000 7f6298845b80 10 client.15273 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:32:34.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.510+0000 7f6298845b80 10 client.15273 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-10T00:32:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.510+0000 7f6298845b80 20 client.15273 populate_metadata read hostname 'smithi062' 2024-06-10T00:32:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.510+0000 7f6298845b80 1 --2- 172.21.15.62:0/1771060698 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x5647bb406080 0x5647bb404d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.510+0000 7f6298845b80 4 client.15273 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:32:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.510+0000 7f6298845b80 1 -- 172.21.15.62:0/1771060698 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "status"}) -- 0x5647bb2d6240 con 0x5647bb406080 2024-06-10T00:32:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.510+0000 7f62877fe700 1 --2- 172.21.15.62:0/1771060698 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x5647bb406080 0x5647bb404d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f62877fe700 1 --2- 172.21.15.62:0/1771060698 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x5647bb406080 0x5647bb404d90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=439c1bfaed7291db server_cookie=eeeead8333cbf816 in_seq=0 out_seq=0 2024-06-10T00:32:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6266ffd700 10 client.15273 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:32:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6266ffd700 1 -- 172.21.15.62:0/1771060698 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5647bb2d6240 con 0x5647bb406080 2024-06-10T00:32:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6266ffd700 10 client.15273 handle_command_reply: tid=0 2024-06-10T00:32:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6298845b80 1 client.15273 shutdown 2024-06-10T00:32:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6298845b80 1 -- 172.21.15.62:0/1771060698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 msgr2=0x5647bb39cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6298845b80 1 --2- 172.21.15.62:0/1771060698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb39cba0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f628009acc0 tx=0x7f628009ae90 comp rx=0 tx=0).stop 2024-06-10T00:32:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6298845b80 1 -- 172.21.15.62:0/1771060698 shutdown_connections 2024-06-10T00:32:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6298845b80 1 --2- 172.21.15.62:0/1771060698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647bb382f10 0x5647bb39cba0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6298845b80 1 --2- 172.21.15.62:0/1771060698 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x5647bb406080 0x5647bb404d90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6298845b80 1 --2- 172.21.15.62:0/1771060698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647bb37ced0 0x5647bb3a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6298845b80 1 --2- 172.21.15.62:0/1771060698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647bb3825f0 0x5647bb39c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6266ffd700 0 client.15273 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:32:34.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6298845b80 1 -- 172.21.15.62:0/1771060698 >> 172.21.15.62:0/1771060698 conn(0x5647bb2a7c70 msgr2=0x5647bb3832f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:34.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6298845b80 1 -- 172.21.15.62:0/1771060698 shutdown_connections 2024-06-10T00:32:34.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6298845b80 1 -- 172.21.15.62:0/1771060698 wait complete. 2024-06-10T00:32:34.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.511+0000 7f6298845b80 20 client.15273 trim_cache size 0 max 16384 2024-06-10T00:32:34.517 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:32:34.517 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:32:34.518 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2024-06-10T00:32:34.518 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24431, 2024-06-10T00:32:34.518 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:32:34.518 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:32:34.518 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:32:34.518 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 171.65938158399999, 2024-06-10T00:32:34.518 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 29, 2024-06-10T00:32:34.518 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 99, 2024-06-10T00:32:34.518 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T00:32:34.518 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 266.290557036 2024-06-10T00:32:34.518 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:32:34.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.512+0000 7f628dfc2700 1 -- 172.21.15.62:0/2442558896 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f62700a2c20 msgr2=0x7f62700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:34.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.512+0000 7f628dfc2700 1 --2- 172.21.15.62:0/2442558896 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f62700a2c20 0x7f62700a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6278009750 tx=0x7f627800b540 comp rx=0 tx=0).stop 2024-06-10T00:32:34.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.512+0000 7f628dfc2700 1 -- 172.21.15.62:0/2442558896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6288107550 msgr2=0x7f6288102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:34.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.512+0000 7f628dfc2700 1 --2- 172.21.15.62:0/2442558896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6288107550 0x7f6288102a90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f628000b970 tx=0x7f628000bd30 comp rx=0 tx=0).stop 2024-06-10T00:32:34.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.512+0000 7f628dfc2700 1 -- 172.21.15.62:0/2442558896 shutdown_connections 2024-06-10T00:32:34.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.512+0000 7f628dfc2700 1 --2- 172.21.15.62:0/2442558896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6288107550 0x7f6288102a90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.512+0000 7f628dfc2700 1 --2- 172.21.15.62:0/2442558896 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f62700a2c20 0x7f62700a50e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.512+0000 7f628dfc2700 1 --2- 172.21.15.62:0/2442558896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6288100e30 0x7f6288102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.512+0000 7f628dfc2700 1 --2- 172.21.15.62:0/2442558896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62881017f0 0x7f6288102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:34.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.512+0000 7f628dfc2700 1 -- 172.21.15.62:0/2442558896 >> 172.21.15.62:0/2442558896 conn(0x7f62880766b0 msgr2=0x7f62880ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:34.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.512+0000 7f628dfc2700 1 -- 172.21.15.62:0/2442558896 shutdown_connections 2024-06-10T00:32:34.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:34.512+0000 7f628dfc2700 1 -- 172.21.15.62:0/2442558896 wait complete. 2024-06-10T00:32:35.335 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T00:32:35.588 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:35.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:35 smithi062 ceph-mon[32390]: pgmap v425: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:32:35.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:35 smithi105 ceph-mon[36321]: pgmap v425: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:32:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:35 smithi138 ceph-mon[36549]: pgmap v425: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:32:37.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.393+0000 7f88d04db700 1 -- 172.21.15.62:0/840262885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8101510 msgr2=0x7f88c810d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:37.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.393+0000 7f88d04db700 1 --2- 172.21.15.62:0/840262885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8101510 0x7f88c810d9d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f88c000b0a0 tx=0x7f88c000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.393+0000 7f88d04db700 1 -- 172.21.15.62:0/840262885 shutdown_connections 2024-06-10T00:32:37.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.393+0000 7f88d04db700 1 --2- 172.21.15.62:0/840262885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8101510 0x7f88c810d9d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.393+0000 7f88d04db700 1 --2- 172.21.15.62:0/840262885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88c8106b90 0x7f88c8106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.393+0000 7f88d04db700 1 --2- 172.21.15.62:0/840262885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88c8100b50 0x7f88c8100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.393+0000 7f88d04db700 1 -- 172.21.15.62:0/840262885 >> 172.21.15.62:0/840262885 conn(0x7f88c8077a00 msgr2=0x7f88c80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:37.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.393+0000 7f88d04db700 1 -- 172.21.15.62:0/840262885 shutdown_connections 2024-06-10T00:32:37.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.393+0000 7f88d04db700 1 -- 172.21.15.62:0/840262885 wait complete. 2024-06-10T00:32:37.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.394+0000 7f88d04db700 1 Processor -- start 2024-06-10T00:32:37.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.394+0000 7f88d04db700 1 -- start start 2024-06-10T00:32:37.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.394+0000 7f88d04db700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88c8100b50 0x7f88c80fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.394+0000 7f88d04db700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88c8101510 0x7f88c80fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.394+0000 7f88d04db700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8106b90 0x7f88c80fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.394+0000 7f88d04db700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8110e50 con 0x7f88c8106b90 2024-06-10T00:32:37.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.394+0000 7f88d04db700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8110cd0 con 0x7f88c8100b50 2024-06-10T00:32:37.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.394+0000 7f88d04db700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8110fd0 con 0x7f88c8101510 2024-06-10T00:32:37.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.394+0000 7f88cea78700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8106b90 0x7f88c80fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.394+0000 7f88cea78700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8106b90 0x7f88c80fd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45574/0 (socket says 172.21.15.62:45574) 2024-06-10T00:32:37.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.394+0000 7f88cea78700 1 -- 172.21.15.62:0/843746189 learned_addr learned my addr 172.21.15.62:0/843746189 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:37.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.395+0000 7f88cda76700 1 --2- 172.21.15.62:0/843746189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88c8101510 0x7f88c80fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.395+0000 7f88ce277700 1 --2- 172.21.15.62:0/843746189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88c8100b50 0x7f88c80fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.395+0000 7f88cea78700 1 -- 172.21.15.62:0/843746189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88c8101510 msgr2=0x7f88c80fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:37.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.395+0000 7f88cea78700 1 --2- 172.21.15.62:0/843746189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88c8101510 0x7f88c80fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.395+0000 7f88cea78700 1 -- 172.21.15.62:0/843746189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88c8100b50 msgr2=0x7f88c80fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:37.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.395+0000 7f88cea78700 1 --2- 172.21.15.62:0/843746189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88c8100b50 0x7f88c80fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.395+0000 7f88cea78700 1 -- 172.21.15.62:0/843746189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c81a70a0 con 0x7f88c8106b90 2024-06-10T00:32:37.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.395+0000 7f88cea78700 1 --2- 172.21.15.62:0/843746189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8106b90 0x7f88c80fd8f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f88c000b780 tx=0x7f88c0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:37.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.395+0000 7f88bf7fe700 1 -- 172.21.15.62:0/843746189 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0021070 con 0x7f88c8106b90 2024-06-10T00:32:37.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.395+0000 7f88bf7fe700 1 -- 172.21.15.62:0/843746189 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c0007b80 con 0x7f88c8106b90 2024-06-10T00:32:37.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.395+0000 7f88d04db700 1 -- 172.21.15.62:0/843746189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c81a7330 con 0x7f88c8106b90 2024-06-10T00:32:37.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.395+0000 7f88bf7fe700 1 -- 172.21.15.62:0/843746189 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0005370 con 0x7f88c8106b90 2024-06-10T00:32:37.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.395+0000 7f88d04db700 1 -- 172.21.15.62:0/843746189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c81a77e0 con 0x7f88c8106b90 2024-06-10T00:32:37.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.397+0000 7f88bf7fe700 1 -- 172.21.15.62:0/843746189 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f88c0024070 con 0x7f88c8106b90 2024-06-10T00:32:37.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.398+0000 7f88bf7fe700 1 --2- 172.21.15.62:0/843746189 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f88b40a2c20 0x7f88b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.398+0000 7f88ce277700 1 --2- 172.21.15.62:0/843746189 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f88b40a2c20 0x7f88b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.398+0000 7f88bf7fe700 1 -- 172.21.15.62:0/843746189 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f88c000f020 con 0x7f88c8106b90 2024-06-10T00:32:37.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.399+0000 7f88ce277700 1 --2- 172.21.15.62:0/843746189 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f88b40a2c20 0x7f88b40a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009bd0 tx=0x7f88c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:37.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.399+0000 7f88dad5eb80 1 Processor -- start 2024-06-10T00:32:37.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.399+0000 7f88dad5eb80 1 -- start start 2024-06-10T00:32:37.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.399+0000 7f88dad5eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a2f10 0x55c31c8a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.399+0000 7f88dad5eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a3830 0x55c31c89d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.399+0000 7f88dad5eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c89db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.399+0000 7f88dad5eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c31c78ff10 con 0x55c31c89d750 2024-06-10T00:32:37.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.399+0000 7f88dad5eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c31c7ee800 con 0x55c31c8a3830 2024-06-10T00:32:37.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.399+0000 7f88dad5eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c31c756410 con 0x55c31c8a2f10 2024-06-10T00:32:37.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.399+0000 7f88cea78700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c89db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.399+0000 7f88cea78700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c89db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45582/0 (socket says 172.21.15.62:45582) 2024-06-10T00:32:37.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.399+0000 7f88cea78700 1 -- 172.21.15.62:0/330387487 learned_addr learned my addr 172.21.15.62:0/330387487 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:37.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.400+0000 7f88cda76700 1 --2- 172.21.15.62:0/330387487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a3830 0x55c31c89d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.400+0000 7f88ce277700 1 --2- 172.21.15.62:0/330387487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a2f10 0x55c31c8a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.400+0000 7f88cea78700 1 -- 172.21.15.62:0/330387487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a2f10 msgr2=0x55c31c8a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:37.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.400+0000 7f88cea78700 1 --2- 172.21.15.62:0/330387487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a2f10 0x55c31c8a32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.400+0000 7f88cea78700 1 -- 172.21.15.62:0/330387487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a3830 msgr2=0x55c31c89d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:37.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.400+0000 7f88cea78700 1 --2- 172.21.15.62:0/330387487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a3830 0x55c31c89d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.400+0000 7f88cea78700 1 -- 172.21.15.62:0/330387487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c31c73d880 con 0x55c31c89d750 2024-06-10T00:32:37.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.400+0000 7f88ce277700 1 --2- 172.21.15.62:0/330387487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a2f10 0x55c31c8a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:32:37.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.400+0000 7f88cea78700 1 --2- 172.21.15.62:0/330387487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c89db70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f88c009ce40 tx=0x7f88c000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c24f43ecbe7585 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:37.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.400+0000 7f88bcd70700 1 -- 172.21.15.62:0/330387487 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c00a1260 con 0x55c31c89d750 2024-06-10T00:32:37.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.400+0000 7f88bcd70700 1 -- 172.21.15.62:0/330387487 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c00a5a90 con 0x55c31c89d750 2024-06-10T00:32:37.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.400+0000 7f88bcd70700 1 -- 172.21.15.62:0/330387487 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c009d370 con 0x55c31c89d750 2024-06-10T00:32:37.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.401+0000 7f88dad5eb80 1 -- 172.21.15.62:0/330387487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 msgr2=0x55c31c89db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:37.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.401+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/330387487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c89db70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f88c009ce40 tx=0x7f88c000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.401+0000 7f88dad5eb80 1 -- 172.21.15.62:0/330387487 shutdown_connections 2024-06-10T00:32:37.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.401+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/330387487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c89db70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.401+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/330387487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a2f10 0x55c31c8a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.401+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/330387487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a3830 0x55c31c89d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.401+0000 7f88dad5eb80 1 -- 172.21.15.62:0/330387487 >> 172.21.15.62:0/330387487 conn(0x55c31c7ca030 msgr2=0x55c31c8981d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:37.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.401+0000 7f88dad5eb80 1 -- 172.21.15.62:0/330387487 shutdown_connections 2024-06-10T00:32:37.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.401+0000 7f88dad5eb80 1 -- 172.21.15.62:0/330387487 wait complete. 2024-06-10T00:32:37.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.401+0000 7f88dad5eb80 1 -- start start 2024-06-10T00:32:37.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.401+0000 7f88dad5eb80 1 Processor -- start 2024-06-10T00:32:37.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88dad5eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c8a9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88dad5eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a2f10 0x55c31c8a6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88dad5eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a3830 0x55c31c8a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88dad5eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c31c78ef70 con 0x55c31c89d750 2024-06-10T00:32:37.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88dad5eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c31c7ea7c0 con 0x55c31c8a3830 2024-06-10T00:32:37.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88dad5eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c31c8a9fc0 con 0x55c31c8a2f10 2024-06-10T00:32:37.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88dad5eb80 10 client.0 _ll_register_callbacks cb 0x55c31c784560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:32:37.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88ce277700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c8a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88ce277700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c8a9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45584/0 (socket says 172.21.15.62:45584) 2024-06-10T00:32:37.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88ce277700 1 -- 172.21.15.62:0/2260509380 learned_addr learned my addr 172.21.15.62:0/2260509380 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:37.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88cea78700 1 --2- 172.21.15.62:0/2260509380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a3830 0x55c31c8a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88cda76700 1 --2- 172.21.15.62:0/2260509380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a2f10 0x55c31c8a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88ce277700 1 -- 172.21.15.62:0/2260509380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a2f10 msgr2=0x55c31c8a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:37.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88ce277700 1 --2- 172.21.15.62:0/2260509380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a2f10 0x55c31c8a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88ce277700 1 -- 172.21.15.62:0/2260509380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a3830 msgr2=0x55c31c8a6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:37.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88ce277700 1 --2- 172.21.15.62:0/2260509380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a3830 0x55c31c8a6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88ce277700 1 -- 172.21.15.62:0/2260509380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c31c7b4700 con 0x55c31c89d750 2024-06-10T00:32:37.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88ce277700 1 --2- 172.21.15.62:0/2260509380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c8a9590 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f88c4011770 tx=0x7f88c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eff0309f4456056 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:37.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.402+0000 7f88b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:32:37.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.403+0000 7f88b1ffb700 1 -- 172.21.15.62:0/2260509380 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c401f840 con 0x55c31c89d750 2024-06-10T00:32:37.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.403+0000 7f88dad5eb80 1 -- 172.21.15.62:0/2260509380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88c00a5a90 con 0x55c31c89d750 2024-06-10T00:32:37.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.403+0000 7f88b1ffb700 1 -- 172.21.15.62:0/2260509380 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f88c401f9e0 con 0x55c31c89d750 2024-06-10T00:32:37.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.403+0000 7f88b1ffb700 1 -- 172.21.15.62:0/2260509380 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f88c4031050 con 0x55c31c89d750 2024-06-10T00:32:37.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.403+0000 7f88dad5eb80 10 client.15285 fetch_fsmap learned FSMap version 29 2024-06-10T00:32:37.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.403+0000 7f88dad5eb80 1 -- 172.21.15.62:0/2260509380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c31c8b4770 con 0x55c31c89d750 2024-06-10T00:32:37.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.403+0000 7f88b1ffb700 1 -- 172.21.15.62:0/2260509380 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c00a5a90 con 0x55c31c89d750 2024-06-10T00:32:37.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.403+0000 7f88b1ffb700 1 -- 172.21.15.62:0/2260509380 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f88c401e430 con 0x55c31c89d750 2024-06-10T00:32:37.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.403+0000 7f88dad5eb80 10 client.15285 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:32:37.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.403+0000 7f88dad5eb80 10 client.15285 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T00:32:37.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.403+0000 7f88dad5eb80 20 client.15285 populate_metadata read hostname 'smithi062' 2024-06-10T00:32:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.403+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/2260509380 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c31c8b6db0 0x55c31c8b8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.403+0000 7f88dad5eb80 4 client.15285 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:32:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.403+0000 7f88dad5eb80 1 -- 172.21.15.62:0/2260509380 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c31c7fbd00 con 0x55c31c8b6db0 2024-06-10T00:32:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.404+0000 7f88cda76700 1 --2- 172.21.15.62:0/2260509380 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c31c8b6db0 0x55c31c8b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.404+0000 7f88b1ffb700 10 client.15285 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:32:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.404+0000 7f88cda76700 1 --2- 172.21.15.62:0/2260509380 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c31c8b6db0 0x55c31c8b8220 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0beebba09d3b860 server_cookie=777b82bdbd9cc7ba in_seq=0 out_seq=0 2024-06-10T00:32:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88b1ffb700 1 -- 172.21.15.62:0/2260509380 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c31c7fbd00 con 0x55c31c8b6db0 2024-06-10T00:32:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88b1ffb700 10 client.15285 handle_command_reply: tid=0 2024-06-10T00:32:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88dad5eb80 1 client.15285 shutdown 2024-06-10T00:32:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88dad5eb80 1 -- 172.21.15.62:0/2260509380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 msgr2=0x55c31c8a9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/2260509380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c8a9590 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f88c4011770 tx=0x7f88c40047c0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88dad5eb80 1 -- 172.21.15.62:0/2260509380 shutdown_connections 2024-06-10T00:32:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/2260509380 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c31c8b6db0 0x55c31c8b8220 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/2260509380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c8a9590 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/2260509380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a3830 0x55c31c8a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/2260509380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a2f10 0x55c31c8a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88dad5eb80 1 -- 172.21.15.62:0/2260509380 >> 172.21.15.62:0/2260509380 conn(0x55c31c7ca030 msgr2=0x55c31c8976b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:37.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88b1ffb700 0 client.15285 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:32:37.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88dad5eb80 1 -- 172.21.15.62:0/2260509380 shutdown_connections 2024-06-10T00:32:37.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88dad5eb80 1 -- 172.21.15.62:0/2260509380 wait complete. 2024-06-10T00:32:37.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.405+0000 7f88dad5eb80 20 client.15285 trim_cache size 0 max 16384 2024-06-10T00:32:37.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88dad5eb80 1 Processor -- start 2024-06-10T00:32:37.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88dad5eb80 1 -- start start 2024-06-10T00:32:37.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88dad5eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c89d750 0x55c31c89d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88dad5eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c8a2f10 0x55c31c8b5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88dad5eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a3830 0x55c31c8a5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88dad5eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c31c8b6560 con 0x55c31c8a2f10 2024-06-10T00:32:37.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88dad5eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c31c8b6c30 con 0x55c31c89d750 2024-06-10T00:32:37.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88dad5eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c31c8b7300 con 0x55c31c8a3830 2024-06-10T00:32:37.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88cda76700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c8a2f10 0x55c31c8b5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88cda76700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c8a2f10 0x55c31c8b5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45590/0 (socket says 172.21.15.62:45590) 2024-06-10T00:32:37.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88cda76700 1 -- 172.21.15.62:0/3138248806 learned_addr learned my addr 172.21.15.62:0/3138248806 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:37.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88cea78700 1 --2- 172.21.15.62:0/3138248806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a3830 0x55c31c8a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88cda76700 1 -- 172.21.15.62:0/3138248806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a3830 msgr2=0x55c31c8a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:37.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88cda76700 1 --2- 172.21.15.62:0/3138248806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a3830 0x55c31c8a5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88cda76700 1 -- 172.21.15.62:0/3138248806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c89d750 msgr2=0x55c31c89d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:37.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88cda76700 1 --2- 172.21.15.62:0/3138248806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c89d750 0x55c31c89d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.427+0000 7f88cda76700 1 -- 172.21.15.62:0/3138248806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c31c8a4e00 con 0x55c31c8a2f10 2024-06-10T00:32:37.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.428+0000 7f88cda76700 1 --2- 172.21.15.62:0/3138248806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c8a2f10 0x55c31c8b5f50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55c31c8a7e60 tx=0x7f88b80073d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217c4884470ab1c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:37.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.428+0000 7f88b3fff700 1 -- 172.21.15.62:0/3138248806 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8003c60 con 0x55c31c8a2f10 2024-06-10T00:32:37.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.428+0000 7f88b3fff700 1 -- 172.21.15.62:0/3138248806 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b8004540 con 0x55c31c8a2f10 2024-06-10T00:32:37.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.428+0000 7f88b3fff700 1 -- 172.21.15.62:0/3138248806 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8011440 con 0x55c31c8a2f10 2024-06-10T00:32:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.428+0000 7f88dad5eb80 1 -- 172.21.15.62:0/3138248806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c8a2f10 msgr2=0x55c31c8b5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.428+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/3138248806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c8a2f10 0x55c31c8b5f50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55c31c8a7e60 tx=0x7f88b80073d0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.428+0000 7f88dad5eb80 1 -- 172.21.15.62:0/3138248806 shutdown_connections 2024-06-10T00:32:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.428+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/3138248806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a3830 0x55c31c8a5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.428+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/3138248806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c89d750 0x55c31c89d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.428+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/3138248806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c8a2f10 0x55c31c8b5f50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.428+0000 7f88dad5eb80 1 -- 172.21.15.62:0/3138248806 >> 172.21.15.62:0/3138248806 conn(0x55c31c7ca030 msgr2=0x55c31c8a5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.429+0000 7f88dad5eb80 1 -- 172.21.15.62:0/3138248806 shutdown_connections 2024-06-10T00:32:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.429+0000 7f88dad5eb80 1 -- 172.21.15.62:0/3138248806 wait complete. 2024-06-10T00:32:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.429+0000 7f88dad5eb80 1 -- start start 2024-06-10T00:32:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.429+0000 7f88dad5eb80 1 Processor -- start 2024-06-10T00:32:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.429+0000 7f88dad5eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c8c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.429+0000 7f88dad5eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a2f10 0x55c31c8c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.429+0000 7f88dad5eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a3830 0x55c31c8bc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.429+0000 7f88dad5eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c31c8b80a0 con 0x55c31c89d750 2024-06-10T00:32:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.429+0000 7f88dad5eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c31c8b79d0 con 0x55c31c8a2f10 2024-06-10T00:32:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.429+0000 7f88dad5eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c31c8b8770 con 0x55c31c8a3830 2024-06-10T00:32:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.429+0000 7f88dad5eb80 10 client.0 _ll_register_callbacks cb 0x55c31c784560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:32:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.429+0000 7f88ce277700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c8c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88ce277700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c8c16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45606/0 (socket says 172.21.15.62:45606) 2024-06-10T00:32:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88ce277700 1 -- 172.21.15.62:0/4024706013 learned_addr learned my addr 172.21.15.62:0/4024706013 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88cea78700 1 --2- 172.21.15.62:0/4024706013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a3830 0x55c31c8bc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88cda76700 1 --2- 172.21.15.62:0/4024706013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a2f10 0x55c31c8c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88ce277700 1 -- 172.21.15.62:0/4024706013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a3830 msgr2=0x55c31c8bc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88ce277700 1 --2- 172.21.15.62:0/4024706013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a3830 0x55c31c8bc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88ce277700 1 -- 172.21.15.62:0/4024706013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a2f10 msgr2=0x55c31c8c1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88ce277700 1 --2- 172.21.15.62:0/4024706013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a2f10 0x55c31c8c1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88ce277700 1 -- 172.21.15.62:0/4024706013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c31c898670 con 0x55c31c89d750 2024-06-10T00:32:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88cda76700 1 --2- 172.21.15.62:0/4024706013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a2f10 0x55c31c8c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:32:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88ce277700 1 --2- 172.21.15.62:0/4024706013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c8c16a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f88c401e880 tx=0x7f88c4010810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8faf96a8662752 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:32:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88b1ffb700 1 -- 172.21.15.62:0/4024706013 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4039070 con 0x55c31c89d750 2024-06-10T00:32:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88b1ffb700 1 -- 172.21.15.62:0/4024706013 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f88c401ea70 con 0x55c31c89d750 2024-06-10T00:32:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88dad5eb80 1 -- 172.21.15.62:0/4024706013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b8004540 con 0x55c31c89d750 2024-06-10T00:32:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88b1ffb700 1 -- 172.21.15.62:0/4024706013 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f88c401ec70 con 0x55c31c89d750 2024-06-10T00:32:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88dad5eb80 10 client.15291 fetch_fsmap learned FSMap version 29 2024-06-10T00:32:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.430+0000 7f88dad5eb80 1 -- 172.21.15.62:0/4024706013 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c31c922c70 con 0x55c31c89d750 2024-06-10T00:32:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.431+0000 7f88b1ffb700 1 -- 172.21.15.62:0/4024706013 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b8004540 con 0x55c31c89d750 2024-06-10T00:32:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.431+0000 7f88b1ffb700 1 -- 172.21.15.62:0/4024706013 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f88c4028500 con 0x55c31c89d750 2024-06-10T00:32:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.431+0000 7f88dad5eb80 10 client.15291 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:32:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.431+0000 7f88dad5eb80 10 client.15291 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T00:32:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.431+0000 7f88dad5eb80 20 client.15291 populate_metadata read hostname 'smithi062' 2024-06-10T00:32:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.431+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/4024706013 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c31c9278f0 0x55c31c926600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.431+0000 7f88dad5eb80 4 client.15291 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:32:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.431+0000 7f88dad5eb80 1 -- 172.21.15.62:0/4024706013 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "status"}) -- 0x55c31c7f6280 con 0x55c31c9278f0 2024-06-10T00:32:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.431+0000 7f88cda76700 1 --2- 172.21.15.62:0/4024706013 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c31c9278f0 0x55c31c926600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88cda76700 1 --2- 172.21.15.62:0/4024706013 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c31c9278f0 0x55c31c926600 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed59a1e1f9c336ed server_cookie=856ec126bedcdd0a in_seq=0 out_seq=0 2024-06-10T00:32:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88b1ffb700 10 client.15291 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:32:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88b1ffb700 1 -- 172.21.15.62:0/4024706013 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c31c7f6280 con 0x55c31c9278f0 2024-06-10T00:32:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88b1ffb700 10 client.15291 handle_command_reply: tid=0 2024-06-10T00:32:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88dad5eb80 1 client.15291 shutdown 2024-06-10T00:32:37.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88dad5eb80 1 -- 172.21.15.62:0/4024706013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 msgr2=0x55c31c8c16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:37.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/4024706013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c8c16a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f88c401e880 tx=0x7f88c4010810 comp rx=0 tx=0).stop 2024-06-10T00:32:37.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88dad5eb80 1 -- 172.21.15.62:0/4024706013 shutdown_connections 2024-06-10T00:32:37.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/4024706013 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c31c9278f0 0x55c31c926600 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/4024706013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c31c89d750 0x55c31c8c16a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/4024706013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c31c8a3830 0x55c31c8bc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88dad5eb80 1 --2- 172.21.15.62:0/4024706013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c31c8a2f10 0x55c31c8c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88dad5eb80 1 -- 172.21.15.62:0/4024706013 >> 172.21.15.62:0/4024706013 conn(0x55c31c7ca030 msgr2=0x55c31c8a80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:37.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88b1ffb700 0 client.15291 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:32:37.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88dad5eb80 1 -- 172.21.15.62:0/4024706013 shutdown_connections 2024-06-10T00:32:37.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88dad5eb80 1 -- 172.21.15.62:0/4024706013 wait complete. 2024-06-10T00:32:37.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.432+0000 7f88dad5eb80 20 client.15291 trim_cache size 0 max 16384 2024-06-10T00:32:37.440 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:32:37.440 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:32:37.440 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2024-06-10T00:32:37.440 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24433, 2024-06-10T00:32:37.440 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:32:37.440 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:32:37.440 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:32:37.440 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 175.58936666599999, 2024-06-10T00:32:37.440 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 29, 2024-06-10T00:32:37.440 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 99, 2024-06-10T00:32:37.440 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T00:32:37.441 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 266.47569460699998 2024-06-10T00:32:37.441 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:32:37.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.433+0000 7f88d04db700 1 -- 172.21.15.62:0/843746189 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f88b40a2c20 msgr2=0x7f88b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:37.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.433+0000 7f88d04db700 1 --2- 172.21.15.62:0/843746189 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f88b40a2c20 0x7f88b40a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009bd0 tx=0x7f88c4009260 comp rx=0 tx=0).stop 2024-06-10T00:32:37.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.433+0000 7f88d04db700 1 -- 172.21.15.62:0/843746189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8106b90 msgr2=0x7f88c80fd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:37.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.433+0000 7f88d04db700 1 --2- 172.21.15.62:0/843746189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8106b90 0x7f88c80fd8f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f88c000b780 tx=0x7f88c0004d00 comp rx=0 tx=0).stop 2024-06-10T00:32:37.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.433+0000 7f88d04db700 1 -- 172.21.15.62:0/843746189 shutdown_connections 2024-06-10T00:32:37.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.433+0000 7f88d04db700 1 --2- 172.21.15.62:0/843746189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c8106b90 0x7f88c80fd8f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.433+0000 7f88d04db700 1 --2- 172.21.15.62:0/843746189 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f88b40a2c20 0x7f88b40a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.433+0000 7f88d04db700 1 --2- 172.21.15.62:0/843746189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88c8100b50 0x7f88c80fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.433+0000 7f88d04db700 1 --2- 172.21.15.62:0/843746189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88c8101510 0x7f88c80fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:37.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.433+0000 7f88d04db700 1 -- 172.21.15.62:0/843746189 >> 172.21.15.62:0/843746189 conn(0x7f88c8077a00 msgr2=0x7f88c8107960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:37.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.433+0000 7f88d04db700 1 -- 172.21.15.62:0/843746189 shutdown_connections 2024-06-10T00:32:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:37.433+0000 7f88d04db700 1 -- 172.21.15.62:0/843746189 wait complete. 2024-06-10T00:32:37.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:37 smithi105 ceph-mon[36321]: pgmap v426: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:32:37.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:37 smithi062 ceph-mon[32390]: pgmap v426: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:32:37.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:37 smithi138 ceph-mon[36549]: pgmap v426: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:32:38.169 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-10T00:32:38.421 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:39.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:39 smithi062 ceph-mon[32390]: pgmap v427: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:39.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:39 smithi105 ceph-mon[36321]: pgmap v427: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:39.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:39 smithi138 ceph-mon[36549]: pgmap v427: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-10T00:32:40.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.218+0000 7fabb4bb8700 1 -- 172.21.15.62:0/3524795063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb00746c0 msgr2=0x7fabb0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:40.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.218+0000 7fabb4bb8700 1 --2- 172.21.15.62:0/3524795063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb00746c0 0x7fabb0074aa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7faba0009990 tx=0x7faba0009ca0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.218+0000 7fabb4bb8700 1 -- 172.21.15.62:0/3524795063 shutdown_connections 2024-06-10T00:32:40.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.218+0000 7fabb4bb8700 1 --2- 172.21.15.62:0/3524795063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb00746c0 0x7fabb0074aa0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.218+0000 7fabb4bb8700 1 --2- 172.21.15.62:0/3524795063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb010ee90 0x7fabb0111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.218+0000 7fabb4bb8700 1 --2- 172.21.15.62:0/3524795063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabb0074fe0 0x7fabb010e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.218+0000 7fabb4bb8700 1 -- 172.21.15.62:0/3524795063 >> 172.21.15.62:0/3524795063 conn(0x7fabb00795f0 msgr2=0x7fabb0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:40.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.218+0000 7fabb4bb8700 1 -- 172.21.15.62:0/3524795063 shutdown_connections 2024-06-10T00:32:40.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.218+0000 7fabb4bb8700 1 -- 172.21.15.62:0/3524795063 wait complete. 2024-06-10T00:32:40.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.219+0000 7fabb4bb8700 1 Processor -- start 2024-06-10T00:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.219+0000 7fabb4bb8700 1 -- start start 2024-06-10T00:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.219+0000 7fabb4bb8700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabb00746c0 0x7fabb019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.219+0000 7fabb4bb8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb0074fe0 0x7fabb019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.219+0000 7fabb4bb8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb010ee90 0x7fabb019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.219+0000 7fabb4bb8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0114a10 con 0x7fabb010ee90 2024-06-10T00:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.219+0000 7fabb4bb8700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0114890 con 0x7fabb00746c0 2024-06-10T00:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.219+0000 7fabb4bb8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0114b90 con 0x7fabb0074fe0 2024-06-10T00:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.219+0000 7fabadd9b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb0074fe0 0x7fabb019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.219+0000 7fabadd9b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb0074fe0 0x7fabb019b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:38718/0 (socket says 172.21.15.62:38718) 2024-06-10T00:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.219+0000 7fabadd9b700 1 -- 172.21.15.62:0/3684964928 learned_addr learned my addr 172.21.15.62:0/3684964928 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.219+0000 7fabae59c700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabb00746c0 0x7fabb019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.219+0000 7fabaed9d700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb010ee90 0x7fabb019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.220+0000 7fabaed9d700 1 -- 172.21.15.62:0/3684964928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb0074fe0 msgr2=0x7fabb019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:40.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.220+0000 7fabaed9d700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb0074fe0 0x7fabb019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.220+0000 7fabaed9d700 1 -- 172.21.15.62:0/3684964928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabb00746c0 msgr2=0x7fabb019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:40.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.220+0000 7fabaed9d700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabb00746c0 0x7fabb019ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.220+0000 7fabaed9d700 1 -- 172.21.15.62:0/3684964928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb019fe90 con 0x7fabb010ee90 2024-06-10T00:32:40.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.220+0000 7fabadd9b700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb0074fe0 0x7fabb019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:32:40.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.220+0000 7fabae59c700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabb00746c0 0x7fabb019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:32:40.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.220+0000 7fabaed9d700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb010ee90 0x7fabb019f750 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7faba800ab40 tx=0x7faba800ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:40.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.220+0000 7fab9f7fe700 1 -- 172.21.15.62:0/3684964928 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8017020 con 0x7fabb010ee90 2024-06-10T00:32:40.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.220+0000 7fab9f7fe700 1 -- 172.21.15.62:0/3684964928 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba8004bb0 con 0x7fabb010ee90 2024-06-10T00:32:40.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.220+0000 7fabb4bb8700 1 -- 172.21.15.62:0/3684964928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb01a0180 con 0x7fabb010ee90 2024-06-10T00:32:40.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.220+0000 7fab9f7fe700 1 -- 172.21.15.62:0/3684964928 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8005260 con 0x7fabb010ee90 2024-06-10T00:32:40.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.220+0000 7fabb4bb8700 1 -- 172.21.15.62:0/3684964928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb01a7a00 con 0x7fabb010ee90 2024-06-10T00:32:40.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.223+0000 7fab9f7fe700 1 -- 172.21.15.62:0/3684964928 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faba8005500 con 0x7fabb010ee90 2024-06-10T00:32:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.223+0000 7fab9f7fe700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fab980a2c20 0x7fab980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.224+0000 7fab9f7fe700 1 -- 172.21.15.62:0/3684964928 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7faba80ce930 con 0x7fabb010ee90 2024-06-10T00:32:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.224+0000 7fabae59c700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fab980a2c20 0x7fab980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.224+0000 7fabae59c700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fab980a2c20 0x7fab980a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faba000b5c0 tx=0x7faba0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.224+0000 7fabbf43bb80 1 Processor -- start 2024-06-10T00:32:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.224+0000 7fabbf43bb80 1 -- start start 2024-06-10T00:32:40.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabbf43bb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 0x55dfc653a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabbf43bb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc653a830 0x55dfc6534210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabbf43bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6534750 0x55dfc6534b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabbf43bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55dfc6426f10 con 0x55dfc6534750 2024-06-10T00:32:40.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabbf43bb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dfc6485800 con 0x55dfc6539f10 2024-06-10T00:32:40.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabbf43bb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dfc63ed410 con 0x55dfc653a830 2024-06-10T00:32:40.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabaed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6534750 0x55dfc6534b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabaed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6534750 0x55dfc6534b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55708/0 (socket says 172.21.15.62:55708) 2024-06-10T00:32:40.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabaed9d700 1 -- 172.21.15.62:0/2070236640 learned_addr learned my addr 172.21.15.62:0/2070236640 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:40.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabae59c700 1 --2- 172.21.15.62:0/2070236640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 0x55dfc653a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabadd9b700 1 --2- 172.21.15.62:0/2070236640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc653a830 0x55dfc6534210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabaed9d700 1 -- 172.21.15.62:0/2070236640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc653a830 msgr2=0x55dfc6534210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:40.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabaed9d700 1 --2- 172.21.15.62:0/2070236640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc653a830 0x55dfc6534210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabaed9d700 1 -- 172.21.15.62:0/2070236640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 msgr2=0x55dfc653a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:40.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabaed9d700 1 --2- 172.21.15.62:0/2070236640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 0x55dfc653a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabaed9d700 1 -- 172.21.15.62:0/2070236640 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfc63d4880 con 0x55dfc6534750 2024-06-10T00:32:40.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fabaed9d700 1 --2- 172.21.15.62:0/2070236640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6534750 0x55dfc6534b70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faba809ade0 tx=0x7faba809aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307a285605f45fe2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:40.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.225+0000 7fab9cff9700 1 -- 172.21.15.62:0/2070236640 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba809ae50 con 0x55dfc6534750 2024-06-10T00:32:40.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.226+0000 7fab9cff9700 1 -- 172.21.15.62:0/2070236640 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba809d070 con 0x55dfc6534750 2024-06-10T00:32:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.226+0000 7fab9cff9700 1 -- 172.21.15.62:0/2070236640 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba809b2b0 con 0x55dfc6534750 2024-06-10T00:32:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.226+0000 7fabbf43bb80 1 -- 172.21.15.62:0/2070236640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6534750 msgr2=0x55dfc6534b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.226+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/2070236640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6534750 0x55dfc6534b70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faba809ade0 tx=0x7faba809aa80 comp rx=0 tx=0).stop 2024-06-10T00:32:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.226+0000 7fabbf43bb80 1 -- 172.21.15.62:0/2070236640 shutdown_connections 2024-06-10T00:32:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.226+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/2070236640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6534750 0x55dfc6534b70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.226+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/2070236640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 0x55dfc653a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.226+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/2070236640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc653a830 0x55dfc6534210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.226+0000 7fabbf43bb80 1 -- 172.21.15.62:0/2070236640 >> 172.21.15.62:0/2070236640 conn(0x55dfc6461030 msgr2=0x55dfc652f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.226+0000 7fabbf43bb80 1 -- 172.21.15.62:0/2070236640 shutdown_connections 2024-06-10T00:32:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.226+0000 7fabbf43bb80 1 -- 172.21.15.62:0/2070236640 wait complete. 2024-06-10T00:32:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.227+0000 7fabbf43bb80 1 -- start start 2024-06-10T00:32:40.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.227+0000 7fabbf43bb80 1 Processor -- start 2024-06-10T00:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.227+0000 7fabbf43bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6534750 0x55dfc6540500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.227+0000 7fabbf43bb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 0x55dfc653d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.227+0000 7fabbf43bb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc653a830 0x55dfc653dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.227+0000 7fabbf43bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55dfc6425f70 con 0x55dfc6534750 2024-06-10T00:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.227+0000 7fabbf43bb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dfc64817c0 con 0x55dfc6539f10 2024-06-10T00:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.227+0000 7fabbf43bb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dfc6541210 con 0x55dfc653a830 2024-06-10T00:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.227+0000 7fabbf43bb80 10 client.0 _ll_register_callbacks cb 0x55dfc641b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.227+0000 7fabae59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6534750 0x55dfc6540500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.227+0000 7fabae59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6534750 0x55dfc6540500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55718/0 (socket says 172.21.15.62:55718) 2024-06-10T00:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.227+0000 7fabae59c700 1 -- 172.21.15.62:0/3932278335 learned_addr learned my addr 172.21.15.62:0/3932278335 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.227+0000 7fabadd9b700 1 --2- 172.21.15.62:0/3932278335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 0x55dfc653d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.228+0000 7fabae59c700 1 -- 172.21.15.62:0/3932278335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc653a830 msgr2=0x55dfc653dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.228+0000 7fabae59c700 1 --2- 172.21.15.62:0/3932278335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc653a830 0x55dfc653dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.228+0000 7fabae59c700 1 -- 172.21.15.62:0/3932278335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 msgr2=0x55dfc653d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.228+0000 7fabae59c700 1 --2- 172.21.15.62:0/3932278335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 0x55dfc653d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.228+0000 7fabae59c700 1 -- 172.21.15.62:0/3932278335 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfc644b700 con 0x55dfc6534750 2024-06-10T00:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.228+0000 7fabae59c700 1 --2- 172.21.15.62:0/3932278335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6534750 0x55dfc6540500 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faba000c010 tx=0x7faba0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d1e651d788f226 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.228+0000 7fab95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.228+0000 7fab95ffb700 1 -- 172.21.15.62:0/3932278335 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba000f870 con 0x55dfc6534750 2024-06-10T00:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.228+0000 7fab95ffb700 1 -- 172.21.15.62:0/3932278335 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7faba000fa10 con 0x55dfc6534750 2024-06-10T00:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.228+0000 7fabbf43bb80 1 -- 172.21.15.62:0/3932278335 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba809d070 con 0x55dfc6534750 2024-06-10T00:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.228+0000 7fab95ffb700 1 -- 172.21.15.62:0/3932278335 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faba003d050 con 0x55dfc6534750 2024-06-10T00:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.228+0000 7fabbf43bb80 10 client.15303 fetch_fsmap learned FSMap version 29 2024-06-10T00:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.228+0000 7fabbf43bb80 1 -- 172.21.15.62:0/3932278335 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dfc654b930 con 0x55dfc6534750 2024-06-10T00:32:40.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.228+0000 7fab95ffb700 1 -- 172.21.15.62:0/3932278335 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba809d070 con 0x55dfc6534750 2024-06-10T00:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.229+0000 7fab95ffb700 1 -- 172.21.15.62:0/3932278335 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faba0024b20 con 0x55dfc6534750 2024-06-10T00:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.229+0000 7fabbf43bb80 10 client.15303 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.229+0000 7fabbf43bb80 10 client.15303 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T00:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.229+0000 7fabbf43bb80 20 client.15303 populate_metadata read hostname 'smithi062' 2024-06-10T00:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.229+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/3932278335 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfc654e050 0x55dfc654f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.229+0000 7fabbf43bb80 4 client.15303 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.229+0000 7fabbf43bb80 1 -- 172.21.15.62:0/3932278335 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfc6492d00 con 0x55dfc654e050 2024-06-10T00:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.229+0000 7fabadd9b700 1 --2- 172.21.15.62:0/3932278335 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfc654e050 0x55dfc654f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.229+0000 7fab95ffb700 10 client.15303 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.229+0000 7fabadd9b700 1 --2- 172.21.15.62:0/3932278335 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfc654e050 0x55dfc654f4c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77c3f4d064ed38dd server_cookie=dfdda5dca1f57f01 in_seq=0 out_seq=0 2024-06-10T00:32:40.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.230+0000 7fab95ffb700 1 -- 172.21.15.62:0/3932278335 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dfc6492d00 con 0x55dfc654e050 2024-06-10T00:32:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.230+0000 7fab95ffb700 10 client.15303 handle_command_reply: tid=0 2024-06-10T00:32:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.230+0000 7fabbf43bb80 1 client.15303 shutdown 2024-06-10T00:32:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.230+0000 7fabbf43bb80 1 -- 172.21.15.62:0/3932278335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6534750 msgr2=0x55dfc6540500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.230+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/3932278335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6534750 0x55dfc6540500 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faba000c010 tx=0x7faba0009ca0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.230+0000 7fabbf43bb80 1 -- 172.21.15.62:0/3932278335 shutdown_connections 2024-06-10T00:32:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.230+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/3932278335 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfc654e050 0x55dfc654f4c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.230+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/3932278335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6534750 0x55dfc6540500 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.230+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/3932278335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc653a830 0x55dfc653dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.230+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/3932278335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 0x55dfc653d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.230+0000 7fabbf43bb80 1 -- 172.21.15.62:0/3932278335 >> 172.21.15.62:0/3932278335 conn(0x55dfc6461030 msgr2=0x55dfc652e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:40.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.230+0000 7fab95ffb700 0 client.15303 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:32:40.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.230+0000 7fabbf43bb80 1 -- 172.21.15.62:0/3932278335 shutdown_connections 2024-06-10T00:32:40.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.230+0000 7fabbf43bb80 1 -- 172.21.15.62:0/3932278335 wait complete. 2024-06-10T00:32:40.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.231+0000 7fabbf43bb80 20 client.15303 trim_cache size 0 max 16384 2024-06-10T00:32:40.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.252+0000 7fabbf43bb80 1 Processor -- start 2024-06-10T00:32:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.252+0000 7fabbf43bb80 1 -- start start 2024-06-10T00:32:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.252+0000 7fabbf43bb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc6534750 0x55dfc6534210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.252+0000 7fabbf43bb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 0x55dfc652ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.252+0000 7fabbf43bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc653a830 0x55dfc652f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.252+0000 7fabbf43bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55dfc654d800 con 0x55dfc653a830 2024-06-10T00:32:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.252+0000 7fabbf43bb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dfc654ded0 con 0x55dfc6539f10 2024-06-10T00:32:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.252+0000 7fabbf43bb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dfc654e5a0 con 0x55dfc6534750 2024-06-10T00:32:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.252+0000 7fabaed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc653a830 0x55dfc652f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.253+0000 7fabaed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc653a830 0x55dfc652f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55728/0 (socket says 172.21.15.62:55728) 2024-06-10T00:32:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.253+0000 7fabaed9d700 1 -- 172.21.15.62:0/24498150 learned_addr learned my addr 172.21.15.62:0/24498150 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.253+0000 7fabadd9b700 1 --2- 172.21.15.62:0/24498150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 0x55dfc652ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.253+0000 7fabae59c700 1 --2- 172.21.15.62:0/24498150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc6534750 0x55dfc6534210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.253+0000 7fabaed9d700 1 -- 172.21.15.62:0/24498150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc6534750 msgr2=0x55dfc6534210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.253+0000 7fabaed9d700 1 --2- 172.21.15.62:0/24498150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc6534750 0x55dfc6534210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.253+0000 7fabaed9d700 1 -- 172.21.15.62:0/24498150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 msgr2=0x55dfc652ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.253+0000 7fabaed9d700 1 --2- 172.21.15.62:0/24498150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 0x55dfc652ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.253+0000 7fabaed9d700 1 -- 172.21.15.62:0/24498150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfc653bd90 con 0x55dfc653a830 2024-06-10T00:32:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.253+0000 7fabaed9d700 1 --2- 172.21.15.62:0/24498150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc653a830 0x55dfc652f370 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faba800a0a0 tx=0x7faba8007a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336b2dcf356d2486 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.253+0000 7fab97fff700 1 -- 172.21.15.62:0/24498150 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba809f200 con 0x55dfc653a830 2024-06-10T00:32:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.253+0000 7fab97fff700 1 -- 172.21.15.62:0/24498150 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba809d070 con 0x55dfc653a830 2024-06-10T00:32:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.253+0000 7fab97fff700 1 -- 172.21.15.62:0/24498150 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba80a3800 con 0x55dfc653a830 2024-06-10T00:32:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 -- 172.21.15.62:0/24498150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc653a830 msgr2=0x55dfc652f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/24498150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc653a830 0x55dfc652f370 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faba800a0a0 tx=0x7faba8007a70 comp rx=0 tx=0).stop 2024-06-10T00:32:40.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 -- 172.21.15.62:0/24498150 shutdown_connections 2024-06-10T00:32:40.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/24498150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc653a830 0x55dfc652f370 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/24498150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc6534750 0x55dfc6534210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/24498150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc6539f10 0x55dfc652ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 -- 172.21.15.62:0/24498150 >> 172.21.15.62:0/24498150 conn(0x55dfc6461030 msgr2=0x55dfc653c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:40.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 -- 172.21.15.62:0/24498150 shutdown_connections 2024-06-10T00:32:40.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 -- 172.21.15.62:0/24498150 wait complete. 2024-06-10T00:32:40.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 -- start start 2024-06-10T00:32:40.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 Processor -- start 2024-06-10T00:32:40.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc6534750 0x55dfc652e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6539f10 0x55dfc652e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc653a830 0x55dfc6557930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55dfc654f340 con 0x55dfc6539f10 2024-06-10T00:32:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dfc654ec70 con 0x55dfc653a830 2024-06-10T00:32:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.254+0000 7fabbf43bb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dfc654fa10 con 0x55dfc6534750 2024-06-10T00:32:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fabbf43bb80 10 client.0 _ll_register_callbacks cb 0x55dfc641b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:32:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fabadd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6539f10 0x55dfc652e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fabadd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6539f10 0x55dfc652e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55740/0 (socket says 172.21.15.62:55740) 2024-06-10T00:32:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fabadd9b700 1 -- 172.21.15.62:0/2126653746 learned_addr learned my addr 172.21.15.62:0/2126653746 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fabae59c700 1 --2- 172.21.15.62:0/2126653746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc6534750 0x55dfc652e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fabaed9d700 1 --2- 172.21.15.62:0/2126653746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc653a830 0x55dfc6557930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fabadd9b700 1 -- 172.21.15.62:0/2126653746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc6534750 msgr2=0x55dfc652e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fabadd9b700 1 --2- 172.21.15.62:0/2126653746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc6534750 0x55dfc652e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fabadd9b700 1 -- 172.21.15.62:0/2126653746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc653a830 msgr2=0x55dfc6557930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:32:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fabadd9b700 1 --2- 172.21.15.62:0/2126653746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc653a830 0x55dfc6557930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fabadd9b700 1 -- 172.21.15.62:0/2126653746 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfc6557e70 con 0x55dfc6539f10 2024-06-10T00:32:40.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fabadd9b700 1 --2- 172.21.15.62:0/2126653746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6539f10 0x55dfc652e540 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faba4004790 tx=0x7faba4004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abe3a4812d1147d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:40.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fab95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:32:40.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fab95ffb700 1 -- 172.21.15.62:0/2126653746 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4023070 con 0x55dfc6539f10 2024-06-10T00:32:40.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fab95ffb700 1 -- 172.21.15.62:0/2126653746 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7faba4003880 con 0x55dfc6539f10 2024-06-10T00:32:40.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.255+0000 7fabbf43bb80 1 -- 172.21.15.62:0/2126653746 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba809d070 con 0x55dfc6539f10 2024-06-10T00:32:40.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.256+0000 7fab95ffb700 1 -- 172.21.15.62:0/2126653746 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faba402f050 con 0x55dfc6539f10 2024-06-10T00:32:40.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.256+0000 7fabbf43bb80 10 client.15309 fetch_fsmap learned FSMap version 29 2024-06-10T00:32:40.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.256+0000 7fabbf43bb80 1 -- 172.21.15.62:0/2126653746 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dfc6559190 con 0x55dfc6539f10 2024-06-10T00:32:40.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.256+0000 7fab95ffb700 1 -- 172.21.15.62:0/2126653746 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba809d070 con 0x55dfc6539f10 2024-06-10T00:32:40.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.256+0000 7fab95ffb700 1 -- 172.21.15.62:0/2126653746 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faba4010430 con 0x55dfc6539f10 2024-06-10T00:32:40.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.256+0000 7fabbf43bb80 10 client.15309 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T00:32:40.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.256+0000 7fabbf43bb80 10 client.15309 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T00:32:40.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.256+0000 7fabbf43bb80 20 client.15309 populate_metadata read hostname 'smithi062' 2024-06-10T00:32:40.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.256+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/2126653746 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfc65bded0 0x55dfc65bcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:40.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.256+0000 7fabbf43bb80 4 client.15309 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:32:40.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.256+0000 7fabbf43bb80 1 -- 172.21.15.62:0/2126653746 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "status"}) -- 0x55dfc648d280 con 0x55dfc65bded0 2024-06-10T00:32:40.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.256+0000 7fabae59c700 1 --2- 172.21.15.62:0/2126653746 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfc65bded0 0x55dfc65bcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:40.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fabae59c700 1 --2- 172.21.15.62:0/2126653746 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfc65bded0 0x55dfc65bcbe0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e55224d952b9e61 server_cookie=ac0aee27d30a0d3a in_seq=0 out_seq=0 2024-06-10T00:32:40.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fab95ffb700 10 client.15309 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:32:40.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fab95ffb700 1 -- 172.21.15.62:0/2126653746 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55dfc648d280 con 0x55dfc65bded0 2024-06-10T00:32:40.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fab95ffb700 10 client.15309 handle_command_reply: tid=0 2024-06-10T00:32:40.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fabbf43bb80 1 client.15309 shutdown 2024-06-10T00:32:40.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fabbf43bb80 1 -- 172.21.15.62:0/2126653746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6539f10 msgr2=0x55dfc652e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:40.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/2126653746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6539f10 0x55dfc652e540 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faba4004790 tx=0x7faba4004560 comp rx=0 tx=0).stop 2024-06-10T00:32:40.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fabbf43bb80 1 -- 172.21.15.62:0/2126653746 shutdown_connections 2024-06-10T00:32:40.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/2126653746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfc653a830 0x55dfc6557930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/2126653746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfc6534750 0x55dfc652e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/2126653746 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfc65bded0 0x55dfc65bcbe0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fabbf43bb80 1 --2- 172.21.15.62:0/2126653746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfc6539f10 0x55dfc652e540 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fab95ffb700 0 client.15309 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:32:40.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fabbf43bb80 1 -- 172.21.15.62:0/2126653746 >> 172.21.15.62:0/2126653746 conn(0x55dfc6461030 msgr2=0x55dfc653ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:40.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fabbf43bb80 1 -- 172.21.15.62:0/2126653746 shutdown_connections 2024-06-10T00:32:40.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fabbf43bb80 1 -- 172.21.15.62:0/2126653746 wait complete. 2024-06-10T00:32:40.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.257+0000 7fabbf43bb80 20 client.15309 trim_cache size 0 max 16384 2024-06-10T00:32:40.264 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:32:40.264 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:32:40.264 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-06-10T00:32:40.264 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24464, 2024-06-10T00:32:40.264 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:32:40.264 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:32:40.265 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:32:40.265 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 240.61259426500001, 2024-06-10T00:32:40.265 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 29, 2024-06-10T00:32:40.265 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 99, 2024-06-10T00:32:40.265 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 97, 2024-06-10T00:32:40.265 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 258.33772898699999 2024-06-10T00:32:40.265 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:32:40.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.258+0000 7fabb4bb8700 1 -- 172.21.15.62:0/3684964928 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fab980a2c20 msgr2=0x7fab980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:40.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.258+0000 7fabb4bb8700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fab980a2c20 0x7fab980a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faba000b5c0 tx=0x7faba0005fd0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.258+0000 7fabb4bb8700 1 -- 172.21.15.62:0/3684964928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb010ee90 msgr2=0x7fabb019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:40.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.258+0000 7fabb4bb8700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb010ee90 0x7fabb019f750 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7faba800ab40 tx=0x7faba800ae50 comp rx=0 tx=0).stop 2024-06-10T00:32:40.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.258+0000 7fabb4bb8700 1 -- 172.21.15.62:0/3684964928 shutdown_connections 2024-06-10T00:32:40.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.258+0000 7fabb4bb8700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabb010ee90 0x7fabb019f750 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.258+0000 7fabb4bb8700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fab980a2c20 0x7fab980a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.258+0000 7fabb4bb8700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fabb00746c0 0x7fabb019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.258+0000 7fabb4bb8700 1 --2- 172.21.15.62:0/3684964928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb0074fe0 0x7fabb019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:40.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.258+0000 7fabb4bb8700 1 -- 172.21.15.62:0/3684964928 >> 172.21.15.62:0/3684964928 conn(0x7fabb00795f0 msgr2=0x7fabb00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:40.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.258+0000 7fabb4bb8700 1 -- 172.21.15.62:0/3684964928 shutdown_connections 2024-06-10T00:32:40.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:40.258+0000 7fabb4bb8700 1 -- 172.21.15.62:0/3684964928 wait complete. 2024-06-10T00:32:41.078 INFO:teuthology.run_tasks:Running task sequential... 2024-06-10T00:32:41.088 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-10T00:32:41.088 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-10T00:32:41.089 INFO:teuthology.task.print:Enabling mgr modules 2024-06-10T00:32:41.089 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-06-10T00:32:41.090 INFO:teuthology.task.exec:Executing custom commands... 2024-06-10T00:32:41.091 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-06-10T00:32:41.091 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-06-10T00:32:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.206+0000 7f0a7bd14640 1 -- 172.21.15.62:0/3276040369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a74102bf0 msgr2=0x7f0a74103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.206+0000 7f0a7bd14640 1 --2- 172.21.15.62:0/3276040369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a74102bf0 0x7f0a74103070 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009900 tx=0x7f0a6402a410 comp rx=0 tx=0).stop 2024-06-10T00:32:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.206+0000 7f0a7bd14640 1 -- 172.21.15.62:0/3276040369 shutdown_connections 2024-06-10T00:32:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.206+0000 7f0a7bd14640 1 --2- 172.21.15.62:0/3276040369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a741035b0 0x7f0a74109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.206+0000 7f0a7bd14640 1 --2- 172.21.15.62:0/3276040369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a74102bf0 0x7f0a74103070 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.206+0000 7f0a7bd14640 1 --2- 172.21.15.62:0/3276040369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a74101a00 0x7f0a74101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.206+0000 7f0a7bd14640 1 -- 172.21.15.62:0/3276040369 >> 172.21.15.62:0/3276040369 conn(0x7f0a740fcf90 msgr2=0x7f0a740ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.206+0000 7f0a7bd14640 1 -- 172.21.15.62:0/3276040369 shutdown_connections 2024-06-10T00:32:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.206+0000 7f0a7bd14640 1 -- 172.21.15.62:0/3276040369 wait complete. 2024-06-10T00:32:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.206+0000 7f0a7bd14640 1 Processor -- start 2024-06-10T00:32:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.206+0000 7f0a7bd14640 1 -- start start 2024-06-10T00:32:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a7bd14640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a74101a00 0x7f0a74197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a7bd14640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a74102bf0 0x7f0a74198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a7bd14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a741035b0 0x7f0a7419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a7bd14640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7410d680 con 0x7f0a74101a00 2024-06-10T00:32:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a7bd14640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7410d500 con 0x7f0a741035b0 2024-06-10T00:32:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a7bd14640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7410d800 con 0x7f0a74102bf0 2024-06-10T00:32:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a79ab0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a74101a00 0x7f0a74197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a79ab0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a74101a00 0x7f0a74197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55768/0 (socket says 172.21.15.62:55768) 2024-06-10T00:32:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a79ab0640 1 -- 172.21.15.62:0/4265175959 learned_addr learned my addr 172.21.15.62:0/4265175959 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a792af640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a74102bf0 0x7f0a74198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a7a2b1640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a741035b0 0x7f0a7419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a79ab0640 1 -- 172.21.15.62:0/4265175959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a74102bf0 msgr2=0x7f0a74198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a79ab0640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a74102bf0 0x7f0a74198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a79ab0640 1 -- 172.21.15.62:0/4265175959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a741035b0 msgr2=0x7f0a7419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a79ab0640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a741035b0 0x7f0a7419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a79ab0640 1 -- 172.21.15.62:0/4265175959 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7419f840 con 0x7f0a74101a00 2024-06-10T00:32:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a7a2b1640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a741035b0 0x7f0a7419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:32:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a79ab0640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a74101a00 0x7f0a74197c30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000d770 tx=0x7f0a7000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a6affd640 1 -- 172.21.15.62:0/4265175959 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70017070 con 0x7f0a74101a00 2024-06-10T00:32:41.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a6affd640 1 -- 172.21.15.62:0/4265175959 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a70009c30 con 0x7f0a74101a00 2024-06-10T00:32:41.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a7bd14640 1 -- 172.21.15.62:0/4265175959 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7419fb30 con 0x7f0a74101a00 2024-06-10T00:32:41.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a6affd640 1 -- 172.21.15.62:0/4265175959 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70003dd0 con 0x7f0a74101a00 2024-06-10T00:32:41.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.207+0000 7f0a7bd14640 1 -- 172.21.15.62:0/4265175959 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a741a0080 con 0x7f0a74101a00 2024-06-10T00:32:41.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.209+0000 7f0a7bd14640 1 -- 172.21.15.62:0/4265175959 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a74108c50 con 0x7f0a74101a00 2024-06-10T00:32:41.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.210+0000 7f0a6affd640 1 -- 172.21.15.62:0/4265175959 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0a70003f70 con 0x7f0a74101a00 2024-06-10T00:32:41.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.210+0000 7f0a6affd640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0a600a2c30 0x7f0a600a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:41.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.211+0000 7f0a792af640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0a600a2c30 0x7f0a600a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:41.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.211+0000 7f0a6affd640 1 -- 172.21.15.62:0/4265175959 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f0a700cef00 con 0x7f0a74101a00 2024-06-10T00:32:41.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.211+0000 7f0a792af640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0a600a2c30 0x7f0a600a50f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a64000c00 tx=0x7f0a640315d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:41.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.215+0000 7f0a6affd640 1 -- 172.21.15.62:0/4265175959 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0a7009ba90 con 0x7f0a74101a00 2024-06-10T00:32:41.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:41.451+0000 7f0a7bd14640 1 -- 172.21.15.62:0/4265175959 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f0a74062640 con 0x7f0a74101a00 2024-06-10T00:32:41.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:41 smithi105 ceph-mon[36321]: pgmap v428: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:41.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:41 smithi062 ceph-mon[32390]: pgmap v428: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:41.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:41 smithi138 ceph-mon[36549]: pgmap v428: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-10T00:32:42.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:42.401+0000 7f0a6affd640 1 -- 172.21.15.62:0/4265175959 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f0a70009de0 con 0x7f0a74101a00 2024-06-10T00:32:42.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:42.405+0000 7f0a7bd14640 1 -- 172.21.15.62:0/4265175959 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0a600a2c30 msgr2=0x7f0a600a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:42.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:42.405+0000 7f0a7bd14640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0a600a2c30 0x7f0a600a50f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a64000c00 tx=0x7f0a640315d0 comp rx=0 tx=0).stop 2024-06-10T00:32:42.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:42.405+0000 7f0a7bd14640 1 -- 172.21.15.62:0/4265175959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a74101a00 msgr2=0x7f0a74197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:42.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:42.405+0000 7f0a7bd14640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a74101a00 0x7f0a74197c30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000d770 tx=0x7f0a7000dc30 comp rx=0 tx=0).stop 2024-06-10T00:32:42.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:42.405+0000 7f0a7bd14640 1 -- 172.21.15.62:0/4265175959 shutdown_connections 2024-06-10T00:32:42.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:42.405+0000 7f0a7bd14640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a741035b0 0x7f0a7419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:42.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:42.405+0000 7f0a7bd14640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f0a600a2c30 0x7f0a600a50f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:42.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:42.405+0000 7f0a7bd14640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a74102bf0 0x7f0a74198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:42.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:42.405+0000 7f0a7bd14640 1 --2- 172.21.15.62:0/4265175959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a74101a00 0x7f0a74197c30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:42.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:42.405+0000 7f0a7bd14640 1 -- 172.21.15.62:0/4265175959 >> 172.21.15.62:0/4265175959 conn(0x7f0a740fcf90 msgr2=0x7f0a74105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:42.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:42.405+0000 7f0a7bd14640 1 -- 172.21.15.62:0/4265175959 shutdown_connections 2024-06-10T00:32:42.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:42.405+0000 7f0a7bd14640 1 -- 172.21.15.62:0/4265175959 wait complete. 2024-06-10T00:32:42.424 INFO:teuthology.run_tasks:Running task kclient... 2024-06-10T00:32:42.438 INFO:tasks.kclient:Mounting kernel clients... 2024-06-10T00:32:42.438 INFO:tasks.kclient:config is {'client.0': {}} 2024-06-10T00:32:42.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:32:42.439 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-06-10T00:32:42.457 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-06-10T00:32:42.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:32:42.458 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T00:32:42.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:32:42.525 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-06-10T00:32:42.633 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-06-10T00:32:42.633 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-10T00:32:42.634 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-10T00:32:42.634 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-10T00:32:42.647 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:42 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/4265175959' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-10T00:32:42.648 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:42 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: ignoring --setuser ceph since I am not root 2024-06-10T00:32:42.648 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:42 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: ignoring --setgroup ceph since I am not root 2024-06-10T00:32:42.648 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:42 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:42.600+0000 7f47305dd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:32:42.764 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:42 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/4265175959' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-10T00:32:42.764 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:42 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: ignoring --setuser ceph since I am not root 2024-06-10T00:32:42.764 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:42 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: ignoring --setgroup ceph since I am not root 2024-06-10T00:32:42.765 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:42 smithi105 ceph-mgr[37698]: -- 172.21.15.105:0/1416444077 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f3bb425a0 con 0x560f3bb16c00 2024-06-10T00:32:42.765 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:42 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:42.582+0000 7fb63cde0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:32:42.765 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:42 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:42.764+0000 7fb63cde0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:32:42.765 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:42 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/4265175959' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-10T00:32:42.765 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:42 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: ignoring --setuser ceph since I am not root 2024-06-10T00:32:42.765 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:42 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: ignoring --setgroup ceph since I am not root 2024-06-10T00:32:42.766 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:42 smithi138 ceph-mgr[38384]: -- 172.21.15.138:0/969448331 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5567920825a0 con 0x556792056c00 2024-06-10T00:32:42.766 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:42 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:42.581+0000 7f94b63b0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:32:42.766 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:42 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:42.764+0000 7f94b63b0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:32:42.900 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:42.935 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:42 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:42.801+0000 7f47305dd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:32:42.935 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:42 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:42.935+0000 7f47305dd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:32:43.032 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:42 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:42.864+0000 7fb63cde0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:32:43.123 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:42 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:42.867+0000 7f94b63b0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:32:43.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:43 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/4265175959' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-10T00:32:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:43 smithi105 ceph-mon[36321]: mgrmap e19: x(active, since 11m), standbys: y, z 2024-06-10T00:32:43.532 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:43 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:43.168+0000 7fb63cde0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:32:43.532 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:43 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:43.253+0000 7fb63cde0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:32:43.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:43 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/4265175959' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-10T00:32:43.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:43 smithi062 ceph-mon[32390]: mgrmap e19: x(active, since 11m), standbys: y, z 2024-06-10T00:32:43.608 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:43 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:43.266+0000 7f47305dd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:32:43.608 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:43 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:43.360+0000 7f47305dd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:32:43.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:43 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/4265175959' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-10T00:32:43.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:43 smithi138 ceph-mon[36549]: mgrmap e19: x(active, since 11m), standbys: y, z 2024-06-10T00:32:43.623 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:43 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:43.179+0000 7f94b63b0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:32:43.623 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:43 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:43.265+0000 7f94b63b0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:32:44.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.701+0000 7f04b5912700 1 -- 172.21.15.62:0/2648817976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04b01084f0 msgr2=0x7f04b01088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:44.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.701+0000 7f04b5912700 1 --2- 172.21.15.62:0/2648817976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04b01084f0 0x7f04b01088d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009990 tx=0x7f04a0009ca0 comp rx=0 tx=0).stop 2024-06-10T00:32:44.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.701+0000 7f04b5912700 1 -- 172.21.15.62:0/2648817976 shutdown_connections 2024-06-10T00:32:44.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.701+0000 7f04b5912700 1 --2- 172.21.15.62:0/2648817976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04b01084f0 0x7f04b01088d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:44.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.701+0000 7f04b5912700 1 --2- 172.21.15.62:0/2648817976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b006b3c0 0x7f04b010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:44.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.701+0000 7f04b5912700 1 --2- 172.21.15.62:0/2648817976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b006aa00 0x7f04b006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:44.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.701+0000 7f04b5912700 1 -- 172.21.15.62:0/2648817976 >> 172.21.15.62:0/2648817976 conn(0x7f04b0078300 msgr2=0x7f04b0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:44.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.701+0000 7f04b5912700 1 -- 172.21.15.62:0/2648817976 shutdown_connections 2024-06-10T00:32:44.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.701+0000 7f04b5912700 1 -- 172.21.15.62:0/2648817976 wait complete. 2024-06-10T00:32:44.703 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:44 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:44.648+0000 7f47305dd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:32:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.702+0000 7f04b5912700 1 Processor -- start 2024-06-10T00:32:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.702+0000 7f04b5912700 1 -- start start 2024-06-10T00:32:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04b5912700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b006aa00 0x7f04b019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04b5912700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b006b3c0 0x7f04b019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04b5912700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04b01084f0 0x7f04b0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04ae7fc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b006b3c0 0x7f04b019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04ae7fc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b006b3c0 0x7f04b019ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:59834/0 (socket says 172.21.15.62:59834) 2024-06-10T00:32:44.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04ae7fc700 1 -- 172.21.15.62:0/3463893882 learned_addr learned my addr 172.21.15.62:0/3463893882 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:44.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04b5912700 1 -- 172.21.15.62:0/3463893882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f04b0112770 con 0x7f04b01084f0 2024-06-10T00:32:44.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04b5912700 1 -- 172.21.15.62:0/3463893882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01125f0 con 0x7f04b006b3c0 2024-06-10T00:32:44.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04b5912700 1 -- 172.21.15.62:0/3463893882 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01128f0 con 0x7f04b006aa00 2024-06-10T00:32:44.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04af7fe700 1 --2- 172.21.15.62:0/3463893882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04b01084f0 0x7f04b0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:44.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04aeffd700 1 --2- 172.21.15.62:0/3463893882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b006aa00 0x7f04b019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:44.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04ae7fc700 1 -- 172.21.15.62:0/3463893882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b006aa00 msgr2=0x7f04b019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:44.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04ae7fc700 1 --2- 172.21.15.62:0/3463893882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b006aa00 0x7f04b019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:44.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04ae7fc700 1 -- 172.21.15.62:0/3463893882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04b01084f0 msgr2=0x7f04b0198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:44.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04ae7fc700 1 --2- 172.21.15.62:0/3463893882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04b01084f0 0x7f04b0198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:44.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.703+0000 7f04ae7fc700 1 -- 172.21.15.62:0/3463893882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b0199230 con 0x7f04b006b3c0 2024-06-10T00:32:44.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.704+0000 7f04ae7fc700 1 --2- 172.21.15.62:0/3463893882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b006b3c0 0x7f04b019ee50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f04a400e9c0 tx=0x7f04a400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:44.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.704+0000 7f04b4910700 1 -- 172.21.15.62:0/3463893882 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a400cbd0 con 0x7f04b006b3c0 2024-06-10T00:32:44.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.704+0000 7f04b4910700 1 -- 172.21.15.62:0/3463893882 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04a400ee60 con 0x7f04b006b3c0 2024-06-10T00:32:44.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.704+0000 7f04b5912700 1 -- 172.21.15.62:0/3463893882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b0199520 con 0x7f04b006b3c0 2024-06-10T00:32:44.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.704+0000 7f04b4910700 1 -- 172.21.15.62:0/3463893882 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4005630 con 0x7f04b006b3c0 2024-06-10T00:32:44.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.704+0000 7f04b5912700 1 -- 172.21.15.62:0/3463893882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b0101380 con 0x7f04b006b3c0 2024-06-10T00:32:44.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.706+0000 7f04b5912700 1 -- 172.21.15.62:0/3463893882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b010a9d0 con 0x7f04b006b3c0 2024-06-10T00:32:44.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.707+0000 7f04b4910700 1 -- 172.21.15.62:0/3463893882 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f04a4004830 con 0x7f04b006b3c0 2024-06-10T00:32:44.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.708+0000 7f04b4910700 1 --2- 172.21.15.62:0/3463893882 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04980a2c70 0x7f04980a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:44.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.708+0000 7f04aeffd700 1 -- 172.21.15.62:0/3463893882 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04980a2c70 msgr2=0x7f04980a5130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3602410963 2024-06-10T00:32:44.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.708+0000 7f04b4910700 1 -- 172.21.15.62:0/3463893882 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f04a40cf730 con 0x7f04b006b3c0 2024-06-10T00:32:44.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.708+0000 7f04aeffd700 1 --2- 172.21.15.62:0/3463893882 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04980a2c70 0x7f04980a5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:32:44.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.712+0000 7f04b4910700 1 -- 172.21.15.62:0/3463893882 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f04a409c2c0 con 0x7f04b006b3c0 2024-06-10T00:32:44.760 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:44 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:44.424+0000 7fb63cde0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:32:44.761 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:44 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:44.592+0000 7fb63cde0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:32:44.783 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:44 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:44.444+0000 7f94b63b0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:32:44.783 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:44 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:44.613+0000 7f94b63b0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:32:44.783 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:44 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:44.782+0000 7f94b63b0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:32:44.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.909+0000 7f04aeffd700 1 -- 172.21.15.62:0/3463893882 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04980a2c70 msgr2=0x7f04980a5130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3602410963 2024-06-10T00:32:44.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.909+0000 7f04aeffd700 1 --2- 172.21.15.62:0/3463893882 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04980a2c70 0x7f04980a5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T00:32:44.992 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:44 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:44.855+0000 7f47305dd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:32:44.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.991+0000 7f04b5912700 1 -- 172.21.15.62:0/3463893882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f04b0199fc0 con 0x7f04b006b3c0 2024-06-10T00:32:44.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.992+0000 7f04b4910700 1 -- 172.21.15.62:0/3463893882 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f04a409bbc0 con 0x7f04b006b3c0 2024-06-10T00:32:44.993 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-10T00:32:44.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.996+0000 7f04b5912700 1 -- 172.21.15.62:0/3463893882 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04980a2c70 msgr2=0x7f04980a5130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:32:44.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.996+0000 7f04b5912700 1 --2- 172.21.15.62:0/3463893882 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04980a2c70 0x7f04980a5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.996+0000 7f04b5912700 1 -- 172.21.15.62:0/3463893882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b006b3c0 msgr2=0x7f04b019ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.996+0000 7f04b5912700 1 --2- 172.21.15.62:0/3463893882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b006b3c0 0x7f04b019ee50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f04a400e9c0 tx=0x7f04a400ecd0 comp rx=0 tx=0).stop 2024-06-10T00:32:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.996+0000 7f04b5912700 1 -- 172.21.15.62:0/3463893882 shutdown_connections 2024-06-10T00:32:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.996+0000 7f04b5912700 1 --2- 172.21.15.62:0/3463893882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04b01084f0 0x7f04b0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.996+0000 7f04b5912700 1 --2- 172.21.15.62:0/3463893882 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f04980a2c70 0x7f04980a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.996+0000 7f04b5912700 1 --2- 172.21.15.62:0/3463893882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b006aa00 0x7f04b019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.996+0000 7f04b5912700 1 --2- 172.21.15.62:0/3463893882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04b006b3c0 0x7f04b019ee50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.996+0000 7f04b5912700 1 -- 172.21.15.62:0/3463893882 >> 172.21.15.62:0/3463893882 conn(0x7f04b0078300 msgr2=0x7f04b0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.996+0000 7f04b5912700 1 -- 172.21.15.62:0/3463893882 shutdown_connections 2024-06-10T00:32:44.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:44.996+0000 7f04b5912700 1 -- 172.21.15.62:0/3463893882 wait complete. 2024-06-10T00:32:45.032 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:44 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:44.760+0000 7fb63cde0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:32:45.342 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:45 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3463893882' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T00:32:45.343 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:45 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:45.049+0000 7fb63cde0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:32:45.343 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:45 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:45.142+0000 7fb63cde0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:32:45.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:45 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3463893882' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T00:32:45.344 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:45 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:45.052+0000 7f47305dd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:32:45.371 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:45 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3463893882' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T00:32:45.372 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:45 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:45.074+0000 7f94b63b0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:32:45.372 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:45 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:45.167+0000 7f94b63b0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:32:45.372 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:45 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:45.371+0000 7f94b63b0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:32:45.607 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:45 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:45.363+0000 7f47305dd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:32:45.608 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:45 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:45.463+0000 7f47305dd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:32:45.623 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:45 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:45.564+0000 7f94b63b0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:32:45.781 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:45 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:45.342+0000 7fb63cde0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:32:45.782 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:45 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:45.535+0000 7fb63cde0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:32:45.783 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-10T00:32:45.783 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-10T00:32:45.783 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-06-10T00:32:45.783 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-10T00:32:45.783 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-10T00:32:45.783 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-10T00:32:45.783 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-10T00:32:45.783 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-10T00:32:45.783 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-10T00:32:45.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:32:45.783 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-06-10T00:32:45.840 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-10T00:32:45.840 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-10T00:32:45.840 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-06-10T00:32:45.840 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-06-10T00:32:45.840 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-06-10T00:32:45.841 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-06-10T00:32:45.841 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T00:32:45.841 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:32:45.841 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-06-10T00:32:45.841 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T00:32:45.841 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:32:45.841 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-06-10T00:32:45.841 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T00:32:45.841 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:32:45.841 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-10T00:32:45.841 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:32:45.841 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-06-10T00:32:45.842 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41967sec preferred_lft 41967sec 2024-06-10T00:32:45.842 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-06-10T00:32:45.842 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-06-10T00:32:45.842 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-10T00:32:45.842 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:32:45.842 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-06-10T00:32:45.842 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2024-06-10T00:32:45.842 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2024-06-10T00:32:45.842 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2024-06-10T00:32:45.842 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-10T00:32:45.843 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-06-10T00:32:45.900 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:45 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:45.679+0000 7f47305dd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:32:45.900 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:45 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:45.900+0000 7f47305dd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:32:45.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:32:45.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:32:45.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:32:45.990 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-10T00:32:46.055 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-06-10T00:32:46.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:32:46.057 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-06-10T00:32:46.115 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-06-10T00:32:46.115 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-10T00:32:46.115 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-10T00:32:46.115 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-10T00:32:46.115 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-10T00:32:46.115 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:32:46.116 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-06-10T00:32:46.116 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-10T00:32:46.116 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-10T00:32:46.116 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-10T00:32:46.116 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-06-10T00:32:46.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:32:46.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:32:46.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:32:46.249 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-06-10T00:32:46.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:32:46.266 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-06-10T00:32:46.323 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:32:46.323 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-06-10T00:32:46.324 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-10T00:32:46.324 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-10T00:32:46.324 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-06-10T00:32:46.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:32:46.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:32:46.448 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-10T00:32:46.449 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:32:46.449 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-06-10T00:32:46.449 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-10T00:32:46.449 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-10T00:32:46.449 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-10T00:32:46.449 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-10T00:32:46.449 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-10T00:32:46.449 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-06-10T00:32:46.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:32:46.531 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:46 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:46.132+0000 7fb63cde0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:32:46.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:32:46.572 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:32:46.572 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-06-10T00:32:46.572 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-06-10T00:32:46.572 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2024-06-10T00:32:46.572 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-06-10T00:32:46.590 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:46 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:46.167+0000 7f94b63b0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:32:46.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:32:46.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:32:46.654 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T00:32:46.654 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-10T00:32:46.670 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-10T00:32:46.670 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T00:32:46.670 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-10T00:32:46.726 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-06-10T00:32:46.726 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-06-10T00:32:46.726 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/4ba2ab46-adb2-4be3-aedd-0c97cb2421cc 2024-06-10T00:32:46.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:32:46.727 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/4ba2ab46-adb2-4be3-aedd-0c97cb2421cc /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-06-10T00:32:46.789 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:46 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:46.569+0000 7f47305dd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:32:46.812 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-06-10T00:32:46.813 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-06-10T00:32:46.813 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-06-10T00:32:46.844 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:46 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:46.548+0000 7fb63cde0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:32:46.845 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:46 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:46.645+0000 7fb63cde0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:32:46.873 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:46 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:46.589+0000 7f94b63b0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:32:46.873 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:46 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:46.686+0000 7f94b63b0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:32:46.973 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-06-10T00:32:46.973 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-06-10T00:32:46.973 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-06-10T00:32:46.973 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.105:3300,172.21.15.138:3300" 2024-06-10T00:32:46.974 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.62:3300,172.21.15.105:3300,172.21.15.138:3300:/volumes/qa/sv_1/4ba2ab46-adb2-4be3-aedd-0c97cb2421cc 2024-06-10T00:32:46.974 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,key=0,fsid=1353b2ce-26bf-11ef-bc9b-c7b262605968" will pass to kernel 2024-06-10T00:32:46.974 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-06-10T00:32:46.974 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-06-10T00:32:46.974 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-06-10T00:32:46.974 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-06-10T00:32:46.975 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-06-10T00:32:46.976 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-10T00:32:46.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:32:46.976 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-10T00:32:47.041 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:47 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:47.033+0000 7f47305dd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:32:47.044 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-10T00:32:47.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T00:32:47.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:32:47.145 INFO:teuthology.orchestra.run.smithi062.stdout:client24980 2024-06-10T00:32:47.145 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:32:47.145 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T00:32:47.163 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-10T00:32:47.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T00:32:47.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:32:47.262 INFO:teuthology.orchestra.run.smithi062.stdout:client24980 2024-06-10T00:32:47.263 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-10T00:32:47.281 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:46 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:46.844+0000 7fb63cde0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:32:47.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T00:32:47.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:32:47.323 INFO:teuthology.orchestra.run.smithi062.stdout:client24980 2024-06-10T00:32:47.323 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968.client24980/status') 2024-06-10T00:32:47.357 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:47 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:47.141+0000 7f47305dd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:32:47.373 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:46 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:46.888+0000 7f94b63b0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:32:47.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:32:47.385 INFO:teuthology.orchestra.run.smithi062.stdout:instance: client.24980 (3)172.21.15.62:0/2725431333 2024-06-10T00:32:47.385 INFO:teuthology.orchestra.run.smithi062.stdout:blocklisted: false 2024-06-10T00:32:47.385 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2024-06-10T00:32:47.386 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2024-06-10T00:32:47.386 INFO:teuthology.orchestra.run.smithi062.stderr:71 bytes copied, 2.5806e-05 s, 2.8 MB/s 2024-06-10T00:32:47.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:32:47.393 INFO:teuthology.run_tasks:Running task exec... 2024-06-10T00:32:47.402 INFO:teuthology.task.exec:Executing custom commands... 2024-06-10T00:32:47.402 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-06-10T00:32:47.402 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-06-10T00:32:47.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.542+0000 7fb143f16640 1 -- 172.21.15.62:0/199937684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c101b10 msgr2=0x7fb13c101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:47.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.542+0000 7fb143f16640 1 --2- 172.21.15.62:0/199937684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c101b10 0x7fb13c101f90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb130009900 tx=0x7fb13002a410 comp rx=0 tx=0).stop 2024-06-10T00:32:47.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.542+0000 7fb143f16640 1 -- 172.21.15.62:0/199937684 shutdown_connections 2024-06-10T00:32:47.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.542+0000 7fb143f16640 1 --2- 172.21.15.62:0/199937684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb13c1024d0 0x7fb13c108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.542+0000 7fb143f16640 1 --2- 172.21.15.62:0/199937684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c101b10 0x7fb13c101f90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.542+0000 7fb143f16640 1 --2- 172.21.15.62:0/199937684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb13c100920 0x7fb13c100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.542+0000 7fb143f16640 1 -- 172.21.15.62:0/199937684 >> 172.21.15.62:0/199937684 conn(0x7fb13c0fbeb0 msgr2=0x7fb13c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.542+0000 7fb143f16640 1 -- 172.21.15.62:0/199937684 shutdown_connections 2024-06-10T00:32:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.542+0000 7fb143f16640 1 -- 172.21.15.62:0/199937684 wait complete. 2024-06-10T00:32:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.543+0000 7fb143f16640 1 Processor -- start 2024-06-10T00:32:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.543+0000 7fb143f16640 1 -- start start 2024-06-10T00:32:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.543+0000 7fb143f16640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c100920 0x7fb13c199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.543+0000 7fb143f16640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb13c101b10 0x7fb13c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.543+0000 7fb143f16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb13c1024d0 0x7fb13c1a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.543+0000 7fb143f16640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c10b4a0 con 0x7fb13c100920 2024-06-10T00:32:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.543+0000 7fb143f16640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c10b320 con 0x7fb13c1024d0 2024-06-10T00:32:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.543+0000 7fb143f16640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c10b620 con 0x7fb13c101b10 2024-06-10T00:32:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.543+0000 7fb141cb2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c100920 0x7fb13c199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:47.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.543+0000 7fb141cb2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c100920 0x7fb13c199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55830/0 (socket says 172.21.15.62:55830) 2024-06-10T00:32:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.543+0000 7fb141cb2640 1 -- 172.21.15.62:0/1201032272 learned_addr learned my addr 172.21.15.62:0/1201032272 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.543+0000 7fb1414b1640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb13c101b10 0x7fb13c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.543+0000 7fb1424b3640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb13c1024d0 0x7fb13c1a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.544+0000 7fb141cb2640 1 -- 172.21.15.62:0/1201032272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb13c101b10 msgr2=0x7fb13c19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.544+0000 7fb141cb2640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb13c101b10 0x7fb13c19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.544+0000 7fb141cb2640 1 -- 172.21.15.62:0/1201032272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb13c1024d0 msgr2=0x7fb13c1a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.544+0000 7fb141cb2640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb13c1024d0 0x7fb13c1a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.544+0000 7fb141cb2640 1 -- 172.21.15.62:0/1201032272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13c1a1a90 con 0x7fb13c100920 2024-06-10T00:32:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.544+0000 7fb1424b3640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb13c1024d0 0x7fb13c1a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:32:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.544+0000 7fb141cb2640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c100920 0x7fb13c199ec0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb13800e9c0 tx=0x7fb13800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.544+0000 7fb12effd640 1 -- 172.21.15.62:0/1201032272 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13800ccc0 con 0x7fb13c100920 2024-06-10T00:32:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.544+0000 7fb12effd640 1 -- 172.21.15.62:0/1201032272 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1380035f0 con 0x7fb13c100920 2024-06-10T00:32:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.544+0000 7fb143f16640 1 -- 172.21.15.62:0/1201032272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13c1a1d80 con 0x7fb13c100920 2024-06-10T00:32:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.544+0000 7fb12effd640 1 -- 172.21.15.62:0/1201032272 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13800ce60 con 0x7fb13c100920 2024-06-10T00:32:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.544+0000 7fb143f16640 1 -- 172.21.15.62:0/1201032272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb13c1a2190 con 0x7fb13c100920 2024-06-10T00:32:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.545+0000 7fb143f16640 1 -- 172.21.15.62:0/1201032272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb13c193990 con 0x7fb13c100920 2024-06-10T00:32:47.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.546+0000 7fb12effd640 1 -- 172.21.15.62:0/1201032272 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fb1380037a0 con 0x7fb13c100920 2024-06-10T00:32:47.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.547+0000 7fb12effd640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb1280a2c80 0x7fb1280a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:47.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.547+0000 7fb1414b1640 1 -- 172.21.15.62:0/1201032272 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb1280a2c80 msgr2=0x7fb1280a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3602410963 2024-06-10T00:32:47.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.547+0000 7fb12effd640 1 -- 172.21.15.62:0/1201032272 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fb1380cf7d0 con 0x7fb13c100920 2024-06-10T00:32:47.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.547+0000 7fb1414b1640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb1280a2c80 0x7fb1280a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:32:47.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.552+0000 7fb12effd640 1 -- 172.21.15.62:0/1201032272 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb13809c360 con 0x7fb13c100920 2024-06-10T00:32:47.700 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:47 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:47.366+0000 7fb63cde0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:32:47.700 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:47 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:47.459+0000 7fb63cde0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:32:47.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.748+0000 7fb1414b1640 1 -- 172.21.15.62:0/1201032272 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb1280a2c80 msgr2=0x7fb1280a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3602410963 2024-06-10T00:32:47.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.748+0000 7fb1414b1640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb1280a2c80 0x7fb1280a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T00:32:47.748 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:47 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:47.410+0000 7f94b63b0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:32:47.748 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:47 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:47.504+0000 7f94b63b0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:32:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.781+0000 7fb143f16640 1 -- 172.21.15.62:0/1201032272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fb13c02d2e0 con 0x7fb13c100920 2024-06-10T00:32:47.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.833+0000 7fb12effd640 1 -- 172.21.15.62:0/1201032272 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v30) ==== 117+0+0 (secure 0 0 0) 0x7fb13809bc60 con 0x7fb13c100920 2024-06-10T00:32:47.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.836+0000 7fb143f16640 1 -- 172.21.15.62:0/1201032272 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb1280a2c80 msgr2=0x7fb1280a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:32:47.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.836+0000 7fb143f16640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb1280a2c80 0x7fb1280a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.836+0000 7fb143f16640 1 -- 172.21.15.62:0/1201032272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c100920 msgr2=0x7fb13c199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:47.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.836+0000 7fb143f16640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c100920 0x7fb13c199ec0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb13800e9c0 tx=0x7fb13800edc0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.836+0000 7fb143f16640 1 -- 172.21.15.62:0/1201032272 shutdown_connections 2024-06-10T00:32:47.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.836+0000 7fb143f16640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb13c1024d0 0x7fb13c1a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.836+0000 7fb143f16640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fb1280a2c80 0x7fb1280a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.836+0000 7fb143f16640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb13c101b10 0x7fb13c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.836+0000 7fb143f16640 1 --2- 172.21.15.62:0/1201032272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c100920 0x7fb13c199ec0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.836+0000 7fb143f16640 1 -- 172.21.15.62:0/1201032272 >> 172.21.15.62:0/1201032272 conn(0x7fb13c0fbeb0 msgr2=0x7fb13c190b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:47.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.836+0000 7fb143f16640 1 -- 172.21.15.62:0/1201032272 shutdown_connections 2024-06-10T00:32:47.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.836+0000 7fb143f16640 1 -- 172.21.15.62:0/1201032272 wait complete. 2024-06-10T00:32:47.837 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:47 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:47.361+0000 7f47305dd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:32:47.848 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-10T00:32:47.859 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-06-10T00:32:47.877 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-10T00:32:47.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.983+0000 7fd7cfd09640 1 -- 172.21.15.62:0/1297149075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c8101ad0 msgr2=0x7fd7c8101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:47.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.983+0000 7fd7cfd09640 1 --2- 172.21.15.62:0/1297149075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c8101ad0 0x7fd7c8101f50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009900 tx=0x7fd7b802a410 comp rx=0 tx=0).stop 2024-06-10T00:32:47.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.983+0000 7fd7cfd09640 1 -- 172.21.15.62:0/1297149075 shutdown_connections 2024-06-10T00:32:47.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.983+0000 7fd7cfd09640 1 --2- 172.21.15.62:0/1297149075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7c8102490 0x7fd7c8108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.983+0000 7fd7cfd09640 1 --2- 172.21.15.62:0/1297149075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c8101ad0 0x7fd7c8101f50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.983+0000 7fd7cfd09640 1 --2- 172.21.15.62:0/1297149075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7c81008e0 0x7fd7c8100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.983+0000 7fd7cfd09640 1 -- 172.21.15.62:0/1297149075 >> 172.21.15.62:0/1297149075 conn(0x7fd7c80fbe50 msgr2=0x7fd7c80fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:47.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.983+0000 7fd7cfd09640 1 -- 172.21.15.62:0/1297149075 shutdown_connections 2024-06-10T00:32:47.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.983+0000 7fd7cfd09640 1 -- 172.21.15.62:0/1297149075 wait complete. 2024-06-10T00:32:47.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cfd09640 1 Processor -- start 2024-06-10T00:32:47.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cfd09640 1 -- start start 2024-06-10T00:32:47.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cfd09640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c81008e0 0x7fd7c80696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:47.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cfd09640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7c8101ad0 0x7fd7c819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:47.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cfd09640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7c8102490 0x7fd7c81a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:47.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cfd09640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c810c560 con 0x7fd7c81008e0 2024-06-10T00:32:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cfd09640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c810c3e0 con 0x7fd7c8101ad0 2024-06-10T00:32:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cfd09640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c810c6e0 con 0x7fd7c8102490 2024-06-10T00:32:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cdaa5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c81008e0 0x7fd7c80696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cdaa5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c81008e0 0x7fd7c80696a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45116/0 (socket says 172.21.15.62:45116) 2024-06-10T00:32:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cdaa5640 1 -- 172.21.15.62:0/2516614212 learned_addr learned my addr 172.21.15.62:0/2516614212 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cd2a4640 1 --2- 172.21.15.62:0/2516614212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7c8101ad0 0x7fd7c819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7ce2a6640 1 --2- 172.21.15.62:0/2516614212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7c8102490 0x7fd7c81a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cdaa5640 1 -- 172.21.15.62:0/2516614212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7c8102490 msgr2=0x7fd7c81a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cdaa5640 1 --2- 172.21.15.62:0/2516614212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7c8102490 0x7fd7c81a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cdaa5640 1 -- 172.21.15.62:0/2516614212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7c8101ad0 msgr2=0x7fd7c819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cdaa5640 1 --2- 172.21.15.62:0/2516614212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7c8101ad0 0x7fd7c819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.984+0000 7fd7cdaa5640 1 -- 172.21.15.62:0/2516614212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c81a1b90 con 0x7fd7c81008e0 2024-06-10T00:32:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.985+0000 7fd7cdaa5640 1 --2- 172.21.15.62:0/2516614212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c81008e0 0x7fd7c80696a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400e9c0 tx=0x7fd7c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.985+0000 7fd7beffd640 1 -- 172.21.15.62:0/2516614212 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c400cd80 con 0x7fd7c81008e0 2024-06-10T00:32:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.985+0000 7fd7beffd640 1 -- 172.21.15.62:0/2516614212 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7c40035f0 con 0x7fd7c81008e0 2024-06-10T00:32:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.985+0000 7fd7cfd09640 1 -- 172.21.15.62:0/2516614212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c81a1e20 con 0x7fd7c81008e0 2024-06-10T00:32:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.985+0000 7fd7beffd640 1 -- 172.21.15.62:0/2516614212 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4010690 con 0x7fd7c81008e0 2024-06-10T00:32:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.985+0000 7fd7cfd09640 1 -- 172.21.15.62:0/2516614212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c81a2280 con 0x7fd7c81008e0 2024-06-10T00:32:47.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.986+0000 7fd7cfd09640 1 -- 172.21.15.62:0/2516614212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c8107c00 con 0x7fd7c81008e0 2024-06-10T00:32:47.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.988+0000 7fd7beffd640 1 -- 172.21.15.62:0/2516614212 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fd7c40037a0 con 0x7fd7c81008e0 2024-06-10T00:32:47.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.989+0000 7fd7beffd640 1 --2- 172.21.15.62:0/2516614212 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd7b40a2c10 0x7fd7b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:47.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.989+0000 7fd7cd2a4640 1 -- 172.21.15.62:0/2516614212 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd7b40a2c10 msgr2=0x7fd7b40a50d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3602410963 2024-06-10T00:32:47.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.989+0000 7fd7beffd640 1 -- 172.21.15.62:0/2516614212 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7fd7c40cf8a0 con 0x7fd7c81008e0 2024-06-10T00:32:47.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.989+0000 7fd7cd2a4640 1 --2- 172.21.15.62:0/2516614212 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd7b40a2c10 0x7fd7b40a50d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:32:47.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:47.991+0000 7fd7beffd640 1 -- 172.21.15.62:0/2516614212 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd7c409c4c0 con 0x7fd7c81008e0 2024-06-10T00:32:48.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:47 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1201032272' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-10T00:32:48.032 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:47 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:47.699+0000 7fb63cde0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:32:48.032 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:47 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:47.833+0000 7fb63cde0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:32:48.032 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:47 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:47.929+0000 7fb63cde0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:32:48.098 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:47 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1201032272' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-10T00:32:48.099 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:47 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:47.747+0000 7f94b63b0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:32:48.099 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:47 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:47.884+0000 7f94b63b0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:32:48.099 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:47 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:47.981+0000 7f94b63b0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:32:48.099 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:48 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:48.098+0000 7f94b63b0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:32:48.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:47 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1201032272' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-10T00:32:48.108 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:47 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:47.927+0000 7f47305dd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:32:48.108 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:48 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:48.031+0000 7f47305dd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:32:48.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.179+0000 7fd7cfd09640 1 -- 172.21.15.62:0/2516614212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7c802d590 con 0x7fd7c81008e0 2024-06-10T00:32:48.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.181+0000 7fd7beffd640 1 -- 172.21.15.62:0/2516614212 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24839 (secure 0 0 0) 0x7fd7c409bdc0 con 0x7fd7c81008e0 2024-06-10T00:32:48.181 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:32:48.182 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":99,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:30:24.104116+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:32:48.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.183+0000 7fd7cfd09640 1 -- 172.21.15.62:0/2516614212 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd7b40a2c10 msgr2=0x7fd7b40a50d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:32:48.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.183+0000 7fd7cfd09640 1 --2- 172.21.15.62:0/2516614212 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd7b40a2c10 0x7fd7b40a50d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.183+0000 7fd7cfd09640 1 -- 172.21.15.62:0/2516614212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c81008e0 msgr2=0x7fd7c80696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:48.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.183+0000 7fd7cfd09640 1 --2- 172.21.15.62:0/2516614212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c81008e0 0x7fd7c80696a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400e9c0 tx=0x7fd7c400ee80 comp rx=0 tx=0).stop 2024-06-10T00:32:48.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.183+0000 7fd7cfd09640 1 -- 172.21.15.62:0/2516614212 shutdown_connections 2024-06-10T00:32:48.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.183+0000 7fd7cfd09640 1 --2- 172.21.15.62:0/2516614212 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7fd7b40a2c10 0x7fd7b40a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.183+0000 7fd7cfd09640 1 --2- 172.21.15.62:0/2516614212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7c8102490 0x7fd7c81a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.183+0000 7fd7cfd09640 1 --2- 172.21.15.62:0/2516614212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd7c8101ad0 0x7fd7c819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.183+0000 7fd7cfd09640 1 --2- 172.21.15.62:0/2516614212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c81008e0 0x7fd7c80696a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.183+0000 7fd7cfd09640 1 -- 172.21.15.62:0/2516614212 >> 172.21.15.62:0/2516614212 conn(0x7fd7c80fbe50 msgr2=0x7fd7c8104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:48.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.183+0000 7fd7cfd09640 1 -- 172.21.15.62:0/2516614212 shutdown_connections 2024-06-10T00:32:48.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.183+0000 7fd7cfd09640 1 -- 172.21.15.62:0/2516614212 wait complete. 2024-06-10T00:32:48.195 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-10T00:23:46.372672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-10T00:28:33.319774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-10T00:28:36.578569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-10T00:28:47.112439+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-10T00:32:48.195 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-10T00:32:48.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.304+0000 7f069b75c640 1 -- 172.21.15.62:0/3262625747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06941008e0 msgr2=0x7f0694100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:48.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.304+0000 7f069b75c640 1 --2- 172.21.15.62:0/3262625747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06941008e0 0x7f0694100d00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0690009900 tx=0x7f069002a410 comp rx=0 tx=0).stop 2024-06-10T00:32:48.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.304+0000 7f069b75c640 1 -- 172.21.15.62:0/3262625747 shutdown_connections 2024-06-10T00:32:48.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.304+0000 7f069b75c640 1 --2- 172.21.15.62:0/3262625747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0694102490 0x7f0694108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.304+0000 7f069b75c640 1 --2- 172.21.15.62:0/3262625747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0694101ad0 0x7f0694101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.304+0000 7f069b75c640 1 --2- 172.21.15.62:0/3262625747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06941008e0 0x7f0694100d00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.304+0000 7f069b75c640 1 -- 172.21.15.62:0/3262625747 >> 172.21.15.62:0/3262625747 conn(0x7f06940fbe50 msgr2=0x7f06940fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:48.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.304+0000 7f069b75c640 1 -- 172.21.15.62:0/3262625747 shutdown_connections 2024-06-10T00:32:48.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.304+0000 7f069b75c640 1 -- 172.21.15.62:0/3262625747 wait complete. 2024-06-10T00:32:48.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f069b75c640 1 Processor -- start 2024-06-10T00:32:48.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f069b75c640 1 -- start start 2024-06-10T00:32:48.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f069b75c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06941008e0 0x7f0694197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:48.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f069b75c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0694101ad0 0x7f06941982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:48.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f069b75c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0694102490 0x7f069419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:48.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f069b75c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f069410c530 con 0x7f06941008e0 2024-06-10T00:32:48.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f069b75c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f069410c3b0 con 0x7f0694102490 2024-06-10T00:32:48.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f069b75c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f069410c6b0 con 0x7f0694101ad0 2024-06-10T00:32:48.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f06994f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06941008e0 0x7f0694197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:48.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f06994f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06941008e0 0x7f0694197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45130/0 (socket says 172.21.15.62:45130) 2024-06-10T00:32:48.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f06994f8640 1 -- 172.21.15.62:0/1018999699 learned_addr learned my addr 172.21.15.62:0/1018999699 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:48.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f0698cf7640 1 --2- 172.21.15.62:0/1018999699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0694101ad0 0x7f06941982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:48.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f0699cf9640 1 --2- 172.21.15.62:0/1018999699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0694102490 0x7f069419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:48.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f06994f8640 1 -- 172.21.15.62:0/1018999699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0694101ad0 msgr2=0x7f06941982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:48.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f06994f8640 1 --2- 172.21.15.62:0/1018999699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0694101ad0 0x7f06941982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.305+0000 7f06994f8640 1 -- 172.21.15.62:0/1018999699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0694102490 msgr2=0x7f069419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:48.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.306+0000 7f06994f8640 1 --2- 172.21.15.62:0/1018999699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0694102490 0x7f069419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.306+0000 7f06994f8640 1 -- 172.21.15.62:0/1018999699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069419f9b0 con 0x7f06941008e0 2024-06-10T00:32:48.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.306+0000 7f06994f8640 1 --2- 172.21.15.62:0/1018999699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06941008e0 0x7f0694197da0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0690031610 tx=0x7f06900043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:48.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.306+0000 7f068a7fc640 1 -- 172.21.15.62:0/1018999699 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069003f030 con 0x7f06941008e0 2024-06-10T00:32:48.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.306+0000 7f068a7fc640 1 -- 172.21.15.62:0/1018999699 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0690003ce0 con 0x7f06941008e0 2024-06-10T00:32:48.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.306+0000 7f069b75c640 1 -- 172.21.15.62:0/1018999699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069419fc40 con 0x7f06941008e0 2024-06-10T00:32:48.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.306+0000 7f068a7fc640 1 -- 172.21.15.62:0/1018999699 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690035070 con 0x7f06941008e0 2024-06-10T00:32:48.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.306+0000 7f069b75c640 1 -- 172.21.15.62:0/1018999699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06941a0130 con 0x7f06941008e0 2024-06-10T00:32:48.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.307+0000 7f069b75c640 1 -- 172.21.15.62:0/1018999699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06941a0300 con 0x7f06941008e0 2024-06-10T00:32:48.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.308+0000 7f068a7fc640 1 -- 172.21.15.62:0/1018999699 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f0690042030 con 0x7f06941008e0 2024-06-10T00:32:48.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.309+0000 7f068a7fc640 1 --2- 172.21.15.62:0/1018999699 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f06800a2be0 0x7f06800a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:48.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.309+0000 7f0698cf7640 1 -- 172.21.15.62:0/1018999699 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f06800a2be0 msgr2=0x7f06800a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3602410963 2024-06-10T00:32:48.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.309+0000 7f068a7fc640 1 -- 172.21.15.62:0/1018999699 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f06900ef9f0 con 0x7f06941008e0 2024-06-10T00:32:48.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.309+0000 7f0698cf7640 1 --2- 172.21.15.62:0/1018999699 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f06800a2be0 0x7f06800a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:32:48.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.313+0000 7f068a7fc640 1 -- 172.21.15.62:0/1018999699 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f06900bbf30 con 0x7f06941008e0 2024-06-10T00:32:48.342 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:48 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:48.045+0000 7fb63cde0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:32:48.445 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:48 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:48.298+0000 7f47305dd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:32:48.445 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:48 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:48.445+0000 7f47305dd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:32:48.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.500+0000 7f069b75c640 1 -- 172.21.15.62:0/1018999699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f06941a04d0 con 0x7f06941008e0 2024-06-10T00:32:48.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.501+0000 7f068a7fc640 1 -- 172.21.15.62:0/1018999699 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f06900c0230 con 0x7f06941008e0 2024-06-10T00:32:48.501 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-06-10T00:32:48.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.503+0000 7f069b75c640 1 -- 172.21.15.62:0/1018999699 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f06800a2be0 msgr2=0x7f06800a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:32:48.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.503+0000 7f069b75c640 1 --2- 172.21.15.62:0/1018999699 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f06800a2be0 0x7f06800a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.503+0000 7f069b75c640 1 -- 172.21.15.62:0/1018999699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06941008e0 msgr2=0x7f0694197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:48.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.503+0000 7f069b75c640 1 --2- 172.21.15.62:0/1018999699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06941008e0 0x7f0694197da0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0690031610 tx=0x7f06900043b0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.503+0000 7f069b75c640 1 -- 172.21.15.62:0/1018999699 shutdown_connections 2024-06-10T00:32:48.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.503+0000 7f069b75c640 1 --2- 172.21.15.62:0/1018999699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0694102490 0x7f069419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.503+0000 7f069b75c640 1 --2- 172.21.15.62:0/1018999699 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f06800a2be0 0x7f06800a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.503+0000 7f069b75c640 1 --2- 172.21.15.62:0/1018999699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0694101ad0 0x7f06941982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.503+0000 7f069b75c640 1 --2- 172.21.15.62:0/1018999699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06941008e0 0x7f0694197da0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.503+0000 7f069b75c640 1 -- 172.21.15.62:0/1018999699 >> 172.21.15.62:0/1018999699 conn(0x7f06940fbe50 msgr2=0x7f0694068be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.503+0000 7f069b75c640 1 -- 172.21.15.62:0/1018999699 shutdown_connections 2024-06-10T00:32:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.503+0000 7f069b75c640 1 -- 172.21.15.62:0/1018999699 wait complete. 2024-06-10T00:32:48.514 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-10T00:32:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.623+0000 7f2cc32b3640 1 -- 172.21.15.62:0/3106531319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc1076c0 msgr2=0x7f2cbc109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.623+0000 7f2cc32b3640 1 --2- 172.21.15.62:0/3106531319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc1076c0 0x7f2cbc109b10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009ca0 tx=0x7f2cb402a8c0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.623+0000 7f2cc32b3640 1 -- 172.21.15.62:0/3106531319 shutdown_connections 2024-06-10T00:32:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.623+0000 7f2cc32b3640 1 --2- 172.21.15.62:0/3106531319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc1076c0 0x7f2cbc109b10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.623+0000 7f2cc32b3640 1 --2- 172.21.15.62:0/3106531319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cbc104d90 0x7f2cbc107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.623+0000 7f2cc32b3640 1 --2- 172.21.15.62:0/3106531319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc102460 0x7f2cbc104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.623+0000 7f2cc32b3640 1 -- 172.21.15.62:0/3106531319 >> 172.21.15.62:0/3106531319 conn(0x7f2cbc0fbe50 msgr2=0x7f2cbc0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.623+0000 7f2cc32b3640 1 -- 172.21.15.62:0/3106531319 shutdown_connections 2024-06-10T00:32:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.623+0000 7f2cc32b3640 1 -- 172.21.15.62:0/3106531319 wait complete. 2024-06-10T00:32:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.624+0000 7f2cc32b3640 1 Processor -- start 2024-06-10T00:32:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.624+0000 7f2cc32b3640 1 -- start start 2024-06-10T00:32:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.624+0000 7f2cc32b3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc102460 0x7f2cbc197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.624+0000 7f2cc32b3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc104d90 0x7f2cbc198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.624+0000 7f2cc32b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cbc1076c0 0x7f2cbc19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.624+0000 7f2cc32b3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc10d660 con 0x7f2cbc102460 2024-06-10T00:32:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.624+0000 7f2cc32b3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc10d4e0 con 0x7f2cbc1076c0 2024-06-10T00:32:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.624+0000 7f2cc32b3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc10d7e0 con 0x7f2cbc104d90 2024-06-10T00:32:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.624+0000 7f2cc104f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc102460 0x7f2cbc197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.624+0000 7f2cc104f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc102460 0x7f2cbc197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45148/0 (socket says 172.21.15.62:45148) 2024-06-10T00:32:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.624+0000 7f2cc104f640 1 -- 172.21.15.62:0/2533032684 learned_addr learned my addr 172.21.15.62:0/2533032684 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.624+0000 7f2cc084e640 1 --2- 172.21.15.62:0/2533032684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc104d90 0x7f2cbc198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.624+0000 7f2cc1850640 1 --2- 172.21.15.62:0/2533032684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cbc1076c0 0x7f2cbc19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.625+0000 7f2cc104f640 1 -- 172.21.15.62:0/2533032684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc104d90 msgr2=0x7f2cbc198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.625+0000 7f2cc104f640 1 --2- 172.21.15.62:0/2533032684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc104d90 0x7f2cbc198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.625+0000 7f2cc104f640 1 -- 172.21.15.62:0/2533032684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cbc1076c0 msgr2=0x7f2cbc19f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.625+0000 7f2cc104f640 1 --2- 172.21.15.62:0/2533032684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cbc1076c0 0x7f2cbc19f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.625+0000 7f2cc104f640 1 -- 172.21.15.62:0/2533032684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc19f930 con 0x7f2cbc102460 2024-06-10T00:32:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.625+0000 7f2cc104f640 1 --2- 172.21.15.62:0/2533032684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc102460 0x7f2cbc197d20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800e9f0 tx=0x7f2cb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.625+0000 7f2cb27fc640 1 -- 172.21.15.62:0/2533032684 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb800cd80 con 0x7f2cbc102460 2024-06-10T00:32:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.625+0000 7f2cb27fc640 1 -- 172.21.15.62:0/2533032684 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cb8010910 con 0x7f2cbc102460 2024-06-10T00:32:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.625+0000 7f2cc32b3640 1 -- 172.21.15.62:0/2533032684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cbc19fc20 con 0x7f2cbc102460 2024-06-10T00:32:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.625+0000 7f2cb27fc640 1 -- 172.21.15.62:0/2533032684 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8018990 con 0x7f2cbc102460 2024-06-10T00:32:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.625+0000 7f2cc32b3640 1 -- 172.21.15.62:0/2533032684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cbc1a0170 con 0x7f2cbc102460 2024-06-10T00:32:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.626+0000 7f2cc32b3640 1 -- 172.21.15.62:0/2533032684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cbc0fd940 con 0x7f2cbc102460 2024-06-10T00:32:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.627+0000 7f2cb27fc640 1 -- 172.21.15.62:0/2533032684 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f2cb8010ac0 con 0x7f2cbc102460 2024-06-10T00:32:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.628+0000 7f2cb27fc640 1 --2- 172.21.15.62:0/2533032684 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2ca80a2be0 0x7f2ca80a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.628+0000 7f2cc084e640 1 -- 172.21.15.62:0/2533032684 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2ca80a2be0 msgr2=0x7f2ca80a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3602410963 2024-06-10T00:32:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.628+0000 7f2cb27fc640 1 -- 172.21.15.62:0/2533032684 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f2cb80cf870 con 0x7f2cbc102460 2024-06-10T00:32:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.629+0000 7f2cc084e640 1 --2- 172.21.15.62:0/2533032684 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2ca80a2be0 0x7f2ca80a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:32:48.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.632+0000 7f2cb27fc640 1 -- 172.21.15.62:0/2533032684 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2cb809bd50 con 0x7f2cbc102460 2024-06-10T00:32:48.781 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:48 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:48.342+0000 7fb63cde0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:32:48.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.822+0000 7f2cc32b3640 1 -- 172.21.15.62:0/2533032684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2cbc0625a0 con 0x7f2cbc102460 2024-06-10T00:32:48.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.822+0000 7f2cb27fc640 1 -- 172.21.15.62:0/2533032684 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f2cb80a0050 con 0x7f2cbc102460 2024-06-10T00:32:48.822 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-06-10T00:32:48.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.824+0000 7f2cc32b3640 1 -- 172.21.15.62:0/2533032684 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2ca80a2be0 msgr2=0x7f2ca80a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:32:48.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.824+0000 7f2cc32b3640 1 --2- 172.21.15.62:0/2533032684 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2ca80a2be0 0x7f2ca80a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.824+0000 7f2cc32b3640 1 -- 172.21.15.62:0/2533032684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc102460 msgr2=0x7f2cbc197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:48.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.824+0000 7f2cc32b3640 1 --2- 172.21.15.62:0/2533032684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc102460 0x7f2cbc197d20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800e9f0 tx=0x7f2cb800eeb0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.824+0000 7f2cc32b3640 1 -- 172.21.15.62:0/2533032684 shutdown_connections 2024-06-10T00:32:48.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.824+0000 7f2cc32b3640 1 --2- 172.21.15.62:0/2533032684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cbc1076c0 0x7f2cbc19f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.824+0000 7f2cc32b3640 1 --2- 172.21.15.62:0/2533032684 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f2ca80a2be0 0x7f2ca80a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.824+0000 7f2cc32b3640 1 --2- 172.21.15.62:0/2533032684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cbc104d90 0x7f2cbc198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.824+0000 7f2cc32b3640 1 --2- 172.21.15.62:0/2533032684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2cbc102460 0x7f2cbc197d20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.824+0000 7f2cc32b3640 1 -- 172.21.15.62:0/2533032684 >> 172.21.15.62:0/2533032684 conn(0x7f2cbc0fbe50 msgr2=0x7f2cbc102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:48.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.824+0000 7f2cc32b3640 1 -- 172.21.15.62:0/2533032684 shutdown_connections 2024-06-10T00:32:48.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.824+0000 7f2cc32b3640 1 -- 172.21.15.62:0/2533032684 wait complete. 2024-06-10T00:32:48.835 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:48 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:48.550+0000 7f47305dd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:32:48.835 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:48 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:48.679+0000 7f47305dd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:32:48.836 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-10T00:32:48.839 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:48 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:48.403+0000 7f94b63b0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:32:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.947+0000 7f8ab50ba640 1 -- 172.21.15.62:0/3331122417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab01098a0 msgr2=0x7f8ab010bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.947+0000 7f8ab50ba640 1 --2- 172.21.15.62:0/3331122417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab01098a0 0x7f8ab010bd30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009ca0 tx=0x7f8aa802a950 comp rx=0 tx=0).stop 2024-06-10T00:32:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.948+0000 7f8ab50ba640 1 -- 172.21.15.62:0/3331122417 shutdown_connections 2024-06-10T00:32:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.948+0000 7f8ab50ba640 1 --2- 172.21.15.62:0/3331122417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab01098a0 0x7f8ab010bd30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.948+0000 7f8ab50ba640 1 --2- 172.21.15.62:0/3331122417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab0074380 0x7f8ab00747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.948+0000 7f8ab50ba640 1 --2- 172.21.15.62:0/3331122417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ab0075960 0x7f8ab0073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.948+0000 7f8ab50ba640 1 -- 172.21.15.62:0/3331122417 >> 172.21.15.62:0/3331122417 conn(0x7f8ab00fcf80 msgr2=0x7f8ab00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.948+0000 7f8ab50ba640 1 -- 172.21.15.62:0/3331122417 shutdown_connections 2024-06-10T00:32:48.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.948+0000 7f8ab50ba640 1 -- 172.21.15.62:0/3331122417 wait complete. 2024-06-10T00:32:48.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.948+0000 7f8ab50ba640 1 Processor -- start 2024-06-10T00:32:48.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8ab50ba640 1 -- start start 2024-06-10T00:32:48.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8ab50ba640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab0074380 0x7f8ab0199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:48.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8ab50ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ab0075960 0x7f8ab019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:48.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8ab50ba640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab01098a0 0x7f8ab01a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:48.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8ab50ba640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab010f880 con 0x7f8ab0074380 2024-06-10T00:32:48.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8ab50ba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab010f700 con 0x7f8ab0075960 2024-06-10T00:32:48.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8ab50ba640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab010fa00 con 0x7f8ab01098a0 2024-06-10T00:32:48.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8aaed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab0074380 0x7f8ab0199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:48.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8aaed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab0074380 0x7f8ab0199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45176/0 (socket says 172.21.15.62:45176) 2024-06-10T00:32:48.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8aaed9d640 1 -- 172.21.15.62:0/1363117180 learned_addr learned my addr 172.21.15.62:0/1363117180 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:48.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8aae59c640 1 --2- 172.21.15.62:0/1363117180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ab0075960 0x7f8ab019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:48.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8aaf59e640 1 --2- 172.21.15.62:0/1363117180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab01098a0 0x7f8ab01a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:48.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8aaed9d640 1 -- 172.21.15.62:0/1363117180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab01098a0 msgr2=0x7f8ab01a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:48.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8aaed9d640 1 --2- 172.21.15.62:0/1363117180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab01098a0 0x7f8ab01a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8aaed9d640 1 -- 172.21.15.62:0/1363117180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ab0075960 msgr2=0x7f8ab019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:48.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8aaed9d640 1 --2- 172.21.15.62:0/1363117180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ab0075960 0x7f8ab019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:48.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8aaed9d640 1 -- 172.21.15.62:0/1363117180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab01a1ae0 con 0x7f8ab0074380 2024-06-10T00:32:48.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.949+0000 7f8aaed9d640 1 --2- 172.21.15.62:0/1363117180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab0074380 0x7f8ab0199ec0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000e9f0 tx=0x7f8aa000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:48.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.950+0000 7f8a97fff640 1 -- 172.21.15.62:0/1363117180 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa000cd80 con 0x7f8ab0074380 2024-06-10T00:32:48.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.950+0000 7f8ab50ba640 1 -- 172.21.15.62:0/1363117180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab01a1dd0 con 0x7f8ab0074380 2024-06-10T00:32:48.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.950+0000 7f8a97fff640 1 -- 172.21.15.62:0/1363117180 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa0010910 con 0x7f8ab0074380 2024-06-10T00:32:48.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.950+0000 7f8a97fff640 1 -- 172.21.15.62:0/1363117180 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa00188f0 con 0x7f8ab0074380 2024-06-10T00:32:48.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.950+0000 7f8ab50ba640 1 -- 172.21.15.62:0/1363117180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab01a2240 con 0x7f8ab0074380 2024-06-10T00:32:48.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.951+0000 7f8ab50ba640 1 -- 172.21.15.62:0/1363117180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab0108660 con 0x7f8ab0074380 2024-06-10T00:32:48.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.953+0000 7f8a97fff640 1 -- 172.21.15.62:0/1363117180 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f8aa0010ac0 con 0x7f8ab0074380 2024-06-10T00:32:48.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.953+0000 7f8a97fff640 1 --2- 172.21.15.62:0/1363117180 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8a980a2be0 0x7f8a980a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:48.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.953+0000 7f8aae59c640 1 -- 172.21.15.62:0/1363117180 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8a980a2be0 msgr2=0x7f8a980a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3602410963 2024-06-10T00:32:48.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.953+0000 7f8a97fff640 1 -- 172.21.15.62:0/1363117180 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f8aa00cfb80 con 0x7f8ab0074380 2024-06-10T00:32:48.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.953+0000 7f8aae59c640 1 --2- 172.21.15.62:0/1363117180 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8a980a2be0 0x7f8a980a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:32:48.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:48.957+0000 7f8a97fff640 1 -- 172.21.15.62:0/1363117180 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8aa009c0c0 con 0x7f8ab0074380 2024-06-10T00:32:49.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:48 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1201032272' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-10T00:32:49.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:48 smithi062 ceph-mon[32390]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:32:49.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:48 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2516614212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:32:49.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:48 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1018999699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:32:49.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:48 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2533032684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T00:32:49.109 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:49.004+0000 7f47305dd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:32:49.123 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:49.011+0000 7f94b63b0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:32:49.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:48 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1201032272' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-10T00:32:49.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:48 smithi138 ceph-mon[36549]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:32:49.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:48 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2516614212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:32:49.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:48 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1018999699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:32:49.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:48 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2533032684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T00:32:49.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.143+0000 7f8ab50ba640 1 -- 172.21.15.62:0/1363117180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8ab002d2e0 con 0x7f8ab0074380 2024-06-10T00:32:49.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.143+0000 7f8a97fff640 1 -- 172.21.15.62:0/1363117180 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f8aa00a03c0 con 0x7f8ab0074380 2024-06-10T00:32:49.143 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-06-10T00:32:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.145+0000 7f8ab50ba640 1 -- 172.21.15.62:0/1363117180 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8a980a2be0 msgr2=0x7f8a980a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:32:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.145+0000 7f8ab50ba640 1 --2- 172.21.15.62:0/1363117180 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8a980a2be0 0x7f8a980a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.145+0000 7f8ab50ba640 1 -- 172.21.15.62:0/1363117180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab0074380 msgr2=0x7f8ab0199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.145+0000 7f8ab50ba640 1 --2- 172.21.15.62:0/1363117180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab0074380 0x7f8ab0199ec0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000e9f0 tx=0x7f8aa000eeb0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.145+0000 7f8ab50ba640 1 -- 172.21.15.62:0/1363117180 shutdown_connections 2024-06-10T00:32:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.145+0000 7f8ab50ba640 1 --2- 172.21.15.62:0/1363117180 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f8a980a2be0 0x7f8a980a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.145+0000 7f8ab50ba640 1 --2- 172.21.15.62:0/1363117180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab01098a0 0x7f8ab01a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.145+0000 7f8ab50ba640 1 --2- 172.21.15.62:0/1363117180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ab0075960 0x7f8ab019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.145+0000 7f8ab50ba640 1 --2- 172.21.15.62:0/1363117180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ab0074380 0x7f8ab0199ec0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.145+0000 7f8ab50ba640 1 -- 172.21.15.62:0/1363117180 >> 172.21.15.62:0/1363117180 conn(0x7f8ab00fcf80 msgr2=0x7f8ab0105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.146+0000 7f8ab50ba640 1 -- 172.21.15.62:0/1363117180 shutdown_connections 2024-06-10T00:32:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.146+0000 7f8ab50ba640 1 -- 172.21.15.62:0/1363117180 wait complete. 2024-06-10T00:32:49.156 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-10T00:32:49.159 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:48 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:48.943+0000 7fb63cde0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:32:49.160 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:48 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1201032272' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-10T00:32:49.160 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:48 smithi105 ceph-mon[36321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:32:49.160 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:48 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2516614212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:32:49.160 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:48 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1018999699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:32:49.160 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:48 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2533032684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T00:32:49.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 -- 172.21.15.62:0/4178525999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d64101ad0 msgr2=0x7f6d64101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:49.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 --2- 172.21.15.62:0/4178525999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d64101ad0 0x7f6d64101f50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009900 tx=0x7f6d4c02a410 comp rx=0 tx=0).stop 2024-06-10T00:32:49.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 -- 172.21.15.62:0/4178525999 shutdown_connections 2024-06-10T00:32:49.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 --2- 172.21.15.62:0/4178525999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64102490 0x7f6d64108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 --2- 172.21.15.62:0/4178525999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d64101ad0 0x7f6d64101f50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 --2- 172.21.15.62:0/4178525999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d641008e0 0x7f6d64100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 -- 172.21.15.62:0/4178525999 >> 172.21.15.62:0/4178525999 conn(0x7f6d640fbe50 msgr2=0x7f6d640fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:49.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 -- 172.21.15.62:0/4178525999 shutdown_connections 2024-06-10T00:32:49.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 -- 172.21.15.62:0/4178525999 wait complete. 2024-06-10T00:32:49.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 Processor -- start 2024-06-10T00:32:49.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 -- start start 2024-06-10T00:32:49.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d641008e0 0x7f6d64069700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:49.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64101ad0 0x7f6d6419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:49.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d64102490 0x7f6d641a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:49.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6410c560 con 0x7f6d64102490 2024-06-10T00:32:49.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.266+0000 7f6d695f9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6410c3e0 con 0x7f6d64101ad0 2024-06-10T00:32:49.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d695f9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6410c6e0 con 0x7f6d641008e0 2024-06-10T00:32:49.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d637fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d64102490 0x7f6d641a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:49.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d637fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d64102490 0x7f6d641a13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45206/0 (socket says 172.21.15.62:45206) 2024-06-10T00:32:49.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d637fe640 1 -- 172.21.15.62:0/3141896448 learned_addr learned my addr 172.21.15.62:0/3141896448 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:49.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d62ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d641008e0 0x7f6d64069700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:49.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d5bfff640 1 --2- 172.21.15.62:0/3141896448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64101ad0 0x7f6d6419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:49.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d62ffd640 1 -- 172.21.15.62:0/3141896448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64101ad0 msgr2=0x7f6d6419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:49.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d62ffd640 1 --2- 172.21.15.62:0/3141896448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64101ad0 0x7f6d6419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d62ffd640 1 -- 172.21.15.62:0/3141896448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d64102490 msgr2=0x7f6d641a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:49.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d62ffd640 1 --2- 172.21.15.62:0/3141896448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d64102490 0x7f6d641a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d62ffd640 1 -- 172.21.15.62:0/3141896448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d641a1ba0 con 0x7f6d641008e0 2024-06-10T00:32:49.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d637fe640 1 --2- 172.21.15.62:0/3141896448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d64102490 0x7f6d641a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:32:49.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d62ffd640 1 --2- 172.21.15.62:0/3141896448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d641008e0 0x7f6d64069700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400e9c0 tx=0x7f6d5400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:49.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d60ff9640 1 -- 172.21.15.62:0/3141896448 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5400ccc0 con 0x7f6d641008e0 2024-06-10T00:32:49.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d695f9640 1 -- 172.21.15.62:0/3141896448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d641a1e90 con 0x7f6d641008e0 2024-06-10T00:32:49.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d60ff9640 1 -- 172.21.15.62:0/3141896448 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d540035f0 con 0x7f6d641008e0 2024-06-10T00:32:49.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.267+0000 7f6d60ff9640 1 -- 172.21.15.62:0/3141896448 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5400ce60 con 0x7f6d641008e0 2024-06-10T00:32:49.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.268+0000 7f6d695f9640 1 -- 172.21.15.62:0/3141896448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d641a22f0 con 0x7f6d641008e0 2024-06-10T00:32:49.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.268+0000 7f6d695f9640 1 -- 172.21.15.62:0/3141896448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d64107c50 con 0x7f6d641008e0 2024-06-10T00:32:49.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.270+0000 7f6d60ff9640 1 -- 172.21.15.62:0/3141896448 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f6d540037a0 con 0x7f6d641008e0 2024-06-10T00:32:49.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.270+0000 7f6d60ff9640 1 --2- 172.21.15.62:0/3141896448 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6d500a2be0 0x7f6d500a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:49.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.270+0000 7f6d5bfff640 1 -- 172.21.15.62:0/3141896448 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6d500a2be0 msgr2=0x7f6d500a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3602410963 2024-06-10T00:32:49.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.270+0000 7f6d60ff9640 1 -- 172.21.15.62:0/3141896448 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9208+0+0 (secure 0 0 0) 0x7f6d540cf9c0 con 0x7f6d641008e0 2024-06-10T00:32:49.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.270+0000 7f6d5bfff640 1 --2- 172.21.15.62:0/3141896448 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6d500a2be0 0x7f6d500a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:32:49.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.274+0000 7f6d60ff9640 1 -- 172.21.15.62:0/3141896448 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6d5409c4f0 con 0x7f6d641008e0 2024-06-10T00:32:49.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.458+0000 7f6d695f9640 1 -- 172.21.15.62:0/3141896448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6d6402d2e0 con 0x7f6d641008e0 2024-06-10T00:32:49.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.459+0000 7f6d60ff9640 1 -- 172.21.15.62:0/3141896448 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v99) ==== 103+0+11 (secure 0 0 0) 0x7f6d5409bdf0 con 0x7f6d641008e0 2024-06-10T00:32:49.459 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-06-10T00:32:49.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.461+0000 7f6d695f9640 1 -- 172.21.15.62:0/3141896448 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6d500a2be0 msgr2=0x7f6d500a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:32:49.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.461+0000 7f6d695f9640 1 --2- 172.21.15.62:0/3141896448 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6d500a2be0 0x7f6d500a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.461+0000 7f6d695f9640 1 -- 172.21.15.62:0/3141896448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d641008e0 msgr2=0x7f6d64069700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:49.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.461+0000 7f6d695f9640 1 --2- 172.21.15.62:0/3141896448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d641008e0 0x7f6d64069700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400e9c0 tx=0x7f6d5400ee80 comp rx=0 tx=0).stop 2024-06-10T00:32:49.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.461+0000 7f6d695f9640 1 -- 172.21.15.62:0/3141896448 shutdown_connections 2024-06-10T00:32:49.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.461+0000 7f6d695f9640 1 --2- 172.21.15.62:0/3141896448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d64102490 0x7f6d641a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.461+0000 7f6d695f9640 1 --2- 172.21.15.62:0/3141896448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64101ad0 0x7f6d6419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.461+0000 7f6d695f9640 1 --2- 172.21.15.62:0/3141896448 >> [v2:172.21.15.62:6800/3602410963,v1:172.21.15.62:6801/3602410963] conn(0x7f6d500a2be0 0x7f6d500a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.461+0000 7f6d695f9640 1 --2- 172.21.15.62:0/3141896448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d641008e0 0x7f6d64069700 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:49.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.461+0000 7f6d695f9640 1 -- 172.21.15.62:0/3141896448 >> 172.21.15.62:0/3141896448 conn(0x7f6d640fbe50 msgr2=0x7f6d64104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:49.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.461+0000 7f6d695f9640 1 -- 172.21.15.62:0/3141896448 shutdown_connections 2024-06-10T00:32:49.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:49.461+0000 7f6d695f9640 1 -- 172.21.15.62:0/3141896448 wait complete. 2024-06-10T00:32:49.472 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-06-10T00:32:49.472 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:32:49.531 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y[37693]: 2024-06-10T00:32:49.159+0000 7fb63cde0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:32:49.623 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z[38380]: 2024-06-10T00:32:49.229+0000 7f94b63b0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:32:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:49.849 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:49.667+0000 7f47305dd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:32:50.103 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1363117180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T00:32:50.103 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-mon[36549]: Standby manager daemon y restarted 2024-06-10T00:32:50.103 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-mon[36549]: Standby manager daemon y started 2024-06-10T00:32:50.103 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.105:0/4093137096' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T00:32:50.104 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.105:0/4093137096' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:32:50.104 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.105:0/4093137096' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T00:32:50.104 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.105:0/4093137096' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:32:50.104 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-mon[36549]: Standby manager daemon z restarted 2024-06-10T00:32:50.104 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-mon[36549]: Standby manager daemon z started 2024-06-10T00:32:50.104 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.138:0/1971857517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T00:32:50.104 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.138:0/1971857517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:32:50.104 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.138:0/1971857517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T00:32:50.104 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-mon[36549]: from='mgr.? 172.21.15.138:0/1971857517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:32:50.105 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:49 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3141896448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T00:32:50.105 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:49.911+0000 7f47305dd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:32:50.105 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1363117180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T00:32:50.106 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-mon[36321]: Standby manager daemon y restarted 2024-06-10T00:32:50.106 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-mon[36321]: Standby manager daemon y started 2024-06-10T00:32:50.106 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.105:0/4093137096' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T00:32:50.106 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.105:0/4093137096' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:32:50.106 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.105:0/4093137096' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T00:32:50.106 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.105:0/4093137096' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:32:50.106 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-mon[36321]: Standby manager daemon z restarted 2024-06-10T00:32:50.106 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-mon[36321]: Standby manager daemon z started 2024-06-10T00:32:50.106 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.138:0/1971857517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T00:32:50.106 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.138:0/1971857517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:32:50.106 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.138:0/1971857517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T00:32:50.106 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-mon[36321]: from='mgr.? 172.21.15.138:0/1971857517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:32:50.107 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:49 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3141896448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T00:32:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1363117180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T00:32:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-mon[32390]: Standby manager daemon y restarted 2024-06-10T00:32:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-mon[32390]: Standby manager daemon y started 2024-06-10T00:32:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.105:0/4093137096' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T00:32:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.105:0/4093137096' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:32:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.105:0/4093137096' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T00:32:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.105:0/4093137096' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:32:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-mon[32390]: Standby manager daemon z restarted 2024-06-10T00:32:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-mon[32390]: Standby manager daemon z started 2024-06-10T00:32:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.138:0/1971857517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T00:32:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.138:0/1971857517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:32:50.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.138:0/1971857517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T00:32:50.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-mon[32390]: from='mgr.? 172.21.15.138:0/1971857517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:32:50.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:49 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3141896448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T00:32:50.358 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:50.182+0000 7f4692edd700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:32:50.358 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:50.182+0000 7f4692edd700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:32:50.358 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:50.182+0000 7f4692edd700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:32:50.358 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:50.182+0000 7f4692edd700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:32:50.358 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:50.182+0000 7f4692edd700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:32:50.358 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:50.183+0000 7f468f6d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:32:50.358 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:50.183+0000 7f468f6d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:32:50.359 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:50.183+0000 7f468f6d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:32:50.359 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:50.183+0000 7f468f6d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:32:50.359 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:32:50.183+0000 7f468f6d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:32:51.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-06-10T00:32:51.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: Active manager daemon x restarted 2024-06-10T00:32:51.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: Activating manager daemon x 2024-06-10T00:32:51.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: osdmap e100: 12 total, 12 up, 12 in 2024-06-10T00:32:51.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: mgrmap e21: x(active, starting, since 0.0149334s), standbys: z, y 2024-06-10T00:32:51.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:32:51.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:32:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:32:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T00:32:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T00:32:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T00:32:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T00:32:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T00:32:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T00:32:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T00:32:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T00:32:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T00:32:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T00:32:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T00:32:51.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T00:32:51.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T00:32:51.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T00:32:51.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T00:32:51.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:32:51.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:32:51.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:32:51.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:32:51.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:32:51.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:32:51.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:32:51.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:32:51.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:32:51.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:32:51.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:32:51.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:32:51.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T00:32:51.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T00:32:51.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T00:32:51.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: Manager daemon x is now available 2024-06-10T00:32:51.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:32:51.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T00:32:51.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:50 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T00:32:51.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-06-10T00:32:51.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: Active manager daemon x restarted 2024-06-10T00:32:51.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: Activating manager daemon x 2024-06-10T00:32:51.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: osdmap e100: 12 total, 12 up, 12 in 2024-06-10T00:32:51.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: mgrmap e21: x(active, starting, since 0.0149334s), standbys: z, y 2024-06-10T00:32:51.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:32:51.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:32:51.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:32:51.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T00:32:51.124 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T00:32:51.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T00:32:51.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T00:32:51.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T00:32:51.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T00:32:51.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T00:32:51.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T00:32:51.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T00:32:51.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T00:32:51.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T00:32:51.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T00:32:51.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T00:32:51.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T00:32:51.125 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T00:32:51.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:32:51.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:32:51.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:32:51.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:32:51.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:32:51.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:32:51.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:32:51.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:32:51.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:32:51.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:32:51.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:32:51.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:32:51.126 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T00:32:51.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T00:32:51.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T00:32:51.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: Manager daemon x is now available 2024-06-10T00:32:51.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:32:51.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T00:32:51.127 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:50 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T00:32:51.178 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-06-10T00:32:51.178 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: Active manager daemon x restarted 2024-06-10T00:32:51.178 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: Activating manager daemon x 2024-06-10T00:32:51.179 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: osdmap e100: 12 total, 12 up, 12 in 2024-06-10T00:32:51.179 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: mgrmap e21: x(active, starting, since 0.0149334s), standbys: z, y 2024-06-10T00:32:51.179 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:32:51.179 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:32:51.179 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:32:51.179 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T00:32:51.179 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T00:32:51.179 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T00:32:51.179 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T00:32:51.179 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T00:32:51.179 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T00:32:51.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T00:32:51.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T00:32:51.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T00:32:51.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T00:32:51.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T00:32:51.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T00:32:51.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T00:32:51.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T00:32:51.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T00:32:51.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:32:51.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:32:51.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:32:51.180 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:32:51.181 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:32:51.181 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:32:51.181 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:32:51.181 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:32:51.181 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:32:51.181 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:32:51.181 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:32:51.181 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:32:51.181 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T00:32:51.181 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T00:32:51.181 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T00:32:51.181 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: Manager daemon x is now available 2024-06-10T00:32:51.182 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:32:51.182 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T00:32:51.182 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:50 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T00:32:51.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.603+0000 7f9140c52700 1 -- 172.21.15.62:0/565404740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913c1085d0 msgr2=0x7f913c1089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:51.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.603+0000 7f9140c52700 1 --2- 172.21.15.62:0/565404740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913c1085d0 0x7f913c1089b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f912c0099c0 tx=0x7f912c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:32:51.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.603+0000 7f9140c52700 1 -- 172.21.15.62:0/565404740 shutdown_connections 2024-06-10T00:32:51.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.603+0000 7f9140c52700 1 --2- 172.21.15.62:0/565404740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913c1085d0 0x7f913c1089b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:51.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.603+0000 7f9140c52700 1 --2- 172.21.15.62:0/565404740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c102ef0 0x7f913c10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:51.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.603+0000 7f9140c52700 1 --2- 172.21.15.62:0/565404740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f913c102530 0x7f913c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:51.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.603+0000 7f9140c52700 1 -- 172.21.15.62:0/565404740 >> 172.21.15.62:0/565404740 conn(0x7f913c079640 msgr2=0x7f913c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:51.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.603+0000 7f9140c52700 1 -- 172.21.15.62:0/565404740 shutdown_connections 2024-06-10T00:32:51.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.603+0000 7f9140c52700 1 -- 172.21.15.62:0/565404740 wait complete. 2024-06-10T00:32:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.604+0000 7f9140c52700 1 Processor -- start 2024-06-10T00:32:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.604+0000 7f9140c52700 1 -- start start 2024-06-10T00:32:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.604+0000 7f9140c52700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f913c102530 0x7f913c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.604+0000 7f9140c52700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913c102ef0 0x7f913c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.604+0000 7f9140c52700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c1085d0 0x7f913c196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.604+0000 7f9140c52700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f913c1106c0 con 0x7f913c102ef0 2024-06-10T00:32:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.604+0000 7f9140c52700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f913c110540 con 0x7f913c1085d0 2024-06-10T00:32:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.604+0000 7f9140c52700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f913c110840 con 0x7f913c102530 2024-06-10T00:32:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f913a59c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f913c102530 0x7f913c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:51.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f9139d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913c102ef0 0x7f913c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:51.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f913a59c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f913c102530 0x7f913c19c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:47610/0 (socket says 172.21.15.62:47610) 2024-06-10T00:32:51.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f9139d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913c102ef0 0x7f913c19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45334/0 (socket says 172.21.15.62:45334) 2024-06-10T00:32:51.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f913a59c700 1 -- 172.21.15.62:0/964352745 learned_addr learned my addr 172.21.15.62:0/964352745 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:51.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f913ad9d700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c1085d0 0x7f913c196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:51.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f9139d9b700 1 -- 172.21.15.62:0/964352745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f913c102530 msgr2=0x7f913c19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:51.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f9139d9b700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f913c102530 0x7f913c19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:51.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f9139d9b700 1 -- 172.21.15.62:0/964352745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c1085d0 msgr2=0x7f913c196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:51.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f9139d9b700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c1085d0 0x7f913c196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:51.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f9139d9b700 1 -- 172.21.15.62:0/964352745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c1970c0 con 0x7f913c102ef0 2024-06-10T00:32:51.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f913a59c700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f913c102530 0x7f913c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:32:51.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f913ad9d700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c1085d0 0x7f913c196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:32:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f9139d9b700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913c102ef0 0x7f913c19cce0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f913000e9f0 tx=0x7f913000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.605+0000 7f912b7fe700 1 -- 172.21.15.62:0/964352745 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130004220 con 0x7f913c102ef0 2024-06-10T00:32:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.606+0000 7f9140c52700 1 -- 172.21.15.62:0/964352745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913c1973b0 con 0x7f913c102ef0 2024-06-10T00:32:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.606+0000 7f912b7fe700 1 -- 172.21.15.62:0/964352745 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9130004d10 con 0x7f913c102ef0 2024-06-10T00:32:51.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.606+0000 7f912b7fe700 1 -- 172.21.15.62:0/964352745 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130005020 con 0x7f913c102ef0 2024-06-10T00:32:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.606+0000 7f9140c52700 1 -- 172.21.15.62:0/964352745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f913c1a3670 con 0x7f913c102ef0 2024-06-10T00:32:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.606+0000 7f9140c52700 1 -- 172.21.15.62:0/964352745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f913c10aa60 con 0x7f913c102ef0 2024-06-10T00:32:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.608+0000 7f912b7fe700 1 -- 172.21.15.62:0/964352745 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f9130004750 con 0x7f913c102ef0 2024-06-10T00:32:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.609+0000 7f912b7fe700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f91240a2a20 0x7f91240a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.609+0000 7f913a59c700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f91240a2a20 0x7f91240a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.609+0000 7f912b7fe700 1 -- 172.21.15.62:0/964352745 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f91300cf070 con 0x7f913c102ef0 2024-06-10T00:32:51.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.609+0000 7f913a59c700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f91240a2a20 0x7f91240a4ee0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f912c000c00 tx=0x7f912c005fb0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:51.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.614+0000 7f912b7fe700 1 -- 172.21.15.62:0/964352745 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f913009be80 con 0x7f913c102ef0 2024-06-10T00:32:51.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.870+0000 7f9140c52700 1 -- 172.21.15.62:0/964352745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f913c197e90 con 0x7f913c102ef0 2024-06-10T00:32:51.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.871+0000 7f912b7fe700 1 -- 172.21.15.62:0/964352745 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f913009b780 con 0x7f913c102ef0 2024-06-10T00:32:51.871 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:32:51.871 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:32:47.783154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:32:51.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.873+0000 7f9140c52700 1 -- 172.21.15.62:0/964352745 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f91240a2a20 msgr2=0x7f91240a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:51.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.873+0000 7f9140c52700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f91240a2a20 0x7f91240a4ee0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f912c000c00 tx=0x7f912c005fb0 comp rx=0 tx=0).stop 2024-06-10T00:32:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.873+0000 7f9140c52700 1 -- 172.21.15.62:0/964352745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913c102ef0 msgr2=0x7f913c19cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.873+0000 7f9140c52700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913c102ef0 0x7f913c19cce0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f913000e9f0 tx=0x7f913000ed00 comp rx=0 tx=0).stop 2024-06-10T00:32:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.873+0000 7f9140c52700 1 -- 172.21.15.62:0/964352745 shutdown_connections 2024-06-10T00:32:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.873+0000 7f9140c52700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f91240a2a20 0x7f91240a4ee0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.873+0000 7f9140c52700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c1085d0 0x7f913c196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:51.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.873+0000 7f9140c52700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f913c102530 0x7f913c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:51.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.873+0000 7f9140c52700 1 --2- 172.21.15.62:0/964352745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f913c102ef0 0x7f913c19cce0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:51.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.873+0000 7f9140c52700 1 -- 172.21.15.62:0/964352745 >> 172.21.15.62:0/964352745 conn(0x7f913c079640 msgr2=0x7f913c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:51.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.873+0000 7f9140c52700 1 -- 172.21.15.62:0/964352745 shutdown_connections 2024-06-10T00:32:51.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:51.873+0000 7f9140c52700 1 -- 172.21.15.62:0/964352745 wait complete. 2024-06-10T00:32:51.875 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-06-10T00:32:52.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:51 smithi105 ceph-mon[36321]: [10/Jun/2024:00:32:50] ENGINE Bus STARTING 2024-06-10T00:32:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:51 smithi105 ceph-mon[36321]: [10/Jun/2024:00:32:50] ENGINE Serving on https://172.21.15.62:7150 2024-06-10T00:32:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:51 smithi105 ceph-mon[36321]: mgrmap e22: x(active, since 1.01861s), standbys: z, y 2024-06-10T00:32:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:51 smithi105 ceph-mon[36321]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:32:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:51 smithi105 ceph-mon[36321]: [10/Jun/2024:00:32:50] ENGINE Serving on http://172.21.15.62:8765 2024-06-10T00:32:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:51 smithi105 ceph-mon[36321]: [10/Jun/2024:00:32:50] ENGINE Bus STARTED 2024-06-10T00:32:52.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:51 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/964352745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:32:52.325 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:51 smithi062 ceph-mon[32390]: [10/Jun/2024:00:32:50] ENGINE Bus STARTING 2024-06-10T00:32:52.325 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:51 smithi062 ceph-mon[32390]: [10/Jun/2024:00:32:50] ENGINE Serving on https://172.21.15.62:7150 2024-06-10T00:32:52.326 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:51 smithi062 ceph-mon[32390]: mgrmap e22: x(active, since 1.01861s), standbys: z, y 2024-06-10T00:32:52.326 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:51 smithi062 ceph-mon[32390]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:32:52.326 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:51 smithi062 ceph-mon[32390]: [10/Jun/2024:00:32:50] ENGINE Serving on http://172.21.15.62:8765 2024-06-10T00:32:52.326 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:51 smithi062 ceph-mon[32390]: [10/Jun/2024:00:32:50] ENGINE Bus STARTED 2024-06-10T00:32:52.326 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:51 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/964352745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:32:52.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:51 smithi138 ceph-mon[36549]: [10/Jun/2024:00:32:50] ENGINE Bus STARTING 2024-06-10T00:32:52.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:51 smithi138 ceph-mon[36549]: [10/Jun/2024:00:32:50] ENGINE Serving on https://172.21.15.62:7150 2024-06-10T00:32:52.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:51 smithi138 ceph-mon[36549]: mgrmap e22: x(active, since 1.01861s), standbys: z, y 2024-06-10T00:32:52.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:51 smithi138 ceph-mon[36549]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:32:52.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:51 smithi138 ceph-mon[36549]: [10/Jun/2024:00:32:50] ENGINE Serving on http://172.21.15.62:8765 2024-06-10T00:32:52.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:51 smithi138 ceph-mon[36549]: [10/Jun/2024:00:32:50] ENGINE Bus STARTED 2024-06-10T00:32:52.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:51 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/964352745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:32:53.057 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:52 smithi062 ceph-mon[32390]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:32:53.058 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:52 smithi062 ceph-mon[32390]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-06-10T00:32:53.058 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:52 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:53.058 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:52 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:53.058 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:52 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:53.058 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:52 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:53.086 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-06-10T00:32:53.086 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-06-10T00:32:53.086 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-10T00:32:53.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.208+0000 7fb0aaa88640 1 -- 172.21.15.62:0/4290514907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0a4104d90 msgr2=0x7fb0a4107180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:53.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.208+0000 7fb0aaa88640 1 --2- 172.21.15.62:0/4290514907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0a4104d90 0x7fb0a4107180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb090009860 tx=0x7fb09002a340 comp rx=0 tx=0).stop 2024-06-10T00:32:53.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.208+0000 7fb0aaa88640 1 -- 172.21.15.62:0/4290514907 shutdown_connections 2024-06-10T00:32:53.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.208+0000 7fb0aaa88640 1 --2- 172.21.15.62:0/4290514907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0a41076c0 0x7fb0a4109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.208+0000 7fb0aaa88640 1 --2- 172.21.15.62:0/4290514907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0a4104d90 0x7fb0a4107180 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.208+0000 7fb0aaa88640 1 --2- 172.21.15.62:0/4290514907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0a4102460 0x7fb0a4104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.208+0000 7fb0aaa88640 1 -- 172.21.15.62:0/4290514907 >> 172.21.15.62:0/4290514907 conn(0x7fb0a40fbe50 msgr2=0x7fb0a40fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:53.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.208+0000 7fb0aaa88640 1 -- 172.21.15.62:0/4290514907 shutdown_connections 2024-06-10T00:32:53.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.208+0000 7fb0aaa88640 1 -- 172.21.15.62:0/4290514907 wait complete. 2024-06-10T00:32:53.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.208+0000 7fb0aaa88640 1 Processor -- start 2024-06-10T00:32:53.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0aaa88640 1 -- start start 2024-06-10T00:32:53.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0aaa88640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0a4102460 0x7fb0a4195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:53.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0aaa88640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0a4104d90 0x7fb0a4196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:53.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0aaa88640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0a41076c0 0x7fb0a419d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:53.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0aaa88640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a410b4a0 con 0x7fb0a4104d90 2024-06-10T00:32:53.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0aaa88640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a410b320 con 0x7fb0a41076c0 2024-06-10T00:32:53.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0aaa88640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a410b620 con 0x7fb0a4102460 2024-06-10T00:32:53.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0a3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0a4104d90 0x7fb0a4196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:53.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0a3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0a4104d90 0x7fb0a4196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45374/0 (socket says 172.21.15.62:45374) 2024-06-10T00:32:53.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0a3fff640 1 -- 172.21.15.62:0/1752308529 learned_addr learned my addr 172.21.15.62:0/1752308529 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:53.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0a8824640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0a4102460 0x7fb0a4195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:53.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0a9025640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0a41076c0 0x7fb0a419d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:53.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0a3fff640 1 -- 172.21.15.62:0/1752308529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0a4102460 msgr2=0x7fb0a4195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:53.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0a3fff640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0a4102460 0x7fb0a4195b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0a3fff640 1 -- 172.21.15.62:0/1752308529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0a41076c0 msgr2=0x7fb0a419d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0a3fff640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0a41076c0 0x7fb0a419d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0a3fff640 1 -- 172.21.15.62:0/1752308529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a419d740 con 0x7fb0a4104d90 2024-06-10T00:32:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0a8824640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0a4102460 0x7fb0a4195b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:32:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0a9025640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0a41076c0 0x7fb0a419d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:32:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.209+0000 7fb0a3fff640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0a4104d90 0x7fb0a4196060 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb0900305b0 tx=0x7fb090033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.210+0000 7fb0a1ffb640 1 -- 172.21.15.62:0/1752308529 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09003f020 con 0x7fb0a4104d90 2024-06-10T00:32:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.210+0000 7fb0aaa88640 1 -- 172.21.15.62:0/1752308529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a419d9d0 con 0x7fb0a4104d90 2024-06-10T00:32:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.210+0000 7fb0a1ffb640 1 -- 172.21.15.62:0/1752308529 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb090030850 con 0x7fb0a4104d90 2024-06-10T00:32:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.210+0000 7fb0a1ffb640 1 -- 172.21.15.62:0/1752308529 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090037040 con 0x7fb0a4104d90 2024-06-10T00:32:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.210+0000 7fb0aaa88640 1 -- 172.21.15.62:0/1752308529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a419dec0 con 0x7fb0a4104d90 2024-06-10T00:32:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.211+0000 7fb0aaa88640 1 -- 172.21.15.62:0/1752308529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a40fd940 con 0x7fb0a4104d90 2024-06-10T00:32:53.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.212+0000 7fb0a1ffb640 1 -- 172.21.15.62:0/1752308529 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 137953+0+0 (secure 0 0 0) 0x7fb090042020 con 0x7fb0a4104d90 2024-06-10T00:32:53.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.213+0000 7fb0a1ffb640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb0940a2d00 0x7fb0940a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:53.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.213+0000 7fb0a1ffb640 1 -- 172.21.15.62:0/1752308529 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fb0900efc30 con 0x7fb0a4104d90 2024-06-10T00:32:53.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.213+0000 7fb0a8824640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb0940a2d00 0x7fb0940a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:53.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.214+0000 7fb0a8824640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb0940a2d00 0x7fb0940a51c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb098005fd0 tx=0x7fb098015040 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:53.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.218+0000 7fb0a1ffb640 1 -- 172.21.15.62:0/1752308529 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb090049020 con 0x7fb0a4104d90 2024-06-10T00:32:53.281 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:52 smithi105 ceph-mon[36321]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:32:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:52 smithi105 ceph-mon[36321]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-06-10T00:32:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:52 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:52 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:52 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:52 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:53.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:52 smithi138 ceph-mon[36549]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:32:53.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:52 smithi138 ceph-mon[36549]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-06-10T00:32:53.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:52 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:53.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:52 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:53.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:52 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:53.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:52 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:53.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.429+0000 7fb0aaa88640 1 -- 172.21.15.62:0/1752308529 --> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb0a4061de0 con 0x7fb0940a2d00 2024-06-10T00:32:53.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.440+0000 7fb0a1ffb640 1 -- 172.21.15.62:0/1752308529 <== mgr.15333 v2:172.21.15.62:6800/518034449 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339578 (secure 0 0 0) 0x7fb0a4061de0 con 0x7fb0940a2d00 2024-06-10T00:32:53.445 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:32:53.447 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-06-10T00:32:53.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.444+0000 7fb0aaa88640 1 -- 172.21.15.62:0/1752308529 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb0940a2d00 msgr2=0x7fb0940a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.444+0000 7fb0aaa88640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb0940a2d00 0x7fb0940a51c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb098005fd0 tx=0x7fb098015040 comp rx=0 tx=0).stop 2024-06-10T00:32:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.444+0000 7fb0aaa88640 1 -- 172.21.15.62:0/1752308529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0a4104d90 msgr2=0x7fb0a4196060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.444+0000 7fb0aaa88640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0a4104d90 0x7fb0a4196060 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb0900305b0 tx=0x7fb090033d90 comp rx=0 tx=0).stop 2024-06-10T00:32:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.444+0000 7fb0aaa88640 1 -- 172.21.15.62:0/1752308529 shutdown_connections 2024-06-10T00:32:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.444+0000 7fb0aaa88640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb0940a2d00 0x7fb0940a51c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.444+0000 7fb0aaa88640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0a41076c0 0x7fb0a419d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.444+0000 7fb0aaa88640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0a4104d90 0x7fb0a4196060 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.444+0000 7fb0aaa88640 1 --2- 172.21.15.62:0/1752308529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0a4102460 0x7fb0a4195b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.444+0000 7fb0aaa88640 1 -- 172.21.15.62:0/1752308529 >> 172.21.15.62:0/1752308529 conn(0x7fb0a40fbe50 msgr2=0x7fb0a4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.444+0000 7fb0aaa88640 1 -- 172.21.15.62:0/1752308529 shutdown_connections 2024-06-10T00:32:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.444+0000 7fb0aaa88640 1 -- 172.21.15.62:0/1752308529 wait complete. 2024-06-10T00:32:53.456 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-06-10T00:32:51.934091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194915,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2124,"num_read_kb":1226,"num_write":569,"num_write_kb":4418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":950,"ondisk_log_size":950,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387164,"kb_used_data":17176,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124554660,"statfs":{"total":1151940427776,"available":1151543971840,"internally_reserved":0,"allocated":17588224,"data_stored":10356804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":378713403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.993748"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199389+0000","last_change":"2024-06-10T00:28:37.657234+0000","last_active":"2024-06-10T00:32:50.199389+0000","last_peered":"2024-06-10T00:32:50.199389+0000","last_clean":"2024-06-10T00:32:50.199389+0000","last_became_active":"2024-06-10T00:28:37.656879+0000","last_became_peered":"2024-06-10T00:28:37.656879+0000","last_unstale":"2024-06-10T00:32:50.199389+0000","last_undegraded":"2024-06-10T00:32:50.199389+0000","last_fullsized":"2024-06-10T00:32:50.199389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:01:02.907597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193442+0000","last_change":"2024-06-10T00:28:37.656440+0000","last_active":"2024-06-10T00:32:50.193442+0000","last_peered":"2024-06-10T00:32:50.193442+0000","last_clean":"2024-06-10T00:32:50.193442+0000","last_became_active":"2024-06-10T00:28:37.656113+0000","last_became_peered":"2024-06-10T00:28:37.656113+0000","last_unstale":"2024-06-10T00:32:50.193442+0000","last_undegraded":"2024-06-10T00:32:50.193442+0000","last_fullsized":"2024-06-10T00:32:50.193442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:49:31.447951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193070+0000","last_change":"2024-06-10T00:28:37.659114+0000","last_active":"2024-06-10T00:32:50.193070+0000","last_peered":"2024-06-10T00:32:50.193070+0000","last_clean":"2024-06-10T00:32:50.193070+0000","last_became_active":"2024-06-10T00:28:37.658763+0000","last_became_peered":"2024-06-10T00:28:37.658763+0000","last_unstale":"2024-06-10T00:32:50.193070+0000","last_undegraded":"2024-06-10T00:32:50.193070+0000","last_fullsized":"2024-06-10T00:32:50.193070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:39:54.963590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921896+0000","last_change":"2024-06-10T00:28:37.656786+0000","last_active":"2024-06-10T00:32:49.921896+0000","last_peered":"2024-06-10T00:32:49.921896+0000","last_clean":"2024-06-10T00:32:49.921896+0000","last_became_active":"2024-06-10T00:28:37.656073+0000","last_became_peered":"2024-06-10T00:28:37.656073+0000","last_unstale":"2024-06-10T00:32:49.921896+0000","last_undegraded":"2024-06-10T00:32:49.921896+0000","last_fullsized":"2024-06-10T00:32:49.921896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:19.176608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031746+0000","last_change":"2024-06-10T00:28:37.650250+0000","last_active":"2024-06-10T00:32:50.031746+0000","last_peered":"2024-06-10T00:32:50.031746+0000","last_clean":"2024-06-10T00:32:50.031746+0000","last_became_active":"2024-06-10T00:28:37.649852+0000","last_became_peered":"2024-06-10T00:28:37.649852+0000","last_unstale":"2024-06-10T00:32:50.031746+0000","last_undegraded":"2024-06-10T00:32:50.031746+0000","last_fullsized":"2024-06-10T00:32:50.031746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:39.492884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042392+0000","last_change":"2024-06-10T00:28:37.646042+0000","last_active":"2024-06-10T00:32:50.042392+0000","last_peered":"2024-06-10T00:32:50.042392+0000","last_clean":"2024-06-10T00:32:50.042392+0000","last_became_active":"2024-06-10T00:28:37.644591+0000","last_became_peered":"2024-06-10T00:28:37.644591+0000","last_unstale":"2024-06-10T00:32:50.042392+0000","last_undegraded":"2024-06-10T00:32:50.042392+0000","last_fullsized":"2024-06-10T00:32:50.042392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:29.932303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134284+0000","last_change":"2024-06-10T00:28:37.664076+0000","last_active":"2024-06-10T00:32:50.134284+0000","last_peered":"2024-06-10T00:32:50.134284+0000","last_clean":"2024-06-10T00:32:50.134284+0000","last_became_active":"2024-06-10T00:28:37.663741+0000","last_became_peered":"2024-06-10T00:28:37.663741+0000","last_unstale":"2024-06-10T00:32:50.134284+0000","last_undegraded":"2024-06-10T00:32:50.134284+0000","last_fullsized":"2024-06-10T00:32:50.134284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:14.581409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032039+0000","last_change":"2024-06-10T00:28:37.648758+0000","last_active":"2024-06-10T00:32:50.032039+0000","last_peered":"2024-06-10T00:32:50.032039+0000","last_clean":"2024-06-10T00:32:50.032039+0000","last_became_active":"2024-06-10T00:28:37.648387+0000","last_became_peered":"2024-06-10T00:28:37.648387+0000","last_unstale":"2024-06-10T00:32:50.032039+0000","last_undegraded":"2024-06-10T00:32:50.032039+0000","last_fullsized":"2024-06-10T00:32:50.032039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:17.187265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.189039+0000","last_change":"2024-06-10T00:28:37.664650+0000","last_active":"2024-06-10T00:32:50.189039+0000","last_peered":"2024-06-10T00:32:50.189039+0000","last_clean":"2024-06-10T00:32:50.189039+0000","last_became_active":"2024-06-10T00:28:37.663178+0000","last_became_peered":"2024-06-10T00:28:37.663178+0000","last_unstale":"2024-06-10T00:32:50.189039+0000","last_undegraded":"2024-06-10T00:32:50.189039+0000","last_fullsized":"2024-06-10T00:32:50.189039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:46.032078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042870+0000","last_change":"2024-06-10T00:28:37.658353+0000","last_active":"2024-06-10T00:32:50.042870+0000","last_peered":"2024-06-10T00:32:50.042870+0000","last_clean":"2024-06-10T00:32:50.042870+0000","last_became_active":"2024-06-10T00:28:37.657961+0000","last_became_peered":"2024-06-10T00:28:37.657961+0000","last_unstale":"2024-06-10T00:32:50.042870+0000","last_undegraded":"2024-06-10T00:32:50.042870+0000","last_fullsized":"2024-06-10T00:32:50.042870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:22:27.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134878+0000","last_change":"2024-06-10T00:28:37.663823+0000","last_active":"2024-06-10T00:32:50.134878+0000","last_peered":"2024-06-10T00:32:50.134878+0000","last_clean":"2024-06-10T00:32:50.134878+0000","last_became_active":"2024-06-10T00:28:37.663499+0000","last_became_peered":"2024-06-10T00:28:37.663499+0000","last_unstale":"2024-06-10T00:32:50.134878+0000","last_undegraded":"2024-06-10T00:32:50.134878+0000","last_fullsized":"2024-06-10T00:32:50.134878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.100923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921877+0000","last_change":"2024-06-10T00:30:24.112722+0000","last_active":"2024-06-10T00:32:49.921877+0000","last_peered":"2024-06-10T00:32:49.921877+0000","last_clean":"2024-06-10T00:32:49.921877+0000","last_became_active":"2024-06-10T00:30:24.112325+0000","last_became_peered":"2024-06-10T00:30:24.112325+0000","last_unstale":"2024-06-10T00:32:49.921877+0000","last_undegraded":"2024-06-10T00:32:49.921877+0000","last_fullsized":"2024-06-10T00:32:49.921877+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:26.671583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133292+0000","last_change":"2024-06-10T00:28:37.649997+0000","last_active":"2024-06-10T00:32:50.133292+0000","last_peered":"2024-06-10T00:32:50.133292+0000","last_clean":"2024-06-10T00:32:50.133292+0000","last_became_active":"2024-06-10T00:28:37.649494+0000","last_became_peered":"2024-06-10T00:28:37.649494+0000","last_unstale":"2024-06-10T00:32:50.133292+0000","last_undegraded":"2024-06-10T00:32:50.133292+0000","last_fullsized":"2024-06-10T00:32:50.133292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:44:07.467206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192453+0000","last_change":"2024-06-10T00:28:37.653064+0000","last_active":"2024-06-10T00:32:50.192453+0000","last_peered":"2024-06-10T00:32:50.192453+0000","last_clean":"2024-06-10T00:32:50.192453+0000","last_became_active":"2024-06-10T00:28:37.652684+0000","last_became_peered":"2024-06-10T00:28:37.652684+0000","last_unstale":"2024-06-10T00:32:50.192453+0000","last_undegraded":"2024-06-10T00:32:50.192453+0000","last_fullsized":"2024-06-10T00:32:50.192453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:30.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192384+0000","last_change":"2024-06-10T00:28:37.655570+0000","last_active":"2024-06-10T00:32:50.192384+0000","last_peered":"2024-06-10T00:32:50.192384+0000","last_clean":"2024-06-10T00:32:50.192384+0000","last_became_active":"2024-06-10T00:28:37.655241+0000","last_became_peered":"2024-06-10T00:28:37.655241+0000","last_unstale":"2024-06-10T00:32:50.192384+0000","last_undegraded":"2024-06-10T00:32:50.192384+0000","last_fullsized":"2024-06-10T00:32:50.192384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:07:03.543064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188211+0000","last_change":"2024-06-10T00:28:37.665607+0000","last_active":"2024-06-10T00:32:50.188211+0000","last_peered":"2024-06-10T00:32:50.188211+0000","last_clean":"2024-06-10T00:32:50.188211+0000","last_became_active":"2024-06-10T00:28:37.664943+0000","last_became_peered":"2024-06-10T00:28:37.664943+0000","last_unstale":"2024-06-10T00:32:50.188211+0000","last_undegraded":"2024-06-10T00:32:50.188211+0000","last_fullsized":"2024-06-10T00:32:50.188211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:43:02.940298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199188+0000","last_change":"2024-06-10T00:28:37.658214+0000","last_active":"2024-06-10T00:32:50.199188+0000","last_peered":"2024-06-10T00:32:50.199188+0000","last_clean":"2024-06-10T00:32:50.199188+0000","last_became_active":"2024-06-10T00:28:37.657295+0000","last_became_peered":"2024-06-10T00:28:37.657295+0000","last_unstale":"2024-06-10T00:32:50.199188+0000","last_undegraded":"2024-06-10T00:32:50.199188+0000","last_fullsized":"2024-06-10T00:32:50.199188+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:20.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922079+0000","last_change":"2024-06-10T00:28:37.656645+0000","last_active":"2024-06-10T00:32:49.922079+0000","last_peered":"2024-06-10T00:32:49.922079+0000","last_clean":"2024-06-10T00:32:49.922079+0000","last_became_active":"2024-06-10T00:28:37.656157+0000","last_became_peered":"2024-06-10T00:28:37.656157+0000","last_unstale":"2024-06-10T00:32:49.922079+0000","last_undegraded":"2024-06-10T00:32:49.922079+0000","last_fullsized":"2024-06-10T00:32:49.922079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:00.953804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921572+0000","last_change":"2024-06-10T00:28:37.664336+0000","last_active":"2024-06-10T00:32:49.921572+0000","last_peered":"2024-06-10T00:32:49.921572+0000","last_clean":"2024-06-10T00:32:49.921572+0000","last_became_active":"2024-06-10T00:28:37.663960+0000","last_became_peered":"2024-06-10T00:28:37.663960+0000","last_unstale":"2024-06-10T00:32:49.921572+0000","last_undegraded":"2024-06-10T00:32:49.921572+0000","last_fullsized":"2024-06-10T00:32:49.921572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:59:32.774442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920810+0000","last_change":"2024-06-10T00:28:48.705930+0000","last_active":"2024-06-10T00:32:49.920810+0000","last_peered":"2024-06-10T00:32:49.920810+0000","last_clean":"2024-06-10T00:32:49.920810+0000","last_became_active":"2024-06-10T00:28:48.705558+0000","last_became_peered":"2024-06-10T00:28:48.705558+0000","last_unstale":"2024-06-10T00:32:49.920810+0000","last_undegraded":"2024-06-10T00:32:49.920810+0000","last_fullsized":"2024-06-10T00:32:49.920810+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:41:44.914378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"99'14","reported_seq":260,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132362+0000","last_change":"2024-06-10T00:29:23.862141+0000","last_active":"2024-06-10T00:32:50.132362+0000","last_peered":"2024-06-10T00:32:50.132362+0000","last_clean":"2024-06-10T00:32:50.132362+0000","last_became_active":"2024-06-10T00:29:23.857051+0000","last_became_peered":"2024-06-10T00:29:23.857051+0000","last_unstale":"2024-06-10T00:32:50.132362+0000","last_undegraded":"2024-06-10T00:32:50.132362+0000","last_fullsized":"2024-06-10T00:32:50.132362+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:41:41.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032309+0000","last_change":"2024-06-10T00:28:37.653938+0000","last_active":"2024-06-10T00:32:50.032309+0000","last_peered":"2024-06-10T00:32:50.032309+0000","last_clean":"2024-06-10T00:32:50.032309+0000","last_became_active":"2024-06-10T00:28:37.653629+0000","last_became_peered":"2024-06-10T00:28:37.653629+0000","last_unstale":"2024-06-10T00:32:50.032309+0000","last_undegraded":"2024-06-10T00:32:50.032309+0000","last_fullsized":"2024-06-10T00:32:50.032309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:21:39.285961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921432+0000","last_change":"2024-06-10T00:28:48.704413+0000","last_active":"2024-06-10T00:32:49.921432+0000","last_peered":"2024-06-10T00:32:49.921432+0000","last_clean":"2024-06-10T00:32:49.921432+0000","last_became_active":"2024-06-10T00:28:48.703972+0000","last_became_peered":"2024-06-10T00:28:48.703972+0000","last_unstale":"2024-06-10T00:32:49.921432+0000","last_undegraded":"2024-06-10T00:32:49.921432+0000","last_fullsized":"2024-06-10T00:32:49.921432+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:51.927435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032677+0000","last_change":"2024-06-10T00:28:34.445413+0000","last_active":"2024-06-10T00:32:50.032677+0000","last_peered":"2024-06-10T00:32:50.032677+0000","last_clean":"2024-06-10T00:32:50.032677+0000","last_became_active":"2024-06-10T00:28:34.445077+0000","last_became_peered":"2024-06-10T00:28:34.445077+0000","last_unstale":"2024-06-10T00:32:50.032677+0000","last_undegraded":"2024-06-10T00:32:50.032677+0000","last_fullsized":"2024-06-10T00:32:50.032677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:19:19.535976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193506+0000","last_change":"2024-06-10T00:28:37.657123+0000","last_active":"2024-06-10T00:32:50.193506+0000","last_peered":"2024-06-10T00:32:50.193506+0000","last_clean":"2024-06-10T00:32:50.193506+0000","last_became_active":"2024-06-10T00:28:37.656791+0000","last_became_peered":"2024-06-10T00:28:37.656791+0000","last_unstale":"2024-06-10T00:32:50.193506+0000","last_undegraded":"2024-06-10T00:32:50.193506+0000","last_fullsized":"2024-06-10T00:32:50.193506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:11.102478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921858+0000","last_change":"2024-06-10T00:28:48.705989+0000","last_active":"2024-06-10T00:32:49.921858+0000","last_peered":"2024-06-10T00:32:49.921858+0000","last_clean":"2024-06-10T00:32:49.921858+0000","last_became_active":"2024-06-10T00:28:48.705555+0000","last_became_peered":"2024-06-10T00:28:48.705555+0000","last_unstale":"2024-06-10T00:32:49.921858+0000","last_undegraded":"2024-06-10T00:32:49.921858+0000","last_fullsized":"2024-06-10T00:32:49.921858+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:36.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"100'13","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032133+0000","last_change":"2024-06-10T00:28:34.400117+0000","last_active":"2024-06-10T00:32:50.032133+0000","last_peered":"2024-06-10T00:32:50.032133+0000","last_clean":"2024-06-10T00:32:50.032133+0000","last_became_active":"2024-06-10T00:28:34.399743+0000","last_became_peered":"2024-06-10T00:28:34.399743+0000","last_unstale":"2024-06-10T00:32:50.032133+0000","last_undegraded":"2024-06-10T00:32:50.032133+0000","last_fullsized":"2024-06-10T00:32:50.032133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:20:57.580146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188627+0000","last_change":"2024-06-10T00:28:37.664005+0000","last_active":"2024-06-10T00:32:50.188627+0000","last_peered":"2024-06-10T00:32:50.188627+0000","last_clean":"2024-06-10T00:32:50.188627+0000","last_became_active":"2024-06-10T00:28:37.661297+0000","last_became_peered":"2024-06-10T00:28:37.661297+0000","last_unstale":"2024-06-10T00:32:50.188627+0000","last_undegraded":"2024-06-10T00:32:50.188627+0000","last_fullsized":"2024-06-10T00:32:50.188627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:57.795080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921561+0000","last_change":"2024-06-10T00:29:23.859091+0000","last_active":"2024-06-10T00:32:49.921561+0000","last_peered":"2024-06-10T00:32:49.921561+0000","last_clean":"2024-06-10T00:32:49.921561+0000","last_became_active":"2024-06-10T00:29:23.858485+0000","last_became_peered":"2024-06-10T00:29:23.858485+0000","last_unstale":"2024-06-10T00:32:49.921561+0000","last_undegraded":"2024-06-10T00:32:49.921561+0000","last_fullsized":"2024-06-10T00:32:49.921561+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:07:10.925323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031945+0000","last_change":"2024-06-10T00:28:34.400588+0000","last_active":"2024-06-10T00:32:50.031945+0000","last_peered":"2024-06-10T00:32:50.031945+0000","last_clean":"2024-06-10T00:32:50.031945+0000","last_became_active":"2024-06-10T00:28:34.400027+0000","last_became_peered":"2024-06-10T00:28:34.400027+0000","last_unstale":"2024-06-10T00:32:50.031945+0000","last_undegraded":"2024-06-10T00:32:50.031945+0000","last_fullsized":"2024-06-10T00:32:50.031945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:25:59.890076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039025+0000","last_change":"2024-06-10T00:28:37.659401+0000","last_active":"2024-06-10T00:32:50.039025+0000","last_peered":"2024-06-10T00:32:50.039025+0000","last_clean":"2024-06-10T00:32:50.039025+0000","last_became_active":"2024-06-10T00:28:37.659056+0000","last_became_peered":"2024-06-10T00:28:37.659056+0000","last_unstale":"2024-06-10T00:32:50.039025+0000","last_undegraded":"2024-06-10T00:32:50.039025+0000","last_fullsized":"2024-06-10T00:32:50.039025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:35:01.415630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921379+0000","last_change":"2024-06-10T00:28:48.702961+0000","last_active":"2024-06-10T00:32:49.921379+0000","last_peered":"2024-06-10T00:32:49.921379+0000","last_clean":"2024-06-10T00:32:49.921379+0000","last_became_active":"2024-06-10T00:28:48.702502+0000","last_became_peered":"2024-06-10T00:28:48.702502+0000","last_unstale":"2024-06-10T00:32:49.921379+0000","last_undegraded":"2024-06-10T00:32:49.921379+0000","last_fullsized":"2024-06-10T00:32:49.921379+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:54.881595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'9","reported_seq":244,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.191304+0000","last_change":"2024-06-10T00:28:34.396310+0000","last_active":"2024-06-10T00:32:50.191304+0000","last_peered":"2024-06-10T00:32:50.191304+0000","last_clean":"2024-06-10T00:32:50.191304+0000","last_became_active":"2024-06-10T00:28:34.395401+0000","last_became_peered":"2024-06-10T00:28:34.395401+0000","last_unstale":"2024-06-10T00:32:50.191304+0000","last_undegraded":"2024-06-10T00:32:50.191304+0000","last_fullsized":"2024-06-10T00:32:50.191304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:33:54.300308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031647+0000","last_change":"2024-06-10T00:28:37.651427+0000","last_active":"2024-06-10T00:32:50.031647+0000","last_peered":"2024-06-10T00:32:50.031647+0000","last_clean":"2024-06-10T00:32:50.031647+0000","last_became_active":"2024-06-10T00:28:37.650510+0000","last_became_peered":"2024-06-10T00:28:37.650510+0000","last_unstale":"2024-06-10T00:32:50.031647+0000","last_undegraded":"2024-06-10T00:32:50.031647+0000","last_fullsized":"2024-06-10T00:32:50.031647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:42.953919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920970+0000","last_change":"2024-06-10T00:28:48.705918+0000","last_active":"2024-06-10T00:32:49.920970+0000","last_peered":"2024-06-10T00:32:49.920970+0000","last_clean":"2024-06-10T00:32:49.920970+0000","last_became_active":"2024-06-10T00:28:48.705462+0000","last_became_peered":"2024-06-10T00:28:48.705462+0000","last_unstale":"2024-06-10T00:32:49.920970+0000","last_undegraded":"2024-06-10T00:32:49.920970+0000","last_fullsized":"2024-06-10T00:32:49.920970+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:59:23.176511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042358+0000","last_change":"2024-06-10T00:28:34.393199+0000","last_active":"2024-06-10T00:32:50.042358+0000","last_peered":"2024-06-10T00:32:50.042358+0000","last_clean":"2024-06-10T00:32:50.042358+0000","last_became_active":"2024-06-10T00:28:34.392777+0000","last_became_peered":"2024-06-10T00:28:34.392777+0000","last_unstale":"2024-06-10T00:32:50.042358+0000","last_undegraded":"2024-06-10T00:32:50.042358+0000","last_fullsized":"2024-06-10T00:32:50.042358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:29.614242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038961+0000","last_change":"2024-06-10T00:30:24.229721+0000","last_active":"2024-06-10T00:32:50.038961+0000","last_peered":"2024-06-10T00:32:50.038961+0000","last_clean":"2024-06-10T00:32:50.038961+0000","last_became_active":"2024-06-10T00:30:24.229407+0000","last_became_peered":"2024-06-10T00:30:24.229407+0000","last_unstale":"2024-06-10T00:32:50.038961+0000","last_undegraded":"2024-06-10T00:32:50.038961+0000","last_fullsized":"2024-06-10T00:32:50.038961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:39.523197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199084+0000","last_change":"2024-06-10T00:29:23.780971+0000","last_active":"2024-06-10T00:32:50.199084+0000","last_peered":"2024-06-10T00:32:50.199084+0000","last_clean":"2024-06-10T00:32:50.199084+0000","last_became_active":"2024-06-10T00:29:23.780034+0000","last_became_peered":"2024-06-10T00:29:23.780034+0000","last_unstale":"2024-06-10T00:32:50.199084+0000","last_undegraded":"2024-06-10T00:32:50.199084+0000","last_fullsized":"2024-06-10T00:32:50.199084+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:47:06.444181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132954+0000","last_change":"2024-06-10T00:28:34.392486+0000","last_active":"2024-06-10T00:32:50.132954+0000","last_peered":"2024-06-10T00:32:50.132954+0000","last_clean":"2024-06-10T00:32:50.132954+0000","last_became_active":"2024-06-10T00:28:34.392141+0000","last_became_peered":"2024-06-10T00:28:34.392141+0000","last_unstale":"2024-06-10T00:32:50.132954+0000","last_undegraded":"2024-06-10T00:32:50.132954+0000","last_fullsized":"2024-06-10T00:32:50.132954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:10:01.534195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031869+0000","last_change":"2024-06-10T00:28:37.651033+0000","last_active":"2024-06-10T00:32:50.031869+0000","last_peered":"2024-06-10T00:32:50.031869+0000","last_clean":"2024-06-10T00:32:50.031869+0000","last_became_active":"2024-06-10T00:28:37.650505+0000","last_became_peered":"2024-06-10T00:28:37.650505+0000","last_unstale":"2024-06-10T00:32:50.031869+0000","last_undegraded":"2024-06-10T00:32:50.031869+0000","last_fullsized":"2024-06-10T00:32:50.031869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:54.965865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193367+0000","last_change":"2024-06-10T00:28:48.710748+0000","last_active":"2024-06-10T00:32:50.193367+0000","last_peered":"2024-06-10T00:32:50.193367+0000","last_clean":"2024-06-10T00:32:50.193367+0000","last_became_active":"2024-06-10T00:28:48.710353+0000","last_became_peered":"2024-06-10T00:28:48.710353+0000","last_unstale":"2024-06-10T00:32:50.193367+0000","last_undegraded":"2024-06-10T00:32:50.193367+0000","last_fullsized":"2024-06-10T00:32:50.193367+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:30:47.546650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038780+0000","last_change":"2024-06-10T00:28:34.398911+0000","last_active":"2024-06-10T00:32:50.038780+0000","last_peered":"2024-06-10T00:32:50.038780+0000","last_clean":"2024-06-10T00:32:50.038780+0000","last_became_active":"2024-06-10T00:28:34.398373+0000","last_became_peered":"2024-06-10T00:28:34.398373+0000","last_unstale":"2024-06-10T00:32:50.038780+0000","last_undegraded":"2024-06-10T00:32:50.038780+0000","last_fullsized":"2024-06-10T00:32:50.038780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:05:34.352825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921526+0000","last_change":"2024-06-10T00:28:37.647492+0000","last_active":"2024-06-10T00:32:49.921526+0000","last_peered":"2024-06-10T00:32:49.921526+0000","last_clean":"2024-06-10T00:32:49.921526+0000","last_became_active":"2024-06-10T00:28:37.647103+0000","last_became_peered":"2024-06-10T00:28:37.647103+0000","last_unstale":"2024-06-10T00:32:49.921526+0000","last_undegraded":"2024-06-10T00:32:49.921526+0000","last_fullsized":"2024-06-10T00:32:49.921526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:10.429256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031883+0000","last_change":"2024-06-10T00:28:48.703095+0000","last_active":"2024-06-10T00:32:50.031883+0000","last_peered":"2024-06-10T00:32:50.031883+0000","last_clean":"2024-06-10T00:32:50.031883+0000","last_became_active":"2024-06-10T00:28:48.702687+0000","last_became_peered":"2024-06-10T00:28:48.702687+0000","last_unstale":"2024-06-10T00:32:50.031883+0000","last_undegraded":"2024-06-10T00:32:50.031883+0000","last_fullsized":"2024-06-10T00:32:50.031883+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:10:20.074173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":423,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.191573+0000","last_change":"2024-06-10T00:28:34.394807+0000","last_active":"2024-06-10T00:32:50.191573+0000","last_peered":"2024-06-10T00:32:50.191573+0000","last_clean":"2024-06-10T00:32:50.191573+0000","last_became_active":"2024-06-10T00:28:34.394022+0000","last_became_peered":"2024-06-10T00:28:34.394022+0000","last_unstale":"2024-06-10T00:32:50.191573+0000","last_undegraded":"2024-06-10T00:32:50.191573+0000","last_fullsized":"2024-06-10T00:32:50.191573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:21.626079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":370,"num_read_kb":370,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.189056+0000","last_change":"2024-06-10T00:28:37.662231+0000","last_active":"2024-06-10T00:32:50.189056+0000","last_peered":"2024-06-10T00:32:50.189056+0000","last_clean":"2024-06-10T00:32:50.189056+0000","last_became_active":"2024-06-10T00:28:37.661861+0000","last_became_peered":"2024-06-10T00:28:37.661861+0000","last_unstale":"2024-06-10T00:32:50.189056+0000","last_undegraded":"2024-06-10T00:32:50.189056+0000","last_fullsized":"2024-06-10T00:32:50.189056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:09.705349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921535+0000","last_change":"2024-06-10T00:28:48.706643+0000","last_active":"2024-06-10T00:32:49.921535+0000","last_peered":"2024-06-10T00:32:49.921535+0000","last_clean":"2024-06-10T00:32:49.921535+0000","last_became_active":"2024-06-10T00:28:48.706127+0000","last_became_peered":"2024-06-10T00:28:48.706127+0000","last_unstale":"2024-06-10T00:32:49.921535+0000","last_undegraded":"2024-06-10T00:32:49.921535+0000","last_fullsized":"2024-06-10T00:32:49.921535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:55.031303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134861+0000","last_change":"2024-06-10T00:28:34.392808+0000","last_active":"2024-06-10T00:32:50.134861+0000","last_peered":"2024-06-10T00:32:50.134861+0000","last_clean":"2024-06-10T00:32:50.134861+0000","last_became_active":"2024-06-10T00:28:34.392450+0000","last_became_peered":"2024-06-10T00:28:34.392450+0000","last_unstale":"2024-06-10T00:32:50.134861+0000","last_undegraded":"2024-06-10T00:32:50.134861+0000","last_fullsized":"2024-06-10T00:32:50.134861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:06:00.493316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193616+0000","last_change":"2024-06-10T00:28:37.656099+0000","last_active":"2024-06-10T00:32:50.193616+0000","last_peered":"2024-06-10T00:32:50.193616+0000","last_clean":"2024-06-10T00:32:50.193616+0000","last_became_active":"2024-06-10T00:28:37.655640+0000","last_became_peered":"2024-06-10T00:28:37.655640+0000","last_unstale":"2024-06-10T00:32:50.193616+0000","last_undegraded":"2024-06-10T00:32:50.193616+0000","last_fullsized":"2024-06-10T00:32:50.193616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:59:26.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922109+0000","last_change":"2024-06-10T00:28:48.702628+0000","last_active":"2024-06-10T00:32:49.922109+0000","last_peered":"2024-06-10T00:32:49.922109+0000","last_clean":"2024-06-10T00:32:49.922109+0000","last_became_active":"2024-06-10T00:28:48.702109+0000","last_became_peered":"2024-06-10T00:28:48.702109+0000","last_unstale":"2024-06-10T00:32:49.922109+0000","last_undegraded":"2024-06-10T00:32:49.922109+0000","last_fullsized":"2024-06-10T00:32:49.922109+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:25.262348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039027+0000","last_change":"2024-06-10T00:28:34.446788+0000","last_active":"2024-06-10T00:32:50.039027+0000","last_peered":"2024-06-10T00:32:50.039027+0000","last_clean":"2024-06-10T00:32:50.039027+0000","last_became_active":"2024-06-10T00:28:34.446377+0000","last_became_peered":"2024-06-10T00:28:34.446377+0000","last_unstale":"2024-06-10T00:32:50.039027+0000","last_undegraded":"2024-06-10T00:32:50.039027+0000","last_fullsized":"2024-06-10T00:32:50.039027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:16.914244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188829+0000","last_change":"2024-06-10T00:28:37.666159+0000","last_active":"2024-06-10T00:32:50.188829+0000","last_peered":"2024-06-10T00:32:50.188829+0000","last_clean":"2024-06-10T00:32:50.188829+0000","last_became_active":"2024-06-10T00:28:37.665711+0000","last_became_peered":"2024-06-10T00:28:37.665711+0000","last_unstale":"2024-06-10T00:32:50.188829+0000","last_undegraded":"2024-06-10T00:32:50.188829+0000","last_fullsized":"2024-06-10T00:32:50.188829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:03.982375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921636+0000","last_change":"2024-06-10T00:28:48.705071+0000","last_active":"2024-06-10T00:32:49.921636+0000","last_peered":"2024-06-10T00:32:49.921636+0000","last_clean":"2024-06-10T00:32:49.921636+0000","last_became_active":"2024-06-10T00:28:48.702211+0000","last_became_peered":"2024-06-10T00:28:48.702211+0000","last_unstale":"2024-06-10T00:32:49.921636+0000","last_undegraded":"2024-06-10T00:32:49.921636+0000","last_fullsized":"2024-06-10T00:32:49.921636+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:13.361985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":468,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.195703+0000","last_change":"2024-06-10T00:28:34.389784+0000","last_active":"2024-06-10T00:32:50.195703+0000","last_peered":"2024-06-10T00:32:50.195703+0000","last_clean":"2024-06-10T00:32:50.195703+0000","last_became_active":"2024-06-10T00:28:34.389416+0000","last_became_peered":"2024-06-10T00:28:34.389416+0000","last_unstale":"2024-06-10T00:32:50.195703+0000","last_undegraded":"2024-06-10T00:32:50.195703+0000","last_fullsized":"2024-06-10T00:32:50.195703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:23:05.803503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69365,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":381,"num_read_kb":253,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031957+0000","last_change":"2024-06-10T00:28:37.656174+0000","last_active":"2024-06-10T00:32:50.031957+0000","last_peered":"2024-06-10T00:32:50.031957+0000","last_clean":"2024-06-10T00:32:50.031957+0000","last_became_active":"2024-06-10T00:28:37.655794+0000","last_became_peered":"2024-06-10T00:28:37.655794+0000","last_unstale":"2024-06-10T00:32:50.031957+0000","last_undegraded":"2024-06-10T00:32:50.031957+0000","last_fullsized":"2024-06-10T00:32:50.031957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:22:00.310309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039312+0000","last_change":"2024-06-10T00:28:48.709915+0000","last_active":"2024-06-10T00:32:50.039312+0000","last_peered":"2024-06-10T00:32:50.039312+0000","last_clean":"2024-06-10T00:32:50.039312+0000","last_became_active":"2024-06-10T00:28:48.709426+0000","last_became_peered":"2024-06-10T00:28:48.709426+0000","last_unstale":"2024-06-10T00:32:50.039312+0000","last_undegraded":"2024-06-10T00:32:50.039312+0000","last_fullsized":"2024-06-10T00:32:50.039312+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:55.150617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921708+0000","last_change":"2024-06-10T00:28:34.389136+0000","last_active":"2024-06-10T00:32:49.921708+0000","last_peered":"2024-06-10T00:32:49.921708+0000","last_clean":"2024-06-10T00:32:49.921708+0000","last_became_active":"2024-06-10T00:28:34.388701+0000","last_became_peered":"2024-06-10T00:28:34.388701+0000","last_unstale":"2024-06-10T00:32:49.921708+0000","last_undegraded":"2024-06-10T00:32:49.921708+0000","last_fullsized":"2024-06-10T00:32:49.921708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:13:24.907824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133047+0000","last_change":"2024-06-10T00:28:37.659048+0000","last_active":"2024-06-10T00:32:50.133047+0000","last_peered":"2024-06-10T00:32:50.133047+0000","last_clean":"2024-06-10T00:32:50.133047+0000","last_became_active":"2024-06-10T00:28:37.658710+0000","last_became_peered":"2024-06-10T00:28:37.658710+0000","last_unstale":"2024-06-10T00:32:50.133047+0000","last_undegraded":"2024-06-10T00:32:50.133047+0000","last_fullsized":"2024-06-10T00:32:50.133047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:44.314559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198919+0000","last_change":"2024-06-10T00:28:48.709683+0000","last_active":"2024-06-10T00:32:50.198919+0000","last_peered":"2024-06-10T00:32:50.198919+0000","last_clean":"2024-06-10T00:32:50.198919+0000","last_became_active":"2024-06-10T00:28:48.709241+0000","last_became_peered":"2024-06-10T00:28:48.709241+0000","last_unstale":"2024-06-10T00:32:50.198919+0000","last_undegraded":"2024-06-10T00:32:50.198919+0000","last_fullsized":"2024-06-10T00:32:50.198919+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:25.190043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":24,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198772+0000","last_change":"2024-06-10T00:29:23.782029+0000","last_active":"2024-06-10T00:32:50.198772+0000","last_peered":"2024-06-10T00:32:50.198772+0000","last_clean":"2024-06-10T00:32:50.198772+0000","last_became_active":"2024-06-10T00:29:23.781565+0000","last_became_peered":"2024-06-10T00:29:23.781565+0000","last_unstale":"2024-06-10T00:32:50.198772+0000","last_undegraded":"2024-06-10T00:32:50.198772+0000","last_fullsized":"2024-06-10T00:32:50.198772+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:53.258760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134547+0000","last_change":"2024-06-10T00:28:37.653385+0000","last_active":"2024-06-10T00:32:50.134547+0000","last_peered":"2024-06-10T00:32:50.134547+0000","last_clean":"2024-06-10T00:32:50.134547+0000","last_became_active":"2024-06-10T00:28:37.653020+0000","last_became_peered":"2024-06-10T00:28:37.653020+0000","last_unstale":"2024-06-10T00:32:50.134547+0000","last_undegraded":"2024-06-10T00:32:50.134547+0000","last_fullsized":"2024-06-10T00:32:50.134547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:57:54.469544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921940+0000","last_change":"2024-06-10T00:28:48.707540+0000","last_active":"2024-06-10T00:32:49.921940+0000","last_peered":"2024-06-10T00:32:49.921940+0000","last_clean":"2024-06-10T00:32:49.921940+0000","last_became_active":"2024-06-10T00:28:48.707143+0000","last_became_peered":"2024-06-10T00:28:48.707143+0000","last_unstale":"2024-06-10T00:32:49.921940+0000","last_undegraded":"2024-06-10T00:32:49.921940+0000","last_fullsized":"2024-06-10T00:32:49.921940+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:20:29.344808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'8","reported_seq":244,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133792+0000","last_change":"2024-06-10T00:28:34.397496+0000","last_active":"2024-06-10T00:32:50.133792+0000","last_peered":"2024-06-10T00:32:50.133792+0000","last_clean":"2024-06-10T00:32:50.133792+0000","last_became_active":"2024-06-10T00:28:34.397124+0000","last_became_peered":"2024-06-10T00:28:34.397124+0000","last_unstale":"2024-06-10T00:32:50.133792+0000","last_undegraded":"2024-06-10T00:32:50.133792+0000","last_fullsized":"2024-06-10T00:32:50.133792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:41:48.609354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134324+0000","last_change":"2024-06-10T00:28:37.658338+0000","last_active":"2024-06-10T00:32:50.134324+0000","last_peered":"2024-06-10T00:32:50.134324+0000","last_clean":"2024-06-10T00:32:50.134324+0000","last_became_active":"2024-06-10T00:28:37.655910+0000","last_became_peered":"2024-06-10T00:28:37.655910+0000","last_unstale":"2024-06-10T00:32:50.134324+0000","last_undegraded":"2024-06-10T00:32:50.134324+0000","last_fullsized":"2024-06-10T00:32:50.134324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:38.501335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133960+0000","last_change":"2024-06-10T00:28:48.711151+0000","last_active":"2024-06-10T00:32:50.133960+0000","last_peered":"2024-06-10T00:32:50.133960+0000","last_clean":"2024-06-10T00:32:50.133960+0000","last_became_active":"2024-06-10T00:28:48.710768+0000","last_became_peered":"2024-06-10T00:28:48.710768+0000","last_unstale":"2024-06-10T00:32:50.133960+0000","last_undegraded":"2024-06-10T00:32:50.133960+0000","last_fullsized":"2024-06-10T00:32:50.133960+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:15:22.791839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":65,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031568+0000","last_change":"2024-06-10T00:28:34.445812+0000","last_active":"2024-06-10T00:32:50.031568+0000","last_peered":"2024-06-10T00:32:50.031568+0000","last_clean":"2024-06-10T00:32:50.031568+0000","last_became_active":"2024-06-10T00:28:34.445472+0000","last_became_peered":"2024-06-10T00:28:34.445472+0000","last_unstale":"2024-06-10T00:32:50.031568+0000","last_undegraded":"2024-06-10T00:32:50.031568+0000","last_fullsized":"2024-06-10T00:32:50.031568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:00.912050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038769+0000","last_change":"2024-06-10T00:28:37.655345+0000","last_active":"2024-06-10T00:32:50.038769+0000","last_peered":"2024-06-10T00:32:50.038769+0000","last_clean":"2024-06-10T00:32:50.038769+0000","last_became_active":"2024-06-10T00:28:37.654954+0000","last_became_peered":"2024-06-10T00:28:37.654954+0000","last_unstale":"2024-06-10T00:32:50.038769+0000","last_undegraded":"2024-06-10T00:32:50.038769+0000","last_fullsized":"2024-06-10T00:32:50.038769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:39:27.403006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038741+0000","last_change":"2024-06-10T00:28:48.705247+0000","last_active":"2024-06-10T00:32:50.038741+0000","last_peered":"2024-06-10T00:32:50.038741+0000","last_clean":"2024-06-10T00:32:50.038741+0000","last_became_active":"2024-06-10T00:28:48.704822+0000","last_became_peered":"2024-06-10T00:28:48.704822+0000","last_unstale":"2024-06-10T00:32:50.038741+0000","last_undegraded":"2024-06-10T00:32:50.038741+0000","last_fullsized":"2024-06-10T00:32:50.038741+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:24:59.991491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032099+0000","last_change":"2024-06-10T00:28:34.396792+0000","last_active":"2024-06-10T00:32:50.032099+0000","last_peered":"2024-06-10T00:32:50.032099+0000","last_clean":"2024-06-10T00:32:50.032099+0000","last_became_active":"2024-06-10T00:28:34.396442+0000","last_became_peered":"2024-06-10T00:28:34.396442+0000","last_unstale":"2024-06-10T00:32:50.032099+0000","last_undegraded":"2024-06-10T00:32:50.032099+0000","last_fullsized":"2024-06-10T00:32:50.032099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:46.436198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199145+0000","last_change":"2024-06-10T00:28:37.657698+0000","last_active":"2024-06-10T00:32:50.199145+0000","last_peered":"2024-06-10T00:32:50.199145+0000","last_clean":"2024-06-10T00:32:50.199145+0000","last_became_active":"2024-06-10T00:28:37.657364+0000","last_became_peered":"2024-06-10T00:28:37.657364+0000","last_unstale":"2024-06-10T00:32:50.199145+0000","last_undegraded":"2024-06-10T00:32:50.199145+0000","last_fullsized":"2024-06-10T00:32:50.199145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:23.097015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031580+0000","last_change":"2024-06-10T00:28:48.727567+0000","last_active":"2024-06-10T00:32:50.031580+0000","last_peered":"2024-06-10T00:32:50.031580+0000","last_clean":"2024-06-10T00:32:50.031580+0000","last_became_active":"2024-06-10T00:28:48.727108+0000","last_became_peered":"2024-06-10T00:28:48.727108+0000","last_unstale":"2024-06-10T00:32:50.031580+0000","last_undegraded":"2024-06-10T00:32:50.031580+0000","last_fullsized":"2024-06-10T00:32:50.031580+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:35:00.824177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921917+0000","last_change":"2024-06-10T00:28:34.446832+0000","last_active":"2024-06-10T00:32:49.921917+0000","last_peered":"2024-06-10T00:32:49.921917+0000","last_clean":"2024-06-10T00:32:49.921917+0000","last_became_active":"2024-06-10T00:28:34.446435+0000","last_became_peered":"2024-06-10T00:28:34.446435+0000","last_unstale":"2024-06-10T00:32:49.921917+0000","last_undegraded":"2024-06-10T00:32:49.921917+0000","last_fullsized":"2024-06-10T00:32:49.921917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:58.296674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042645+0000","last_change":"2024-06-10T00:30:24.229514+0000","last_active":"2024-06-10T00:32:50.042645+0000","last_peered":"2024-06-10T00:32:50.042645+0000","last_clean":"2024-06-10T00:32:50.042645+0000","last_became_active":"2024-06-10T00:30:24.229163+0000","last_became_peered":"2024-06-10T00:30:24.229163+0000","last_unstale":"2024-06-10T00:32:50.042645+0000","last_undegraded":"2024-06-10T00:32:50.042645+0000","last_fullsized":"2024-06-10T00:32:50.042645+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:52.914144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133741+0000","last_change":"2024-06-10T00:28:48.704683+0000","last_active":"2024-06-10T00:32:50.133741+0000","last_peered":"2024-06-10T00:32:50.133741+0000","last_clean":"2024-06-10T00:32:50.133741+0000","last_became_active":"2024-06-10T00:28:48.704297+0000","last_became_peered":"2024-06-10T00:28:48.704297+0000","last_unstale":"2024-06-10T00:32:50.133741+0000","last_undegraded":"2024-06-10T00:32:50.133741+0000","last_fullsized":"2024-06-10T00:32:50.133741+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:44.980544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'8","reported_seq":242,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188609+0000","last_change":"2024-06-10T00:28:34.393328+0000","last_active":"2024-06-10T00:32:50.188609+0000","last_peered":"2024-06-10T00:32:50.188609+0000","last_clean":"2024-06-10T00:32:50.188609+0000","last_became_active":"2024-06-10T00:28:34.391968+0000","last_became_peered":"2024-06-10T00:28:34.391968+0000","last_unstale":"2024-06-10T00:32:50.188609+0000","last_undegraded":"2024-06-10T00:32:50.188609+0000","last_fullsized":"2024-06-10T00:32:50.188609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:18.898180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1676,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":2,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042474+0000","last_change":"2024-06-10T00:28:37.652555+0000","last_active":"2024-06-10T00:32:50.042474+0000","last_peered":"2024-06-10T00:32:50.042474+0000","last_clean":"2024-06-10T00:32:50.042474+0000","last_became_active":"2024-06-10T00:28:37.651476+0000","last_became_peered":"2024-06-10T00:28:37.651476+0000","last_unstale":"2024-06-10T00:32:50.042474+0000","last_undegraded":"2024-06-10T00:32:50.042474+0000","last_fullsized":"2024-06-10T00:32:50.042474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:04:57.823224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134015+0000","last_change":"2024-06-10T00:28:48.727568+0000","last_active":"2024-06-10T00:32:50.134015+0000","last_peered":"2024-06-10T00:32:50.134015+0000","last_clean":"2024-06-10T00:32:50.134015+0000","last_became_active":"2024-06-10T00:28:48.727018+0000","last_became_peered":"2024-06-10T00:28:48.727018+0000","last_unstale":"2024-06-10T00:32:50.134015+0000","last_undegraded":"2024-06-10T00:32:50.134015+0000","last_fullsized":"2024-06-10T00:32:50.134015+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:02.025212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921381+0000","last_change":"2024-06-10T00:30:24.651947+0000","last_active":"2024-06-10T00:32:49.921381+0000","last_peered":"2024-06-10T00:32:49.921381+0000","last_clean":"2024-06-10T00:32:49.921381+0000","last_became_active":"2024-06-10T00:30:24.651640+0000","last_became_peered":"2024-06-10T00:30:24.651640+0000","last_unstale":"2024-06-10T00:32:49.921381+0000","last_undegraded":"2024-06-10T00:32:49.921381+0000","last_fullsized":"2024-06-10T00:32:49.921381+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:47.070568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"100'7","reported_seq":240,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.191611+0000","last_change":"2024-06-10T00:28:34.445501+0000","last_active":"2024-06-10T00:32:50.191611+0000","last_peered":"2024-06-10T00:32:50.191611+0000","last_clean":"2024-06-10T00:32:50.191611+0000","last_became_active":"2024-06-10T00:28:34.445128+0000","last_became_peered":"2024-06-10T00:28:34.445128+0000","last_unstale":"2024-06-10T00:32:50.191611+0000","last_undegraded":"2024-06-10T00:32:50.191611+0000","last_fullsized":"2024-06-10T00:32:50.191611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:59.231585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134703+0000","last_change":"2024-06-10T00:28:48.709852+0000","last_active":"2024-06-10T00:32:50.134703+0000","last_peered":"2024-06-10T00:32:50.134703+0000","last_clean":"2024-06-10T00:32:50.134703+0000","last_became_active":"2024-06-10T00:28:48.709423+0000","last_became_peered":"2024-06-10T00:28:48.709423+0000","last_unstale":"2024-06-10T00:32:50.134703+0000","last_undegraded":"2024-06-10T00:32:50.134703+0000","last_fullsized":"2024-06-10T00:32:50.134703+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:42.123563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031914+0000","last_change":"2024-06-10T00:28:37.657423+0000","last_active":"2024-06-10T00:32:50.031914+0000","last_peered":"2024-06-10T00:32:50.031914+0000","last_clean":"2024-06-10T00:32:50.031914+0000","last_became_active":"2024-06-10T00:28:37.657140+0000","last_became_peered":"2024-06-10T00:28:37.657140+0000","last_unstale":"2024-06-10T00:32:50.031914+0000","last_undegraded":"2024-06-10T00:32:50.031914+0000","last_fullsized":"2024-06-10T00:32:50.031914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:20:08.943066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922033+0000","last_change":"2024-06-10T00:28:34.392585+0000","last_active":"2024-06-10T00:32:49.922033+0000","last_peered":"2024-06-10T00:32:49.922033+0000","last_clean":"2024-06-10T00:32:49.922033+0000","last_became_active":"2024-06-10T00:28:34.392229+0000","last_became_peered":"2024-06-10T00:28:34.392229+0000","last_unstale":"2024-06-10T00:32:49.922033+0000","last_undegraded":"2024-06-10T00:32:49.922033+0000","last_fullsized":"2024-06-10T00:32:49.922033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:11.275761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":380,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134207+0000","last_change":"2024-06-10T00:27:12.072577+0000","last_active":"2024-06-10T00:32:50.134207+0000","last_peered":"2024-06-10T00:32:50.134207+0000","last_clean":"2024-06-10T00:32:50.134207+0000","last_became_active":"2024-06-10T00:27:12.072193+0000","last_became_peered":"2024-06-10T00:27:12.072193+0000","last_unstale":"2024-06-10T00:32:50.134207+0000","last_undegraded":"2024-06-10T00:32:50.134207+0000","last_fullsized":"2024-06-10T00:32:50.134207+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_clean_scrub_stamp":"2024-06-10T00:23:46.826438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:57.782124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039093+0000","last_change":"2024-06-10T00:28:48.727913+0000","last_active":"2024-06-10T00:32:50.039093+0000","last_peered":"2024-06-10T00:32:50.039093+0000","last_clean":"2024-06-10T00:32:50.039093+0000","last_became_active":"2024-06-10T00:28:48.727402+0000","last_became_peered":"2024-06-10T00:28:48.727402+0000","last_unstale":"2024-06-10T00:32:50.039093+0000","last_undegraded":"2024-06-10T00:32:50.039093+0000","last_fullsized":"2024-06-10T00:32:50.039093+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:10:25.051296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132861+0000","last_change":"2024-06-10T00:28:37.659901+0000","last_active":"2024-06-10T00:32:50.132861+0000","last_peered":"2024-06-10T00:32:50.132861+0000","last_clean":"2024-06-10T00:32:50.132861+0000","last_became_active":"2024-06-10T00:28:37.659553+0000","last_became_peered":"2024-06-10T00:28:37.659553+0000","last_unstale":"2024-06-10T00:32:50.132861+0000","last_undegraded":"2024-06-10T00:32:50.132861+0000","last_fullsized":"2024-06-10T00:32:50.132861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:25.694823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134306+0000","last_change":"2024-06-10T00:28:34.398155+0000","last_active":"2024-06-10T00:32:50.134306+0000","last_peered":"2024-06-10T00:32:50.134306+0000","last_clean":"2024-06-10T00:32:50.134306+0000","last_became_active":"2024-06-10T00:28:34.397793+0000","last_became_peered":"2024-06-10T00:28:34.397793+0000","last_unstale":"2024-06-10T00:32:50.134306+0000","last_undegraded":"2024-06-10T00:32:50.134306+0000","last_fullsized":"2024-06-10T00:32:50.134306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:21.188139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198902+0000","last_change":"2024-06-10T00:28:48.709736+0000","last_active":"2024-06-10T00:32:50.198902+0000","last_peered":"2024-06-10T00:32:50.198902+0000","last_clean":"2024-06-10T00:32:50.198902+0000","last_became_active":"2024-06-10T00:28:48.709293+0000","last_became_peered":"2024-06-10T00:28:48.709293+0000","last_unstale":"2024-06-10T00:32:50.198902+0000","last_undegraded":"2024-06-10T00:32:50.198902+0000","last_fullsized":"2024-06-10T00:32:50.198902+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:30:04.840656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199015+0000","last_change":"2024-06-10T00:29:23.780030+0000","last_active":"2024-06-10T00:32:50.199015+0000","last_peered":"2024-06-10T00:32:50.199015+0000","last_clean":"2024-06-10T00:32:50.199015+0000","last_became_active":"2024-06-10T00:29:23.779589+0000","last_became_peered":"2024-06-10T00:29:23.779589+0000","last_unstale":"2024-06-10T00:32:50.199015+0000","last_undegraded":"2024-06-10T00:32:50.199015+0000","last_fullsized":"2024-06-10T00:32:50.199015+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:53.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133881+0000","last_change":"2024-06-10T00:28:37.658330+0000","last_active":"2024-06-10T00:32:50.133881+0000","last_peered":"2024-06-10T00:32:50.133881+0000","last_clean":"2024-06-10T00:32:50.133881+0000","last_became_active":"2024-06-10T00:28:37.657343+0000","last_became_peered":"2024-06-10T00:28:37.657343+0000","last_unstale":"2024-06-10T00:32:50.133881+0000","last_undegraded":"2024-06-10T00:32:50.133881+0000","last_fullsized":"2024-06-10T00:32:50.133881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:31:52.167402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031759+0000","last_change":"2024-06-10T00:28:48.727506+0000","last_active":"2024-06-10T00:32:50.031759+0000","last_peered":"2024-06-10T00:32:50.031759+0000","last_clean":"2024-06-10T00:32:50.031759+0000","last_became_active":"2024-06-10T00:28:48.727068+0000","last_became_peered":"2024-06-10T00:28:48.727068+0000","last_unstale":"2024-06-10T00:32:50.031759+0000","last_undegraded":"2024-06-10T00:32:50.031759+0000","last_fullsized":"2024-06-10T00:32:50.031759+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:42.147908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'51","reported_seq":121,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.148195+0000","last_change":"2024-06-10T00:28:34.399872+0000","last_active":"2024-06-10T00:32:50.148195+0000","last_peered":"2024-06-10T00:32:50.148195+0000","last_clean":"2024-06-10T00:32:50.148195+0000","last_became_active":"2024-06-10T00:28:34.399493+0000","last_became_peered":"2024-06-10T00:28:34.399493+0000","last_unstale":"2024-06-10T00:32:50.148195+0000","last_undegraded":"2024-06-10T00:32:50.148195+0000","last_fullsized":"2024-06-10T00:32:50.148195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:24:10.018110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032414+0000","last_change":"2024-06-10T00:28:37.656597+0000","last_active":"2024-06-10T00:32:50.032414+0000","last_peered":"2024-06-10T00:32:50.032414+0000","last_clean":"2024-06-10T00:32:50.032414+0000","last_became_active":"2024-06-10T00:28:37.654356+0000","last_became_peered":"2024-06-10T00:28:37.654356+0000","last_unstale":"2024-06-10T00:32:50.032414+0000","last_undegraded":"2024-06-10T00:32:50.032414+0000","last_fullsized":"2024-06-10T00:32:50.032414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:07:12.950683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920721+0000","last_change":"2024-06-10T00:28:48.702686+0000","last_active":"2024-06-10T00:32:49.920721+0000","last_peered":"2024-06-10T00:32:49.920721+0000","last_clean":"2024-06-10T00:32:49.920721+0000","last_became_active":"2024-06-10T00:28:48.702273+0000","last_became_peered":"2024-06-10T00:28:48.702273+0000","last_unstale":"2024-06-10T00:32:49.920721+0000","last_undegraded":"2024-06-10T00:32:49.920721+0000","last_fullsized":"2024-06-10T00:32:49.920721+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:00.078232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921732+0000","last_change":"2024-06-10T00:28:34.447549+0000","last_active":"2024-06-10T00:32:49.921732+0000","last_peered":"2024-06-10T00:32:49.921732+0000","last_clean":"2024-06-10T00:32:49.921732+0000","last_became_active":"2024-06-10T00:28:34.447197+0000","last_became_peered":"2024-06-10T00:28:34.447197+0000","last_unstale":"2024-06-10T00:32:49.921732+0000","last_undegraded":"2024-06-10T00:32:49.921732+0000","last_fullsized":"2024-06-10T00:32:49.921732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:47.525153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193431+0000","last_change":"2024-06-10T00:28:37.655585+0000","last_active":"2024-06-10T00:32:50.193431+0000","last_peered":"2024-06-10T00:32:50.193431+0000","last_clean":"2024-06-10T00:32:50.193431+0000","last_became_active":"2024-06-10T00:28:37.655225+0000","last_became_peered":"2024-06-10T00:28:37.655225+0000","last_unstale":"2024-06-10T00:32:50.193431+0000","last_undegraded":"2024-06-10T00:32:50.193431+0000","last_fullsized":"2024-06-10T00:32:50.193431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:57:25.184924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921606+0000","last_change":"2024-06-10T00:29:23.857447+0000","last_active":"2024-06-10T00:32:49.921606+0000","last_peered":"2024-06-10T00:32:49.921606+0000","last_clean":"2024-06-10T00:32:49.921606+0000","last_became_active":"2024-06-10T00:29:23.856943+0000","last_became_peered":"2024-06-10T00:29:23.856943+0000","last_unstale":"2024-06-10T00:32:49.921606+0000","last_undegraded":"2024-06-10T00:32:49.921606+0000","last_fullsized":"2024-06-10T00:32:49.921606+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:58:44.548818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188309+0000","last_change":"2024-06-10T00:28:34.445637+0000","last_active":"2024-06-10T00:32:50.188309+0000","last_peered":"2024-06-10T00:32:50.188309+0000","last_clean":"2024-06-10T00:32:50.188309+0000","last_became_active":"2024-06-10T00:28:34.445324+0000","last_became_peered":"2024-06-10T00:28:34.445324+0000","last_unstale":"2024-06-10T00:32:50.188309+0000","last_undegraded":"2024-06-10T00:32:50.188309+0000","last_fullsized":"2024-06-10T00:32:50.188309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:05.713557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031570+0000","last_change":"2024-06-10T00:28:37.659037+0000","last_active":"2024-06-10T00:32:50.031570+0000","last_peered":"2024-06-10T00:32:50.031570+0000","last_clean":"2024-06-10T00:32:50.031570+0000","last_became_active":"2024-06-10T00:28:37.658705+0000","last_became_peered":"2024-06-10T00:28:37.658705+0000","last_unstale":"2024-06-10T00:32:50.031570+0000","last_undegraded":"2024-06-10T00:32:50.031570+0000","last_fullsized":"2024-06-10T00:32:50.031570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:20:52.797237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922067+0000","last_change":"2024-06-10T00:28:48.704130+0000","last_active":"2024-06-10T00:32:49.922067+0000","last_peered":"2024-06-10T00:32:49.922067+0000","last_clean":"2024-06-10T00:32:49.922067+0000","last_became_active":"2024-06-10T00:28:48.703738+0000","last_became_peered":"2024-06-10T00:28:48.703738+0000","last_unstale":"2024-06-10T00:32:49.922067+0000","last_undegraded":"2024-06-10T00:32:49.922067+0000","last_fullsized":"2024-06-10T00:32:49.922067+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:01:40.436710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032028+0000","last_change":"2024-06-10T00:28:34.386667+0000","last_active":"2024-06-10T00:32:50.032028+0000","last_peered":"2024-06-10T00:32:50.032028+0000","last_clean":"2024-06-10T00:32:50.032028+0000","last_became_active":"2024-06-10T00:28:34.386363+0000","last_became_peered":"2024-06-10T00:28:34.386363+0000","last_unstale":"2024-06-10T00:32:50.032028+0000","last_undegraded":"2024-06-10T00:32:50.032028+0000","last_fullsized":"2024-06-10T00:32:50.032028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:11.850162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198745+0000","last_change":"2024-06-10T00:28:37.664754+0000","last_active":"2024-06-10T00:32:50.198745+0000","last_peered":"2024-06-10T00:32:50.198745+0000","last_clean":"2024-06-10T00:32:50.198745+0000","last_became_active":"2024-06-10T00:28:37.658574+0000","last_became_peered":"2024-06-10T00:28:37.658574+0000","last_unstale":"2024-06-10T00:32:50.198745+0000","last_undegraded":"2024-06-10T00:32:50.198745+0000","last_fullsized":"2024-06-10T00:32:50.198745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:16:47.437822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921563+0000","last_change":"2024-06-10T00:28:48.706560+0000","last_active":"2024-06-10T00:32:49.921563+0000","last_peered":"2024-06-10T00:32:49.921563+0000","last_clean":"2024-06-10T00:32:49.921563+0000","last_became_active":"2024-06-10T00:28:48.705986+0000","last_became_peered":"2024-06-10T00:28:48.705986+0000","last_unstale":"2024-06-10T00:32:49.921563+0000","last_undegraded":"2024-06-10T00:32:49.921563+0000","last_fullsized":"2024-06-10T00:32:49.921563+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:21:26.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921015+0000","last_change":"2024-06-10T00:28:34.394509+0000","last_active":"2024-06-10T00:32:49.921015+0000","last_peered":"2024-06-10T00:32:49.921015+0000","last_clean":"2024-06-10T00:32:49.921015+0000","last_became_active":"2024-06-10T00:28:34.394105+0000","last_became_peered":"2024-06-10T00:28:34.394105+0000","last_unstale":"2024-06-10T00:32:49.921015+0000","last_undegraded":"2024-06-10T00:32:49.921015+0000","last_fullsized":"2024-06-10T00:32:49.921015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:13.415544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032271+0000","last_change":"2024-06-10T00:28:37.653192+0000","last_active":"2024-06-10T00:32:50.032271+0000","last_peered":"2024-06-10T00:32:50.032271+0000","last_clean":"2024-06-10T00:32:50.032271+0000","last_became_active":"2024-06-10T00:28:37.652868+0000","last_became_peered":"2024-06-10T00:28:37.652868+0000","last_unstale":"2024-06-10T00:32:50.032271+0000","last_undegraded":"2024-06-10T00:32:50.032271+0000","last_fullsized":"2024-06-10T00:32:50.032271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:21.614915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921695+0000","last_change":"2024-06-10T00:28:48.727615+0000","last_active":"2024-06-10T00:32:49.921695+0000","last_peered":"2024-06-10T00:32:49.921695+0000","last_clean":"2024-06-10T00:32:49.921695+0000","last_became_active":"2024-06-10T00:28:48.727190+0000","last_became_peered":"2024-06-10T00:28:48.727190+0000","last_unstale":"2024-06-10T00:32:49.921695+0000","last_undegraded":"2024-06-10T00:32:49.921695+0000","last_fullsized":"2024-06-10T00:32:49.921695+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:57.498183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031723+0000","last_change":"2024-06-10T00:29:23.784131+0000","last_active":"2024-06-10T00:32:50.031723+0000","last_peered":"2024-06-10T00:32:50.031723+0000","last_clean":"2024-06-10T00:32:50.031723+0000","last_became_active":"2024-06-10T00:29:23.780909+0000","last_became_peered":"2024-06-10T00:29:23.780909+0000","last_unstale":"2024-06-10T00:32:50.031723+0000","last_undegraded":"2024-06-10T00:32:50.031723+0000","last_fullsized":"2024-06-10T00:32:50.031723+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.774217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134422+0000","last_change":"2024-06-10T00:28:37.660399+0000","last_active":"2024-06-10T00:32:50.134422+0000","last_peered":"2024-06-10T00:32:50.134422+0000","last_clean":"2024-06-10T00:32:50.134422+0000","last_became_active":"2024-06-10T00:28:37.660069+0000","last_became_peered":"2024-06-10T00:28:37.660069+0000","last_unstale":"2024-06-10T00:32:50.134422+0000","last_undegraded":"2024-06-10T00:32:50.134422+0000","last_fullsized":"2024-06-10T00:32:50.134422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:59.727883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188428+0000","last_change":"2024-06-10T00:28:48.713267+0000","last_active":"2024-06-10T00:32:50.188428+0000","last_peered":"2024-06-10T00:32:50.188428+0000","last_clean":"2024-06-10T00:32:50.188428+0000","last_became_active":"2024-06-10T00:28:48.712870+0000","last_became_peered":"2024-06-10T00:28:48.712870+0000","last_unstale":"2024-06-10T00:32:50.188428+0000","last_undegraded":"2024-06-10T00:32:50.188428+0000","last_fullsized":"2024-06-10T00:32:50.188428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:59.210101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'6","reported_seq":239,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.161235+0000","last_change":"2024-06-10T00:28:34.448652+0000","last_active":"2024-06-10T00:32:50.161235+0000","last_peered":"2024-06-10T00:32:50.161235+0000","last_clean":"2024-06-10T00:32:50.161235+0000","last_became_active":"2024-06-10T00:28:34.448243+0000","last_became_peered":"2024-06-10T00:28:34.448243+0000","last_unstale":"2024-06-10T00:32:50.161235+0000","last_undegraded":"2024-06-10T00:32:50.161235+0000","last_fullsized":"2024-06-10T00:32:50.161235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:51:40.003429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":187,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.135023+0000","last_change":"2024-06-10T00:28:37.662307+0000","last_active":"2024-06-10T00:32:50.135023+0000","last_peered":"2024-06-10T00:32:50.135023+0000","last_clean":"2024-06-10T00:32:50.135023+0000","last_became_active":"2024-06-10T00:28:37.661727+0000","last_became_peered":"2024-06-10T00:28:37.661727+0000","last_unstale":"2024-06-10T00:32:50.135023+0000","last_undegraded":"2024-06-10T00:32:50.135023+0000","last_fullsized":"2024-06-10T00:32:50.135023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:23.989429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192376+0000","last_change":"2024-06-10T00:29:23.857626+0000","last_active":"2024-06-10T00:32:50.192376+0000","last_peered":"2024-06-10T00:32:50.192376+0000","last_clean":"2024-06-10T00:32:50.192376+0000","last_became_active":"2024-06-10T00:29:23.857057+0000","last_became_peered":"2024-06-10T00:29:23.857057+0000","last_unstale":"2024-06-10T00:32:50.192376+0000","last_undegraded":"2024-06-10T00:32:50.192376+0000","last_fullsized":"2024-06-10T00:32:50.192376+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:25:10.340053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'41","reported_seq":95,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.145625+0000","last_change":"2024-06-10T00:28:34.399253+0000","last_active":"2024-06-10T00:32:50.145625+0000","last_peered":"2024-06-10T00:32:50.145625+0000","last_clean":"2024-06-10T00:32:50.145625+0000","last_became_active":"2024-06-10T00:28:34.397763+0000","last_became_peered":"2024-06-10T00:28:34.397763+0000","last_unstale":"2024-06-10T00:32:50.145625+0000","last_undegraded":"2024-06-10T00:32:50.145625+0000","last_fullsized":"2024-06-10T00:32:50.145625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:05:57.371625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134695+0000","last_change":"2024-06-10T00:28:37.658027+0000","last_active":"2024-06-10T00:32:50.134695+0000","last_peered":"2024-06-10T00:32:50.134695+0000","last_clean":"2024-06-10T00:32:50.134695+0000","last_became_active":"2024-06-10T00:28:37.655513+0000","last_became_peered":"2024-06-10T00:28:37.655513+0000","last_unstale":"2024-06-10T00:32:50.134695+0000","last_undegraded":"2024-06-10T00:32:50.134695+0000","last_fullsized":"2024-06-10T00:32:50.134695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:50:23.525855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038558+0000","last_change":"2024-06-10T00:28:48.706997+0000","last_active":"2024-06-10T00:32:50.038558+0000","last_peered":"2024-06-10T00:32:50.038558+0000","last_clean":"2024-06-10T00:32:50.038558+0000","last_became_active":"2024-06-10T00:28:48.706597+0000","last_became_peered":"2024-06-10T00:28:48.706597+0000","last_unstale":"2024-06-10T00:32:50.038558+0000","last_undegraded":"2024-06-10T00:32:50.038558+0000","last_fullsized":"2024-06-10T00:32:50.038558+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:16.385312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133886+0000","last_change":"2024-06-10T00:29:23.785774+0000","last_active":"2024-06-10T00:32:50.133886+0000","last_peered":"2024-06-10T00:32:50.133886+0000","last_clean":"2024-06-10T00:32:50.133886+0000","last_became_active":"2024-06-10T00:29:23.781768+0000","last_became_peered":"2024-06-10T00:29:23.781768+0000","last_unstale":"2024-06-10T00:32:50.133886+0000","last_undegraded":"2024-06-10T00:32:50.133886+0000","last_fullsized":"2024-06-10T00:32:50.133886+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:27.348264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134953+0000","last_change":"2024-06-10T00:28:37.657820+0000","last_active":"2024-06-10T00:32:50.134953+0000","last_peered":"2024-06-10T00:32:50.134953+0000","last_clean":"2024-06-10T00:32:50.134953+0000","last_became_active":"2024-06-10T00:28:37.657487+0000","last_became_peered":"2024-06-10T00:28:37.657487+0000","last_unstale":"2024-06-10T00:32:50.134953+0000","last_undegraded":"2024-06-10T00:32:50.134953+0000","last_fullsized":"2024-06-10T00:32:50.134953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:44:35.930401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198953+0000","last_change":"2024-06-10T00:28:37.664543+0000","last_active":"2024-06-10T00:32:50.198953+0000","last_peered":"2024-06-10T00:32:50.198953+0000","last_clean":"2024-06-10T00:32:50.198953+0000","last_became_active":"2024-06-10T00:28:37.659015+0000","last_became_peered":"2024-06-10T00:28:37.659015+0000","last_unstale":"2024-06-10T00:32:50.198953+0000","last_undegraded":"2024-06-10T00:32:50.198953+0000","last_fullsized":"2024-06-10T00:32:50.198953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:06.555472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921072+0000","last_change":"2024-06-10T00:28:37.659389+0000","last_active":"2024-06-10T00:32:49.921072+0000","last_peered":"2024-06-10T00:32:49.921072+0000","last_clean":"2024-06-10T00:32:49.921072+0000","last_became_active":"2024-06-10T00:28:37.658990+0000","last_became_peered":"2024-06-10T00:28:37.658990+0000","last_unstale":"2024-06-10T00:32:49.921072+0000","last_undegraded":"2024-06-10T00:32:49.921072+0000","last_fullsized":"2024-06-10T00:32:49.921072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:29.532779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920823+0000","last_change":"2024-06-10T00:28:37.655621+0000","last_active":"2024-06-10T00:32:49.920823+0000","last_peered":"2024-06-10T00:32:49.920823+0000","last_clean":"2024-06-10T00:32:49.920823+0000","last_became_active":"2024-06-10T00:28:37.653599+0000","last_became_peered":"2024-06-10T00:28:37.653599+0000","last_unstale":"2024-06-10T00:32:49.920823+0000","last_undegraded":"2024-06-10T00:32:49.920823+0000","last_fullsized":"2024-06-10T00:32:49.920823+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:22.165851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192846+0000","last_change":"2024-06-10T00:29:23.782727+0000","last_active":"2024-06-10T00:32:50.192846+0000","last_peered":"2024-06-10T00:32:50.192846+0000","last_clean":"2024-06-10T00:32:50.192846+0000","last_became_active":"2024-06-10T00:29:23.782309+0000","last_became_peered":"2024-06-10T00:29:23.782309+0000","last_unstale":"2024-06-10T00:32:50.192846+0000","last_undegraded":"2024-06-10T00:32:50.192846+0000","last_fullsized":"2024-06-10T00:32:50.192846+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:20.585955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133856+0000","last_change":"2024-06-10T00:28:37.646750+0000","last_active":"2024-06-10T00:32:50.133856+0000","last_peered":"2024-06-10T00:32:50.133856+0000","last_clean":"2024-06-10T00:32:50.133856+0000","last_became_active":"2024-06-10T00:28:37.646372+0000","last_became_peered":"2024-06-10T00:28:37.646372+0000","last_unstale":"2024-06-10T00:32:50.133856+0000","last_undegraded":"2024-06-10T00:32:50.133856+0000","last_fullsized":"2024-06-10T00:32:50.133856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:45.762956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199287+0000","last_change":"2024-06-10T00:28:37.662278+0000","last_active":"2024-06-10T00:32:50.199287+0000","last_peered":"2024-06-10T00:32:50.199287+0000","last_clean":"2024-06-10T00:32:50.199287+0000","last_became_active":"2024-06-10T00:28:37.661855+0000","last_became_peered":"2024-06-10T00:28:37.661855+0000","last_unstale":"2024-06-10T00:32:50.199287+0000","last_undegraded":"2024-06-10T00:32:50.199287+0000","last_fullsized":"2024-06-10T00:32:50.199287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:20.074208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921288+0000","last_change":"2024-06-10T00:28:37.656738+0000","last_active":"2024-06-10T00:32:49.921288+0000","last_peered":"2024-06-10T00:32:49.921288+0000","last_clean":"2024-06-10T00:32:49.921288+0000","last_became_active":"2024-06-10T00:28:37.656375+0000","last_became_peered":"2024-06-10T00:28:37.656375+0000","last_unstale":"2024-06-10T00:32:49.921288+0000","last_undegraded":"2024-06-10T00:32:49.921288+0000","last_fullsized":"2024-06-10T00:32:49.921288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:49:17.779670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032555+0000","last_change":"2024-06-10T00:28:37.656849+0000","last_active":"2024-06-10T00:32:50.032555+0000","last_peered":"2024-06-10T00:32:50.032555+0000","last_clean":"2024-06-10T00:32:50.032555+0000","last_became_active":"2024-06-10T00:28:37.656261+0000","last_became_peered":"2024-06-10T00:28:37.656261+0000","last_unstale":"2024-06-10T00:32:50.032555+0000","last_undegraded":"2024-06-10T00:32:50.032555+0000","last_fullsized":"2024-06-10T00:32:50.032555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:13.152165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038730+0000","last_change":"2024-06-10T00:28:37.648384+0000","last_active":"2024-06-10T00:32:50.038730+0000","last_peered":"2024-06-10T00:32:50.038730+0000","last_clean":"2024-06-10T00:32:50.038730+0000","last_became_active":"2024-06-10T00:28:37.648009+0000","last_became_peered":"2024-06-10T00:28:37.648009+0000","last_unstale":"2024-06-10T00:32:50.038730+0000","last_undegraded":"2024-06-10T00:32:50.038730+0000","last_fullsized":"2024-06-10T00:32:50.038730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:45:55.808932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032174+0000","last_change":"2024-06-10T00:28:37.655828+0000","last_active":"2024-06-10T00:32:50.032174+0000","last_peered":"2024-06-10T00:32:50.032174+0000","last_clean":"2024-06-10T00:32:50.032174+0000","last_became_active":"2024-06-10T00:28:37.655269+0000","last_became_peered":"2024-06-10T00:28:37.655269+0000","last_unstale":"2024-06-10T00:32:50.032174+0000","last_undegraded":"2024-06-10T00:32:50.032174+0000","last_fullsized":"2024-06-10T00:32:50.032174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:56.618469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188607+0000","last_change":"2024-06-10T00:28:37.663756+0000","last_active":"2024-06-10T00:32:50.188607+0000","last_peered":"2024-06-10T00:32:50.188607+0000","last_clean":"2024-06-10T00:32:50.188607+0000","last_became_active":"2024-06-10T00:28:37.663380+0000","last_became_peered":"2024-06-10T00:28:37.663380+0000","last_unstale":"2024-06-10T00:32:50.188607+0000","last_undegraded":"2024-06-10T00:32:50.188607+0000","last_fullsized":"2024-06-10T00:32:50.188607+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:45.534648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133127+0000","last_change":"2024-06-10T00:28:37.655246+0000","last_active":"2024-06-10T00:32:50.133127+0000","last_peered":"2024-06-10T00:32:50.133127+0000","last_clean":"2024-06-10T00:32:50.133127+0000","last_became_active":"2024-06-10T00:28:37.654605+0000","last_became_peered":"2024-06-10T00:28:37.654605+0000","last_unstale":"2024-06-10T00:32:50.133127+0000","last_undegraded":"2024-06-10T00:32:50.133127+0000","last_fullsized":"2024-06-10T00:32:50.133127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:15.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199118+0000","last_change":"2024-06-10T00:28:37.662485+0000","last_active":"2024-06-10T00:32:50.199118+0000","last_peered":"2024-06-10T00:32:50.199118+0000","last_clean":"2024-06-10T00:32:50.199118+0000","last_became_active":"2024-06-10T00:28:37.661900+0000","last_became_peered":"2024-06-10T00:28:37.661900+0000","last_unstale":"2024-06-10T00:32:50.199118+0000","last_undegraded":"2024-06-10T00:32:50.199118+0000","last_fullsized":"2024-06-10T00:32:50.199118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:30:03.210634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604313,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1988,"num_read_kb":1107,"num_write":386,"num_write_kb":1478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1812939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":311,"ondisk_log_size":311,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383746,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1175552,"data_stored":581073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645382,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28836,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716316,"statfs":{"total":95995035648,"available":95965507584,"internally_reserved":0,"allocated":1859584,"data_stored":1231040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808907,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1585152,"data_stored":977723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070546,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36864,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708288,"statfs":{"total":95995035648,"available":95957286912,"internally_reserved":0,"allocated":1560576,"data_stored":973221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332182,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1781760,"data_stored":1180215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495707,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":577003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36532,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708620,"statfs":{"total":95995035648,"available":95957626880,"internally_reserved":0,"allocated":1224704,"data_stored":631213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36500,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708652,"statfs":{"total":95995035648,"available":95957659648,"internally_reserved":0,"allocated":1187840,"data_stored":577761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36564,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708588,"statfs":{"total":95995035648,"available":95957594112,"internally_reserved":0,"allocated":1253376,"data_stored":640908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182506,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1703936,"data_stored":1100439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920869,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36484,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708668,"statfs":{"total":95995035648,"available":95957676032,"internally_reserved":0,"allocated":1171456,"data_stored":580910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757344,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28892,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716260,"statfs":{"total":95995035648,"available":95965450240,"internally_reserved":0,"allocated":1921024,"data_stored":1305298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":135641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":203622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:32:53.462 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-10T00:32:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.605+0000 7f14666b3640 1 -- 172.21.15.62:0/4055029413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1460100c10 msgr2=0x7f1460101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.605+0000 7f14666b3640 1 --2- 172.21.15.62:0/4055029413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1460100c10 0x7f1460101070 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1448009900 tx=0x7f144802a410 comp rx=0 tx=0).stop 2024-06-10T00:32:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.605+0000 7f14666b3640 1 -- 172.21.15.62:0/4055029413 shutdown_connections 2024-06-10T00:32:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.605+0000 7f14666b3640 1 --2- 172.21.15.62:0/4055029413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14601015b0 0x7f1460101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.605+0000 7f14666b3640 1 --2- 172.21.15.62:0/4055029413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1460100c10 0x7f1460101070 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.605+0000 7f14666b3640 1 --2- 172.21.15.62:0/4055029413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14601002b0 0x7f14601006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.605+0000 7f14666b3640 1 -- 172.21.15.62:0/4055029413 >> 172.21.15.62:0/4055029413 conn(0x7f14600fbe50 msgr2=0x7f14600fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.605+0000 7f14666b3640 1 -- 172.21.15.62:0/4055029413 shutdown_connections 2024-06-10T00:32:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.605+0000 7f14666b3640 1 -- 172.21.15.62:0/4055029413 wait complete. 2024-06-10T00:32:53.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.605+0000 7f14666b3640 1 Processor -- start 2024-06-10T00:32:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.605+0000 7f14666b3640 1 -- start start 2024-06-10T00:32:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f14666b3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14601002b0 0x7f1460199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f14666b3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1460100c10 0x7f146019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f14666b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14601015b0 0x7f14601a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f14666b3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f146010f870 con 0x7f14601002b0 2024-06-10T00:32:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f14666b3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f146010f6f0 con 0x7f14601015b0 2024-06-10T00:32:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f14666b3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f146010f9f0 con 0x7f1460100c10 2024-06-10T00:32:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f145ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14601002b0 0x7f1460199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f145ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14601002b0 0x7f1460199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45394/0 (socket says 172.21.15.62:45394) 2024-06-10T00:32:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f145ffff640 1 -- 172.21.15.62:0/209981318 learned_addr learned my addr 172.21.15.62:0/209981318 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f1457fff640 1 --2- 172.21.15.62:0/209981318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1460100c10 0x7f146019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:53.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f1464c50640 1 --2- 172.21.15.62:0/209981318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14601015b0 0x7f14601a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:53.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f145ffff640 1 -- 172.21.15.62:0/209981318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1460100c10 msgr2=0x7f146019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:53.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f145ffff640 1 --2- 172.21.15.62:0/209981318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1460100c10 0x7f146019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f145ffff640 1 -- 172.21.15.62:0/209981318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14601015b0 msgr2=0x7f14601a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:53.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f145ffff640 1 --2- 172.21.15.62:0/209981318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14601015b0 0x7f14601a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f145ffff640 1 -- 172.21.15.62:0/209981318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14601a1b00 con 0x7f14601002b0 2024-06-10T00:32:53.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f145ffff640 1 --2- 172.21.15.62:0/209981318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14601002b0 0x7f1460199ee0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f145000ea60 tx=0x7f145000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:53.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f145dffb640 1 -- 172.21.15.62:0/209981318 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145000cde0 con 0x7f14601002b0 2024-06-10T00:32:53.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f14666b3640 1 -- 172.21.15.62:0/209981318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14601a1df0 con 0x7f14601002b0 2024-06-10T00:32:53.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f145dffb640 1 -- 172.21.15.62:0/209981318 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14500035f0 con 0x7f14601002b0 2024-06-10T00:32:53.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f145dffb640 1 -- 172.21.15.62:0/209981318 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1450010670 con 0x7f14601002b0 2024-06-10T00:32:53.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.606+0000 7f14666b3640 1 -- 172.21.15.62:0/209981318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14601a2260 con 0x7f14601002b0 2024-06-10T00:32:53.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.608+0000 7f14666b3640 1 -- 172.21.15.62:0/209981318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f146010b190 con 0x7f14601002b0 2024-06-10T00:32:53.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.609+0000 7f145dffb640 1 -- 172.21.15.62:0/209981318 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 137953+0+0 (secure 0 0 0) 0x7f14500037a0 con 0x7f14601002b0 2024-06-10T00:32:53.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.610+0000 7f145dffb640 1 --2- 172.21.15.62:0/209981318 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f144c0a2d00 0x7f144c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:53.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.610+0000 7f1457fff640 1 --2- 172.21.15.62:0/209981318 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f144c0a2d00 0x7f144c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:53.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.610+0000 7f145dffb640 1 -- 172.21.15.62:0/209981318 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f14500cfb80 con 0x7f14601002b0 2024-06-10T00:32:53.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.611+0000 7f1457fff640 1 --2- 172.21.15.62:0/209981318 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f144c0a2d00 0x7f144c0a51c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1448031440 tx=0x7f14480315b0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:53.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.614+0000 7f145dffb640 1 -- 172.21.15.62:0/209981318 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f145009c3f0 con 0x7f14601002b0 2024-06-10T00:32:53.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.801+0000 7f14666b3640 1 -- 172.21.15.62:0/209981318 --> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1460061de0 con 0x7f144c0a2d00 2024-06-10T00:32:53.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.810+0000 7f145dffb640 1 -- 172.21.15.62:0/209981318 <== mgr.15333 v2:172.21.15.62:6800/518034449 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339578 (secure 0 0 0) 0x7f1460061de0 con 0x7f144c0a2d00 2024-06-10T00:32:53.814 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:32:53.816 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-06-10T00:32:53.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.814+0000 7f14666b3640 1 -- 172.21.15.62:0/209981318 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f144c0a2d00 msgr2=0x7f144c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:53.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.814+0000 7f14666b3640 1 --2- 172.21.15.62:0/209981318 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f144c0a2d00 0x7f144c0a51c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1448031440 tx=0x7f14480315b0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.814+0000 7f14666b3640 1 -- 172.21.15.62:0/209981318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14601002b0 msgr2=0x7f1460199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:53.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.814+0000 7f14666b3640 1 --2- 172.21.15.62:0/209981318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14601002b0 0x7f1460199ee0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f145000ea60 tx=0x7f145000ef20 comp rx=0 tx=0).stop 2024-06-10T00:32:53.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.814+0000 7f14666b3640 1 -- 172.21.15.62:0/209981318 shutdown_connections 2024-06-10T00:32:53.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.814+0000 7f14666b3640 1 --2- 172.21.15.62:0/209981318 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f144c0a2d00 0x7f144c0a51c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.814+0000 7f14666b3640 1 --2- 172.21.15.62:0/209981318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14601015b0 0x7f14601a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.814+0000 7f14666b3640 1 --2- 172.21.15.62:0/209981318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1460100c10 0x7f146019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.814+0000 7f14666b3640 1 --2- 172.21.15.62:0/209981318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14601002b0 0x7f1460199ee0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.814+0000 7f14666b3640 1 -- 172.21.15.62:0/209981318 >> 172.21.15.62:0/209981318 conn(0x7f14600fbe50 msgr2=0x7f14600fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:53.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.815+0000 7f14666b3640 1 -- 172.21.15.62:0/209981318 shutdown_connections 2024-06-10T00:32:53.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.815+0000 7f14666b3640 1 -- 172.21.15.62:0/209981318 wait complete. 2024-06-10T00:32:53.826 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-06-10T00:32:51.934091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194915,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2124,"num_read_kb":1226,"num_write":569,"num_write_kb":4418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":950,"ondisk_log_size":950,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387164,"kb_used_data":17176,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124554660,"statfs":{"total":1151940427776,"available":1151543971840,"internally_reserved":0,"allocated":17588224,"data_stored":10356804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":378713403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.993748"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199389+0000","last_change":"2024-06-10T00:28:37.657234+0000","last_active":"2024-06-10T00:32:50.199389+0000","last_peered":"2024-06-10T00:32:50.199389+0000","last_clean":"2024-06-10T00:32:50.199389+0000","last_became_active":"2024-06-10T00:28:37.656879+0000","last_became_peered":"2024-06-10T00:28:37.656879+0000","last_unstale":"2024-06-10T00:32:50.199389+0000","last_undegraded":"2024-06-10T00:32:50.199389+0000","last_fullsized":"2024-06-10T00:32:50.199389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:01:02.907597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193442+0000","last_change":"2024-06-10T00:28:37.656440+0000","last_active":"2024-06-10T00:32:50.193442+0000","last_peered":"2024-06-10T00:32:50.193442+0000","last_clean":"2024-06-10T00:32:50.193442+0000","last_became_active":"2024-06-10T00:28:37.656113+0000","last_became_peered":"2024-06-10T00:28:37.656113+0000","last_unstale":"2024-06-10T00:32:50.193442+0000","last_undegraded":"2024-06-10T00:32:50.193442+0000","last_fullsized":"2024-06-10T00:32:50.193442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:49:31.447951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193070+0000","last_change":"2024-06-10T00:28:37.659114+0000","last_active":"2024-06-10T00:32:50.193070+0000","last_peered":"2024-06-10T00:32:50.193070+0000","last_clean":"2024-06-10T00:32:50.193070+0000","last_became_active":"2024-06-10T00:28:37.658763+0000","last_became_peered":"2024-06-10T00:28:37.658763+0000","last_unstale":"2024-06-10T00:32:50.193070+0000","last_undegraded":"2024-06-10T00:32:50.193070+0000","last_fullsized":"2024-06-10T00:32:50.193070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:39:54.963590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921896+0000","last_change":"2024-06-10T00:28:37.656786+0000","last_active":"2024-06-10T00:32:49.921896+0000","last_peered":"2024-06-10T00:32:49.921896+0000","last_clean":"2024-06-10T00:32:49.921896+0000","last_became_active":"2024-06-10T00:28:37.656073+0000","last_became_peered":"2024-06-10T00:28:37.656073+0000","last_unstale":"2024-06-10T00:32:49.921896+0000","last_undegraded":"2024-06-10T00:32:49.921896+0000","last_fullsized":"2024-06-10T00:32:49.921896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:19.176608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031746+0000","last_change":"2024-06-10T00:28:37.650250+0000","last_active":"2024-06-10T00:32:50.031746+0000","last_peered":"2024-06-10T00:32:50.031746+0000","last_clean":"2024-06-10T00:32:50.031746+0000","last_became_active":"2024-06-10T00:28:37.649852+0000","last_became_peered":"2024-06-10T00:28:37.649852+0000","last_unstale":"2024-06-10T00:32:50.031746+0000","last_undegraded":"2024-06-10T00:32:50.031746+0000","last_fullsized":"2024-06-10T00:32:50.031746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:39.492884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042392+0000","last_change":"2024-06-10T00:28:37.646042+0000","last_active":"2024-06-10T00:32:50.042392+0000","last_peered":"2024-06-10T00:32:50.042392+0000","last_clean":"2024-06-10T00:32:50.042392+0000","last_became_active":"2024-06-10T00:28:37.644591+0000","last_became_peered":"2024-06-10T00:28:37.644591+0000","last_unstale":"2024-06-10T00:32:50.042392+0000","last_undegraded":"2024-06-10T00:32:50.042392+0000","last_fullsized":"2024-06-10T00:32:50.042392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:29.932303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134284+0000","last_change":"2024-06-10T00:28:37.664076+0000","last_active":"2024-06-10T00:32:50.134284+0000","last_peered":"2024-06-10T00:32:50.134284+0000","last_clean":"2024-06-10T00:32:50.134284+0000","last_became_active":"2024-06-10T00:28:37.663741+0000","last_became_peered":"2024-06-10T00:28:37.663741+0000","last_unstale":"2024-06-10T00:32:50.134284+0000","last_undegraded":"2024-06-10T00:32:50.134284+0000","last_fullsized":"2024-06-10T00:32:50.134284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:14.581409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032039+0000","last_change":"2024-06-10T00:28:37.648758+0000","last_active":"2024-06-10T00:32:50.032039+0000","last_peered":"2024-06-10T00:32:50.032039+0000","last_clean":"2024-06-10T00:32:50.032039+0000","last_became_active":"2024-06-10T00:28:37.648387+0000","last_became_peered":"2024-06-10T00:28:37.648387+0000","last_unstale":"2024-06-10T00:32:50.032039+0000","last_undegraded":"2024-06-10T00:32:50.032039+0000","last_fullsized":"2024-06-10T00:32:50.032039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:17.187265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.189039+0000","last_change":"2024-06-10T00:28:37.664650+0000","last_active":"2024-06-10T00:32:50.189039+0000","last_peered":"2024-06-10T00:32:50.189039+0000","last_clean":"2024-06-10T00:32:50.189039+0000","last_became_active":"2024-06-10T00:28:37.663178+0000","last_became_peered":"2024-06-10T00:28:37.663178+0000","last_unstale":"2024-06-10T00:32:50.189039+0000","last_undegraded":"2024-06-10T00:32:50.189039+0000","last_fullsized":"2024-06-10T00:32:50.189039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:46.032078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042870+0000","last_change":"2024-06-10T00:28:37.658353+0000","last_active":"2024-06-10T00:32:50.042870+0000","last_peered":"2024-06-10T00:32:50.042870+0000","last_clean":"2024-06-10T00:32:50.042870+0000","last_became_active":"2024-06-10T00:28:37.657961+0000","last_became_peered":"2024-06-10T00:28:37.657961+0000","last_unstale":"2024-06-10T00:32:50.042870+0000","last_undegraded":"2024-06-10T00:32:50.042870+0000","last_fullsized":"2024-06-10T00:32:50.042870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:22:27.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134878+0000","last_change":"2024-06-10T00:28:37.663823+0000","last_active":"2024-06-10T00:32:50.134878+0000","last_peered":"2024-06-10T00:32:50.134878+0000","last_clean":"2024-06-10T00:32:50.134878+0000","last_became_active":"2024-06-10T00:28:37.663499+0000","last_became_peered":"2024-06-10T00:28:37.663499+0000","last_unstale":"2024-06-10T00:32:50.134878+0000","last_undegraded":"2024-06-10T00:32:50.134878+0000","last_fullsized":"2024-06-10T00:32:50.134878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.100923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921877+0000","last_change":"2024-06-10T00:30:24.112722+0000","last_active":"2024-06-10T00:32:49.921877+0000","last_peered":"2024-06-10T00:32:49.921877+0000","last_clean":"2024-06-10T00:32:49.921877+0000","last_became_active":"2024-06-10T00:30:24.112325+0000","last_became_peered":"2024-06-10T00:30:24.112325+0000","last_unstale":"2024-06-10T00:32:49.921877+0000","last_undegraded":"2024-06-10T00:32:49.921877+0000","last_fullsized":"2024-06-10T00:32:49.921877+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:26.671583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133292+0000","last_change":"2024-06-10T00:28:37.649997+0000","last_active":"2024-06-10T00:32:50.133292+0000","last_peered":"2024-06-10T00:32:50.133292+0000","last_clean":"2024-06-10T00:32:50.133292+0000","last_became_active":"2024-06-10T00:28:37.649494+0000","last_became_peered":"2024-06-10T00:28:37.649494+0000","last_unstale":"2024-06-10T00:32:50.133292+0000","last_undegraded":"2024-06-10T00:32:50.133292+0000","last_fullsized":"2024-06-10T00:32:50.133292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:44:07.467206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192453+0000","last_change":"2024-06-10T00:28:37.653064+0000","last_active":"2024-06-10T00:32:50.192453+0000","last_peered":"2024-06-10T00:32:50.192453+0000","last_clean":"2024-06-10T00:32:50.192453+0000","last_became_active":"2024-06-10T00:28:37.652684+0000","last_became_peered":"2024-06-10T00:28:37.652684+0000","last_unstale":"2024-06-10T00:32:50.192453+0000","last_undegraded":"2024-06-10T00:32:50.192453+0000","last_fullsized":"2024-06-10T00:32:50.192453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:30.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192384+0000","last_change":"2024-06-10T00:28:37.655570+0000","last_active":"2024-06-10T00:32:50.192384+0000","last_peered":"2024-06-10T00:32:50.192384+0000","last_clean":"2024-06-10T00:32:50.192384+0000","last_became_active":"2024-06-10T00:28:37.655241+0000","last_became_peered":"2024-06-10T00:28:37.655241+0000","last_unstale":"2024-06-10T00:32:50.192384+0000","last_undegraded":"2024-06-10T00:32:50.192384+0000","last_fullsized":"2024-06-10T00:32:50.192384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:07:03.543064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188211+0000","last_change":"2024-06-10T00:28:37.665607+0000","last_active":"2024-06-10T00:32:50.188211+0000","last_peered":"2024-06-10T00:32:50.188211+0000","last_clean":"2024-06-10T00:32:50.188211+0000","last_became_active":"2024-06-10T00:28:37.664943+0000","last_became_peered":"2024-06-10T00:28:37.664943+0000","last_unstale":"2024-06-10T00:32:50.188211+0000","last_undegraded":"2024-06-10T00:32:50.188211+0000","last_fullsized":"2024-06-10T00:32:50.188211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:43:02.940298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199188+0000","last_change":"2024-06-10T00:28:37.658214+0000","last_active":"2024-06-10T00:32:50.199188+0000","last_peered":"2024-06-10T00:32:50.199188+0000","last_clean":"2024-06-10T00:32:50.199188+0000","last_became_active":"2024-06-10T00:28:37.657295+0000","last_became_peered":"2024-06-10T00:28:37.657295+0000","last_unstale":"2024-06-10T00:32:50.199188+0000","last_undegraded":"2024-06-10T00:32:50.199188+0000","last_fullsized":"2024-06-10T00:32:50.199188+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:20.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922079+0000","last_change":"2024-06-10T00:28:37.656645+0000","last_active":"2024-06-10T00:32:49.922079+0000","last_peered":"2024-06-10T00:32:49.922079+0000","last_clean":"2024-06-10T00:32:49.922079+0000","last_became_active":"2024-06-10T00:28:37.656157+0000","last_became_peered":"2024-06-10T00:28:37.656157+0000","last_unstale":"2024-06-10T00:32:49.922079+0000","last_undegraded":"2024-06-10T00:32:49.922079+0000","last_fullsized":"2024-06-10T00:32:49.922079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:00.953804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921572+0000","last_change":"2024-06-10T00:28:37.664336+0000","last_active":"2024-06-10T00:32:49.921572+0000","last_peered":"2024-06-10T00:32:49.921572+0000","last_clean":"2024-06-10T00:32:49.921572+0000","last_became_active":"2024-06-10T00:28:37.663960+0000","last_became_peered":"2024-06-10T00:28:37.663960+0000","last_unstale":"2024-06-10T00:32:49.921572+0000","last_undegraded":"2024-06-10T00:32:49.921572+0000","last_fullsized":"2024-06-10T00:32:49.921572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:59:32.774442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920810+0000","last_change":"2024-06-10T00:28:48.705930+0000","last_active":"2024-06-10T00:32:49.920810+0000","last_peered":"2024-06-10T00:32:49.920810+0000","last_clean":"2024-06-10T00:32:49.920810+0000","last_became_active":"2024-06-10T00:28:48.705558+0000","last_became_peered":"2024-06-10T00:28:48.705558+0000","last_unstale":"2024-06-10T00:32:49.920810+0000","last_undegraded":"2024-06-10T00:32:49.920810+0000","last_fullsized":"2024-06-10T00:32:49.920810+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:41:44.914378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"99'14","reported_seq":260,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132362+0000","last_change":"2024-06-10T00:29:23.862141+0000","last_active":"2024-06-10T00:32:50.132362+0000","last_peered":"2024-06-10T00:32:50.132362+0000","last_clean":"2024-06-10T00:32:50.132362+0000","last_became_active":"2024-06-10T00:29:23.857051+0000","last_became_peered":"2024-06-10T00:29:23.857051+0000","last_unstale":"2024-06-10T00:32:50.132362+0000","last_undegraded":"2024-06-10T00:32:50.132362+0000","last_fullsized":"2024-06-10T00:32:50.132362+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:41:41.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032309+0000","last_change":"2024-06-10T00:28:37.653938+0000","last_active":"2024-06-10T00:32:50.032309+0000","last_peered":"2024-06-10T00:32:50.032309+0000","last_clean":"2024-06-10T00:32:50.032309+0000","last_became_active":"2024-06-10T00:28:37.653629+0000","last_became_peered":"2024-06-10T00:28:37.653629+0000","last_unstale":"2024-06-10T00:32:50.032309+0000","last_undegraded":"2024-06-10T00:32:50.032309+0000","last_fullsized":"2024-06-10T00:32:50.032309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:21:39.285961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921432+0000","last_change":"2024-06-10T00:28:48.704413+0000","last_active":"2024-06-10T00:32:49.921432+0000","last_peered":"2024-06-10T00:32:49.921432+0000","last_clean":"2024-06-10T00:32:49.921432+0000","last_became_active":"2024-06-10T00:28:48.703972+0000","last_became_peered":"2024-06-10T00:28:48.703972+0000","last_unstale":"2024-06-10T00:32:49.921432+0000","last_undegraded":"2024-06-10T00:32:49.921432+0000","last_fullsized":"2024-06-10T00:32:49.921432+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:51.927435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032677+0000","last_change":"2024-06-10T00:28:34.445413+0000","last_active":"2024-06-10T00:32:50.032677+0000","last_peered":"2024-06-10T00:32:50.032677+0000","last_clean":"2024-06-10T00:32:50.032677+0000","last_became_active":"2024-06-10T00:28:34.445077+0000","last_became_peered":"2024-06-10T00:28:34.445077+0000","last_unstale":"2024-06-10T00:32:50.032677+0000","last_undegraded":"2024-06-10T00:32:50.032677+0000","last_fullsized":"2024-06-10T00:32:50.032677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:19:19.535976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193506+0000","last_change":"2024-06-10T00:28:37.657123+0000","last_active":"2024-06-10T00:32:50.193506+0000","last_peered":"2024-06-10T00:32:50.193506+0000","last_clean":"2024-06-10T00:32:50.193506+0000","last_became_active":"2024-06-10T00:28:37.656791+0000","last_became_peered":"2024-06-10T00:28:37.656791+0000","last_unstale":"2024-06-10T00:32:50.193506+0000","last_undegraded":"2024-06-10T00:32:50.193506+0000","last_fullsized":"2024-06-10T00:32:50.193506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:11.102478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921858+0000","last_change":"2024-06-10T00:28:48.705989+0000","last_active":"2024-06-10T00:32:49.921858+0000","last_peered":"2024-06-10T00:32:49.921858+0000","last_clean":"2024-06-10T00:32:49.921858+0000","last_became_active":"2024-06-10T00:28:48.705555+0000","last_became_peered":"2024-06-10T00:28:48.705555+0000","last_unstale":"2024-06-10T00:32:49.921858+0000","last_undegraded":"2024-06-10T00:32:49.921858+0000","last_fullsized":"2024-06-10T00:32:49.921858+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:36.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"100'13","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032133+0000","last_change":"2024-06-10T00:28:34.400117+0000","last_active":"2024-06-10T00:32:50.032133+0000","last_peered":"2024-06-10T00:32:50.032133+0000","last_clean":"2024-06-10T00:32:50.032133+0000","last_became_active":"2024-06-10T00:28:34.399743+0000","last_became_peered":"2024-06-10T00:28:34.399743+0000","last_unstale":"2024-06-10T00:32:50.032133+0000","last_undegraded":"2024-06-10T00:32:50.032133+0000","last_fullsized":"2024-06-10T00:32:50.032133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:20:57.580146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188627+0000","last_change":"2024-06-10T00:28:37.664005+0000","last_active":"2024-06-10T00:32:50.188627+0000","last_peered":"2024-06-10T00:32:50.188627+0000","last_clean":"2024-06-10T00:32:50.188627+0000","last_became_active":"2024-06-10T00:28:37.661297+0000","last_became_peered":"2024-06-10T00:28:37.661297+0000","last_unstale":"2024-06-10T00:32:50.188627+0000","last_undegraded":"2024-06-10T00:32:50.188627+0000","last_fullsized":"2024-06-10T00:32:50.188627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:57.795080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921561+0000","last_change":"2024-06-10T00:29:23.859091+0000","last_active":"2024-06-10T00:32:49.921561+0000","last_peered":"2024-06-10T00:32:49.921561+0000","last_clean":"2024-06-10T00:32:49.921561+0000","last_became_active":"2024-06-10T00:29:23.858485+0000","last_became_peered":"2024-06-10T00:29:23.858485+0000","last_unstale":"2024-06-10T00:32:49.921561+0000","last_undegraded":"2024-06-10T00:32:49.921561+0000","last_fullsized":"2024-06-10T00:32:49.921561+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:07:10.925323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031945+0000","last_change":"2024-06-10T00:28:34.400588+0000","last_active":"2024-06-10T00:32:50.031945+0000","last_peered":"2024-06-10T00:32:50.031945+0000","last_clean":"2024-06-10T00:32:50.031945+0000","last_became_active":"2024-06-10T00:28:34.400027+0000","last_became_peered":"2024-06-10T00:28:34.400027+0000","last_unstale":"2024-06-10T00:32:50.031945+0000","last_undegraded":"2024-06-10T00:32:50.031945+0000","last_fullsized":"2024-06-10T00:32:50.031945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:25:59.890076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039025+0000","last_change":"2024-06-10T00:28:37.659401+0000","last_active":"2024-06-10T00:32:50.039025+0000","last_peered":"2024-06-10T00:32:50.039025+0000","last_clean":"2024-06-10T00:32:50.039025+0000","last_became_active":"2024-06-10T00:28:37.659056+0000","last_became_peered":"2024-06-10T00:28:37.659056+0000","last_unstale":"2024-06-10T00:32:50.039025+0000","last_undegraded":"2024-06-10T00:32:50.039025+0000","last_fullsized":"2024-06-10T00:32:50.039025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:35:01.415630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921379+0000","last_change":"2024-06-10T00:28:48.702961+0000","last_active":"2024-06-10T00:32:49.921379+0000","last_peered":"2024-06-10T00:32:49.921379+0000","last_clean":"2024-06-10T00:32:49.921379+0000","last_became_active":"2024-06-10T00:28:48.702502+0000","last_became_peered":"2024-06-10T00:28:48.702502+0000","last_unstale":"2024-06-10T00:32:49.921379+0000","last_undegraded":"2024-06-10T00:32:49.921379+0000","last_fullsized":"2024-06-10T00:32:49.921379+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:54.881595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'9","reported_seq":244,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.191304+0000","last_change":"2024-06-10T00:28:34.396310+0000","last_active":"2024-06-10T00:32:50.191304+0000","last_peered":"2024-06-10T00:32:50.191304+0000","last_clean":"2024-06-10T00:32:50.191304+0000","last_became_active":"2024-06-10T00:28:34.395401+0000","last_became_peered":"2024-06-10T00:28:34.395401+0000","last_unstale":"2024-06-10T00:32:50.191304+0000","last_undegraded":"2024-06-10T00:32:50.191304+0000","last_fullsized":"2024-06-10T00:32:50.191304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:33:54.300308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031647+0000","last_change":"2024-06-10T00:28:37.651427+0000","last_active":"2024-06-10T00:32:50.031647+0000","last_peered":"2024-06-10T00:32:50.031647+0000","last_clean":"2024-06-10T00:32:50.031647+0000","last_became_active":"2024-06-10T00:28:37.650510+0000","last_became_peered":"2024-06-10T00:28:37.650510+0000","last_unstale":"2024-06-10T00:32:50.031647+0000","last_undegraded":"2024-06-10T00:32:50.031647+0000","last_fullsized":"2024-06-10T00:32:50.031647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:42.953919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920970+0000","last_change":"2024-06-10T00:28:48.705918+0000","last_active":"2024-06-10T00:32:49.920970+0000","last_peered":"2024-06-10T00:32:49.920970+0000","last_clean":"2024-06-10T00:32:49.920970+0000","last_became_active":"2024-06-10T00:28:48.705462+0000","last_became_peered":"2024-06-10T00:28:48.705462+0000","last_unstale":"2024-06-10T00:32:49.920970+0000","last_undegraded":"2024-06-10T00:32:49.920970+0000","last_fullsized":"2024-06-10T00:32:49.920970+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:59:23.176511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042358+0000","last_change":"2024-06-10T00:28:34.393199+0000","last_active":"2024-06-10T00:32:50.042358+0000","last_peered":"2024-06-10T00:32:50.042358+0000","last_clean":"2024-06-10T00:32:50.042358+0000","last_became_active":"2024-06-10T00:28:34.392777+0000","last_became_peered":"2024-06-10T00:28:34.392777+0000","last_unstale":"2024-06-10T00:32:50.042358+0000","last_undegraded":"2024-06-10T00:32:50.042358+0000","last_fullsized":"2024-06-10T00:32:50.042358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:29.614242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038961+0000","last_change":"2024-06-10T00:30:24.229721+0000","last_active":"2024-06-10T00:32:50.038961+0000","last_peered":"2024-06-10T00:32:50.038961+0000","last_clean":"2024-06-10T00:32:50.038961+0000","last_became_active":"2024-06-10T00:30:24.229407+0000","last_became_peered":"2024-06-10T00:30:24.229407+0000","last_unstale":"2024-06-10T00:32:50.038961+0000","last_undegraded":"2024-06-10T00:32:50.038961+0000","last_fullsized":"2024-06-10T00:32:50.038961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:39.523197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199084+0000","last_change":"2024-06-10T00:29:23.780971+0000","last_active":"2024-06-10T00:32:50.199084+0000","last_peered":"2024-06-10T00:32:50.199084+0000","last_clean":"2024-06-10T00:32:50.199084+0000","last_became_active":"2024-06-10T00:29:23.780034+0000","last_became_peered":"2024-06-10T00:29:23.780034+0000","last_unstale":"2024-06-10T00:32:50.199084+0000","last_undegraded":"2024-06-10T00:32:50.199084+0000","last_fullsized":"2024-06-10T00:32:50.199084+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:47:06.444181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132954+0000","last_change":"2024-06-10T00:28:34.392486+0000","last_active":"2024-06-10T00:32:50.132954+0000","last_peered":"2024-06-10T00:32:50.132954+0000","last_clean":"2024-06-10T00:32:50.132954+0000","last_became_active":"2024-06-10T00:28:34.392141+0000","last_became_peered":"2024-06-10T00:28:34.392141+0000","last_unstale":"2024-06-10T00:32:50.132954+0000","last_undegraded":"2024-06-10T00:32:50.132954+0000","last_fullsized":"2024-06-10T00:32:50.132954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:10:01.534195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031869+0000","last_change":"2024-06-10T00:28:37.651033+0000","last_active":"2024-06-10T00:32:50.031869+0000","last_peered":"2024-06-10T00:32:50.031869+0000","last_clean":"2024-06-10T00:32:50.031869+0000","last_became_active":"2024-06-10T00:28:37.650505+0000","last_became_peered":"2024-06-10T00:28:37.650505+0000","last_unstale":"2024-06-10T00:32:50.031869+0000","last_undegraded":"2024-06-10T00:32:50.031869+0000","last_fullsized":"2024-06-10T00:32:50.031869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:54.965865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193367+0000","last_change":"2024-06-10T00:28:48.710748+0000","last_active":"2024-06-10T00:32:50.193367+0000","last_peered":"2024-06-10T00:32:50.193367+0000","last_clean":"2024-06-10T00:32:50.193367+0000","last_became_active":"2024-06-10T00:28:48.710353+0000","last_became_peered":"2024-06-10T00:28:48.710353+0000","last_unstale":"2024-06-10T00:32:50.193367+0000","last_undegraded":"2024-06-10T00:32:50.193367+0000","last_fullsized":"2024-06-10T00:32:50.193367+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:30:47.546650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038780+0000","last_change":"2024-06-10T00:28:34.398911+0000","last_active":"2024-06-10T00:32:50.038780+0000","last_peered":"2024-06-10T00:32:50.038780+0000","last_clean":"2024-06-10T00:32:50.038780+0000","last_became_active":"2024-06-10T00:28:34.398373+0000","last_became_peered":"2024-06-10T00:28:34.398373+0000","last_unstale":"2024-06-10T00:32:50.038780+0000","last_undegraded":"2024-06-10T00:32:50.038780+0000","last_fullsized":"2024-06-10T00:32:50.038780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:05:34.352825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921526+0000","last_change":"2024-06-10T00:28:37.647492+0000","last_active":"2024-06-10T00:32:49.921526+0000","last_peered":"2024-06-10T00:32:49.921526+0000","last_clean":"2024-06-10T00:32:49.921526+0000","last_became_active":"2024-06-10T00:28:37.647103+0000","last_became_peered":"2024-06-10T00:28:37.647103+0000","last_unstale":"2024-06-10T00:32:49.921526+0000","last_undegraded":"2024-06-10T00:32:49.921526+0000","last_fullsized":"2024-06-10T00:32:49.921526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:10.429256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031883+0000","last_change":"2024-06-10T00:28:48.703095+0000","last_active":"2024-06-10T00:32:50.031883+0000","last_peered":"2024-06-10T00:32:50.031883+0000","last_clean":"2024-06-10T00:32:50.031883+0000","last_became_active":"2024-06-10T00:28:48.702687+0000","last_became_peered":"2024-06-10T00:28:48.702687+0000","last_unstale":"2024-06-10T00:32:50.031883+0000","last_undegraded":"2024-06-10T00:32:50.031883+0000","last_fullsized":"2024-06-10T00:32:50.031883+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:10:20.074173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":423,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.191573+0000","last_change":"2024-06-10T00:28:34.394807+0000","last_active":"2024-06-10T00:32:50.191573+0000","last_peered":"2024-06-10T00:32:50.191573+0000","last_clean":"2024-06-10T00:32:50.191573+0000","last_became_active":"2024-06-10T00:28:34.394022+0000","last_became_peered":"2024-06-10T00:28:34.394022+0000","last_unstale":"2024-06-10T00:32:50.191573+0000","last_undegraded":"2024-06-10T00:32:50.191573+0000","last_fullsized":"2024-06-10T00:32:50.191573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:21.626079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":370,"num_read_kb":370,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.189056+0000","last_change":"2024-06-10T00:28:37.662231+0000","last_active":"2024-06-10T00:32:50.189056+0000","last_peered":"2024-06-10T00:32:50.189056+0000","last_clean":"2024-06-10T00:32:50.189056+0000","last_became_active":"2024-06-10T00:28:37.661861+0000","last_became_peered":"2024-06-10T00:28:37.661861+0000","last_unstale":"2024-06-10T00:32:50.189056+0000","last_undegraded":"2024-06-10T00:32:50.189056+0000","last_fullsized":"2024-06-10T00:32:50.189056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:09.705349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921535+0000","last_change":"2024-06-10T00:28:48.706643+0000","last_active":"2024-06-10T00:32:49.921535+0000","last_peered":"2024-06-10T00:32:49.921535+0000","last_clean":"2024-06-10T00:32:49.921535+0000","last_became_active":"2024-06-10T00:28:48.706127+0000","last_became_peered":"2024-06-10T00:28:48.706127+0000","last_unstale":"2024-06-10T00:32:49.921535+0000","last_undegraded":"2024-06-10T00:32:49.921535+0000","last_fullsized":"2024-06-10T00:32:49.921535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:55.031303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134861+0000","last_change":"2024-06-10T00:28:34.392808+0000","last_active":"2024-06-10T00:32:50.134861+0000","last_peered":"2024-06-10T00:32:50.134861+0000","last_clean":"2024-06-10T00:32:50.134861+0000","last_became_active":"2024-06-10T00:28:34.392450+0000","last_became_peered":"2024-06-10T00:28:34.392450+0000","last_unstale":"2024-06-10T00:32:50.134861+0000","last_undegraded":"2024-06-10T00:32:50.134861+0000","last_fullsized":"2024-06-10T00:32:50.134861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:06:00.493316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193616+0000","last_change":"2024-06-10T00:28:37.656099+0000","last_active":"2024-06-10T00:32:50.193616+0000","last_peered":"2024-06-10T00:32:50.193616+0000","last_clean":"2024-06-10T00:32:50.193616+0000","last_became_active":"2024-06-10T00:28:37.655640+0000","last_became_peered":"2024-06-10T00:28:37.655640+0000","last_unstale":"2024-06-10T00:32:50.193616+0000","last_undegraded":"2024-06-10T00:32:50.193616+0000","last_fullsized":"2024-06-10T00:32:50.193616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:59:26.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922109+0000","last_change":"2024-06-10T00:28:48.702628+0000","last_active":"2024-06-10T00:32:49.922109+0000","last_peered":"2024-06-10T00:32:49.922109+0000","last_clean":"2024-06-10T00:32:49.922109+0000","last_became_active":"2024-06-10T00:28:48.702109+0000","last_became_peered":"2024-06-10T00:28:48.702109+0000","last_unstale":"2024-06-10T00:32:49.922109+0000","last_undegraded":"2024-06-10T00:32:49.922109+0000","last_fullsized":"2024-06-10T00:32:49.922109+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:25.262348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039027+0000","last_change":"2024-06-10T00:28:34.446788+0000","last_active":"2024-06-10T00:32:50.039027+0000","last_peered":"2024-06-10T00:32:50.039027+0000","last_clean":"2024-06-10T00:32:50.039027+0000","last_became_active":"2024-06-10T00:28:34.446377+0000","last_became_peered":"2024-06-10T00:28:34.446377+0000","last_unstale":"2024-06-10T00:32:50.039027+0000","last_undegraded":"2024-06-10T00:32:50.039027+0000","last_fullsized":"2024-06-10T00:32:50.039027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:16.914244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188829+0000","last_change":"2024-06-10T00:28:37.666159+0000","last_active":"2024-06-10T00:32:50.188829+0000","last_peered":"2024-06-10T00:32:50.188829+0000","last_clean":"2024-06-10T00:32:50.188829+0000","last_became_active":"2024-06-10T00:28:37.665711+0000","last_became_peered":"2024-06-10T00:28:37.665711+0000","last_unstale":"2024-06-10T00:32:50.188829+0000","last_undegraded":"2024-06-10T00:32:50.188829+0000","last_fullsized":"2024-06-10T00:32:50.188829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:03.982375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921636+0000","last_change":"2024-06-10T00:28:48.705071+0000","last_active":"2024-06-10T00:32:49.921636+0000","last_peered":"2024-06-10T00:32:49.921636+0000","last_clean":"2024-06-10T00:32:49.921636+0000","last_became_active":"2024-06-10T00:28:48.702211+0000","last_became_peered":"2024-06-10T00:28:48.702211+0000","last_unstale":"2024-06-10T00:32:49.921636+0000","last_undegraded":"2024-06-10T00:32:49.921636+0000","last_fullsized":"2024-06-10T00:32:49.921636+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:13.361985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":468,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.195703+0000","last_change":"2024-06-10T00:28:34.389784+0000","last_active":"2024-06-10T00:32:50.195703+0000","last_peered":"2024-06-10T00:32:50.195703+0000","last_clean":"2024-06-10T00:32:50.195703+0000","last_became_active":"2024-06-10T00:28:34.389416+0000","last_became_peered":"2024-06-10T00:28:34.389416+0000","last_unstale":"2024-06-10T00:32:50.195703+0000","last_undegraded":"2024-06-10T00:32:50.195703+0000","last_fullsized":"2024-06-10T00:32:50.195703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:23:05.803503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69365,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":381,"num_read_kb":253,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031957+0000","last_change":"2024-06-10T00:28:37.656174+0000","last_active":"2024-06-10T00:32:50.031957+0000","last_peered":"2024-06-10T00:32:50.031957+0000","last_clean":"2024-06-10T00:32:50.031957+0000","last_became_active":"2024-06-10T00:28:37.655794+0000","last_became_peered":"2024-06-10T00:28:37.655794+0000","last_unstale":"2024-06-10T00:32:50.031957+0000","last_undegraded":"2024-06-10T00:32:50.031957+0000","last_fullsized":"2024-06-10T00:32:50.031957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:22:00.310309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039312+0000","last_change":"2024-06-10T00:28:48.709915+0000","last_active":"2024-06-10T00:32:50.039312+0000","last_peered":"2024-06-10T00:32:50.039312+0000","last_clean":"2024-06-10T00:32:50.039312+0000","last_became_active":"2024-06-10T00:28:48.709426+0000","last_became_peered":"2024-06-10T00:28:48.709426+0000","last_unstale":"2024-06-10T00:32:50.039312+0000","last_undegraded":"2024-06-10T00:32:50.039312+0000","last_fullsized":"2024-06-10T00:32:50.039312+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:55.150617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921708+0000","last_change":"2024-06-10T00:28:34.389136+0000","last_active":"2024-06-10T00:32:49.921708+0000","last_peered":"2024-06-10T00:32:49.921708+0000","last_clean":"2024-06-10T00:32:49.921708+0000","last_became_active":"2024-06-10T00:28:34.388701+0000","last_became_peered":"2024-06-10T00:28:34.388701+0000","last_unstale":"2024-06-10T00:32:49.921708+0000","last_undegraded":"2024-06-10T00:32:49.921708+0000","last_fullsized":"2024-06-10T00:32:49.921708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:13:24.907824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133047+0000","last_change":"2024-06-10T00:28:37.659048+0000","last_active":"2024-06-10T00:32:50.133047+0000","last_peered":"2024-06-10T00:32:50.133047+0000","last_clean":"2024-06-10T00:32:50.133047+0000","last_became_active":"2024-06-10T00:28:37.658710+0000","last_became_peered":"2024-06-10T00:28:37.658710+0000","last_unstale":"2024-06-10T00:32:50.133047+0000","last_undegraded":"2024-06-10T00:32:50.133047+0000","last_fullsized":"2024-06-10T00:32:50.133047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:44.314559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198919+0000","last_change":"2024-06-10T00:28:48.709683+0000","last_active":"2024-06-10T00:32:50.198919+0000","last_peered":"2024-06-10T00:32:50.198919+0000","last_clean":"2024-06-10T00:32:50.198919+0000","last_became_active":"2024-06-10T00:28:48.709241+0000","last_became_peered":"2024-06-10T00:28:48.709241+0000","last_unstale":"2024-06-10T00:32:50.198919+0000","last_undegraded":"2024-06-10T00:32:50.198919+0000","last_fullsized":"2024-06-10T00:32:50.198919+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:25.190043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":24,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198772+0000","last_change":"2024-06-10T00:29:23.782029+0000","last_active":"2024-06-10T00:32:50.198772+0000","last_peered":"2024-06-10T00:32:50.198772+0000","last_clean":"2024-06-10T00:32:50.198772+0000","last_became_active":"2024-06-10T00:29:23.781565+0000","last_became_peered":"2024-06-10T00:29:23.781565+0000","last_unstale":"2024-06-10T00:32:50.198772+0000","last_undegraded":"2024-06-10T00:32:50.198772+0000","last_fullsized":"2024-06-10T00:32:50.198772+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:53.258760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134547+0000","last_change":"2024-06-10T00:28:37.653385+0000","last_active":"2024-06-10T00:32:50.134547+0000","last_peered":"2024-06-10T00:32:50.134547+0000","last_clean":"2024-06-10T00:32:50.134547+0000","last_became_active":"2024-06-10T00:28:37.653020+0000","last_became_peered":"2024-06-10T00:28:37.653020+0000","last_unstale":"2024-06-10T00:32:50.134547+0000","last_undegraded":"2024-06-10T00:32:50.134547+0000","last_fullsized":"2024-06-10T00:32:50.134547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:57:54.469544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921940+0000","last_change":"2024-06-10T00:28:48.707540+0000","last_active":"2024-06-10T00:32:49.921940+0000","last_peered":"2024-06-10T00:32:49.921940+0000","last_clean":"2024-06-10T00:32:49.921940+0000","last_became_active":"2024-06-10T00:28:48.707143+0000","last_became_peered":"2024-06-10T00:28:48.707143+0000","last_unstale":"2024-06-10T00:32:49.921940+0000","last_undegraded":"2024-06-10T00:32:49.921940+0000","last_fullsized":"2024-06-10T00:32:49.921940+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:20:29.344808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'8","reported_seq":244,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133792+0000","last_change":"2024-06-10T00:28:34.397496+0000","last_active":"2024-06-10T00:32:50.133792+0000","last_peered":"2024-06-10T00:32:50.133792+0000","last_clean":"2024-06-10T00:32:50.133792+0000","last_became_active":"2024-06-10T00:28:34.397124+0000","last_became_peered":"2024-06-10T00:28:34.397124+0000","last_unstale":"2024-06-10T00:32:50.133792+0000","last_undegraded":"2024-06-10T00:32:50.133792+0000","last_fullsized":"2024-06-10T00:32:50.133792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:41:48.609354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134324+0000","last_change":"2024-06-10T00:28:37.658338+0000","last_active":"2024-06-10T00:32:50.134324+0000","last_peered":"2024-06-10T00:32:50.134324+0000","last_clean":"2024-06-10T00:32:50.134324+0000","last_became_active":"2024-06-10T00:28:37.655910+0000","last_became_peered":"2024-06-10T00:28:37.655910+0000","last_unstale":"2024-06-10T00:32:50.134324+0000","last_undegraded":"2024-06-10T00:32:50.134324+0000","last_fullsized":"2024-06-10T00:32:50.134324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:38.501335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133960+0000","last_change":"2024-06-10T00:28:48.711151+0000","last_active":"2024-06-10T00:32:50.133960+0000","last_peered":"2024-06-10T00:32:50.133960+0000","last_clean":"2024-06-10T00:32:50.133960+0000","last_became_active":"2024-06-10T00:28:48.710768+0000","last_became_peered":"2024-06-10T00:28:48.710768+0000","last_unstale":"2024-06-10T00:32:50.133960+0000","last_undegraded":"2024-06-10T00:32:50.133960+0000","last_fullsized":"2024-06-10T00:32:50.133960+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:15:22.791839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":65,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031568+0000","last_change":"2024-06-10T00:28:34.445812+0000","last_active":"2024-06-10T00:32:50.031568+0000","last_peered":"2024-06-10T00:32:50.031568+0000","last_clean":"2024-06-10T00:32:50.031568+0000","last_became_active":"2024-06-10T00:28:34.445472+0000","last_became_peered":"2024-06-10T00:28:34.445472+0000","last_unstale":"2024-06-10T00:32:50.031568+0000","last_undegraded":"2024-06-10T00:32:50.031568+0000","last_fullsized":"2024-06-10T00:32:50.031568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:00.912050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038769+0000","last_change":"2024-06-10T00:28:37.655345+0000","last_active":"2024-06-10T00:32:50.038769+0000","last_peered":"2024-06-10T00:32:50.038769+0000","last_clean":"2024-06-10T00:32:50.038769+0000","last_became_active":"2024-06-10T00:28:37.654954+0000","last_became_peered":"2024-06-10T00:28:37.654954+0000","last_unstale":"2024-06-10T00:32:50.038769+0000","last_undegraded":"2024-06-10T00:32:50.038769+0000","last_fullsized":"2024-06-10T00:32:50.038769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:39:27.403006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038741+0000","last_change":"2024-06-10T00:28:48.705247+0000","last_active":"2024-06-10T00:32:50.038741+0000","last_peered":"2024-06-10T00:32:50.038741+0000","last_clean":"2024-06-10T00:32:50.038741+0000","last_became_active":"2024-06-10T00:28:48.704822+0000","last_became_peered":"2024-06-10T00:28:48.704822+0000","last_unstale":"2024-06-10T00:32:50.038741+0000","last_undegraded":"2024-06-10T00:32:50.038741+0000","last_fullsized":"2024-06-10T00:32:50.038741+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:24:59.991491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032099+0000","last_change":"2024-06-10T00:28:34.396792+0000","last_active":"2024-06-10T00:32:50.032099+0000","last_peered":"2024-06-10T00:32:50.032099+0000","last_clean":"2024-06-10T00:32:50.032099+0000","last_became_active":"2024-06-10T00:28:34.396442+0000","last_became_peered":"2024-06-10T00:28:34.396442+0000","last_unstale":"2024-06-10T00:32:50.032099+0000","last_undegraded":"2024-06-10T00:32:50.032099+0000","last_fullsized":"2024-06-10T00:32:50.032099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:46.436198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199145+0000","last_change":"2024-06-10T00:28:37.657698+0000","last_active":"2024-06-10T00:32:50.199145+0000","last_peered":"2024-06-10T00:32:50.199145+0000","last_clean":"2024-06-10T00:32:50.199145+0000","last_became_active":"2024-06-10T00:28:37.657364+0000","last_became_peered":"2024-06-10T00:28:37.657364+0000","last_unstale":"2024-06-10T00:32:50.199145+0000","last_undegraded":"2024-06-10T00:32:50.199145+0000","last_fullsized":"2024-06-10T00:32:50.199145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:23.097015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031580+0000","last_change":"2024-06-10T00:28:48.727567+0000","last_active":"2024-06-10T00:32:50.031580+0000","last_peered":"2024-06-10T00:32:50.031580+0000","last_clean":"2024-06-10T00:32:50.031580+0000","last_became_active":"2024-06-10T00:28:48.727108+0000","last_became_peered":"2024-06-10T00:28:48.727108+0000","last_unstale":"2024-06-10T00:32:50.031580+0000","last_undegraded":"2024-06-10T00:32:50.031580+0000","last_fullsized":"2024-06-10T00:32:50.031580+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:35:00.824177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921917+0000","last_change":"2024-06-10T00:28:34.446832+0000","last_active":"2024-06-10T00:32:49.921917+0000","last_peered":"2024-06-10T00:32:49.921917+0000","last_clean":"2024-06-10T00:32:49.921917+0000","last_became_active":"2024-06-10T00:28:34.446435+0000","last_became_peered":"2024-06-10T00:28:34.446435+0000","last_unstale":"2024-06-10T00:32:49.921917+0000","last_undegraded":"2024-06-10T00:32:49.921917+0000","last_fullsized":"2024-06-10T00:32:49.921917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:58.296674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042645+0000","last_change":"2024-06-10T00:30:24.229514+0000","last_active":"2024-06-10T00:32:50.042645+0000","last_peered":"2024-06-10T00:32:50.042645+0000","last_clean":"2024-06-10T00:32:50.042645+0000","last_became_active":"2024-06-10T00:30:24.229163+0000","last_became_peered":"2024-06-10T00:30:24.229163+0000","last_unstale":"2024-06-10T00:32:50.042645+0000","last_undegraded":"2024-06-10T00:32:50.042645+0000","last_fullsized":"2024-06-10T00:32:50.042645+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:52.914144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133741+0000","last_change":"2024-06-10T00:28:48.704683+0000","last_active":"2024-06-10T00:32:50.133741+0000","last_peered":"2024-06-10T00:32:50.133741+0000","last_clean":"2024-06-10T00:32:50.133741+0000","last_became_active":"2024-06-10T00:28:48.704297+0000","last_became_peered":"2024-06-10T00:28:48.704297+0000","last_unstale":"2024-06-10T00:32:50.133741+0000","last_undegraded":"2024-06-10T00:32:50.133741+0000","last_fullsized":"2024-06-10T00:32:50.133741+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:44.980544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'8","reported_seq":242,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188609+0000","last_change":"2024-06-10T00:28:34.393328+0000","last_active":"2024-06-10T00:32:50.188609+0000","last_peered":"2024-06-10T00:32:50.188609+0000","last_clean":"2024-06-10T00:32:50.188609+0000","last_became_active":"2024-06-10T00:28:34.391968+0000","last_became_peered":"2024-06-10T00:28:34.391968+0000","last_unstale":"2024-06-10T00:32:50.188609+0000","last_undegraded":"2024-06-10T00:32:50.188609+0000","last_fullsized":"2024-06-10T00:32:50.188609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:18.898180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1676,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":2,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042474+0000","last_change":"2024-06-10T00:28:37.652555+0000","last_active":"2024-06-10T00:32:50.042474+0000","last_peered":"2024-06-10T00:32:50.042474+0000","last_clean":"2024-06-10T00:32:50.042474+0000","last_became_active":"2024-06-10T00:28:37.651476+0000","last_became_peered":"2024-06-10T00:28:37.651476+0000","last_unstale":"2024-06-10T00:32:50.042474+0000","last_undegraded":"2024-06-10T00:32:50.042474+0000","last_fullsized":"2024-06-10T00:32:50.042474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:04:57.823224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134015+0000","last_change":"2024-06-10T00:28:48.727568+0000","last_active":"2024-06-10T00:32:50.134015+0000","last_peered":"2024-06-10T00:32:50.134015+0000","last_clean":"2024-06-10T00:32:50.134015+0000","last_became_active":"2024-06-10T00:28:48.727018+0000","last_became_peered":"2024-06-10T00:28:48.727018+0000","last_unstale":"2024-06-10T00:32:50.134015+0000","last_undegraded":"2024-06-10T00:32:50.134015+0000","last_fullsized":"2024-06-10T00:32:50.134015+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:02.025212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921381+0000","last_change":"2024-06-10T00:30:24.651947+0000","last_active":"2024-06-10T00:32:49.921381+0000","last_peered":"2024-06-10T00:32:49.921381+0000","last_clean":"2024-06-10T00:32:49.921381+0000","last_became_active":"2024-06-10T00:30:24.651640+0000","last_became_peered":"2024-06-10T00:30:24.651640+0000","last_unstale":"2024-06-10T00:32:49.921381+0000","last_undegraded":"2024-06-10T00:32:49.921381+0000","last_fullsized":"2024-06-10T00:32:49.921381+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:47.070568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"100'7","reported_seq":240,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.191611+0000","last_change":"2024-06-10T00:28:34.445501+0000","last_active":"2024-06-10T00:32:50.191611+0000","last_peered":"2024-06-10T00:32:50.191611+0000","last_clean":"2024-06-10T00:32:50.191611+0000","last_became_active":"2024-06-10T00:28:34.445128+0000","last_became_peered":"2024-06-10T00:28:34.445128+0000","last_unstale":"2024-06-10T00:32:50.191611+0000","last_undegraded":"2024-06-10T00:32:50.191611+0000","last_fullsized":"2024-06-10T00:32:50.191611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:59.231585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134703+0000","last_change":"2024-06-10T00:28:48.709852+0000","last_active":"2024-06-10T00:32:50.134703+0000","last_peered":"2024-06-10T00:32:50.134703+0000","last_clean":"2024-06-10T00:32:50.134703+0000","last_became_active":"2024-06-10T00:28:48.709423+0000","last_became_peered":"2024-06-10T00:28:48.709423+0000","last_unstale":"2024-06-10T00:32:50.134703+0000","last_undegraded":"2024-06-10T00:32:50.134703+0000","last_fullsized":"2024-06-10T00:32:50.134703+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:42.123563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031914+0000","last_change":"2024-06-10T00:28:37.657423+0000","last_active":"2024-06-10T00:32:50.031914+0000","last_peered":"2024-06-10T00:32:50.031914+0000","last_clean":"2024-06-10T00:32:50.031914+0000","last_became_active":"2024-06-10T00:28:37.657140+0000","last_became_peered":"2024-06-10T00:28:37.657140+0000","last_unstale":"2024-06-10T00:32:50.031914+0000","last_undegraded":"2024-06-10T00:32:50.031914+0000","last_fullsized":"2024-06-10T00:32:50.031914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:20:08.943066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922033+0000","last_change":"2024-06-10T00:28:34.392585+0000","last_active":"2024-06-10T00:32:49.922033+0000","last_peered":"2024-06-10T00:32:49.922033+0000","last_clean":"2024-06-10T00:32:49.922033+0000","last_became_active":"2024-06-10T00:28:34.392229+0000","last_became_peered":"2024-06-10T00:28:34.392229+0000","last_unstale":"2024-06-10T00:32:49.922033+0000","last_undegraded":"2024-06-10T00:32:49.922033+0000","last_fullsized":"2024-06-10T00:32:49.922033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:11.275761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":380,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134207+0000","last_change":"2024-06-10T00:27:12.072577+0000","last_active":"2024-06-10T00:32:50.134207+0000","last_peered":"2024-06-10T00:32:50.134207+0000","last_clean":"2024-06-10T00:32:50.134207+0000","last_became_active":"2024-06-10T00:27:12.072193+0000","last_became_peered":"2024-06-10T00:27:12.072193+0000","last_unstale":"2024-06-10T00:32:50.134207+0000","last_undegraded":"2024-06-10T00:32:50.134207+0000","last_fullsized":"2024-06-10T00:32:50.134207+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_clean_scrub_stamp":"2024-06-10T00:23:46.826438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:57.782124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039093+0000","last_change":"2024-06-10T00:28:48.727913+0000","last_active":"2024-06-10T00:32:50.039093+0000","last_peered":"2024-06-10T00:32:50.039093+0000","last_clean":"2024-06-10T00:32:50.039093+0000","last_became_active":"2024-06-10T00:28:48.727402+0000","last_became_peered":"2024-06-10T00:28:48.727402+0000","last_unstale":"2024-06-10T00:32:50.039093+0000","last_undegraded":"2024-06-10T00:32:50.039093+0000","last_fullsized":"2024-06-10T00:32:50.039093+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:10:25.051296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132861+0000","last_change":"2024-06-10T00:28:37.659901+0000","last_active":"2024-06-10T00:32:50.132861+0000","last_peered":"2024-06-10T00:32:50.132861+0000","last_clean":"2024-06-10T00:32:50.132861+0000","last_became_active":"2024-06-10T00:28:37.659553+0000","last_became_peered":"2024-06-10T00:28:37.659553+0000","last_unstale":"2024-06-10T00:32:50.132861+0000","last_undegraded":"2024-06-10T00:32:50.132861+0000","last_fullsized":"2024-06-10T00:32:50.132861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:25.694823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134306+0000","last_change":"2024-06-10T00:28:34.398155+0000","last_active":"2024-06-10T00:32:50.134306+0000","last_peered":"2024-06-10T00:32:50.134306+0000","last_clean":"2024-06-10T00:32:50.134306+0000","last_became_active":"2024-06-10T00:28:34.397793+0000","last_became_peered":"2024-06-10T00:28:34.397793+0000","last_unstale":"2024-06-10T00:32:50.134306+0000","last_undegraded":"2024-06-10T00:32:50.134306+0000","last_fullsized":"2024-06-10T00:32:50.134306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:21.188139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198902+0000","last_change":"2024-06-10T00:28:48.709736+0000","last_active":"2024-06-10T00:32:50.198902+0000","last_peered":"2024-06-10T00:32:50.198902+0000","last_clean":"2024-06-10T00:32:50.198902+0000","last_became_active":"2024-06-10T00:28:48.709293+0000","last_became_peered":"2024-06-10T00:28:48.709293+0000","last_unstale":"2024-06-10T00:32:50.198902+0000","last_undegraded":"2024-06-10T00:32:50.198902+0000","last_fullsized":"2024-06-10T00:32:50.198902+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:30:04.840656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199015+0000","last_change":"2024-06-10T00:29:23.780030+0000","last_active":"2024-06-10T00:32:50.199015+0000","last_peered":"2024-06-10T00:32:50.199015+0000","last_clean":"2024-06-10T00:32:50.199015+0000","last_became_active":"2024-06-10T00:29:23.779589+0000","last_became_peered":"2024-06-10T00:29:23.779589+0000","last_unstale":"2024-06-10T00:32:50.199015+0000","last_undegraded":"2024-06-10T00:32:50.199015+0000","last_fullsized":"2024-06-10T00:32:50.199015+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:53.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133881+0000","last_change":"2024-06-10T00:28:37.658330+0000","last_active":"2024-06-10T00:32:50.133881+0000","last_peered":"2024-06-10T00:32:50.133881+0000","last_clean":"2024-06-10T00:32:50.133881+0000","last_became_active":"2024-06-10T00:28:37.657343+0000","last_became_peered":"2024-06-10T00:28:37.657343+0000","last_unstale":"2024-06-10T00:32:50.133881+0000","last_undegraded":"2024-06-10T00:32:50.133881+0000","last_fullsized":"2024-06-10T00:32:50.133881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:31:52.167402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031759+0000","last_change":"2024-06-10T00:28:48.727506+0000","last_active":"2024-06-10T00:32:50.031759+0000","last_peered":"2024-06-10T00:32:50.031759+0000","last_clean":"2024-06-10T00:32:50.031759+0000","last_became_active":"2024-06-10T00:28:48.727068+0000","last_became_peered":"2024-06-10T00:28:48.727068+0000","last_unstale":"2024-06-10T00:32:50.031759+0000","last_undegraded":"2024-06-10T00:32:50.031759+0000","last_fullsized":"2024-06-10T00:32:50.031759+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:42.147908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'51","reported_seq":121,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.148195+0000","last_change":"2024-06-10T00:28:34.399872+0000","last_active":"2024-06-10T00:32:50.148195+0000","last_peered":"2024-06-10T00:32:50.148195+0000","last_clean":"2024-06-10T00:32:50.148195+0000","last_became_active":"2024-06-10T00:28:34.399493+0000","last_became_peered":"2024-06-10T00:28:34.399493+0000","last_unstale":"2024-06-10T00:32:50.148195+0000","last_undegraded":"2024-06-10T00:32:50.148195+0000","last_fullsized":"2024-06-10T00:32:50.148195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:24:10.018110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032414+0000","last_change":"2024-06-10T00:28:37.656597+0000","last_active":"2024-06-10T00:32:50.032414+0000","last_peered":"2024-06-10T00:32:50.032414+0000","last_clean":"2024-06-10T00:32:50.032414+0000","last_became_active":"2024-06-10T00:28:37.654356+0000","last_became_peered":"2024-06-10T00:28:37.654356+0000","last_unstale":"2024-06-10T00:32:50.032414+0000","last_undegraded":"2024-06-10T00:32:50.032414+0000","last_fullsized":"2024-06-10T00:32:50.032414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:07:12.950683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920721+0000","last_change":"2024-06-10T00:28:48.702686+0000","last_active":"2024-06-10T00:32:49.920721+0000","last_peered":"2024-06-10T00:32:49.920721+0000","last_clean":"2024-06-10T00:32:49.920721+0000","last_became_active":"2024-06-10T00:28:48.702273+0000","last_became_peered":"2024-06-10T00:28:48.702273+0000","last_unstale":"2024-06-10T00:32:49.920721+0000","last_undegraded":"2024-06-10T00:32:49.920721+0000","last_fullsized":"2024-06-10T00:32:49.920721+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:00.078232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921732+0000","last_change":"2024-06-10T00:28:34.447549+0000","last_active":"2024-06-10T00:32:49.921732+0000","last_peered":"2024-06-10T00:32:49.921732+0000","last_clean":"2024-06-10T00:32:49.921732+0000","last_became_active":"2024-06-10T00:28:34.447197+0000","last_became_peered":"2024-06-10T00:28:34.447197+0000","last_unstale":"2024-06-10T00:32:49.921732+0000","last_undegraded":"2024-06-10T00:32:49.921732+0000","last_fullsized":"2024-06-10T00:32:49.921732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:47.525153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193431+0000","last_change":"2024-06-10T00:28:37.655585+0000","last_active":"2024-06-10T00:32:50.193431+0000","last_peered":"2024-06-10T00:32:50.193431+0000","last_clean":"2024-06-10T00:32:50.193431+0000","last_became_active":"2024-06-10T00:28:37.655225+0000","last_became_peered":"2024-06-10T00:28:37.655225+0000","last_unstale":"2024-06-10T00:32:50.193431+0000","last_undegraded":"2024-06-10T00:32:50.193431+0000","last_fullsized":"2024-06-10T00:32:50.193431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:57:25.184924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921606+0000","last_change":"2024-06-10T00:29:23.857447+0000","last_active":"2024-06-10T00:32:49.921606+0000","last_peered":"2024-06-10T00:32:49.921606+0000","last_clean":"2024-06-10T00:32:49.921606+0000","last_became_active":"2024-06-10T00:29:23.856943+0000","last_became_peered":"2024-06-10T00:29:23.856943+0000","last_unstale":"2024-06-10T00:32:49.921606+0000","last_undegraded":"2024-06-10T00:32:49.921606+0000","last_fullsized":"2024-06-10T00:32:49.921606+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:58:44.548818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188309+0000","last_change":"2024-06-10T00:28:34.445637+0000","last_active":"2024-06-10T00:32:50.188309+0000","last_peered":"2024-06-10T00:32:50.188309+0000","last_clean":"2024-06-10T00:32:50.188309+0000","last_became_active":"2024-06-10T00:28:34.445324+0000","last_became_peered":"2024-06-10T00:28:34.445324+0000","last_unstale":"2024-06-10T00:32:50.188309+0000","last_undegraded":"2024-06-10T00:32:50.188309+0000","last_fullsized":"2024-06-10T00:32:50.188309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:05.713557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031570+0000","last_change":"2024-06-10T00:28:37.659037+0000","last_active":"2024-06-10T00:32:50.031570+0000","last_peered":"2024-06-10T00:32:50.031570+0000","last_clean":"2024-06-10T00:32:50.031570+0000","last_became_active":"2024-06-10T00:28:37.658705+0000","last_became_peered":"2024-06-10T00:28:37.658705+0000","last_unstale":"2024-06-10T00:32:50.031570+0000","last_undegraded":"2024-06-10T00:32:50.031570+0000","last_fullsized":"2024-06-10T00:32:50.031570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:20:52.797237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922067+0000","last_change":"2024-06-10T00:28:48.704130+0000","last_active":"2024-06-10T00:32:49.922067+0000","last_peered":"2024-06-10T00:32:49.922067+0000","last_clean":"2024-06-10T00:32:49.922067+0000","last_became_active":"2024-06-10T00:28:48.703738+0000","last_became_peered":"2024-06-10T00:28:48.703738+0000","last_unstale":"2024-06-10T00:32:49.922067+0000","last_undegraded":"2024-06-10T00:32:49.922067+0000","last_fullsized":"2024-06-10T00:32:49.922067+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:01:40.436710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032028+0000","last_change":"2024-06-10T00:28:34.386667+0000","last_active":"2024-06-10T00:32:50.032028+0000","last_peered":"2024-06-10T00:32:50.032028+0000","last_clean":"2024-06-10T00:32:50.032028+0000","last_became_active":"2024-06-10T00:28:34.386363+0000","last_became_peered":"2024-06-10T00:28:34.386363+0000","last_unstale":"2024-06-10T00:32:50.032028+0000","last_undegraded":"2024-06-10T00:32:50.032028+0000","last_fullsized":"2024-06-10T00:32:50.032028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:11.850162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198745+0000","last_change":"2024-06-10T00:28:37.664754+0000","last_active":"2024-06-10T00:32:50.198745+0000","last_peered":"2024-06-10T00:32:50.198745+0000","last_clean":"2024-06-10T00:32:50.198745+0000","last_became_active":"2024-06-10T00:28:37.658574+0000","last_became_peered":"2024-06-10T00:28:37.658574+0000","last_unstale":"2024-06-10T00:32:50.198745+0000","last_undegraded":"2024-06-10T00:32:50.198745+0000","last_fullsized":"2024-06-10T00:32:50.198745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:16:47.437822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921563+0000","last_change":"2024-06-10T00:28:48.706560+0000","last_active":"2024-06-10T00:32:49.921563+0000","last_peered":"2024-06-10T00:32:49.921563+0000","last_clean":"2024-06-10T00:32:49.921563+0000","last_became_active":"2024-06-10T00:28:48.705986+0000","last_became_peered":"2024-06-10T00:28:48.705986+0000","last_unstale":"2024-06-10T00:32:49.921563+0000","last_undegraded":"2024-06-10T00:32:49.921563+0000","last_fullsized":"2024-06-10T00:32:49.921563+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:21:26.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921015+0000","last_change":"2024-06-10T00:28:34.394509+0000","last_active":"2024-06-10T00:32:49.921015+0000","last_peered":"2024-06-10T00:32:49.921015+0000","last_clean":"2024-06-10T00:32:49.921015+0000","last_became_active":"2024-06-10T00:28:34.394105+0000","last_became_peered":"2024-06-10T00:28:34.394105+0000","last_unstale":"2024-06-10T00:32:49.921015+0000","last_undegraded":"2024-06-10T00:32:49.921015+0000","last_fullsized":"2024-06-10T00:32:49.921015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:13.415544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032271+0000","last_change":"2024-06-10T00:28:37.653192+0000","last_active":"2024-06-10T00:32:50.032271+0000","last_peered":"2024-06-10T00:32:50.032271+0000","last_clean":"2024-06-10T00:32:50.032271+0000","last_became_active":"2024-06-10T00:28:37.652868+0000","last_became_peered":"2024-06-10T00:28:37.652868+0000","last_unstale":"2024-06-10T00:32:50.032271+0000","last_undegraded":"2024-06-10T00:32:50.032271+0000","last_fullsized":"2024-06-10T00:32:50.032271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:21.614915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921695+0000","last_change":"2024-06-10T00:28:48.727615+0000","last_active":"2024-06-10T00:32:49.921695+0000","last_peered":"2024-06-10T00:32:49.921695+0000","last_clean":"2024-06-10T00:32:49.921695+0000","last_became_active":"2024-06-10T00:28:48.727190+0000","last_became_peered":"2024-06-10T00:28:48.727190+0000","last_unstale":"2024-06-10T00:32:49.921695+0000","last_undegraded":"2024-06-10T00:32:49.921695+0000","last_fullsized":"2024-06-10T00:32:49.921695+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:57.498183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031723+0000","last_change":"2024-06-10T00:29:23.784131+0000","last_active":"2024-06-10T00:32:50.031723+0000","last_peered":"2024-06-10T00:32:50.031723+0000","last_clean":"2024-06-10T00:32:50.031723+0000","last_became_active":"2024-06-10T00:29:23.780909+0000","last_became_peered":"2024-06-10T00:29:23.780909+0000","last_unstale":"2024-06-10T00:32:50.031723+0000","last_undegraded":"2024-06-10T00:32:50.031723+0000","last_fullsized":"2024-06-10T00:32:50.031723+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.774217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134422+0000","last_change":"2024-06-10T00:28:37.660399+0000","last_active":"2024-06-10T00:32:50.134422+0000","last_peered":"2024-06-10T00:32:50.134422+0000","last_clean":"2024-06-10T00:32:50.134422+0000","last_became_active":"2024-06-10T00:28:37.660069+0000","last_became_peered":"2024-06-10T00:28:37.660069+0000","last_unstale":"2024-06-10T00:32:50.134422+0000","last_undegraded":"2024-06-10T00:32:50.134422+0000","last_fullsized":"2024-06-10T00:32:50.134422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:59.727883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188428+0000","last_change":"2024-06-10T00:28:48.713267+0000","last_active":"2024-06-10T00:32:50.188428+0000","last_peered":"2024-06-10T00:32:50.188428+0000","last_clean":"2024-06-10T00:32:50.188428+0000","last_became_active":"2024-06-10T00:28:48.712870+0000","last_became_peered":"2024-06-10T00:28:48.712870+0000","last_unstale":"2024-06-10T00:32:50.188428+0000","last_undegraded":"2024-06-10T00:32:50.188428+0000","last_fullsized":"2024-06-10T00:32:50.188428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:59.210101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'6","reported_seq":239,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.161235+0000","last_change":"2024-06-10T00:28:34.448652+0000","last_active":"2024-06-10T00:32:50.161235+0000","last_peered":"2024-06-10T00:32:50.161235+0000","last_clean":"2024-06-10T00:32:50.161235+0000","last_became_active":"2024-06-10T00:28:34.448243+0000","last_became_peered":"2024-06-10T00:28:34.448243+0000","last_unstale":"2024-06-10T00:32:50.161235+0000","last_undegraded":"2024-06-10T00:32:50.161235+0000","last_fullsized":"2024-06-10T00:32:50.161235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:51:40.003429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":187,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.135023+0000","last_change":"2024-06-10T00:28:37.662307+0000","last_active":"2024-06-10T00:32:50.135023+0000","last_peered":"2024-06-10T00:32:50.135023+0000","last_clean":"2024-06-10T00:32:50.135023+0000","last_became_active":"2024-06-10T00:28:37.661727+0000","last_became_peered":"2024-06-10T00:28:37.661727+0000","last_unstale":"2024-06-10T00:32:50.135023+0000","last_undegraded":"2024-06-10T00:32:50.135023+0000","last_fullsized":"2024-06-10T00:32:50.135023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:23.989429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192376+0000","last_change":"2024-06-10T00:29:23.857626+0000","last_active":"2024-06-10T00:32:50.192376+0000","last_peered":"2024-06-10T00:32:50.192376+0000","last_clean":"2024-06-10T00:32:50.192376+0000","last_became_active":"2024-06-10T00:29:23.857057+0000","last_became_peered":"2024-06-10T00:29:23.857057+0000","last_unstale":"2024-06-10T00:32:50.192376+0000","last_undegraded":"2024-06-10T00:32:50.192376+0000","last_fullsized":"2024-06-10T00:32:50.192376+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:25:10.340053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'41","reported_seq":95,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.145625+0000","last_change":"2024-06-10T00:28:34.399253+0000","last_active":"2024-06-10T00:32:50.145625+0000","last_peered":"2024-06-10T00:32:50.145625+0000","last_clean":"2024-06-10T00:32:50.145625+0000","last_became_active":"2024-06-10T00:28:34.397763+0000","last_became_peered":"2024-06-10T00:28:34.397763+0000","last_unstale":"2024-06-10T00:32:50.145625+0000","last_undegraded":"2024-06-10T00:32:50.145625+0000","last_fullsized":"2024-06-10T00:32:50.145625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:05:57.371625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134695+0000","last_change":"2024-06-10T00:28:37.658027+0000","last_active":"2024-06-10T00:32:50.134695+0000","last_peered":"2024-06-10T00:32:50.134695+0000","last_clean":"2024-06-10T00:32:50.134695+0000","last_became_active":"2024-06-10T00:28:37.655513+0000","last_became_peered":"2024-06-10T00:28:37.655513+0000","last_unstale":"2024-06-10T00:32:50.134695+0000","last_undegraded":"2024-06-10T00:32:50.134695+0000","last_fullsized":"2024-06-10T00:32:50.134695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:50:23.525855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038558+0000","last_change":"2024-06-10T00:28:48.706997+0000","last_active":"2024-06-10T00:32:50.038558+0000","last_peered":"2024-06-10T00:32:50.038558+0000","last_clean":"2024-06-10T00:32:50.038558+0000","last_became_active":"2024-06-10T00:28:48.706597+0000","last_became_peered":"2024-06-10T00:28:48.706597+0000","last_unstale":"2024-06-10T00:32:50.038558+0000","last_undegraded":"2024-06-10T00:32:50.038558+0000","last_fullsized":"2024-06-10T00:32:50.038558+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:16.385312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133886+0000","last_change":"2024-06-10T00:29:23.785774+0000","last_active":"2024-06-10T00:32:50.133886+0000","last_peered":"2024-06-10T00:32:50.133886+0000","last_clean":"2024-06-10T00:32:50.133886+0000","last_became_active":"2024-06-10T00:29:23.781768+0000","last_became_peered":"2024-06-10T00:29:23.781768+0000","last_unstale":"2024-06-10T00:32:50.133886+0000","last_undegraded":"2024-06-10T00:32:50.133886+0000","last_fullsized":"2024-06-10T00:32:50.133886+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:27.348264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134953+0000","last_change":"2024-06-10T00:28:37.657820+0000","last_active":"2024-06-10T00:32:50.134953+0000","last_peered":"2024-06-10T00:32:50.134953+0000","last_clean":"2024-06-10T00:32:50.134953+0000","last_became_active":"2024-06-10T00:28:37.657487+0000","last_became_peered":"2024-06-10T00:28:37.657487+0000","last_unstale":"2024-06-10T00:32:50.134953+0000","last_undegraded":"2024-06-10T00:32:50.134953+0000","last_fullsized":"2024-06-10T00:32:50.134953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:44:35.930401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198953+0000","last_change":"2024-06-10T00:28:37.664543+0000","last_active":"2024-06-10T00:32:50.198953+0000","last_peered":"2024-06-10T00:32:50.198953+0000","last_clean":"2024-06-10T00:32:50.198953+0000","last_became_active":"2024-06-10T00:28:37.659015+0000","last_became_peered":"2024-06-10T00:28:37.659015+0000","last_unstale":"2024-06-10T00:32:50.198953+0000","last_undegraded":"2024-06-10T00:32:50.198953+0000","last_fullsized":"2024-06-10T00:32:50.198953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:06.555472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921072+0000","last_change":"2024-06-10T00:28:37.659389+0000","last_active":"2024-06-10T00:32:49.921072+0000","last_peered":"2024-06-10T00:32:49.921072+0000","last_clean":"2024-06-10T00:32:49.921072+0000","last_became_active":"2024-06-10T00:28:37.658990+0000","last_became_peered":"2024-06-10T00:28:37.658990+0000","last_unstale":"2024-06-10T00:32:49.921072+0000","last_undegraded":"2024-06-10T00:32:49.921072+0000","last_fullsized":"2024-06-10T00:32:49.921072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:29.532779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920823+0000","last_change":"2024-06-10T00:28:37.655621+0000","last_active":"2024-06-10T00:32:49.920823+0000","last_peered":"2024-06-10T00:32:49.920823+0000","last_clean":"2024-06-10T00:32:49.920823+0000","last_became_active":"2024-06-10T00:28:37.653599+0000","last_became_peered":"2024-06-10T00:28:37.653599+0000","last_unstale":"2024-06-10T00:32:49.920823+0000","last_undegraded":"2024-06-10T00:32:49.920823+0000","last_fullsized":"2024-06-10T00:32:49.920823+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:22.165851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192846+0000","last_change":"2024-06-10T00:29:23.782727+0000","last_active":"2024-06-10T00:32:50.192846+0000","last_peered":"2024-06-10T00:32:50.192846+0000","last_clean":"2024-06-10T00:32:50.192846+0000","last_became_active":"2024-06-10T00:29:23.782309+0000","last_became_peered":"2024-06-10T00:29:23.782309+0000","last_unstale":"2024-06-10T00:32:50.192846+0000","last_undegraded":"2024-06-10T00:32:50.192846+0000","last_fullsized":"2024-06-10T00:32:50.192846+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:20.585955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133856+0000","last_change":"2024-06-10T00:28:37.646750+0000","last_active":"2024-06-10T00:32:50.133856+0000","last_peered":"2024-06-10T00:32:50.133856+0000","last_clean":"2024-06-10T00:32:50.133856+0000","last_became_active":"2024-06-10T00:28:37.646372+0000","last_became_peered":"2024-06-10T00:28:37.646372+0000","last_unstale":"2024-06-10T00:32:50.133856+0000","last_undegraded":"2024-06-10T00:32:50.133856+0000","last_fullsized":"2024-06-10T00:32:50.133856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:45.762956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199287+0000","last_change":"2024-06-10T00:28:37.662278+0000","last_active":"2024-06-10T00:32:50.199287+0000","last_peered":"2024-06-10T00:32:50.199287+0000","last_clean":"2024-06-10T00:32:50.199287+0000","last_became_active":"2024-06-10T00:28:37.661855+0000","last_became_peered":"2024-06-10T00:28:37.661855+0000","last_unstale":"2024-06-10T00:32:50.199287+0000","last_undegraded":"2024-06-10T00:32:50.199287+0000","last_fullsized":"2024-06-10T00:32:50.199287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:20.074208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921288+0000","last_change":"2024-06-10T00:28:37.656738+0000","last_active":"2024-06-10T00:32:49.921288+0000","last_peered":"2024-06-10T00:32:49.921288+0000","last_clean":"2024-06-10T00:32:49.921288+0000","last_became_active":"2024-06-10T00:28:37.656375+0000","last_became_peered":"2024-06-10T00:28:37.656375+0000","last_unstale":"2024-06-10T00:32:49.921288+0000","last_undegraded":"2024-06-10T00:32:49.921288+0000","last_fullsized":"2024-06-10T00:32:49.921288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:49:17.779670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032555+0000","last_change":"2024-06-10T00:28:37.656849+0000","last_active":"2024-06-10T00:32:50.032555+0000","last_peered":"2024-06-10T00:32:50.032555+0000","last_clean":"2024-06-10T00:32:50.032555+0000","last_became_active":"2024-06-10T00:28:37.656261+0000","last_became_peered":"2024-06-10T00:28:37.656261+0000","last_unstale":"2024-06-10T00:32:50.032555+0000","last_undegraded":"2024-06-10T00:32:50.032555+0000","last_fullsized":"2024-06-10T00:32:50.032555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:13.152165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038730+0000","last_change":"2024-06-10T00:28:37.648384+0000","last_active":"2024-06-10T00:32:50.038730+0000","last_peered":"2024-06-10T00:32:50.038730+0000","last_clean":"2024-06-10T00:32:50.038730+0000","last_became_active":"2024-06-10T00:28:37.648009+0000","last_became_peered":"2024-06-10T00:28:37.648009+0000","last_unstale":"2024-06-10T00:32:50.038730+0000","last_undegraded":"2024-06-10T00:32:50.038730+0000","last_fullsized":"2024-06-10T00:32:50.038730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:45:55.808932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032174+0000","last_change":"2024-06-10T00:28:37.655828+0000","last_active":"2024-06-10T00:32:50.032174+0000","last_peered":"2024-06-10T00:32:50.032174+0000","last_clean":"2024-06-10T00:32:50.032174+0000","last_became_active":"2024-06-10T00:28:37.655269+0000","last_became_peered":"2024-06-10T00:28:37.655269+0000","last_unstale":"2024-06-10T00:32:50.032174+0000","last_undegraded":"2024-06-10T00:32:50.032174+0000","last_fullsized":"2024-06-10T00:32:50.032174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:56.618469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188607+0000","last_change":"2024-06-10T00:28:37.663756+0000","last_active":"2024-06-10T00:32:50.188607+0000","last_peered":"2024-06-10T00:32:50.188607+0000","last_clean":"2024-06-10T00:32:50.188607+0000","last_became_active":"2024-06-10T00:28:37.663380+0000","last_became_peered":"2024-06-10T00:28:37.663380+0000","last_unstale":"2024-06-10T00:32:50.188607+0000","last_undegraded":"2024-06-10T00:32:50.188607+0000","last_fullsized":"2024-06-10T00:32:50.188607+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:45.534648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133127+0000","last_change":"2024-06-10T00:28:37.655246+0000","last_active":"2024-06-10T00:32:50.133127+0000","last_peered":"2024-06-10T00:32:50.133127+0000","last_clean":"2024-06-10T00:32:50.133127+0000","last_became_active":"2024-06-10T00:28:37.654605+0000","last_became_peered":"2024-06-10T00:28:37.654605+0000","last_unstale":"2024-06-10T00:32:50.133127+0000","last_undegraded":"2024-06-10T00:32:50.133127+0000","last_fullsized":"2024-06-10T00:32:50.133127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:15.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199118+0000","last_change":"2024-06-10T00:28:37.662485+0000","last_active":"2024-06-10T00:32:50.199118+0000","last_peered":"2024-06-10T00:32:50.199118+0000","last_clean":"2024-06-10T00:32:50.199118+0000","last_became_active":"2024-06-10T00:28:37.661900+0000","last_became_peered":"2024-06-10T00:28:37.661900+0000","last_unstale":"2024-06-10T00:32:50.199118+0000","last_undegraded":"2024-06-10T00:32:50.199118+0000","last_fullsized":"2024-06-10T00:32:50.199118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:30:03.210634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604313,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1988,"num_read_kb":1107,"num_write":386,"num_write_kb":1478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1812939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":311,"ondisk_log_size":311,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383746,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1175552,"data_stored":581073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645382,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28836,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716316,"statfs":{"total":95995035648,"available":95965507584,"internally_reserved":0,"allocated":1859584,"data_stored":1231040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808907,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1585152,"data_stored":977723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070546,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36864,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708288,"statfs":{"total":95995035648,"available":95957286912,"internally_reserved":0,"allocated":1560576,"data_stored":973221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332182,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1781760,"data_stored":1180215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495707,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":577003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36532,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708620,"statfs":{"total":95995035648,"available":95957626880,"internally_reserved":0,"allocated":1224704,"data_stored":631213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36500,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708652,"statfs":{"total":95995035648,"available":95957659648,"internally_reserved":0,"allocated":1187840,"data_stored":577761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36564,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708588,"statfs":{"total":95995035648,"available":95957594112,"internally_reserved":0,"allocated":1253376,"data_stored":640908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182506,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1703936,"data_stored":1100439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920869,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36484,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708668,"statfs":{"total":95995035648,"available":95957676032,"internally_reserved":0,"allocated":1171456,"data_stored":580910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757344,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28892,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716260,"statfs":{"total":95995035648,"available":95965450240,"internally_reserved":0,"allocated":1921024,"data_stored":1305298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":135641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":203622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:32:53.831 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-06-10T00:32:53.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-10T00:32:53.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.974+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2497852989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca4102500 msgr2=0x7f5ca41048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:53.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.974+0000 7f5cab1f4640 1 --2- 172.21.15.62:0/2497852989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca4102500 0x7f5ca41048f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009900 tx=0x7f5c9802a410 comp rx=0 tx=0).stop 2024-06-10T00:32:53.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.974+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2497852989 shutdown_connections 2024-06-10T00:32:53.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.974+0000 7f5cab1f4640 1 --2- 172.21.15.62:0/2497852989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca4107760 0x7f5ca4109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.974+0000 7f5cab1f4640 1 --2- 172.21.15.62:0/2497852989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ca4104e30 0x7f5ca4107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.974+0000 7f5cab1f4640 1 --2- 172.21.15.62:0/2497852989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca4102500 0x7f5ca41048f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.974+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2497852989 >> 172.21.15.62:0/2497852989 conn(0x7f5ca40fbe50 msgr2=0x7f5ca40fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.974+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2497852989 shutdown_connections 2024-06-10T00:32:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.974+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2497852989 wait complete. 2024-06-10T00:32:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.974+0000 7f5cab1f4640 1 Processor -- start 2024-06-10T00:32:53.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.974+0000 7f5cab1f4640 1 -- start start 2024-06-10T00:32:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5cab1f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ca4102500 0x7f5ca4197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5cab1f4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca4104e30 0x7f5ca4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5cab1f4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca4107760 0x7f5ca419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5cab1f4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca410d6c0 con 0x7f5ca4104e30 2024-06-10T00:32:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5cab1f4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca410d540 con 0x7f5ca4102500 2024-06-10T00:32:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5cab1f4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca410d840 con 0x7f5ca4107760 2024-06-10T00:32:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5ca3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca4104e30 0x7f5ca4198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:53.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5ca9791640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca4107760 0x7f5ca419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5ca8f90640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ca4102500 0x7f5ca4197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5ca3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca4104e30 0x7f5ca4198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45408/0 (socket says 172.21.15.62:45408) 2024-06-10T00:32:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5ca8f90640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ca4102500 0x7f5ca4197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:43106/0 (socket says 172.21.15.62:43106) 2024-06-10T00:32:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5ca3fff640 1 -- 172.21.15.62:0/2753838162 learned_addr learned my addr 172.21.15.62:0/2753838162 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5ca9791640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca4107760 0x7f5ca419f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:47658/0 (socket says 172.21.15.62:47658) 2024-06-10T00:32:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5ca9791640 1 -- 172.21.15.62:0/2753838162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ca4102500 msgr2=0x7f5ca4197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5ca9791640 1 --2- 172.21.15.62:0/2753838162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ca4102500 0x7f5ca4197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5ca9791640 1 -- 172.21.15.62:0/2753838162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca4104e30 msgr2=0x7f5ca4198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5ca9791640 1 --2- 172.21.15.62:0/2753838162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca4104e30 0x7f5ca4198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5ca9791640 1 -- 172.21.15.62:0/2753838162 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca419f930 con 0x7f5ca4107760 2024-06-10T00:32:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5ca3fff640 1 --2- 172.21.15.62:0/2753838162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca4104e30 0x7f5ca4198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:32:53.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.975+0000 7f5ca9791640 1 --2- 172.21.15.62:0/2753838162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca4107760 0x7f5ca419f290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009fd0 tx=0x7f5c9c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:53.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.976+0000 7f5ca1ffb640 1 -- 172.21.15.62:0/2753838162 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00ce20 con 0x7f5ca4107760 2024-06-10T00:32:53.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.976+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2753838162 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca419fc20 con 0x7f5ca4107760 2024-06-10T00:32:53.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.976+0000 7f5ca1ffb640 1 -- 172.21.15.62:0/2753838162 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c9c005020 con 0x7f5ca4107760 2024-06-10T00:32:53.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.976+0000 7f5ca1ffb640 1 -- 172.21.15.62:0/2753838162 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c010720 con 0x7f5ca4107760 2024-06-10T00:32:53.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.976+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2753838162 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca41a0190 con 0x7f5ca4107760 2024-06-10T00:32:53.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.977+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2753838162 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca40fd940 con 0x7f5ca4107760 2024-06-10T00:32:53.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.978+0000 7f5ca1ffb640 1 -- 172.21.15.62:0/2753838162 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 137953+0+0 (secure 0 0 0) 0x7f5c9c005540 con 0x7f5ca4107760 2024-06-10T00:32:53.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.979+0000 7f5ca1ffb640 1 --2- 172.21.15.62:0/2753838162 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5c940a2d00 0x7f5c940a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:53.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.979+0000 7f5ca8f90640 1 --2- 172.21.15.62:0/2753838162 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5c940a2d00 0x7f5c940a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:53.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.979+0000 7f5ca1ffb640 1 -- 172.21.15.62:0/2753838162 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f5c9c0cff60 con 0x7f5ca4107760 2024-06-10T00:32:53.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.979+0000 7f5ca8f90640 1 --2- 172.21.15.62:0/2753838162 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5c940a2d00 0x7f5c940a51c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c98031440 tx=0x7f5c980315b0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:53.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:53.983+0000 7f5ca1ffb640 1 -- 172.21.15.62:0/2753838162 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5c9c09c7d0 con 0x7f5ca4107760 2024-06-10T00:32:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:54.170+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2753838162 --> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ca4061de0 con 0x7f5c940a2d00 2024-06-10T00:32:54.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:54.179+0000 7f5ca1ffb640 1 -- 172.21.15.62:0/2753838162 <== mgr.15333 v2:172.21.15.62:6800/518034449 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339578 (secure 0 0 0) 0x7f5ca4061de0 con 0x7f5c940a2d00 2024-06-10T00:32:54.183 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:32:54.185 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-06-10T00:32:54.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:54.183+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2753838162 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5c940a2d00 msgr2=0x7f5c940a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:54.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:54.183+0000 7f5cab1f4640 1 --2- 172.21.15.62:0/2753838162 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5c940a2d00 0x7f5c940a51c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c98031440 tx=0x7f5c980315b0 comp rx=0 tx=0).stop 2024-06-10T00:32:54.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:54.183+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2753838162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca4107760 msgr2=0x7f5ca419f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:54.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:54.183+0000 7f5cab1f4640 1 --2- 172.21.15.62:0/2753838162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca4107760 0x7f5ca419f290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009fd0 tx=0x7f5c9c00ef10 comp rx=0 tx=0).stop 2024-06-10T00:32:54.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:54.183+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2753838162 shutdown_connections 2024-06-10T00:32:54.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:54.183+0000 7f5cab1f4640 1 --2- 172.21.15.62:0/2753838162 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5c940a2d00 0x7f5c940a51c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:54.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:54.183+0000 7f5cab1f4640 1 --2- 172.21.15.62:0/2753838162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca4107760 0x7f5ca419f290 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:54.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:54.183+0000 7f5cab1f4640 1 --2- 172.21.15.62:0/2753838162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca4104e30 0x7f5ca4198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:54.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:54.183+0000 7f5cab1f4640 1 --2- 172.21.15.62:0/2753838162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ca4102500 0x7f5ca4197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:54.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:54.183+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2753838162 >> 172.21.15.62:0/2753838162 conn(0x7f5ca40fbe50 msgr2=0x7f5ca4102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:54.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:54.183+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2753838162 shutdown_connections 2024-06-10T00:32:54.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:54.183+0000 7f5cab1f4640 1 -- 172.21.15.62:0/2753838162 wait complete. 2024-06-10T00:32:54.194 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-06-10T00:32:53.934529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194915,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2124,"num_read_kb":1226,"num_write":569,"num_write_kb":4418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":950,"ondisk_log_size":950,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387164,"kb_used_data":17176,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124554660,"statfs":{"total":1151940427776,"available":1151543971840,"internally_reserved":0,"allocated":17588224,"data_stored":10356804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":378713403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.994186"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199389+0000","last_change":"2024-06-10T00:28:37.657234+0000","last_active":"2024-06-10T00:32:50.199389+0000","last_peered":"2024-06-10T00:32:50.199389+0000","last_clean":"2024-06-10T00:32:50.199389+0000","last_became_active":"2024-06-10T00:28:37.656879+0000","last_became_peered":"2024-06-10T00:28:37.656879+0000","last_unstale":"2024-06-10T00:32:50.199389+0000","last_undegraded":"2024-06-10T00:32:50.199389+0000","last_fullsized":"2024-06-10T00:32:50.199389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:01:02.907597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193442+0000","last_change":"2024-06-10T00:28:37.656440+0000","last_active":"2024-06-10T00:32:50.193442+0000","last_peered":"2024-06-10T00:32:50.193442+0000","last_clean":"2024-06-10T00:32:50.193442+0000","last_became_active":"2024-06-10T00:28:37.656113+0000","last_became_peered":"2024-06-10T00:28:37.656113+0000","last_unstale":"2024-06-10T00:32:50.193442+0000","last_undegraded":"2024-06-10T00:32:50.193442+0000","last_fullsized":"2024-06-10T00:32:50.193442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:49:31.447951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193070+0000","last_change":"2024-06-10T00:28:37.659114+0000","last_active":"2024-06-10T00:32:50.193070+0000","last_peered":"2024-06-10T00:32:50.193070+0000","last_clean":"2024-06-10T00:32:50.193070+0000","last_became_active":"2024-06-10T00:28:37.658763+0000","last_became_peered":"2024-06-10T00:28:37.658763+0000","last_unstale":"2024-06-10T00:32:50.193070+0000","last_undegraded":"2024-06-10T00:32:50.193070+0000","last_fullsized":"2024-06-10T00:32:50.193070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:39:54.963590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921896+0000","last_change":"2024-06-10T00:28:37.656786+0000","last_active":"2024-06-10T00:32:49.921896+0000","last_peered":"2024-06-10T00:32:49.921896+0000","last_clean":"2024-06-10T00:32:49.921896+0000","last_became_active":"2024-06-10T00:28:37.656073+0000","last_became_peered":"2024-06-10T00:28:37.656073+0000","last_unstale":"2024-06-10T00:32:49.921896+0000","last_undegraded":"2024-06-10T00:32:49.921896+0000","last_fullsized":"2024-06-10T00:32:49.921896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:19.176608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031746+0000","last_change":"2024-06-10T00:28:37.650250+0000","last_active":"2024-06-10T00:32:50.031746+0000","last_peered":"2024-06-10T00:32:50.031746+0000","last_clean":"2024-06-10T00:32:50.031746+0000","last_became_active":"2024-06-10T00:28:37.649852+0000","last_became_peered":"2024-06-10T00:28:37.649852+0000","last_unstale":"2024-06-10T00:32:50.031746+0000","last_undegraded":"2024-06-10T00:32:50.031746+0000","last_fullsized":"2024-06-10T00:32:50.031746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:39.492884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042392+0000","last_change":"2024-06-10T00:28:37.646042+0000","last_active":"2024-06-10T00:32:50.042392+0000","last_peered":"2024-06-10T00:32:50.042392+0000","last_clean":"2024-06-10T00:32:50.042392+0000","last_became_active":"2024-06-10T00:28:37.644591+0000","last_became_peered":"2024-06-10T00:28:37.644591+0000","last_unstale":"2024-06-10T00:32:50.042392+0000","last_undegraded":"2024-06-10T00:32:50.042392+0000","last_fullsized":"2024-06-10T00:32:50.042392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:29.932303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134284+0000","last_change":"2024-06-10T00:28:37.664076+0000","last_active":"2024-06-10T00:32:50.134284+0000","last_peered":"2024-06-10T00:32:50.134284+0000","last_clean":"2024-06-10T00:32:50.134284+0000","last_became_active":"2024-06-10T00:28:37.663741+0000","last_became_peered":"2024-06-10T00:28:37.663741+0000","last_unstale":"2024-06-10T00:32:50.134284+0000","last_undegraded":"2024-06-10T00:32:50.134284+0000","last_fullsized":"2024-06-10T00:32:50.134284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:14.581409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032039+0000","last_change":"2024-06-10T00:28:37.648758+0000","last_active":"2024-06-10T00:32:50.032039+0000","last_peered":"2024-06-10T00:32:50.032039+0000","last_clean":"2024-06-10T00:32:50.032039+0000","last_became_active":"2024-06-10T00:28:37.648387+0000","last_became_peered":"2024-06-10T00:28:37.648387+0000","last_unstale":"2024-06-10T00:32:50.032039+0000","last_undegraded":"2024-06-10T00:32:50.032039+0000","last_fullsized":"2024-06-10T00:32:50.032039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:17.187265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.189039+0000","last_change":"2024-06-10T00:28:37.664650+0000","last_active":"2024-06-10T00:32:50.189039+0000","last_peered":"2024-06-10T00:32:50.189039+0000","last_clean":"2024-06-10T00:32:50.189039+0000","last_became_active":"2024-06-10T00:28:37.663178+0000","last_became_peered":"2024-06-10T00:28:37.663178+0000","last_unstale":"2024-06-10T00:32:50.189039+0000","last_undegraded":"2024-06-10T00:32:50.189039+0000","last_fullsized":"2024-06-10T00:32:50.189039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:46.032078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042870+0000","last_change":"2024-06-10T00:28:37.658353+0000","last_active":"2024-06-10T00:32:50.042870+0000","last_peered":"2024-06-10T00:32:50.042870+0000","last_clean":"2024-06-10T00:32:50.042870+0000","last_became_active":"2024-06-10T00:28:37.657961+0000","last_became_peered":"2024-06-10T00:28:37.657961+0000","last_unstale":"2024-06-10T00:32:50.042870+0000","last_undegraded":"2024-06-10T00:32:50.042870+0000","last_fullsized":"2024-06-10T00:32:50.042870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:22:27.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134878+0000","last_change":"2024-06-10T00:28:37.663823+0000","last_active":"2024-06-10T00:32:50.134878+0000","last_peered":"2024-06-10T00:32:50.134878+0000","last_clean":"2024-06-10T00:32:50.134878+0000","last_became_active":"2024-06-10T00:28:37.663499+0000","last_became_peered":"2024-06-10T00:28:37.663499+0000","last_unstale":"2024-06-10T00:32:50.134878+0000","last_undegraded":"2024-06-10T00:32:50.134878+0000","last_fullsized":"2024-06-10T00:32:50.134878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.100923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921877+0000","last_change":"2024-06-10T00:30:24.112722+0000","last_active":"2024-06-10T00:32:49.921877+0000","last_peered":"2024-06-10T00:32:49.921877+0000","last_clean":"2024-06-10T00:32:49.921877+0000","last_became_active":"2024-06-10T00:30:24.112325+0000","last_became_peered":"2024-06-10T00:30:24.112325+0000","last_unstale":"2024-06-10T00:32:49.921877+0000","last_undegraded":"2024-06-10T00:32:49.921877+0000","last_fullsized":"2024-06-10T00:32:49.921877+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:26.671583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133292+0000","last_change":"2024-06-10T00:28:37.649997+0000","last_active":"2024-06-10T00:32:50.133292+0000","last_peered":"2024-06-10T00:32:50.133292+0000","last_clean":"2024-06-10T00:32:50.133292+0000","last_became_active":"2024-06-10T00:28:37.649494+0000","last_became_peered":"2024-06-10T00:28:37.649494+0000","last_unstale":"2024-06-10T00:32:50.133292+0000","last_undegraded":"2024-06-10T00:32:50.133292+0000","last_fullsized":"2024-06-10T00:32:50.133292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:44:07.467206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192453+0000","last_change":"2024-06-10T00:28:37.653064+0000","last_active":"2024-06-10T00:32:50.192453+0000","last_peered":"2024-06-10T00:32:50.192453+0000","last_clean":"2024-06-10T00:32:50.192453+0000","last_became_active":"2024-06-10T00:28:37.652684+0000","last_became_peered":"2024-06-10T00:28:37.652684+0000","last_unstale":"2024-06-10T00:32:50.192453+0000","last_undegraded":"2024-06-10T00:32:50.192453+0000","last_fullsized":"2024-06-10T00:32:50.192453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:30.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192384+0000","last_change":"2024-06-10T00:28:37.655570+0000","last_active":"2024-06-10T00:32:50.192384+0000","last_peered":"2024-06-10T00:32:50.192384+0000","last_clean":"2024-06-10T00:32:50.192384+0000","last_became_active":"2024-06-10T00:28:37.655241+0000","last_became_peered":"2024-06-10T00:28:37.655241+0000","last_unstale":"2024-06-10T00:32:50.192384+0000","last_undegraded":"2024-06-10T00:32:50.192384+0000","last_fullsized":"2024-06-10T00:32:50.192384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:07:03.543064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188211+0000","last_change":"2024-06-10T00:28:37.665607+0000","last_active":"2024-06-10T00:32:50.188211+0000","last_peered":"2024-06-10T00:32:50.188211+0000","last_clean":"2024-06-10T00:32:50.188211+0000","last_became_active":"2024-06-10T00:28:37.664943+0000","last_became_peered":"2024-06-10T00:28:37.664943+0000","last_unstale":"2024-06-10T00:32:50.188211+0000","last_undegraded":"2024-06-10T00:32:50.188211+0000","last_fullsized":"2024-06-10T00:32:50.188211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:43:02.940298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199188+0000","last_change":"2024-06-10T00:28:37.658214+0000","last_active":"2024-06-10T00:32:50.199188+0000","last_peered":"2024-06-10T00:32:50.199188+0000","last_clean":"2024-06-10T00:32:50.199188+0000","last_became_active":"2024-06-10T00:28:37.657295+0000","last_became_peered":"2024-06-10T00:28:37.657295+0000","last_unstale":"2024-06-10T00:32:50.199188+0000","last_undegraded":"2024-06-10T00:32:50.199188+0000","last_fullsized":"2024-06-10T00:32:50.199188+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:20.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922079+0000","last_change":"2024-06-10T00:28:37.656645+0000","last_active":"2024-06-10T00:32:49.922079+0000","last_peered":"2024-06-10T00:32:49.922079+0000","last_clean":"2024-06-10T00:32:49.922079+0000","last_became_active":"2024-06-10T00:28:37.656157+0000","last_became_peered":"2024-06-10T00:28:37.656157+0000","last_unstale":"2024-06-10T00:32:49.922079+0000","last_undegraded":"2024-06-10T00:32:49.922079+0000","last_fullsized":"2024-06-10T00:32:49.922079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:00.953804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921572+0000","last_change":"2024-06-10T00:28:37.664336+0000","last_active":"2024-06-10T00:32:49.921572+0000","last_peered":"2024-06-10T00:32:49.921572+0000","last_clean":"2024-06-10T00:32:49.921572+0000","last_became_active":"2024-06-10T00:28:37.663960+0000","last_became_peered":"2024-06-10T00:28:37.663960+0000","last_unstale":"2024-06-10T00:32:49.921572+0000","last_undegraded":"2024-06-10T00:32:49.921572+0000","last_fullsized":"2024-06-10T00:32:49.921572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:59:32.774442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920810+0000","last_change":"2024-06-10T00:28:48.705930+0000","last_active":"2024-06-10T00:32:49.920810+0000","last_peered":"2024-06-10T00:32:49.920810+0000","last_clean":"2024-06-10T00:32:49.920810+0000","last_became_active":"2024-06-10T00:28:48.705558+0000","last_became_peered":"2024-06-10T00:28:48.705558+0000","last_unstale":"2024-06-10T00:32:49.920810+0000","last_undegraded":"2024-06-10T00:32:49.920810+0000","last_fullsized":"2024-06-10T00:32:49.920810+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:41:44.914378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"99'14","reported_seq":260,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132362+0000","last_change":"2024-06-10T00:29:23.862141+0000","last_active":"2024-06-10T00:32:50.132362+0000","last_peered":"2024-06-10T00:32:50.132362+0000","last_clean":"2024-06-10T00:32:50.132362+0000","last_became_active":"2024-06-10T00:29:23.857051+0000","last_became_peered":"2024-06-10T00:29:23.857051+0000","last_unstale":"2024-06-10T00:32:50.132362+0000","last_undegraded":"2024-06-10T00:32:50.132362+0000","last_fullsized":"2024-06-10T00:32:50.132362+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:41:41.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032309+0000","last_change":"2024-06-10T00:28:37.653938+0000","last_active":"2024-06-10T00:32:50.032309+0000","last_peered":"2024-06-10T00:32:50.032309+0000","last_clean":"2024-06-10T00:32:50.032309+0000","last_became_active":"2024-06-10T00:28:37.653629+0000","last_became_peered":"2024-06-10T00:28:37.653629+0000","last_unstale":"2024-06-10T00:32:50.032309+0000","last_undegraded":"2024-06-10T00:32:50.032309+0000","last_fullsized":"2024-06-10T00:32:50.032309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:21:39.285961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921432+0000","last_change":"2024-06-10T00:28:48.704413+0000","last_active":"2024-06-10T00:32:49.921432+0000","last_peered":"2024-06-10T00:32:49.921432+0000","last_clean":"2024-06-10T00:32:49.921432+0000","last_became_active":"2024-06-10T00:28:48.703972+0000","last_became_peered":"2024-06-10T00:28:48.703972+0000","last_unstale":"2024-06-10T00:32:49.921432+0000","last_undegraded":"2024-06-10T00:32:49.921432+0000","last_fullsized":"2024-06-10T00:32:49.921432+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:51.927435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032677+0000","last_change":"2024-06-10T00:28:34.445413+0000","last_active":"2024-06-10T00:32:50.032677+0000","last_peered":"2024-06-10T00:32:50.032677+0000","last_clean":"2024-06-10T00:32:50.032677+0000","last_became_active":"2024-06-10T00:28:34.445077+0000","last_became_peered":"2024-06-10T00:28:34.445077+0000","last_unstale":"2024-06-10T00:32:50.032677+0000","last_undegraded":"2024-06-10T00:32:50.032677+0000","last_fullsized":"2024-06-10T00:32:50.032677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:19:19.535976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193506+0000","last_change":"2024-06-10T00:28:37.657123+0000","last_active":"2024-06-10T00:32:50.193506+0000","last_peered":"2024-06-10T00:32:50.193506+0000","last_clean":"2024-06-10T00:32:50.193506+0000","last_became_active":"2024-06-10T00:28:37.656791+0000","last_became_peered":"2024-06-10T00:28:37.656791+0000","last_unstale":"2024-06-10T00:32:50.193506+0000","last_undegraded":"2024-06-10T00:32:50.193506+0000","last_fullsized":"2024-06-10T00:32:50.193506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:11.102478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921858+0000","last_change":"2024-06-10T00:28:48.705989+0000","last_active":"2024-06-10T00:32:49.921858+0000","last_peered":"2024-06-10T00:32:49.921858+0000","last_clean":"2024-06-10T00:32:49.921858+0000","last_became_active":"2024-06-10T00:28:48.705555+0000","last_became_peered":"2024-06-10T00:28:48.705555+0000","last_unstale":"2024-06-10T00:32:49.921858+0000","last_undegraded":"2024-06-10T00:32:49.921858+0000","last_fullsized":"2024-06-10T00:32:49.921858+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:36.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"100'13","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032133+0000","last_change":"2024-06-10T00:28:34.400117+0000","last_active":"2024-06-10T00:32:50.032133+0000","last_peered":"2024-06-10T00:32:50.032133+0000","last_clean":"2024-06-10T00:32:50.032133+0000","last_became_active":"2024-06-10T00:28:34.399743+0000","last_became_peered":"2024-06-10T00:28:34.399743+0000","last_unstale":"2024-06-10T00:32:50.032133+0000","last_undegraded":"2024-06-10T00:32:50.032133+0000","last_fullsized":"2024-06-10T00:32:50.032133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:20:57.580146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188627+0000","last_change":"2024-06-10T00:28:37.664005+0000","last_active":"2024-06-10T00:32:50.188627+0000","last_peered":"2024-06-10T00:32:50.188627+0000","last_clean":"2024-06-10T00:32:50.188627+0000","last_became_active":"2024-06-10T00:28:37.661297+0000","last_became_peered":"2024-06-10T00:28:37.661297+0000","last_unstale":"2024-06-10T00:32:50.188627+0000","last_undegraded":"2024-06-10T00:32:50.188627+0000","last_fullsized":"2024-06-10T00:32:50.188627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:57.795080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921561+0000","last_change":"2024-06-10T00:29:23.859091+0000","last_active":"2024-06-10T00:32:49.921561+0000","last_peered":"2024-06-10T00:32:49.921561+0000","last_clean":"2024-06-10T00:32:49.921561+0000","last_became_active":"2024-06-10T00:29:23.858485+0000","last_became_peered":"2024-06-10T00:29:23.858485+0000","last_unstale":"2024-06-10T00:32:49.921561+0000","last_undegraded":"2024-06-10T00:32:49.921561+0000","last_fullsized":"2024-06-10T00:32:49.921561+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:07:10.925323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031945+0000","last_change":"2024-06-10T00:28:34.400588+0000","last_active":"2024-06-10T00:32:50.031945+0000","last_peered":"2024-06-10T00:32:50.031945+0000","last_clean":"2024-06-10T00:32:50.031945+0000","last_became_active":"2024-06-10T00:28:34.400027+0000","last_became_peered":"2024-06-10T00:28:34.400027+0000","last_unstale":"2024-06-10T00:32:50.031945+0000","last_undegraded":"2024-06-10T00:32:50.031945+0000","last_fullsized":"2024-06-10T00:32:50.031945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:25:59.890076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039025+0000","last_change":"2024-06-10T00:28:37.659401+0000","last_active":"2024-06-10T00:32:50.039025+0000","last_peered":"2024-06-10T00:32:50.039025+0000","last_clean":"2024-06-10T00:32:50.039025+0000","last_became_active":"2024-06-10T00:28:37.659056+0000","last_became_peered":"2024-06-10T00:28:37.659056+0000","last_unstale":"2024-06-10T00:32:50.039025+0000","last_undegraded":"2024-06-10T00:32:50.039025+0000","last_fullsized":"2024-06-10T00:32:50.039025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:35:01.415630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921379+0000","last_change":"2024-06-10T00:28:48.702961+0000","last_active":"2024-06-10T00:32:49.921379+0000","last_peered":"2024-06-10T00:32:49.921379+0000","last_clean":"2024-06-10T00:32:49.921379+0000","last_became_active":"2024-06-10T00:28:48.702502+0000","last_became_peered":"2024-06-10T00:28:48.702502+0000","last_unstale":"2024-06-10T00:32:49.921379+0000","last_undegraded":"2024-06-10T00:32:49.921379+0000","last_fullsized":"2024-06-10T00:32:49.921379+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:54.881595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'9","reported_seq":244,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.191304+0000","last_change":"2024-06-10T00:28:34.396310+0000","last_active":"2024-06-10T00:32:50.191304+0000","last_peered":"2024-06-10T00:32:50.191304+0000","last_clean":"2024-06-10T00:32:50.191304+0000","last_became_active":"2024-06-10T00:28:34.395401+0000","last_became_peered":"2024-06-10T00:28:34.395401+0000","last_unstale":"2024-06-10T00:32:50.191304+0000","last_undegraded":"2024-06-10T00:32:50.191304+0000","last_fullsized":"2024-06-10T00:32:50.191304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:33:54.300308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031647+0000","last_change":"2024-06-10T00:28:37.651427+0000","last_active":"2024-06-10T00:32:50.031647+0000","last_peered":"2024-06-10T00:32:50.031647+0000","last_clean":"2024-06-10T00:32:50.031647+0000","last_became_active":"2024-06-10T00:28:37.650510+0000","last_became_peered":"2024-06-10T00:28:37.650510+0000","last_unstale":"2024-06-10T00:32:50.031647+0000","last_undegraded":"2024-06-10T00:32:50.031647+0000","last_fullsized":"2024-06-10T00:32:50.031647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:42.953919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920970+0000","last_change":"2024-06-10T00:28:48.705918+0000","last_active":"2024-06-10T00:32:49.920970+0000","last_peered":"2024-06-10T00:32:49.920970+0000","last_clean":"2024-06-10T00:32:49.920970+0000","last_became_active":"2024-06-10T00:28:48.705462+0000","last_became_peered":"2024-06-10T00:28:48.705462+0000","last_unstale":"2024-06-10T00:32:49.920970+0000","last_undegraded":"2024-06-10T00:32:49.920970+0000","last_fullsized":"2024-06-10T00:32:49.920970+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:59:23.176511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042358+0000","last_change":"2024-06-10T00:28:34.393199+0000","last_active":"2024-06-10T00:32:50.042358+0000","last_peered":"2024-06-10T00:32:50.042358+0000","last_clean":"2024-06-10T00:32:50.042358+0000","last_became_active":"2024-06-10T00:28:34.392777+0000","last_became_peered":"2024-06-10T00:28:34.392777+0000","last_unstale":"2024-06-10T00:32:50.042358+0000","last_undegraded":"2024-06-10T00:32:50.042358+0000","last_fullsized":"2024-06-10T00:32:50.042358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:29.614242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038961+0000","last_change":"2024-06-10T00:30:24.229721+0000","last_active":"2024-06-10T00:32:50.038961+0000","last_peered":"2024-06-10T00:32:50.038961+0000","last_clean":"2024-06-10T00:32:50.038961+0000","last_became_active":"2024-06-10T00:30:24.229407+0000","last_became_peered":"2024-06-10T00:30:24.229407+0000","last_unstale":"2024-06-10T00:32:50.038961+0000","last_undegraded":"2024-06-10T00:32:50.038961+0000","last_fullsized":"2024-06-10T00:32:50.038961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:39.523197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199084+0000","last_change":"2024-06-10T00:29:23.780971+0000","last_active":"2024-06-10T00:32:50.199084+0000","last_peered":"2024-06-10T00:32:50.199084+0000","last_clean":"2024-06-10T00:32:50.199084+0000","last_became_active":"2024-06-10T00:29:23.780034+0000","last_became_peered":"2024-06-10T00:29:23.780034+0000","last_unstale":"2024-06-10T00:32:50.199084+0000","last_undegraded":"2024-06-10T00:32:50.199084+0000","last_fullsized":"2024-06-10T00:32:50.199084+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:47:06.444181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132954+0000","last_change":"2024-06-10T00:28:34.392486+0000","last_active":"2024-06-10T00:32:50.132954+0000","last_peered":"2024-06-10T00:32:50.132954+0000","last_clean":"2024-06-10T00:32:50.132954+0000","last_became_active":"2024-06-10T00:28:34.392141+0000","last_became_peered":"2024-06-10T00:28:34.392141+0000","last_unstale":"2024-06-10T00:32:50.132954+0000","last_undegraded":"2024-06-10T00:32:50.132954+0000","last_fullsized":"2024-06-10T00:32:50.132954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:10:01.534195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031869+0000","last_change":"2024-06-10T00:28:37.651033+0000","last_active":"2024-06-10T00:32:50.031869+0000","last_peered":"2024-06-10T00:32:50.031869+0000","last_clean":"2024-06-10T00:32:50.031869+0000","last_became_active":"2024-06-10T00:28:37.650505+0000","last_became_peered":"2024-06-10T00:28:37.650505+0000","last_unstale":"2024-06-10T00:32:50.031869+0000","last_undegraded":"2024-06-10T00:32:50.031869+0000","last_fullsized":"2024-06-10T00:32:50.031869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:54.965865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193367+0000","last_change":"2024-06-10T00:28:48.710748+0000","last_active":"2024-06-10T00:32:50.193367+0000","last_peered":"2024-06-10T00:32:50.193367+0000","last_clean":"2024-06-10T00:32:50.193367+0000","last_became_active":"2024-06-10T00:28:48.710353+0000","last_became_peered":"2024-06-10T00:28:48.710353+0000","last_unstale":"2024-06-10T00:32:50.193367+0000","last_undegraded":"2024-06-10T00:32:50.193367+0000","last_fullsized":"2024-06-10T00:32:50.193367+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:30:47.546650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038780+0000","last_change":"2024-06-10T00:28:34.398911+0000","last_active":"2024-06-10T00:32:50.038780+0000","last_peered":"2024-06-10T00:32:50.038780+0000","last_clean":"2024-06-10T00:32:50.038780+0000","last_became_active":"2024-06-10T00:28:34.398373+0000","last_became_peered":"2024-06-10T00:28:34.398373+0000","last_unstale":"2024-06-10T00:32:50.038780+0000","last_undegraded":"2024-06-10T00:32:50.038780+0000","last_fullsized":"2024-06-10T00:32:50.038780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:05:34.352825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921526+0000","last_change":"2024-06-10T00:28:37.647492+0000","last_active":"2024-06-10T00:32:49.921526+0000","last_peered":"2024-06-10T00:32:49.921526+0000","last_clean":"2024-06-10T00:32:49.921526+0000","last_became_active":"2024-06-10T00:28:37.647103+0000","last_became_peered":"2024-06-10T00:28:37.647103+0000","last_unstale":"2024-06-10T00:32:49.921526+0000","last_undegraded":"2024-06-10T00:32:49.921526+0000","last_fullsized":"2024-06-10T00:32:49.921526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:10.429256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031883+0000","last_change":"2024-06-10T00:28:48.703095+0000","last_active":"2024-06-10T00:32:50.031883+0000","last_peered":"2024-06-10T00:32:50.031883+0000","last_clean":"2024-06-10T00:32:50.031883+0000","last_became_active":"2024-06-10T00:28:48.702687+0000","last_became_peered":"2024-06-10T00:28:48.702687+0000","last_unstale":"2024-06-10T00:32:50.031883+0000","last_undegraded":"2024-06-10T00:32:50.031883+0000","last_fullsized":"2024-06-10T00:32:50.031883+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:10:20.074173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":423,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.191573+0000","last_change":"2024-06-10T00:28:34.394807+0000","last_active":"2024-06-10T00:32:50.191573+0000","last_peered":"2024-06-10T00:32:50.191573+0000","last_clean":"2024-06-10T00:32:50.191573+0000","last_became_active":"2024-06-10T00:28:34.394022+0000","last_became_peered":"2024-06-10T00:28:34.394022+0000","last_unstale":"2024-06-10T00:32:50.191573+0000","last_undegraded":"2024-06-10T00:32:50.191573+0000","last_fullsized":"2024-06-10T00:32:50.191573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:21.626079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":370,"num_read_kb":370,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.189056+0000","last_change":"2024-06-10T00:28:37.662231+0000","last_active":"2024-06-10T00:32:50.189056+0000","last_peered":"2024-06-10T00:32:50.189056+0000","last_clean":"2024-06-10T00:32:50.189056+0000","last_became_active":"2024-06-10T00:28:37.661861+0000","last_became_peered":"2024-06-10T00:28:37.661861+0000","last_unstale":"2024-06-10T00:32:50.189056+0000","last_undegraded":"2024-06-10T00:32:50.189056+0000","last_fullsized":"2024-06-10T00:32:50.189056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:09.705349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921535+0000","last_change":"2024-06-10T00:28:48.706643+0000","last_active":"2024-06-10T00:32:49.921535+0000","last_peered":"2024-06-10T00:32:49.921535+0000","last_clean":"2024-06-10T00:32:49.921535+0000","last_became_active":"2024-06-10T00:28:48.706127+0000","last_became_peered":"2024-06-10T00:28:48.706127+0000","last_unstale":"2024-06-10T00:32:49.921535+0000","last_undegraded":"2024-06-10T00:32:49.921535+0000","last_fullsized":"2024-06-10T00:32:49.921535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:55.031303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134861+0000","last_change":"2024-06-10T00:28:34.392808+0000","last_active":"2024-06-10T00:32:50.134861+0000","last_peered":"2024-06-10T00:32:50.134861+0000","last_clean":"2024-06-10T00:32:50.134861+0000","last_became_active":"2024-06-10T00:28:34.392450+0000","last_became_peered":"2024-06-10T00:28:34.392450+0000","last_unstale":"2024-06-10T00:32:50.134861+0000","last_undegraded":"2024-06-10T00:32:50.134861+0000","last_fullsized":"2024-06-10T00:32:50.134861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:06:00.493316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193616+0000","last_change":"2024-06-10T00:28:37.656099+0000","last_active":"2024-06-10T00:32:50.193616+0000","last_peered":"2024-06-10T00:32:50.193616+0000","last_clean":"2024-06-10T00:32:50.193616+0000","last_became_active":"2024-06-10T00:28:37.655640+0000","last_became_peered":"2024-06-10T00:28:37.655640+0000","last_unstale":"2024-06-10T00:32:50.193616+0000","last_undegraded":"2024-06-10T00:32:50.193616+0000","last_fullsized":"2024-06-10T00:32:50.193616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:59:26.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922109+0000","last_change":"2024-06-10T00:28:48.702628+0000","last_active":"2024-06-10T00:32:49.922109+0000","last_peered":"2024-06-10T00:32:49.922109+0000","last_clean":"2024-06-10T00:32:49.922109+0000","last_became_active":"2024-06-10T00:28:48.702109+0000","last_became_peered":"2024-06-10T00:28:48.702109+0000","last_unstale":"2024-06-10T00:32:49.922109+0000","last_undegraded":"2024-06-10T00:32:49.922109+0000","last_fullsized":"2024-06-10T00:32:49.922109+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:25.262348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039027+0000","last_change":"2024-06-10T00:28:34.446788+0000","last_active":"2024-06-10T00:32:50.039027+0000","last_peered":"2024-06-10T00:32:50.039027+0000","last_clean":"2024-06-10T00:32:50.039027+0000","last_became_active":"2024-06-10T00:28:34.446377+0000","last_became_peered":"2024-06-10T00:28:34.446377+0000","last_unstale":"2024-06-10T00:32:50.039027+0000","last_undegraded":"2024-06-10T00:32:50.039027+0000","last_fullsized":"2024-06-10T00:32:50.039027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:16.914244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188829+0000","last_change":"2024-06-10T00:28:37.666159+0000","last_active":"2024-06-10T00:32:50.188829+0000","last_peered":"2024-06-10T00:32:50.188829+0000","last_clean":"2024-06-10T00:32:50.188829+0000","last_became_active":"2024-06-10T00:28:37.665711+0000","last_became_peered":"2024-06-10T00:28:37.665711+0000","last_unstale":"2024-06-10T00:32:50.188829+0000","last_undegraded":"2024-06-10T00:32:50.188829+0000","last_fullsized":"2024-06-10T00:32:50.188829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:03.982375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921636+0000","last_change":"2024-06-10T00:28:48.705071+0000","last_active":"2024-06-10T00:32:49.921636+0000","last_peered":"2024-06-10T00:32:49.921636+0000","last_clean":"2024-06-10T00:32:49.921636+0000","last_became_active":"2024-06-10T00:28:48.702211+0000","last_became_peered":"2024-06-10T00:28:48.702211+0000","last_unstale":"2024-06-10T00:32:49.921636+0000","last_undegraded":"2024-06-10T00:32:49.921636+0000","last_fullsized":"2024-06-10T00:32:49.921636+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:13.361985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":468,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.195703+0000","last_change":"2024-06-10T00:28:34.389784+0000","last_active":"2024-06-10T00:32:50.195703+0000","last_peered":"2024-06-10T00:32:50.195703+0000","last_clean":"2024-06-10T00:32:50.195703+0000","last_became_active":"2024-06-10T00:28:34.389416+0000","last_became_peered":"2024-06-10T00:28:34.389416+0000","last_unstale":"2024-06-10T00:32:50.195703+0000","last_undegraded":"2024-06-10T00:32:50.195703+0000","last_fullsized":"2024-06-10T00:32:50.195703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:23:05.803503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69365,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":381,"num_read_kb":253,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031957+0000","last_change":"2024-06-10T00:28:37.656174+0000","last_active":"2024-06-10T00:32:50.031957+0000","last_peered":"2024-06-10T00:32:50.031957+0000","last_clean":"2024-06-10T00:32:50.031957+0000","last_became_active":"2024-06-10T00:28:37.655794+0000","last_became_peered":"2024-06-10T00:28:37.655794+0000","last_unstale":"2024-06-10T00:32:50.031957+0000","last_undegraded":"2024-06-10T00:32:50.031957+0000","last_fullsized":"2024-06-10T00:32:50.031957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:22:00.310309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039312+0000","last_change":"2024-06-10T00:28:48.709915+0000","last_active":"2024-06-10T00:32:50.039312+0000","last_peered":"2024-06-10T00:32:50.039312+0000","last_clean":"2024-06-10T00:32:50.039312+0000","last_became_active":"2024-06-10T00:28:48.709426+0000","last_became_peered":"2024-06-10T00:28:48.709426+0000","last_unstale":"2024-06-10T00:32:50.039312+0000","last_undegraded":"2024-06-10T00:32:50.039312+0000","last_fullsized":"2024-06-10T00:32:50.039312+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:55.150617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921708+0000","last_change":"2024-06-10T00:28:34.389136+0000","last_active":"2024-06-10T00:32:49.921708+0000","last_peered":"2024-06-10T00:32:49.921708+0000","last_clean":"2024-06-10T00:32:49.921708+0000","last_became_active":"2024-06-10T00:28:34.388701+0000","last_became_peered":"2024-06-10T00:28:34.388701+0000","last_unstale":"2024-06-10T00:32:49.921708+0000","last_undegraded":"2024-06-10T00:32:49.921708+0000","last_fullsized":"2024-06-10T00:32:49.921708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:13:24.907824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133047+0000","last_change":"2024-06-10T00:28:37.659048+0000","last_active":"2024-06-10T00:32:50.133047+0000","last_peered":"2024-06-10T00:32:50.133047+0000","last_clean":"2024-06-10T00:32:50.133047+0000","last_became_active":"2024-06-10T00:28:37.658710+0000","last_became_peered":"2024-06-10T00:28:37.658710+0000","last_unstale":"2024-06-10T00:32:50.133047+0000","last_undegraded":"2024-06-10T00:32:50.133047+0000","last_fullsized":"2024-06-10T00:32:50.133047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:44.314559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198919+0000","last_change":"2024-06-10T00:28:48.709683+0000","last_active":"2024-06-10T00:32:50.198919+0000","last_peered":"2024-06-10T00:32:50.198919+0000","last_clean":"2024-06-10T00:32:50.198919+0000","last_became_active":"2024-06-10T00:28:48.709241+0000","last_became_peered":"2024-06-10T00:28:48.709241+0000","last_unstale":"2024-06-10T00:32:50.198919+0000","last_undegraded":"2024-06-10T00:32:50.198919+0000","last_fullsized":"2024-06-10T00:32:50.198919+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:25.190043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":24,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198772+0000","last_change":"2024-06-10T00:29:23.782029+0000","last_active":"2024-06-10T00:32:50.198772+0000","last_peered":"2024-06-10T00:32:50.198772+0000","last_clean":"2024-06-10T00:32:50.198772+0000","last_became_active":"2024-06-10T00:29:23.781565+0000","last_became_peered":"2024-06-10T00:29:23.781565+0000","last_unstale":"2024-06-10T00:32:50.198772+0000","last_undegraded":"2024-06-10T00:32:50.198772+0000","last_fullsized":"2024-06-10T00:32:50.198772+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:53.258760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134547+0000","last_change":"2024-06-10T00:28:37.653385+0000","last_active":"2024-06-10T00:32:50.134547+0000","last_peered":"2024-06-10T00:32:50.134547+0000","last_clean":"2024-06-10T00:32:50.134547+0000","last_became_active":"2024-06-10T00:28:37.653020+0000","last_became_peered":"2024-06-10T00:28:37.653020+0000","last_unstale":"2024-06-10T00:32:50.134547+0000","last_undegraded":"2024-06-10T00:32:50.134547+0000","last_fullsized":"2024-06-10T00:32:50.134547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:57:54.469544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921940+0000","last_change":"2024-06-10T00:28:48.707540+0000","last_active":"2024-06-10T00:32:49.921940+0000","last_peered":"2024-06-10T00:32:49.921940+0000","last_clean":"2024-06-10T00:32:49.921940+0000","last_became_active":"2024-06-10T00:28:48.707143+0000","last_became_peered":"2024-06-10T00:28:48.707143+0000","last_unstale":"2024-06-10T00:32:49.921940+0000","last_undegraded":"2024-06-10T00:32:49.921940+0000","last_fullsized":"2024-06-10T00:32:49.921940+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:20:29.344808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'8","reported_seq":244,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133792+0000","last_change":"2024-06-10T00:28:34.397496+0000","last_active":"2024-06-10T00:32:50.133792+0000","last_peered":"2024-06-10T00:32:50.133792+0000","last_clean":"2024-06-10T00:32:50.133792+0000","last_became_active":"2024-06-10T00:28:34.397124+0000","last_became_peered":"2024-06-10T00:28:34.397124+0000","last_unstale":"2024-06-10T00:32:50.133792+0000","last_undegraded":"2024-06-10T00:32:50.133792+0000","last_fullsized":"2024-06-10T00:32:50.133792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:41:48.609354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134324+0000","last_change":"2024-06-10T00:28:37.658338+0000","last_active":"2024-06-10T00:32:50.134324+0000","last_peered":"2024-06-10T00:32:50.134324+0000","last_clean":"2024-06-10T00:32:50.134324+0000","last_became_active":"2024-06-10T00:28:37.655910+0000","last_became_peered":"2024-06-10T00:28:37.655910+0000","last_unstale":"2024-06-10T00:32:50.134324+0000","last_undegraded":"2024-06-10T00:32:50.134324+0000","last_fullsized":"2024-06-10T00:32:50.134324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:38.501335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133960+0000","last_change":"2024-06-10T00:28:48.711151+0000","last_active":"2024-06-10T00:32:50.133960+0000","last_peered":"2024-06-10T00:32:50.133960+0000","last_clean":"2024-06-10T00:32:50.133960+0000","last_became_active":"2024-06-10T00:28:48.710768+0000","last_became_peered":"2024-06-10T00:28:48.710768+0000","last_unstale":"2024-06-10T00:32:50.133960+0000","last_undegraded":"2024-06-10T00:32:50.133960+0000","last_fullsized":"2024-06-10T00:32:50.133960+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:15:22.791839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":65,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031568+0000","last_change":"2024-06-10T00:28:34.445812+0000","last_active":"2024-06-10T00:32:50.031568+0000","last_peered":"2024-06-10T00:32:50.031568+0000","last_clean":"2024-06-10T00:32:50.031568+0000","last_became_active":"2024-06-10T00:28:34.445472+0000","last_became_peered":"2024-06-10T00:28:34.445472+0000","last_unstale":"2024-06-10T00:32:50.031568+0000","last_undegraded":"2024-06-10T00:32:50.031568+0000","last_fullsized":"2024-06-10T00:32:50.031568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:00.912050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038769+0000","last_change":"2024-06-10T00:28:37.655345+0000","last_active":"2024-06-10T00:32:50.038769+0000","last_peered":"2024-06-10T00:32:50.038769+0000","last_clean":"2024-06-10T00:32:50.038769+0000","last_became_active":"2024-06-10T00:28:37.654954+0000","last_became_peered":"2024-06-10T00:28:37.654954+0000","last_unstale":"2024-06-10T00:32:50.038769+0000","last_undegraded":"2024-06-10T00:32:50.038769+0000","last_fullsized":"2024-06-10T00:32:50.038769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:39:27.403006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038741+0000","last_change":"2024-06-10T00:28:48.705247+0000","last_active":"2024-06-10T00:32:50.038741+0000","last_peered":"2024-06-10T00:32:50.038741+0000","last_clean":"2024-06-10T00:32:50.038741+0000","last_became_active":"2024-06-10T00:28:48.704822+0000","last_became_peered":"2024-06-10T00:28:48.704822+0000","last_unstale":"2024-06-10T00:32:50.038741+0000","last_undegraded":"2024-06-10T00:32:50.038741+0000","last_fullsized":"2024-06-10T00:32:50.038741+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:24:59.991491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032099+0000","last_change":"2024-06-10T00:28:34.396792+0000","last_active":"2024-06-10T00:32:50.032099+0000","last_peered":"2024-06-10T00:32:50.032099+0000","last_clean":"2024-06-10T00:32:50.032099+0000","last_became_active":"2024-06-10T00:28:34.396442+0000","last_became_peered":"2024-06-10T00:28:34.396442+0000","last_unstale":"2024-06-10T00:32:50.032099+0000","last_undegraded":"2024-06-10T00:32:50.032099+0000","last_fullsized":"2024-06-10T00:32:50.032099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:46.436198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199145+0000","last_change":"2024-06-10T00:28:37.657698+0000","last_active":"2024-06-10T00:32:50.199145+0000","last_peered":"2024-06-10T00:32:50.199145+0000","last_clean":"2024-06-10T00:32:50.199145+0000","last_became_active":"2024-06-10T00:28:37.657364+0000","last_became_peered":"2024-06-10T00:28:37.657364+0000","last_unstale":"2024-06-10T00:32:50.199145+0000","last_undegraded":"2024-06-10T00:32:50.199145+0000","last_fullsized":"2024-06-10T00:32:50.199145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:23.097015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031580+0000","last_change":"2024-06-10T00:28:48.727567+0000","last_active":"2024-06-10T00:32:50.031580+0000","last_peered":"2024-06-10T00:32:50.031580+0000","last_clean":"2024-06-10T00:32:50.031580+0000","last_became_active":"2024-06-10T00:28:48.727108+0000","last_became_peered":"2024-06-10T00:28:48.727108+0000","last_unstale":"2024-06-10T00:32:50.031580+0000","last_undegraded":"2024-06-10T00:32:50.031580+0000","last_fullsized":"2024-06-10T00:32:50.031580+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:35:00.824177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921917+0000","last_change":"2024-06-10T00:28:34.446832+0000","last_active":"2024-06-10T00:32:49.921917+0000","last_peered":"2024-06-10T00:32:49.921917+0000","last_clean":"2024-06-10T00:32:49.921917+0000","last_became_active":"2024-06-10T00:28:34.446435+0000","last_became_peered":"2024-06-10T00:28:34.446435+0000","last_unstale":"2024-06-10T00:32:49.921917+0000","last_undegraded":"2024-06-10T00:32:49.921917+0000","last_fullsized":"2024-06-10T00:32:49.921917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:58.296674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042645+0000","last_change":"2024-06-10T00:30:24.229514+0000","last_active":"2024-06-10T00:32:50.042645+0000","last_peered":"2024-06-10T00:32:50.042645+0000","last_clean":"2024-06-10T00:32:50.042645+0000","last_became_active":"2024-06-10T00:30:24.229163+0000","last_became_peered":"2024-06-10T00:30:24.229163+0000","last_unstale":"2024-06-10T00:32:50.042645+0000","last_undegraded":"2024-06-10T00:32:50.042645+0000","last_fullsized":"2024-06-10T00:32:50.042645+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:52.914144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133741+0000","last_change":"2024-06-10T00:28:48.704683+0000","last_active":"2024-06-10T00:32:50.133741+0000","last_peered":"2024-06-10T00:32:50.133741+0000","last_clean":"2024-06-10T00:32:50.133741+0000","last_became_active":"2024-06-10T00:28:48.704297+0000","last_became_peered":"2024-06-10T00:28:48.704297+0000","last_unstale":"2024-06-10T00:32:50.133741+0000","last_undegraded":"2024-06-10T00:32:50.133741+0000","last_fullsized":"2024-06-10T00:32:50.133741+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:44.980544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'8","reported_seq":242,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188609+0000","last_change":"2024-06-10T00:28:34.393328+0000","last_active":"2024-06-10T00:32:50.188609+0000","last_peered":"2024-06-10T00:32:50.188609+0000","last_clean":"2024-06-10T00:32:50.188609+0000","last_became_active":"2024-06-10T00:28:34.391968+0000","last_became_peered":"2024-06-10T00:28:34.391968+0000","last_unstale":"2024-06-10T00:32:50.188609+0000","last_undegraded":"2024-06-10T00:32:50.188609+0000","last_fullsized":"2024-06-10T00:32:50.188609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:18.898180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1676,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":2,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042474+0000","last_change":"2024-06-10T00:28:37.652555+0000","last_active":"2024-06-10T00:32:50.042474+0000","last_peered":"2024-06-10T00:32:50.042474+0000","last_clean":"2024-06-10T00:32:50.042474+0000","last_became_active":"2024-06-10T00:28:37.651476+0000","last_became_peered":"2024-06-10T00:28:37.651476+0000","last_unstale":"2024-06-10T00:32:50.042474+0000","last_undegraded":"2024-06-10T00:32:50.042474+0000","last_fullsized":"2024-06-10T00:32:50.042474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:04:57.823224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134015+0000","last_change":"2024-06-10T00:28:48.727568+0000","last_active":"2024-06-10T00:32:50.134015+0000","last_peered":"2024-06-10T00:32:50.134015+0000","last_clean":"2024-06-10T00:32:50.134015+0000","last_became_active":"2024-06-10T00:28:48.727018+0000","last_became_peered":"2024-06-10T00:28:48.727018+0000","last_unstale":"2024-06-10T00:32:50.134015+0000","last_undegraded":"2024-06-10T00:32:50.134015+0000","last_fullsized":"2024-06-10T00:32:50.134015+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:02.025212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921381+0000","last_change":"2024-06-10T00:30:24.651947+0000","last_active":"2024-06-10T00:32:49.921381+0000","last_peered":"2024-06-10T00:32:49.921381+0000","last_clean":"2024-06-10T00:32:49.921381+0000","last_became_active":"2024-06-10T00:30:24.651640+0000","last_became_peered":"2024-06-10T00:30:24.651640+0000","last_unstale":"2024-06-10T00:32:49.921381+0000","last_undegraded":"2024-06-10T00:32:49.921381+0000","last_fullsized":"2024-06-10T00:32:49.921381+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:47.070568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"100'7","reported_seq":240,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.191611+0000","last_change":"2024-06-10T00:28:34.445501+0000","last_active":"2024-06-10T00:32:50.191611+0000","last_peered":"2024-06-10T00:32:50.191611+0000","last_clean":"2024-06-10T00:32:50.191611+0000","last_became_active":"2024-06-10T00:28:34.445128+0000","last_became_peered":"2024-06-10T00:28:34.445128+0000","last_unstale":"2024-06-10T00:32:50.191611+0000","last_undegraded":"2024-06-10T00:32:50.191611+0000","last_fullsized":"2024-06-10T00:32:50.191611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:59.231585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134703+0000","last_change":"2024-06-10T00:28:48.709852+0000","last_active":"2024-06-10T00:32:50.134703+0000","last_peered":"2024-06-10T00:32:50.134703+0000","last_clean":"2024-06-10T00:32:50.134703+0000","last_became_active":"2024-06-10T00:28:48.709423+0000","last_became_peered":"2024-06-10T00:28:48.709423+0000","last_unstale":"2024-06-10T00:32:50.134703+0000","last_undegraded":"2024-06-10T00:32:50.134703+0000","last_fullsized":"2024-06-10T00:32:50.134703+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:42.123563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031914+0000","last_change":"2024-06-10T00:28:37.657423+0000","last_active":"2024-06-10T00:32:50.031914+0000","last_peered":"2024-06-10T00:32:50.031914+0000","last_clean":"2024-06-10T00:32:50.031914+0000","last_became_active":"2024-06-10T00:28:37.657140+0000","last_became_peered":"2024-06-10T00:28:37.657140+0000","last_unstale":"2024-06-10T00:32:50.031914+0000","last_undegraded":"2024-06-10T00:32:50.031914+0000","last_fullsized":"2024-06-10T00:32:50.031914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:20:08.943066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922033+0000","last_change":"2024-06-10T00:28:34.392585+0000","last_active":"2024-06-10T00:32:49.922033+0000","last_peered":"2024-06-10T00:32:49.922033+0000","last_clean":"2024-06-10T00:32:49.922033+0000","last_became_active":"2024-06-10T00:28:34.392229+0000","last_became_peered":"2024-06-10T00:28:34.392229+0000","last_unstale":"2024-06-10T00:32:49.922033+0000","last_undegraded":"2024-06-10T00:32:49.922033+0000","last_fullsized":"2024-06-10T00:32:49.922033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:11.275761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":380,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134207+0000","last_change":"2024-06-10T00:27:12.072577+0000","last_active":"2024-06-10T00:32:50.134207+0000","last_peered":"2024-06-10T00:32:50.134207+0000","last_clean":"2024-06-10T00:32:50.134207+0000","last_became_active":"2024-06-10T00:27:12.072193+0000","last_became_peered":"2024-06-10T00:27:12.072193+0000","last_unstale":"2024-06-10T00:32:50.134207+0000","last_undegraded":"2024-06-10T00:32:50.134207+0000","last_fullsized":"2024-06-10T00:32:50.134207+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_clean_scrub_stamp":"2024-06-10T00:23:46.826438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:57.782124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039093+0000","last_change":"2024-06-10T00:28:48.727913+0000","last_active":"2024-06-10T00:32:50.039093+0000","last_peered":"2024-06-10T00:32:50.039093+0000","last_clean":"2024-06-10T00:32:50.039093+0000","last_became_active":"2024-06-10T00:28:48.727402+0000","last_became_peered":"2024-06-10T00:28:48.727402+0000","last_unstale":"2024-06-10T00:32:50.039093+0000","last_undegraded":"2024-06-10T00:32:50.039093+0000","last_fullsized":"2024-06-10T00:32:50.039093+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:10:25.051296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132861+0000","last_change":"2024-06-10T00:28:37.659901+0000","last_active":"2024-06-10T00:32:50.132861+0000","last_peered":"2024-06-10T00:32:50.132861+0000","last_clean":"2024-06-10T00:32:50.132861+0000","last_became_active":"2024-06-10T00:28:37.659553+0000","last_became_peered":"2024-06-10T00:28:37.659553+0000","last_unstale":"2024-06-10T00:32:50.132861+0000","last_undegraded":"2024-06-10T00:32:50.132861+0000","last_fullsized":"2024-06-10T00:32:50.132861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:25.694823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134306+0000","last_change":"2024-06-10T00:28:34.398155+0000","last_active":"2024-06-10T00:32:50.134306+0000","last_peered":"2024-06-10T00:32:50.134306+0000","last_clean":"2024-06-10T00:32:50.134306+0000","last_became_active":"2024-06-10T00:28:34.397793+0000","last_became_peered":"2024-06-10T00:28:34.397793+0000","last_unstale":"2024-06-10T00:32:50.134306+0000","last_undegraded":"2024-06-10T00:32:50.134306+0000","last_fullsized":"2024-06-10T00:32:50.134306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:21.188139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198902+0000","last_change":"2024-06-10T00:28:48.709736+0000","last_active":"2024-06-10T00:32:50.198902+0000","last_peered":"2024-06-10T00:32:50.198902+0000","last_clean":"2024-06-10T00:32:50.198902+0000","last_became_active":"2024-06-10T00:28:48.709293+0000","last_became_peered":"2024-06-10T00:28:48.709293+0000","last_unstale":"2024-06-10T00:32:50.198902+0000","last_undegraded":"2024-06-10T00:32:50.198902+0000","last_fullsized":"2024-06-10T00:32:50.198902+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:30:04.840656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199015+0000","last_change":"2024-06-10T00:29:23.780030+0000","last_active":"2024-06-10T00:32:50.199015+0000","last_peered":"2024-06-10T00:32:50.199015+0000","last_clean":"2024-06-10T00:32:50.199015+0000","last_became_active":"2024-06-10T00:29:23.779589+0000","last_became_peered":"2024-06-10T00:29:23.779589+0000","last_unstale":"2024-06-10T00:32:50.199015+0000","last_undegraded":"2024-06-10T00:32:50.199015+0000","last_fullsized":"2024-06-10T00:32:50.199015+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:53.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133881+0000","last_change":"2024-06-10T00:28:37.658330+0000","last_active":"2024-06-10T00:32:50.133881+0000","last_peered":"2024-06-10T00:32:50.133881+0000","last_clean":"2024-06-10T00:32:50.133881+0000","last_became_active":"2024-06-10T00:28:37.657343+0000","last_became_peered":"2024-06-10T00:28:37.657343+0000","last_unstale":"2024-06-10T00:32:50.133881+0000","last_undegraded":"2024-06-10T00:32:50.133881+0000","last_fullsized":"2024-06-10T00:32:50.133881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:31:52.167402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031759+0000","last_change":"2024-06-10T00:28:48.727506+0000","last_active":"2024-06-10T00:32:50.031759+0000","last_peered":"2024-06-10T00:32:50.031759+0000","last_clean":"2024-06-10T00:32:50.031759+0000","last_became_active":"2024-06-10T00:28:48.727068+0000","last_became_peered":"2024-06-10T00:28:48.727068+0000","last_unstale":"2024-06-10T00:32:50.031759+0000","last_undegraded":"2024-06-10T00:32:50.031759+0000","last_fullsized":"2024-06-10T00:32:50.031759+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:42.147908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'51","reported_seq":121,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.148195+0000","last_change":"2024-06-10T00:28:34.399872+0000","last_active":"2024-06-10T00:32:50.148195+0000","last_peered":"2024-06-10T00:32:50.148195+0000","last_clean":"2024-06-10T00:32:50.148195+0000","last_became_active":"2024-06-10T00:28:34.399493+0000","last_became_peered":"2024-06-10T00:28:34.399493+0000","last_unstale":"2024-06-10T00:32:50.148195+0000","last_undegraded":"2024-06-10T00:32:50.148195+0000","last_fullsized":"2024-06-10T00:32:50.148195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:24:10.018110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032414+0000","last_change":"2024-06-10T00:28:37.656597+0000","last_active":"2024-06-10T00:32:50.032414+0000","last_peered":"2024-06-10T00:32:50.032414+0000","last_clean":"2024-06-10T00:32:50.032414+0000","last_became_active":"2024-06-10T00:28:37.654356+0000","last_became_peered":"2024-06-10T00:28:37.654356+0000","last_unstale":"2024-06-10T00:32:50.032414+0000","last_undegraded":"2024-06-10T00:32:50.032414+0000","last_fullsized":"2024-06-10T00:32:50.032414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:07:12.950683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920721+0000","last_change":"2024-06-10T00:28:48.702686+0000","last_active":"2024-06-10T00:32:49.920721+0000","last_peered":"2024-06-10T00:32:49.920721+0000","last_clean":"2024-06-10T00:32:49.920721+0000","last_became_active":"2024-06-10T00:28:48.702273+0000","last_became_peered":"2024-06-10T00:28:48.702273+0000","last_unstale":"2024-06-10T00:32:49.920721+0000","last_undegraded":"2024-06-10T00:32:49.920721+0000","last_fullsized":"2024-06-10T00:32:49.920721+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:00.078232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921732+0000","last_change":"2024-06-10T00:28:34.447549+0000","last_active":"2024-06-10T00:32:49.921732+0000","last_peered":"2024-06-10T00:32:49.921732+0000","last_clean":"2024-06-10T00:32:49.921732+0000","last_became_active":"2024-06-10T00:28:34.447197+0000","last_became_peered":"2024-06-10T00:28:34.447197+0000","last_unstale":"2024-06-10T00:32:49.921732+0000","last_undegraded":"2024-06-10T00:32:49.921732+0000","last_fullsized":"2024-06-10T00:32:49.921732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:47.525153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193431+0000","last_change":"2024-06-10T00:28:37.655585+0000","last_active":"2024-06-10T00:32:50.193431+0000","last_peered":"2024-06-10T00:32:50.193431+0000","last_clean":"2024-06-10T00:32:50.193431+0000","last_became_active":"2024-06-10T00:28:37.655225+0000","last_became_peered":"2024-06-10T00:28:37.655225+0000","last_unstale":"2024-06-10T00:32:50.193431+0000","last_undegraded":"2024-06-10T00:32:50.193431+0000","last_fullsized":"2024-06-10T00:32:50.193431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:57:25.184924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921606+0000","last_change":"2024-06-10T00:29:23.857447+0000","last_active":"2024-06-10T00:32:49.921606+0000","last_peered":"2024-06-10T00:32:49.921606+0000","last_clean":"2024-06-10T00:32:49.921606+0000","last_became_active":"2024-06-10T00:29:23.856943+0000","last_became_peered":"2024-06-10T00:29:23.856943+0000","last_unstale":"2024-06-10T00:32:49.921606+0000","last_undegraded":"2024-06-10T00:32:49.921606+0000","last_fullsized":"2024-06-10T00:32:49.921606+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:58:44.548818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188309+0000","last_change":"2024-06-10T00:28:34.445637+0000","last_active":"2024-06-10T00:32:50.188309+0000","last_peered":"2024-06-10T00:32:50.188309+0000","last_clean":"2024-06-10T00:32:50.188309+0000","last_became_active":"2024-06-10T00:28:34.445324+0000","last_became_peered":"2024-06-10T00:28:34.445324+0000","last_unstale":"2024-06-10T00:32:50.188309+0000","last_undegraded":"2024-06-10T00:32:50.188309+0000","last_fullsized":"2024-06-10T00:32:50.188309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:05.713557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031570+0000","last_change":"2024-06-10T00:28:37.659037+0000","last_active":"2024-06-10T00:32:50.031570+0000","last_peered":"2024-06-10T00:32:50.031570+0000","last_clean":"2024-06-10T00:32:50.031570+0000","last_became_active":"2024-06-10T00:28:37.658705+0000","last_became_peered":"2024-06-10T00:28:37.658705+0000","last_unstale":"2024-06-10T00:32:50.031570+0000","last_undegraded":"2024-06-10T00:32:50.031570+0000","last_fullsized":"2024-06-10T00:32:50.031570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:20:52.797237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922067+0000","last_change":"2024-06-10T00:28:48.704130+0000","last_active":"2024-06-10T00:32:49.922067+0000","last_peered":"2024-06-10T00:32:49.922067+0000","last_clean":"2024-06-10T00:32:49.922067+0000","last_became_active":"2024-06-10T00:28:48.703738+0000","last_became_peered":"2024-06-10T00:28:48.703738+0000","last_unstale":"2024-06-10T00:32:49.922067+0000","last_undegraded":"2024-06-10T00:32:49.922067+0000","last_fullsized":"2024-06-10T00:32:49.922067+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:01:40.436710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032028+0000","last_change":"2024-06-10T00:28:34.386667+0000","last_active":"2024-06-10T00:32:50.032028+0000","last_peered":"2024-06-10T00:32:50.032028+0000","last_clean":"2024-06-10T00:32:50.032028+0000","last_became_active":"2024-06-10T00:28:34.386363+0000","last_became_peered":"2024-06-10T00:28:34.386363+0000","last_unstale":"2024-06-10T00:32:50.032028+0000","last_undegraded":"2024-06-10T00:32:50.032028+0000","last_fullsized":"2024-06-10T00:32:50.032028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:11.850162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198745+0000","last_change":"2024-06-10T00:28:37.664754+0000","last_active":"2024-06-10T00:32:50.198745+0000","last_peered":"2024-06-10T00:32:50.198745+0000","last_clean":"2024-06-10T00:32:50.198745+0000","last_became_active":"2024-06-10T00:28:37.658574+0000","last_became_peered":"2024-06-10T00:28:37.658574+0000","last_unstale":"2024-06-10T00:32:50.198745+0000","last_undegraded":"2024-06-10T00:32:50.198745+0000","last_fullsized":"2024-06-10T00:32:50.198745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:16:47.437822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921563+0000","last_change":"2024-06-10T00:28:48.706560+0000","last_active":"2024-06-10T00:32:49.921563+0000","last_peered":"2024-06-10T00:32:49.921563+0000","last_clean":"2024-06-10T00:32:49.921563+0000","last_became_active":"2024-06-10T00:28:48.705986+0000","last_became_peered":"2024-06-10T00:28:48.705986+0000","last_unstale":"2024-06-10T00:32:49.921563+0000","last_undegraded":"2024-06-10T00:32:49.921563+0000","last_fullsized":"2024-06-10T00:32:49.921563+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:21:26.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921015+0000","last_change":"2024-06-10T00:28:34.394509+0000","last_active":"2024-06-10T00:32:49.921015+0000","last_peered":"2024-06-10T00:32:49.921015+0000","last_clean":"2024-06-10T00:32:49.921015+0000","last_became_active":"2024-06-10T00:28:34.394105+0000","last_became_peered":"2024-06-10T00:28:34.394105+0000","last_unstale":"2024-06-10T00:32:49.921015+0000","last_undegraded":"2024-06-10T00:32:49.921015+0000","last_fullsized":"2024-06-10T00:32:49.921015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:13.415544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032271+0000","last_change":"2024-06-10T00:28:37.653192+0000","last_active":"2024-06-10T00:32:50.032271+0000","last_peered":"2024-06-10T00:32:50.032271+0000","last_clean":"2024-06-10T00:32:50.032271+0000","last_became_active":"2024-06-10T00:28:37.652868+0000","last_became_peered":"2024-06-10T00:28:37.652868+0000","last_unstale":"2024-06-10T00:32:50.032271+0000","last_undegraded":"2024-06-10T00:32:50.032271+0000","last_fullsized":"2024-06-10T00:32:50.032271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:21.614915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921695+0000","last_change":"2024-06-10T00:28:48.727615+0000","last_active":"2024-06-10T00:32:49.921695+0000","last_peered":"2024-06-10T00:32:49.921695+0000","last_clean":"2024-06-10T00:32:49.921695+0000","last_became_active":"2024-06-10T00:28:48.727190+0000","last_became_peered":"2024-06-10T00:28:48.727190+0000","last_unstale":"2024-06-10T00:32:49.921695+0000","last_undegraded":"2024-06-10T00:32:49.921695+0000","last_fullsized":"2024-06-10T00:32:49.921695+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:57.498183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031723+0000","last_change":"2024-06-10T00:29:23.784131+0000","last_active":"2024-06-10T00:32:50.031723+0000","last_peered":"2024-06-10T00:32:50.031723+0000","last_clean":"2024-06-10T00:32:50.031723+0000","last_became_active":"2024-06-10T00:29:23.780909+0000","last_became_peered":"2024-06-10T00:29:23.780909+0000","last_unstale":"2024-06-10T00:32:50.031723+0000","last_undegraded":"2024-06-10T00:32:50.031723+0000","last_fullsized":"2024-06-10T00:32:50.031723+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.774217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134422+0000","last_change":"2024-06-10T00:28:37.660399+0000","last_active":"2024-06-10T00:32:50.134422+0000","last_peered":"2024-06-10T00:32:50.134422+0000","last_clean":"2024-06-10T00:32:50.134422+0000","last_became_active":"2024-06-10T00:28:37.660069+0000","last_became_peered":"2024-06-10T00:28:37.660069+0000","last_unstale":"2024-06-10T00:32:50.134422+0000","last_undegraded":"2024-06-10T00:32:50.134422+0000","last_fullsized":"2024-06-10T00:32:50.134422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:59.727883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188428+0000","last_change":"2024-06-10T00:28:48.713267+0000","last_active":"2024-06-10T00:32:50.188428+0000","last_peered":"2024-06-10T00:32:50.188428+0000","last_clean":"2024-06-10T00:32:50.188428+0000","last_became_active":"2024-06-10T00:28:48.712870+0000","last_became_peered":"2024-06-10T00:28:48.712870+0000","last_unstale":"2024-06-10T00:32:50.188428+0000","last_undegraded":"2024-06-10T00:32:50.188428+0000","last_fullsized":"2024-06-10T00:32:50.188428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:59.210101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'6","reported_seq":239,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.161235+0000","last_change":"2024-06-10T00:28:34.448652+0000","last_active":"2024-06-10T00:32:50.161235+0000","last_peered":"2024-06-10T00:32:50.161235+0000","last_clean":"2024-06-10T00:32:50.161235+0000","last_became_active":"2024-06-10T00:28:34.448243+0000","last_became_peered":"2024-06-10T00:28:34.448243+0000","last_unstale":"2024-06-10T00:32:50.161235+0000","last_undegraded":"2024-06-10T00:32:50.161235+0000","last_fullsized":"2024-06-10T00:32:50.161235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:51:40.003429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":187,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.135023+0000","last_change":"2024-06-10T00:28:37.662307+0000","last_active":"2024-06-10T00:32:50.135023+0000","last_peered":"2024-06-10T00:32:50.135023+0000","last_clean":"2024-06-10T00:32:50.135023+0000","last_became_active":"2024-06-10T00:28:37.661727+0000","last_became_peered":"2024-06-10T00:28:37.661727+0000","last_unstale":"2024-06-10T00:32:50.135023+0000","last_undegraded":"2024-06-10T00:32:50.135023+0000","last_fullsized":"2024-06-10T00:32:50.135023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:23.989429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192376+0000","last_change":"2024-06-10T00:29:23.857626+0000","last_active":"2024-06-10T00:32:50.192376+0000","last_peered":"2024-06-10T00:32:50.192376+0000","last_clean":"2024-06-10T00:32:50.192376+0000","last_became_active":"2024-06-10T00:29:23.857057+0000","last_became_peered":"2024-06-10T00:29:23.857057+0000","last_unstale":"2024-06-10T00:32:50.192376+0000","last_undegraded":"2024-06-10T00:32:50.192376+0000","last_fullsized":"2024-06-10T00:32:50.192376+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:25:10.340053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'41","reported_seq":95,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.145625+0000","last_change":"2024-06-10T00:28:34.399253+0000","last_active":"2024-06-10T00:32:50.145625+0000","last_peered":"2024-06-10T00:32:50.145625+0000","last_clean":"2024-06-10T00:32:50.145625+0000","last_became_active":"2024-06-10T00:28:34.397763+0000","last_became_peered":"2024-06-10T00:28:34.397763+0000","last_unstale":"2024-06-10T00:32:50.145625+0000","last_undegraded":"2024-06-10T00:32:50.145625+0000","last_fullsized":"2024-06-10T00:32:50.145625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:05:57.371625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134695+0000","last_change":"2024-06-10T00:28:37.658027+0000","last_active":"2024-06-10T00:32:50.134695+0000","last_peered":"2024-06-10T00:32:50.134695+0000","last_clean":"2024-06-10T00:32:50.134695+0000","last_became_active":"2024-06-10T00:28:37.655513+0000","last_became_peered":"2024-06-10T00:28:37.655513+0000","last_unstale":"2024-06-10T00:32:50.134695+0000","last_undegraded":"2024-06-10T00:32:50.134695+0000","last_fullsized":"2024-06-10T00:32:50.134695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:50:23.525855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038558+0000","last_change":"2024-06-10T00:28:48.706997+0000","last_active":"2024-06-10T00:32:50.038558+0000","last_peered":"2024-06-10T00:32:50.038558+0000","last_clean":"2024-06-10T00:32:50.038558+0000","last_became_active":"2024-06-10T00:28:48.706597+0000","last_became_peered":"2024-06-10T00:28:48.706597+0000","last_unstale":"2024-06-10T00:32:50.038558+0000","last_undegraded":"2024-06-10T00:32:50.038558+0000","last_fullsized":"2024-06-10T00:32:50.038558+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:16.385312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133886+0000","last_change":"2024-06-10T00:29:23.785774+0000","last_active":"2024-06-10T00:32:50.133886+0000","last_peered":"2024-06-10T00:32:50.133886+0000","last_clean":"2024-06-10T00:32:50.133886+0000","last_became_active":"2024-06-10T00:29:23.781768+0000","last_became_peered":"2024-06-10T00:29:23.781768+0000","last_unstale":"2024-06-10T00:32:50.133886+0000","last_undegraded":"2024-06-10T00:32:50.133886+0000","last_fullsized":"2024-06-10T00:32:50.133886+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:27.348264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134953+0000","last_change":"2024-06-10T00:28:37.657820+0000","last_active":"2024-06-10T00:32:50.134953+0000","last_peered":"2024-06-10T00:32:50.134953+0000","last_clean":"2024-06-10T00:32:50.134953+0000","last_became_active":"2024-06-10T00:28:37.657487+0000","last_became_peered":"2024-06-10T00:28:37.657487+0000","last_unstale":"2024-06-10T00:32:50.134953+0000","last_undegraded":"2024-06-10T00:32:50.134953+0000","last_fullsized":"2024-06-10T00:32:50.134953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:44:35.930401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198953+0000","last_change":"2024-06-10T00:28:37.664543+0000","last_active":"2024-06-10T00:32:50.198953+0000","last_peered":"2024-06-10T00:32:50.198953+0000","last_clean":"2024-06-10T00:32:50.198953+0000","last_became_active":"2024-06-10T00:28:37.659015+0000","last_became_peered":"2024-06-10T00:28:37.659015+0000","last_unstale":"2024-06-10T00:32:50.198953+0000","last_undegraded":"2024-06-10T00:32:50.198953+0000","last_fullsized":"2024-06-10T00:32:50.198953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:06.555472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921072+0000","last_change":"2024-06-10T00:28:37.659389+0000","last_active":"2024-06-10T00:32:49.921072+0000","last_peered":"2024-06-10T00:32:49.921072+0000","last_clean":"2024-06-10T00:32:49.921072+0000","last_became_active":"2024-06-10T00:28:37.658990+0000","last_became_peered":"2024-06-10T00:28:37.658990+0000","last_unstale":"2024-06-10T00:32:49.921072+0000","last_undegraded":"2024-06-10T00:32:49.921072+0000","last_fullsized":"2024-06-10T00:32:49.921072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:29.532779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920823+0000","last_change":"2024-06-10T00:28:37.655621+0000","last_active":"2024-06-10T00:32:49.920823+0000","last_peered":"2024-06-10T00:32:49.920823+0000","last_clean":"2024-06-10T00:32:49.920823+0000","last_became_active":"2024-06-10T00:28:37.653599+0000","last_became_peered":"2024-06-10T00:28:37.653599+0000","last_unstale":"2024-06-10T00:32:49.920823+0000","last_undegraded":"2024-06-10T00:32:49.920823+0000","last_fullsized":"2024-06-10T00:32:49.920823+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:22.165851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192846+0000","last_change":"2024-06-10T00:29:23.782727+0000","last_active":"2024-06-10T00:32:50.192846+0000","last_peered":"2024-06-10T00:32:50.192846+0000","last_clean":"2024-06-10T00:32:50.192846+0000","last_became_active":"2024-06-10T00:29:23.782309+0000","last_became_peered":"2024-06-10T00:29:23.782309+0000","last_unstale":"2024-06-10T00:32:50.192846+0000","last_undegraded":"2024-06-10T00:32:50.192846+0000","last_fullsized":"2024-06-10T00:32:50.192846+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:20.585955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133856+0000","last_change":"2024-06-10T00:28:37.646750+0000","last_active":"2024-06-10T00:32:50.133856+0000","last_peered":"2024-06-10T00:32:50.133856+0000","last_clean":"2024-06-10T00:32:50.133856+0000","last_became_active":"2024-06-10T00:28:37.646372+0000","last_became_peered":"2024-06-10T00:28:37.646372+0000","last_unstale":"2024-06-10T00:32:50.133856+0000","last_undegraded":"2024-06-10T00:32:50.133856+0000","last_fullsized":"2024-06-10T00:32:50.133856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:45.762956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199287+0000","last_change":"2024-06-10T00:28:37.662278+0000","last_active":"2024-06-10T00:32:50.199287+0000","last_peered":"2024-06-10T00:32:50.199287+0000","last_clean":"2024-06-10T00:32:50.199287+0000","last_became_active":"2024-06-10T00:28:37.661855+0000","last_became_peered":"2024-06-10T00:28:37.661855+0000","last_unstale":"2024-06-10T00:32:50.199287+0000","last_undegraded":"2024-06-10T00:32:50.199287+0000","last_fullsized":"2024-06-10T00:32:50.199287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:20.074208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921288+0000","last_change":"2024-06-10T00:28:37.656738+0000","last_active":"2024-06-10T00:32:49.921288+0000","last_peered":"2024-06-10T00:32:49.921288+0000","last_clean":"2024-06-10T00:32:49.921288+0000","last_became_active":"2024-06-10T00:28:37.656375+0000","last_became_peered":"2024-06-10T00:28:37.656375+0000","last_unstale":"2024-06-10T00:32:49.921288+0000","last_undegraded":"2024-06-10T00:32:49.921288+0000","last_fullsized":"2024-06-10T00:32:49.921288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:49:17.779670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032555+0000","last_change":"2024-06-10T00:28:37.656849+0000","last_active":"2024-06-10T00:32:50.032555+0000","last_peered":"2024-06-10T00:32:50.032555+0000","last_clean":"2024-06-10T00:32:50.032555+0000","last_became_active":"2024-06-10T00:28:37.656261+0000","last_became_peered":"2024-06-10T00:28:37.656261+0000","last_unstale":"2024-06-10T00:32:50.032555+0000","last_undegraded":"2024-06-10T00:32:50.032555+0000","last_fullsized":"2024-06-10T00:32:50.032555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:13.152165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038730+0000","last_change":"2024-06-10T00:28:37.648384+0000","last_active":"2024-06-10T00:32:50.038730+0000","last_peered":"2024-06-10T00:32:50.038730+0000","last_clean":"2024-06-10T00:32:50.038730+0000","last_became_active":"2024-06-10T00:28:37.648009+0000","last_became_peered":"2024-06-10T00:28:37.648009+0000","last_unstale":"2024-06-10T00:32:50.038730+0000","last_undegraded":"2024-06-10T00:32:50.038730+0000","last_fullsized":"2024-06-10T00:32:50.038730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:45:55.808932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032174+0000","last_change":"2024-06-10T00:28:37.655828+0000","last_active":"2024-06-10T00:32:50.032174+0000","last_peered":"2024-06-10T00:32:50.032174+0000","last_clean":"2024-06-10T00:32:50.032174+0000","last_became_active":"2024-06-10T00:28:37.655269+0000","last_became_peered":"2024-06-10T00:28:37.655269+0000","last_unstale":"2024-06-10T00:32:50.032174+0000","last_undegraded":"2024-06-10T00:32:50.032174+0000","last_fullsized":"2024-06-10T00:32:50.032174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:56.618469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188607+0000","last_change":"2024-06-10T00:28:37.663756+0000","last_active":"2024-06-10T00:32:50.188607+0000","last_peered":"2024-06-10T00:32:50.188607+0000","last_clean":"2024-06-10T00:32:50.188607+0000","last_became_active":"2024-06-10T00:28:37.663380+0000","last_became_peered":"2024-06-10T00:28:37.663380+0000","last_unstale":"2024-06-10T00:32:50.188607+0000","last_undegraded":"2024-06-10T00:32:50.188607+0000","last_fullsized":"2024-06-10T00:32:50.188607+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:45.534648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133127+0000","last_change":"2024-06-10T00:28:37.655246+0000","last_active":"2024-06-10T00:32:50.133127+0000","last_peered":"2024-06-10T00:32:50.133127+0000","last_clean":"2024-06-10T00:32:50.133127+0000","last_became_active":"2024-06-10T00:28:37.654605+0000","last_became_peered":"2024-06-10T00:28:37.654605+0000","last_unstale":"2024-06-10T00:32:50.133127+0000","last_undegraded":"2024-06-10T00:32:50.133127+0000","last_fullsized":"2024-06-10T00:32:50.133127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:15.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199118+0000","last_change":"2024-06-10T00:28:37.662485+0000","last_active":"2024-06-10T00:32:50.199118+0000","last_peered":"2024-06-10T00:32:50.199118+0000","last_clean":"2024-06-10T00:32:50.199118+0000","last_became_active":"2024-06-10T00:28:37.661900+0000","last_became_peered":"2024-06-10T00:28:37.661900+0000","last_unstale":"2024-06-10T00:32:50.199118+0000","last_undegraded":"2024-06-10T00:32:50.199118+0000","last_fullsized":"2024-06-10T00:32:50.199118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:30:03.210634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604313,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1988,"num_read_kb":1107,"num_write":386,"num_write_kb":1478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1812939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":311,"ondisk_log_size":311,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383746,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1175552,"data_stored":581073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645382,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28836,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716316,"statfs":{"total":95995035648,"available":95965507584,"internally_reserved":0,"allocated":1859584,"data_stored":1231040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808907,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1585152,"data_stored":977723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070546,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36864,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708288,"statfs":{"total":95995035648,"available":95957286912,"internally_reserved":0,"allocated":1560576,"data_stored":973221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332182,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1781760,"data_stored":1180215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495707,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":577003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36532,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708620,"statfs":{"total":95995035648,"available":95957626880,"internally_reserved":0,"allocated":1224704,"data_stored":631213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36500,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708652,"statfs":{"total":95995035648,"available":95957659648,"internally_reserved":0,"allocated":1187840,"data_stored":577761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36564,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708588,"statfs":{"total":95995035648,"available":95957594112,"internally_reserved":0,"allocated":1253376,"data_stored":640908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182506,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1703936,"data_stored":1100439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920869,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36484,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708668,"statfs":{"total":95995035648,"available":95957676032,"internally_reserved":0,"allocated":1171456,"data_stored":580910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757344,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28892,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716260,"statfs":{"total":95995035648,"available":95965450240,"internally_reserved":0,"allocated":1921024,"data_stored":1305298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":135641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":203622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:32:54.200 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:32:54.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-06-10T00:32:54.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: Unable to set osd_memory_target on smithi138 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T00:32:54.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-06-10T00:32:54.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: Unable to set osd_memory_target on smithi105 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-06-10T00:32:54.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:32:54.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:54 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-06-10T00:32:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: Unable to set osd_memory_target on smithi138 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T00:32:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-06-10T00:32:54.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: Unable to set osd_memory_target on smithi105 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-06-10T00:32:54.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:32:54.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:54 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.494 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:54.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-06-10T00:32:54.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: Unable to set osd_memory_target on smithi138 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T00:32:54.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:54.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-06-10T00:32:54.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: Unable to set osd_memory_target on smithi105 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-06-10T00:32:54.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:32:54.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:54.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:54 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:55 smithi062 ceph-mon[32390]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:32:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:55 smithi062 ceph-mon[32390]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:32:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:55 smithi062 ceph-mon[32390]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:32:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:55 smithi062 ceph-mon[32390]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-06-10T00:32:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:55 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:55.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:55 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:55.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:55 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:55.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:55 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:55.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:55 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:55.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:55 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:55.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:55 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:32:55.360 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:55 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:32:55.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:55 smithi138 ceph-mon[36549]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:32:55.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:55 smithi138 ceph-mon[36549]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:32:55.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:55 smithi138 ceph-mon[36549]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:32:55.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:55 smithi138 ceph-mon[36549]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-06-10T00:32:55.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:55 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:55.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:55 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:55.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:55 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:55.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:55 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:55.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:55 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:55.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:55 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:55.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:55 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:32:55.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:55 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:32:55.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:55 smithi105 ceph-mon[36321]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:32:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:55 smithi105 ceph-mon[36321]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:32:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:55 smithi105 ceph-mon[36321]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:32:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:55 smithi105 ceph-mon[36321]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-06-10T00:32:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:55 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:55 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:55 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:55 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:55 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:55 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:32:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:55 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:32:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:55 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:32:56.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi062 to 176.1M 2024-06-10T00:32:56.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: Unable to set osd_memory_target on smithi062 to 184742297: error parsing value: Value '184742297' is below minimum 939524096 2024-06-10T00:32:56.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: Updating smithi062:/etc/ceph/ceph.conf 2024-06-10T00:32:56.343 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: Updating smithi105:/etc/ceph/ceph.conf 2024-06-10T00:32:56.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: Updating smithi138:/etc/ceph/ceph.conf 2024-06-10T00:32:56.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: Updating smithi062:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:32:56.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: Updating smithi105:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:32:56.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: Updating smithi138:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:32:56.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:32:56.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:32:56.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:32:56.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.344 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:32:56.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:32:56.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:32:56.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:56 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi062 to 176.1M 2024-06-10T00:32:56.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: Unable to set osd_memory_target on smithi062 to 184742297: error parsing value: Value '184742297' is below minimum 939524096 2024-06-10T00:32:56.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: Updating smithi062:/etc/ceph/ceph.conf 2024-06-10T00:32:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: Updating smithi105:/etc/ceph/ceph.conf 2024-06-10T00:32:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: Updating smithi138:/etc/ceph/ceph.conf 2024-06-10T00:32:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: Updating smithi062:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:32:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: Updating smithi105:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:32:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: Updating smithi138:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:32:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:32:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:32:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:32:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:32:56.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:32:56.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:32:56.375 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:56 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.448+0000 7f35f8122700 1 -- 172.21.15.62:0/152201254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f006ac40 msgr2=0x7f35f006b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:56.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.448+0000 7f35f8122700 1 --2- 172.21.15.62:0/152201254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f006ac40 0x7f35f006b020 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009990 tx=0x7f35ec009ca0 comp rx=0 tx=0).stop 2024-06-10T00:32:56.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.449+0000 7f35f8122700 1 -- 172.21.15.62:0/152201254 shutdown_connections 2024-06-10T00:32:56.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.449+0000 7f35f8122700 1 --2- 172.21.15.62:0/152201254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f006ac40 0x7f35f006b020 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:56.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.449+0000 7f35f8122700 1 --2- 172.21.15.62:0/152201254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35f010eee0 0x7f35f01112d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:56.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.449+0000 7f35f8122700 1 --2- 172.21.15.62:0/152201254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35f006b560 0x7f35f010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:56.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.449+0000 7f35f8122700 1 -- 172.21.15.62:0/152201254 >> 172.21.15.62:0/152201254 conn(0x7f35f0078430 msgr2=0x7f35f0078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:56.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.449+0000 7f35f8122700 1 -- 172.21.15.62:0/152201254 shutdown_connections 2024-06-10T00:32:56.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.449+0000 7f35f8122700 1 -- 172.21.15.62:0/152201254 wait complete. 2024-06-10T00:32:56.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.449+0000 7f35f8122700 1 Processor -- start 2024-06-10T00:32:56.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.449+0000 7f35f8122700 1 -- start start 2024-06-10T00:32:56.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f8122700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35f006ac40 0x7f35f019af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:56.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f8122700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f006b560 0x7f35f019b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:56.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f8122700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35f010eee0 0x7f35f019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:56.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f8122700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0114a30 con 0x7f35f006b560 2024-06-10T00:32:56.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f8122700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f35f01148b0 con 0x7f35f006ac40 2024-06-10T00:32:56.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f8122700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0114bb0 con 0x7f35f010eee0 2024-06-10T00:32:56.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f56bd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f006b560 0x7f35f019b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:56.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f56bd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f006b560 0x7f35f019b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45432/0 (socket says 172.21.15.62:45432) 2024-06-10T00:32:56.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f56bd700 1 -- 172.21.15.62:0/2219480523 learned_addr learned my addr 172.21.15.62:0/2219480523 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:56.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f5ebe700 1 --2- 172.21.15.62:0/2219480523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35f006ac40 0x7f35f019af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f66bf700 1 --2- 172.21.15.62:0/2219480523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35f010eee0 0x7f35f019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f56bd700 1 -- 172.21.15.62:0/2219480523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35f010eee0 msgr2=0x7f35f019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f56bd700 1 --2- 172.21.15.62:0/2219480523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35f010eee0 0x7f35f019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f56bd700 1 -- 172.21.15.62:0/2219480523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35f006ac40 msgr2=0x7f35f019af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f56bd700 1 --2- 172.21.15.62:0/2219480523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35f006ac40 0x7f35f019af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f56bd700 1 -- 172.21.15.62:0/2219480523 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f019ff50 con 0x7f35f006b560 2024-06-10T00:32:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f56bd700 1 --2- 172.21.15.62:0/2219480523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f006b560 0x7f35f019b480 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f35e000d980 tx=0x7f35e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35e6ffd700 1 -- 172.21.15.62:0/2219480523 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0016070 con 0x7f35f006b560 2024-06-10T00:32:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35e6ffd700 1 -- 172.21.15.62:0/2219480523 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e0004d10 con 0x7f35f006b560 2024-06-10T00:32:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f8122700 1 -- 172.21.15.62:0/2219480523 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f01a0240 con 0x7f35f006b560 2024-06-10T00:32:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35e6ffd700 1 -- 172.21.15.62:0/2219480523 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e00055f0 con 0x7f35f006b560 2024-06-10T00:32:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.450+0000 7f35f8122700 1 -- 172.21.15.62:0/2219480523 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f01a7ac0 con 0x7f35f006b560 2024-06-10T00:32:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.451+0000 7f35f8122700 1 -- 172.21.15.62:0/2219480523 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f0069180 con 0x7f35f006b560 2024-06-10T00:32:56.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.452+0000 7f35e6ffd700 1 -- 172.21.15.62:0/2219480523 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f35e0004750 con 0x7f35f006b560 2024-06-10T00:32:56.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.453+0000 7f35e6ffd700 1 --2- 172.21.15.62:0/2219480523 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f35dc0a2ea0 0x7f35dc0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:56.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.453+0000 7f35f5ebe700 1 --2- 172.21.15.62:0/2219480523 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f35dc0a2ea0 0x7f35dc0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:56.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.453+0000 7f35e6ffd700 1 -- 172.21.15.62:0/2219480523 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f35e00cf750 con 0x7f35f006b560 2024-06-10T00:32:56.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.454+0000 7f35f5ebe700 1 --2- 172.21.15.62:0/2219480523 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f35dc0a2ea0 0x7f35dc0a5360 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f35ec006010 tx=0x7f35ec00b540 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:56.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.458+0000 7f35e6ffd700 1 -- 172.21.15.62:0/2219480523 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f35e001f020 con 0x7f35f006b560 2024-06-10T00:32:56.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi062 to 176.1M 2024-06-10T00:32:56.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: Unable to set osd_memory_target on smithi062 to 184742297: error parsing value: Value '184742297' is below minimum 939524096 2024-06-10T00:32:56.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: Updating smithi062:/etc/ceph/ceph.conf 2024-06-10T00:32:56.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: Updating smithi105:/etc/ceph/ceph.conf 2024-06-10T00:32:56.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: Updating smithi138:/etc/ceph/ceph.conf 2024-06-10T00:32:56.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: Updating smithi062:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:32:56.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: Updating smithi105:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:32:56.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: Updating smithi138:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:32:56.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:32:56.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:32:56.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:32:56.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:32:56.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:32:56.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:32:56.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:56 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:32:56.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.713+0000 7f35f8122700 1 -- 172.21.15.62:0/2219480523 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35f019c2d0 con 0x7f35f006b560 2024-06-10T00:32:56.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.713+0000 7f35e6ffd700 1 -- 172.21.15.62:0/2219480523 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f35f019c2d0 con 0x7f35f006b560 2024-06-10T00:32:56.714 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:32:56.714 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:32:47.783154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:32:56.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.716+0000 7f35f8122700 1 -- 172.21.15.62:0/2219480523 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f35dc0a2ea0 msgr2=0x7f35dc0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:56.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.716+0000 7f35f8122700 1 --2- 172.21.15.62:0/2219480523 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f35dc0a2ea0 0x7f35dc0a5360 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f35ec006010 tx=0x7f35ec00b540 comp rx=0 tx=0).stop 2024-06-10T00:32:56.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.716+0000 7f35f8122700 1 -- 172.21.15.62:0/2219480523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f006b560 msgr2=0x7f35f019b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:56.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.716+0000 7f35f8122700 1 --2- 172.21.15.62:0/2219480523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f006b560 0x7f35f019b480 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f35e000d980 tx=0x7f35e000dc90 comp rx=0 tx=0).stop 2024-06-10T00:32:56.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.716+0000 7f35f8122700 1 -- 172.21.15.62:0/2219480523 shutdown_connections 2024-06-10T00:32:56.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.716+0000 7f35f8122700 1 --2- 172.21.15.62:0/2219480523 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f35dc0a2ea0 0x7f35dc0a5360 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:56.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.716+0000 7f35f8122700 1 --2- 172.21.15.62:0/2219480523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35f010eee0 0x7f35f019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:56.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.716+0000 7f35f8122700 1 --2- 172.21.15.62:0/2219480523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35f006ac40 0x7f35f019af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:56.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.716+0000 7f35f8122700 1 --2- 172.21.15.62:0/2219480523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f006b560 0x7f35f019b480 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:56.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.716+0000 7f35f8122700 1 -- 172.21.15.62:0/2219480523 >> 172.21.15.62:0/2219480523 conn(0x7f35f0078430 msgr2=0x7f35f0110330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:56.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.716+0000 7f35f8122700 1 -- 172.21.15.62:0/2219480523 shutdown_connections 2024-06-10T00:32:56.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:56.716+0000 7f35f8122700 1 -- 172.21.15.62:0/2219480523 wait complete. 2024-06-10T00:32:56.717 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-06-10T00:32:57.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:57 smithi062 ceph-mon[32390]: Updating smithi062:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:32:57.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:57 smithi062 ceph-mon[32390]: Updating smithi138:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:32:57.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:57 smithi062 ceph-mon[32390]: Updating smithi105:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:32:57.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:57 smithi062 ceph-mon[32390]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6 op/s 2024-06-10T00:32:57.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:57 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2219480523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:32:57.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:57 smithi138 ceph-mon[36549]: Updating smithi062:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:32:57.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:57 smithi138 ceph-mon[36549]: Updating smithi138:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:32:57.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:57 smithi138 ceph-mon[36549]: Updating smithi105:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:32:57.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:57 smithi138 ceph-mon[36549]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6 op/s 2024-06-10T00:32:57.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:57 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2219480523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:32:57.531 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:32:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:57 smithi105 ceph-mon[36321]: Updating smithi062:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:32:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:57 smithi105 ceph-mon[36321]: Updating smithi138:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:32:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:57 smithi105 ceph-mon[36321]: Updating smithi105:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:32:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:57 smithi105 ceph-mon[36321]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6 op/s 2024-06-10T00:32:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:57 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2219480523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:32:57.785 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:32:59.231 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:32:59 smithi062 ceph-mon[32390]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 0 B/s wr, 25 op/s 2024-06-10T00:32:59.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:32:59 smithi138 ceph-mon[36549]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 0 B/s wr, 25 op/s 2024-06-10T00:32:59.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.471+0000 7f155e082700 1 -- 172.21.15.62:0/1884470427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15581066f0 msgr2=0x7f155810e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:59.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.471+0000 7f155e082700 1 --2- 172.21.15.62:0/1884470427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15581066f0 0x7f155810e2c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f15500099c0 tx=0x7f1550009cd0 comp rx=0 tx=0).stop 2024-06-10T00:32:59.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.471+0000 7f155e082700 1 -- 172.21.15.62:0/1884470427 shutdown_connections 2024-06-10T00:32:59.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.471+0000 7f155e082700 1 --2- 172.21.15.62:0/1884470427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15581066f0 0x7f155810e2c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:59.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.471+0000 7f155e082700 1 --2- 172.21.15.62:0/1884470427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1558101610 0x7f15581019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:59.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.471+0000 7f155e082700 1 --2- 172.21.15.62:0/1884470427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1558101fc0 0x7f1558106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:59.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.471+0000 7f155e082700 1 -- 172.21.15.62:0/1884470427 >> 172.21.15.62:0/1884470427 conn(0x7f1558078430 msgr2=0x7f1558078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:59.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.471+0000 7f155e082700 1 -- 172.21.15.62:0/1884470427 shutdown_connections 2024-06-10T00:32:59.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.471+0000 7f155e082700 1 -- 172.21.15.62:0/1884470427 wait complete. 2024-06-10T00:32:59.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.472+0000 7f155e082700 1 Processor -- start 2024-06-10T00:32:59.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.472+0000 7f155e082700 1 -- start start 2024-06-10T00:32:59.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.472+0000 7f155e082700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1558101610 0x7f155819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:59.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.472+0000 7f155e082700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1558101fc0 0x7f155819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:59.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.472+0000 7f155e082700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15581066f0 0x7f1558198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:59.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.472+0000 7f155e082700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f15581117a0 con 0x7f1558101fc0 2024-06-10T00:32:59.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.472+0000 7f155e082700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1558111620 con 0x7f15581066f0 2024-06-10T00:32:59.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.472+0000 7f155e082700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1558111920 con 0x7f1558101610 2024-06-10T00:32:59.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.472+0000 7f1556ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1558101fc0 0x7f155819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:59.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.472+0000 7f1556ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1558101fc0 0x7f155819ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48046/0 (socket says 172.21.15.62:48046) 2024-06-10T00:32:59.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.472+0000 7f1556ffd700 1 -- 172.21.15.62:0/3921878541 learned_addr learned my addr 172.21.15.62:0/3921878541 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:32:59.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.472+0000 7f15577fe700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1558101610 0x7f155819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:59.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.473+0000 7f1557fff700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15581066f0 0x7f1558198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:59.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.473+0000 7f1556ffd700 1 -- 172.21.15.62:0/3921878541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1558101610 msgr2=0x7f155819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:59.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.473+0000 7f1556ffd700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1558101610 0x7f155819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:59.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.473+0000 7f1556ffd700 1 -- 172.21.15.62:0/3921878541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15581066f0 msgr2=0x7f1558198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:59.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.473+0000 7f1556ffd700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15581066f0 0x7f1558198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:59.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.473+0000 7f1556ffd700 1 -- 172.21.15.62:0/3921878541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1558199210 con 0x7f1558101fc0 2024-06-10T00:32:59.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.473+0000 7f1557fff700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15581066f0 0x7f1558198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:32:59.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.473+0000 7f1556ffd700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1558101fc0 0x7f155819ede0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f154000c9e0 tx=0x7f154000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:59.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.473+0000 7f1554ff9700 1 -- 172.21.15.62:0/3921878541 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540007aa0 con 0x7f1558101fc0 2024-06-10T00:32:59.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.473+0000 7f155e082700 1 -- 172.21.15.62:0/3921878541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1558199500 con 0x7f1558101fc0 2024-06-10T00:32:59.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.473+0000 7f1554ff9700 1 -- 172.21.15.62:0/3921878541 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1540007c40 con 0x7f1558101fc0 2024-06-10T00:32:59.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.473+0000 7f1554ff9700 1 -- 172.21.15.62:0/3921878541 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540018660 con 0x7f1558101fc0 2024-06-10T00:32:59.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.473+0000 7f155e082700 1 -- 172.21.15.62:0/3921878541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15581a5770 con 0x7f1558101fc0 2024-06-10T00:32:59.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.474+0000 7f155e082700 1 -- 172.21.15.62:0/3921878541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1558069180 con 0x7f1558101fc0 2024-06-10T00:32:59.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.475+0000 7f1554ff9700 1 -- 172.21.15.62:0/3921878541 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f15400188c0 con 0x7f1558101fc0 2024-06-10T00:32:59.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.476+0000 7f1554ff9700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f15440a2e20 0x7f15440a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:32:59.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.476+0000 7f15577fe700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f15440a2e20 0x7f15440a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:32:59.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.476+0000 7f1554ff9700 1 -- 172.21.15.62:0/3921878541 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f15400cfbe0 con 0x7f1558101fc0 2024-06-10T00:32:59.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.476+0000 7f15577fe700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f15440a2e20 0x7f15440a52e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1558078b60 tx=0x7f154800d040 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:32:59.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.480+0000 7f1554ff9700 1 -- 172.21.15.62:0/3921878541 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1540014030 con 0x7f1558101fc0 2024-06-10T00:32:59.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:32:59 smithi105 ceph-mon[36321]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 0 B/s wr, 25 op/s 2024-06-10T00:32:59.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.686+0000 7f155e082700 1 -- 172.21.15.62:0/3921878541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f155802dca0 con 0x7f1558101fc0 2024-06-10T00:32:59.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.687+0000 7f1554ff9700 1 -- 172.21.15.62:0/3921878541 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+25218 (secure 0 0 0) 0x7f15400a2090 con 0x7f1558101fc0 2024-06-10T00:32:59.688 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:32:59.688 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":100,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:32:49.913791+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3602410963":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/1557827118":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/2350700567":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/3602410963":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/1086776708":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/2528347688":"2024-06-11T00:32:49.913741+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:32:59.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.689+0000 7f155e082700 1 -- 172.21.15.62:0/3921878541 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f15440a2e20 msgr2=0x7f15440a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.689+0000 7f155e082700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f15440a2e20 0x7f15440a52e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1558078b60 tx=0x7f154800d040 comp rx=0 tx=0).stop 2024-06-10T00:32:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.689+0000 7f155e082700 1 -- 172.21.15.62:0/3921878541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1558101fc0 msgr2=0x7f155819ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:32:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.689+0000 7f155e082700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1558101fc0 0x7f155819ede0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f154000c9e0 tx=0x7f154000ccf0 comp rx=0 tx=0).stop 2024-06-10T00:32:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.689+0000 7f155e082700 1 -- 172.21.15.62:0/3921878541 shutdown_connections 2024-06-10T00:32:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.689+0000 7f155e082700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f15440a2e20 0x7f15440a52e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.689+0000 7f155e082700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15581066f0 0x7f1558198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.689+0000 7f155e082700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1558101610 0x7f155819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.689+0000 7f155e082700 1 --2- 172.21.15.62:0/3921878541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1558101fc0 0x7f155819ede0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:32:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.689+0000 7f155e082700 1 -- 172.21.15.62:0/3921878541 >> 172.21.15.62:0/3921878541 conn(0x7f1558078430 msgr2=0x7f155806ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:32:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.689+0000 7f155e082700 1 -- 172.21.15.62:0/3921878541 shutdown_connections 2024-06-10T00:32:59.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:32:59.689+0000 7f155e082700 1 -- 172.21.15.62:0/3921878541 wait complete. 2024-06-10T00:33:00.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:00 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3921878541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:33:00.486 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:00 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3921878541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:33:00.519 DEBUG:tasks.fwd_scrub:Yielding 2024-06-10T00:33:00.520 INFO:teuthology.run_tasks:Running task exec... 2024-06-10T00:33:00.529 INFO:teuthology.task.exec:Executing custom commands... 2024-06-10T00:33:00.529 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-06-10T00:33:00.529 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-06-10T00:33:00.530 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-06-10T00:33:00.530 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T00:33:00.530 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T00:33:00.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:00 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3921878541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:33:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.646+0000 7f2a134f5640 1 -- 172.21.15.62:0/3908314477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c103570 msgr2=0x7f2a0c10ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.646+0000 7f2a134f5640 1 --2- 172.21.15.62:0/3908314477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c103570 0x7f2a0c10ba50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2a08009900 tx=0x7f2a0802a480 comp rx=0 tx=0).stop 2024-06-10T00:33:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.646+0000 7f2a134f5640 1 -- 172.21.15.62:0/3908314477 shutdown_connections 2024-06-10T00:33:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.646+0000 7f2a134f5640 1 --2- 172.21.15.62:0/3908314477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a0c10bf90 0x7f2a0c10e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.646+0000 7f2a134f5640 1 --2- 172.21.15.62:0/3908314477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c103570 0x7f2a0c10ba50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.646+0000 7f2a134f5640 1 --2- 172.21.15.62:0/3908314477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a0c102c50 0x7f2a0c103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.646+0000 7f2a134f5640 1 -- 172.21.15.62:0/3908314477 >> 172.21.15.62:0/3908314477 conn(0x7f2a0c077a20 msgr2=0x7f2a0c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:00.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.646+0000 7f2a134f5640 1 -- 172.21.15.62:0/3908314477 shutdown_connections 2024-06-10T00:33:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.646+0000 7f2a134f5640 1 -- 172.21.15.62:0/3908314477 wait complete. 2024-06-10T00:33:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a134f5640 1 Processor -- start 2024-06-10T00:33:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a134f5640 1 -- start start 2024-06-10T00:33:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a134f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a0c102c50 0x7f2a0c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a134f5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a0c103570 0x7f2a0c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a134f5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c10bf90 0x7f2a0c19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a134f5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c111d60 con 0x7f2a0c10bf90 2024-06-10T00:33:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a134f5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c111be0 con 0x7f2a0c102c50 2024-06-10T00:33:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a134f5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c111ee0 con 0x7f2a0c103570 2024-06-10T00:33:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a11a92640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c10bf90 0x7f2a0c19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a11a92640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c10bf90 0x7f2a0c19ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48070/0 (socket says 172.21.15.62:48070) 2024-06-10T00:33:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a11a92640 1 -- 172.21.15.62:0/3362602725 learned_addr learned my addr 172.21.15.62:0/3362602725 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a10a90640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a0c103570 0x7f2a0c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:00.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a11291640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a0c102c50 0x7f2a0c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:00.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a11a92640 1 -- 172.21.15.62:0/3362602725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a0c103570 msgr2=0x7f2a0c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:00.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a11a92640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a0c103570 0x7f2a0c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a11a92640 1 -- 172.21.15.62:0/3362602725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a0c102c50 msgr2=0x7f2a0c198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:00.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a11a92640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a0c102c50 0x7f2a0c198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a11a92640 1 -- 172.21.15.62:0/3362602725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c19d1c0 con 0x7f2a0c10bf90 2024-06-10T00:33:00.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.647+0000 7f2a10a90640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a0c103570 0x7f2a0c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:00.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.648+0000 7f2a11a92640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c10bf90 0x7f2a0c19ca90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400eaa0 tx=0x7f2a0400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:00.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.648+0000 7f2a027fc640 1 -- 172.21.15.62:0/3362602725 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0400cdb0 con 0x7f2a0c10bf90 2024-06-10T00:33:00.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.648+0000 7f2a027fc640 1 -- 172.21.15.62:0/3362602725 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a04005020 con 0x7f2a0c10bf90 2024-06-10T00:33:00.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.648+0000 7f2a134f5640 1 -- 172.21.15.62:0/3362602725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c19d4b0 con 0x7f2a0c10bf90 2024-06-10T00:33:00.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.648+0000 7f2a027fc640 1 -- 172.21.15.62:0/3362602725 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a040116b0 con 0x7f2a0c10bf90 2024-06-10T00:33:00.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.648+0000 7f2a134f5640 1 -- 172.21.15.62:0/3362602725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c1a4d40 con 0x7f2a0c10bf90 2024-06-10T00:33:00.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.649+0000 7f2a134f5640 1 -- 172.21.15.62:0/3362602725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0c04f760 con 0x7f2a0c10bf90 2024-06-10T00:33:00.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.650+0000 7f2a027fc640 1 -- 172.21.15.62:0/3362602725 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f2a04005560 con 0x7f2a0c10bf90 2024-06-10T00:33:00.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.651+0000 7f2a027fc640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f29f80a2e40 0x7f29f80a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:00.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.651+0000 7f2a11291640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f29f80a2e40 0x7f29f80a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:00.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.651+0000 7f2a027fc640 1 -- 172.21.15.62:0/3362602725 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f2a040cfc60 con 0x7f2a0c10bf90 2024-06-10T00:33:00.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.651+0000 7f2a11291640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f29f80a2e40 0x7f29f80a5300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f29fc005fd0 tx=0x7f29fc00b590 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:00.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.655+0000 7f2a027fc640 1 -- 172.21.15.62:0/3362602725 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2a04016030 con 0x7f2a0c10bf90 2024-06-10T00:33:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:00.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.852+0000 7f2a134f5640 1 -- 172.21.15.62:0/3362602725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f2a0c02d2e0 con 0x7f2a0c10bf90 2024-06-10T00:33:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.856+0000 7f2a027fc640 1 -- 172.21.15.62:0/3362602725 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7f2a0409c360 con 0x7f2a0c10bf90 2024-06-10T00:33:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.859+0000 7f2a134f5640 1 -- 172.21.15.62:0/3362602725 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f29f80a2e40 msgr2=0x7f29f80a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.859+0000 7f2a134f5640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f29f80a2e40 0x7f29f80a5300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f29fc005fd0 tx=0x7f29fc00b590 comp rx=0 tx=0).stop 2024-06-10T00:33:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.859+0000 7f2a134f5640 1 -- 172.21.15.62:0/3362602725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c10bf90 msgr2=0x7f2a0c19ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.859+0000 7f2a134f5640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c10bf90 0x7f2a0c19ca90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400eaa0 tx=0x7f2a0400ef60 comp rx=0 tx=0).stop 2024-06-10T00:33:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.859+0000 7f2a134f5640 1 -- 172.21.15.62:0/3362602725 shutdown_connections 2024-06-10T00:33:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.859+0000 7f2a134f5640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f29f80a2e40 0x7f29f80a5300 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.859+0000 7f2a134f5640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a0c10bf90 0x7f2a0c19ca90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.859+0000 7f2a134f5640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a0c103570 0x7f2a0c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.859+0000 7f2a134f5640 1 --2- 172.21.15.62:0/3362602725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a0c102c50 0x7f2a0c198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.859+0000 7f2a134f5640 1 -- 172.21.15.62:0/3362602725 >> 172.21.15.62:0/3362602725 conn(0x7f2a0c077a20 msgr2=0x7f2a0c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.859+0000 7f2a134f5640 1 -- 172.21.15.62:0/3362602725 shutdown_connections 2024-06-10T00:33:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.859+0000 7f2a134f5640 1 -- 172.21.15.62:0/3362602725 wait complete. 2024-06-10T00:33:00.871 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-06-10T00:33:00.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.980+0000 7f16ca161640 1 -- 172.21.15.62:0/1715950239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c406ab70 msgr2=0x7f16c410e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:00.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.980+0000 7f16ca161640 1 --2- 172.21.15.62:0/1715950239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c406ab70 0x7f16c410e7e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009ca0 tx=0x7f16bc02a8c0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.980+0000 7f16ca161640 1 -- 172.21.15.62:0/1715950239 shutdown_connections 2024-06-10T00:33:00.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.980+0000 7f16ca161640 1 --2- 172.21.15.62:0/1715950239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c406ab70 0x7f16c410e7e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.980+0000 7f16ca161640 1 --2- 172.21.15.62:0/1715950239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c406a1d0 0x7f16c406a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.980+0000 7f16ca161640 1 --2- 172.21.15.62:0/1715950239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16c4107990 0x7f16c4107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.980+0000 7f16ca161640 1 -- 172.21.15.62:0/1715950239 >> 172.21.15.62:0/1715950239 conn(0x7f16c40766a0 msgr2=0x7f16c4076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:00.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.981+0000 7f16ca161640 1 -- 172.21.15.62:0/1715950239 shutdown_connections 2024-06-10T00:33:00.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.981+0000 7f16ca161640 1 -- 172.21.15.62:0/1715950239 wait complete. 2024-06-10T00:33:00.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.981+0000 7f16ca161640 1 Processor -- start 2024-06-10T00:33:00.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.981+0000 7f16ca161640 1 -- start start 2024-06-10T00:33:00.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.981+0000 7f16ca161640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c406a1d0 0x7f16c4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.981+0000 7f16ca161640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16c406ab70 0x7f16c4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16ca161640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4107990 0x7f16c419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16ca161640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4111dd0 con 0x7f16c406a1d0 2024-06-10T00:33:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16ca161640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4111c50 con 0x7f16c406ab70 2024-06-10T00:33:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16ca161640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4111f50 con 0x7f16c4107990 2024-06-10T00:33:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c406a1d0 0x7f16c4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c406a1d0 0x7f16c4198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48096/0 (socket says 172.21.15.62:48096) 2024-06-10T00:33:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16c37fe640 1 -- 172.21.15.62:0/3239139584 learned_addr learned my addr 172.21.15.62:0/3239139584 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16baffd640 1 --2- 172.21.15.62:0/3239139584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16c406ab70 0x7f16c4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16c3fff640 1 --2- 172.21.15.62:0/3239139584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4107990 0x7f16c419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16c37fe640 1 -- 172.21.15.62:0/3239139584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4107990 msgr2=0x7f16c419c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:00.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16c37fe640 1 --2- 172.21.15.62:0/3239139584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4107990 0x7f16c419c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16c37fe640 1 -- 172.21.15.62:0/3239139584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16c406ab70 msgr2=0x7f16c4198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:00.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16c37fe640 1 --2- 172.21.15.62:0/3239139584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16c406ab70 0x7f16c4198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:00.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16c37fe640 1 -- 172.21.15.62:0/3239139584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c419d100 con 0x7f16c406a1d0 2024-06-10T00:33:00.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16c37fe640 1 --2- 172.21.15.62:0/3239139584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c406a1d0 0x7f16c4198110 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f16b400e9f0 tx=0x7f16b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:00.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16c17fa640 1 -- 172.21.15.62:0/3239139584 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b400cd80 con 0x7f16c406a1d0 2024-06-10T00:33:00.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16c17fa640 1 -- 172.21.15.62:0/3239139584 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b4010910 con 0x7f16c406a1d0 2024-06-10T00:33:00.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16ca161640 1 -- 172.21.15.62:0/3239139584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c419d3f0 con 0x7f16c406a1d0 2024-06-10T00:33:00.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16c17fa640 1 -- 172.21.15.62:0/3239139584 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b40188f0 con 0x7f16c406a1d0 2024-06-10T00:33:00.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.982+0000 7f16ca161640 1 -- 172.21.15.62:0/3239139584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c41a4c80 con 0x7f16c406a1d0 2024-06-10T00:33:00.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.983+0000 7f16ca161640 1 -- 172.21.15.62:0/3239139584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c41030b0 con 0x7f16c406a1d0 2024-06-10T00:33:00.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.984+0000 7f16c17fa640 1 -- 172.21.15.62:0/3239139584 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f16b4010ac0 con 0x7f16c406a1d0 2024-06-10T00:33:00.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.985+0000 7f16c17fa640 1 --2- 172.21.15.62:0/3239139584 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f16b00a2e40 0x7f16b00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:00.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.985+0000 7f16baffd640 1 --2- 172.21.15.62:0/3239139584 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f16b00a2e40 0x7f16b00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:00.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.985+0000 7f16c17fa640 1 -- 172.21.15.62:0/3239139584 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f16b40cfce0 con 0x7f16c406a1d0 2024-06-10T00:33:00.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.986+0000 7f16baffd640 1 --2- 172.21.15.62:0/3239139584 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f16b00a2e40 0x7f16b00a5300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f16c4106f30 tx=0x7f16ac00b390 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:00.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:00.990+0000 7f16c17fa640 1 -- 172.21.15.62:0/3239139584 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f16b409bd90 con 0x7f16c406a1d0 2024-06-10T00:33:01.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.183+0000 7f16ca161640 1 -- 172.21.15.62:0/3239139584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f16c4109e30 con 0x7f16c406a1d0 2024-06-10T00:33:01.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.187+0000 7f16c17fa640 1 -- 172.21.15.62:0/3239139584 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f16b40a0090 con 0x7f16c406a1d0 2024-06-10T00:33:01.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.189+0000 7f16ca161640 1 -- 172.21.15.62:0/3239139584 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f16b00a2e40 msgr2=0x7f16b00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:01.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.189+0000 7f16ca161640 1 --2- 172.21.15.62:0/3239139584 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f16b00a2e40 0x7f16b00a5300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f16c4106f30 tx=0x7f16ac00b390 comp rx=0 tx=0).stop 2024-06-10T00:33:01.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.189+0000 7f16ca161640 1 -- 172.21.15.62:0/3239139584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c406a1d0 msgr2=0x7f16c4198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:01.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.189+0000 7f16ca161640 1 --2- 172.21.15.62:0/3239139584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c406a1d0 0x7f16c4198110 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f16b400e9f0 tx=0x7f16b400eeb0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.189+0000 7f16ca161640 1 -- 172.21.15.62:0/3239139584 shutdown_connections 2024-06-10T00:33:01.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.189+0000 7f16ca161640 1 --2- 172.21.15.62:0/3239139584 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f16b00a2e40 0x7f16b00a5300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.189+0000 7f16ca161640 1 --2- 172.21.15.62:0/3239139584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16c4107990 0x7f16c419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.189+0000 7f16ca161640 1 --2- 172.21.15.62:0/3239139584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16c406ab70 0x7f16c4198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.189+0000 7f16ca161640 1 --2- 172.21.15.62:0/3239139584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16c406a1d0 0x7f16c4198110 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.189+0000 7f16ca161640 1 -- 172.21.15.62:0/3239139584 >> 172.21.15.62:0/3239139584 conn(0x7f16c40766a0 msgr2=0x7f16c40ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:01.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.189+0000 7f16ca161640 1 -- 172.21.15.62:0/3239139584 shutdown_connections 2024-06-10T00:33:01.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.189+0000 7f16ca161640 1 -- 172.21.15.62:0/3239139584 wait complete. 2024-06-10T00:33:01.199 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:01 smithi062 ceph-mon[32390]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 0 B/s wr, 19 op/s 2024-06-10T00:33:01.199 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:01 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3362602725' entity='client.admin' 2024-06-10T00:33:01.199 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:01 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:33:01.199 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:01 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:33:01.199 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:01 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:33:01.200 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:01 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:33:01.201 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-06-10T00:33:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.321+0000 7fa749d7e640 1 -- 172.21.15.62:0/462880389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7441008d0 msgr2=0x7fa744100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.321+0000 7fa749d7e640 1 --2- 172.21.15.62:0/462880389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7441008d0 0x7fa744100d50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa738009cd0 tx=0x7fa73802a8e0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.321+0000 7fa749d7e640 1 -- 172.21.15.62:0/462880389 shutdown_connections 2024-06-10T00:33:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.321+0000 7fa749d7e640 1 --2- 172.21.15.62:0/462880389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa744101290 0x7fa74410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.321+0000 7fa749d7e640 1 --2- 172.21.15.62:0/462880389 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7441008d0 0x7fa744100d50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.321+0000 7fa749d7e640 1 --2- 172.21.15.62:0/462880389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa744106890 0x7fa744106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.321+0000 7fa749d7e640 1 -- 172.21.15.62:0/462880389 >> 172.21.15.62:0/462880389 conn(0x7fa744077a20 msgr2=0x7fa744077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.321+0000 7fa749d7e640 1 -- 172.21.15.62:0/462880389 shutdown_connections 2024-06-10T00:33:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.321+0000 7fa749d7e640 1 -- 172.21.15.62:0/462880389 wait complete. 2024-06-10T00:33:01.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.322+0000 7fa749d7e640 1 Processor -- start 2024-06-10T00:33:01.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.322+0000 7fa749d7e640 1 -- start start 2024-06-10T00:33:01.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.322+0000 7fa749d7e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7441008d0 0x7fa74410e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:01.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.322+0000 7fa749d7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa744101290 0x7fa74410e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:01.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.322+0000 7fa749d7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa744106890 0x7fa74410ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:01.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.322+0000 7fa749d7e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa744111da0 con 0x7fa744106890 2024-06-10T00:33:01.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.322+0000 7fa749d7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa744111c20 con 0x7fa744101290 2024-06-10T00:33:01.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.322+0000 7fa749d7e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa744111f20 con 0x7fa7441008d0 2024-06-10T00:33:01.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.322+0000 7fa743fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa744106890 0x7fa74410ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:01.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.322+0000 7fa743fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa744106890 0x7fa74410ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48116/0 (socket says 172.21.15.62:48116) 2024-06-10T00:33:01.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.322+0000 7fa743fff640 1 -- 172.21.15.62:0/936225693 learned_addr learned my addr 172.21.15.62:0/936225693 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:01.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.322+0000 7fa7437fe640 1 --2- 172.21.15.62:0/936225693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7441008d0 0x7fa74410e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:01.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.322+0000 7fa742ffd640 1 --2- 172.21.15.62:0/936225693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa744101290 0x7fa74410e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:01.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.323+0000 7fa743fff640 1 -- 172.21.15.62:0/936225693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7441008d0 msgr2=0x7fa74410e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:01.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.323+0000 7fa743fff640 1 --2- 172.21.15.62:0/936225693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7441008d0 0x7fa74410e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.323+0000 7fa743fff640 1 -- 172.21.15.62:0/936225693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa744101290 msgr2=0x7fa74410e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:01.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.323+0000 7fa743fff640 1 --2- 172.21.15.62:0/936225693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa744101290 0x7fa74410e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.323+0000 7fa743fff640 1 -- 172.21.15.62:0/936225693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7441a66c0 con 0x7fa744106890 2024-06-10T00:33:01.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.323+0000 7fa743fff640 1 --2- 172.21.15.62:0/936225693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa744106890 0x7fa74410ec60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00d750 tx=0x7fa73c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:01.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.323+0000 7fa740ff9640 1 -- 172.21.15.62:0/936225693 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c017070 con 0x7fa744106890 2024-06-10T00:33:01.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.323+0000 7fa740ff9640 1 -- 172.21.15.62:0/936225693 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa73c005500 con 0x7fa744106890 2024-06-10T00:33:01.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.323+0000 7fa749d7e640 1 -- 172.21.15.62:0/936225693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7441a69b0 con 0x7fa744106890 2024-06-10T00:33:01.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.323+0000 7fa740ff9640 1 -- 172.21.15.62:0/936225693 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c004d90 con 0x7fa744106890 2024-06-10T00:33:01.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.323+0000 7fa749d7e640 1 -- 172.21.15.62:0/936225693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7441a6e80 con 0x7fa744106890 2024-06-10T00:33:01.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.324+0000 7fa749d7e640 1 -- 172.21.15.62:0/936225693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa74404f760 con 0x7fa744106890 2024-06-10T00:33:01.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.325+0000 7fa740ff9640 1 -- 172.21.15.62:0/936225693 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fa73c004f30 con 0x7fa744106890 2024-06-10T00:33:01.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.326+0000 7fa740ff9640 1 --2- 172.21.15.62:0/936225693 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa72c0a2e40 0x7fa72c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:01.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.326+0000 7fa740ff9640 1 -- 172.21.15.62:0/936225693 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fa73c0cf330 con 0x7fa744106890 2024-06-10T00:33:01.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.326+0000 7fa7437fe640 1 --2- 172.21.15.62:0/936225693 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa72c0a2e40 0x7fa72c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:01.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.327+0000 7fa7437fe640 1 --2- 172.21.15.62:0/936225693 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa72c0a2e40 0x7fa72c0a5300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa734005d00 tx=0x7fa73400b410 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:01.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.331+0000 7fa740ff9640 1 -- 172.21.15.62:0/936225693 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa73c09b3e0 con 0x7fa744106890 2024-06-10T00:33:01.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:01 smithi138 ceph-mon[36549]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 0 B/s wr, 19 op/s 2024-06-10T00:33:01.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:01 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3362602725' entity='client.admin' 2024-06-10T00:33:01.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:01 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:33:01.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:01 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:33:01.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:01 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:33:01.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:01 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:33:01.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.521+0000 7fa749d7e640 1 -- 172.21.15.62:0/936225693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fa744108e10 con 0x7fa744106890 2024-06-10T00:33:01.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.527+0000 7fa740ff9640 1 -- 172.21.15.62:0/936225693 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7fa73c09f6e0 con 0x7fa744106890 2024-06-10T00:33:01.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.530+0000 7fa749d7e640 1 -- 172.21.15.62:0/936225693 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa72c0a2e40 msgr2=0x7fa72c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:01.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.530+0000 7fa749d7e640 1 --2- 172.21.15.62:0/936225693 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa72c0a2e40 0x7fa72c0a5300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa734005d00 tx=0x7fa73400b410 comp rx=0 tx=0).stop 2024-06-10T00:33:01.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.530+0000 7fa749d7e640 1 -- 172.21.15.62:0/936225693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa744106890 msgr2=0x7fa74410ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:01.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.530+0000 7fa749d7e640 1 --2- 172.21.15.62:0/936225693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa744106890 0x7fa74410ec60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00d750 tx=0x7fa73c00dc10 comp rx=0 tx=0).stop 2024-06-10T00:33:01.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.530+0000 7fa749d7e640 1 -- 172.21.15.62:0/936225693 shutdown_connections 2024-06-10T00:33:01.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.530+0000 7fa749d7e640 1 --2- 172.21.15.62:0/936225693 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa72c0a2e40 0x7fa72c0a5300 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.530+0000 7fa749d7e640 1 --2- 172.21.15.62:0/936225693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa744106890 0x7fa74410ec60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.530+0000 7fa749d7e640 1 --2- 172.21.15.62:0/936225693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa744101290 0x7fa74410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.530+0000 7fa749d7e640 1 --2- 172.21.15.62:0/936225693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7441008d0 0x7fa74410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.530+0000 7fa749d7e640 1 -- 172.21.15.62:0/936225693 >> 172.21.15.62:0/936225693 conn(0x7fa744077a20 msgr2=0x7fa7441033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:01.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.530+0000 7fa749d7e640 1 -- 172.21.15.62:0/936225693 shutdown_connections 2024-06-10T00:33:01.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.530+0000 7fa749d7e640 1 -- 172.21.15.62:0/936225693 wait complete. 2024-06-10T00:33:01.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:01 smithi105 ceph-mon[36321]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 0 B/s wr, 19 op/s 2024-06-10T00:33:01.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:01 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3362602725' entity='client.admin' 2024-06-10T00:33:01.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:01 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:33:01.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:01 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:33:01.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:01 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:33:01.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:01 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:33:01.541 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-06-10T00:33:01.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.651+0000 7f91d7ac4640 1 -- 172.21.15.62:0/268549569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d010bf70 msgr2=0x7f91d010e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:01.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.651+0000 7f91d7ac4640 1 --2- 172.21.15.62:0/268549569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d010bf70 0x7f91d010e410 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009ca0 tx=0x7f91c802a9e0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.651+0000 7f91d7ac4640 1 -- 172.21.15.62:0/268549569 shutdown_connections 2024-06-10T00:33:01.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.651+0000 7f91d7ac4640 1 --2- 172.21.15.62:0/268549569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d010bf70 0x7f91d010e410 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.651+0000 7f91d7ac4640 1 --2- 172.21.15.62:0/268549569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d0103550 0x7f91d010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.651+0000 7f91d7ac4640 1 --2- 172.21.15.62:0/268549569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d0102c30 0x7f91d0103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.651+0000 7f91d7ac4640 1 -- 172.21.15.62:0/268549569 >> 172.21.15.62:0/268549569 conn(0x7f91d0077a20 msgr2=0x7f91d0077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:01.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.651+0000 7f91d7ac4640 1 -- 172.21.15.62:0/268549569 shutdown_connections 2024-06-10T00:33:01.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.651+0000 7f91d7ac4640 1 -- 172.21.15.62:0/268549569 wait complete. 2024-06-10T00:33:01.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d7ac4640 1 Processor -- start 2024-06-10T00:33:01.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d7ac4640 1 -- start start 2024-06-10T00:33:01.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d7ac4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d0102c30 0x7f91d0100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:01.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d7ac4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d0103550 0x7f91d0100dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:01.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d7ac4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d010bf70 0x7f91d0101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:01.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d7ac4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0111d70 con 0x7f91d0103550 2024-06-10T00:33:01.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d7ac4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0111bf0 con 0x7f91d0102c30 2024-06-10T00:33:01.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d7ac4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0111ef0 con 0x7f91d010bf70 2024-06-10T00:33:01.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d505f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d0103550 0x7f91d0100dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:01.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d505f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d0103550 0x7f91d0100dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48122/0 (socket says 172.21.15.62:48122) 2024-06-10T00:33:01.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d505f640 1 -- 172.21.15.62:0/1691691616 learned_addr learned my addr 172.21.15.62:0/1691691616 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:01.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d5860640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d0102c30 0x7f91d0100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:01.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d6061640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d010bf70 0x7f91d0101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:01.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d505f640 1 -- 172.21.15.62:0/1691691616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d010bf70 msgr2=0x7f91d0101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:01.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d505f640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d010bf70 0x7f91d0101300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d505f640 1 -- 172.21.15.62:0/1691691616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d0102c30 msgr2=0x7f91d0100880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:01.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d505f640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d0102c30 0x7f91d0100880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d505f640 1 -- 172.21.15.62:0/1691691616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d01a6690 con 0x7f91d0103550 2024-06-10T00:33:01.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.652+0000 7f91d6061640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d010bf70 0x7f91d0101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:01.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.653+0000 7f91d505f640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d0103550 0x7f91d0100dc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f91c000e9f0 tx=0x7f91c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:01.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.653+0000 7f91c6ffd640 1 -- 172.21.15.62:0/1691691616 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c000cd80 con 0x7f91d0103550 2024-06-10T00:33:01.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.653+0000 7f91c6ffd640 1 -- 172.21.15.62:0/1691691616 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91c0010910 con 0x7f91d0103550 2024-06-10T00:33:01.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.653+0000 7f91c6ffd640 1 -- 172.21.15.62:0/1691691616 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0018990 con 0x7f91d0103550 2024-06-10T00:33:01.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.653+0000 7f91d7ac4640 1 -- 172.21.15.62:0/1691691616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d01a6980 con 0x7f91d0103550 2024-06-10T00:33:01.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.653+0000 7f91d7ac4640 1 -- 172.21.15.62:0/1691691616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d01a6f20 con 0x7f91d0103550 2024-06-10T00:33:01.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.654+0000 7f91d7ac4640 1 -- 172.21.15.62:0/1691691616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d004f760 con 0x7f91d0103550 2024-06-10T00:33:01.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.656+0000 7f91c6ffd640 1 -- 172.21.15.62:0/1691691616 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f91c0010ac0 con 0x7f91d0103550 2024-06-10T00:33:01.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.657+0000 7f91c6ffd640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f91bc0a2ec0 0x7f91bc0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:01.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.657+0000 7f91c6ffd640 1 -- 172.21.15.62:0/1691691616 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f91c00d0070 con 0x7f91d0103550 2024-06-10T00:33:01.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.657+0000 7f91d5860640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f91bc0a2ec0 0x7f91bc0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:01.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.658+0000 7f91d5860640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f91bc0a2ec0 0x7f91bc0a5380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009bd0 tx=0x7f91cc005f90 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:01.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.663+0000 7f91c6ffd640 1 -- 172.21.15.62:0/1691691616 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f91c009c770 con 0x7f91d0103550 2024-06-10T00:33:01.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.871+0000 7f91d7ac4640 1 -- 172.21.15.62:0/1691691616 --> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f91d00fe040 con 0x7f91bc0a2ec0 2024-06-10T00:33:01.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.935+0000 7f91c6ffd640 1 -- 172.21.15.62:0/1691691616 <== mgr.15333 v2:172.21.15.62:6800/518034449 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f91d00fe040 con 0x7f91bc0a2ec0 2024-06-10T00:33:01.936 INFO:teuthology.orchestra.run.smithi062.stdout:Schedule set for path / 2024-06-10T00:33:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.937+0000 7f91d7ac4640 1 -- 172.21.15.62:0/1691691616 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f91bc0a2ec0 msgr2=0x7f91bc0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.937+0000 7f91d7ac4640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f91bc0a2ec0 0x7f91bc0a5380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009bd0 tx=0x7f91cc005f90 comp rx=0 tx=0).stop 2024-06-10T00:33:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.937+0000 7f91d7ac4640 1 -- 172.21.15.62:0/1691691616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d0103550 msgr2=0x7f91d0100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.937+0000 7f91d7ac4640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d0103550 0x7f91d0100dc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f91c000e9f0 tx=0x7f91c000eeb0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.937+0000 7f91d7ac4640 1 -- 172.21.15.62:0/1691691616 shutdown_connections 2024-06-10T00:33:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.937+0000 7f91d7ac4640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f91bc0a2ec0 0x7f91bc0a5380 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.937+0000 7f91d7ac4640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d010bf70 0x7f91d0101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.937+0000 7f91d7ac4640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91d0103550 0x7f91d0100dc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.937+0000 7f91d7ac4640 1 --2- 172.21.15.62:0/1691691616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91d0102c30 0x7f91d0100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.937+0000 7f91d7ac4640 1 -- 172.21.15.62:0/1691691616 >> 172.21.15.62:0/1691691616 conn(0x7f91d0077a20 msgr2=0x7f91d00fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.938+0000 7f91d7ac4640 1 -- 172.21.15.62:0/1691691616 shutdown_connections 2024-06-10T00:33:01.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:01.938+0000 7f91d7ac4640 1 -- 172.21.15.62:0/1691691616 wait complete. 2024-06-10T00:33:01.948 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-06-10T00:33:02.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.057+0000 7efc47c6d640 1 -- 172.21.15.62:0/1597212905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc401019c0 msgr2=0x7efc40101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.057+0000 7efc47c6d640 1 --2- 172.21.15.62:0/1597212905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc401019c0 0x7efc40101e40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efc30009930 tx=0x7efc3002a420 comp rx=0 tx=0).stop 2024-06-10T00:33:02.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.057+0000 7efc47c6d640 1 -- 172.21.15.62:0/1597212905 shutdown_connections 2024-06-10T00:33:02.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.057+0000 7efc47c6d640 1 --2- 172.21.15.62:0/1597212905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc40102380 0x7efc4010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.057+0000 7efc47c6d640 1 --2- 172.21.15.62:0/1597212905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc401019c0 0x7efc40101e40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.057+0000 7efc47c6d640 1 --2- 172.21.15.62:0/1597212905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc401079d0 0x7efc40107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.057+0000 7efc47c6d640 1 -- 172.21.15.62:0/1597212905 >> 172.21.15.62:0/1597212905 conn(0x7efc400fd450 msgr2=0x7efc400ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:02.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.057+0000 7efc47c6d640 1 -- 172.21.15.62:0/1597212905 shutdown_connections 2024-06-10T00:33:02.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.057+0000 7efc47c6d640 1 -- 172.21.15.62:0/1597212905 wait complete. 2024-06-10T00:33:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.058+0000 7efc47c6d640 1 Processor -- start 2024-06-10T00:33:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.058+0000 7efc47c6d640 1 -- start start 2024-06-10T00:33:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.058+0000 7efc47c6d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc401019c0 0x7efc40198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.058+0000 7efc47c6d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc40102380 0x7efc401987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.058+0000 7efc47c6d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc401079d0 0x7efc4019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.058+0000 7efc47c6d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efc40111db0 con 0x7efc401019c0 2024-06-10T00:33:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.058+0000 7efc47c6d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efc40111c30 con 0x7efc40102380 2024-06-10T00:33:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.058+0000 7efc47c6d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc40111f30 con 0x7efc401079d0 2024-06-10T00:33:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.058+0000 7efc45a09640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc401019c0 0x7efc40198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.058+0000 7efc45a09640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc401019c0 0x7efc40198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48146/0 (socket says 172.21.15.62:48146) 2024-06-10T00:33:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.058+0000 7efc45a09640 1 -- 172.21.15.62:0/1413266133 learned_addr learned my addr 172.21.15.62:0/1413266133 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.058+0000 7efc4620a640 1 --2- 172.21.15.62:0/1413266133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc401079d0 0x7efc4019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.058+0000 7efc45208640 1 --2- 172.21.15.62:0/1413266133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc40102380 0x7efc401987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.059+0000 7efc45a09640 1 -- 172.21.15.62:0/1413266133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc401079d0 msgr2=0x7efc4019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.059+0000 7efc45a09640 1 --2- 172.21.15.62:0/1413266133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc401079d0 0x7efc4019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.059+0000 7efc45a09640 1 -- 172.21.15.62:0/1413266133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc40102380 msgr2=0x7efc401987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.059+0000 7efc45a09640 1 --2- 172.21.15.62:0/1413266133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc40102380 0x7efc401987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.059+0000 7efc45a09640 1 -- 172.21.15.62:0/1413266133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4019d230 con 0x7efc401019c0 2024-06-10T00:33:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.059+0000 7efc45a09640 1 --2- 172.21.15.62:0/1413266133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc401019c0 0x7efc40198290 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00d950 tx=0x7efc3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.059+0000 7efc36ffd640 1 -- 172.21.15.62:0/1413266133 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c017030 con 0x7efc401019c0 2024-06-10T00:33:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.059+0000 7efc36ffd640 1 -- 172.21.15.62:0/1413266133 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc3c004020 con 0x7efc401019c0 2024-06-10T00:33:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.059+0000 7efc36ffd640 1 -- 172.21.15.62:0/1413266133 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c0047a0 con 0x7efc401019c0 2024-06-10T00:33:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.059+0000 7efc47c6d640 1 -- 172.21.15.62:0/1413266133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4019d520 con 0x7efc401019c0 2024-06-10T00:33:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.059+0000 7efc47c6d640 1 -- 172.21.15.62:0/1413266133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc401a4db0 con 0x7efc401019c0 2024-06-10T00:33:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.060+0000 7efc47c6d640 1 -- 172.21.15.62:0/1413266133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4004f760 con 0x7efc401019c0 2024-06-10T00:33:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.061+0000 7efc36ffd640 1 -- 172.21.15.62:0/1413266133 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7efc3c0041d0 con 0x7efc401019c0 2024-06-10T00:33:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.062+0000 7efc36ffd640 1 --2- 172.21.15.62:0/1413266133 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7efc2c0a2e40 0x7efc2c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.062+0000 7efc45208640 1 --2- 172.21.15.62:0/1413266133 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7efc2c0a2e40 0x7efc2c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.062+0000 7efc36ffd640 1 -- 172.21.15.62:0/1413266133 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7efc3c0cf240 con 0x7efc401019c0 2024-06-10T00:33:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.063+0000 7efc45208640 1 --2- 172.21.15.62:0/1413266133 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7efc2c0a2e40 0x7efc2c0a5300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efc30009900 tx=0x7efc300058d0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:02.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.071+0000 7efc36ffd640 1 -- 172.21.15.62:0/1413266133 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efc3c09b940 con 0x7efc401019c0 2024-06-10T00:33:02.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:02 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3239139584' entity='client.admin' 2024-06-10T00:33:02.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:02 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:33:02.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:02 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:33:02.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:02 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:33:02.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:02 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:33:02.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:02 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/936225693' entity='client.admin' 2024-06-10T00:33:02.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:02 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:33:02.207 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:02 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:33:02.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:02 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:33:02.208 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:02 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:33:02.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.294+0000 7efc47c6d640 1 -- 172.21.15.62:0/1413266133 --> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7efc40061de0 con 0x7efc2c0a2e40 2024-06-10T00:33:02.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.355+0000 7efc36ffd640 1 -- 172.21.15.62:0/1413266133 <== mgr.15333 v2:172.21.15.62:6800/518034449 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7efc40061de0 con 0x7efc2c0a2e40 2024-06-10T00:33:02.356 INFO:teuthology.orchestra.run.smithi062.stdout:Retention added to path / 2024-06-10T00:33:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.357+0000 7efc47c6d640 1 -- 172.21.15.62:0/1413266133 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7efc2c0a2e40 msgr2=0x7efc2c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.357+0000 7efc47c6d640 1 --2- 172.21.15.62:0/1413266133 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7efc2c0a2e40 0x7efc2c0a5300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efc30009900 tx=0x7efc300058d0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.357+0000 7efc47c6d640 1 -- 172.21.15.62:0/1413266133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc401019c0 msgr2=0x7efc40198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.357+0000 7efc47c6d640 1 --2- 172.21.15.62:0/1413266133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc401019c0 0x7efc40198290 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00d950 tx=0x7efc3c00de10 comp rx=0 tx=0).stop 2024-06-10T00:33:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.358+0000 7efc47c6d640 1 -- 172.21.15.62:0/1413266133 shutdown_connections 2024-06-10T00:33:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.358+0000 7efc47c6d640 1 --2- 172.21.15.62:0/1413266133 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7efc2c0a2e40 0x7efc2c0a5300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.358+0000 7efc47c6d640 1 --2- 172.21.15.62:0/1413266133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc401079d0 0x7efc4019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.358+0000 7efc47c6d640 1 --2- 172.21.15.62:0/1413266133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc40102380 0x7efc401987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.358+0000 7efc47c6d640 1 --2- 172.21.15.62:0/1413266133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc401019c0 0x7efc40198290 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.358+0000 7efc47c6d640 1 -- 172.21.15.62:0/1413266133 >> 172.21.15.62:0/1413266133 conn(0x7efc400fd450 msgr2=0x7efc400fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:02.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.358+0000 7efc47c6d640 1 -- 172.21.15.62:0/1413266133 shutdown_connections 2024-06-10T00:33:02.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.358+0000 7efc47c6d640 1 -- 172.21.15.62:0/1413266133 wait complete. 2024-06-10T00:33:02.369 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-06-10T00:33:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.474+0000 7f2f3c91d640 1 -- 172.21.15.62:0/2123232892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f38106890 msgr2=0x7f2f38106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.474+0000 7f2f3c91d640 1 --2- 172.21.15.62:0/2123232892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f38106890 0x7f2f38106c70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009930 tx=0x7f2f3002a420 comp rx=0 tx=0).stop 2024-06-10T00:33:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.475+0000 7f2f3c91d640 1 -- 172.21.15.62:0/2123232892 shutdown_connections 2024-06-10T00:33:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.475+0000 7f2f3c91d640 1 --2- 172.21.15.62:0/2123232892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f38101290 0x7f2f3810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.475+0000 7f2f3c91d640 1 --2- 172.21.15.62:0/2123232892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f381008d0 0x7f2f38100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.475+0000 7f2f3c91d640 1 --2- 172.21.15.62:0/2123232892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f38106890 0x7f2f38106c70 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.475+0000 7f2f3c91d640 1 -- 172.21.15.62:0/2123232892 >> 172.21.15.62:0/2123232892 conn(0x7f2f38077a20 msgr2=0x7f2f38077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:02.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.475+0000 7f2f3c91d640 1 -- 172.21.15.62:0/2123232892 shutdown_connections 2024-06-10T00:33:02.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.475+0000 7f2f3c91d640 1 -- 172.21.15.62:0/2123232892 wait complete. 2024-06-10T00:33:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.475+0000 7f2f3c91d640 1 Processor -- start 2024-06-10T00:33:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.475+0000 7f2f3c91d640 1 -- start start 2024-06-10T00:33:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f3c91d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f381008d0 0x7f2f38198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f3c91d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f38101290 0x7f2f38198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f3c91d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f38106890 0x7f2f3819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f3c91d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2f38110c60 con 0x7f2f38106890 2024-06-10T00:33:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f3c91d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f38110ae0 con 0x7f2f38101290 2024-06-10T00:33:02.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f3c91d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f38110de0 con 0x7f2f381008d0 2024-06-10T00:33:02.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f3659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f381008d0 0x7f2f38198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f3659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f381008d0 0x7f2f38198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:57776/0 (socket says 172.21.15.62:57776) 2024-06-10T00:33:02.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f3659c640 1 -- 172.21.15.62:0/4125574345 learned_addr learned my addr 172.21.15.62:0/4125574345 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:02.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f36d9d640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f38106890 0x7f2f3819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f35d9b640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f38101290 0x7f2f38198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f3659c640 1 -- 172.21.15.62:0/4125574345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f38101290 msgr2=0x7f2f38198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f3659c640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f38101290 0x7f2f38198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f3659c640 1 -- 172.21.15.62:0/4125574345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f38106890 msgr2=0x7f2f3819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f3659c640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f38106890 0x7f2f3819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f3659c640 1 -- 172.21.15.62:0/4125574345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3819d1f0 con 0x7f2f381008d0 2024-06-10T00:33:02.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.476+0000 7f2f36d9d640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f38106890 0x7f2f3819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:02.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.477+0000 7f2f3659c640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f381008d0 0x7f2f38198200 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009230 tx=0x7f2f30004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:02.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.477+0000 7f2f2f7fe640 1 -- 172.21.15.62:0/4125574345 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3003f030 con 0x7f2f381008d0 2024-06-10T00:33:02.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.477+0000 7f2f2f7fe640 1 -- 172.21.15.62:0/4125574345 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f30003ad0 con 0x7f2f381008d0 2024-06-10T00:33:02.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.477+0000 7f2f3c91d640 1 -- 172.21.15.62:0/4125574345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3819d480 con 0x7f2f381008d0 2024-06-10T00:33:02.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.477+0000 7f2f2f7fe640 1 -- 172.21.15.62:0/4125574345 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30035070 con 0x7f2f381008d0 2024-06-10T00:33:02.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.477+0000 7f2f3c91d640 1 -- 172.21.15.62:0/4125574345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f381a4d70 con 0x7f2f381008d0 2024-06-10T00:33:02.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.478+0000 7f2f3c91d640 1 -- 172.21.15.62:0/4125574345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f3804f760 con 0x7f2f381008d0 2024-06-10T00:33:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.479+0000 7f2f2f7fe640 1 -- 172.21.15.62:0/4125574345 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f2f30042030 con 0x7f2f381008d0 2024-06-10T00:33:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.480+0000 7f2f2f7fe640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2f240a2ec0 0x7f2f240a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.481+0000 7f2f35d9b640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2f240a2ec0 0x7f2f240a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.481+0000 7f2f2f7fe640 1 -- 172.21.15.62:0/4125574345 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f2f300f0300 con 0x7f2f381008d0 2024-06-10T00:33:02.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.481+0000 7f2f35d9b640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2f240a2ec0 0x7f2f240a5380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2f38078100 tx=0x7f2f20005f90 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:02.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.484+0000 7f2f2f7fe640 1 -- 172.21.15.62:0/4125574345 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2f300bc350 con 0x7f2f381008d0 2024-06-10T00:33:02.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:02 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3239139584' entity='client.admin' 2024-06-10T00:33:02.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:02 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:33:02.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:02 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:33:02.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:02 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:33:02.507 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:02 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:33:02.508 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:02 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/936225693' entity='client.admin' 2024-06-10T00:33:02.508 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:02 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:33:02.508 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:02 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:33:02.508 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:02 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:33:02.508 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:02 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:33:02.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:02 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3239139584' entity='client.admin' 2024-06-10T00:33:02.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:02 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:33:02.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:02 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:33:02.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:02 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:33:02.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:02 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:33:02.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:02 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/936225693' entity='client.admin' 2024-06-10T00:33:02.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:02 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:33:02.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:02 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:33:02.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:02 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:33:02.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:02 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:33:02.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.589+0000 7f92672db700 1 -- 172.21.15.62:0/1189625651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f926006ab60 msgr2=0x7f926006afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.589+0000 7f92672db700 1 --2- 172.21.15.62:0/1189625651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f926006ab60 0x7f926006afe0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9254009990 tx=0x7f9254009ca0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.589+0000 7f92672db700 1 -- 172.21.15.62:0/1189625651 shutdown_connections 2024-06-10T00:33:02.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.589+0000 7f92672db700 1 --2- 172.21.15.62:0/1189625651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f926006b520 0x7f926010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.589+0000 7f92672db700 1 --2- 172.21.15.62:0/1189625651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9260108530 0x7f9260108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.589+0000 7f92672db700 1 --2- 172.21.15.62:0/1189625651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f926006ab60 0x7f926006afe0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.589+0000 7f92672db700 1 -- 172.21.15.62:0/1189625651 >> 172.21.15.62:0/1189625651 conn(0x7f92600783d0 msgr2=0x7f92600787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:02.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.589+0000 7f92672db700 1 -- 172.21.15.62:0/1189625651 shutdown_connections 2024-06-10T00:33:02.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.589+0000 7f92672db700 1 -- 172.21.15.62:0/1189625651 wait complete. 2024-06-10T00:33:02.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.590+0000 7f92672db700 1 Processor -- start 2024-06-10T00:33:02.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.590+0000 7f92672db700 1 -- start start 2024-06-10T00:33:02.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.590+0000 7f92672db700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f926006ab60 0x7f9260101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.590+0000 7f92672db700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926006b520 0x7f92601019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.590+0000 7f92672db700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9260108530 0x7f9260101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.590+0000 7f92672db700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9260112790 con 0x7f926006ab60 2024-06-10T00:33:02.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.590+0000 7f92672db700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9260112610 con 0x7f926006b520 2024-06-10T00:33:02.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.590+0000 7f92672db700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9260112910 con 0x7f9260108530 2024-06-10T00:33:02.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.590+0000 7f9265077700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f926006ab60 0x7f9260101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.591+0000 7f9265077700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f926006ab60 0x7f9260101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48162/0 (socket says 172.21.15.62:48162) 2024-06-10T00:33:02.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.591+0000 7f9265077700 1 -- 172.21.15.62:0/1388165233 learned_addr learned my addr 172.21.15.62:0/1388165233 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:02.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.591+0000 7f9265878700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9260108530 0x7f9260101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.591+0000 7f9264876700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926006b520 0x7f92601019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.591+0000 7f9265878700 1 -- 172.21.15.62:0/1388165233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926006b520 msgr2=0x7f92601019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.591+0000 7f9265878700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926006b520 0x7f92601019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.591+0000 7f9265878700 1 -- 172.21.15.62:0/1388165233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f926006ab60 msgr2=0x7f9260101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.591+0000 7f9265878700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f926006ab60 0x7f9260101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.591+0000 7f9265878700 1 -- 172.21.15.62:0/1388165233 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92601a71f0 con 0x7f9260108530 2024-06-10T00:33:02.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.591+0000 7f9265077700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f926006ab60 0x7f9260101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:02.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.591+0000 7f9265878700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9260108530 0x7f9260101f10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9258007f10 tx=0x7f925800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:02.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.592+0000 7f92527fc700 1 -- 172.21.15.62:0/1388165233 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258004db0 con 0x7f9260108530 2024-06-10T00:33:02.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.592+0000 7f92527fc700 1 -- 172.21.15.62:0/1388165233 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9258009e20 con 0x7f9260108530 2024-06-10T00:33:02.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.592+0000 7f92672db700 1 -- 172.21.15.62:0/1388165233 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92601a74e0 con 0x7f9260108530 2024-06-10T00:33:02.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.592+0000 7f92527fc700 1 -- 172.21.15.62:0/1388165233 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258015400 con 0x7f9260108530 2024-06-10T00:33:02.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.592+0000 7f92672db700 1 -- 172.21.15.62:0/1388165233 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92601a79f0 con 0x7f9260108530 2024-06-10T00:33:02.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.594+0000 7f92527fc700 1 -- 172.21.15.62:0/1388165233 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f925800db20 con 0x7f9260108530 2024-06-10T00:33:02.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.594+0000 7f92527fc700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f924c0a2ea0 0x7f924c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.595+0000 7f9265077700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f924c0a2ea0 0x7f924c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.595+0000 7f92527fc700 1 -- 172.21.15.62:0/1388165233 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f92580cf2a0 con 0x7f9260108530 2024-06-10T00:33:02.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.595+0000 7f9265077700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f924c0a2ea0 0x7f924c0a5360 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f925c005950 tx=0x7f925c00b500 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:02.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.596+0000 7f9271b5eb80 1 Processor -- start 2024-06-10T00:33:02.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.596+0000 7f9271b5eb80 1 -- start start 2024-06-10T00:33:02.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.596+0000 7f9271b5eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622de6c0 0x560d622deaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.596+0000 7f9271b5eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622d8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.596+0000 7f9271b5eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622d8fa0 0x560d622d93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.596+0000 7f9271b5eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560d6222a890 con 0x560d622defe0 2024-06-10T00:33:02.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.596+0000 7f9271b5eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560d621d27a0 con 0x560d622d8fa0 2024-06-10T00:33:02.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.596+0000 7f9271b5eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560d62226850 con 0x560d622de6c0 2024-06-10T00:33:02.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.596+0000 7f9264876700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622d8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.596+0000 7f9265077700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622de6c0 0x560d622deaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.596+0000 7f9264876700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622d8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48174/0 (socket says 172.21.15.62:48174) 2024-06-10T00:33:02.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.596+0000 7f9265077700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622de6c0 0x560d622deaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:57816/0 (socket says 172.21.15.62:57816) 2024-06-10T00:33:02.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.596+0000 7f9265077700 1 -- 172.21.15.62:0/3587797316 learned_addr learned my addr 172.21.15.62:0/3587797316 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:02.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.596+0000 7f9265878700 1 --2- 172.21.15.62:0/3587797316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622d8fa0 0x560d622d93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.597+0000 7f9265077700 1 -- 172.21.15.62:0/3587797316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622d8fa0 msgr2=0x560d622d93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:02.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.597+0000 7f9265077700 1 --2- 172.21.15.62:0/3587797316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622d8fa0 0x560d622d93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.597+0000 7f9265077700 1 -- 172.21.15.62:0/3587797316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 msgr2=0x560d622d8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:02.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.597+0000 7f9265077700 1 --2- 172.21.15.62:0/3587797316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622d8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.597+0000 7f9265077700 1 -- 172.21.15.62:0/3587797316 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d621c8fd0 con 0x560d622de6c0 2024-06-10T00:33:02.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.597+0000 7f9264876700 1 --2- 172.21.15.62:0/3587797316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622d8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:02.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.597+0000 7f9265077700 1 --2- 172.21.15.62:0/3587797316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622de6c0 0x560d622deaa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f925c011790 tx=0x7f925c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe3d83fbf855f1c4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:02.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.597+0000 7f924b7fe700 1 -- 172.21.15.62:0/3587797316 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c01e620 con 0x560d622de6c0 2024-06-10T00:33:02.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.597+0000 7f924b7fe700 1 -- 172.21.15.62:0/3587797316 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925c01eca0 con 0x560d622de6c0 2024-06-10T00:33:02.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.597+0000 7f924b7fe700 1 -- 172.21.15.62:0/3587797316 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c026db0 con 0x560d622de6c0 2024-06-10T00:33:02.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.598+0000 7f9271b5eb80 1 -- 172.21.15.62:0/3587797316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622de6c0 msgr2=0x560d622deaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.598+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/3587797316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622de6c0 0x560d622deaa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f925c011790 tx=0x7f925c013000 comp rx=0 tx=0).stop 2024-06-10T00:33:02.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.598+0000 7f9271b5eb80 1 -- 172.21.15.62:0/3587797316 shutdown_connections 2024-06-10T00:33:02.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.598+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/3587797316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622d8fa0 0x560d622d93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.598+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/3587797316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622de6c0 0x560d622deaa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.598+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/3587797316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622d8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.598+0000 7f9271b5eb80 1 -- 172.21.15.62:0/3587797316 >> 172.21.15.62:0/3587797316 conn(0x560d62203d40 msgr2=0x560d622d38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:02.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.598+0000 7f9271b5eb80 1 -- 172.21.15.62:0/3587797316 shutdown_connections 2024-06-10T00:33:02.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.598+0000 7f9271b5eb80 1 -- 172.21.15.62:0/3587797316 wait complete. 2024-06-10T00:33:02.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.598+0000 7f9271b5eb80 1 -- start start 2024-06-10T00:33:02.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.598+0000 7f9271b5eb80 1 Processor -- start 2024-06-10T00:33:02.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9271b5eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622d8fa0 0x560d622e4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9271b5eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622de6c0 0x560d622e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9271b5eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622defe0 0x560d622e25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9271b5eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560d622e5a50 con 0x560d622d8fa0 2024-06-10T00:33:02.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9271b5eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560d621cac70 con 0x560d622de6c0 2024-06-10T00:33:02.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9271b5eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560d622e5bd0 con 0x560d622defe0 2024-06-10T00:33:02.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9271b5eb80 10 client.0 _ll_register_callbacks cb 0x560d621c49a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:02.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9265077700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622d8fa0 0x560d622e4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9265077700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622d8fa0 0x560d622e4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48178/0 (socket says 172.21.15.62:48178) 2024-06-10T00:33:02.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9265077700 1 -- 172.21.15.62:0/1893678706 learned_addr learned my addr 172.21.15.62:0/1893678706 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:02.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9264876700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622de6c0 0x560d622e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9265878700 1 --2- 172.21.15.62:0/1893678706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622defe0 0x560d622e25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9265077700 1 -- 172.21.15.62:0/1893678706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622defe0 msgr2=0x560d622e25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:02.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9265077700 1 --2- 172.21.15.62:0/1893678706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622defe0 0x560d622e25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9265077700 1 -- 172.21.15.62:0/1893678706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622de6c0 msgr2=0x560d622e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:02.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9265077700 1 --2- 172.21.15.62:0/1893678706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622de6c0 0x560d622e2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.599+0000 7f9265077700 1 -- 172.21.15.62:0/1893678706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d62179950 con 0x560d622d8fa0 2024-06-10T00:33:02.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.600+0000 7f9265077700 1 --2- 172.21.15.62:0/1893678706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622d8fa0 0x560d622e4d90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f925c011760 tx=0x7f925c026630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2c7a7b27895b39 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:02.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.600+0000 7f9248ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:33:02.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.600+0000 7f9248ff9700 1 -- 172.21.15.62:0/1893678706 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c027d50 con 0x560d622d8fa0 2024-06-10T00:33:02.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.600+0000 7f9248ff9700 1 -- 172.21.15.62:0/1893678706 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f925c004020 con 0x560d622d8fa0 2024-06-10T00:33:02.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.600+0000 7f9271b5eb80 1 -- 172.21.15.62:0/1893678706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f925c01eca0 con 0x560d622d8fa0 2024-06-10T00:33:02.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.600+0000 7f9248ff9700 1 -- 172.21.15.62:0/1893678706 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f925c03e050 con 0x560d622d8fa0 2024-06-10T00:33:02.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.600+0000 7f9271b5eb80 10 client.15498 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:02.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.600+0000 7f9271b5eb80 1 -- 172.21.15.62:0/1893678706 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d622f0390 con 0x560d622d8fa0 2024-06-10T00:33:02.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.600+0000 7f9248ff9700 1 -- 172.21.15.62:0/1893678706 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925c01eca0 con 0x560d622d8fa0 2024-06-10T00:33:02.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.600+0000 7f9248ff9700 1 -- 172.21.15.62:0/1893678706 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f925c003680 con 0x560d622d8fa0 2024-06-10T00:33:02.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.600+0000 7f9271b5eb80 10 client.15498 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:02.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.600+0000 7f9271b5eb80 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:33:02.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.600+0000 7f9271b5eb80 20 client.15498 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:02.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.600+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/1893678706 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d622f2970 0x560d622f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.601+0000 7f9271b5eb80 4 client.15498 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:33:02.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.601+0000 7f9271b5eb80 1 -- 172.21.15.62:0/1893678706 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d62237d90 con 0x560d622f2970 2024-06-10T00:33:02.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.601+0000 7f9264876700 1 --2- 172.21.15.62:0/1893678706 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d622f2970 0x560d622f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.601+0000 7f9248ff9700 10 client.15498 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:02.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.601+0000 7f9264876700 1 --2- 172.21.15.62:0/1893678706 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d622f2970 0x560d622f4d60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd40bec8565b955e server_cookie=31102ce5f5a7621c in_seq=0 out_seq=0 2024-06-10T00:33:02.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9248ff9700 1 -- 172.21.15.62:0/1893678706 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560d62237d90 con 0x560d622f2970 2024-06-10T00:33:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9248ff9700 10 client.15498 handle_command_reply: tid=0 2024-06-10T00:33:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9271b5eb80 1 client.15498 shutdown 2024-06-10T00:33:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9271b5eb80 1 -- 172.21.15.62:0/1893678706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622d8fa0 msgr2=0x560d622e4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/1893678706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622d8fa0 0x560d622e4d90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f925c011760 tx=0x7f925c026630 comp rx=0 tx=0).stop 2024-06-10T00:33:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9271b5eb80 1 -- 172.21.15.62:0/1893678706 shutdown_connections 2024-06-10T00:33:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/1893678706 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d622f2970 0x560d622f4d60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/1893678706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622d8fa0 0x560d622e4d90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/1893678706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622defe0 0x560d622e25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/1893678706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622de6c0 0x560d622e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9271b5eb80 1 -- 172.21.15.62:0/1893678706 >> 172.21.15.62:0/1893678706 conn(0x560d62203d40 msgr2=0x560d622d7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9248ff9700 0 client.15498 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9271b5eb80 1 -- 172.21.15.62:0/1893678706 shutdown_connections 2024-06-10T00:33:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9271b5eb80 1 -- 172.21.15.62:0/1893678706 wait complete. 2024-06-10T00:33:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.602+0000 7f9271b5eb80 20 client.15498 trim_cache size 0 max 16384 2024-06-10T00:33:02.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.623+0000 7f9271b5eb80 1 Processor -- start 2024-06-10T00:33:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.623+0000 7f9271b5eb80 1 -- start start 2024-06-10T00:33:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.623+0000 7f9271b5eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622d8fa0 0x560d622d8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9271b5eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622de6c0 0x560d622cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9271b5eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622e38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9271b5eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f92601b07a0 con 0x560d622defe0 2024-06-10T00:33:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9271b5eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f92601b22e0 con 0x560d622d8fa0 2024-06-10T00:33:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9271b5eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92601b0e70 con 0x560d622de6c0 2024-06-10T00:33:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9265878700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622e38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9265878700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622e38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48188/0 (socket says 172.21.15.62:48188) 2024-06-10T00:33:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9265878700 1 -- 172.21.15.62:0/579375972 learned_addr learned my addr 172.21.15.62:0/579375972 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9264876700 1 --2- 172.21.15.62:0/579375972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622de6c0 0x560d622cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9265077700 1 --2- 172.21.15.62:0/579375972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622d8fa0 0x560d622d8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9265878700 1 -- 172.21.15.62:0/579375972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622de6c0 msgr2=0x560d622cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:02.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9265878700 1 --2- 172.21.15.62:0/579375972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622de6c0 0x560d622cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9265878700 1 -- 172.21.15.62:0/579375972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622d8fa0 msgr2=0x560d622d8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:02.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9265878700 1 --2- 172.21.15.62:0/579375972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622d8fa0 0x560d622d8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9265878700 1 -- 172.21.15.62:0/579375972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d622e0600 con 0x560d622defe0 2024-06-10T00:33:02.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f9265878700 1 --2- 172.21.15.62:0/579375972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622e38c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f92580cc610 tx=0x7f92580bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fabcd6bd1c869d4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:02.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.624+0000 7f924affd700 1 -- 172.21.15.62:0/579375972 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92580b4270 con 0x560d622defe0 2024-06-10T00:33:02.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.625+0000 7f924affd700 1 -- 172.21.15.62:0/579375972 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925809b2a0 con 0x560d622defe0 2024-06-10T00:33:02.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.625+0000 7f924affd700 1 -- 172.21.15.62:0/579375972 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92580cc830 con 0x560d622defe0 2024-06-10T00:33:02.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.625+0000 7f9271b5eb80 1 -- 172.21.15.62:0/579375972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 msgr2=0x560d622e38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.625+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/579375972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622e38c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f92580cc610 tx=0x7f92580bb040 comp rx=0 tx=0).stop 2024-06-10T00:33:02.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.625+0000 7f9271b5eb80 1 -- 172.21.15.62:0/579375972 shutdown_connections 2024-06-10T00:33:02.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.625+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/579375972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622e38c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.625+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/579375972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622d8fa0 0x560d622d8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.625+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/579375972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622de6c0 0x560d622cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.625+0000 7f9271b5eb80 1 -- 172.21.15.62:0/579375972 >> 172.21.15.62:0/579375972 conn(0x560d62203d40 msgr2=0x560d622dfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:02.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.625+0000 7f9271b5eb80 1 -- 172.21.15.62:0/579375972 shutdown_connections 2024-06-10T00:33:02.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.625+0000 7f9271b5eb80 1 -- 172.21.15.62:0/579375972 wait complete. 2024-06-10T00:33:02.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.626+0000 7f9271b5eb80 1 -- start start 2024-06-10T00:33:02.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.626+0000 7f9271b5eb80 1 Processor -- start 2024-06-10T00:33:02.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.626+0000 7f9271b5eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622d8fa0 0x560d622f5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.626+0000 7f9271b5eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622de6c0 0x560d622f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.626+0000 7f9271b5eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622f6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.626+0000 7f9271b5eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f92601b1c10 con 0x560d622defe0 2024-06-10T00:33:02.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.626+0000 7f9271b5eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f92601b1540 con 0x560d622de6c0 2024-06-10T00:33:02.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.626+0000 7f9271b5eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92601ae740 con 0x560d622d8fa0 2024-06-10T00:33:02.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.626+0000 7f9271b5eb80 10 client.0 _ll_register_callbacks cb 0x560d621c49a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:02.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.626+0000 7f9265077700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622d8fa0 0x560d622f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.626+0000 7f9265077700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622d8fa0 0x560d622f5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:57836/0 (socket says 172.21.15.62:57836) 2024-06-10T00:33:02.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.626+0000 7f9265077700 1 -- 172.21.15.62:0/2565539982 learned_addr learned my addr 172.21.15.62:0/2565539982 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:02.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.626+0000 7f9265878700 1 --2- 172.21.15.62:0/2565539982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.626+0000 7f9264876700 1 --2- 172.21.15.62:0/2565539982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622de6c0 0x560d622f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9265077700 1 -- 172.21.15.62:0/2565539982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622de6c0 msgr2=0x560d622f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9265077700 1 --2- 172.21.15.62:0/2565539982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622de6c0 0x560d622f5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9265077700 1 -- 172.21.15.62:0/2565539982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 msgr2=0x560d622f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9265077700 1 --2- 172.21.15.62:0/2565539982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9265077700 1 -- 172.21.15.62:0/2565539982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d6235c490 con 0x560d622d8fa0 2024-06-10T00:33:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9265878700 1 --2- 172.21.15.62:0/2565539982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9265077700 1 --2- 172.21.15.62:0/2565539982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622d8fa0 0x560d622f5990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f925c011730 tx=0x7f925c005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11a7f6980dbd63b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9248ff9700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:33:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9248ff9700 1 -- 172.21.15.62:0/2565539982 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c046020 con 0x560d622d8fa0 2024-06-10T00:33:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9248ff9700 1 -- 172.21.15.62:0/2565539982 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f925c0266a0 con 0x560d622d8fa0 2024-06-10T00:33:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9271b5eb80 1 -- 172.21.15.62:0/2565539982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f925809b2a0 con 0x560d622d8fa0 2024-06-10T00:33:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9248ff9700 1 -- 172.21.15.62:0/2565539982 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f925c051050 con 0x560d622d8fa0 2024-06-10T00:33:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9271b5eb80 10 client.25120 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9271b5eb80 1 -- 172.21.15.62:0/2565539982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d6235d7b0 con 0x560d622d8fa0 2024-06-10T00:33:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.627+0000 7f9248ff9700 1 -- 172.21.15.62:0/2565539982 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925809b2a0 con 0x560d622d8fa0 2024-06-10T00:33:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.628+0000 7f9248ff9700 1 -- 172.21.15.62:0/2565539982 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f925c0275e0 con 0x560d622d8fa0 2024-06-10T00:33:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.628+0000 7f9271b5eb80 10 client.25120 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.628+0000 7f9271b5eb80 10 client.25120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:33:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.628+0000 7f9271b5eb80 20 client.25120 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.628+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/2565539982 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d6235ddf0 0x560d62360210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.628+0000 7f9271b5eb80 4 client.25120 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T00:33:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.628+0000 7f9271b5eb80 1 -- 172.21.15.62:0/2565539982 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560d62232310 con 0x560d6235ddf0 2024-06-10T00:33:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.628+0000 7f9264876700 1 --2- 172.21.15.62:0/2565539982 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d6235ddf0 0x560d62360210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.628+0000 7f9264876700 1 --2- 172.21.15.62:0/2565539982 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d6235ddf0 0x560d62360210 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a88c68851214ff86 server_cookie=9314ccd928d8a423 in_seq=0 out_seq=0 2024-06-10T00:33:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.628+0000 7f9248ff9700 10 client.25120 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:02.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9248ff9700 1 -- 172.21.15.62:0/2565539982 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560d62232310 con 0x560d6235ddf0 2024-06-10T00:33:02.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9248ff9700 10 client.25120 handle_command_reply: tid=0 2024-06-10T00:33:02.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9271b5eb80 1 client.25120 shutdown 2024-06-10T00:33:02.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9271b5eb80 1 -- 172.21.15.62:0/2565539982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622d8fa0 msgr2=0x560d622f5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/2565539982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622d8fa0 0x560d622f5990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f925c011730 tx=0x7f925c005700 comp rx=0 tx=0).stop 2024-06-10T00:33:02.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9271b5eb80 1 -- 172.21.15.62:0/2565539982 shutdown_connections 2024-06-10T00:33:02.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/2565539982 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d6235ddf0 0x560d62360210 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/2565539982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d622defe0 0x560d622f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/2565539982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d622d8fa0 0x560d622f5990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9271b5eb80 1 --2- 172.21.15.62:0/2565539982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d622de6c0 0x560d622f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9271b5eb80 1 -- 172.21.15.62:0/2565539982 >> 172.21.15.62:0/2565539982 conn(0x560d62203d40 msgr2=0x560d622e38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:02.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9248ff9700 0 client.25120 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:02.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9271b5eb80 1 -- 172.21.15.62:0/2565539982 shutdown_connections 2024-06-10T00:33:02.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9271b5eb80 1 -- 172.21.15.62:0/2565539982 wait complete. 2024-06-10T00:33:02.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.630+0000 7f9271b5eb80 20 client.25120 trim_cache size 0 max 16384 2024-06-10T00:33:02.635 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:33:02.635 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-06-10T00:33:02.635 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "efdc32f2-c0f1-4917-8e4f-8f9bb83fb21d", 2024-06-10T00:33:02.635 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-06-10T00:33:02.635 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:33:02.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.631+0000 7f92672db700 1 -- 172.21.15.62:0/1388165233 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f924c0a2ea0 msgr2=0x7f924c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.631+0000 7f92672db700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f924c0a2ea0 0x7f924c0a5360 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f925c005950 tx=0x7f925c00b500 comp rx=0 tx=0).stop 2024-06-10T00:33:02.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.631+0000 7f92672db700 1 -- 172.21.15.62:0/1388165233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9260108530 msgr2=0x7f9260101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.631+0000 7f92672db700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9260108530 0x7f9260101f10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9258007f10 tx=0x7f925800d3b0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.631+0000 7f92672db700 1 -- 172.21.15.62:0/1388165233 shutdown_connections 2024-06-10T00:33:02.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.631+0000 7f92672db700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f924c0a2ea0 0x7f924c0a5360 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.631+0000 7f92672db700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f926006ab60 0x7f9260101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.631+0000 7f92672db700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9260108530 0x7f9260101f10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.631+0000 7f92672db700 1 --2- 172.21.15.62:0/1388165233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f926006b520 0x7f92601019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.631+0000 7f92672db700 1 -- 172.21.15.62:0/1388165233 >> 172.21.15.62:0/1388165233 conn(0x7f92600783d0 msgr2=0x7f9260100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:02.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.631+0000 7f92672db700 1 -- 172.21.15.62:0/1388165233 shutdown_connections 2024-06-10T00:33:02.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.631+0000 7f92672db700 1 -- 172.21.15.62:0/1388165233 wait complete. 2024-06-10T00:33:02.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.702+0000 7f2f3c91d640 1 -- 172.21.15.62:0/4125574345 --> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2f38109e70 con 0x7f2f240a2ec0 2024-06-10T00:33:02.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.710+0000 7f2f2f7fe640 1 -- 172.21.15.62:0/4125574345 <== mgr.15333 v2:172.21.15.62:6800/518034449 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f2f2001b3f0 con 0x7f2f240a2ec0 2024-06-10T00:33:02.710 INFO:teuthology.orchestra.run.smithi062.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-06-10T00:00:00", "created": "2024-06-10T00:33:01", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-06-10T00:33:02.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.712+0000 7f2f3c91d640 1 -- 172.21.15.62:0/4125574345 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2f240a2ec0 msgr2=0x7f2f240a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.712+0000 7f2f3c91d640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2f240a2ec0 0x7f2f240a5380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2f38078100 tx=0x7f2f20005f90 comp rx=0 tx=0).stop 2024-06-10T00:33:02.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.712+0000 7f2f3c91d640 1 -- 172.21.15.62:0/4125574345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f381008d0 msgr2=0x7f2f38198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.712+0000 7f2f3c91d640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f381008d0 0x7f2f38198200 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009230 tx=0x7f2f30004270 comp rx=0 tx=0).stop 2024-06-10T00:33:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.712+0000 7f2f3c91d640 1 -- 172.21.15.62:0/4125574345 shutdown_connections 2024-06-10T00:33:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.712+0000 7f2f3c91d640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2f240a2ec0 0x7f2f240a5380 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.712+0000 7f2f3c91d640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f38106890 0x7f2f3819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.712+0000 7f2f3c91d640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f38101290 0x7f2f38198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.712+0000 7f2f3c91d640 1 --2- 172.21.15.62:0/4125574345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f381008d0 0x7f2f38198200 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.712+0000 7f2f3c91d640 1 -- 172.21.15.62:0/4125574345 >> 172.21.15.62:0/4125574345 conn(0x7f2f38077a20 msgr2=0x7f2f38069390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.712+0000 7f2f3c91d640 1 -- 172.21.15.62:0/4125574345 shutdown_connections 2024-06-10T00:33:02.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.712+0000 7f2f3c91d640 1 -- 172.21.15.62:0/4125574345 wait complete. 2024-06-10T00:33:02.723 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-06-10T00:33:02.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.831+0000 7fb8c17a0640 1 -- 172.21.15.62:0/1996961214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8bc102180 msgr2=0x7fb8bc10e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.831+0000 7fb8c17a0640 1 --2- 172.21.15.62:0/1996961214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8bc102180 0x7fb8bc10e690 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009cd0 tx=0x7fb8b402a8e0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.831+0000 7fb8c17a0640 1 -- 172.21.15.62:0/1996961214 shutdown_connections 2024-06-10T00:33:02.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.831+0000 7fb8c17a0640 1 --2- 172.21.15.62:0/1996961214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8bc102180 0x7fb8bc10e690 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.831+0000 7fb8c17a0640 1 --2- 172.21.15.62:0/1996961214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8bc1017c0 0x7fb8bc101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.831+0000 7fb8c17a0640 1 --2- 172.21.15.62:0/1996961214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8bc1077d0 0x7fb8bc107bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.831+0000 7fb8c17a0640 1 -- 172.21.15.62:0/1996961214 >> 172.21.15.62:0/1996961214 conn(0x7fb8bc0fd2f0 msgr2=0x7fb8bc0ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:02.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.831+0000 7fb8c17a0640 1 -- 172.21.15.62:0/1996961214 shutdown_connections 2024-06-10T00:33:02.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.831+0000 7fb8c17a0640 1 -- 172.21.15.62:0/1996961214 wait complete. 2024-06-10T00:33:02.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8c17a0640 1 Processor -- start 2024-06-10T00:33:02.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8c17a0640 1 -- start start 2024-06-10T00:33:02.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8c17a0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8bc1017c0 0x7fb8bc1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8c17a0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8bc102180 0x7fb8bc198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8c17a0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8bc1077d0 0x7fb8bc19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8c17a0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc111be0 con 0x7fb8bc1017c0 2024-06-10T00:33:02.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8c17a0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc111a60 con 0x7fb8bc102180 2024-06-10T00:33:02.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8c17a0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc111d60 con 0x7fb8bc1077d0 2024-06-10T00:33:02.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8bbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8bc1017c0 0x7fb8bc1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8bbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8bc1017c0 0x7fb8bc1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48198/0 (socket says 172.21.15.62:48198) 2024-06-10T00:33:02.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8bbfff640 1 -- 172.21.15.62:0/4159633293 learned_addr learned my addr 172.21.15.62:0/4159633293 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:02.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8bb7fe640 1 --2- 172.21.15.62:0/4159633293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8bc102180 0x7fb8bc198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8c0f9f640 1 --2- 172.21.15.62:0/4159633293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8bc1077d0 0x7fb8bc19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8bbfff640 1 -- 172.21.15.62:0/4159633293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8bc1077d0 msgr2=0x7fb8bc19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8bbfff640 1 --2- 172.21.15.62:0/4159633293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8bc1077d0 0x7fb8bc19c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8bbfff640 1 -- 172.21.15.62:0/4159633293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8bc102180 msgr2=0x7fb8bc198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:02.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8bbfff640 1 --2- 172.21.15.62:0/4159633293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8bc102180 0x7fb8bc198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:02.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.832+0000 7fb8bbfff640 1 -- 172.21.15.62:0/4159633293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8bc19d0b0 con 0x7fb8bc1017c0 2024-06-10T00:33:02.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.833+0000 7fb8bbfff640 1 --2- 172.21.15.62:0/4159633293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8bc1017c0 0x7fb8bc1980c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00d950 tx=0x7fb8ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:02.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.833+0000 7fb8b97fa640 1 -- 172.21.15.62:0/4159633293 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac017030 con 0x7fb8bc1017c0 2024-06-10T00:33:02.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.833+0000 7fb8b97fa640 1 -- 172.21.15.62:0/4159633293 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8ac004020 con 0x7fb8bc1017c0 2024-06-10T00:33:02.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.833+0000 7fb8c17a0640 1 -- 172.21.15.62:0/4159633293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8bc19d3a0 con 0x7fb8bc1017c0 2024-06-10T00:33:02.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.833+0000 7fb8b97fa640 1 -- 172.21.15.62:0/4159633293 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0047a0 con 0x7fb8bc1017c0 2024-06-10T00:33:02.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.833+0000 7fb8c17a0640 1 -- 172.21.15.62:0/4159633293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8bc1a4c30 con 0x7fb8bc1017c0 2024-06-10T00:33:02.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.834+0000 7fb8c17a0640 1 -- 172.21.15.62:0/4159633293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8bc04ffb0 con 0x7fb8bc1017c0 2024-06-10T00:33:02.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.835+0000 7fb8b97fa640 1 -- 172.21.15.62:0/4159633293 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fb8ac0041d0 con 0x7fb8bc1017c0 2024-06-10T00:33:02.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.836+0000 7fb8b97fa640 1 --2- 172.21.15.62:0/4159633293 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb8a40a2e00 0x7fb8a40a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:02.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.836+0000 7fb8bb7fe640 1 --2- 172.21.15.62:0/4159633293 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb8a40a2e00 0x7fb8a40a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:02.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.836+0000 7fb8b97fa640 1 -- 172.21.15.62:0/4159633293 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fb8ac0cf320 con 0x7fb8bc1017c0 2024-06-10T00:33:02.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.836+0000 7fb8bb7fe640 1 --2- 172.21.15.62:0/4159633293 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb8a40a2e00 0x7fb8a40a52c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0ff900 tx=0x7fb8b0005f90 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:02.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:02.840+0000 7fb8b97fa640 1 -- 172.21.15.62:0/4159633293 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb8ac09ba20 con 0x7fb8bc1017c0 2024-06-10T00:33:03.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:03.051+0000 7fb8c17a0640 1 -- 172.21.15.62:0/4159633293 --> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb8bc061de0 con 0x7fb8a40a2e00 2024-06-10T00:33:03.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:03.058+0000 7fb8b97fa640 1 -- 172.21.15.62:0/4159633293 <== mgr.15333 v2:172.21.15.62:6800/518034449 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb8bc061de0 con 0x7fb8a40a2e00 2024-06-10T00:33:03.058 INFO:teuthology.orchestra.run.smithi062.stdout:/ 1m 6m3h 2024-06-10T00:33:03.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:03.060+0000 7fb8c17a0640 1 -- 172.21.15.62:0/4159633293 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb8a40a2e00 msgr2=0x7fb8a40a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:03.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:03.060+0000 7fb8c17a0640 1 --2- 172.21.15.62:0/4159633293 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb8a40a2e00 0x7fb8a40a52c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0ff900 tx=0x7fb8b0005f90 comp rx=0 tx=0).stop 2024-06-10T00:33:03.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:03.060+0000 7fb8c17a0640 1 -- 172.21.15.62:0/4159633293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8bc1017c0 msgr2=0x7fb8bc1980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:03.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:03.060+0000 7fb8c17a0640 1 --2- 172.21.15.62:0/4159633293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8bc1017c0 0x7fb8bc1980c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00d950 tx=0x7fb8ac00de10 comp rx=0 tx=0).stop 2024-06-10T00:33:03.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:03.060+0000 7fb8c17a0640 1 -- 172.21.15.62:0/4159633293 shutdown_connections 2024-06-10T00:33:03.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:03.060+0000 7fb8c17a0640 1 --2- 172.21.15.62:0/4159633293 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb8a40a2e00 0x7fb8a40a52c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:03.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:03.060+0000 7fb8c17a0640 1 --2- 172.21.15.62:0/4159633293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8bc1077d0 0x7fb8bc19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:03.060+0000 7fb8c17a0640 1 --2- 172.21.15.62:0/4159633293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8bc102180 0x7fb8bc198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:03.060+0000 7fb8c17a0640 1 --2- 172.21.15.62:0/4159633293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8bc1017c0 0x7fb8bc1980c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:03.060+0000 7fb8c17a0640 1 -- 172.21.15.62:0/4159633293 >> 172.21.15.62:0/4159633293 conn(0x7fb8bc0fd2f0 msgr2=0x7fb8bc0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:03.060+0000 7fb8c17a0640 1 -- 172.21.15.62:0/4159633293 shutdown_connections 2024-06-10T00:33:03.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:03.060+0000 7fb8c17a0640 1 -- 172.21.15.62:0/4159633293 wait complete. 2024-06-10T00:33:03.071 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-06-10T00:33:03.098 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-06-10T00:33:03.108 INFO:teuthology.run_tasks:Running task quiescer... 2024-06-10T00:33:03.120 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-10T00:33:03.121 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T00:33:03.321 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:33:03.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:03 smithi062 ceph-mon[32390]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:33:03.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:03 smithi062 ceph-mon[32390]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 20 op/s 2024-06-10T00:33:03.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:03 smithi062 ceph-mon[32390]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:33:03.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:03 smithi062 ceph-mon[32390]: scrub queued for path: / 2024-06-10T00:33:03.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:03 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:33:03.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:03 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:33:03.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:03 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:33:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:03.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:03 smithi105 ceph-mon[36321]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:33:03.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:03 smithi105 ceph-mon[36321]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 20 op/s 2024-06-10T00:33:03.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:03 smithi105 ceph-mon[36321]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:33:03.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:03 smithi105 ceph-mon[36321]: scrub queued for path: / 2024-06-10T00:33:03.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:03 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:33:03.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:03 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:33:03.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:03 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:33:03.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:03 smithi138 ceph-mon[36549]: from='client.15474 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:33:03.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:03 smithi138 ceph-mon[36549]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 20 op/s 2024-06-10T00:33:03.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:03 smithi138 ceph-mon[36549]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:33:03.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:03 smithi138 ceph-mon[36549]: scrub queued for path: / 2024-06-10T00:33:03.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:03 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:33:03.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:03 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:33:03.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:03 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:33:04.140 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:04.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:04 smithi105 ceph-mon[36321]: from='client.25102 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:33:04.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:04 smithi105 ceph-mon[36321]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:33:04.544 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:04 smithi062 ceph-mon[32390]: from='client.25102 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:33:04.544 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:04 smithi062 ceph-mon[32390]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:33:04.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:04 smithi138 ceph-mon[36549]: from='client.25102 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:33:04.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:04 smithi138 ceph-mon[36549]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:33:05.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.340+0000 7fd427052700 1 -- 172.21.15.62:0/2961499835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42006a170 msgr2=0x7fd42006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:05.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.340+0000 7fd427052700 1 --2- 172.21.15.62:0/2961499835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42006a170 0x7fd42006a5f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd408009990 tx=0x7fd408009ca0 comp rx=0 tx=0).stop 2024-06-10T00:33:05.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.340+0000 7fd427052700 1 -- 172.21.15.62:0/2961499835 shutdown_connections 2024-06-10T00:33:05.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.340+0000 7fd427052700 1 --2- 172.21.15.62:0/2961499835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd42006ab30 0x7fd42010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:05.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.340+0000 7fd427052700 1 --2- 172.21.15.62:0/2961499835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd420108500 0x7fd4201088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:05.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.340+0000 7fd427052700 1 --2- 172.21.15.62:0/2961499835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42006a170 0x7fd42006a5f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:05.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.340+0000 7fd427052700 1 -- 172.21.15.62:0/2961499835 >> 172.21.15.62:0/2961499835 conn(0x7fd420078340 msgr2=0x7fd420078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.341+0000 7fd427052700 1 -- 172.21.15.62:0/2961499835 shutdown_connections 2024-06-10T00:33:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.341+0000 7fd427052700 1 -- 172.21.15.62:0/2961499835 wait complete. 2024-06-10T00:33:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.341+0000 7fd427052700 1 Processor -- start 2024-06-10T00:33:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.341+0000 7fd427052700 1 -- start start 2024-06-10T00:33:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.341+0000 7fd427052700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd42006a170 0x7fd420101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd427052700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42006ab30 0x7fd4201019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd427052700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd420108500 0x7fd420101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd427052700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd420112760 con 0x7fd42006ab30 2024-06-10T00:33:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd427052700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201125e0 con 0x7fd420108500 2024-06-10T00:33:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd427052700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201128e0 con 0x7fd42006a170 2024-06-10T00:33:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd41ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42006ab30 0x7fd4201019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd41ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42006ab30 0x7fd4201019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48216/0 (socket says 172.21.15.62:48216) 2024-06-10T00:33:05.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd41ffff700 1 -- 172.21.15.62:0/4114637913 learned_addr learned my addr 172.21.15.62:0/4114637913 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:05.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd4255ef700 1 --2- 172.21.15.62:0/4114637913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd420108500 0x7fd420101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:05.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd424dee700 1 --2- 172.21.15.62:0/4114637913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd42006a170 0x7fd420101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:05.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd41ffff700 1 -- 172.21.15.62:0/4114637913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd42006a170 msgr2=0x7fd420101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:05.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd41ffff700 1 --2- 172.21.15.62:0/4114637913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd42006a170 0x7fd420101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:05.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd41ffff700 1 -- 172.21.15.62:0/4114637913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd420108500 msgr2=0x7fd420101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:05.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd41ffff700 1 --2- 172.21.15.62:0/4114637913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd420108500 0x7fd420101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:05.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd41ffff700 1 -- 172.21.15.62:0/4114637913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4201a7140 con 0x7fd42006ab30 2024-06-10T00:33:05.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd41ffff700 1 --2- 172.21.15.62:0/4114637913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42006ab30 0x7fd4201019a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd40800bff0 tx=0x7fd408005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:05.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd41dffb700 1 -- 172.21.15.62:0/4114637913 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408027070 con 0x7fd42006ab30 2024-06-10T00:33:05.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd41dffb700 1 -- 172.21.15.62:0/4114637913 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4080040d0 con 0x7fd42006ab30 2024-06-10T00:33:05.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd427052700 1 -- 172.21.15.62:0/4114637913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4201a73d0 con 0x7fd42006ab30 2024-06-10T00:33:05.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.342+0000 7fd41dffb700 1 -- 172.21.15.62:0/4114637913 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408022e00 con 0x7fd42006ab30 2024-06-10T00:33:05.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.343+0000 7fd427052700 1 -- 172.21.15.62:0/4114637913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4201a78b0 con 0x7fd42006ab30 2024-06-10T00:33:05.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.344+0000 7fd427052700 1 -- 172.21.15.62:0/4114637913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd420069180 con 0x7fd42006ab30 2024-06-10T00:33:05.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.345+0000 7fd41dffb700 1 -- 172.21.15.62:0/4114637913 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fd40802a070 con 0x7fd42006ab30 2024-06-10T00:33:05.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.346+0000 7fd41dffb700 1 --2- 172.21.15.62:0/4114637913 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd40c0a2e20 0x7fd40c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:05.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.346+0000 7fd424dee700 1 --2- 172.21.15.62:0/4114637913 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd40c0a2e20 0x7fd40c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:05.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.346+0000 7fd41dffb700 1 -- 172.21.15.62:0/4114637913 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fd4080d0e50 con 0x7fd42006ab30 2024-06-10T00:33:05.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.346+0000 7fd424dee700 1 --2- 172.21.15.62:0/4114637913 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd40c0a2e20 0x7fd40c0a52e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd41000e270 tx=0x7fd410009040 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:05.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.349+0000 7fd41dffb700 1 -- 172.21.15.62:0/4114637913 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd408031020 con 0x7fd42006ab30 2024-06-10T00:33:05.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:05 smithi062 ceph-mon[32390]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 0 B/s wr, 18 op/s 2024-06-10T00:33:05.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:05 smithi105 ceph-mon[36321]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 0 B/s wr, 18 op/s 2024-06-10T00:33:05.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.555+0000 7fd427052700 1 -- 172.21.15.62:0/4114637913 --> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd420102990 con 0x7fd40c0a2e20 2024-06-10T00:33:05.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.565+0000 7fd41dffb700 1 -- 172.21.15.62:0/4114637913 <== mgr.15333 v2:172.21.15.62:6800/518034449 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339586 (secure 0 0 0) 0x7fd420102990 con 0x7fd40c0a2e20 2024-06-10T00:33:05.567 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:33:05.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.570+0000 7fd427052700 1 -- 172.21.15.62:0/4114637913 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd40c0a2e20 msgr2=0x7fd40c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:05.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.570+0000 7fd427052700 1 --2- 172.21.15.62:0/4114637913 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd40c0a2e20 0x7fd40c0a52e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd41000e270 tx=0x7fd410009040 comp rx=0 tx=0).stop 2024-06-10T00:33:05.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.570+0000 7fd427052700 1 -- 172.21.15.62:0/4114637913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42006ab30 msgr2=0x7fd4201019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:05.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.570+0000 7fd427052700 1 --2- 172.21.15.62:0/4114637913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42006ab30 0x7fd4201019a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd40800bff0 tx=0x7fd408005590 comp rx=0 tx=0).stop 2024-06-10T00:33:05.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.570+0000 7fd427052700 1 -- 172.21.15.62:0/4114637913 shutdown_connections 2024-06-10T00:33:05.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.570+0000 7fd427052700 1 --2- 172.21.15.62:0/4114637913 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd40c0a2e20 0x7fd40c0a52e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:05.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.570+0000 7fd427052700 1 --2- 172.21.15.62:0/4114637913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd420108500 0x7fd420101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:05.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.570+0000 7fd427052700 1 --2- 172.21.15.62:0/4114637913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd42006a170 0x7fd420101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:05.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.570+0000 7fd427052700 1 --2- 172.21.15.62:0/4114637913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42006ab30 0x7fd4201019a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:05.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.570+0000 7fd427052700 1 -- 172.21.15.62:0/4114637913 >> 172.21.15.62:0/4114637913 conn(0x7fd420078340 msgr2=0x7fd4200fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:05.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.570+0000 7fd427052700 1 -- 172.21.15.62:0/4114637913 shutdown_connections 2024-06-10T00:33:05.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:05.570+0000 7fd427052700 1 -- 172.21.15.62:0/4114637913 wait complete. 2024-06-10T00:33:05.582 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-06-10T00:33:05.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:05 smithi138 ceph-mon[36549]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 0 B/s wr, 18 op/s 2024-06-10T00:33:07.072 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-06-10T00:33:03.937047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194915,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2330,"num_read_kb":1498,"num_write":589,"num_write_kb":4418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":961,"ondisk_log_size":961,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387316,"kb_used_data":17336,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124554508,"statfs":{"total":1151940427776,"available":1151543816192,"internally_reserved":0,"allocated":17752064,"data_stored":10519127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":378713403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":206,"num_read_kb":272,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002956"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199389+0000","last_change":"2024-06-10T00:28:37.657234+0000","last_active":"2024-06-10T00:32:50.199389+0000","last_peered":"2024-06-10T00:32:50.199389+0000","last_clean":"2024-06-10T00:32:50.199389+0000","last_became_active":"2024-06-10T00:28:37.656879+0000","last_became_peered":"2024-06-10T00:28:37.656879+0000","last_unstale":"2024-06-10T00:32:50.199389+0000","last_undegraded":"2024-06-10T00:32:50.199389+0000","last_fullsized":"2024-06-10T00:32:50.199389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:01:02.907597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193442+0000","last_change":"2024-06-10T00:28:37.656440+0000","last_active":"2024-06-10T00:32:50.193442+0000","last_peered":"2024-06-10T00:32:50.193442+0000","last_clean":"2024-06-10T00:32:50.193442+0000","last_became_active":"2024-06-10T00:28:37.656113+0000","last_became_peered":"2024-06-10T00:28:37.656113+0000","last_unstale":"2024-06-10T00:32:50.193442+0000","last_undegraded":"2024-06-10T00:32:50.193442+0000","last_fullsized":"2024-06-10T00:32:50.193442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:49:31.447951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193070+0000","last_change":"2024-06-10T00:28:37.659114+0000","last_active":"2024-06-10T00:32:50.193070+0000","last_peered":"2024-06-10T00:32:50.193070+0000","last_clean":"2024-06-10T00:32:50.193070+0000","last_became_active":"2024-06-10T00:28:37.658763+0000","last_became_peered":"2024-06-10T00:28:37.658763+0000","last_unstale":"2024-06-10T00:32:50.193070+0000","last_undegraded":"2024-06-10T00:32:50.193070+0000","last_fullsized":"2024-06-10T00:32:50.193070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:39:54.963590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921896+0000","last_change":"2024-06-10T00:28:37.656786+0000","last_active":"2024-06-10T00:32:49.921896+0000","last_peered":"2024-06-10T00:32:49.921896+0000","last_clean":"2024-06-10T00:32:49.921896+0000","last_became_active":"2024-06-10T00:28:37.656073+0000","last_became_peered":"2024-06-10T00:28:37.656073+0000","last_unstale":"2024-06-10T00:32:49.921896+0000","last_undegraded":"2024-06-10T00:32:49.921896+0000","last_fullsized":"2024-06-10T00:32:49.921896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:19.176608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031746+0000","last_change":"2024-06-10T00:28:37.650250+0000","last_active":"2024-06-10T00:32:50.031746+0000","last_peered":"2024-06-10T00:32:50.031746+0000","last_clean":"2024-06-10T00:32:50.031746+0000","last_became_active":"2024-06-10T00:28:37.649852+0000","last_became_peered":"2024-06-10T00:28:37.649852+0000","last_unstale":"2024-06-10T00:32:50.031746+0000","last_undegraded":"2024-06-10T00:32:50.031746+0000","last_fullsized":"2024-06-10T00:32:50.031746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:39.492884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042392+0000","last_change":"2024-06-10T00:28:37.646042+0000","last_active":"2024-06-10T00:32:50.042392+0000","last_peered":"2024-06-10T00:32:50.042392+0000","last_clean":"2024-06-10T00:32:50.042392+0000","last_became_active":"2024-06-10T00:28:37.644591+0000","last_became_peered":"2024-06-10T00:28:37.644591+0000","last_unstale":"2024-06-10T00:32:50.042392+0000","last_undegraded":"2024-06-10T00:32:50.042392+0000","last_fullsized":"2024-06-10T00:32:50.042392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:29.932303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134284+0000","last_change":"2024-06-10T00:28:37.664076+0000","last_active":"2024-06-10T00:32:50.134284+0000","last_peered":"2024-06-10T00:32:50.134284+0000","last_clean":"2024-06-10T00:32:50.134284+0000","last_became_active":"2024-06-10T00:28:37.663741+0000","last_became_peered":"2024-06-10T00:28:37.663741+0000","last_unstale":"2024-06-10T00:32:50.134284+0000","last_undegraded":"2024-06-10T00:32:50.134284+0000","last_fullsized":"2024-06-10T00:32:50.134284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:14.581409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032039+0000","last_change":"2024-06-10T00:28:37.648758+0000","last_active":"2024-06-10T00:32:50.032039+0000","last_peered":"2024-06-10T00:32:50.032039+0000","last_clean":"2024-06-10T00:32:50.032039+0000","last_became_active":"2024-06-10T00:28:37.648387+0000","last_became_peered":"2024-06-10T00:28:37.648387+0000","last_unstale":"2024-06-10T00:32:50.032039+0000","last_undegraded":"2024-06-10T00:32:50.032039+0000","last_fullsized":"2024-06-10T00:32:50.032039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:17.187265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.189039+0000","last_change":"2024-06-10T00:28:37.664650+0000","last_active":"2024-06-10T00:32:50.189039+0000","last_peered":"2024-06-10T00:32:50.189039+0000","last_clean":"2024-06-10T00:32:50.189039+0000","last_became_active":"2024-06-10T00:28:37.663178+0000","last_became_peered":"2024-06-10T00:28:37.663178+0000","last_unstale":"2024-06-10T00:32:50.189039+0000","last_undegraded":"2024-06-10T00:32:50.189039+0000","last_fullsized":"2024-06-10T00:32:50.189039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:46.032078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042870+0000","last_change":"2024-06-10T00:28:37.658353+0000","last_active":"2024-06-10T00:32:50.042870+0000","last_peered":"2024-06-10T00:32:50.042870+0000","last_clean":"2024-06-10T00:32:50.042870+0000","last_became_active":"2024-06-10T00:28:37.657961+0000","last_became_peered":"2024-06-10T00:28:37.657961+0000","last_unstale":"2024-06-10T00:32:50.042870+0000","last_undegraded":"2024-06-10T00:32:50.042870+0000","last_fullsized":"2024-06-10T00:32:50.042870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:22:27.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134878+0000","last_change":"2024-06-10T00:28:37.663823+0000","last_active":"2024-06-10T00:32:50.134878+0000","last_peered":"2024-06-10T00:32:50.134878+0000","last_clean":"2024-06-10T00:32:50.134878+0000","last_became_active":"2024-06-10T00:28:37.663499+0000","last_became_peered":"2024-06-10T00:28:37.663499+0000","last_unstale":"2024-06-10T00:32:50.134878+0000","last_undegraded":"2024-06-10T00:32:50.134878+0000","last_fullsized":"2024-06-10T00:32:50.134878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.100923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921877+0000","last_change":"2024-06-10T00:30:24.112722+0000","last_active":"2024-06-10T00:32:49.921877+0000","last_peered":"2024-06-10T00:32:49.921877+0000","last_clean":"2024-06-10T00:32:49.921877+0000","last_became_active":"2024-06-10T00:30:24.112325+0000","last_became_peered":"2024-06-10T00:30:24.112325+0000","last_unstale":"2024-06-10T00:32:49.921877+0000","last_undegraded":"2024-06-10T00:32:49.921877+0000","last_fullsized":"2024-06-10T00:32:49.921877+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:26.671583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133292+0000","last_change":"2024-06-10T00:28:37.649997+0000","last_active":"2024-06-10T00:32:50.133292+0000","last_peered":"2024-06-10T00:32:50.133292+0000","last_clean":"2024-06-10T00:32:50.133292+0000","last_became_active":"2024-06-10T00:28:37.649494+0000","last_became_peered":"2024-06-10T00:28:37.649494+0000","last_unstale":"2024-06-10T00:32:50.133292+0000","last_undegraded":"2024-06-10T00:32:50.133292+0000","last_fullsized":"2024-06-10T00:32:50.133292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:44:07.467206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192453+0000","last_change":"2024-06-10T00:28:37.653064+0000","last_active":"2024-06-10T00:32:50.192453+0000","last_peered":"2024-06-10T00:32:50.192453+0000","last_clean":"2024-06-10T00:32:50.192453+0000","last_became_active":"2024-06-10T00:28:37.652684+0000","last_became_peered":"2024-06-10T00:28:37.652684+0000","last_unstale":"2024-06-10T00:32:50.192453+0000","last_undegraded":"2024-06-10T00:32:50.192453+0000","last_fullsized":"2024-06-10T00:32:50.192453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:30.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192384+0000","last_change":"2024-06-10T00:28:37.655570+0000","last_active":"2024-06-10T00:32:50.192384+0000","last_peered":"2024-06-10T00:32:50.192384+0000","last_clean":"2024-06-10T00:32:50.192384+0000","last_became_active":"2024-06-10T00:28:37.655241+0000","last_became_peered":"2024-06-10T00:28:37.655241+0000","last_unstale":"2024-06-10T00:32:50.192384+0000","last_undegraded":"2024-06-10T00:32:50.192384+0000","last_fullsized":"2024-06-10T00:32:50.192384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:07:03.543064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188211+0000","last_change":"2024-06-10T00:28:37.665607+0000","last_active":"2024-06-10T00:32:50.188211+0000","last_peered":"2024-06-10T00:32:50.188211+0000","last_clean":"2024-06-10T00:32:50.188211+0000","last_became_active":"2024-06-10T00:28:37.664943+0000","last_became_peered":"2024-06-10T00:28:37.664943+0000","last_unstale":"2024-06-10T00:32:50.188211+0000","last_undegraded":"2024-06-10T00:32:50.188211+0000","last_fullsized":"2024-06-10T00:32:50.188211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:43:02.940298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199188+0000","last_change":"2024-06-10T00:28:37.658214+0000","last_active":"2024-06-10T00:32:50.199188+0000","last_peered":"2024-06-10T00:32:50.199188+0000","last_clean":"2024-06-10T00:32:50.199188+0000","last_became_active":"2024-06-10T00:28:37.657295+0000","last_became_peered":"2024-06-10T00:28:37.657295+0000","last_unstale":"2024-06-10T00:32:50.199188+0000","last_undegraded":"2024-06-10T00:32:50.199188+0000","last_fullsized":"2024-06-10T00:32:50.199188+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:20.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922079+0000","last_change":"2024-06-10T00:28:37.656645+0000","last_active":"2024-06-10T00:32:49.922079+0000","last_peered":"2024-06-10T00:32:49.922079+0000","last_clean":"2024-06-10T00:32:49.922079+0000","last_became_active":"2024-06-10T00:28:37.656157+0000","last_became_peered":"2024-06-10T00:28:37.656157+0000","last_unstale":"2024-06-10T00:32:49.922079+0000","last_undegraded":"2024-06-10T00:32:49.922079+0000","last_fullsized":"2024-06-10T00:32:49.922079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:00.953804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921572+0000","last_change":"2024-06-10T00:28:37.664336+0000","last_active":"2024-06-10T00:32:49.921572+0000","last_peered":"2024-06-10T00:32:49.921572+0000","last_clean":"2024-06-10T00:32:49.921572+0000","last_became_active":"2024-06-10T00:28:37.663960+0000","last_became_peered":"2024-06-10T00:28:37.663960+0000","last_unstale":"2024-06-10T00:32:49.921572+0000","last_undegraded":"2024-06-10T00:32:49.921572+0000","last_fullsized":"2024-06-10T00:32:49.921572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:59:32.774442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920810+0000","last_change":"2024-06-10T00:28:48.705930+0000","last_active":"2024-06-10T00:32:49.920810+0000","last_peered":"2024-06-10T00:32:49.920810+0000","last_clean":"2024-06-10T00:32:49.920810+0000","last_became_active":"2024-06-10T00:28:48.705558+0000","last_became_peered":"2024-06-10T00:28:48.705558+0000","last_unstale":"2024-06-10T00:32:49.920810+0000","last_undegraded":"2024-06-10T00:32:49.920810+0000","last_fullsized":"2024-06-10T00:32:49.920810+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:41:44.914378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"99'14","reported_seq":270,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:59.144476+0000","last_change":"2024-06-10T00:29:23.862141+0000","last_active":"2024-06-10T00:32:59.144476+0000","last_peered":"2024-06-10T00:32:59.144476+0000","last_clean":"2024-06-10T00:32:59.144476+0000","last_became_active":"2024-06-10T00:29:23.857051+0000","last_became_peered":"2024-06-10T00:29:23.857051+0000","last_unstale":"2024-06-10T00:32:59.144476+0000","last_undegraded":"2024-06-10T00:32:59.144476+0000","last_fullsized":"2024-06-10T00:32:59.144476+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:41:41.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":196,"num_read_kb":196,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032309+0000","last_change":"2024-06-10T00:28:37.653938+0000","last_active":"2024-06-10T00:32:50.032309+0000","last_peered":"2024-06-10T00:32:50.032309+0000","last_clean":"2024-06-10T00:32:50.032309+0000","last_became_active":"2024-06-10T00:28:37.653629+0000","last_became_peered":"2024-06-10T00:28:37.653629+0000","last_unstale":"2024-06-10T00:32:50.032309+0000","last_undegraded":"2024-06-10T00:32:50.032309+0000","last_fullsized":"2024-06-10T00:32:50.032309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:21:39.285961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921432+0000","last_change":"2024-06-10T00:28:48.704413+0000","last_active":"2024-06-10T00:32:49.921432+0000","last_peered":"2024-06-10T00:32:49.921432+0000","last_clean":"2024-06-10T00:32:49.921432+0000","last_became_active":"2024-06-10T00:28:48.703972+0000","last_became_peered":"2024-06-10T00:28:48.703972+0000","last_unstale":"2024-06-10T00:32:49.921432+0000","last_undegraded":"2024-06-10T00:32:49.921432+0000","last_fullsized":"2024-06-10T00:32:49.921432+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:51.927435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032677+0000","last_change":"2024-06-10T00:28:34.445413+0000","last_active":"2024-06-10T00:32:50.032677+0000","last_peered":"2024-06-10T00:32:50.032677+0000","last_clean":"2024-06-10T00:32:50.032677+0000","last_became_active":"2024-06-10T00:28:34.445077+0000","last_became_peered":"2024-06-10T00:28:34.445077+0000","last_unstale":"2024-06-10T00:32:50.032677+0000","last_undegraded":"2024-06-10T00:32:50.032677+0000","last_fullsized":"2024-06-10T00:32:50.032677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:19:19.535976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193506+0000","last_change":"2024-06-10T00:28:37.657123+0000","last_active":"2024-06-10T00:32:50.193506+0000","last_peered":"2024-06-10T00:32:50.193506+0000","last_clean":"2024-06-10T00:32:50.193506+0000","last_became_active":"2024-06-10T00:28:37.656791+0000","last_became_peered":"2024-06-10T00:28:37.656791+0000","last_unstale":"2024-06-10T00:32:50.193506+0000","last_undegraded":"2024-06-10T00:32:50.193506+0000","last_fullsized":"2024-06-10T00:32:50.193506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:11.102478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921858+0000","last_change":"2024-06-10T00:28:48.705989+0000","last_active":"2024-06-10T00:32:49.921858+0000","last_peered":"2024-06-10T00:32:49.921858+0000","last_clean":"2024-06-10T00:32:49.921858+0000","last_became_active":"2024-06-10T00:28:48.705555+0000","last_became_peered":"2024-06-10T00:28:48.705555+0000","last_unstale":"2024-06-10T00:32:49.921858+0000","last_undegraded":"2024-06-10T00:32:49.921858+0000","last_fullsized":"2024-06-10T00:32:49.921858+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:36.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"100'13","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032133+0000","last_change":"2024-06-10T00:28:34.400117+0000","last_active":"2024-06-10T00:32:50.032133+0000","last_peered":"2024-06-10T00:32:50.032133+0000","last_clean":"2024-06-10T00:32:50.032133+0000","last_became_active":"2024-06-10T00:28:34.399743+0000","last_became_peered":"2024-06-10T00:28:34.399743+0000","last_unstale":"2024-06-10T00:32:50.032133+0000","last_undegraded":"2024-06-10T00:32:50.032133+0000","last_fullsized":"2024-06-10T00:32:50.032133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:20:57.580146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188627+0000","last_change":"2024-06-10T00:28:37.664005+0000","last_active":"2024-06-10T00:32:50.188627+0000","last_peered":"2024-06-10T00:32:50.188627+0000","last_clean":"2024-06-10T00:32:50.188627+0000","last_became_active":"2024-06-10T00:28:37.661297+0000","last_became_peered":"2024-06-10T00:28:37.661297+0000","last_unstale":"2024-06-10T00:32:50.188627+0000","last_undegraded":"2024-06-10T00:32:50.188627+0000","last_fullsized":"2024-06-10T00:32:50.188627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:57.795080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921561+0000","last_change":"2024-06-10T00:29:23.859091+0000","last_active":"2024-06-10T00:32:49.921561+0000","last_peered":"2024-06-10T00:32:49.921561+0000","last_clean":"2024-06-10T00:32:49.921561+0000","last_became_active":"2024-06-10T00:29:23.858485+0000","last_became_peered":"2024-06-10T00:29:23.858485+0000","last_unstale":"2024-06-10T00:32:49.921561+0000","last_undegraded":"2024-06-10T00:32:49.921561+0000","last_fullsized":"2024-06-10T00:32:49.921561+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:07:10.925323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031945+0000","last_change":"2024-06-10T00:28:34.400588+0000","last_active":"2024-06-10T00:32:50.031945+0000","last_peered":"2024-06-10T00:32:50.031945+0000","last_clean":"2024-06-10T00:32:50.031945+0000","last_became_active":"2024-06-10T00:28:34.400027+0000","last_became_peered":"2024-06-10T00:28:34.400027+0000","last_unstale":"2024-06-10T00:32:50.031945+0000","last_undegraded":"2024-06-10T00:32:50.031945+0000","last_fullsized":"2024-06-10T00:32:50.031945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:25:59.890076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039025+0000","last_change":"2024-06-10T00:28:37.659401+0000","last_active":"2024-06-10T00:32:50.039025+0000","last_peered":"2024-06-10T00:32:50.039025+0000","last_clean":"2024-06-10T00:32:50.039025+0000","last_became_active":"2024-06-10T00:28:37.659056+0000","last_became_peered":"2024-06-10T00:28:37.659056+0000","last_unstale":"2024-06-10T00:32:50.039025+0000","last_undegraded":"2024-06-10T00:32:50.039025+0000","last_fullsized":"2024-06-10T00:32:50.039025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:35:01.415630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921379+0000","last_change":"2024-06-10T00:28:48.702961+0000","last_active":"2024-06-10T00:32:49.921379+0000","last_peered":"2024-06-10T00:32:49.921379+0000","last_clean":"2024-06-10T00:32:49.921379+0000","last_became_active":"2024-06-10T00:28:48.702502+0000","last_became_peered":"2024-06-10T00:28:48.702502+0000","last_unstale":"2024-06-10T00:32:49.921379+0000","last_undegraded":"2024-06-10T00:32:49.921379+0000","last_fullsized":"2024-06-10T00:32:49.921379+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:54.881595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'9","reported_seq":255,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:59.143683+0000","last_change":"2024-06-10T00:28:34.396310+0000","last_active":"2024-06-10T00:32:59.143683+0000","last_peered":"2024-06-10T00:32:59.143683+0000","last_clean":"2024-06-10T00:32:59.143683+0000","last_became_active":"2024-06-10T00:28:34.395401+0000","last_became_peered":"2024-06-10T00:28:34.395401+0000","last_unstale":"2024-06-10T00:32:59.143683+0000","last_undegraded":"2024-06-10T00:32:59.143683+0000","last_fullsized":"2024-06-10T00:32:59.143683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:33:54.300308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":200,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031647+0000","last_change":"2024-06-10T00:28:37.651427+0000","last_active":"2024-06-10T00:32:50.031647+0000","last_peered":"2024-06-10T00:32:50.031647+0000","last_clean":"2024-06-10T00:32:50.031647+0000","last_became_active":"2024-06-10T00:28:37.650510+0000","last_became_peered":"2024-06-10T00:28:37.650510+0000","last_unstale":"2024-06-10T00:32:50.031647+0000","last_undegraded":"2024-06-10T00:32:50.031647+0000","last_fullsized":"2024-06-10T00:32:50.031647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:42.953919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920970+0000","last_change":"2024-06-10T00:28:48.705918+0000","last_active":"2024-06-10T00:32:49.920970+0000","last_peered":"2024-06-10T00:32:49.920970+0000","last_clean":"2024-06-10T00:32:49.920970+0000","last_became_active":"2024-06-10T00:28:48.705462+0000","last_became_peered":"2024-06-10T00:28:48.705462+0000","last_unstale":"2024-06-10T00:32:49.920970+0000","last_undegraded":"2024-06-10T00:32:49.920970+0000","last_fullsized":"2024-06-10T00:32:49.920970+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:59:23.176511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042358+0000","last_change":"2024-06-10T00:28:34.393199+0000","last_active":"2024-06-10T00:32:50.042358+0000","last_peered":"2024-06-10T00:32:50.042358+0000","last_clean":"2024-06-10T00:32:50.042358+0000","last_became_active":"2024-06-10T00:28:34.392777+0000","last_became_peered":"2024-06-10T00:28:34.392777+0000","last_unstale":"2024-06-10T00:32:50.042358+0000","last_undegraded":"2024-06-10T00:32:50.042358+0000","last_fullsized":"2024-06-10T00:32:50.042358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:29.614242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038961+0000","last_change":"2024-06-10T00:30:24.229721+0000","last_active":"2024-06-10T00:32:50.038961+0000","last_peered":"2024-06-10T00:32:50.038961+0000","last_clean":"2024-06-10T00:32:50.038961+0000","last_became_active":"2024-06-10T00:30:24.229407+0000","last_became_peered":"2024-06-10T00:30:24.229407+0000","last_unstale":"2024-06-10T00:32:50.038961+0000","last_undegraded":"2024-06-10T00:32:50.038961+0000","last_fullsized":"2024-06-10T00:32:50.038961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:39.523197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199129+0000","last_change":"2024-06-10T00:29:23.780971+0000","last_active":"2024-06-10T00:32:50.199129+0000","last_peered":"2024-06-10T00:32:50.199129+0000","last_clean":"2024-06-10T00:32:50.199129+0000","last_became_active":"2024-06-10T00:29:23.780034+0000","last_became_peered":"2024-06-10T00:29:23.780034+0000","last_unstale":"2024-06-10T00:32:50.199129+0000","last_undegraded":"2024-06-10T00:32:50.199129+0000","last_fullsized":"2024-06-10T00:32:50.199129+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:47:06.444181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132954+0000","last_change":"2024-06-10T00:28:34.392486+0000","last_active":"2024-06-10T00:32:50.132954+0000","last_peered":"2024-06-10T00:32:50.132954+0000","last_clean":"2024-06-10T00:32:50.132954+0000","last_became_active":"2024-06-10T00:28:34.392141+0000","last_became_peered":"2024-06-10T00:28:34.392141+0000","last_unstale":"2024-06-10T00:32:50.132954+0000","last_undegraded":"2024-06-10T00:32:50.132954+0000","last_fullsized":"2024-06-10T00:32:50.132954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:10:01.534195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031869+0000","last_change":"2024-06-10T00:28:37.651033+0000","last_active":"2024-06-10T00:32:50.031869+0000","last_peered":"2024-06-10T00:32:50.031869+0000","last_clean":"2024-06-10T00:32:50.031869+0000","last_became_active":"2024-06-10T00:28:37.650505+0000","last_became_peered":"2024-06-10T00:28:37.650505+0000","last_unstale":"2024-06-10T00:32:50.031869+0000","last_undegraded":"2024-06-10T00:32:50.031869+0000","last_fullsized":"2024-06-10T00:32:50.031869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:54.965865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193367+0000","last_change":"2024-06-10T00:28:48.710748+0000","last_active":"2024-06-10T00:32:50.193367+0000","last_peered":"2024-06-10T00:32:50.193367+0000","last_clean":"2024-06-10T00:32:50.193367+0000","last_became_active":"2024-06-10T00:28:48.710353+0000","last_became_peered":"2024-06-10T00:28:48.710353+0000","last_unstale":"2024-06-10T00:32:50.193367+0000","last_undegraded":"2024-06-10T00:32:50.193367+0000","last_fullsized":"2024-06-10T00:32:50.193367+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:30:47.546650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038780+0000","last_change":"2024-06-10T00:28:34.398911+0000","last_active":"2024-06-10T00:32:50.038780+0000","last_peered":"2024-06-10T00:32:50.038780+0000","last_clean":"2024-06-10T00:32:50.038780+0000","last_became_active":"2024-06-10T00:28:34.398373+0000","last_became_peered":"2024-06-10T00:28:34.398373+0000","last_unstale":"2024-06-10T00:32:50.038780+0000","last_undegraded":"2024-06-10T00:32:50.038780+0000","last_fullsized":"2024-06-10T00:32:50.038780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:05:34.352825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921526+0000","last_change":"2024-06-10T00:28:37.647492+0000","last_active":"2024-06-10T00:32:49.921526+0000","last_peered":"2024-06-10T00:32:49.921526+0000","last_clean":"2024-06-10T00:32:49.921526+0000","last_became_active":"2024-06-10T00:28:37.647103+0000","last_became_peered":"2024-06-10T00:28:37.647103+0000","last_unstale":"2024-06-10T00:32:49.921526+0000","last_undegraded":"2024-06-10T00:32:49.921526+0000","last_fullsized":"2024-06-10T00:32:49.921526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:10.429256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031883+0000","last_change":"2024-06-10T00:28:48.703095+0000","last_active":"2024-06-10T00:32:50.031883+0000","last_peered":"2024-06-10T00:32:50.031883+0000","last_clean":"2024-06-10T00:32:50.031883+0000","last_became_active":"2024-06-10T00:28:48.702687+0000","last_became_peered":"2024-06-10T00:28:48.702687+0000","last_unstale":"2024-06-10T00:32:50.031883+0000","last_undegraded":"2024-06-10T00:32:50.031883+0000","last_fullsized":"2024-06-10T00:32:50.031883+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:10:20.074173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":443,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:00.144542+0000","last_change":"2024-06-10T00:28:34.394807+0000","last_active":"2024-06-10T00:33:00.144542+0000","last_peered":"2024-06-10T00:33:00.144542+0000","last_clean":"2024-06-10T00:33:00.144542+0000","last_became_active":"2024-06-10T00:28:34.394022+0000","last_became_peered":"2024-06-10T00:28:34.394022+0000","last_unstale":"2024-06-10T00:33:00.144542+0000","last_undegraded":"2024-06-10T00:33:00.144542+0000","last_fullsized":"2024-06-10T00:33:00.144542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:21.626079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":390,"num_read_kb":390,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.189056+0000","last_change":"2024-06-10T00:28:37.662231+0000","last_active":"2024-06-10T00:32:50.189056+0000","last_peered":"2024-06-10T00:32:50.189056+0000","last_clean":"2024-06-10T00:32:50.189056+0000","last_became_active":"2024-06-10T00:28:37.661861+0000","last_became_peered":"2024-06-10T00:28:37.661861+0000","last_unstale":"2024-06-10T00:32:50.189056+0000","last_undegraded":"2024-06-10T00:32:50.189056+0000","last_fullsized":"2024-06-10T00:32:50.189056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:09.705349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921535+0000","last_change":"2024-06-10T00:28:48.706643+0000","last_active":"2024-06-10T00:32:49.921535+0000","last_peered":"2024-06-10T00:32:49.921535+0000","last_clean":"2024-06-10T00:32:49.921535+0000","last_became_active":"2024-06-10T00:28:48.706127+0000","last_became_peered":"2024-06-10T00:28:48.706127+0000","last_unstale":"2024-06-10T00:32:49.921535+0000","last_undegraded":"2024-06-10T00:32:49.921535+0000","last_fullsized":"2024-06-10T00:32:49.921535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:55.031303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134861+0000","last_change":"2024-06-10T00:28:34.392808+0000","last_active":"2024-06-10T00:32:50.134861+0000","last_peered":"2024-06-10T00:32:50.134861+0000","last_clean":"2024-06-10T00:32:50.134861+0000","last_became_active":"2024-06-10T00:28:34.392450+0000","last_became_peered":"2024-06-10T00:28:34.392450+0000","last_unstale":"2024-06-10T00:32:50.134861+0000","last_undegraded":"2024-06-10T00:32:50.134861+0000","last_fullsized":"2024-06-10T00:32:50.134861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:06:00.493316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193616+0000","last_change":"2024-06-10T00:28:37.656099+0000","last_active":"2024-06-10T00:32:50.193616+0000","last_peered":"2024-06-10T00:32:50.193616+0000","last_clean":"2024-06-10T00:32:50.193616+0000","last_became_active":"2024-06-10T00:28:37.655640+0000","last_became_peered":"2024-06-10T00:28:37.655640+0000","last_unstale":"2024-06-10T00:32:50.193616+0000","last_undegraded":"2024-06-10T00:32:50.193616+0000","last_fullsized":"2024-06-10T00:32:50.193616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:59:26.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922109+0000","last_change":"2024-06-10T00:28:48.702628+0000","last_active":"2024-06-10T00:32:49.922109+0000","last_peered":"2024-06-10T00:32:49.922109+0000","last_clean":"2024-06-10T00:32:49.922109+0000","last_became_active":"2024-06-10T00:28:48.702109+0000","last_became_peered":"2024-06-10T00:28:48.702109+0000","last_unstale":"2024-06-10T00:32:49.922109+0000","last_undegraded":"2024-06-10T00:32:49.922109+0000","last_fullsized":"2024-06-10T00:32:49.922109+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:25.262348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039027+0000","last_change":"2024-06-10T00:28:34.446788+0000","last_active":"2024-06-10T00:32:50.039027+0000","last_peered":"2024-06-10T00:32:50.039027+0000","last_clean":"2024-06-10T00:32:50.039027+0000","last_became_active":"2024-06-10T00:28:34.446377+0000","last_became_peered":"2024-06-10T00:28:34.446377+0000","last_unstale":"2024-06-10T00:32:50.039027+0000","last_undegraded":"2024-06-10T00:32:50.039027+0000","last_fullsized":"2024-06-10T00:32:50.039027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:16.914244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188829+0000","last_change":"2024-06-10T00:28:37.666159+0000","last_active":"2024-06-10T00:32:50.188829+0000","last_peered":"2024-06-10T00:32:50.188829+0000","last_clean":"2024-06-10T00:32:50.188829+0000","last_became_active":"2024-06-10T00:28:37.665711+0000","last_became_peered":"2024-06-10T00:28:37.665711+0000","last_unstale":"2024-06-10T00:32:50.188829+0000","last_undegraded":"2024-06-10T00:32:50.188829+0000","last_fullsized":"2024-06-10T00:32:50.188829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:03.982375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921636+0000","last_change":"2024-06-10T00:28:48.705071+0000","last_active":"2024-06-10T00:32:49.921636+0000","last_peered":"2024-06-10T00:32:49.921636+0000","last_clean":"2024-06-10T00:32:49.921636+0000","last_became_active":"2024-06-10T00:28:48.702211+0000","last_became_peered":"2024-06-10T00:28:48.702211+0000","last_unstale":"2024-06-10T00:32:49.921636+0000","last_undegraded":"2024-06-10T00:32:49.921636+0000","last_fullsized":"2024-06-10T00:32:49.921636+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:13.361985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":489,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:00.144415+0000","last_change":"2024-06-10T00:28:34.389784+0000","last_active":"2024-06-10T00:33:00.144415+0000","last_peered":"2024-06-10T00:33:00.144415+0000","last_clean":"2024-06-10T00:33:00.144415+0000","last_became_active":"2024-06-10T00:28:34.389416+0000","last_became_peered":"2024-06-10T00:28:34.389416+0000","last_unstale":"2024-06-10T00:33:00.144415+0000","last_undegraded":"2024-06-10T00:33:00.144415+0000","last_fullsized":"2024-06-10T00:33:00.144415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:23:05.803503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69365,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":402,"num_read_kb":270,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031957+0000","last_change":"2024-06-10T00:28:37.656174+0000","last_active":"2024-06-10T00:32:50.031957+0000","last_peered":"2024-06-10T00:32:50.031957+0000","last_clean":"2024-06-10T00:32:50.031957+0000","last_became_active":"2024-06-10T00:28:37.655794+0000","last_became_peered":"2024-06-10T00:28:37.655794+0000","last_unstale":"2024-06-10T00:32:50.031957+0000","last_undegraded":"2024-06-10T00:32:50.031957+0000","last_fullsized":"2024-06-10T00:32:50.031957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:22:00.310309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039312+0000","last_change":"2024-06-10T00:28:48.709915+0000","last_active":"2024-06-10T00:32:50.039312+0000","last_peered":"2024-06-10T00:32:50.039312+0000","last_clean":"2024-06-10T00:32:50.039312+0000","last_became_active":"2024-06-10T00:28:48.709426+0000","last_became_peered":"2024-06-10T00:28:48.709426+0000","last_unstale":"2024-06-10T00:32:50.039312+0000","last_undegraded":"2024-06-10T00:32:50.039312+0000","last_fullsized":"2024-06-10T00:32:50.039312+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:55.150617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921708+0000","last_change":"2024-06-10T00:28:34.389136+0000","last_active":"2024-06-10T00:32:49.921708+0000","last_peered":"2024-06-10T00:32:49.921708+0000","last_clean":"2024-06-10T00:32:49.921708+0000","last_became_active":"2024-06-10T00:28:34.388701+0000","last_became_peered":"2024-06-10T00:28:34.388701+0000","last_unstale":"2024-06-10T00:32:49.921708+0000","last_undegraded":"2024-06-10T00:32:49.921708+0000","last_fullsized":"2024-06-10T00:32:49.921708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:13:24.907824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133047+0000","last_change":"2024-06-10T00:28:37.659048+0000","last_active":"2024-06-10T00:32:50.133047+0000","last_peered":"2024-06-10T00:32:50.133047+0000","last_clean":"2024-06-10T00:32:50.133047+0000","last_became_active":"2024-06-10T00:28:37.658710+0000","last_became_peered":"2024-06-10T00:28:37.658710+0000","last_unstale":"2024-06-10T00:32:50.133047+0000","last_undegraded":"2024-06-10T00:32:50.133047+0000","last_fullsized":"2024-06-10T00:32:50.133047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:44.314559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198919+0000","last_change":"2024-06-10T00:28:48.709683+0000","last_active":"2024-06-10T00:32:50.198919+0000","last_peered":"2024-06-10T00:32:50.198919+0000","last_clean":"2024-06-10T00:32:50.198919+0000","last_became_active":"2024-06-10T00:28:48.709241+0000","last_became_peered":"2024-06-10T00:28:48.709241+0000","last_unstale":"2024-06-10T00:32:50.198919+0000","last_undegraded":"2024-06-10T00:32:50.198919+0000","last_fullsized":"2024-06-10T00:32:50.198919+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:25.190043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":24,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198772+0000","last_change":"2024-06-10T00:29:23.782029+0000","last_active":"2024-06-10T00:32:50.198772+0000","last_peered":"2024-06-10T00:32:50.198772+0000","last_clean":"2024-06-10T00:32:50.198772+0000","last_became_active":"2024-06-10T00:29:23.781565+0000","last_became_peered":"2024-06-10T00:29:23.781565+0000","last_unstale":"2024-06-10T00:32:50.198772+0000","last_undegraded":"2024-06-10T00:32:50.198772+0000","last_fullsized":"2024-06-10T00:32:50.198772+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:53.258760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134547+0000","last_change":"2024-06-10T00:28:37.653385+0000","last_active":"2024-06-10T00:32:50.134547+0000","last_peered":"2024-06-10T00:32:50.134547+0000","last_clean":"2024-06-10T00:32:50.134547+0000","last_became_active":"2024-06-10T00:28:37.653020+0000","last_became_peered":"2024-06-10T00:28:37.653020+0000","last_unstale":"2024-06-10T00:32:50.134547+0000","last_undegraded":"2024-06-10T00:32:50.134547+0000","last_fullsized":"2024-06-10T00:32:50.134547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:57:54.469544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921940+0000","last_change":"2024-06-10T00:28:48.707540+0000","last_active":"2024-06-10T00:32:49.921940+0000","last_peered":"2024-06-10T00:32:49.921940+0000","last_clean":"2024-06-10T00:32:49.921940+0000","last_became_active":"2024-06-10T00:28:48.707143+0000","last_became_peered":"2024-06-10T00:28:48.707143+0000","last_unstale":"2024-06-10T00:32:49.921940+0000","last_undegraded":"2024-06-10T00:32:49.921940+0000","last_fullsized":"2024-06-10T00:32:49.921940+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:20:29.344808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'8","reported_seq":255,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:59.144853+0000","last_change":"2024-06-10T00:28:34.397496+0000","last_active":"2024-06-10T00:32:59.144853+0000","last_peered":"2024-06-10T00:32:59.144853+0000","last_clean":"2024-06-10T00:32:59.144853+0000","last_became_active":"2024-06-10T00:28:34.397124+0000","last_became_peered":"2024-06-10T00:28:34.397124+0000","last_unstale":"2024-06-10T00:32:59.144853+0000","last_undegraded":"2024-06-10T00:32:59.144853+0000","last_fullsized":"2024-06-10T00:32:59.144853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:41:48.609354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":201,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134324+0000","last_change":"2024-06-10T00:28:37.658338+0000","last_active":"2024-06-10T00:32:50.134324+0000","last_peered":"2024-06-10T00:32:50.134324+0000","last_clean":"2024-06-10T00:32:50.134324+0000","last_became_active":"2024-06-10T00:28:37.655910+0000","last_became_peered":"2024-06-10T00:28:37.655910+0000","last_unstale":"2024-06-10T00:32:50.134324+0000","last_undegraded":"2024-06-10T00:32:50.134324+0000","last_fullsized":"2024-06-10T00:32:50.134324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:38.501335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133960+0000","last_change":"2024-06-10T00:28:48.711151+0000","last_active":"2024-06-10T00:32:50.133960+0000","last_peered":"2024-06-10T00:32:50.133960+0000","last_clean":"2024-06-10T00:32:50.133960+0000","last_became_active":"2024-06-10T00:28:48.710768+0000","last_became_peered":"2024-06-10T00:28:48.710768+0000","last_unstale":"2024-06-10T00:32:50.133960+0000","last_undegraded":"2024-06-10T00:32:50.133960+0000","last_fullsized":"2024-06-10T00:32:50.133960+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:15:22.791839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":65,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031568+0000","last_change":"2024-06-10T00:28:34.445812+0000","last_active":"2024-06-10T00:32:50.031568+0000","last_peered":"2024-06-10T00:32:50.031568+0000","last_clean":"2024-06-10T00:32:50.031568+0000","last_became_active":"2024-06-10T00:28:34.445472+0000","last_became_peered":"2024-06-10T00:28:34.445472+0000","last_unstale":"2024-06-10T00:32:50.031568+0000","last_undegraded":"2024-06-10T00:32:50.031568+0000","last_fullsized":"2024-06-10T00:32:50.031568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:00.912050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038769+0000","last_change":"2024-06-10T00:28:37.655345+0000","last_active":"2024-06-10T00:32:50.038769+0000","last_peered":"2024-06-10T00:32:50.038769+0000","last_clean":"2024-06-10T00:32:50.038769+0000","last_became_active":"2024-06-10T00:28:37.654954+0000","last_became_peered":"2024-06-10T00:28:37.654954+0000","last_unstale":"2024-06-10T00:32:50.038769+0000","last_undegraded":"2024-06-10T00:32:50.038769+0000","last_fullsized":"2024-06-10T00:32:50.038769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:39:27.403006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038741+0000","last_change":"2024-06-10T00:28:48.705247+0000","last_active":"2024-06-10T00:32:50.038741+0000","last_peered":"2024-06-10T00:32:50.038741+0000","last_clean":"2024-06-10T00:32:50.038741+0000","last_became_active":"2024-06-10T00:28:48.704822+0000","last_became_peered":"2024-06-10T00:28:48.704822+0000","last_unstale":"2024-06-10T00:32:50.038741+0000","last_undegraded":"2024-06-10T00:32:50.038741+0000","last_fullsized":"2024-06-10T00:32:50.038741+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:24:59.991491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032099+0000","last_change":"2024-06-10T00:28:34.396792+0000","last_active":"2024-06-10T00:32:50.032099+0000","last_peered":"2024-06-10T00:32:50.032099+0000","last_clean":"2024-06-10T00:32:50.032099+0000","last_became_active":"2024-06-10T00:28:34.396442+0000","last_became_peered":"2024-06-10T00:28:34.396442+0000","last_unstale":"2024-06-10T00:32:50.032099+0000","last_undegraded":"2024-06-10T00:32:50.032099+0000","last_fullsized":"2024-06-10T00:32:50.032099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:46.436198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199145+0000","last_change":"2024-06-10T00:28:37.657698+0000","last_active":"2024-06-10T00:32:50.199145+0000","last_peered":"2024-06-10T00:32:50.199145+0000","last_clean":"2024-06-10T00:32:50.199145+0000","last_became_active":"2024-06-10T00:28:37.657364+0000","last_became_peered":"2024-06-10T00:28:37.657364+0000","last_unstale":"2024-06-10T00:32:50.199145+0000","last_undegraded":"2024-06-10T00:32:50.199145+0000","last_fullsized":"2024-06-10T00:32:50.199145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:23.097015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031651+0000","last_change":"2024-06-10T00:28:48.727567+0000","last_active":"2024-06-10T00:32:50.031651+0000","last_peered":"2024-06-10T00:32:50.031651+0000","last_clean":"2024-06-10T00:32:50.031651+0000","last_became_active":"2024-06-10T00:28:48.727108+0000","last_became_peered":"2024-06-10T00:28:48.727108+0000","last_unstale":"2024-06-10T00:32:50.031651+0000","last_undegraded":"2024-06-10T00:32:50.031651+0000","last_fullsized":"2024-06-10T00:32:50.031651+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:35:00.824177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921917+0000","last_change":"2024-06-10T00:28:34.446832+0000","last_active":"2024-06-10T00:32:49.921917+0000","last_peered":"2024-06-10T00:32:49.921917+0000","last_clean":"2024-06-10T00:32:49.921917+0000","last_became_active":"2024-06-10T00:28:34.446435+0000","last_became_peered":"2024-06-10T00:28:34.446435+0000","last_unstale":"2024-06-10T00:32:49.921917+0000","last_undegraded":"2024-06-10T00:32:49.921917+0000","last_fullsized":"2024-06-10T00:32:49.921917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:58.296674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042645+0000","last_change":"2024-06-10T00:30:24.229514+0000","last_active":"2024-06-10T00:32:50.042645+0000","last_peered":"2024-06-10T00:32:50.042645+0000","last_clean":"2024-06-10T00:32:50.042645+0000","last_became_active":"2024-06-10T00:30:24.229163+0000","last_became_peered":"2024-06-10T00:30:24.229163+0000","last_unstale":"2024-06-10T00:32:50.042645+0000","last_undegraded":"2024-06-10T00:32:50.042645+0000","last_fullsized":"2024-06-10T00:32:50.042645+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:52.914144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133795+0000","last_change":"2024-06-10T00:28:48.704683+0000","last_active":"2024-06-10T00:32:50.133795+0000","last_peered":"2024-06-10T00:32:50.133795+0000","last_clean":"2024-06-10T00:32:50.133795+0000","last_became_active":"2024-06-10T00:28:48.704297+0000","last_became_peered":"2024-06-10T00:28:48.704297+0000","last_unstale":"2024-06-10T00:32:50.133795+0000","last_undegraded":"2024-06-10T00:32:50.133795+0000","last_fullsized":"2024-06-10T00:32:50.133795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:44.980544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'8","reported_seq":253,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:59.144983+0000","last_change":"2024-06-10T00:28:34.393328+0000","last_active":"2024-06-10T00:32:59.144983+0000","last_peered":"2024-06-10T00:32:59.144983+0000","last_clean":"2024-06-10T00:32:59.144983+0000","last_became_active":"2024-06-10T00:28:34.391968+0000","last_became_peered":"2024-06-10T00:28:34.391968+0000","last_unstale":"2024-06-10T00:32:59.144983+0000","last_undegraded":"2024-06-10T00:32:59.144983+0000","last_fullsized":"2024-06-10T00:32:59.144983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:18.898180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1676,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":198,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042474+0000","last_change":"2024-06-10T00:28:37.652555+0000","last_active":"2024-06-10T00:32:50.042474+0000","last_peered":"2024-06-10T00:32:50.042474+0000","last_clean":"2024-06-10T00:32:50.042474+0000","last_became_active":"2024-06-10T00:28:37.651476+0000","last_became_peered":"2024-06-10T00:28:37.651476+0000","last_unstale":"2024-06-10T00:32:50.042474+0000","last_undegraded":"2024-06-10T00:32:50.042474+0000","last_fullsized":"2024-06-10T00:32:50.042474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:04:57.823224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134015+0000","last_change":"2024-06-10T00:28:48.727568+0000","last_active":"2024-06-10T00:32:50.134015+0000","last_peered":"2024-06-10T00:32:50.134015+0000","last_clean":"2024-06-10T00:32:50.134015+0000","last_became_active":"2024-06-10T00:28:48.727018+0000","last_became_peered":"2024-06-10T00:28:48.727018+0000","last_unstale":"2024-06-10T00:32:50.134015+0000","last_undegraded":"2024-06-10T00:32:50.134015+0000","last_fullsized":"2024-06-10T00:32:50.134015+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:02.025212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921381+0000","last_change":"2024-06-10T00:30:24.651947+0000","last_active":"2024-06-10T00:32:49.921381+0000","last_peered":"2024-06-10T00:32:49.921381+0000","last_clean":"2024-06-10T00:32:49.921381+0000","last_became_active":"2024-06-10T00:30:24.651640+0000","last_became_peered":"2024-06-10T00:30:24.651640+0000","last_unstale":"2024-06-10T00:32:49.921381+0000","last_undegraded":"2024-06-10T00:32:49.921381+0000","last_fullsized":"2024-06-10T00:32:49.921381+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:47.070568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"100'7","reported_seq":250,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:59.144331+0000","last_change":"2024-06-10T00:28:34.445501+0000","last_active":"2024-06-10T00:32:59.144331+0000","last_peered":"2024-06-10T00:32:59.144331+0000","last_clean":"2024-06-10T00:32:59.144331+0000","last_became_active":"2024-06-10T00:28:34.445128+0000","last_became_peered":"2024-06-10T00:28:34.445128+0000","last_unstale":"2024-06-10T00:32:59.144331+0000","last_undegraded":"2024-06-10T00:32:59.144331+0000","last_fullsized":"2024-06-10T00:32:59.144331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:59.231585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":196,"num_read_kb":196,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134703+0000","last_change":"2024-06-10T00:28:48.709852+0000","last_active":"2024-06-10T00:32:50.134703+0000","last_peered":"2024-06-10T00:32:50.134703+0000","last_clean":"2024-06-10T00:32:50.134703+0000","last_became_active":"2024-06-10T00:28:48.709423+0000","last_became_peered":"2024-06-10T00:28:48.709423+0000","last_unstale":"2024-06-10T00:32:50.134703+0000","last_undegraded":"2024-06-10T00:32:50.134703+0000","last_fullsized":"2024-06-10T00:32:50.134703+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:42.123563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031914+0000","last_change":"2024-06-10T00:28:37.657423+0000","last_active":"2024-06-10T00:32:50.031914+0000","last_peered":"2024-06-10T00:32:50.031914+0000","last_clean":"2024-06-10T00:32:50.031914+0000","last_became_active":"2024-06-10T00:28:37.657140+0000","last_became_peered":"2024-06-10T00:28:37.657140+0000","last_unstale":"2024-06-10T00:32:50.031914+0000","last_undegraded":"2024-06-10T00:32:50.031914+0000","last_fullsized":"2024-06-10T00:32:50.031914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:20:08.943066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922033+0000","last_change":"2024-06-10T00:28:34.392585+0000","last_active":"2024-06-10T00:32:49.922033+0000","last_peered":"2024-06-10T00:32:49.922033+0000","last_clean":"2024-06-10T00:32:49.922033+0000","last_became_active":"2024-06-10T00:28:34.392229+0000","last_became_peered":"2024-06-10T00:28:34.392229+0000","last_unstale":"2024-06-10T00:32:49.922033+0000","last_undegraded":"2024-06-10T00:32:49.922033+0000","last_fullsized":"2024-06-10T00:32:49.922033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:11.275761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"100'148","reported_seq":424,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:52.045212+0000","last_change":"2024-06-10T00:27:12.072577+0000","last_active":"2024-06-10T00:32:52.045212+0000","last_peered":"2024-06-10T00:32:52.045212+0000","last_clean":"2024-06-10T00:32:52.045212+0000","last_became_active":"2024-06-10T00:27:12.072193+0000","last_became_peered":"2024-06-10T00:27:12.072193+0000","last_unstale":"2024-06-10T00:32:52.045212+0000","last_undegraded":"2024-06-10T00:32:52.045212+0000","last_fullsized":"2024-06-10T00:32:52.045212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_clean_scrub_stamp":"2024-06-10T00:23:46.826438+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:57.782124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039093+0000","last_change":"2024-06-10T00:28:48.727913+0000","last_active":"2024-06-10T00:32:50.039093+0000","last_peered":"2024-06-10T00:32:50.039093+0000","last_clean":"2024-06-10T00:32:50.039093+0000","last_became_active":"2024-06-10T00:28:48.727402+0000","last_became_peered":"2024-06-10T00:28:48.727402+0000","last_unstale":"2024-06-10T00:32:50.039093+0000","last_undegraded":"2024-06-10T00:32:50.039093+0000","last_fullsized":"2024-06-10T00:32:50.039093+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:10:25.051296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132861+0000","last_change":"2024-06-10T00:28:37.659901+0000","last_active":"2024-06-10T00:32:50.132861+0000","last_peered":"2024-06-10T00:32:50.132861+0000","last_clean":"2024-06-10T00:32:50.132861+0000","last_became_active":"2024-06-10T00:28:37.659553+0000","last_became_peered":"2024-06-10T00:28:37.659553+0000","last_unstale":"2024-06-10T00:32:50.132861+0000","last_undegraded":"2024-06-10T00:32:50.132861+0000","last_fullsized":"2024-06-10T00:32:50.132861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:25.694823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134306+0000","last_change":"2024-06-10T00:28:34.398155+0000","last_active":"2024-06-10T00:32:50.134306+0000","last_peered":"2024-06-10T00:32:50.134306+0000","last_clean":"2024-06-10T00:32:50.134306+0000","last_became_active":"2024-06-10T00:28:34.397793+0000","last_became_peered":"2024-06-10T00:28:34.397793+0000","last_unstale":"2024-06-10T00:32:50.134306+0000","last_undegraded":"2024-06-10T00:32:50.134306+0000","last_fullsized":"2024-06-10T00:32:50.134306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:21.188139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198954+0000","last_change":"2024-06-10T00:28:48.709736+0000","last_active":"2024-06-10T00:32:50.198954+0000","last_peered":"2024-06-10T00:32:50.198954+0000","last_clean":"2024-06-10T00:32:50.198954+0000","last_became_active":"2024-06-10T00:28:48.709293+0000","last_became_peered":"2024-06-10T00:28:48.709293+0000","last_unstale":"2024-06-10T00:32:50.198954+0000","last_undegraded":"2024-06-10T00:32:50.198954+0000","last_fullsized":"2024-06-10T00:32:50.198954+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:30:04.840656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199015+0000","last_change":"2024-06-10T00:29:23.780030+0000","last_active":"2024-06-10T00:32:50.199015+0000","last_peered":"2024-06-10T00:32:50.199015+0000","last_clean":"2024-06-10T00:32:50.199015+0000","last_became_active":"2024-06-10T00:29:23.779589+0000","last_became_peered":"2024-06-10T00:29:23.779589+0000","last_unstale":"2024-06-10T00:32:50.199015+0000","last_undegraded":"2024-06-10T00:32:50.199015+0000","last_fullsized":"2024-06-10T00:32:50.199015+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:53.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133881+0000","last_change":"2024-06-10T00:28:37.658330+0000","last_active":"2024-06-10T00:32:50.133881+0000","last_peered":"2024-06-10T00:32:50.133881+0000","last_clean":"2024-06-10T00:32:50.133881+0000","last_became_active":"2024-06-10T00:28:37.657343+0000","last_became_peered":"2024-06-10T00:28:37.657343+0000","last_unstale":"2024-06-10T00:32:50.133881+0000","last_undegraded":"2024-06-10T00:32:50.133881+0000","last_fullsized":"2024-06-10T00:32:50.133881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:31:52.167402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031813+0000","last_change":"2024-06-10T00:28:48.727506+0000","last_active":"2024-06-10T00:32:50.031813+0000","last_peered":"2024-06-10T00:32:50.031813+0000","last_clean":"2024-06-10T00:32:50.031813+0000","last_became_active":"2024-06-10T00:28:48.727068+0000","last_became_peered":"2024-06-10T00:28:48.727068+0000","last_unstale":"2024-06-10T00:32:50.031813+0000","last_undegraded":"2024-06-10T00:32:50.031813+0000","last_fullsized":"2024-06-10T00:32:50.031813+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:42.147908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'51","reported_seq":122,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.151546+0000","last_change":"2024-06-10T00:28:34.399872+0000","last_active":"2024-06-10T00:32:50.151546+0000","last_peered":"2024-06-10T00:32:50.151546+0000","last_clean":"2024-06-10T00:32:50.151546+0000","last_became_active":"2024-06-10T00:28:34.399493+0000","last_became_peered":"2024-06-10T00:28:34.399493+0000","last_unstale":"2024-06-10T00:32:50.151546+0000","last_undegraded":"2024-06-10T00:32:50.151546+0000","last_fullsized":"2024-06-10T00:32:50.151546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:24:10.018110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032464+0000","last_change":"2024-06-10T00:28:37.656597+0000","last_active":"2024-06-10T00:32:50.032464+0000","last_peered":"2024-06-10T00:32:50.032464+0000","last_clean":"2024-06-10T00:32:50.032464+0000","last_became_active":"2024-06-10T00:28:37.654356+0000","last_became_peered":"2024-06-10T00:28:37.654356+0000","last_unstale":"2024-06-10T00:32:50.032464+0000","last_undegraded":"2024-06-10T00:32:50.032464+0000","last_fullsized":"2024-06-10T00:32:50.032464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:07:12.950683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920789+0000","last_change":"2024-06-10T00:28:48.702686+0000","last_active":"2024-06-10T00:32:49.920789+0000","last_peered":"2024-06-10T00:32:49.920789+0000","last_clean":"2024-06-10T00:32:49.920789+0000","last_became_active":"2024-06-10T00:28:48.702273+0000","last_became_peered":"2024-06-10T00:28:48.702273+0000","last_unstale":"2024-06-10T00:32:49.920789+0000","last_undegraded":"2024-06-10T00:32:49.920789+0000","last_fullsized":"2024-06-10T00:32:49.920789+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:00.078232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921789+0000","last_change":"2024-06-10T00:28:34.447549+0000","last_active":"2024-06-10T00:32:49.921789+0000","last_peered":"2024-06-10T00:32:49.921789+0000","last_clean":"2024-06-10T00:32:49.921789+0000","last_became_active":"2024-06-10T00:28:34.447197+0000","last_became_peered":"2024-06-10T00:28:34.447197+0000","last_unstale":"2024-06-10T00:32:49.921789+0000","last_undegraded":"2024-06-10T00:32:49.921789+0000","last_fullsized":"2024-06-10T00:32:49.921789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:47.525153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193483+0000","last_change":"2024-06-10T00:28:37.655585+0000","last_active":"2024-06-10T00:32:50.193483+0000","last_peered":"2024-06-10T00:32:50.193483+0000","last_clean":"2024-06-10T00:32:50.193483+0000","last_became_active":"2024-06-10T00:28:37.655225+0000","last_became_peered":"2024-06-10T00:28:37.655225+0000","last_unstale":"2024-06-10T00:32:50.193483+0000","last_undegraded":"2024-06-10T00:32:50.193483+0000","last_fullsized":"2024-06-10T00:32:50.193483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:57:25.184924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921606+0000","last_change":"2024-06-10T00:29:23.857447+0000","last_active":"2024-06-10T00:32:49.921606+0000","last_peered":"2024-06-10T00:32:49.921606+0000","last_clean":"2024-06-10T00:32:49.921606+0000","last_became_active":"2024-06-10T00:29:23.856943+0000","last_became_peered":"2024-06-10T00:29:23.856943+0000","last_unstale":"2024-06-10T00:32:49.921606+0000","last_undegraded":"2024-06-10T00:32:49.921606+0000","last_fullsized":"2024-06-10T00:32:49.921606+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:58:44.548818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188371+0000","last_change":"2024-06-10T00:28:34.445637+0000","last_active":"2024-06-10T00:32:50.188371+0000","last_peered":"2024-06-10T00:32:50.188371+0000","last_clean":"2024-06-10T00:32:50.188371+0000","last_became_active":"2024-06-10T00:28:34.445324+0000","last_became_peered":"2024-06-10T00:28:34.445324+0000","last_unstale":"2024-06-10T00:32:50.188371+0000","last_undegraded":"2024-06-10T00:32:50.188371+0000","last_fullsized":"2024-06-10T00:32:50.188371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:05.713557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031670+0000","last_change":"2024-06-10T00:28:37.659037+0000","last_active":"2024-06-10T00:32:50.031670+0000","last_peered":"2024-06-10T00:32:50.031670+0000","last_clean":"2024-06-10T00:32:50.031670+0000","last_became_active":"2024-06-10T00:28:37.658705+0000","last_became_peered":"2024-06-10T00:28:37.658705+0000","last_unstale":"2024-06-10T00:32:50.031670+0000","last_undegraded":"2024-06-10T00:32:50.031670+0000","last_fullsized":"2024-06-10T00:32:50.031670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:20:52.797237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922067+0000","last_change":"2024-06-10T00:28:48.704130+0000","last_active":"2024-06-10T00:32:49.922067+0000","last_peered":"2024-06-10T00:32:49.922067+0000","last_clean":"2024-06-10T00:32:49.922067+0000","last_became_active":"2024-06-10T00:28:48.703738+0000","last_became_peered":"2024-06-10T00:28:48.703738+0000","last_unstale":"2024-06-10T00:32:49.922067+0000","last_undegraded":"2024-06-10T00:32:49.922067+0000","last_fullsized":"2024-06-10T00:32:49.922067+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:01:40.436710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032084+0000","last_change":"2024-06-10T00:28:34.386667+0000","last_active":"2024-06-10T00:32:50.032084+0000","last_peered":"2024-06-10T00:32:50.032084+0000","last_clean":"2024-06-10T00:32:50.032084+0000","last_became_active":"2024-06-10T00:28:34.386363+0000","last_became_peered":"2024-06-10T00:28:34.386363+0000","last_unstale":"2024-06-10T00:32:50.032084+0000","last_undegraded":"2024-06-10T00:32:50.032084+0000","last_fullsized":"2024-06-10T00:32:50.032084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:11.850162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198807+0000","last_change":"2024-06-10T00:28:37.664754+0000","last_active":"2024-06-10T00:32:50.198807+0000","last_peered":"2024-06-10T00:32:50.198807+0000","last_clean":"2024-06-10T00:32:50.198807+0000","last_became_active":"2024-06-10T00:28:37.658574+0000","last_became_peered":"2024-06-10T00:28:37.658574+0000","last_unstale":"2024-06-10T00:32:50.198807+0000","last_undegraded":"2024-06-10T00:32:50.198807+0000","last_fullsized":"2024-06-10T00:32:50.198807+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:16:47.437822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921563+0000","last_change":"2024-06-10T00:28:48.706560+0000","last_active":"2024-06-10T00:32:49.921563+0000","last_peered":"2024-06-10T00:32:49.921563+0000","last_clean":"2024-06-10T00:32:49.921563+0000","last_became_active":"2024-06-10T00:28:48.705986+0000","last_became_peered":"2024-06-10T00:28:48.705986+0000","last_unstale":"2024-06-10T00:32:49.921563+0000","last_undegraded":"2024-06-10T00:32:49.921563+0000","last_fullsized":"2024-06-10T00:32:49.921563+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:21:26.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.892250+0000","last_change":"2024-06-10T00:28:34.394509+0000","last_active":"2024-06-10T00:32:50.892250+0000","last_peered":"2024-06-10T00:32:50.892250+0000","last_clean":"2024-06-10T00:32:50.892250+0000","last_became_active":"2024-06-10T00:28:34.394105+0000","last_became_peered":"2024-06-10T00:28:34.394105+0000","last_unstale":"2024-06-10T00:32:50.892250+0000","last_undegraded":"2024-06-10T00:32:50.892250+0000","last_fullsized":"2024-06-10T00:32:50.892250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:13.415544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032324+0000","last_change":"2024-06-10T00:28:37.653192+0000","last_active":"2024-06-10T00:32:50.032324+0000","last_peered":"2024-06-10T00:32:50.032324+0000","last_clean":"2024-06-10T00:32:50.032324+0000","last_became_active":"2024-06-10T00:28:37.652868+0000","last_became_peered":"2024-06-10T00:28:37.652868+0000","last_unstale":"2024-06-10T00:32:50.032324+0000","last_undegraded":"2024-06-10T00:32:50.032324+0000","last_fullsized":"2024-06-10T00:32:50.032324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:21.614915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921695+0000","last_change":"2024-06-10T00:28:48.727615+0000","last_active":"2024-06-10T00:32:49.921695+0000","last_peered":"2024-06-10T00:32:49.921695+0000","last_clean":"2024-06-10T00:32:49.921695+0000","last_became_active":"2024-06-10T00:28:48.727190+0000","last_became_peered":"2024-06-10T00:28:48.727190+0000","last_unstale":"2024-06-10T00:32:49.921695+0000","last_undegraded":"2024-06-10T00:32:49.921695+0000","last_fullsized":"2024-06-10T00:32:49.921695+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:57.498183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031723+0000","last_change":"2024-06-10T00:29:23.784131+0000","last_active":"2024-06-10T00:32:50.031723+0000","last_peered":"2024-06-10T00:32:50.031723+0000","last_clean":"2024-06-10T00:32:50.031723+0000","last_became_active":"2024-06-10T00:29:23.780909+0000","last_became_peered":"2024-06-10T00:29:23.780909+0000","last_unstale":"2024-06-10T00:32:50.031723+0000","last_undegraded":"2024-06-10T00:32:50.031723+0000","last_fullsized":"2024-06-10T00:32:50.031723+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.774217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134422+0000","last_change":"2024-06-10T00:28:37.660399+0000","last_active":"2024-06-10T00:32:50.134422+0000","last_peered":"2024-06-10T00:32:50.134422+0000","last_clean":"2024-06-10T00:32:50.134422+0000","last_became_active":"2024-06-10T00:28:37.660069+0000","last_became_peered":"2024-06-10T00:28:37.660069+0000","last_unstale":"2024-06-10T00:32:50.134422+0000","last_undegraded":"2024-06-10T00:32:50.134422+0000","last_fullsized":"2024-06-10T00:32:50.134422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:59.727883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188428+0000","last_change":"2024-06-10T00:28:48.713267+0000","last_active":"2024-06-10T00:32:50.188428+0000","last_peered":"2024-06-10T00:32:50.188428+0000","last_clean":"2024-06-10T00:32:50.188428+0000","last_became_active":"2024-06-10T00:28:48.712870+0000","last_became_peered":"2024-06-10T00:28:48.712870+0000","last_unstale":"2024-06-10T00:32:50.188428+0000","last_undegraded":"2024-06-10T00:32:50.188428+0000","last_fullsized":"2024-06-10T00:32:50.188428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:59.210101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'6","reported_seq":250,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:59.174585+0000","last_change":"2024-06-10T00:28:34.448652+0000","last_active":"2024-06-10T00:32:59.174585+0000","last_peered":"2024-06-10T00:32:59.174585+0000","last_clean":"2024-06-10T00:32:59.174585+0000","last_became_active":"2024-06-10T00:28:34.448243+0000","last_became_peered":"2024-06-10T00:28:34.448243+0000","last_unstale":"2024-06-10T00:32:59.174585+0000","last_undegraded":"2024-06-10T00:32:59.174585+0000","last_fullsized":"2024-06-10T00:32:59.174585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:51:40.003429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":198,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.135023+0000","last_change":"2024-06-10T00:28:37.662307+0000","last_active":"2024-06-10T00:32:50.135023+0000","last_peered":"2024-06-10T00:32:50.135023+0000","last_clean":"2024-06-10T00:32:50.135023+0000","last_became_active":"2024-06-10T00:28:37.661727+0000","last_became_peered":"2024-06-10T00:28:37.661727+0000","last_unstale":"2024-06-10T00:32:50.135023+0000","last_undegraded":"2024-06-10T00:32:50.135023+0000","last_fullsized":"2024-06-10T00:32:50.135023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:23.989429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192376+0000","last_change":"2024-06-10T00:29:23.857626+0000","last_active":"2024-06-10T00:32:50.192376+0000","last_peered":"2024-06-10T00:32:50.192376+0000","last_clean":"2024-06-10T00:32:50.192376+0000","last_became_active":"2024-06-10T00:29:23.857057+0000","last_became_peered":"2024-06-10T00:29:23.857057+0000","last_unstale":"2024-06-10T00:32:50.192376+0000","last_undegraded":"2024-06-10T00:32:50.192376+0000","last_fullsized":"2024-06-10T00:32:50.192376+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:25:10.340053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'41","reported_seq":95,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.145625+0000","last_change":"2024-06-10T00:28:34.399253+0000","last_active":"2024-06-10T00:32:50.145625+0000","last_peered":"2024-06-10T00:32:50.145625+0000","last_clean":"2024-06-10T00:32:50.145625+0000","last_became_active":"2024-06-10T00:28:34.397763+0000","last_became_peered":"2024-06-10T00:28:34.397763+0000","last_unstale":"2024-06-10T00:32:50.145625+0000","last_undegraded":"2024-06-10T00:32:50.145625+0000","last_fullsized":"2024-06-10T00:32:50.145625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:05:57.371625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134695+0000","last_change":"2024-06-10T00:28:37.658027+0000","last_active":"2024-06-10T00:32:50.134695+0000","last_peered":"2024-06-10T00:32:50.134695+0000","last_clean":"2024-06-10T00:32:50.134695+0000","last_became_active":"2024-06-10T00:28:37.655513+0000","last_became_peered":"2024-06-10T00:28:37.655513+0000","last_unstale":"2024-06-10T00:32:50.134695+0000","last_undegraded":"2024-06-10T00:32:50.134695+0000","last_fullsized":"2024-06-10T00:32:50.134695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:50:23.525855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038558+0000","last_change":"2024-06-10T00:28:48.706997+0000","last_active":"2024-06-10T00:32:50.038558+0000","last_peered":"2024-06-10T00:32:50.038558+0000","last_clean":"2024-06-10T00:32:50.038558+0000","last_became_active":"2024-06-10T00:28:48.706597+0000","last_became_peered":"2024-06-10T00:28:48.706597+0000","last_unstale":"2024-06-10T00:32:50.038558+0000","last_undegraded":"2024-06-10T00:32:50.038558+0000","last_fullsized":"2024-06-10T00:32:50.038558+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:16.385312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133886+0000","last_change":"2024-06-10T00:29:23.785774+0000","last_active":"2024-06-10T00:32:50.133886+0000","last_peered":"2024-06-10T00:32:50.133886+0000","last_clean":"2024-06-10T00:32:50.133886+0000","last_became_active":"2024-06-10T00:29:23.781768+0000","last_became_peered":"2024-06-10T00:29:23.781768+0000","last_unstale":"2024-06-10T00:32:50.133886+0000","last_undegraded":"2024-06-10T00:32:50.133886+0000","last_fullsized":"2024-06-10T00:32:50.133886+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:27.348264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134953+0000","last_change":"2024-06-10T00:28:37.657820+0000","last_active":"2024-06-10T00:32:50.134953+0000","last_peered":"2024-06-10T00:32:50.134953+0000","last_clean":"2024-06-10T00:32:50.134953+0000","last_became_active":"2024-06-10T00:28:37.657487+0000","last_became_peered":"2024-06-10T00:28:37.657487+0000","last_unstale":"2024-06-10T00:32:50.134953+0000","last_undegraded":"2024-06-10T00:32:50.134953+0000","last_fullsized":"2024-06-10T00:32:50.134953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:44:35.930401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198953+0000","last_change":"2024-06-10T00:28:37.664543+0000","last_active":"2024-06-10T00:32:50.198953+0000","last_peered":"2024-06-10T00:32:50.198953+0000","last_clean":"2024-06-10T00:32:50.198953+0000","last_became_active":"2024-06-10T00:28:37.659015+0000","last_became_peered":"2024-06-10T00:28:37.659015+0000","last_unstale":"2024-06-10T00:32:50.198953+0000","last_undegraded":"2024-06-10T00:32:50.198953+0000","last_fullsized":"2024-06-10T00:32:50.198953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:06.555472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921072+0000","last_change":"2024-06-10T00:28:37.659389+0000","last_active":"2024-06-10T00:32:49.921072+0000","last_peered":"2024-06-10T00:32:49.921072+0000","last_clean":"2024-06-10T00:32:49.921072+0000","last_became_active":"2024-06-10T00:28:37.658990+0000","last_became_peered":"2024-06-10T00:28:37.658990+0000","last_unstale":"2024-06-10T00:32:49.921072+0000","last_undegraded":"2024-06-10T00:32:49.921072+0000","last_fullsized":"2024-06-10T00:32:49.921072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:29.532779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920823+0000","last_change":"2024-06-10T00:28:37.655621+0000","last_active":"2024-06-10T00:32:49.920823+0000","last_peered":"2024-06-10T00:32:49.920823+0000","last_clean":"2024-06-10T00:32:49.920823+0000","last_became_active":"2024-06-10T00:28:37.653599+0000","last_became_peered":"2024-06-10T00:28:37.653599+0000","last_unstale":"2024-06-10T00:32:49.920823+0000","last_undegraded":"2024-06-10T00:32:49.920823+0000","last_fullsized":"2024-06-10T00:32:49.920823+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:22.165851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192846+0000","last_change":"2024-06-10T00:29:23.782727+0000","last_active":"2024-06-10T00:32:50.192846+0000","last_peered":"2024-06-10T00:32:50.192846+0000","last_clean":"2024-06-10T00:32:50.192846+0000","last_became_active":"2024-06-10T00:29:23.782309+0000","last_became_peered":"2024-06-10T00:29:23.782309+0000","last_unstale":"2024-06-10T00:32:50.192846+0000","last_undegraded":"2024-06-10T00:32:50.192846+0000","last_fullsized":"2024-06-10T00:32:50.192846+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:20.585955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133931+0000","last_change":"2024-06-10T00:28:37.646750+0000","last_active":"2024-06-10T00:32:50.133931+0000","last_peered":"2024-06-10T00:32:50.133931+0000","last_clean":"2024-06-10T00:32:50.133931+0000","last_became_active":"2024-06-10T00:28:37.646372+0000","last_became_peered":"2024-06-10T00:28:37.646372+0000","last_unstale":"2024-06-10T00:32:50.133931+0000","last_undegraded":"2024-06-10T00:32:50.133931+0000","last_fullsized":"2024-06-10T00:32:50.133931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:45.762956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199378+0000","last_change":"2024-06-10T00:28:37.662278+0000","last_active":"2024-06-10T00:32:50.199378+0000","last_peered":"2024-06-10T00:32:50.199378+0000","last_clean":"2024-06-10T00:32:50.199378+0000","last_became_active":"2024-06-10T00:28:37.661855+0000","last_became_peered":"2024-06-10T00:28:37.661855+0000","last_unstale":"2024-06-10T00:32:50.199378+0000","last_undegraded":"2024-06-10T00:32:50.199378+0000","last_fullsized":"2024-06-10T00:32:50.199378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:20.074208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921342+0000","last_change":"2024-06-10T00:28:37.656738+0000","last_active":"2024-06-10T00:32:49.921342+0000","last_peered":"2024-06-10T00:32:49.921342+0000","last_clean":"2024-06-10T00:32:49.921342+0000","last_became_active":"2024-06-10T00:28:37.656375+0000","last_became_peered":"2024-06-10T00:28:37.656375+0000","last_unstale":"2024-06-10T00:32:49.921342+0000","last_undegraded":"2024-06-10T00:32:49.921342+0000","last_fullsized":"2024-06-10T00:32:49.921342+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:49:17.779670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032601+0000","last_change":"2024-06-10T00:28:37.656849+0000","last_active":"2024-06-10T00:32:50.032601+0000","last_peered":"2024-06-10T00:32:50.032601+0000","last_clean":"2024-06-10T00:32:50.032601+0000","last_became_active":"2024-06-10T00:28:37.656261+0000","last_became_peered":"2024-06-10T00:28:37.656261+0000","last_unstale":"2024-06-10T00:32:50.032601+0000","last_undegraded":"2024-06-10T00:32:50.032601+0000","last_fullsized":"2024-06-10T00:32:50.032601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:13.152165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038730+0000","last_change":"2024-06-10T00:28:37.648384+0000","last_active":"2024-06-10T00:32:50.038730+0000","last_peered":"2024-06-10T00:32:50.038730+0000","last_clean":"2024-06-10T00:32:50.038730+0000","last_became_active":"2024-06-10T00:28:37.648009+0000","last_became_peered":"2024-06-10T00:28:37.648009+0000","last_unstale":"2024-06-10T00:32:50.038730+0000","last_undegraded":"2024-06-10T00:32:50.038730+0000","last_fullsized":"2024-06-10T00:32:50.038730+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:45:55.808932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032228+0000","last_change":"2024-06-10T00:28:37.655828+0000","last_active":"2024-06-10T00:32:50.032228+0000","last_peered":"2024-06-10T00:32:50.032228+0000","last_clean":"2024-06-10T00:32:50.032228+0000","last_became_active":"2024-06-10T00:28:37.655269+0000","last_became_peered":"2024-06-10T00:28:37.655269+0000","last_unstale":"2024-06-10T00:32:50.032228+0000","last_undegraded":"2024-06-10T00:32:50.032228+0000","last_fullsized":"2024-06-10T00:32:50.032228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:56.618469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188607+0000","last_change":"2024-06-10T00:28:37.663756+0000","last_active":"2024-06-10T00:32:50.188607+0000","last_peered":"2024-06-10T00:32:50.188607+0000","last_clean":"2024-06-10T00:32:50.188607+0000","last_became_active":"2024-06-10T00:28:37.663380+0000","last_became_peered":"2024-06-10T00:28:37.663380+0000","last_unstale":"2024-06-10T00:32:50.188607+0000","last_undegraded":"2024-06-10T00:32:50.188607+0000","last_fullsized":"2024-06-10T00:32:50.188607+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:45.534648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133127+0000","last_change":"2024-06-10T00:28:37.655246+0000","last_active":"2024-06-10T00:32:50.133127+0000","last_peered":"2024-06-10T00:32:50.133127+0000","last_clean":"2024-06-10T00:32:50.133127+0000","last_became_active":"2024-06-10T00:28:37.654605+0000","last_became_peered":"2024-06-10T00:28:37.654605+0000","last_unstale":"2024-06-10T00:32:50.133127+0000","last_undegraded":"2024-06-10T00:32:50.133127+0000","last_fullsized":"2024-06-10T00:32:50.133127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:15.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199118+0000","last_change":"2024-06-10T00:28:37.662485+0000","last_active":"2024-06-10T00:32:50.199118+0000","last_peered":"2024-06-10T00:32:50.199118+0000","last_clean":"2024-06-10T00:32:50.199118+0000","last_became_active":"2024-06-10T00:28:37.661900+0000","last_became_peered":"2024-06-10T00:28:37.661900+0000","last_unstale":"2024-06-10T00:32:50.199118+0000","last_undegraded":"2024-06-10T00:32:50.199118+0000","last_fullsized":"2024-06-10T00:32:50.199118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:30:03.210634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604313,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2093,"num_read_kb":1168,"num_write":386,"num_write_kb":1478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1812939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":311,"ondisk_log_size":311,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383748,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1175552,"data_stored":581073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645384,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28968,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716184,"statfs":{"total":95995035648,"available":95965372416,"internally_reserved":0,"allocated":1998848,"data_stored":1369183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808909,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1585152,"data_stored":977723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070548,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36860,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708292,"statfs":{"total":95995035648,"available":95957291008,"internally_reserved":0,"allocated":1560576,"data_stored":973221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332184,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1781760,"data_stored":1180215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495709,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":577003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36556,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708596,"statfs":{"total":95995035648,"available":95957602304,"internally_reserved":0,"allocated":1245184,"data_stored":648070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36500,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708652,"statfs":{"total":95995035648,"available":95957659648,"internally_reserved":0,"allocated":1187840,"data_stored":577761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36564,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708588,"statfs":{"total":95995035648,"available":95957594112,"internally_reserved":0,"allocated":1257472,"data_stored":648231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182508,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1703936,"data_stored":1100439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920871,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36484,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708668,"statfs":{"total":95995035648,"available":95957676032,"internally_reserved":0,"allocated":1171456,"data_stored":580910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757346,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28892,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716260,"statfs":{"total":95995035648,"available":95965450240,"internally_reserved":0,"allocated":1921024,"data_stored":1305298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":135641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":203622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:33:07.078 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T00:33:07.302 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:07 smithi062 ceph-mon[32390]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:33:07.302 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:07 smithi062 ceph-mon[32390]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 21 op/s 2024-06-10T00:33:07.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.478+0000 7f360f039700 1 -- 172.21.15.62:0/798934358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3608101680 msgr2=0x7f3608101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:07.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.478+0000 7f360f039700 1 --2- 172.21.15.62:0/798934358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3608101680 0x7f3608101a60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0099c0 tx=0x7f35fc009cd0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.478+0000 7f360f039700 1 -- 172.21.15.62:0/798934358 shutdown_connections 2024-06-10T00:33:07.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.478+0000 7f360f039700 1 --2- 172.21.15.62:0/798934358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3608101680 0x7f3608101a60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.478+0000 7f360f039700 1 --2- 172.21.15.62:0/798934358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36081066d0 0x7f360810e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.478+0000 7f360f039700 1 --2- 172.21.15.62:0/798934358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3608102030 0x7f3608105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.478+0000 7f360f039700 1 -- 172.21.15.62:0/798934358 >> 172.21.15.62:0/798934358 conn(0x7f36080773e0 msgr2=0x7f36080777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:07.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.478+0000 7f360f039700 1 -- 172.21.15.62:0/798934358 shutdown_connections 2024-06-10T00:33:07.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.478+0000 7f360f039700 1 -- 172.21.15.62:0/798934358 wait complete. 2024-06-10T00:33:07.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.478+0000 7f360f039700 1 Processor -- start 2024-06-10T00:33:07.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360f039700 1 -- start start 2024-06-10T00:33:07.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360f039700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3608101680 0x7f3608103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360f039700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3608102030 0x7f36081035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360f039700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36081066d0 0x7f3608103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360f039700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36081116b0 con 0x7f36081066d0 2024-06-10T00:33:07.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360f039700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3608111530 con 0x7f3608101680 2024-06-10T00:33:07.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360f039700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3608111830 con 0x7f3608102030 2024-06-10T00:33:07.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360d5d6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36081066d0 0x7f3608103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360cdd5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3608101680 0x7f3608103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360d5d6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36081066d0 0x7f3608103b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48244/0 (socket says 172.21.15.62:48244) 2024-06-10T00:33:07.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360cdd5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3608101680 0x7f3608103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:58126/0 (socket says 172.21.15.62:58126) 2024-06-10T00:33:07.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360d5d6700 1 -- 172.21.15.62:0/3852191478 learned_addr learned my addr 172.21.15.62:0/3852191478 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f3607fff700 1 --2- 172.21.15.62:0/3852191478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3608102030 0x7f36081035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360d5d6700 1 -- 172.21.15.62:0/3852191478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3608102030 msgr2=0x7f36081035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360d5d6700 1 --2- 172.21.15.62:0/3852191478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3608102030 0x7f36081035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360d5d6700 1 -- 172.21.15.62:0/3852191478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3608101680 msgr2=0x7f3608103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360d5d6700 1 --2- 172.21.15.62:0/3852191478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3608101680 0x7f3608103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360d5d6700 1 -- 172.21.15.62:0/3852191478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36081a71f0 con 0x7f36081066d0 2024-06-10T00:33:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360cdd5700 1 --2- 172.21.15.62:0/3852191478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3608101680 0x7f3608103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.479+0000 7f360d5d6700 1 --2- 172.21.15.62:0/3852191478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36081066d0 0x7f3608103b10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f360000ef30 tx=0x7f360000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.480+0000 7f3605ffb700 1 -- 172.21.15.62:0/3852191478 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36000092e0 con 0x7f36081066d0 2024-06-10T00:33:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.480+0000 7f360f039700 1 -- 172.21.15.62:0/3852191478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36081a7480 con 0x7f36081066d0 2024-06-10T00:33:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.480+0000 7f3605ffb700 1 -- 172.21.15.62:0/3852191478 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3600004bb0 con 0x7f36081066d0 2024-06-10T00:33:07.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.480+0000 7f3605ffb700 1 -- 172.21.15.62:0/3852191478 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600004280 con 0x7f36081066d0 2024-06-10T00:33:07.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.480+0000 7f360f039700 1 -- 172.21.15.62:0/3852191478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36081a7970 con 0x7f36081066d0 2024-06-10T00:33:07.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.483+0000 7f3605ffb700 1 -- 172.21.15.62:0/3852191478 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f360001b030 con 0x7f36081066d0 2024-06-10T00:33:07.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.483+0000 7f3605ffb700 1 --2- 172.21.15.62:0/3852191478 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f35f80a2e20 0x7f35f80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.484+0000 7f360cdd5700 1 --2- 172.21.15.62:0/3852191478 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f35f80a2e20 0x7f35f80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.484+0000 7f3605ffb700 1 -- 172.21.15.62:0/3852191478 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f36000cf510 con 0x7f36081066d0 2024-06-10T00:33:07.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.484+0000 7f360cdd5700 1 --2- 172.21.15.62:0/3852191478 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f35f80a2e20 0x7f35f80a52e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35fc000c00 tx=0x7f35fc005fb0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:07.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.484+0000 7f36198bcb80 1 Processor -- start 2024-06-10T00:33:07.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.484+0000 7f36198bcb80 1 -- start start 2024-06-10T00:33:07.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f36198bcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7dec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f36198bcb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7dec880 0x5590e7de6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f36198bcb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7de67a0 0x5590e7de6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f36198bcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5590e7cd8f60 con 0x5590e7debf60 2024-06-10T00:33:07.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f36198bcb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5590e7d37850 con 0x5590e7dec880 2024-06-10T00:33:07.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f36198bcb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5590e7c9f460 con 0x5590e7de67a0 2024-06-10T00:33:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f3607fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7dec880 0x5590e7de6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f3607fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7dec880 0x5590e7de6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:58136/0 (socket says 172.21.15.62:58136) 2024-06-10T00:33:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f3607fff700 1 -- 172.21.15.62:0/1237829857 learned_addr learned my addr 172.21.15.62:0/1237829857 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f360cdd5700 1 --2- 172.21.15.62:0/1237829857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7dec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f3607fff700 1 -- 172.21.15.62:0/1237829857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7de67a0 msgr2=0x5590e7de6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f360d5d6700 1 --2- 172.21.15.62:0/1237829857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7de67a0 0x5590e7de6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f3607fff700 1 --2- 172.21.15.62:0/1237829857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7de67a0 0x5590e7de6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f3607fff700 1 -- 172.21.15.62:0/1237829857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 msgr2=0x5590e7dec340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f3607fff700 1 --2- 172.21.15.62:0/1237829857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7dec340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f3607fff700 1 -- 172.21.15.62:0/1237829857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590e7c868d0 con 0x5590e7dec880 2024-06-10T00:33:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.485+0000 7f360cdd5700 1 --2- 172.21.15.62:0/1237829857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7dec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.486+0000 7f360d5d6700 1 --2- 172.21.15.62:0/1237829857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7de67a0 0x5590e7de6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:33:07.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.486+0000 7f3607fff700 1 --2- 172.21.15.62:0/1237829857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7dec880 0x5590e7de6260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f35f400bcb0 tx=0x7f35f400f360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b633e9413b6201d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:07.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.486+0000 7f35f2ffd700 1 -- 172.21.15.62:0/1237829857 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4015070 con 0x5590e7dec880 2024-06-10T00:33:07.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.486+0000 7f35f2ffd700 1 -- 172.21.15.62:0/1237829857 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f400fbe0 con 0x5590e7dec880 2024-06-10T00:33:07.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.486+0000 7f35f2ffd700 1 -- 172.21.15.62:0/1237829857 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f40116a0 con 0x5590e7dec880 2024-06-10T00:33:07.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.487+0000 7f36198bcb80 1 -- 172.21.15.62:0/1237829857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7dec880 msgr2=0x5590e7de6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:07.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.487+0000 7f36198bcb80 1 --2- 172.21.15.62:0/1237829857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7dec880 0x5590e7de6260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f35f400bcb0 tx=0x7f35f400f360 comp rx=0 tx=0).stop 2024-06-10T00:33:07.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.487+0000 7f36198bcb80 1 -- 172.21.15.62:0/1237829857 shutdown_connections 2024-06-10T00:33:07.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.487+0000 7f36198bcb80 1 --2- 172.21.15.62:0/1237829857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7dec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.487+0000 7f36198bcb80 1 --2- 172.21.15.62:0/1237829857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7de67a0 0x5590e7de6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.487+0000 7f36198bcb80 1 --2- 172.21.15.62:0/1237829857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7dec880 0x5590e7de6260 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.487+0000 7f36198bcb80 1 -- 172.21.15.62:0/1237829857 >> 172.21.15.62:0/1237829857 conn(0x5590e7d13080 msgr2=0x5590e7de1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:07.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.487+0000 7f36198bcb80 1 -- 172.21.15.62:0/1237829857 shutdown_connections 2024-06-10T00:33:07.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.487+0000 7f36198bcb80 1 -- 172.21.15.62:0/1237829857 wait complete. 2024-06-10T00:33:07.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.487+0000 7f36198bcb80 1 -- start start 2024-06-10T00:33:07.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.488+0000 7f36198bcb80 1 Processor -- start 2024-06-10T00:33:07.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.488+0000 7f36198bcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7de67a0 0x5590e7df2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.488+0000 7f36198bcb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7debf60 0x5590e7def8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.488+0000 7f36198bcb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7defde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.488+0000 7f36198bcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5590e7cd7fc0 con 0x5590e7de67a0 2024-06-10T00:33:07.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.488+0000 7f36198bcb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5590e7d33810 con 0x5590e7debf60 2024-06-10T00:33:07.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.488+0000 7f36198bcb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5590e7dfba70 con 0x5590e7dec880 2024-06-10T00:33:07.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.488+0000 7f36198bcb80 10 client.0 _ll_register_callbacks cb 0x5590e7ccd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:07.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.488+0000 7f3607fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7debf60 0x5590e7def8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.488+0000 7f360cdd5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7de67a0 0x5590e7df2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.488+0000 7f3607fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7debf60 0x5590e7def8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:58142/0 (socket says 172.21.15.62:58142) 2024-06-10T00:33:07.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.488+0000 7f3607fff700 1 -- 172.21.15.62:0/2883302591 learned_addr learned my addr 172.21.15.62:0/2883302591 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:07.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.488+0000 7f360d5d6700 1 --2- 172.21.15.62:0/2883302591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7defde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f3607fff700 1 -- 172.21.15.62:0/2883302591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 msgr2=0x5590e7defde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:07.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f3607fff700 1 --2- 172.21.15.62:0/2883302591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7defde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f3607fff700 1 -- 172.21.15.62:0/2883302591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7de67a0 msgr2=0x5590e7df2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:07.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f3607fff700 1 --2- 172.21.15.62:0/2883302591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7de67a0 0x5590e7df2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f3607fff700 1 -- 172.21.15.62:0/2883302591 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590e7cfd750 con 0x5590e7debf60 2024-06-10T00:33:07.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f360d5d6700 1 --2- 172.21.15.62:0/2883302591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7defde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:33:07.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f3607fff700 1 --2- 172.21.15.62:0/2883302591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7debf60 0x5590e7def8a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f35f4005d90 tx=0x7f35f401aee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ccaacb3692f867 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:07.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f35e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:33:07.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f35e7fff700 1 -- 172.21.15.62:0/2883302591 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4028060 con 0x5590e7debf60 2024-06-10T00:33:07.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f36198bcb80 1 -- 172.21.15.62:0/2883302591 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f400fbe0 con 0x5590e7debf60 2024-06-10T00:33:07.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f35e7fff700 1 -- 172.21.15.62:0/2883302591 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f35f401a670 con 0x5590e7debf60 2024-06-10T00:33:07.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f35e7fff700 1 -- 172.21.15.62:0/2883302591 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f35f4015070 con 0x5590e7debf60 2024-06-10T00:33:07.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f36198bcb80 10 client.25151 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:07.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f36198bcb80 1 -- 172.21.15.62:0/2883302591 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5590e7e06160 con 0x5590e7debf60 2024-06-10T00:33:07.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.489+0000 7f35e7fff700 1 -- 172.21.15.62:0/2883302591 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f400fbe0 con 0x5590e7debf60 2024-06-10T00:33:07.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.490+0000 7f35e7fff700 1 -- 172.21.15.62:0/2883302591 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f35f4004980 con 0x5590e7debf60 2024-06-10T00:33:07.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.490+0000 7f36198bcb80 10 client.25151 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:07.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.490+0000 7f36198bcb80 10 client.25151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:33:07.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.490+0000 7f36198bcb80 20 client.25151 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:07.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.490+0000 7f36198bcb80 1 --2- 172.21.15.62:0/2883302591 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590e7e08690 0x5590e7e0aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.490+0000 7f36198bcb80 4 client.25151 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:33:07.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.490+0000 7f36198bcb80 1 -- 172.21.15.62:0/2883302591 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590e7d44d50 con 0x5590e7e08690 2024-06-10T00:33:07.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.490+0000 7f360cdd5700 1 --2- 172.21.15.62:0/2883302591 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590e7e08690 0x5590e7e0aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.490+0000 7f360cdd5700 1 --2- 172.21.15.62:0/2883302591 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590e7e08690 0x5590e7e0aa80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36153af1ff4d9aba server_cookie=3707f4e77a9ffd2a in_seq=0 out_seq=0 2024-06-10T00:33:07.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.490+0000 7f35e7fff700 10 client.25151 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:07.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.491+0000 7f35e7fff700 1 -- 172.21.15.62:0/2883302591 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5590e7d44d50 con 0x5590e7e08690 2024-06-10T00:33:07.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.491+0000 7f35e7fff700 10 client.25151 handle_command_reply: tid=0 2024-06-10T00:33:07.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.491+0000 7f36198bcb80 1 client.25151 shutdown 2024-06-10T00:33:07.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.491+0000 7f36198bcb80 1 -- 172.21.15.62:0/2883302591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7debf60 msgr2=0x5590e7def8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:07.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.491+0000 7f36198bcb80 1 --2- 172.21.15.62:0/2883302591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7debf60 0x5590e7def8a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f35f4005d90 tx=0x7f35f401aee0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.491+0000 7f36198bcb80 1 -- 172.21.15.62:0/2883302591 shutdown_connections 2024-06-10T00:33:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.491+0000 7f36198bcb80 1 --2- 172.21.15.62:0/2883302591 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590e7e08690 0x5590e7e0aa80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.491+0000 7f36198bcb80 1 --2- 172.21.15.62:0/2883302591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7de67a0 0x5590e7df2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.491+0000 7f36198bcb80 1 --2- 172.21.15.62:0/2883302591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7defde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.491+0000 7f36198bcb80 1 --2- 172.21.15.62:0/2883302591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7debf60 0x5590e7def8a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.491+0000 7f35e7fff700 0 client.25151 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.491+0000 7f36198bcb80 1 -- 172.21.15.62:0/2883302591 >> 172.21.15.62:0/2883302591 conn(0x5590e7d13080 msgr2=0x5590e7de0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.492+0000 7f36198bcb80 1 -- 172.21.15.62:0/2883302591 shutdown_connections 2024-06-10T00:33:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.492+0000 7f36198bcb80 1 -- 172.21.15.62:0/2883302591 wait complete. 2024-06-10T00:33:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.492+0000 7f36198bcb80 20 client.25151 trim_cache size 0 max 16384 2024-06-10T00:33:07.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f36198bcb80 1 Processor -- start 2024-06-10T00:33:07.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f36198bcb80 1 -- start start 2024-06-10T00:33:07.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f36198bcb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7de67a0 0x5590e7de0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f36198bcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7de13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f36198bcb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7deb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f36198bcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36081ac4f0 con 0x5590e7debf60 2024-06-10T00:33:07.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f36198bcb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36081ae030 con 0x5590e7de67a0 2024-06-10T00:33:07.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f36198bcb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36081acbc0 con 0x5590e7dec880 2024-06-10T00:33:07.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f360cdd5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7de67a0 0x5590e7de0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f360cdd5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7de67a0 0x5590e7de0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:58146/0 (socket says 172.21.15.62:58146) 2024-06-10T00:33:07.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f360cdd5700 1 -- 172.21.15.62:0/506608201 learned_addr learned my addr 172.21.15.62:0/506608201 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:07.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f3607fff700 1 --2- 172.21.15.62:0/506608201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7de13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f360d5d6700 1 --2- 172.21.15.62:0/506608201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7deb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f360cdd5700 1 -- 172.21.15.62:0/506608201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 msgr2=0x5590e7deb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:07.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f360cdd5700 1 --2- 172.21.15.62:0/506608201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7deb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.514+0000 7f360cdd5700 1 -- 172.21.15.62:0/506608201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 msgr2=0x5590e7de13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:07.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.515+0000 7f360cdd5700 1 --2- 172.21.15.62:0/506608201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7de13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.515+0000 7f360cdd5700 1 -- 172.21.15.62:0/506608201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590e7de2a50 con 0x5590e7de67a0 2024-06-10T00:33:07.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.515+0000 7f360d5d6700 1 --2- 172.21.15.62:0/506608201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7deb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:07.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.515+0000 7f3607fff700 1 --2- 172.21.15.62:0/506608201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7de13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:07.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.515+0000 7f360cdd5700 1 --2- 172.21.15.62:0/506608201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7de67a0 0x5590e7de0e80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0098e0 tx=0x7f35fc023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a3911e868a9d551 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:07.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.515+0000 7f35f27fc700 1 -- 172.21.15.62:0/506608201 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc005690 con 0x5590e7de67a0 2024-06-10T00:33:07.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.515+0000 7f35f27fc700 1 -- 172.21.15.62:0/506608201 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35fc00f940 con 0x5590e7de67a0 2024-06-10T00:33:07.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.515+0000 7f35f27fc700 1 -- 172.21.15.62:0/506608201 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0246e0 con 0x5590e7de67a0 2024-06-10T00:33:07.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.516+0000 7f36198bcb80 1 -- 172.21.15.62:0/506608201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7de67a0 msgr2=0x5590e7de0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:07.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.516+0000 7f36198bcb80 1 --2- 172.21.15.62:0/506608201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7de67a0 0x5590e7de0e80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0098e0 tx=0x7f35fc023000 comp rx=0 tx=0).stop 2024-06-10T00:33:07.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.516+0000 7f36198bcb80 1 -- 172.21.15.62:0/506608201 shutdown_connections 2024-06-10T00:33:07.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.516+0000 7f36198bcb80 1 --2- 172.21.15.62:0/506608201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7deb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.516+0000 7f36198bcb80 1 --2- 172.21.15.62:0/506608201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7de67a0 0x5590e7de0e80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.516+0000 7f36198bcb80 1 --2- 172.21.15.62:0/506608201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7de13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.516+0000 7f36198bcb80 1 -- 172.21.15.62:0/506608201 >> 172.21.15.62:0/506608201 conn(0x5590e7d13080 msgr2=0x5590e7dea9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:07.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.516+0000 7f36198bcb80 1 -- 172.21.15.62:0/506608201 shutdown_connections 2024-06-10T00:33:07.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.516+0000 7f36198bcb80 1 -- 172.21.15.62:0/506608201 wait complete. 2024-06-10T00:33:07.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.516+0000 7f36198bcb80 1 -- start start 2024-06-10T00:33:07.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.516+0000 7f36198bcb80 1 Processor -- start 2024-06-10T00:33:07.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f36198bcb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7de67a0 0x5590e7e0daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f36198bcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7e0dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f36198bcb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7e6a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f36198bcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36081ad960 con 0x5590e7debf60 2024-06-10T00:33:07.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f36198bcb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36081ad290 con 0x5590e7de67a0 2024-06-10T00:33:07.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f36198bcb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36081aa4b0 con 0x5590e7dec880 2024-06-10T00:33:07.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f36198bcb80 10 client.0 _ll_register_callbacks cb 0x5590e7ccd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:07.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f360cdd5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7de67a0 0x5590e7e0daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f360cdd5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7de67a0 0x5590e7e0daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:58162/0 (socket says 172.21.15.62:58162) 2024-06-10T00:33:07.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f360cdd5700 1 -- 172.21.15.62:0/793380471 learned_addr learned my addr 172.21.15.62:0/793380471 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:07.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f3607fff700 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7e0dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f360d5d6700 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7e6a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f3607fff700 1 -- 172.21.15.62:0/793380471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 msgr2=0x5590e7e6a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:07.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f3607fff700 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7e6a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f3607fff700 1 -- 172.21.15.62:0/793380471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7de67a0 msgr2=0x5590e7e0daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f3607fff700 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7de67a0 0x5590e7e0daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.517+0000 7f3607fff700 1 -- 172.21.15.62:0/793380471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590e7e6ada0 con 0x5590e7debf60 2024-06-10T00:33:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.518+0000 7f360d5d6700 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7e6a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:33:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.518+0000 7f360cdd5700 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7de67a0 0x5590e7e0daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.518+0000 7f3607fff700 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7e0dfe0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f35f4004da0 tx=0x7f35f400b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2976cf14c8500741 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.518+0000 7f35e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:33:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.518+0000 7f35e7fff700 1 -- 172.21.15.62:0/793380471 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4028070 con 0x5590e7debf60 2024-06-10T00:33:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.518+0000 7f35e7fff700 1 -- 172.21.15.62:0/793380471 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f35f401aa10 con 0x5590e7debf60 2024-06-10T00:33:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.518+0000 7f36198bcb80 1 -- 172.21.15.62:0/793380471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35fc00f940 con 0x5590e7debf60 2024-06-10T00:33:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.519+0000 7f35e7fff700 1 -- 172.21.15.62:0/793380471 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f35f400fa90 con 0x5590e7debf60 2024-06-10T00:33:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.519+0000 7f36198bcb80 10 client.15534 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.519+0000 7f36198bcb80 1 -- 172.21.15.62:0/793380471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5590e7e73320 con 0x5590e7debf60 2024-06-10T00:33:07.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.519+0000 7f35e7fff700 1 -- 172.21.15.62:0/793380471 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35fc00f940 con 0x5590e7debf60 2024-06-10T00:33:07.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.519+0000 7f35e7fff700 1 -- 172.21.15.62:0/793380471 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f35f4004f90 con 0x5590e7debf60 2024-06-10T00:33:07.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.519+0000 7f36198bcb80 10 client.15534 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:07.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.519+0000 7f36198bcb80 10 client.15534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:33:07.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.519+0000 7f36198bcb80 20 client.15534 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:07.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.519+0000 7f36198bcb80 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590e7e73a60 0x5590e7e75e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:07.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.519+0000 7f36198bcb80 4 client.15534 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:33:07.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.519+0000 7f36198bcb80 1 -- 172.21.15.62:0/793380471 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5590e7d3f2d0 con 0x5590e7e73a60 2024-06-10T00:33:07.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f360cdd5700 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590e7e73a60 0x5590e7e75e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:07.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f35e7fff700 10 client.15534 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:07.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f360cdd5700 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590e7e73a60 0x5590e7e75e80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=148e942e670af7ae server_cookie=aa133e6e02851171 in_seq=0 out_seq=0 2024-06-10T00:33:07.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f35e7fff700 1 -- 172.21.15.62:0/793380471 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5590e7d3f2d0 con 0x5590e7e73a60 2024-06-10T00:33:07.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f35e7fff700 10 client.15534 handle_command_reply: tid=0 2024-06-10T00:33:07.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f36198bcb80 1 client.15534 shutdown 2024-06-10T00:33:07.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f36198bcb80 1 -- 172.21.15.62:0/793380471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 msgr2=0x5590e7e0dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:07.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f36198bcb80 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7e0dfe0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f35f4004da0 tx=0x7f35f400b520 comp rx=0 tx=0).stop 2024-06-10T00:33:07.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f36198bcb80 1 -- 172.21.15.62:0/793380471 shutdown_connections 2024-06-10T00:33:07.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f36198bcb80 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590e7dec880 0x5590e7e6a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f36198bcb80 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590e7de67a0 0x5590e7e0daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f36198bcb80 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590e7e73a60 0x5590e7e75e80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f36198bcb80 1 --2- 172.21.15.62:0/793380471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590e7debf60 0x5590e7e0dfe0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f36198bcb80 1 -- 172.21.15.62:0/793380471 >> 172.21.15.62:0/793380471 conn(0x5590e7d13080 msgr2=0x5590e7deb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:07.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.520+0000 7f35e7fff700 0 client.15534 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:07.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.521+0000 7f36198bcb80 1 -- 172.21.15.62:0/793380471 shutdown_connections 2024-06-10T00:33:07.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.521+0000 7f36198bcb80 1 -- 172.21.15.62:0/793380471 wait complete. 2024-06-10T00:33:07.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.521+0000 7f36198bcb80 20 client.15534 trim_cache size 0 max 16384 2024-06-10T00:33:07.527 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:33:07.527 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T00:33:07.527 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-06-10T00:33:07.527 INFO:teuthology.orchestra.run.smithi062.stdout: "efdc32f2-c0f1-4917-8e4f-8f9bb83fb21d": { 2024-06-10T00:33:07.527 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-06-10T00:33:07.527 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "efdc32f2-c0f1-4917-8e4f-8f9bb83fb21d", 2024-06-10T00:33:07.527 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-06-10T00:33:07.527 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:33:07.527 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:33:07.527 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:33:07.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.521+0000 7f360f039700 1 -- 172.21.15.62:0/3852191478 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f35f80a2e20 msgr2=0x7f35f80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:07.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.521+0000 7f360f039700 1 --2- 172.21.15.62:0/3852191478 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f35f80a2e20 0x7f35f80a52e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35fc000c00 tx=0x7f35fc005fb0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.522+0000 7f360f039700 1 -- 172.21.15.62:0/3852191478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36081066d0 msgr2=0x7f3608103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:07.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.522+0000 7f360f039700 1 --2- 172.21.15.62:0/3852191478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36081066d0 0x7f3608103b10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f360000ef30 tx=0x7f360000c3a0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.522+0000 7f360f039700 1 -- 172.21.15.62:0/3852191478 shutdown_connections 2024-06-10T00:33:07.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.522+0000 7f360f039700 1 --2- 172.21.15.62:0/3852191478 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f35f80a2e20 0x7f35f80a52e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.522+0000 7f360f039700 1 --2- 172.21.15.62:0/3852191478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36081066d0 0x7f3608103b10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.522+0000 7f360f039700 1 --2- 172.21.15.62:0/3852191478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3608101680 0x7f3608103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.522+0000 7f360f039700 1 --2- 172.21.15.62:0/3852191478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3608102030 0x7f36081035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:07.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.522+0000 7f360f039700 1 -- 172.21.15.62:0/3852191478 >> 172.21.15.62:0/3852191478 conn(0x7f36080773e0 msgr2=0x7f360806afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:07.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.522+0000 7f360f039700 1 -- 172.21.15.62:0/3852191478 shutdown_connections 2024-06-10T00:33:07.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:07.522+0000 7f360f039700 1 -- 172.21.15.62:0/3852191478 wait complete. 2024-06-10T00:33:07.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:07 smithi105 ceph-mon[36321]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:33:07.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:07 smithi105 ceph-mon[36321]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 21 op/s 2024-06-10T00:33:07.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:07 smithi138 ceph-mon[36549]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:33:07.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:07 smithi138 ceph-mon[36549]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 21 op/s 2024-06-10T00:33:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:08.290 INFO:tasks.cephfs.filesystem:scrub status for tag:efdc32f2-c0f1-4917-8e4f-8f9bb83fb21d - {'path': '/', 'tag': 'efdc32f2-c0f1-4917-8e4f-8f9bb83fb21d', 'options': 'recursive,force'} 2024-06-10T00:33:09.336 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:09 smithi062 ceph-mon[32390]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 52 op/s 2024-06-10T00:33:09.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:09 smithi105 ceph-mon[36321]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 52 op/s 2024-06-10T00:33:09.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:09 smithi138 ceph-mon[36549]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 52 op/s 2024-06-10T00:33:09.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.836+0000 7f4ea8257700 1 -- 172.21.15.62:0/2982228777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea0102740 msgr2=0x7f4ea0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:09.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.836+0000 7f4ea8257700 1 --2- 172.21.15.62:0/2982228777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea0102740 0x7f4ea0102b20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0099c0 tx=0x7f4e9c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:33:09.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.836+0000 7f4ea8257700 1 -- 172.21.15.62:0/2982228777 shutdown_connections 2024-06-10T00:33:09.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.836+0000 7f4ea8257700 1 --2- 172.21.15.62:0/2982228777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea0102740 0x7f4ea0102b20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:09.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.836+0000 7f4ea8257700 1 --2- 172.21.15.62:0/2982228777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ea0107840 0x7f4ea010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:09.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.836+0000 7f4ea8257700 1 --2- 172.21.15.62:0/2982228777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea01030f0 0x7f4ea0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:09.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.836+0000 7f4ea8257700 1 -- 172.21.15.62:0/2982228777 >> 172.21.15.62:0/2982228777 conn(0x7f4ea00795f0 msgr2=0x7f4ea0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:09.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.836+0000 7f4ea8257700 1 -- 172.21.15.62:0/2982228777 shutdown_connections 2024-06-10T00:33:09.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.836+0000 7f4ea8257700 1 -- 172.21.15.62:0/2982228777 wait complete. 2024-06-10T00:33:09.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea8257700 1 Processor -- start 2024-06-10T00:33:09.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea8257700 1 -- start start 2024-06-10T00:33:09.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea8257700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea0102740 0x7f4ea0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:09.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea8257700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea01030f0 0x7f4ea0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:09.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea8257700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ea0107840 0x7f4ea0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:09.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea8257700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea0112780 con 0x7f4ea01030f0 2024-06-10T00:33:09.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea8257700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea0112600 con 0x7f4ea0107840 2024-06-10T00:33:09.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea8257700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea0112900 con 0x7f4ea0102740 2024-06-10T00:33:09.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea57f2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea01030f0 0x7f4ea0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:09.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea57f2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea01030f0 0x7f4ea0074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47790/0 (socket says 172.21.15.62:47790) 2024-06-10T00:33:09.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea57f2700 1 -- 172.21.15.62:0/2994925227 learned_addr learned my addr 172.21.15.62:0/2994925227 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:09.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea5ff3700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea0102740 0x7f4ea0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:09.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea67f4700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ea0107840 0x7f4ea0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:09.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea57f2700 1 -- 172.21.15.62:0/2994925227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea0102740 msgr2=0x7f4ea0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:09.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea57f2700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea0102740 0x7f4ea0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:09.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea57f2700 1 -- 172.21.15.62:0/2994925227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ea0107840 msgr2=0x7f4ea0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:09.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea57f2700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ea0107840 0x7f4ea0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:09.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea57f2700 1 -- 172.21.15.62:0/2994925227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea01a71b0 con 0x7f4ea01030f0 2024-06-10T00:33:09.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea67f4700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ea0107840 0x7f4ea0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:09.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.837+0000 7f4ea57f2700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea01030f0 0x7f4ea0074cd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000d980 tx=0x7f4e9000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:09.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.838+0000 7f4e96ffd700 1 -- 172.21.15.62:0/2994925227 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90017070 con 0x7f4ea01030f0 2024-06-10T00:33:09.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.838+0000 7f4e96ffd700 1 -- 172.21.15.62:0/2994925227 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e90005020 con 0x7f4ea01030f0 2024-06-10T00:33:09.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.838+0000 7f4ea8257700 1 -- 172.21.15.62:0/2994925227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea01a74a0 con 0x7f4ea01030f0 2024-06-10T00:33:09.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.838+0000 7f4e96ffd700 1 -- 172.21.15.62:0/2994925227 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90009bc0 con 0x7f4ea01030f0 2024-06-10T00:33:09.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.838+0000 7f4ea8257700 1 -- 172.21.15.62:0/2994925227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea01a79e0 con 0x7f4ea01030f0 2024-06-10T00:33:09.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.839+0000 7f4ea8257700 1 -- 172.21.15.62:0/2994925227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea010aa80 con 0x7f4ea01030f0 2024-06-10T00:33:09.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.841+0000 7f4e96ffd700 1 -- 172.21.15.62:0/2994925227 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f4e900051d0 con 0x7f4ea01030f0 2024-06-10T00:33:09.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.841+0000 7f4e96ffd700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4e8c0a2ea0 0x7f4e8c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:09.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.842+0000 7f4ea5ff3700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4e8c0a2ea0 0x7f4e8c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:09.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.842+0000 7f4e96ffd700 1 -- 172.21.15.62:0/2994925227 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f4e900cfc70 con 0x7f4ea01030f0 2024-06-10T00:33:09.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.842+0000 7f4ea5ff3700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4e8c0a2ea0 0x7f4e8c0a5360 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c000c00 tx=0x7f4e9c005fb0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:09.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:09.845+0000 7f4e96ffd700 1 -- 172.21.15.62:0/2994925227 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4e90020020 con 0x7f4ea01030f0 2024-06-10T00:33:10.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:10.046+0000 7f4ea8257700 1 -- 172.21.15.62:0/2994925227 --> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ea0075c70 con 0x7f4e8c0a2ea0 2024-06-10T00:33:10.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:10.057+0000 7f4e96ffd700 1 -- 172.21.15.62:0/2994925227 <== mgr.15333 v2:172.21.15.62:6800/518034449 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339604 (secure 0 0 0) 0x7f4ea0075c70 con 0x7f4e8c0a2ea0 2024-06-10T00:33:10.063 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:33:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:10.060+0000 7f4ea8257700 1 -- 172.21.15.62:0/2994925227 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4e8c0a2ea0 msgr2=0x7f4e8c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:10.060+0000 7f4ea8257700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4e8c0a2ea0 0x7f4e8c0a5360 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c000c00 tx=0x7f4e9c005fb0 comp rx=0 tx=0).stop 2024-06-10T00:33:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:10.060+0000 7f4ea8257700 1 -- 172.21.15.62:0/2994925227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea01030f0 msgr2=0x7f4ea0074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:10.060+0000 7f4ea8257700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea01030f0 0x7f4ea0074cd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000d980 tx=0x7f4e9000dc90 comp rx=0 tx=0).stop 2024-06-10T00:33:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:10.060+0000 7f4ea8257700 1 -- 172.21.15.62:0/2994925227 shutdown_connections 2024-06-10T00:33:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:10.060+0000 7f4ea8257700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4e8c0a2ea0 0x7f4e8c0a5360 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:10.060+0000 7f4ea8257700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ea0107840 0x7f4ea0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:10.060+0000 7f4ea8257700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea0102740 0x7f4ea0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:10.060+0000 7f4ea8257700 1 --2- 172.21.15.62:0/2994925227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ea01030f0 0x7f4ea0074cd0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:10.060+0000 7f4ea8257700 1 -- 172.21.15.62:0/2994925227 >> 172.21.15.62:0/2994925227 conn(0x7f4ea00795f0 msgr2=0x7f4ea0108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:10.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:10.060+0000 7f4ea8257700 1 -- 172.21.15.62:0/2994925227 shutdown_connections 2024-06-10T00:33:10.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:10.060+0000 7f4ea8257700 1 -- 172.21.15.62:0/2994925227 wait complete. 2024-06-10T00:33:10.069 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-06-10T00:33:10.868 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-06-10T00:33:09.939022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391547,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2635,"num_read_kb":1755,"num_write":715,"num_write_kb":5332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1040,"ondisk_log_size":1040,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388248,"kb_used_data":17912,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124553576,"statfs":{"total":1151940427776,"available":1151542861824,"internally_reserved":0,"allocated":18341888,"data_stored":11109023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":378713403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":355,"num_read_kb":282,"num_write":126,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003593"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199389+0000","last_change":"2024-06-10T00:28:37.657234+0000","last_active":"2024-06-10T00:32:50.199389+0000","last_peered":"2024-06-10T00:32:50.199389+0000","last_clean":"2024-06-10T00:32:50.199389+0000","last_became_active":"2024-06-10T00:28:37.656879+0000","last_became_peered":"2024-06-10T00:28:37.656879+0000","last_unstale":"2024-06-10T00:32:50.199389+0000","last_undegraded":"2024-06-10T00:32:50.199389+0000","last_fullsized":"2024-06-10T00:32:50.199389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:01:02.907597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193442+0000","last_change":"2024-06-10T00:28:37.656440+0000","last_active":"2024-06-10T00:32:50.193442+0000","last_peered":"2024-06-10T00:32:50.193442+0000","last_clean":"2024-06-10T00:32:50.193442+0000","last_became_active":"2024-06-10T00:28:37.656113+0000","last_became_peered":"2024-06-10T00:28:37.656113+0000","last_unstale":"2024-06-10T00:32:50.193442+0000","last_undegraded":"2024-06-10T00:32:50.193442+0000","last_fullsized":"2024-06-10T00:32:50.193442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:49:31.447951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193070+0000","last_change":"2024-06-10T00:28:37.659114+0000","last_active":"2024-06-10T00:32:50.193070+0000","last_peered":"2024-06-10T00:32:50.193070+0000","last_clean":"2024-06-10T00:32:50.193070+0000","last_became_active":"2024-06-10T00:28:37.658763+0000","last_became_peered":"2024-06-10T00:28:37.658763+0000","last_unstale":"2024-06-10T00:32:50.193070+0000","last_undegraded":"2024-06-10T00:32:50.193070+0000","last_fullsized":"2024-06-10T00:32:50.193070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:39:54.963590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921896+0000","last_change":"2024-06-10T00:28:37.656786+0000","last_active":"2024-06-10T00:32:49.921896+0000","last_peered":"2024-06-10T00:32:49.921896+0000","last_clean":"2024-06-10T00:32:49.921896+0000","last_became_active":"2024-06-10T00:28:37.656073+0000","last_became_peered":"2024-06-10T00:28:37.656073+0000","last_unstale":"2024-06-10T00:32:49.921896+0000","last_undegraded":"2024-06-10T00:32:49.921896+0000","last_fullsized":"2024-06-10T00:32:49.921896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:19.176608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031746+0000","last_change":"2024-06-10T00:28:37.650250+0000","last_active":"2024-06-10T00:32:50.031746+0000","last_peered":"2024-06-10T00:32:50.031746+0000","last_clean":"2024-06-10T00:32:50.031746+0000","last_became_active":"2024-06-10T00:28:37.649852+0000","last_became_peered":"2024-06-10T00:28:37.649852+0000","last_unstale":"2024-06-10T00:32:50.031746+0000","last_undegraded":"2024-06-10T00:32:50.031746+0000","last_fullsized":"2024-06-10T00:32:50.031746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:39.492884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042392+0000","last_change":"2024-06-10T00:28:37.646042+0000","last_active":"2024-06-10T00:32:50.042392+0000","last_peered":"2024-06-10T00:32:50.042392+0000","last_clean":"2024-06-10T00:32:50.042392+0000","last_became_active":"2024-06-10T00:28:37.644591+0000","last_became_peered":"2024-06-10T00:28:37.644591+0000","last_unstale":"2024-06-10T00:32:50.042392+0000","last_undegraded":"2024-06-10T00:32:50.042392+0000","last_fullsized":"2024-06-10T00:32:50.042392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:29.932303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134284+0000","last_change":"2024-06-10T00:28:37.664076+0000","last_active":"2024-06-10T00:32:50.134284+0000","last_peered":"2024-06-10T00:32:50.134284+0000","last_clean":"2024-06-10T00:32:50.134284+0000","last_became_active":"2024-06-10T00:28:37.663741+0000","last_became_peered":"2024-06-10T00:28:37.663741+0000","last_unstale":"2024-06-10T00:32:50.134284+0000","last_undegraded":"2024-06-10T00:32:50.134284+0000","last_fullsized":"2024-06-10T00:32:50.134284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:14.581409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032039+0000","last_change":"2024-06-10T00:28:37.648758+0000","last_active":"2024-06-10T00:32:50.032039+0000","last_peered":"2024-06-10T00:32:50.032039+0000","last_clean":"2024-06-10T00:32:50.032039+0000","last_became_active":"2024-06-10T00:28:37.648387+0000","last_became_peered":"2024-06-10T00:28:37.648387+0000","last_unstale":"2024-06-10T00:32:50.032039+0000","last_undegraded":"2024-06-10T00:32:50.032039+0000","last_fullsized":"2024-06-10T00:32:50.032039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:17.187265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.189039+0000","last_change":"2024-06-10T00:28:37.664650+0000","last_active":"2024-06-10T00:32:50.189039+0000","last_peered":"2024-06-10T00:32:50.189039+0000","last_clean":"2024-06-10T00:32:50.189039+0000","last_became_active":"2024-06-10T00:28:37.663178+0000","last_became_peered":"2024-06-10T00:28:37.663178+0000","last_unstale":"2024-06-10T00:32:50.189039+0000","last_undegraded":"2024-06-10T00:32:50.189039+0000","last_fullsized":"2024-06-10T00:32:50.189039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:46.032078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042870+0000","last_change":"2024-06-10T00:28:37.658353+0000","last_active":"2024-06-10T00:32:50.042870+0000","last_peered":"2024-06-10T00:32:50.042870+0000","last_clean":"2024-06-10T00:32:50.042870+0000","last_became_active":"2024-06-10T00:28:37.657961+0000","last_became_peered":"2024-06-10T00:28:37.657961+0000","last_unstale":"2024-06-10T00:32:50.042870+0000","last_undegraded":"2024-06-10T00:32:50.042870+0000","last_fullsized":"2024-06-10T00:32:50.042870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:22:27.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134878+0000","last_change":"2024-06-10T00:28:37.663823+0000","last_active":"2024-06-10T00:32:50.134878+0000","last_peered":"2024-06-10T00:32:50.134878+0000","last_clean":"2024-06-10T00:32:50.134878+0000","last_became_active":"2024-06-10T00:28:37.663499+0000","last_became_peered":"2024-06-10T00:28:37.663499+0000","last_unstale":"2024-06-10T00:32:50.134878+0000","last_undegraded":"2024-06-10T00:32:50.134878+0000","last_fullsized":"2024-06-10T00:32:50.134878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.100923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921877+0000","last_change":"2024-06-10T00:30:24.112722+0000","last_active":"2024-06-10T00:32:49.921877+0000","last_peered":"2024-06-10T00:32:49.921877+0000","last_clean":"2024-06-10T00:32:49.921877+0000","last_became_active":"2024-06-10T00:30:24.112325+0000","last_became_peered":"2024-06-10T00:30:24.112325+0000","last_unstale":"2024-06-10T00:32:49.921877+0000","last_undegraded":"2024-06-10T00:32:49.921877+0000","last_fullsized":"2024-06-10T00:32:49.921877+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:26.671583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133292+0000","last_change":"2024-06-10T00:28:37.649997+0000","last_active":"2024-06-10T00:32:50.133292+0000","last_peered":"2024-06-10T00:32:50.133292+0000","last_clean":"2024-06-10T00:32:50.133292+0000","last_became_active":"2024-06-10T00:28:37.649494+0000","last_became_peered":"2024-06-10T00:28:37.649494+0000","last_unstale":"2024-06-10T00:32:50.133292+0000","last_undegraded":"2024-06-10T00:32:50.133292+0000","last_fullsized":"2024-06-10T00:32:50.133292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:44:07.467206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192453+0000","last_change":"2024-06-10T00:28:37.653064+0000","last_active":"2024-06-10T00:32:50.192453+0000","last_peered":"2024-06-10T00:32:50.192453+0000","last_clean":"2024-06-10T00:32:50.192453+0000","last_became_active":"2024-06-10T00:28:37.652684+0000","last_became_peered":"2024-06-10T00:28:37.652684+0000","last_unstale":"2024-06-10T00:32:50.192453+0000","last_undegraded":"2024-06-10T00:32:50.192453+0000","last_fullsized":"2024-06-10T00:32:50.192453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:30.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192384+0000","last_change":"2024-06-10T00:28:37.655570+0000","last_active":"2024-06-10T00:32:50.192384+0000","last_peered":"2024-06-10T00:32:50.192384+0000","last_clean":"2024-06-10T00:32:50.192384+0000","last_became_active":"2024-06-10T00:28:37.655241+0000","last_became_peered":"2024-06-10T00:28:37.655241+0000","last_unstale":"2024-06-10T00:32:50.192384+0000","last_undegraded":"2024-06-10T00:32:50.192384+0000","last_fullsized":"2024-06-10T00:32:50.192384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:07:03.543064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188211+0000","last_change":"2024-06-10T00:28:37.665607+0000","last_active":"2024-06-10T00:32:50.188211+0000","last_peered":"2024-06-10T00:32:50.188211+0000","last_clean":"2024-06-10T00:32:50.188211+0000","last_became_active":"2024-06-10T00:28:37.664943+0000","last_became_peered":"2024-06-10T00:28:37.664943+0000","last_unstale":"2024-06-10T00:32:50.188211+0000","last_undegraded":"2024-06-10T00:32:50.188211+0000","last_fullsized":"2024-06-10T00:32:50.188211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:43:02.940298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199188+0000","last_change":"2024-06-10T00:28:37.658214+0000","last_active":"2024-06-10T00:32:50.199188+0000","last_peered":"2024-06-10T00:32:50.199188+0000","last_clean":"2024-06-10T00:32:50.199188+0000","last_became_active":"2024-06-10T00:28:37.657295+0000","last_became_peered":"2024-06-10T00:28:37.657295+0000","last_unstale":"2024-06-10T00:32:50.199188+0000","last_undegraded":"2024-06-10T00:32:50.199188+0000","last_fullsized":"2024-06-10T00:32:50.199188+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:20.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922079+0000","last_change":"2024-06-10T00:28:37.656645+0000","last_active":"2024-06-10T00:32:49.922079+0000","last_peered":"2024-06-10T00:32:49.922079+0000","last_clean":"2024-06-10T00:32:49.922079+0000","last_became_active":"2024-06-10T00:28:37.656157+0000","last_became_peered":"2024-06-10T00:28:37.656157+0000","last_unstale":"2024-06-10T00:32:49.922079+0000","last_undegraded":"2024-06-10T00:32:49.922079+0000","last_fullsized":"2024-06-10T00:32:49.922079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:00.953804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921572+0000","last_change":"2024-06-10T00:28:37.664336+0000","last_active":"2024-06-10T00:32:49.921572+0000","last_peered":"2024-06-10T00:32:49.921572+0000","last_clean":"2024-06-10T00:32:49.921572+0000","last_became_active":"2024-06-10T00:28:37.663960+0000","last_became_peered":"2024-06-10T00:28:37.663960+0000","last_unstale":"2024-06-10T00:32:49.921572+0000","last_undegraded":"2024-06-10T00:32:49.921572+0000","last_fullsized":"2024-06-10T00:32:49.921572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:59:32.774442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920810+0000","last_change":"2024-06-10T00:28:48.705930+0000","last_active":"2024-06-10T00:32:49.920810+0000","last_peered":"2024-06-10T00:32:49.920810+0000","last_clean":"2024-06-10T00:32:49.920810+0000","last_became_active":"2024-06-10T00:28:48.705558+0000","last_became_peered":"2024-06-10T00:28:48.705558+0000","last_unstale":"2024-06-10T00:32:49.920810+0000","last_undegraded":"2024-06-10T00:32:49.920810+0000","last_fullsized":"2024-06-10T00:32:49.920810+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:41:44.914378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"99'14","reported_seq":276,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:04.150139+0000","last_change":"2024-06-10T00:29:23.862141+0000","last_active":"2024-06-10T00:33:04.150139+0000","last_peered":"2024-06-10T00:33:04.150139+0000","last_clean":"2024-06-10T00:33:04.150139+0000","last_became_active":"2024-06-10T00:29:23.857051+0000","last_became_peered":"2024-06-10T00:29:23.857051+0000","last_unstale":"2024-06-10T00:33:04.150139+0000","last_undegraded":"2024-06-10T00:33:04.150139+0000","last_fullsized":"2024-06-10T00:33:04.150139+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:41:41.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":202,"num_read_kb":202,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032309+0000","last_change":"2024-06-10T00:28:37.653938+0000","last_active":"2024-06-10T00:32:50.032309+0000","last_peered":"2024-06-10T00:32:50.032309+0000","last_clean":"2024-06-10T00:32:50.032309+0000","last_became_active":"2024-06-10T00:28:37.653629+0000","last_became_peered":"2024-06-10T00:28:37.653629+0000","last_unstale":"2024-06-10T00:32:50.032309+0000","last_undegraded":"2024-06-10T00:32:50.032309+0000","last_fullsized":"2024-06-10T00:32:50.032309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:21:39.285961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921432+0000","last_change":"2024-06-10T00:28:48.704413+0000","last_active":"2024-06-10T00:32:49.921432+0000","last_peered":"2024-06-10T00:32:49.921432+0000","last_clean":"2024-06-10T00:32:49.921432+0000","last_became_active":"2024-06-10T00:28:48.703972+0000","last_became_peered":"2024-06-10T00:28:48.703972+0000","last_unstale":"2024-06-10T00:32:49.921432+0000","last_undegraded":"2024-06-10T00:32:49.921432+0000","last_fullsized":"2024-06-10T00:32:49.921432+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:51.927435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032677+0000","last_change":"2024-06-10T00:28:34.445413+0000","last_active":"2024-06-10T00:32:50.032677+0000","last_peered":"2024-06-10T00:32:50.032677+0000","last_clean":"2024-06-10T00:32:50.032677+0000","last_became_active":"2024-06-10T00:28:34.445077+0000","last_became_peered":"2024-06-10T00:28:34.445077+0000","last_unstale":"2024-06-10T00:32:50.032677+0000","last_undegraded":"2024-06-10T00:32:50.032677+0000","last_fullsized":"2024-06-10T00:32:50.032677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:19:19.535976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193506+0000","last_change":"2024-06-10T00:28:37.657123+0000","last_active":"2024-06-10T00:32:50.193506+0000","last_peered":"2024-06-10T00:32:50.193506+0000","last_clean":"2024-06-10T00:32:50.193506+0000","last_became_active":"2024-06-10T00:28:37.656791+0000","last_became_peered":"2024-06-10T00:28:37.656791+0000","last_unstale":"2024-06-10T00:32:50.193506+0000","last_undegraded":"2024-06-10T00:32:50.193506+0000","last_fullsized":"2024-06-10T00:32:50.193506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:11.102478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921858+0000","last_change":"2024-06-10T00:28:48.705989+0000","last_active":"2024-06-10T00:32:49.921858+0000","last_peered":"2024-06-10T00:32:49.921858+0000","last_clean":"2024-06-10T00:32:49.921858+0000","last_became_active":"2024-06-10T00:28:48.705555+0000","last_became_peered":"2024-06-10T00:28:48.705555+0000","last_unstale":"2024-06-10T00:32:49.921858+0000","last_undegraded":"2024-06-10T00:32:49.921858+0000","last_fullsized":"2024-06-10T00:32:49.921858+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:36.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"100'13","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032133+0000","last_change":"2024-06-10T00:28:34.400117+0000","last_active":"2024-06-10T00:32:50.032133+0000","last_peered":"2024-06-10T00:32:50.032133+0000","last_clean":"2024-06-10T00:32:50.032133+0000","last_became_active":"2024-06-10T00:28:34.399743+0000","last_became_peered":"2024-06-10T00:28:34.399743+0000","last_unstale":"2024-06-10T00:32:50.032133+0000","last_undegraded":"2024-06-10T00:32:50.032133+0000","last_fullsized":"2024-06-10T00:32:50.032133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:20:57.580146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188627+0000","last_change":"2024-06-10T00:28:37.664005+0000","last_active":"2024-06-10T00:32:50.188627+0000","last_peered":"2024-06-10T00:32:50.188627+0000","last_clean":"2024-06-10T00:32:50.188627+0000","last_became_active":"2024-06-10T00:28:37.661297+0000","last_became_peered":"2024-06-10T00:28:37.661297+0000","last_unstale":"2024-06-10T00:32:50.188627+0000","last_undegraded":"2024-06-10T00:32:50.188627+0000","last_fullsized":"2024-06-10T00:32:50.188627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:57.795080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921561+0000","last_change":"2024-06-10T00:29:23.859091+0000","last_active":"2024-06-10T00:32:49.921561+0000","last_peered":"2024-06-10T00:32:49.921561+0000","last_clean":"2024-06-10T00:32:49.921561+0000","last_became_active":"2024-06-10T00:29:23.858485+0000","last_became_peered":"2024-06-10T00:29:23.858485+0000","last_unstale":"2024-06-10T00:32:49.921561+0000","last_undegraded":"2024-06-10T00:32:49.921561+0000","last_fullsized":"2024-06-10T00:32:49.921561+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:07:10.925323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031945+0000","last_change":"2024-06-10T00:28:34.400588+0000","last_active":"2024-06-10T00:32:50.031945+0000","last_peered":"2024-06-10T00:32:50.031945+0000","last_clean":"2024-06-10T00:32:50.031945+0000","last_became_active":"2024-06-10T00:28:34.400027+0000","last_became_peered":"2024-06-10T00:28:34.400027+0000","last_unstale":"2024-06-10T00:32:50.031945+0000","last_undegraded":"2024-06-10T00:32:50.031945+0000","last_fullsized":"2024-06-10T00:32:50.031945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:25:59.890076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039025+0000","last_change":"2024-06-10T00:28:37.659401+0000","last_active":"2024-06-10T00:32:50.039025+0000","last_peered":"2024-06-10T00:32:50.039025+0000","last_clean":"2024-06-10T00:32:50.039025+0000","last_became_active":"2024-06-10T00:28:37.659056+0000","last_became_peered":"2024-06-10T00:28:37.659056+0000","last_unstale":"2024-06-10T00:32:50.039025+0000","last_undegraded":"2024-06-10T00:32:50.039025+0000","last_fullsized":"2024-06-10T00:32:50.039025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:35:01.415630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921379+0000","last_change":"2024-06-10T00:28:48.702961+0000","last_active":"2024-06-10T00:32:49.921379+0000","last_peered":"2024-06-10T00:32:49.921379+0000","last_clean":"2024-06-10T00:32:49.921379+0000","last_became_active":"2024-06-10T00:28:48.702502+0000","last_became_peered":"2024-06-10T00:28:48.702502+0000","last_unstale":"2024-06-10T00:32:49.921379+0000","last_undegraded":"2024-06-10T00:32:49.921379+0000","last_fullsized":"2024-06-10T00:32:49.921379+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:54.881595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'9","reported_seq":260,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:04.150766+0000","last_change":"2024-06-10T00:28:34.396310+0000","last_active":"2024-06-10T00:33:04.150766+0000","last_peered":"2024-06-10T00:33:04.150766+0000","last_clean":"2024-06-10T00:33:04.150766+0000","last_became_active":"2024-06-10T00:28:34.395401+0000","last_became_peered":"2024-06-10T00:28:34.395401+0000","last_unstale":"2024-06-10T00:33:04.150766+0000","last_undegraded":"2024-06-10T00:33:04.150766+0000","last_fullsized":"2024-06-10T00:33:04.150766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:33:54.300308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":205,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031647+0000","last_change":"2024-06-10T00:28:37.651427+0000","last_active":"2024-06-10T00:32:50.031647+0000","last_peered":"2024-06-10T00:32:50.031647+0000","last_clean":"2024-06-10T00:32:50.031647+0000","last_became_active":"2024-06-10T00:28:37.650510+0000","last_became_peered":"2024-06-10T00:28:37.650510+0000","last_unstale":"2024-06-10T00:32:50.031647+0000","last_undegraded":"2024-06-10T00:32:50.031647+0000","last_fullsized":"2024-06-10T00:32:50.031647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:42.953919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920970+0000","last_change":"2024-06-10T00:28:48.705918+0000","last_active":"2024-06-10T00:32:49.920970+0000","last_peered":"2024-06-10T00:32:49.920970+0000","last_clean":"2024-06-10T00:32:49.920970+0000","last_became_active":"2024-06-10T00:28:48.705462+0000","last_became_peered":"2024-06-10T00:28:48.705462+0000","last_unstale":"2024-06-10T00:32:49.920970+0000","last_undegraded":"2024-06-10T00:32:49.920970+0000","last_fullsized":"2024-06-10T00:32:49.920970+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:59:23.176511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042358+0000","last_change":"2024-06-10T00:28:34.393199+0000","last_active":"2024-06-10T00:32:50.042358+0000","last_peered":"2024-06-10T00:32:50.042358+0000","last_clean":"2024-06-10T00:32:50.042358+0000","last_became_active":"2024-06-10T00:28:34.392777+0000","last_became_peered":"2024-06-10T00:28:34.392777+0000","last_unstale":"2024-06-10T00:32:50.042358+0000","last_undegraded":"2024-06-10T00:32:50.042358+0000","last_fullsized":"2024-06-10T00:32:50.042358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:29.614242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038961+0000","last_change":"2024-06-10T00:30:24.229721+0000","last_active":"2024-06-10T00:32:50.038961+0000","last_peered":"2024-06-10T00:32:50.038961+0000","last_clean":"2024-06-10T00:32:50.038961+0000","last_became_active":"2024-06-10T00:30:24.229407+0000","last_became_peered":"2024-06-10T00:30:24.229407+0000","last_unstale":"2024-06-10T00:32:50.038961+0000","last_undegraded":"2024-06-10T00:32:50.038961+0000","last_fullsized":"2024-06-10T00:32:50.038961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:39.523197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199129+0000","last_change":"2024-06-10T00:29:23.780971+0000","last_active":"2024-06-10T00:32:50.199129+0000","last_peered":"2024-06-10T00:32:50.199129+0000","last_clean":"2024-06-10T00:32:50.199129+0000","last_became_active":"2024-06-10T00:29:23.780034+0000","last_became_peered":"2024-06-10T00:29:23.780034+0000","last_unstale":"2024-06-10T00:32:50.199129+0000","last_undegraded":"2024-06-10T00:32:50.199129+0000","last_fullsized":"2024-06-10T00:32:50.199129+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:47:06.444181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132954+0000","last_change":"2024-06-10T00:28:34.392486+0000","last_active":"2024-06-10T00:32:50.132954+0000","last_peered":"2024-06-10T00:32:50.132954+0000","last_clean":"2024-06-10T00:32:50.132954+0000","last_became_active":"2024-06-10T00:28:34.392141+0000","last_became_peered":"2024-06-10T00:28:34.392141+0000","last_unstale":"2024-06-10T00:32:50.132954+0000","last_undegraded":"2024-06-10T00:32:50.132954+0000","last_fullsized":"2024-06-10T00:32:50.132954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:10:01.534195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031869+0000","last_change":"2024-06-10T00:28:37.651033+0000","last_active":"2024-06-10T00:32:50.031869+0000","last_peered":"2024-06-10T00:32:50.031869+0000","last_clean":"2024-06-10T00:32:50.031869+0000","last_became_active":"2024-06-10T00:28:37.650505+0000","last_became_peered":"2024-06-10T00:28:37.650505+0000","last_unstale":"2024-06-10T00:32:50.031869+0000","last_undegraded":"2024-06-10T00:32:50.031869+0000","last_fullsized":"2024-06-10T00:32:50.031869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:54.965865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193367+0000","last_change":"2024-06-10T00:28:48.710748+0000","last_active":"2024-06-10T00:32:50.193367+0000","last_peered":"2024-06-10T00:32:50.193367+0000","last_clean":"2024-06-10T00:32:50.193367+0000","last_became_active":"2024-06-10T00:28:48.710353+0000","last_became_peered":"2024-06-10T00:28:48.710353+0000","last_unstale":"2024-06-10T00:32:50.193367+0000","last_undegraded":"2024-06-10T00:32:50.193367+0000","last_fullsized":"2024-06-10T00:32:50.193367+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:30:47.546650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038780+0000","last_change":"2024-06-10T00:28:34.398911+0000","last_active":"2024-06-10T00:32:50.038780+0000","last_peered":"2024-06-10T00:32:50.038780+0000","last_clean":"2024-06-10T00:32:50.038780+0000","last_became_active":"2024-06-10T00:28:34.398373+0000","last_became_peered":"2024-06-10T00:28:34.398373+0000","last_unstale":"2024-06-10T00:32:50.038780+0000","last_undegraded":"2024-06-10T00:32:50.038780+0000","last_fullsized":"2024-06-10T00:32:50.038780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:05:34.352825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921526+0000","last_change":"2024-06-10T00:28:37.647492+0000","last_active":"2024-06-10T00:32:49.921526+0000","last_peered":"2024-06-10T00:32:49.921526+0000","last_clean":"2024-06-10T00:32:49.921526+0000","last_became_active":"2024-06-10T00:28:37.647103+0000","last_became_peered":"2024-06-10T00:28:37.647103+0000","last_unstale":"2024-06-10T00:32:49.921526+0000","last_undegraded":"2024-06-10T00:32:49.921526+0000","last_fullsized":"2024-06-10T00:32:49.921526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:10.429256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031883+0000","last_change":"2024-06-10T00:28:48.703095+0000","last_active":"2024-06-10T00:32:50.031883+0000","last_peered":"2024-06-10T00:32:50.031883+0000","last_clean":"2024-06-10T00:32:50.031883+0000","last_became_active":"2024-06-10T00:28:48.702687+0000","last_became_peered":"2024-06-10T00:28:48.702687+0000","last_unstale":"2024-06-10T00:32:50.031883+0000","last_undegraded":"2024-06-10T00:32:50.031883+0000","last_fullsized":"2024-06-10T00:32:50.031883+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:10:20.074173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":453,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:05.151511+0000","last_change":"2024-06-10T00:28:34.394807+0000","last_active":"2024-06-10T00:33:05.151511+0000","last_peered":"2024-06-10T00:33:05.151511+0000","last_clean":"2024-06-10T00:33:05.151511+0000","last_became_active":"2024-06-10T00:28:34.394022+0000","last_became_peered":"2024-06-10T00:28:34.394022+0000","last_unstale":"2024-06-10T00:33:05.151511+0000","last_undegraded":"2024-06-10T00:33:05.151511+0000","last_fullsized":"2024-06-10T00:33:05.151511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:21.626079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":400,"num_read_kb":400,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.189056+0000","last_change":"2024-06-10T00:28:37.662231+0000","last_active":"2024-06-10T00:32:50.189056+0000","last_peered":"2024-06-10T00:32:50.189056+0000","last_clean":"2024-06-10T00:32:50.189056+0000","last_became_active":"2024-06-10T00:28:37.661861+0000","last_became_peered":"2024-06-10T00:28:37.661861+0000","last_unstale":"2024-06-10T00:32:50.189056+0000","last_undegraded":"2024-06-10T00:32:50.189056+0000","last_fullsized":"2024-06-10T00:32:50.189056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:09.705349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921535+0000","last_change":"2024-06-10T00:28:48.706643+0000","last_active":"2024-06-10T00:32:49.921535+0000","last_peered":"2024-06-10T00:32:49.921535+0000","last_clean":"2024-06-10T00:32:49.921535+0000","last_became_active":"2024-06-10T00:28:48.706127+0000","last_became_peered":"2024-06-10T00:28:48.706127+0000","last_unstale":"2024-06-10T00:32:49.921535+0000","last_undegraded":"2024-06-10T00:32:49.921535+0000","last_fullsized":"2024-06-10T00:32:49.921535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:55.031303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134861+0000","last_change":"2024-06-10T00:28:34.392808+0000","last_active":"2024-06-10T00:32:50.134861+0000","last_peered":"2024-06-10T00:32:50.134861+0000","last_clean":"2024-06-10T00:32:50.134861+0000","last_became_active":"2024-06-10T00:28:34.392450+0000","last_became_peered":"2024-06-10T00:28:34.392450+0000","last_unstale":"2024-06-10T00:32:50.134861+0000","last_undegraded":"2024-06-10T00:32:50.134861+0000","last_fullsized":"2024-06-10T00:32:50.134861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:06:00.493316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193616+0000","last_change":"2024-06-10T00:28:37.656099+0000","last_active":"2024-06-10T00:32:50.193616+0000","last_peered":"2024-06-10T00:32:50.193616+0000","last_clean":"2024-06-10T00:32:50.193616+0000","last_became_active":"2024-06-10T00:28:37.655640+0000","last_became_peered":"2024-06-10T00:28:37.655640+0000","last_unstale":"2024-06-10T00:32:50.193616+0000","last_undegraded":"2024-06-10T00:32:50.193616+0000","last_fullsized":"2024-06-10T00:32:50.193616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:59:26.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922109+0000","last_change":"2024-06-10T00:28:48.702628+0000","last_active":"2024-06-10T00:32:49.922109+0000","last_peered":"2024-06-10T00:32:49.922109+0000","last_clean":"2024-06-10T00:32:49.922109+0000","last_became_active":"2024-06-10T00:28:48.702109+0000","last_became_peered":"2024-06-10T00:28:48.702109+0000","last_unstale":"2024-06-10T00:32:49.922109+0000","last_undegraded":"2024-06-10T00:32:49.922109+0000","last_fullsized":"2024-06-10T00:32:49.922109+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:25.262348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039027+0000","last_change":"2024-06-10T00:28:34.446788+0000","last_active":"2024-06-10T00:32:50.039027+0000","last_peered":"2024-06-10T00:32:50.039027+0000","last_clean":"2024-06-10T00:32:50.039027+0000","last_became_active":"2024-06-10T00:28:34.446377+0000","last_became_peered":"2024-06-10T00:28:34.446377+0000","last_unstale":"2024-06-10T00:32:50.039027+0000","last_undegraded":"2024-06-10T00:32:50.039027+0000","last_fullsized":"2024-06-10T00:32:50.039027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:16.914244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188829+0000","last_change":"2024-06-10T00:28:37.666159+0000","last_active":"2024-06-10T00:32:50.188829+0000","last_peered":"2024-06-10T00:32:50.188829+0000","last_clean":"2024-06-10T00:32:50.188829+0000","last_became_active":"2024-06-10T00:28:37.665711+0000","last_became_peered":"2024-06-10T00:28:37.665711+0000","last_unstale":"2024-06-10T00:32:50.188829+0000","last_undegraded":"2024-06-10T00:32:50.188829+0000","last_fullsized":"2024-06-10T00:32:50.188829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:03.982375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921636+0000","last_change":"2024-06-10T00:28:48.705071+0000","last_active":"2024-06-10T00:32:49.921636+0000","last_peered":"2024-06-10T00:32:49.921636+0000","last_clean":"2024-06-10T00:32:49.921636+0000","last_became_active":"2024-06-10T00:28:48.702211+0000","last_became_peered":"2024-06-10T00:28:48.702211+0000","last_unstale":"2024-06-10T00:32:49.921636+0000","last_undegraded":"2024-06-10T00:32:49.921636+0000","last_fullsized":"2024-06-10T00:32:49.921636+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:13.361985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":500,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:05.151462+0000","last_change":"2024-06-10T00:28:34.389784+0000","last_active":"2024-06-10T00:33:05.151462+0000","last_peered":"2024-06-10T00:33:05.151462+0000","last_clean":"2024-06-10T00:33:05.151462+0000","last_became_active":"2024-06-10T00:28:34.389416+0000","last_became_peered":"2024-06-10T00:28:34.389416+0000","last_unstale":"2024-06-10T00:33:05.151462+0000","last_undegraded":"2024-06-10T00:33:05.151462+0000","last_fullsized":"2024-06-10T00:33:05.151462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:23:05.803503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69365,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":413,"num_read_kb":276,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031957+0000","last_change":"2024-06-10T00:28:37.656174+0000","last_active":"2024-06-10T00:32:50.031957+0000","last_peered":"2024-06-10T00:32:50.031957+0000","last_clean":"2024-06-10T00:32:50.031957+0000","last_became_active":"2024-06-10T00:28:37.655794+0000","last_became_peered":"2024-06-10T00:28:37.655794+0000","last_unstale":"2024-06-10T00:32:50.031957+0000","last_undegraded":"2024-06-10T00:32:50.031957+0000","last_fullsized":"2024-06-10T00:32:50.031957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:22:00.310309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039312+0000","last_change":"2024-06-10T00:28:48.709915+0000","last_active":"2024-06-10T00:32:50.039312+0000","last_peered":"2024-06-10T00:32:50.039312+0000","last_clean":"2024-06-10T00:32:50.039312+0000","last_became_active":"2024-06-10T00:28:48.709426+0000","last_became_peered":"2024-06-10T00:28:48.709426+0000","last_unstale":"2024-06-10T00:32:50.039312+0000","last_undegraded":"2024-06-10T00:32:50.039312+0000","last_fullsized":"2024-06-10T00:32:50.039312+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:55.150617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921708+0000","last_change":"2024-06-10T00:28:34.389136+0000","last_active":"2024-06-10T00:32:49.921708+0000","last_peered":"2024-06-10T00:32:49.921708+0000","last_clean":"2024-06-10T00:32:49.921708+0000","last_became_active":"2024-06-10T00:28:34.388701+0000","last_became_peered":"2024-06-10T00:28:34.388701+0000","last_unstale":"2024-06-10T00:32:49.921708+0000","last_undegraded":"2024-06-10T00:32:49.921708+0000","last_fullsized":"2024-06-10T00:32:49.921708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:13:24.907824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133047+0000","last_change":"2024-06-10T00:28:37.659048+0000","last_active":"2024-06-10T00:32:50.133047+0000","last_peered":"2024-06-10T00:32:50.133047+0000","last_clean":"2024-06-10T00:32:50.133047+0000","last_became_active":"2024-06-10T00:28:37.658710+0000","last_became_peered":"2024-06-10T00:28:37.658710+0000","last_unstale":"2024-06-10T00:32:50.133047+0000","last_undegraded":"2024-06-10T00:32:50.133047+0000","last_fullsized":"2024-06-10T00:32:50.133047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:44.314559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198919+0000","last_change":"2024-06-10T00:28:48.709683+0000","last_active":"2024-06-10T00:32:50.198919+0000","last_peered":"2024-06-10T00:32:50.198919+0000","last_clean":"2024-06-10T00:32:50.198919+0000","last_became_active":"2024-06-10T00:28:48.709241+0000","last_became_peered":"2024-06-10T00:28:48.709241+0000","last_unstale":"2024-06-10T00:32:50.198919+0000","last_undegraded":"2024-06-10T00:32:50.198919+0000","last_fullsized":"2024-06-10T00:32:50.198919+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:25.190043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":24,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198772+0000","last_change":"2024-06-10T00:29:23.782029+0000","last_active":"2024-06-10T00:32:50.198772+0000","last_peered":"2024-06-10T00:32:50.198772+0000","last_clean":"2024-06-10T00:32:50.198772+0000","last_became_active":"2024-06-10T00:29:23.781565+0000","last_became_peered":"2024-06-10T00:29:23.781565+0000","last_unstale":"2024-06-10T00:32:50.198772+0000","last_undegraded":"2024-06-10T00:32:50.198772+0000","last_fullsized":"2024-06-10T00:32:50.198772+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:53.258760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134547+0000","last_change":"2024-06-10T00:28:37.653385+0000","last_active":"2024-06-10T00:32:50.134547+0000","last_peered":"2024-06-10T00:32:50.134547+0000","last_clean":"2024-06-10T00:32:50.134547+0000","last_became_active":"2024-06-10T00:28:37.653020+0000","last_became_peered":"2024-06-10T00:28:37.653020+0000","last_unstale":"2024-06-10T00:32:50.134547+0000","last_undegraded":"2024-06-10T00:32:50.134547+0000","last_fullsized":"2024-06-10T00:32:50.134547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:57:54.469544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921940+0000","last_change":"2024-06-10T00:28:48.707540+0000","last_active":"2024-06-10T00:32:49.921940+0000","last_peered":"2024-06-10T00:32:49.921940+0000","last_clean":"2024-06-10T00:32:49.921940+0000","last_became_active":"2024-06-10T00:28:48.707143+0000","last_became_peered":"2024-06-10T00:28:48.707143+0000","last_unstale":"2024-06-10T00:32:49.921940+0000","last_undegraded":"2024-06-10T00:32:49.921940+0000","last_fullsized":"2024-06-10T00:32:49.921940+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:20:29.344808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'8","reported_seq":260,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:04.150653+0000","last_change":"2024-06-10T00:28:34.397496+0000","last_active":"2024-06-10T00:33:04.150653+0000","last_peered":"2024-06-10T00:33:04.150653+0000","last_clean":"2024-06-10T00:33:04.150653+0000","last_became_active":"2024-06-10T00:28:34.397124+0000","last_became_peered":"2024-06-10T00:28:34.397124+0000","last_unstale":"2024-06-10T00:33:04.150653+0000","last_undegraded":"2024-06-10T00:33:04.150653+0000","last_fullsized":"2024-06-10T00:33:04.150653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:41:48.609354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134324+0000","last_change":"2024-06-10T00:28:37.658338+0000","last_active":"2024-06-10T00:32:50.134324+0000","last_peered":"2024-06-10T00:32:50.134324+0000","last_clean":"2024-06-10T00:32:50.134324+0000","last_became_active":"2024-06-10T00:28:37.655910+0000","last_became_peered":"2024-06-10T00:28:37.655910+0000","last_unstale":"2024-06-10T00:32:50.134324+0000","last_undegraded":"2024-06-10T00:32:50.134324+0000","last_fullsized":"2024-06-10T00:32:50.134324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:38.501335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133960+0000","last_change":"2024-06-10T00:28:48.711151+0000","last_active":"2024-06-10T00:32:50.133960+0000","last_peered":"2024-06-10T00:32:50.133960+0000","last_clean":"2024-06-10T00:32:50.133960+0000","last_became_active":"2024-06-10T00:28:48.710768+0000","last_became_peered":"2024-06-10T00:28:48.710768+0000","last_unstale":"2024-06-10T00:32:50.133960+0000","last_undegraded":"2024-06-10T00:32:50.133960+0000","last_fullsized":"2024-06-10T00:32:50.133960+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:15:22.791839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":65,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031568+0000","last_change":"2024-06-10T00:28:34.445812+0000","last_active":"2024-06-10T00:32:50.031568+0000","last_peered":"2024-06-10T00:32:50.031568+0000","last_clean":"2024-06-10T00:32:50.031568+0000","last_became_active":"2024-06-10T00:28:34.445472+0000","last_became_peered":"2024-06-10T00:28:34.445472+0000","last_unstale":"2024-06-10T00:32:50.031568+0000","last_undegraded":"2024-06-10T00:32:50.031568+0000","last_fullsized":"2024-06-10T00:32:50.031568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:00.912050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038769+0000","last_change":"2024-06-10T00:28:37.655345+0000","last_active":"2024-06-10T00:32:50.038769+0000","last_peered":"2024-06-10T00:32:50.038769+0000","last_clean":"2024-06-10T00:32:50.038769+0000","last_became_active":"2024-06-10T00:28:37.654954+0000","last_became_peered":"2024-06-10T00:28:37.654954+0000","last_unstale":"2024-06-10T00:32:50.038769+0000","last_undegraded":"2024-06-10T00:32:50.038769+0000","last_fullsized":"2024-06-10T00:32:50.038769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:39:27.403006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038741+0000","last_change":"2024-06-10T00:28:48.705247+0000","last_active":"2024-06-10T00:32:50.038741+0000","last_peered":"2024-06-10T00:32:50.038741+0000","last_clean":"2024-06-10T00:32:50.038741+0000","last_became_active":"2024-06-10T00:28:48.704822+0000","last_became_peered":"2024-06-10T00:28:48.704822+0000","last_unstale":"2024-06-10T00:32:50.038741+0000","last_undegraded":"2024-06-10T00:32:50.038741+0000","last_fullsized":"2024-06-10T00:32:50.038741+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:24:59.991491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032099+0000","last_change":"2024-06-10T00:28:34.396792+0000","last_active":"2024-06-10T00:32:50.032099+0000","last_peered":"2024-06-10T00:32:50.032099+0000","last_clean":"2024-06-10T00:32:50.032099+0000","last_became_active":"2024-06-10T00:28:34.396442+0000","last_became_peered":"2024-06-10T00:28:34.396442+0000","last_unstale":"2024-06-10T00:32:50.032099+0000","last_undegraded":"2024-06-10T00:32:50.032099+0000","last_fullsized":"2024-06-10T00:32:50.032099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:46.436198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199145+0000","last_change":"2024-06-10T00:28:37.657698+0000","last_active":"2024-06-10T00:32:50.199145+0000","last_peered":"2024-06-10T00:32:50.199145+0000","last_clean":"2024-06-10T00:32:50.199145+0000","last_became_active":"2024-06-10T00:28:37.657364+0000","last_became_peered":"2024-06-10T00:28:37.657364+0000","last_unstale":"2024-06-10T00:32:50.199145+0000","last_undegraded":"2024-06-10T00:32:50.199145+0000","last_fullsized":"2024-06-10T00:32:50.199145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:23.097015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031651+0000","last_change":"2024-06-10T00:28:48.727567+0000","last_active":"2024-06-10T00:32:50.031651+0000","last_peered":"2024-06-10T00:32:50.031651+0000","last_clean":"2024-06-10T00:32:50.031651+0000","last_became_active":"2024-06-10T00:28:48.727108+0000","last_became_peered":"2024-06-10T00:28:48.727108+0000","last_unstale":"2024-06-10T00:32:50.031651+0000","last_undegraded":"2024-06-10T00:32:50.031651+0000","last_fullsized":"2024-06-10T00:32:50.031651+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:35:00.824177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921917+0000","last_change":"2024-06-10T00:28:34.446832+0000","last_active":"2024-06-10T00:32:49.921917+0000","last_peered":"2024-06-10T00:32:49.921917+0000","last_clean":"2024-06-10T00:32:49.921917+0000","last_became_active":"2024-06-10T00:28:34.446435+0000","last_became_peered":"2024-06-10T00:28:34.446435+0000","last_unstale":"2024-06-10T00:32:49.921917+0000","last_undegraded":"2024-06-10T00:32:49.921917+0000","last_fullsized":"2024-06-10T00:32:49.921917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:58.296674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042645+0000","last_change":"2024-06-10T00:30:24.229514+0000","last_active":"2024-06-10T00:32:50.042645+0000","last_peered":"2024-06-10T00:32:50.042645+0000","last_clean":"2024-06-10T00:32:50.042645+0000","last_became_active":"2024-06-10T00:30:24.229163+0000","last_became_peered":"2024-06-10T00:30:24.229163+0000","last_unstale":"2024-06-10T00:32:50.042645+0000","last_undegraded":"2024-06-10T00:32:50.042645+0000","last_fullsized":"2024-06-10T00:32:50.042645+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:52.914144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133795+0000","last_change":"2024-06-10T00:28:48.704683+0000","last_active":"2024-06-10T00:32:50.133795+0000","last_peered":"2024-06-10T00:32:50.133795+0000","last_clean":"2024-06-10T00:32:50.133795+0000","last_became_active":"2024-06-10T00:28:48.704297+0000","last_became_peered":"2024-06-10T00:28:48.704297+0000","last_unstale":"2024-06-10T00:32:50.133795+0000","last_undegraded":"2024-06-10T00:32:50.133795+0000","last_fullsized":"2024-06-10T00:32:50.133795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:44.980544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'8","reported_seq":258,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:04.150665+0000","last_change":"2024-06-10T00:28:34.393328+0000","last_active":"2024-06-10T00:33:04.150665+0000","last_peered":"2024-06-10T00:33:04.150665+0000","last_clean":"2024-06-10T00:33:04.150665+0000","last_became_active":"2024-06-10T00:28:34.391968+0000","last_became_peered":"2024-06-10T00:28:34.391968+0000","last_unstale":"2024-06-10T00:33:04.150665+0000","last_undegraded":"2024-06-10T00:33:04.150665+0000","last_fullsized":"2024-06-10T00:33:04.150665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:18.898180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1676,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":203,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042474+0000","last_change":"2024-06-10T00:28:37.652555+0000","last_active":"2024-06-10T00:32:50.042474+0000","last_peered":"2024-06-10T00:32:50.042474+0000","last_clean":"2024-06-10T00:32:50.042474+0000","last_became_active":"2024-06-10T00:28:37.651476+0000","last_became_peered":"2024-06-10T00:28:37.651476+0000","last_unstale":"2024-06-10T00:32:50.042474+0000","last_undegraded":"2024-06-10T00:32:50.042474+0000","last_fullsized":"2024-06-10T00:32:50.042474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:04:57.823224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134015+0000","last_change":"2024-06-10T00:28:48.727568+0000","last_active":"2024-06-10T00:32:50.134015+0000","last_peered":"2024-06-10T00:32:50.134015+0000","last_clean":"2024-06-10T00:32:50.134015+0000","last_became_active":"2024-06-10T00:28:48.727018+0000","last_became_peered":"2024-06-10T00:28:48.727018+0000","last_unstale":"2024-06-10T00:32:50.134015+0000","last_undegraded":"2024-06-10T00:32:50.134015+0000","last_fullsized":"2024-06-10T00:32:50.134015+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:02.025212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921381+0000","last_change":"2024-06-10T00:30:24.651947+0000","last_active":"2024-06-10T00:32:49.921381+0000","last_peered":"2024-06-10T00:32:49.921381+0000","last_clean":"2024-06-10T00:32:49.921381+0000","last_became_active":"2024-06-10T00:30:24.651640+0000","last_became_peered":"2024-06-10T00:30:24.651640+0000","last_unstale":"2024-06-10T00:32:49.921381+0000","last_undegraded":"2024-06-10T00:32:49.921381+0000","last_fullsized":"2024-06-10T00:32:49.921381+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:47.070568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"100'7","reported_seq":255,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:04.150169+0000","last_change":"2024-06-10T00:28:34.445501+0000","last_active":"2024-06-10T00:33:04.150169+0000","last_peered":"2024-06-10T00:33:04.150169+0000","last_clean":"2024-06-10T00:33:04.150169+0000","last_became_active":"2024-06-10T00:28:34.445128+0000","last_became_peered":"2024-06-10T00:28:34.445128+0000","last_unstale":"2024-06-10T00:33:04.150169+0000","last_undegraded":"2024-06-10T00:33:04.150169+0000","last_fullsized":"2024-06-10T00:33:04.150169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:59.231585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":201,"num_read_kb":201,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":33,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134764+0000","last_change":"2024-06-10T00:28:48.709852+0000","last_active":"2024-06-10T00:32:50.134764+0000","last_peered":"2024-06-10T00:32:50.134764+0000","last_clean":"2024-06-10T00:32:50.134764+0000","last_became_active":"2024-06-10T00:28:48.709423+0000","last_became_peered":"2024-06-10T00:28:48.709423+0000","last_unstale":"2024-06-10T00:32:50.134764+0000","last_undegraded":"2024-06-10T00:32:50.134764+0000","last_fullsized":"2024-06-10T00:32:50.134764+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:42.123563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031914+0000","last_change":"2024-06-10T00:28:37.657423+0000","last_active":"2024-06-10T00:32:50.031914+0000","last_peered":"2024-06-10T00:32:50.031914+0000","last_clean":"2024-06-10T00:32:50.031914+0000","last_became_active":"2024-06-10T00:28:37.657140+0000","last_became_peered":"2024-06-10T00:28:37.657140+0000","last_unstale":"2024-06-10T00:32:50.031914+0000","last_undegraded":"2024-06-10T00:32:50.031914+0000","last_fullsized":"2024-06-10T00:32:50.031914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:20:08.943066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922033+0000","last_change":"2024-06-10T00:28:34.392585+0000","last_active":"2024-06-10T00:32:49.922033+0000","last_peered":"2024-06-10T00:32:49.922033+0000","last_clean":"2024-06-10T00:32:49.922033+0000","last_became_active":"2024-06-10T00:28:34.392229+0000","last_became_peered":"2024-06-10T00:28:34.392229+0000","last_unstale":"2024-06-10T00:32:49.922033+0000","last_undegraded":"2024-06-10T00:32:49.922033+0000","last_fullsized":"2024-06-10T00:32:49.922033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:11.275761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"100'148","reported_seq":424,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:52.045212+0000","last_change":"2024-06-10T00:27:12.072577+0000","last_active":"2024-06-10T00:32:52.045212+0000","last_peered":"2024-06-10T00:32:52.045212+0000","last_clean":"2024-06-10T00:32:52.045212+0000","last_became_active":"2024-06-10T00:27:12.072193+0000","last_became_peered":"2024-06-10T00:27:12.072193+0000","last_unstale":"2024-06-10T00:32:52.045212+0000","last_undegraded":"2024-06-10T00:32:52.045212+0000","last_fullsized":"2024-06-10T00:32:52.045212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_clean_scrub_stamp":"2024-06-10T00:23:46.826438+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:57.782124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039093+0000","last_change":"2024-06-10T00:28:48.727913+0000","last_active":"2024-06-10T00:32:50.039093+0000","last_peered":"2024-06-10T00:32:50.039093+0000","last_clean":"2024-06-10T00:32:50.039093+0000","last_became_active":"2024-06-10T00:28:48.727402+0000","last_became_peered":"2024-06-10T00:28:48.727402+0000","last_unstale":"2024-06-10T00:32:50.039093+0000","last_undegraded":"2024-06-10T00:32:50.039093+0000","last_fullsized":"2024-06-10T00:32:50.039093+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:10:25.051296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132861+0000","last_change":"2024-06-10T00:28:37.659901+0000","last_active":"2024-06-10T00:32:50.132861+0000","last_peered":"2024-06-10T00:32:50.132861+0000","last_clean":"2024-06-10T00:32:50.132861+0000","last_became_active":"2024-06-10T00:28:37.659553+0000","last_became_peered":"2024-06-10T00:28:37.659553+0000","last_unstale":"2024-06-10T00:32:50.132861+0000","last_undegraded":"2024-06-10T00:32:50.132861+0000","last_fullsized":"2024-06-10T00:32:50.132861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:25.694823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134306+0000","last_change":"2024-06-10T00:28:34.398155+0000","last_active":"2024-06-10T00:32:50.134306+0000","last_peered":"2024-06-10T00:32:50.134306+0000","last_clean":"2024-06-10T00:32:50.134306+0000","last_became_active":"2024-06-10T00:28:34.397793+0000","last_became_peered":"2024-06-10T00:28:34.397793+0000","last_unstale":"2024-06-10T00:32:50.134306+0000","last_undegraded":"2024-06-10T00:32:50.134306+0000","last_fullsized":"2024-06-10T00:32:50.134306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:21.188139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198954+0000","last_change":"2024-06-10T00:28:48.709736+0000","last_active":"2024-06-10T00:32:50.198954+0000","last_peered":"2024-06-10T00:32:50.198954+0000","last_clean":"2024-06-10T00:32:50.198954+0000","last_became_active":"2024-06-10T00:28:48.709293+0000","last_became_peered":"2024-06-10T00:28:48.709293+0000","last_unstale":"2024-06-10T00:32:50.198954+0000","last_undegraded":"2024-06-10T00:32:50.198954+0000","last_fullsized":"2024-06-10T00:32:50.198954+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:30:04.840656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199015+0000","last_change":"2024-06-10T00:29:23.780030+0000","last_active":"2024-06-10T00:32:50.199015+0000","last_peered":"2024-06-10T00:32:50.199015+0000","last_clean":"2024-06-10T00:32:50.199015+0000","last_became_active":"2024-06-10T00:29:23.779589+0000","last_became_peered":"2024-06-10T00:29:23.779589+0000","last_unstale":"2024-06-10T00:32:50.199015+0000","last_undegraded":"2024-06-10T00:32:50.199015+0000","last_fullsized":"2024-06-10T00:32:50.199015+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:53.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133881+0000","last_change":"2024-06-10T00:28:37.658330+0000","last_active":"2024-06-10T00:32:50.133881+0000","last_peered":"2024-06-10T00:32:50.133881+0000","last_clean":"2024-06-10T00:32:50.133881+0000","last_became_active":"2024-06-10T00:28:37.657343+0000","last_became_peered":"2024-06-10T00:28:37.657343+0000","last_unstale":"2024-06-10T00:32:50.133881+0000","last_undegraded":"2024-06-10T00:32:50.133881+0000","last_fullsized":"2024-06-10T00:32:50.133881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:31:52.167402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031813+0000","last_change":"2024-06-10T00:28:48.727506+0000","last_active":"2024-06-10T00:32:50.031813+0000","last_peered":"2024-06-10T00:32:50.031813+0000","last_clean":"2024-06-10T00:32:50.031813+0000","last_became_active":"2024-06-10T00:28:48.727068+0000","last_became_peered":"2024-06-10T00:28:48.727068+0000","last_unstale":"2024-06-10T00:32:50.031813+0000","last_undegraded":"2024-06-10T00:32:50.031813+0000","last_fullsized":"2024-06-10T00:32:50.031813+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:42.147908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'104","reported_seq":244,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:03.056325+0000","last_change":"2024-06-10T00:28:34.399872+0000","last_active":"2024-06-10T00:33:03.056325+0000","last_peered":"2024-06-10T00:33:03.056325+0000","last_clean":"2024-06-10T00:33:03.056325+0000","last_became_active":"2024-06-10T00:28:34.399493+0000","last_became_peered":"2024-06-10T00:28:34.399493+0000","last_unstale":"2024-06-10T00:33:03.056325+0000","last_undegraded":"2024-06-10T00:33:03.056325+0000","last_fullsized":"2024-06-10T00:33:03.056325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:24:10.018110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":190,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032464+0000","last_change":"2024-06-10T00:28:37.656597+0000","last_active":"2024-06-10T00:32:50.032464+0000","last_peered":"2024-06-10T00:32:50.032464+0000","last_clean":"2024-06-10T00:32:50.032464+0000","last_became_active":"2024-06-10T00:28:37.654356+0000","last_became_peered":"2024-06-10T00:28:37.654356+0000","last_unstale":"2024-06-10T00:32:50.032464+0000","last_undegraded":"2024-06-10T00:32:50.032464+0000","last_fullsized":"2024-06-10T00:32:50.032464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:07:12.950683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920789+0000","last_change":"2024-06-10T00:28:48.702686+0000","last_active":"2024-06-10T00:32:49.920789+0000","last_peered":"2024-06-10T00:32:49.920789+0000","last_clean":"2024-06-10T00:32:49.920789+0000","last_became_active":"2024-06-10T00:28:48.702273+0000","last_became_peered":"2024-06-10T00:28:48.702273+0000","last_unstale":"2024-06-10T00:32:49.920789+0000","last_undegraded":"2024-06-10T00:32:49.920789+0000","last_fullsized":"2024-06-10T00:32:49.920789+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:00.078232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921789+0000","last_change":"2024-06-10T00:28:34.447549+0000","last_active":"2024-06-10T00:32:49.921789+0000","last_peered":"2024-06-10T00:32:49.921789+0000","last_clean":"2024-06-10T00:32:49.921789+0000","last_became_active":"2024-06-10T00:28:34.447197+0000","last_became_peered":"2024-06-10T00:28:34.447197+0000","last_unstale":"2024-06-10T00:32:49.921789+0000","last_undegraded":"2024-06-10T00:32:49.921789+0000","last_fullsized":"2024-06-10T00:32:49.921789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:47.525153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193483+0000","last_change":"2024-06-10T00:28:37.655585+0000","last_active":"2024-06-10T00:32:50.193483+0000","last_peered":"2024-06-10T00:32:50.193483+0000","last_clean":"2024-06-10T00:32:50.193483+0000","last_became_active":"2024-06-10T00:28:37.655225+0000","last_became_peered":"2024-06-10T00:28:37.655225+0000","last_unstale":"2024-06-10T00:32:50.193483+0000","last_undegraded":"2024-06-10T00:32:50.193483+0000","last_fullsized":"2024-06-10T00:32:50.193483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:57:25.184924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921606+0000","last_change":"2024-06-10T00:29:23.857447+0000","last_active":"2024-06-10T00:32:49.921606+0000","last_peered":"2024-06-10T00:32:49.921606+0000","last_clean":"2024-06-10T00:32:49.921606+0000","last_became_active":"2024-06-10T00:29:23.856943+0000","last_became_peered":"2024-06-10T00:29:23.856943+0000","last_unstale":"2024-06-10T00:32:49.921606+0000","last_undegraded":"2024-06-10T00:32:49.921606+0000","last_fullsized":"2024-06-10T00:32:49.921606+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:58:44.548818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188371+0000","last_change":"2024-06-10T00:28:34.445637+0000","last_active":"2024-06-10T00:32:50.188371+0000","last_peered":"2024-06-10T00:32:50.188371+0000","last_clean":"2024-06-10T00:32:50.188371+0000","last_became_active":"2024-06-10T00:28:34.445324+0000","last_became_peered":"2024-06-10T00:28:34.445324+0000","last_unstale":"2024-06-10T00:32:50.188371+0000","last_undegraded":"2024-06-10T00:32:50.188371+0000","last_fullsized":"2024-06-10T00:32:50.188371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:05.713557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031670+0000","last_change":"2024-06-10T00:28:37.659037+0000","last_active":"2024-06-10T00:32:50.031670+0000","last_peered":"2024-06-10T00:32:50.031670+0000","last_clean":"2024-06-10T00:32:50.031670+0000","last_became_active":"2024-06-10T00:28:37.658705+0000","last_became_peered":"2024-06-10T00:28:37.658705+0000","last_unstale":"2024-06-10T00:32:50.031670+0000","last_undegraded":"2024-06-10T00:32:50.031670+0000","last_fullsized":"2024-06-10T00:32:50.031670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:20:52.797237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922067+0000","last_change":"2024-06-10T00:28:48.704130+0000","last_active":"2024-06-10T00:32:49.922067+0000","last_peered":"2024-06-10T00:32:49.922067+0000","last_clean":"2024-06-10T00:32:49.922067+0000","last_became_active":"2024-06-10T00:28:48.703738+0000","last_became_peered":"2024-06-10T00:28:48.703738+0000","last_unstale":"2024-06-10T00:32:49.922067+0000","last_undegraded":"2024-06-10T00:32:49.922067+0000","last_fullsized":"2024-06-10T00:32:49.922067+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:01:40.436710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032084+0000","last_change":"2024-06-10T00:28:34.386667+0000","last_active":"2024-06-10T00:32:50.032084+0000","last_peered":"2024-06-10T00:32:50.032084+0000","last_clean":"2024-06-10T00:32:50.032084+0000","last_became_active":"2024-06-10T00:28:34.386363+0000","last_became_peered":"2024-06-10T00:28:34.386363+0000","last_unstale":"2024-06-10T00:32:50.032084+0000","last_undegraded":"2024-06-10T00:32:50.032084+0000","last_fullsized":"2024-06-10T00:32:50.032084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:11.850162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198807+0000","last_change":"2024-06-10T00:28:37.664754+0000","last_active":"2024-06-10T00:32:50.198807+0000","last_peered":"2024-06-10T00:32:50.198807+0000","last_clean":"2024-06-10T00:32:50.198807+0000","last_became_active":"2024-06-10T00:28:37.658574+0000","last_became_peered":"2024-06-10T00:28:37.658574+0000","last_unstale":"2024-06-10T00:32:50.198807+0000","last_undegraded":"2024-06-10T00:32:50.198807+0000","last_fullsized":"2024-06-10T00:32:50.198807+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:16:47.437822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921563+0000","last_change":"2024-06-10T00:28:48.706560+0000","last_active":"2024-06-10T00:32:49.921563+0000","last_peered":"2024-06-10T00:32:49.921563+0000","last_clean":"2024-06-10T00:32:49.921563+0000","last_became_active":"2024-06-10T00:28:48.705986+0000","last_became_peered":"2024-06-10T00:28:48.705986+0000","last_unstale":"2024-06-10T00:32:49.921563+0000","last_undegraded":"2024-06-10T00:32:49.921563+0000","last_fullsized":"2024-06-10T00:32:49.921563+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:21:26.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.892250+0000","last_change":"2024-06-10T00:28:34.394509+0000","last_active":"2024-06-10T00:32:50.892250+0000","last_peered":"2024-06-10T00:32:50.892250+0000","last_clean":"2024-06-10T00:32:50.892250+0000","last_became_active":"2024-06-10T00:28:34.394105+0000","last_became_peered":"2024-06-10T00:28:34.394105+0000","last_unstale":"2024-06-10T00:32:50.892250+0000","last_undegraded":"2024-06-10T00:32:50.892250+0000","last_fullsized":"2024-06-10T00:32:50.892250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:13.415544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032324+0000","last_change":"2024-06-10T00:28:37.653192+0000","last_active":"2024-06-10T00:32:50.032324+0000","last_peered":"2024-06-10T00:32:50.032324+0000","last_clean":"2024-06-10T00:32:50.032324+0000","last_became_active":"2024-06-10T00:28:37.652868+0000","last_became_peered":"2024-06-10T00:28:37.652868+0000","last_unstale":"2024-06-10T00:32:50.032324+0000","last_undegraded":"2024-06-10T00:32:50.032324+0000","last_fullsized":"2024-06-10T00:32:50.032324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:21.614915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921695+0000","last_change":"2024-06-10T00:28:48.727615+0000","last_active":"2024-06-10T00:32:49.921695+0000","last_peered":"2024-06-10T00:32:49.921695+0000","last_clean":"2024-06-10T00:32:49.921695+0000","last_became_active":"2024-06-10T00:28:48.727190+0000","last_became_peered":"2024-06-10T00:28:48.727190+0000","last_unstale":"2024-06-10T00:32:49.921695+0000","last_undegraded":"2024-06-10T00:32:49.921695+0000","last_fullsized":"2024-06-10T00:32:49.921695+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:57.498183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031777+0000","last_change":"2024-06-10T00:29:23.784131+0000","last_active":"2024-06-10T00:32:50.031777+0000","last_peered":"2024-06-10T00:32:50.031777+0000","last_clean":"2024-06-10T00:32:50.031777+0000","last_became_active":"2024-06-10T00:29:23.780909+0000","last_became_peered":"2024-06-10T00:29:23.780909+0000","last_unstale":"2024-06-10T00:32:50.031777+0000","last_undegraded":"2024-06-10T00:32:50.031777+0000","last_fullsized":"2024-06-10T00:32:50.031777+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.774217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134481+0000","last_change":"2024-06-10T00:28:37.660399+0000","last_active":"2024-06-10T00:32:50.134481+0000","last_peered":"2024-06-10T00:32:50.134481+0000","last_clean":"2024-06-10T00:32:50.134481+0000","last_became_active":"2024-06-10T00:28:37.660069+0000","last_became_peered":"2024-06-10T00:28:37.660069+0000","last_unstale":"2024-06-10T00:32:50.134481+0000","last_undegraded":"2024-06-10T00:32:50.134481+0000","last_fullsized":"2024-06-10T00:32:50.134481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:59.727883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188428+0000","last_change":"2024-06-10T00:28:48.713267+0000","last_active":"2024-06-10T00:32:50.188428+0000","last_peered":"2024-06-10T00:32:50.188428+0000","last_clean":"2024-06-10T00:32:50.188428+0000","last_became_active":"2024-06-10T00:28:48.712870+0000","last_became_peered":"2024-06-10T00:28:48.712870+0000","last_unstale":"2024-06-10T00:32:50.188428+0000","last_undegraded":"2024-06-10T00:32:50.188428+0000","last_fullsized":"2024-06-10T00:32:50.188428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:59.210101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'6","reported_seq":256,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:05.184542+0000","last_change":"2024-06-10T00:28:34.448652+0000","last_active":"2024-06-10T00:33:05.184542+0000","last_peered":"2024-06-10T00:33:05.184542+0000","last_clean":"2024-06-10T00:33:05.184542+0000","last_became_active":"2024-06-10T00:28:34.448243+0000","last_became_peered":"2024-06-10T00:28:34.448243+0000","last_unstale":"2024-06-10T00:33:05.184542+0000","last_undegraded":"2024-06-10T00:33:05.184542+0000","last_fullsized":"2024-06-10T00:33:05.184542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:51:40.003429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.135023+0000","last_change":"2024-06-10T00:28:37.662307+0000","last_active":"2024-06-10T00:32:50.135023+0000","last_peered":"2024-06-10T00:32:50.135023+0000","last_clean":"2024-06-10T00:32:50.135023+0000","last_became_active":"2024-06-10T00:28:37.661727+0000","last_became_peered":"2024-06-10T00:28:37.661727+0000","last_unstale":"2024-06-10T00:32:50.135023+0000","last_undegraded":"2024-06-10T00:32:50.135023+0000","last_fullsized":"2024-06-10T00:32:50.135023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:23.989429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192376+0000","last_change":"2024-06-10T00:29:23.857626+0000","last_active":"2024-06-10T00:32:50.192376+0000","last_peered":"2024-06-10T00:32:50.192376+0000","last_clean":"2024-06-10T00:32:50.192376+0000","last_became_active":"2024-06-10T00:29:23.857057+0000","last_became_peered":"2024-06-10T00:29:23.857057+0000","last_unstale":"2024-06-10T00:32:50.192376+0000","last_undegraded":"2024-06-10T00:32:50.192376+0000","last_fullsized":"2024-06-10T00:32:50.192376+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:25:10.340053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'67","reported_seq":145,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:03.056001+0000","last_change":"2024-06-10T00:28:34.399253+0000","last_active":"2024-06-10T00:33:03.056001+0000","last_peered":"2024-06-10T00:33:03.056001+0000","last_clean":"2024-06-10T00:33:03.056001+0000","last_became_active":"2024-06-10T00:28:34.397763+0000","last_became_peered":"2024-06-10T00:28:34.397763+0000","last_unstale":"2024-06-10T00:33:03.056001+0000","last_undegraded":"2024-06-10T00:33:03.056001+0000","last_fullsized":"2024-06-10T00:33:03.056001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:05:57.371625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134695+0000","last_change":"2024-06-10T00:28:37.658027+0000","last_active":"2024-06-10T00:32:50.134695+0000","last_peered":"2024-06-10T00:32:50.134695+0000","last_clean":"2024-06-10T00:32:50.134695+0000","last_became_active":"2024-06-10T00:28:37.655513+0000","last_became_peered":"2024-06-10T00:28:37.655513+0000","last_unstale":"2024-06-10T00:32:50.134695+0000","last_undegraded":"2024-06-10T00:32:50.134695+0000","last_fullsized":"2024-06-10T00:32:50.134695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:50:23.525855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038558+0000","last_change":"2024-06-10T00:28:48.706997+0000","last_active":"2024-06-10T00:32:50.038558+0000","last_peered":"2024-06-10T00:32:50.038558+0000","last_clean":"2024-06-10T00:32:50.038558+0000","last_became_active":"2024-06-10T00:28:48.706597+0000","last_became_peered":"2024-06-10T00:28:48.706597+0000","last_unstale":"2024-06-10T00:32:50.038558+0000","last_undegraded":"2024-06-10T00:32:50.038558+0000","last_fullsized":"2024-06-10T00:32:50.038558+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:16.385312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133955+0000","last_change":"2024-06-10T00:29:23.785774+0000","last_active":"2024-06-10T00:32:50.133955+0000","last_peered":"2024-06-10T00:32:50.133955+0000","last_clean":"2024-06-10T00:32:50.133955+0000","last_became_active":"2024-06-10T00:29:23.781768+0000","last_became_peered":"2024-06-10T00:29:23.781768+0000","last_unstale":"2024-06-10T00:32:50.133955+0000","last_undegraded":"2024-06-10T00:32:50.133955+0000","last_fullsized":"2024-06-10T00:32:50.133955+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:27.348264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134953+0000","last_change":"2024-06-10T00:28:37.657820+0000","last_active":"2024-06-10T00:32:50.134953+0000","last_peered":"2024-06-10T00:32:50.134953+0000","last_clean":"2024-06-10T00:32:50.134953+0000","last_became_active":"2024-06-10T00:28:37.657487+0000","last_became_peered":"2024-06-10T00:28:37.657487+0000","last_unstale":"2024-06-10T00:32:50.134953+0000","last_undegraded":"2024-06-10T00:32:50.134953+0000","last_fullsized":"2024-06-10T00:32:50.134953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:44:35.930401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198953+0000","last_change":"2024-06-10T00:28:37.664543+0000","last_active":"2024-06-10T00:32:50.198953+0000","last_peered":"2024-06-10T00:32:50.198953+0000","last_clean":"2024-06-10T00:32:50.198953+0000","last_became_active":"2024-06-10T00:28:37.659015+0000","last_became_peered":"2024-06-10T00:28:37.659015+0000","last_unstale":"2024-06-10T00:32:50.198953+0000","last_undegraded":"2024-06-10T00:32:50.198953+0000","last_fullsized":"2024-06-10T00:32:50.198953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:06.555472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921072+0000","last_change":"2024-06-10T00:28:37.659389+0000","last_active":"2024-06-10T00:32:49.921072+0000","last_peered":"2024-06-10T00:32:49.921072+0000","last_clean":"2024-06-10T00:32:49.921072+0000","last_became_active":"2024-06-10T00:28:37.658990+0000","last_became_peered":"2024-06-10T00:28:37.658990+0000","last_unstale":"2024-06-10T00:32:49.921072+0000","last_undegraded":"2024-06-10T00:32:49.921072+0000","last_fullsized":"2024-06-10T00:32:49.921072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:29.532779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920823+0000","last_change":"2024-06-10T00:28:37.655621+0000","last_active":"2024-06-10T00:32:49.920823+0000","last_peered":"2024-06-10T00:32:49.920823+0000","last_clean":"2024-06-10T00:32:49.920823+0000","last_became_active":"2024-06-10T00:28:37.653599+0000","last_became_peered":"2024-06-10T00:28:37.653599+0000","last_unstale":"2024-06-10T00:32:49.920823+0000","last_undegraded":"2024-06-10T00:32:49.920823+0000","last_fullsized":"2024-06-10T00:32:49.920823+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:22.165851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192846+0000","last_change":"2024-06-10T00:29:23.782727+0000","last_active":"2024-06-10T00:32:50.192846+0000","last_peered":"2024-06-10T00:32:50.192846+0000","last_clean":"2024-06-10T00:32:50.192846+0000","last_became_active":"2024-06-10T00:29:23.782309+0000","last_became_peered":"2024-06-10T00:29:23.782309+0000","last_unstale":"2024-06-10T00:32:50.192846+0000","last_undegraded":"2024-06-10T00:32:50.192846+0000","last_fullsized":"2024-06-10T00:32:50.192846+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:20.585955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133931+0000","last_change":"2024-06-10T00:28:37.646750+0000","last_active":"2024-06-10T00:32:50.133931+0000","last_peered":"2024-06-10T00:32:50.133931+0000","last_clean":"2024-06-10T00:32:50.133931+0000","last_became_active":"2024-06-10T00:28:37.646372+0000","last_became_peered":"2024-06-10T00:28:37.646372+0000","last_unstale":"2024-06-10T00:32:50.133931+0000","last_undegraded":"2024-06-10T00:32:50.133931+0000","last_fullsized":"2024-06-10T00:32:50.133931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:45.762956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199378+0000","last_change":"2024-06-10T00:28:37.662278+0000","last_active":"2024-06-10T00:32:50.199378+0000","last_peered":"2024-06-10T00:32:50.199378+0000","last_clean":"2024-06-10T00:32:50.199378+0000","last_became_active":"2024-06-10T00:28:37.661855+0000","last_became_peered":"2024-06-10T00:28:37.661855+0000","last_unstale":"2024-06-10T00:32:50.199378+0000","last_undegraded":"2024-06-10T00:32:50.199378+0000","last_fullsized":"2024-06-10T00:32:50.199378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:20.074208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921342+0000","last_change":"2024-06-10T00:28:37.656738+0000","last_active":"2024-06-10T00:32:49.921342+0000","last_peered":"2024-06-10T00:32:49.921342+0000","last_clean":"2024-06-10T00:32:49.921342+0000","last_became_active":"2024-06-10T00:28:37.656375+0000","last_became_peered":"2024-06-10T00:28:37.656375+0000","last_unstale":"2024-06-10T00:32:49.921342+0000","last_undegraded":"2024-06-10T00:32:49.921342+0000","last_fullsized":"2024-06-10T00:32:49.921342+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:49:17.779670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032601+0000","last_change":"2024-06-10T00:28:37.656849+0000","last_active":"2024-06-10T00:32:50.032601+0000","last_peered":"2024-06-10T00:32:50.032601+0000","last_clean":"2024-06-10T00:32:50.032601+0000","last_became_active":"2024-06-10T00:28:37.656261+0000","last_became_peered":"2024-06-10T00:28:37.656261+0000","last_unstale":"2024-06-10T00:32:50.032601+0000","last_undegraded":"2024-06-10T00:32:50.032601+0000","last_fullsized":"2024-06-10T00:32:50.032601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:13.152165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038849+0000","last_change":"2024-06-10T00:28:37.648384+0000","last_active":"2024-06-10T00:32:50.038849+0000","last_peered":"2024-06-10T00:32:50.038849+0000","last_clean":"2024-06-10T00:32:50.038849+0000","last_became_active":"2024-06-10T00:28:37.648009+0000","last_became_peered":"2024-06-10T00:28:37.648009+0000","last_unstale":"2024-06-10T00:32:50.038849+0000","last_undegraded":"2024-06-10T00:32:50.038849+0000","last_fullsized":"2024-06-10T00:32:50.038849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:45:55.808932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032228+0000","last_change":"2024-06-10T00:28:37.655828+0000","last_active":"2024-06-10T00:32:50.032228+0000","last_peered":"2024-06-10T00:32:50.032228+0000","last_clean":"2024-06-10T00:32:50.032228+0000","last_became_active":"2024-06-10T00:28:37.655269+0000","last_became_peered":"2024-06-10T00:28:37.655269+0000","last_unstale":"2024-06-10T00:32:50.032228+0000","last_undegraded":"2024-06-10T00:32:50.032228+0000","last_fullsized":"2024-06-10T00:32:50.032228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:56.618469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188660+0000","last_change":"2024-06-10T00:28:37.663756+0000","last_active":"2024-06-10T00:32:50.188660+0000","last_peered":"2024-06-10T00:32:50.188660+0000","last_clean":"2024-06-10T00:32:50.188660+0000","last_became_active":"2024-06-10T00:28:37.663380+0000","last_became_peered":"2024-06-10T00:28:37.663380+0000","last_unstale":"2024-06-10T00:32:50.188660+0000","last_undegraded":"2024-06-10T00:32:50.188660+0000","last_fullsized":"2024-06-10T00:32:50.188660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:45.534648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133127+0000","last_change":"2024-06-10T00:28:37.655246+0000","last_active":"2024-06-10T00:32:50.133127+0000","last_peered":"2024-06-10T00:32:50.133127+0000","last_clean":"2024-06-10T00:32:50.133127+0000","last_became_active":"2024-06-10T00:28:37.654605+0000","last_became_peered":"2024-06-10T00:28:37.654605+0000","last_unstale":"2024-06-10T00:32:50.133127+0000","last_undegraded":"2024-06-10T00:32:50.133127+0000","last_fullsized":"2024-06-10T00:32:50.133127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:15.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199118+0000","last_change":"2024-06-10T00:28:37.662485+0000","last_active":"2024-06-10T00:32:50.199118+0000","last_peered":"2024-06-10T00:32:50.199118+0000","last_clean":"2024-06-10T00:32:50.199118+0000","last_became_active":"2024-06-10T00:28:37.661900+0000","last_became_peered":"2024-06-10T00:28:37.661900+0000","last_unstale":"2024-06-10T00:32:50.199118+0000","last_undegraded":"2024-06-10T00:32:50.199118+0000","last_fullsized":"2024-06-10T00:32:50.199118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:30:03.210634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800945,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2398,"num_read_kb":1425,"num_write":512,"num_write_kb":2392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2402835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":390,"ondisk_log_size":390,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383749,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1175552,"data_stored":581073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645385,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29280,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715872,"statfs":{"total":95995035648,"available":95965052928,"internally_reserved":0,"allocated":2195456,"data_stored":1565815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808910,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":977723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070549,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36860,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708292,"statfs":{"total":95995035648,"available":95957291008,"internally_reserved":0,"allocated":1560576,"data_stored":973221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332185,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1781760,"data_stored":1180215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495710,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":577003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36556,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708596,"statfs":{"total":95995035648,"available":95957602304,"internally_reserved":0,"allocated":1245184,"data_stored":648070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36500,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708652,"statfs":{"total":95995035648,"available":95957659648,"internally_reserved":0,"allocated":1187840,"data_stored":577761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36564,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708588,"statfs":{"total":95995035648,"available":95957594112,"internally_reserved":0,"allocated":1257472,"data_stored":648231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182509,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28992,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716160,"statfs":{"total":95995035648,"available":95965347840,"internally_reserved":0,"allocated":1900544,"data_stored":1297071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920872,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36484,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708668,"statfs":{"total":95995035648,"available":95957676032,"internally_reserved":0,"allocated":1171456,"data_stored":580910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757347,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29204,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715948,"statfs":{"total":95995035648,"available":95965130752,"internally_reserved":0,"allocated":2117632,"data_stored":1501930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":332273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":400254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:33:10.873 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-10T00:33:10.874 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T00:33:11.128 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:11.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:11 smithi062 ceph-mon[32390]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-06-10T00:33:11.466 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:11 smithi062 ceph-mon[32390]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:33:11.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:11 smithi105 ceph-mon[36321]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-06-10T00:33:11.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:11 smithi105 ceph-mon[36321]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:33:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:11 smithi138 ceph-mon[36549]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-06-10T00:33:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:11 smithi138 ceph-mon[36549]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:33:12.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.838+0000 7f1cf9a10700 1 -- 172.21.15.62:0/556552072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf4101d30 msgr2=0x7f1cf410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:12.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.838+0000 7f1cf9a10700 1 --2- 172.21.15.62:0/556552072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf4101d30 0x7f1cf410e210 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00b0a0 tx=0x7f1cec00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:33:12.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.838+0000 7f1cf9a10700 1 -- 172.21.15.62:0/556552072 shutdown_connections 2024-06-10T00:33:12.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.838+0000 7f1cf9a10700 1 --2- 172.21.15.62:0/556552072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf4101d30 0x7f1cf410e210 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:12.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.838+0000 7f1cf9a10700 1 --2- 172.21.15.62:0/556552072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cf41073d0 0x7f1cf41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:12.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.838+0000 7f1cf9a10700 1 --2- 172.21.15.62:0/556552072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf4101390 0x7f1cf41017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:12.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.838+0000 7f1cf9a10700 1 -- 172.21.15.62:0/556552072 >> 172.21.15.62:0/556552072 conn(0x7f1cf40782c0 msgr2=0x7f1cf40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:12.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.838+0000 7f1cf9a10700 1 -- 172.21.15.62:0/556552072 shutdown_connections 2024-06-10T00:33:12.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.838+0000 7f1cf9a10700 1 -- 172.21.15.62:0/556552072 wait complete. 2024-06-10T00:33:12.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.839+0000 7f1cf9a10700 1 Processor -- start 2024-06-10T00:33:12.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.839+0000 7f1cf9a10700 1 -- start start 2024-06-10T00:33:12.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.839+0000 7f1cf9a10700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cf4101390 0x7f1cf419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:12.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.839+0000 7f1cf9a10700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf4101d30 0x7f1cf419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:12.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.839+0000 7f1cf9a10700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf41073d0 0x7f1cf4198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:12.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.839+0000 7f1cf9a10700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4111620 con 0x7f1cf4101d30 2024-06-10T00:33:12.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.839+0000 7f1cf9a10700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41114a0 con 0x7f1cf4101390 2024-06-10T00:33:12.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.839+0000 7f1cf9a10700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41117a0 con 0x7f1cf41073d0 2024-06-10T00:33:12.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf4101d30 0x7f1cf419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:12.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf4101d30 0x7f1cf419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47796/0 (socket says 172.21.15.62:47796) 2024-06-10T00:33:12.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf27fc700 1 -- 172.21.15.62:0/843179553 learned_addr learned my addr 172.21.15.62:0/843179553 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:12.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf2ffd700 1 --2- 172.21.15.62:0/843179553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cf4101390 0x7f1cf419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:12.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf37fe700 1 --2- 172.21.15.62:0/843179553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf41073d0 0x7f1cf4198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:12.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf27fc700 1 -- 172.21.15.62:0/843179553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf41073d0 msgr2=0x7f1cf4198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:12.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf27fc700 1 --2- 172.21.15.62:0/843179553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf41073d0 0x7f1cf4198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:12.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf27fc700 1 -- 172.21.15.62:0/843179553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cf4101390 msgr2=0x7f1cf419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:12.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf27fc700 1 --2- 172.21.15.62:0/843179553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cf4101390 0x7f1cf419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:12.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf27fc700 1 -- 172.21.15.62:0/843179553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf4199160 con 0x7f1cf4101d30 2024-06-10T00:33:12.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf27fc700 1 --2- 172.21.15.62:0/843179553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf4101d30 0x7f1cf419ed30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800d950 tx=0x7f1ce800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:12.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf8a0e700 1 -- 172.21.15.62:0/843179553 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8016070 con 0x7f1cf4101d30 2024-06-10T00:33:12.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf9a10700 1 -- 172.21.15.62:0/843179553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf4199450 con 0x7f1cf4101d30 2024-06-10T00:33:12.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf8a0e700 1 -- 172.21.15.62:0/843179553 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce800fd70 con 0x7f1cf4101d30 2024-06-10T00:33:12.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf9a10700 1 -- 172.21.15.62:0/843179553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf41a56c0 con 0x7f1cf4101d30 2024-06-10T00:33:12.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.840+0000 7f1cf8a0e700 1 -- 172.21.15.62:0/843179553 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce800e610 con 0x7f1cf4101d30 2024-06-10T00:33:12.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.841+0000 7f1cf9a10700 1 -- 172.21.15.62:0/843179553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf4069180 con 0x7f1cf4101d30 2024-06-10T00:33:12.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.843+0000 7f1cf8a0e700 1 -- 172.21.15.62:0/843179553 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f1ce8019070 con 0x7f1cf4101d30 2024-06-10T00:33:12.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.843+0000 7f1cf8a0e700 1 --2- 172.21.15.62:0/843179553 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1cdc0a72b0 0x7f1cdc0a9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:12.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.843+0000 7f1cf2ffd700 1 --2- 172.21.15.62:0/843179553 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1cdc0a72b0 0x7f1cdc0a9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:12.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.843+0000 7f1cf8a0e700 1 -- 172.21.15.62:0/843179553 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f1ce80cf830 con 0x7f1cf4101d30 2024-06-10T00:33:12.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.844+0000 7f1cf2ffd700 1 --2- 172.21.15.62:0/843179553 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1cdc0a72b0 0x7f1cdc0a9770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40ffbe0 tx=0x7f1ce4009260 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:12.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:12.847+0000 7f1cf8a0e700 1 -- 172.21.15.62:0/843179553 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1ce809bf30 con 0x7f1cf4101d30 2024-06-10T00:33:13.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:13.049+0000 7f1cf9a10700 1 -- 172.21.15.62:0/843179553 --> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1cf40628d0 con 0x7f1cdc0a72b0 2024-06-10T00:33:13.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:13.059+0000 7f1cf8a0e700 1 -- 172.21.15.62:0/843179553 <== mgr.15333 v2:172.21.15.62:6800/518034449 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339604 (secure 0 0 0) 0x7f1cf40628d0 con 0x7f1cdc0a72b0 2024-06-10T00:33:13.063 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:33:13.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:13.062+0000 7f1cf9a10700 1 -- 172.21.15.62:0/843179553 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1cdc0a72b0 msgr2=0x7f1cdc0a9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:13.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:13.062+0000 7f1cf9a10700 1 --2- 172.21.15.62:0/843179553 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1cdc0a72b0 0x7f1cdc0a9770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40ffbe0 tx=0x7f1ce4009260 comp rx=0 tx=0).stop 2024-06-10T00:33:13.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:13.062+0000 7f1cf9a10700 1 -- 172.21.15.62:0/843179553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf4101d30 msgr2=0x7f1cf419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:13.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:13.062+0000 7f1cf9a10700 1 --2- 172.21.15.62:0/843179553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf4101d30 0x7f1cf419ed30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800d950 tx=0x7f1ce800dc60 comp rx=0 tx=0).stop 2024-06-10T00:33:13.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:13.062+0000 7f1cf9a10700 1 -- 172.21.15.62:0/843179553 shutdown_connections 2024-06-10T00:33:13.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:13.062+0000 7f1cf9a10700 1 --2- 172.21.15.62:0/843179553 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1cdc0a72b0 0x7f1cdc0a9770 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:13.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:13.062+0000 7f1cf9a10700 1 --2- 172.21.15.62:0/843179553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf41073d0 0x7f1cf4198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:13.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:13.062+0000 7f1cf9a10700 1 --2- 172.21.15.62:0/843179553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1cf4101390 0x7f1cf419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:13.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:13.062+0000 7f1cf9a10700 1 --2- 172.21.15.62:0/843179553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf4101d30 0x7f1cf419ed30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:13.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:13.062+0000 7f1cf9a10700 1 -- 172.21.15.62:0/843179553 >> 172.21.15.62:0/843179553 conn(0x7f1cf40782c0 msgr2=0x7f1cf41006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:13.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:13.063+0000 7f1cf9a10700 1 -- 172.21.15.62:0/843179553 shutdown_connections 2024-06-10T00:33:13.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:13.063+0000 7f1cf9a10700 1 -- 172.21.15.62:0/843179553 wait complete. 2024-06-10T00:33:13.067 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-06-10T00:33:13.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:13 smithi062 ceph-mon[32390]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-06-10T00:33:13.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:13 smithi105 ceph-mon[36321]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-06-10T00:33:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:13 smithi138 ceph-mon[36549]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-06-10T00:33:13.735 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-06-10T00:33:11.939332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391547,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2685,"num_read_kb":1780,"num_write":715,"num_write_kb":5332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1040,"ondisk_log_size":1040,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388248,"kb_used_data":17912,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124553576,"statfs":{"total":1151940427776,"available":1151542861824,"internally_reserved":0,"allocated":18341888,"data_stored":11109023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":378713403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":405,"num_read_kb":307,"num_write":126,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003223"},"pg_stats":[{"pgid":"3.3f","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199389+0000","last_change":"2024-06-10T00:28:37.657234+0000","last_active":"2024-06-10T00:32:50.199389+0000","last_peered":"2024-06-10T00:32:50.199389+0000","last_clean":"2024-06-10T00:32:50.199389+0000","last_became_active":"2024-06-10T00:28:37.656879+0000","last_became_peered":"2024-06-10T00:28:37.656879+0000","last_unstale":"2024-06-10T00:32:50.199389+0000","last_undegraded":"2024-06-10T00:32:50.199389+0000","last_fullsized":"2024-06-10T00:32:50.199389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:01:02.907597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"99'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193442+0000","last_change":"2024-06-10T00:28:37.656440+0000","last_active":"2024-06-10T00:32:50.193442+0000","last_peered":"2024-06-10T00:32:50.193442+0000","last_clean":"2024-06-10T00:32:50.193442+0000","last_became_active":"2024-06-10T00:28:37.656113+0000","last_became_peered":"2024-06-10T00:28:37.656113+0000","last_unstale":"2024-06-10T00:32:50.193442+0000","last_undegraded":"2024-06-10T00:32:50.193442+0000","last_fullsized":"2024-06-10T00:32:50.193442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:49:31.447951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193070+0000","last_change":"2024-06-10T00:28:37.659114+0000","last_active":"2024-06-10T00:32:50.193070+0000","last_peered":"2024-06-10T00:32:50.193070+0000","last_clean":"2024-06-10T00:32:50.193070+0000","last_became_active":"2024-06-10T00:28:37.658763+0000","last_became_peered":"2024-06-10T00:28:37.658763+0000","last_unstale":"2024-06-10T00:32:50.193070+0000","last_undegraded":"2024-06-10T00:32:50.193070+0000","last_fullsized":"2024-06-10T00:32:50.193070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:39:54.963590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921896+0000","last_change":"2024-06-10T00:28:37.656786+0000","last_active":"2024-06-10T00:32:49.921896+0000","last_peered":"2024-06-10T00:32:49.921896+0000","last_clean":"2024-06-10T00:32:49.921896+0000","last_became_active":"2024-06-10T00:28:37.656073+0000","last_became_peered":"2024-06-10T00:28:37.656073+0000","last_unstale":"2024-06-10T00:32:49.921896+0000","last_undegraded":"2024-06-10T00:32:49.921896+0000","last_fullsized":"2024-06-10T00:32:49.921896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:19.176608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"99'4","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031746+0000","last_change":"2024-06-10T00:28:37.650250+0000","last_active":"2024-06-10T00:32:50.031746+0000","last_peered":"2024-06-10T00:32:50.031746+0000","last_clean":"2024-06-10T00:32:50.031746+0000","last_became_active":"2024-06-10T00:28:37.649852+0000","last_became_peered":"2024-06-10T00:28:37.649852+0000","last_unstale":"2024-06-10T00:32:50.031746+0000","last_undegraded":"2024-06-10T00:32:50.031746+0000","last_fullsized":"2024-06-10T00:32:50.031746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:51:39.492884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042392+0000","last_change":"2024-06-10T00:28:37.646042+0000","last_active":"2024-06-10T00:32:50.042392+0000","last_peered":"2024-06-10T00:32:50.042392+0000","last_clean":"2024-06-10T00:32:50.042392+0000","last_became_active":"2024-06-10T00:28:37.644591+0000","last_became_peered":"2024-06-10T00:28:37.644591+0000","last_unstale":"2024-06-10T00:32:50.042392+0000","last_undegraded":"2024-06-10T00:32:50.042392+0000","last_fullsized":"2024-06-10T00:32:50.042392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:29.932303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"99'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134284+0000","last_change":"2024-06-10T00:28:37.664076+0000","last_active":"2024-06-10T00:32:50.134284+0000","last_peered":"2024-06-10T00:32:50.134284+0000","last_clean":"2024-06-10T00:32:50.134284+0000","last_became_active":"2024-06-10T00:28:37.663741+0000","last_became_peered":"2024-06-10T00:28:37.663741+0000","last_unstale":"2024-06-10T00:32:50.134284+0000","last_undegraded":"2024-06-10T00:32:50.134284+0000","last_fullsized":"2024-06-10T00:32:50.134284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:14.581409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032039+0000","last_change":"2024-06-10T00:28:37.648758+0000","last_active":"2024-06-10T00:32:50.032039+0000","last_peered":"2024-06-10T00:32:50.032039+0000","last_clean":"2024-06-10T00:32:50.032039+0000","last_became_active":"2024-06-10T00:28:37.648387+0000","last_became_peered":"2024-06-10T00:28:37.648387+0000","last_unstale":"2024-06-10T00:32:50.032039+0000","last_undegraded":"2024-06-10T00:32:50.032039+0000","last_fullsized":"2024-06-10T00:32:50.032039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:17.187265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.189039+0000","last_change":"2024-06-10T00:28:37.664650+0000","last_active":"2024-06-10T00:32:50.189039+0000","last_peered":"2024-06-10T00:32:50.189039+0000","last_clean":"2024-06-10T00:32:50.189039+0000","last_became_active":"2024-06-10T00:28:37.663178+0000","last_became_peered":"2024-06-10T00:28:37.663178+0000","last_unstale":"2024-06-10T00:32:50.189039+0000","last_undegraded":"2024-06-10T00:32:50.189039+0000","last_fullsized":"2024-06-10T00:32:50.189039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:46.032078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"99'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042870+0000","last_change":"2024-06-10T00:28:37.658353+0000","last_active":"2024-06-10T00:32:50.042870+0000","last_peered":"2024-06-10T00:32:50.042870+0000","last_clean":"2024-06-10T00:32:50.042870+0000","last_became_active":"2024-06-10T00:28:37.657961+0000","last_became_peered":"2024-06-10T00:28:37.657961+0000","last_unstale":"2024-06-10T00:32:50.042870+0000","last_undegraded":"2024-06-10T00:32:50.042870+0000","last_fullsized":"2024-06-10T00:32:50.042870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:22:27.050687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134878+0000","last_change":"2024-06-10T00:28:37.663823+0000","last_active":"2024-06-10T00:32:50.134878+0000","last_peered":"2024-06-10T00:32:50.134878+0000","last_clean":"2024-06-10T00:32:50.134878+0000","last_became_active":"2024-06-10T00:28:37.663499+0000","last_became_peered":"2024-06-10T00:28:37.663499+0000","last_unstale":"2024-06-10T00:32:50.134878+0000","last_undegraded":"2024-06-10T00:32:50.134878+0000","last_fullsized":"2024-06-10T00:32:50.134878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:30.100923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"99'7","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921877+0000","last_change":"2024-06-10T00:30:24.112722+0000","last_active":"2024-06-10T00:32:49.921877+0000","last_peered":"2024-06-10T00:32:49.921877+0000","last_clean":"2024-06-10T00:32:49.921877+0000","last_became_active":"2024-06-10T00:30:24.112325+0000","last_became_peered":"2024-06-10T00:30:24.112325+0000","last_unstale":"2024-06-10T00:32:49.921877+0000","last_undegraded":"2024-06-10T00:32:49.921877+0000","last_fullsized":"2024-06-10T00:32:49.921877+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:27:26.671583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133292+0000","last_change":"2024-06-10T00:28:37.649997+0000","last_active":"2024-06-10T00:32:50.133292+0000","last_peered":"2024-06-10T00:32:50.133292+0000","last_clean":"2024-06-10T00:32:50.133292+0000","last_became_active":"2024-06-10T00:28:37.649494+0000","last_became_peered":"2024-06-10T00:28:37.649494+0000","last_unstale":"2024-06-10T00:32:50.133292+0000","last_undegraded":"2024-06-10T00:32:50.133292+0000","last_fullsized":"2024-06-10T00:32:50.133292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:44:07.467206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192453+0000","last_change":"2024-06-10T00:28:37.653064+0000","last_active":"2024-06-10T00:32:50.192453+0000","last_peered":"2024-06-10T00:32:50.192453+0000","last_clean":"2024-06-10T00:32:50.192453+0000","last_became_active":"2024-06-10T00:28:37.652684+0000","last_became_peered":"2024-06-10T00:28:37.652684+0000","last_unstale":"2024-06-10T00:32:50.192453+0000","last_undegraded":"2024-06-10T00:32:50.192453+0000","last_fullsized":"2024-06-10T00:32:50.192453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:30.650669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192384+0000","last_change":"2024-06-10T00:28:37.655570+0000","last_active":"2024-06-10T00:32:50.192384+0000","last_peered":"2024-06-10T00:32:50.192384+0000","last_clean":"2024-06-10T00:32:50.192384+0000","last_became_active":"2024-06-10T00:28:37.655241+0000","last_became_peered":"2024-06-10T00:28:37.655241+0000","last_unstale":"2024-06-10T00:32:50.192384+0000","last_undegraded":"2024-06-10T00:32:50.192384+0000","last_fullsized":"2024-06-10T00:32:50.192384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:07:03.543064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188211+0000","last_change":"2024-06-10T00:28:37.665607+0000","last_active":"2024-06-10T00:32:50.188211+0000","last_peered":"2024-06-10T00:32:50.188211+0000","last_clean":"2024-06-10T00:32:50.188211+0000","last_became_active":"2024-06-10T00:28:37.664943+0000","last_became_peered":"2024-06-10T00:28:37.664943+0000","last_unstale":"2024-06-10T00:32:50.188211+0000","last_undegraded":"2024-06-10T00:32:50.188211+0000","last_fullsized":"2024-06-10T00:32:50.188211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:43:02.940298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"99'8","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199188+0000","last_change":"2024-06-10T00:28:37.658214+0000","last_active":"2024-06-10T00:32:50.199188+0000","last_peered":"2024-06-10T00:32:50.199188+0000","last_clean":"2024-06-10T00:32:50.199188+0000","last_became_active":"2024-06-10T00:28:37.657295+0000","last_became_peered":"2024-06-10T00:28:37.657295+0000","last_unstale":"2024-06-10T00:32:50.199188+0000","last_undegraded":"2024-06-10T00:32:50.199188+0000","last_fullsized":"2024-06-10T00:32:50.199188+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:07:20.091598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"99'10","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922079+0000","last_change":"2024-06-10T00:28:37.656645+0000","last_active":"2024-06-10T00:32:49.922079+0000","last_peered":"2024-06-10T00:32:49.922079+0000","last_clean":"2024-06-10T00:32:49.922079+0000","last_became_active":"2024-06-10T00:28:37.656157+0000","last_became_peered":"2024-06-10T00:28:37.656157+0000","last_unstale":"2024-06-10T00:32:49.922079+0000","last_undegraded":"2024-06-10T00:32:49.922079+0000","last_fullsized":"2024-06-10T00:32:49.922079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:00.953804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921620+0000","last_change":"2024-06-10T00:28:37.664336+0000","last_active":"2024-06-10T00:32:49.921620+0000","last_peered":"2024-06-10T00:32:49.921620+0000","last_clean":"2024-06-10T00:32:49.921620+0000","last_became_active":"2024-06-10T00:28:37.663960+0000","last_became_peered":"2024-06-10T00:28:37.663960+0000","last_unstale":"2024-06-10T00:32:49.921620+0000","last_undegraded":"2024-06-10T00:32:49.921620+0000","last_fullsized":"2024-06-10T00:32:49.921620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:59:32.774442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920810+0000","last_change":"2024-06-10T00:28:48.705930+0000","last_active":"2024-06-10T00:32:49.920810+0000","last_peered":"2024-06-10T00:32:49.920810+0000","last_clean":"2024-06-10T00:32:49.920810+0000","last_became_active":"2024-06-10T00:28:48.705558+0000","last_became_peered":"2024-06-10T00:28:48.705558+0000","last_unstale":"2024-06-10T00:32:49.920810+0000","last_undegraded":"2024-06-10T00:32:49.920810+0000","last_fullsized":"2024-06-10T00:32:49.920810+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:41:44.914378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"99'14","reported_seq":281,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:09.156591+0000","last_change":"2024-06-10T00:29:23.862141+0000","last_active":"2024-06-10T00:33:09.156591+0000","last_peered":"2024-06-10T00:33:09.156591+0000","last_clean":"2024-06-10T00:33:09.156591+0000","last_became_active":"2024-06-10T00:29:23.857051+0000","last_became_peered":"2024-06-10T00:29:23.857051+0000","last_unstale":"2024-06-10T00:33:09.156591+0000","last_undegraded":"2024-06-10T00:33:09.156591+0000","last_fullsized":"2024-06-10T00:33:09.156591+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:41:41.865144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":207,"num_read_kb":207,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032309+0000","last_change":"2024-06-10T00:28:37.653938+0000","last_active":"2024-06-10T00:32:50.032309+0000","last_peered":"2024-06-10T00:32:50.032309+0000","last_clean":"2024-06-10T00:32:50.032309+0000","last_became_active":"2024-06-10T00:28:37.653629+0000","last_became_peered":"2024-06-10T00:28:37.653629+0000","last_unstale":"2024-06-10T00:32:50.032309+0000","last_undegraded":"2024-06-10T00:32:50.032309+0000","last_fullsized":"2024-06-10T00:32:50.032309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:21:39.285961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921432+0000","last_change":"2024-06-10T00:28:48.704413+0000","last_active":"2024-06-10T00:32:49.921432+0000","last_peered":"2024-06-10T00:32:49.921432+0000","last_clean":"2024-06-10T00:32:49.921432+0000","last_became_active":"2024-06-10T00:28:48.703972+0000","last_became_peered":"2024-06-10T00:28:48.703972+0000","last_unstale":"2024-06-10T00:32:49.921432+0000","last_undegraded":"2024-06-10T00:32:49.921432+0000","last_fullsized":"2024-06-10T00:32:49.921432+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:51.927435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032677+0000","last_change":"2024-06-10T00:28:34.445413+0000","last_active":"2024-06-10T00:32:50.032677+0000","last_peered":"2024-06-10T00:32:50.032677+0000","last_clean":"2024-06-10T00:32:50.032677+0000","last_became_active":"2024-06-10T00:28:34.445077+0000","last_became_peered":"2024-06-10T00:28:34.445077+0000","last_unstale":"2024-06-10T00:32:50.032677+0000","last_undegraded":"2024-06-10T00:32:50.032677+0000","last_fullsized":"2024-06-10T00:32:50.032677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:19:19.535976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"99'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193506+0000","last_change":"2024-06-10T00:28:37.657123+0000","last_active":"2024-06-10T00:32:50.193506+0000","last_peered":"2024-06-10T00:32:50.193506+0000","last_clean":"2024-06-10T00:32:50.193506+0000","last_became_active":"2024-06-10T00:28:37.656791+0000","last_became_peered":"2024-06-10T00:28:37.656791+0000","last_unstale":"2024-06-10T00:32:50.193506+0000","last_undegraded":"2024-06-10T00:32:50.193506+0000","last_fullsized":"2024-06-10T00:32:50.193506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:11.102478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921858+0000","last_change":"2024-06-10T00:28:48.705989+0000","last_active":"2024-06-10T00:32:49.921858+0000","last_peered":"2024-06-10T00:32:49.921858+0000","last_clean":"2024-06-10T00:32:49.921858+0000","last_became_active":"2024-06-10T00:28:48.705555+0000","last_became_peered":"2024-06-10T00:28:48.705555+0000","last_unstale":"2024-06-10T00:32:49.921858+0000","last_undegraded":"2024-06-10T00:32:49.921858+0000","last_fullsized":"2024-06-10T00:32:49.921858+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:36.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"100'13","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032133+0000","last_change":"2024-06-10T00:28:34.400117+0000","last_active":"2024-06-10T00:32:50.032133+0000","last_peered":"2024-06-10T00:32:50.032133+0000","last_clean":"2024-06-10T00:32:50.032133+0000","last_became_active":"2024-06-10T00:28:34.399743+0000","last_became_peered":"2024-06-10T00:28:34.399743+0000","last_unstale":"2024-06-10T00:32:50.032133+0000","last_undegraded":"2024-06-10T00:32:50.032133+0000","last_fullsized":"2024-06-10T00:32:50.032133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:20:57.580146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188627+0000","last_change":"2024-06-10T00:28:37.664005+0000","last_active":"2024-06-10T00:32:50.188627+0000","last_peered":"2024-06-10T00:32:50.188627+0000","last_clean":"2024-06-10T00:32:50.188627+0000","last_became_active":"2024-06-10T00:28:37.661297+0000","last_became_peered":"2024-06-10T00:28:37.661297+0000","last_unstale":"2024-06-10T00:32:50.188627+0000","last_undegraded":"2024-06-10T00:32:50.188627+0000","last_fullsized":"2024-06-10T00:32:50.188627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:57.795080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921561+0000","last_change":"2024-06-10T00:29:23.859091+0000","last_active":"2024-06-10T00:32:49.921561+0000","last_peered":"2024-06-10T00:32:49.921561+0000","last_clean":"2024-06-10T00:32:49.921561+0000","last_became_active":"2024-06-10T00:29:23.858485+0000","last_became_peered":"2024-06-10T00:29:23.858485+0000","last_unstale":"2024-06-10T00:32:49.921561+0000","last_undegraded":"2024-06-10T00:32:49.921561+0000","last_fullsized":"2024-06-10T00:32:49.921561+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:07:10.925323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"97'5","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031945+0000","last_change":"2024-06-10T00:28:34.400588+0000","last_active":"2024-06-10T00:32:50.031945+0000","last_peered":"2024-06-10T00:32:50.031945+0000","last_clean":"2024-06-10T00:32:50.031945+0000","last_became_active":"2024-06-10T00:28:34.400027+0000","last_became_peered":"2024-06-10T00:28:34.400027+0000","last_unstale":"2024-06-10T00:32:50.031945+0000","last_undegraded":"2024-06-10T00:32:50.031945+0000","last_fullsized":"2024-06-10T00:32:50.031945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:25:59.890076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039025+0000","last_change":"2024-06-10T00:28:37.659401+0000","last_active":"2024-06-10T00:32:50.039025+0000","last_peered":"2024-06-10T00:32:50.039025+0000","last_clean":"2024-06-10T00:32:50.039025+0000","last_became_active":"2024-06-10T00:28:37.659056+0000","last_became_peered":"2024-06-10T00:28:37.659056+0000","last_unstale":"2024-06-10T00:32:50.039025+0000","last_undegraded":"2024-06-10T00:32:50.039025+0000","last_fullsized":"2024-06-10T00:32:50.039025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:35:01.415630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921379+0000","last_change":"2024-06-10T00:28:48.702961+0000","last_active":"2024-06-10T00:32:49.921379+0000","last_peered":"2024-06-10T00:32:49.921379+0000","last_clean":"2024-06-10T00:32:49.921379+0000","last_became_active":"2024-06-10T00:28:48.702502+0000","last_became_peered":"2024-06-10T00:28:48.702502+0000","last_unstale":"2024-06-10T00:32:49.921379+0000","last_undegraded":"2024-06-10T00:32:49.921379+0000","last_fullsized":"2024-06-10T00:32:49.921379+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:54.881595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'9","reported_seq":265,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:09.157332+0000","last_change":"2024-06-10T00:28:34.396310+0000","last_active":"2024-06-10T00:33:09.157332+0000","last_peered":"2024-06-10T00:33:09.157332+0000","last_clean":"2024-06-10T00:33:09.157332+0000","last_became_active":"2024-06-10T00:28:34.395401+0000","last_became_peered":"2024-06-10T00:28:34.395401+0000","last_unstale":"2024-06-10T00:33:09.157332+0000","last_undegraded":"2024-06-10T00:33:09.157332+0000","last_fullsized":"2024-06-10T00:33:09.157332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:33:54.300308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":210,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"99'7","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031647+0000","last_change":"2024-06-10T00:28:37.651427+0000","last_active":"2024-06-10T00:32:50.031647+0000","last_peered":"2024-06-10T00:32:50.031647+0000","last_clean":"2024-06-10T00:32:50.031647+0000","last_became_active":"2024-06-10T00:28:37.650510+0000","last_became_peered":"2024-06-10T00:28:37.650510+0000","last_unstale":"2024-06-10T00:32:50.031647+0000","last_undegraded":"2024-06-10T00:32:50.031647+0000","last_fullsized":"2024-06-10T00:32:50.031647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:42.953919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920970+0000","last_change":"2024-06-10T00:28:48.705918+0000","last_active":"2024-06-10T00:32:49.920970+0000","last_peered":"2024-06-10T00:32:49.920970+0000","last_clean":"2024-06-10T00:32:49.920970+0000","last_became_active":"2024-06-10T00:28:48.705462+0000","last_became_peered":"2024-06-10T00:28:48.705462+0000","last_unstale":"2024-06-10T00:32:49.920970+0000","last_undegraded":"2024-06-10T00:32:49.920970+0000","last_fullsized":"2024-06-10T00:32:49.920970+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:59:23.176511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"97'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042358+0000","last_change":"2024-06-10T00:28:34.393199+0000","last_active":"2024-06-10T00:32:50.042358+0000","last_peered":"2024-06-10T00:32:50.042358+0000","last_clean":"2024-06-10T00:32:50.042358+0000","last_became_active":"2024-06-10T00:28:34.392777+0000","last_became_peered":"2024-06-10T00:28:34.392777+0000","last_unstale":"2024-06-10T00:32:50.042358+0000","last_undegraded":"2024-06-10T00:32:50.042358+0000","last_fullsized":"2024-06-10T00:32:50.042358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:31:29.614242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"99'8","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038961+0000","last_change":"2024-06-10T00:30:24.229721+0000","last_active":"2024-06-10T00:32:50.038961+0000","last_peered":"2024-06-10T00:32:50.038961+0000","last_clean":"2024-06-10T00:32:50.038961+0000","last_became_active":"2024-06-10T00:30:24.229407+0000","last_became_peered":"2024-06-10T00:30:24.229407+0000","last_unstale":"2024-06-10T00:32:50.038961+0000","last_undegraded":"2024-06-10T00:32:50.038961+0000","last_fullsized":"2024-06-10T00:32:50.038961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:39.523197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199129+0000","last_change":"2024-06-10T00:29:23.780971+0000","last_active":"2024-06-10T00:32:50.199129+0000","last_peered":"2024-06-10T00:32:50.199129+0000","last_clean":"2024-06-10T00:32:50.199129+0000","last_became_active":"2024-06-10T00:29:23.780034+0000","last_became_peered":"2024-06-10T00:29:23.780034+0000","last_unstale":"2024-06-10T00:32:50.199129+0000","last_undegraded":"2024-06-10T00:32:50.199129+0000","last_fullsized":"2024-06-10T00:32:50.199129+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:47:06.444181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"97'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132954+0000","last_change":"2024-06-10T00:28:34.392486+0000","last_active":"2024-06-10T00:32:50.132954+0000","last_peered":"2024-06-10T00:32:50.132954+0000","last_clean":"2024-06-10T00:32:50.132954+0000","last_became_active":"2024-06-10T00:28:34.392141+0000","last_became_peered":"2024-06-10T00:28:34.392141+0000","last_unstale":"2024-06-10T00:32:50.132954+0000","last_undegraded":"2024-06-10T00:32:50.132954+0000","last_fullsized":"2024-06-10T00:32:50.132954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:10:01.534195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"99'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031869+0000","last_change":"2024-06-10T00:28:37.651033+0000","last_active":"2024-06-10T00:32:50.031869+0000","last_peered":"2024-06-10T00:32:50.031869+0000","last_clean":"2024-06-10T00:32:50.031869+0000","last_became_active":"2024-06-10T00:28:37.650505+0000","last_became_peered":"2024-06-10T00:28:37.650505+0000","last_unstale":"2024-06-10T00:32:50.031869+0000","last_undegraded":"2024-06-10T00:32:50.031869+0000","last_fullsized":"2024-06-10T00:32:50.031869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:11:54.965865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193367+0000","last_change":"2024-06-10T00:28:48.710748+0000","last_active":"2024-06-10T00:32:50.193367+0000","last_peered":"2024-06-10T00:32:50.193367+0000","last_clean":"2024-06-10T00:32:50.193367+0000","last_became_active":"2024-06-10T00:28:48.710353+0000","last_became_peered":"2024-06-10T00:28:48.710353+0000","last_unstale":"2024-06-10T00:32:50.193367+0000","last_undegraded":"2024-06-10T00:32:50.193367+0000","last_fullsized":"2024-06-10T00:32:50.193367+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:30:47.546650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"97'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038780+0000","last_change":"2024-06-10T00:28:34.398911+0000","last_active":"2024-06-10T00:32:50.038780+0000","last_peered":"2024-06-10T00:32:50.038780+0000","last_clean":"2024-06-10T00:32:50.038780+0000","last_became_active":"2024-06-10T00:28:34.398373+0000","last_became_peered":"2024-06-10T00:28:34.398373+0000","last_unstale":"2024-06-10T00:32:50.038780+0000","last_undegraded":"2024-06-10T00:32:50.038780+0000","last_fullsized":"2024-06-10T00:32:50.038780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:05:34.352825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'11","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921526+0000","last_change":"2024-06-10T00:28:37.647492+0000","last_active":"2024-06-10T00:32:49.921526+0000","last_peered":"2024-06-10T00:32:49.921526+0000","last_clean":"2024-06-10T00:32:49.921526+0000","last_became_active":"2024-06-10T00:28:37.647103+0000","last_became_peered":"2024-06-10T00:28:37.647103+0000","last_unstale":"2024-06-10T00:32:49.921526+0000","last_undegraded":"2024-06-10T00:32:49.921526+0000","last_fullsized":"2024-06-10T00:32:49.921526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:50:10.429256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031883+0000","last_change":"2024-06-10T00:28:48.703095+0000","last_active":"2024-06-10T00:32:50.031883+0000","last_peered":"2024-06-10T00:32:50.031883+0000","last_clean":"2024-06-10T00:32:50.031883+0000","last_became_active":"2024-06-10T00:28:48.702687+0000","last_became_peered":"2024-06-10T00:28:48.702687+0000","last_unstale":"2024-06-10T00:32:50.031883+0000","last_undegraded":"2024-06-10T00:32:50.031883+0000","last_fullsized":"2024-06-10T00:32:50.031883+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:10:20.074173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":463,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:10.158185+0000","last_change":"2024-06-10T00:28:34.394807+0000","last_active":"2024-06-10T00:33:10.158185+0000","last_peered":"2024-06-10T00:33:10.158185+0000","last_clean":"2024-06-10T00:33:10.158185+0000","last_became_active":"2024-06-10T00:28:34.394022+0000","last_became_peered":"2024-06-10T00:28:34.394022+0000","last_unstale":"2024-06-10T00:33:10.158185+0000","last_undegraded":"2024-06-10T00:33:10.158185+0000","last_fullsized":"2024-06-10T00:33:10.158185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:13:21.626079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":410,"num_read_kb":410,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"99'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.189056+0000","last_change":"2024-06-10T00:28:37.662231+0000","last_active":"2024-06-10T00:32:50.189056+0000","last_peered":"2024-06-10T00:32:50.189056+0000","last_clean":"2024-06-10T00:32:50.189056+0000","last_became_active":"2024-06-10T00:28:37.661861+0000","last_became_peered":"2024-06-10T00:28:37.661861+0000","last_unstale":"2024-06-10T00:32:50.189056+0000","last_undegraded":"2024-06-10T00:32:50.189056+0000","last_fullsized":"2024-06-10T00:32:50.189056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:09.705349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921535+0000","last_change":"2024-06-10T00:28:48.706643+0000","last_active":"2024-06-10T00:32:49.921535+0000","last_peered":"2024-06-10T00:32:49.921535+0000","last_clean":"2024-06-10T00:32:49.921535+0000","last_became_active":"2024-06-10T00:28:48.706127+0000","last_became_peered":"2024-06-10T00:28:48.706127+0000","last_unstale":"2024-06-10T00:32:49.921535+0000","last_undegraded":"2024-06-10T00:32:49.921535+0000","last_fullsized":"2024-06-10T00:32:49.921535+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:55.031303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"97'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134861+0000","last_change":"2024-06-10T00:28:34.392808+0000","last_active":"2024-06-10T00:32:50.134861+0000","last_peered":"2024-06-10T00:32:50.134861+0000","last_clean":"2024-06-10T00:32:50.134861+0000","last_became_active":"2024-06-10T00:28:34.392450+0000","last_became_peered":"2024-06-10T00:28:34.392450+0000","last_unstale":"2024-06-10T00:32:50.134861+0000","last_undegraded":"2024-06-10T00:32:50.134861+0000","last_fullsized":"2024-06-10T00:32:50.134861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:06:00.493316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193616+0000","last_change":"2024-06-10T00:28:37.656099+0000","last_active":"2024-06-10T00:32:50.193616+0000","last_peered":"2024-06-10T00:32:50.193616+0000","last_clean":"2024-06-10T00:32:50.193616+0000","last_became_active":"2024-06-10T00:28:37.655640+0000","last_became_peered":"2024-06-10T00:28:37.655640+0000","last_unstale":"2024-06-10T00:32:50.193616+0000","last_undegraded":"2024-06-10T00:32:50.193616+0000","last_fullsized":"2024-06-10T00:32:50.193616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:59:26.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922109+0000","last_change":"2024-06-10T00:28:48.702628+0000","last_active":"2024-06-10T00:32:49.922109+0000","last_peered":"2024-06-10T00:32:49.922109+0000","last_clean":"2024-06-10T00:32:49.922109+0000","last_became_active":"2024-06-10T00:28:48.702109+0000","last_became_peered":"2024-06-10T00:28:48.702109+0000","last_unstale":"2024-06-10T00:32:49.922109+0000","last_undegraded":"2024-06-10T00:32:49.922109+0000","last_fullsized":"2024-06-10T00:32:49.922109+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:23:25.262348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"97'5","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039027+0000","last_change":"2024-06-10T00:28:34.446788+0000","last_active":"2024-06-10T00:32:50.039027+0000","last_peered":"2024-06-10T00:32:50.039027+0000","last_clean":"2024-06-10T00:32:50.039027+0000","last_became_active":"2024-06-10T00:28:34.446377+0000","last_became_peered":"2024-06-10T00:28:34.446377+0000","last_unstale":"2024-06-10T00:32:50.039027+0000","last_undegraded":"2024-06-10T00:32:50.039027+0000","last_fullsized":"2024-06-10T00:32:50.039027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:16.914244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"99'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188829+0000","last_change":"2024-06-10T00:28:37.666159+0000","last_active":"2024-06-10T00:32:50.188829+0000","last_peered":"2024-06-10T00:32:50.188829+0000","last_clean":"2024-06-10T00:32:50.188829+0000","last_became_active":"2024-06-10T00:28:37.665711+0000","last_became_peered":"2024-06-10T00:28:37.665711+0000","last_unstale":"2024-06-10T00:32:50.188829+0000","last_undegraded":"2024-06-10T00:32:50.188829+0000","last_fullsized":"2024-06-10T00:32:50.188829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:03.982375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921636+0000","last_change":"2024-06-10T00:28:48.705071+0000","last_active":"2024-06-10T00:32:49.921636+0000","last_peered":"2024-06-10T00:32:49.921636+0000","last_clean":"2024-06-10T00:32:49.921636+0000","last_became_active":"2024-06-10T00:28:48.702211+0000","last_became_peered":"2024-06-10T00:28:48.702211+0000","last_unstale":"2024-06-10T00:32:49.921636+0000","last_undegraded":"2024-06-10T00:32:49.921636+0000","last_fullsized":"2024-06-10T00:32:49.921636+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:38:13.361985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":510,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:10.157773+0000","last_change":"2024-06-10T00:28:34.389784+0000","last_active":"2024-06-10T00:33:10.157773+0000","last_peered":"2024-06-10T00:33:10.157773+0000","last_clean":"2024-06-10T00:33:10.157773+0000","last_became_active":"2024-06-10T00:28:34.389416+0000","last_became_peered":"2024-06-10T00:28:34.389416+0000","last_unstale":"2024-06-10T00:33:10.157773+0000","last_undegraded":"2024-06-10T00:33:10.157773+0000","last_fullsized":"2024-06-10T00:33:10.157773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:23:05.803503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69365,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":423,"num_read_kb":281,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"99'6","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031957+0000","last_change":"2024-06-10T00:28:37.656174+0000","last_active":"2024-06-10T00:32:50.031957+0000","last_peered":"2024-06-10T00:32:50.031957+0000","last_clean":"2024-06-10T00:32:50.031957+0000","last_became_active":"2024-06-10T00:28:37.655794+0000","last_became_peered":"2024-06-10T00:28:37.655794+0000","last_unstale":"2024-06-10T00:32:50.031957+0000","last_undegraded":"2024-06-10T00:32:50.031957+0000","last_fullsized":"2024-06-10T00:32:50.031957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:22:00.310309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039312+0000","last_change":"2024-06-10T00:28:48.709915+0000","last_active":"2024-06-10T00:32:50.039312+0000","last_peered":"2024-06-10T00:32:50.039312+0000","last_clean":"2024-06-10T00:32:50.039312+0000","last_became_active":"2024-06-10T00:28:48.709426+0000","last_became_peered":"2024-06-10T00:28:48.709426+0000","last_unstale":"2024-06-10T00:32:50.039312+0000","last_undegraded":"2024-06-10T00:32:50.039312+0000","last_fullsized":"2024-06-10T00:32:50.039312+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:06:55.150617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"97'9","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921708+0000","last_change":"2024-06-10T00:28:34.389136+0000","last_active":"2024-06-10T00:32:49.921708+0000","last_peered":"2024-06-10T00:32:49.921708+0000","last_clean":"2024-06-10T00:32:49.921708+0000","last_became_active":"2024-06-10T00:28:34.388701+0000","last_became_peered":"2024-06-10T00:28:34.388701+0000","last_unstale":"2024-06-10T00:32:49.921708+0000","last_undegraded":"2024-06-10T00:32:49.921708+0000","last_fullsized":"2024-06-10T00:32:49.921708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:13:24.907824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133047+0000","last_change":"2024-06-10T00:28:37.659048+0000","last_active":"2024-06-10T00:32:50.133047+0000","last_peered":"2024-06-10T00:32:50.133047+0000","last_clean":"2024-06-10T00:32:50.133047+0000","last_became_active":"2024-06-10T00:28:37.658710+0000","last_became_peered":"2024-06-10T00:28:37.658710+0000","last_unstale":"2024-06-10T00:32:50.133047+0000","last_undegraded":"2024-06-10T00:32:50.133047+0000","last_fullsized":"2024-06-10T00:32:50.133047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:44.314559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198919+0000","last_change":"2024-06-10T00:28:48.709683+0000","last_active":"2024-06-10T00:32:50.198919+0000","last_peered":"2024-06-10T00:32:50.198919+0000","last_clean":"2024-06-10T00:32:50.198919+0000","last_became_active":"2024-06-10T00:28:48.709241+0000","last_became_peered":"2024-06-10T00:28:48.709241+0000","last_unstale":"2024-06-10T00:32:50.198919+0000","last_undegraded":"2024-06-10T00:32:50.198919+0000","last_fullsized":"2024-06-10T00:32:50.198919+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:01:25.190043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"97'5","reported_seq":24,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198772+0000","last_change":"2024-06-10T00:29:23.782029+0000","last_active":"2024-06-10T00:32:50.198772+0000","last_peered":"2024-06-10T00:32:50.198772+0000","last_clean":"2024-06-10T00:32:50.198772+0000","last_became_active":"2024-06-10T00:29:23.781565+0000","last_became_peered":"2024-06-10T00:29:23.781565+0000","last_unstale":"2024-06-10T00:32:50.198772+0000","last_undegraded":"2024-06-10T00:32:50.198772+0000","last_fullsized":"2024-06-10T00:32:50.198772+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:53.258760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"99'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134547+0000","last_change":"2024-06-10T00:28:37.653385+0000","last_active":"2024-06-10T00:32:50.134547+0000","last_peered":"2024-06-10T00:32:50.134547+0000","last_clean":"2024-06-10T00:32:50.134547+0000","last_became_active":"2024-06-10T00:28:37.653020+0000","last_became_peered":"2024-06-10T00:28:37.653020+0000","last_unstale":"2024-06-10T00:32:50.134547+0000","last_undegraded":"2024-06-10T00:32:50.134547+0000","last_fullsized":"2024-06-10T00:32:50.134547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:57:54.469544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921940+0000","last_change":"2024-06-10T00:28:48.707540+0000","last_active":"2024-06-10T00:32:49.921940+0000","last_peered":"2024-06-10T00:32:49.921940+0000","last_clean":"2024-06-10T00:32:49.921940+0000","last_became_active":"2024-06-10T00:28:48.707143+0000","last_became_peered":"2024-06-10T00:28:48.707143+0000","last_unstale":"2024-06-10T00:32:49.921940+0000","last_undegraded":"2024-06-10T00:32:49.921940+0000","last_fullsized":"2024-06-10T00:32:49.921940+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:20:29.344808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'8","reported_seq":265,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:09.157185+0000","last_change":"2024-06-10T00:28:34.397496+0000","last_active":"2024-06-10T00:33:09.157185+0000","last_peered":"2024-06-10T00:33:09.157185+0000","last_clean":"2024-06-10T00:33:09.157185+0000","last_became_active":"2024-06-10T00:28:34.397124+0000","last_became_peered":"2024-06-10T00:28:34.397124+0000","last_unstale":"2024-06-10T00:33:09.157185+0000","last_undegraded":"2024-06-10T00:33:09.157185+0000","last_fullsized":"2024-06-10T00:33:09.157185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:41:48.609354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":211,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134324+0000","last_change":"2024-06-10T00:28:37.658338+0000","last_active":"2024-06-10T00:32:50.134324+0000","last_peered":"2024-06-10T00:32:50.134324+0000","last_clean":"2024-06-10T00:32:50.134324+0000","last_became_active":"2024-06-10T00:28:37.655910+0000","last_became_peered":"2024-06-10T00:28:37.655910+0000","last_unstale":"2024-06-10T00:32:50.134324+0000","last_undegraded":"2024-06-10T00:32:50.134324+0000","last_fullsized":"2024-06-10T00:32:50.134324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:38.501335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133960+0000","last_change":"2024-06-10T00:28:48.711151+0000","last_active":"2024-06-10T00:32:50.133960+0000","last_peered":"2024-06-10T00:32:50.133960+0000","last_clean":"2024-06-10T00:32:50.133960+0000","last_became_active":"2024-06-10T00:28:48.710768+0000","last_became_peered":"2024-06-10T00:28:48.710768+0000","last_unstale":"2024-06-10T00:32:50.133960+0000","last_undegraded":"2024-06-10T00:32:50.133960+0000","last_fullsized":"2024-06-10T00:32:50.133960+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:15:22.791839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"97'17","reported_seq":65,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031568+0000","last_change":"2024-06-10T00:28:34.445812+0000","last_active":"2024-06-10T00:32:50.031568+0000","last_peered":"2024-06-10T00:32:50.031568+0000","last_clean":"2024-06-10T00:32:50.031568+0000","last_became_active":"2024-06-10T00:28:34.445472+0000","last_became_peered":"2024-06-10T00:28:34.445472+0000","last_unstale":"2024-06-10T00:32:50.031568+0000","last_undegraded":"2024-06-10T00:32:50.031568+0000","last_fullsized":"2024-06-10T00:32:50.031568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:27:00.912050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"99'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038769+0000","last_change":"2024-06-10T00:28:37.655345+0000","last_active":"2024-06-10T00:32:50.038769+0000","last_peered":"2024-06-10T00:32:50.038769+0000","last_clean":"2024-06-10T00:32:50.038769+0000","last_became_active":"2024-06-10T00:28:37.654954+0000","last_became_peered":"2024-06-10T00:28:37.654954+0000","last_unstale":"2024-06-10T00:32:50.038769+0000","last_undegraded":"2024-06-10T00:32:50.038769+0000","last_fullsized":"2024-06-10T00:32:50.038769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:39:27.403006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038815+0000","last_change":"2024-06-10T00:28:48.705247+0000","last_active":"2024-06-10T00:32:50.038815+0000","last_peered":"2024-06-10T00:32:50.038815+0000","last_clean":"2024-06-10T00:32:50.038815+0000","last_became_active":"2024-06-10T00:28:48.704822+0000","last_became_peered":"2024-06-10T00:28:48.704822+0000","last_unstale":"2024-06-10T00:32:50.038815+0000","last_undegraded":"2024-06-10T00:32:50.038815+0000","last_fullsized":"2024-06-10T00:32:50.038815+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:24:59.991491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"97'2","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032130+0000","last_change":"2024-06-10T00:28:34.396792+0000","last_active":"2024-06-10T00:32:50.032130+0000","last_peered":"2024-06-10T00:32:50.032130+0000","last_clean":"2024-06-10T00:32:50.032130+0000","last_became_active":"2024-06-10T00:28:34.396442+0000","last_became_peered":"2024-06-10T00:28:34.396442+0000","last_unstale":"2024-06-10T00:32:50.032130+0000","last_undegraded":"2024-06-10T00:32:50.032130+0000","last_fullsized":"2024-06-10T00:32:50.032130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:15:46.436198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"99'5","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199215+0000","last_change":"2024-06-10T00:28:37.657698+0000","last_active":"2024-06-10T00:32:50.199215+0000","last_peered":"2024-06-10T00:32:50.199215+0000","last_clean":"2024-06-10T00:32:50.199215+0000","last_became_active":"2024-06-10T00:28:37.657364+0000","last_became_peered":"2024-06-10T00:28:37.657364+0000","last_unstale":"2024-06-10T00:32:50.199215+0000","last_undegraded":"2024-06-10T00:32:50.199215+0000","last_fullsized":"2024-06-10T00:32:50.199215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:58:23.097015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031651+0000","last_change":"2024-06-10T00:28:48.727567+0000","last_active":"2024-06-10T00:32:50.031651+0000","last_peered":"2024-06-10T00:32:50.031651+0000","last_clean":"2024-06-10T00:32:50.031651+0000","last_became_active":"2024-06-10T00:28:48.727108+0000","last_became_peered":"2024-06-10T00:28:48.727108+0000","last_unstale":"2024-06-10T00:32:50.031651+0000","last_undegraded":"2024-06-10T00:32:50.031651+0000","last_fullsized":"2024-06-10T00:32:50.031651+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:35:00.824177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"97'1","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921917+0000","last_change":"2024-06-10T00:28:34.446832+0000","last_active":"2024-06-10T00:32:49.921917+0000","last_peered":"2024-06-10T00:32:49.921917+0000","last_clean":"2024-06-10T00:32:49.921917+0000","last_became_active":"2024-06-10T00:28:34.446435+0000","last_became_peered":"2024-06-10T00:28:34.446435+0000","last_unstale":"2024-06-10T00:32:49.921917+0000","last_undegraded":"2024-06-10T00:32:49.921917+0000","last_fullsized":"2024-06-10T00:32:49.921917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:13:58.296674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"99'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042645+0000","last_change":"2024-06-10T00:30:24.229514+0000","last_active":"2024-06-10T00:32:50.042645+0000","last_peered":"2024-06-10T00:32:50.042645+0000","last_clean":"2024-06-10T00:32:50.042645+0000","last_became_active":"2024-06-10T00:30:24.229163+0000","last_became_peered":"2024-06-10T00:30:24.229163+0000","last_unstale":"2024-06-10T00:32:50.042645+0000","last_undegraded":"2024-06-10T00:32:50.042645+0000","last_fullsized":"2024-06-10T00:32:50.042645+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:52.914144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133795+0000","last_change":"2024-06-10T00:28:48.704683+0000","last_active":"2024-06-10T00:32:50.133795+0000","last_peered":"2024-06-10T00:32:50.133795+0000","last_clean":"2024-06-10T00:32:50.133795+0000","last_became_active":"2024-06-10T00:28:48.704297+0000","last_became_peered":"2024-06-10T00:28:48.704297+0000","last_unstale":"2024-06-10T00:32:50.133795+0000","last_undegraded":"2024-06-10T00:32:50.133795+0000","last_fullsized":"2024-06-10T00:32:50.133795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:57:44.980544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'8","reported_seq":263,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:09.157112+0000","last_change":"2024-06-10T00:28:34.393328+0000","last_active":"2024-06-10T00:33:09.157112+0000","last_peered":"2024-06-10T00:33:09.157112+0000","last_clean":"2024-06-10T00:33:09.157112+0000","last_became_active":"2024-06-10T00:28:34.391968+0000","last_became_peered":"2024-06-10T00:28:34.391968+0000","last_unstale":"2024-06-10T00:33:09.157112+0000","last_undegraded":"2024-06-10T00:33:09.157112+0000","last_fullsized":"2024-06-10T00:33:09.157112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:14:18.898180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1676,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":208,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.042474+0000","last_change":"2024-06-10T00:28:37.652555+0000","last_active":"2024-06-10T00:32:50.042474+0000","last_peered":"2024-06-10T00:32:50.042474+0000","last_clean":"2024-06-10T00:32:50.042474+0000","last_became_active":"2024-06-10T00:28:37.651476+0000","last_became_peered":"2024-06-10T00:28:37.651476+0000","last_unstale":"2024-06-10T00:32:50.042474+0000","last_undegraded":"2024-06-10T00:32:50.042474+0000","last_fullsized":"2024-06-10T00:32:50.042474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:04:57.823224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134063+0000","last_change":"2024-06-10T00:28:48.727568+0000","last_active":"2024-06-10T00:32:50.134063+0000","last_peered":"2024-06-10T00:32:50.134063+0000","last_clean":"2024-06-10T00:32:50.134063+0000","last_became_active":"2024-06-10T00:28:48.727018+0000","last_became_peered":"2024-06-10T00:28:48.727018+0000","last_unstale":"2024-06-10T00:32:50.134063+0000","last_undegraded":"2024-06-10T00:32:50.134063+0000","last_fullsized":"2024-06-10T00:32:50.134063+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:02.025212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921381+0000","last_change":"2024-06-10T00:30:24.651947+0000","last_active":"2024-06-10T00:32:49.921381+0000","last_peered":"2024-06-10T00:32:49.921381+0000","last_clean":"2024-06-10T00:32:49.921381+0000","last_became_active":"2024-06-10T00:30:24.651640+0000","last_became_peered":"2024-06-10T00:30:24.651640+0000","last_unstale":"2024-06-10T00:32:49.921381+0000","last_undegraded":"2024-06-10T00:32:49.921381+0000","last_fullsized":"2024-06-10T00:32:49.921381+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:47.070568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"100'7","reported_seq":260,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:09.156629+0000","last_change":"2024-06-10T00:28:34.445501+0000","last_active":"2024-06-10T00:33:09.156629+0000","last_peered":"2024-06-10T00:33:09.156629+0000","last_clean":"2024-06-10T00:33:09.156629+0000","last_became_active":"2024-06-10T00:28:34.445128+0000","last_became_peered":"2024-06-10T00:28:34.445128+0000","last_unstale":"2024-06-10T00:33:09.156629+0000","last_undegraded":"2024-06-10T00:33:09.156629+0000","last_fullsized":"2024-06-10T00:33:09.156629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:59.231585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"97'1","reported_seq":33,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134764+0000","last_change":"2024-06-10T00:28:48.709852+0000","last_active":"2024-06-10T00:32:50.134764+0000","last_peered":"2024-06-10T00:32:50.134764+0000","last_clean":"2024-06-10T00:32:50.134764+0000","last_became_active":"2024-06-10T00:28:48.709423+0000","last_became_peered":"2024-06-10T00:28:48.709423+0000","last_unstale":"2024-06-10T00:32:50.134764+0000","last_undegraded":"2024-06-10T00:32:50.134764+0000","last_fullsized":"2024-06-10T00:32:50.134764+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:44:42.123563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031914+0000","last_change":"2024-06-10T00:28:37.657423+0000","last_active":"2024-06-10T00:32:50.031914+0000","last_peered":"2024-06-10T00:32:50.031914+0000","last_clean":"2024-06-10T00:32:50.031914+0000","last_became_active":"2024-06-10T00:28:37.657140+0000","last_became_peered":"2024-06-10T00:28:37.657140+0000","last_unstale":"2024-06-10T00:32:50.031914+0000","last_undegraded":"2024-06-10T00:32:50.031914+0000","last_fullsized":"2024-06-10T00:32:50.031914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:20:08.943066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"97'6","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922033+0000","last_change":"2024-06-10T00:28:34.392585+0000","last_active":"2024-06-10T00:32:49.922033+0000","last_peered":"2024-06-10T00:32:49.922033+0000","last_clean":"2024-06-10T00:32:49.922033+0000","last_became_active":"2024-06-10T00:28:34.392229+0000","last_became_peered":"2024-06-10T00:28:34.392229+0000","last_unstale":"2024-06-10T00:32:49.922033+0000","last_undegraded":"2024-06-10T00:32:49.922033+0000","last_fullsized":"2024-06-10T00:32:49.922033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:15:11.275761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"100'148","reported_seq":424,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:52.045212+0000","last_change":"2024-06-10T00:27:12.072577+0000","last_active":"2024-06-10T00:32:52.045212+0000","last_peered":"2024-06-10T00:32:52.045212+0000","last_clean":"2024-06-10T00:32:52.045212+0000","last_became_active":"2024-06-10T00:27:12.072193+0000","last_became_peered":"2024-06-10T00:27:12.072193+0000","last_unstale":"2024-06-10T00:32:52.045212+0000","last_undegraded":"2024-06-10T00:32:52.045212+0000","last_fullsized":"2024-06-10T00:32:52.045212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:23:46.826438+0000","last_clean_scrub_stamp":"2024-06-10T00:23:46.826438+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:59:57.782124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.039141+0000","last_change":"2024-06-10T00:28:48.727913+0000","last_active":"2024-06-10T00:32:50.039141+0000","last_peered":"2024-06-10T00:32:50.039141+0000","last_clean":"2024-06-10T00:32:50.039141+0000","last_became_active":"2024-06-10T00:28:48.727402+0000","last_became_peered":"2024-06-10T00:28:48.727402+0000","last_unstale":"2024-06-10T00:32:50.039141+0000","last_undegraded":"2024-06-10T00:32:50.039141+0000","last_fullsized":"2024-06-10T00:32:50.039141+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:10:25.051296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.132861+0000","last_change":"2024-06-10T00:28:37.659901+0000","last_active":"2024-06-10T00:32:50.132861+0000","last_peered":"2024-06-10T00:32:50.132861+0000","last_clean":"2024-06-10T00:32:50.132861+0000","last_became_active":"2024-06-10T00:28:37.659553+0000","last_became_peered":"2024-06-10T00:28:37.659553+0000","last_unstale":"2024-06-10T00:32:50.132861+0000","last_undegraded":"2024-06-10T00:32:50.132861+0000","last_fullsized":"2024-06-10T00:32:50.132861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:54:25.694823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"97'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134306+0000","last_change":"2024-06-10T00:28:34.398155+0000","last_active":"2024-06-10T00:32:50.134306+0000","last_peered":"2024-06-10T00:32:50.134306+0000","last_clean":"2024-06-10T00:32:50.134306+0000","last_became_active":"2024-06-10T00:28:34.397793+0000","last_became_peered":"2024-06-10T00:28:34.397793+0000","last_unstale":"2024-06-10T00:32:50.134306+0000","last_undegraded":"2024-06-10T00:32:50.134306+0000","last_fullsized":"2024-06-10T00:32:50.134306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:32:21.188139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198954+0000","last_change":"2024-06-10T00:28:48.709736+0000","last_active":"2024-06-10T00:32:50.198954+0000","last_peered":"2024-06-10T00:32:50.198954+0000","last_clean":"2024-06-10T00:32:50.198954+0000","last_became_active":"2024-06-10T00:28:48.709293+0000","last_became_peered":"2024-06-10T00:28:48.709293+0000","last_unstale":"2024-06-10T00:32:50.198954+0000","last_undegraded":"2024-06-10T00:32:50.198954+0000","last_fullsized":"2024-06-10T00:32:50.198954+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:30:04.840656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"97'4","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199015+0000","last_change":"2024-06-10T00:29:23.780030+0000","last_active":"2024-06-10T00:32:50.199015+0000","last_peered":"2024-06-10T00:32:50.199015+0000","last_clean":"2024-06-10T00:32:50.199015+0000","last_became_active":"2024-06-10T00:29:23.779589+0000","last_became_peered":"2024-06-10T00:29:23.779589+0000","last_unstale":"2024-06-10T00:32:50.199015+0000","last_undegraded":"2024-06-10T00:32:50.199015+0000","last_fullsized":"2024-06-10T00:32:50.199015+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:51:53.048086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"99'9","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133881+0000","last_change":"2024-06-10T00:28:37.658330+0000","last_active":"2024-06-10T00:32:50.133881+0000","last_peered":"2024-06-10T00:32:50.133881+0000","last_clean":"2024-06-10T00:32:50.133881+0000","last_became_active":"2024-06-10T00:28:37.657343+0000","last_became_peered":"2024-06-10T00:28:37.657343+0000","last_unstale":"2024-06-10T00:32:50.133881+0000","last_undegraded":"2024-06-10T00:32:50.133881+0000","last_fullsized":"2024-06-10T00:32:50.133881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:31:52.167402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031813+0000","last_change":"2024-06-10T00:28:48.727506+0000","last_active":"2024-06-10T00:32:50.031813+0000","last_peered":"2024-06-10T00:32:50.031813+0000","last_clean":"2024-06-10T00:32:50.031813+0000","last_became_active":"2024-06-10T00:28:48.727068+0000","last_became_peered":"2024-06-10T00:28:48.727068+0000","last_unstale":"2024-06-10T00:32:50.031813+0000","last_undegraded":"2024-06-10T00:32:50.031813+0000","last_fullsized":"2024-06-10T00:32:50.031813+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:42.147908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'104","reported_seq":244,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:03.056325+0000","last_change":"2024-06-10T00:28:34.399872+0000","last_active":"2024-06-10T00:33:03.056325+0000","last_peered":"2024-06-10T00:33:03.056325+0000","last_clean":"2024-06-10T00:33:03.056325+0000","last_became_active":"2024-06-10T00:28:34.399493+0000","last_became_peered":"2024-06-10T00:28:34.399493+0000","last_unstale":"2024-06-10T00:33:03.056325+0000","last_undegraded":"2024-06-10T00:33:03.056325+0000","last_fullsized":"2024-06-10T00:33:03.056325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:24:10.018110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":190,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032464+0000","last_change":"2024-06-10T00:28:37.656597+0000","last_active":"2024-06-10T00:32:50.032464+0000","last_peered":"2024-06-10T00:32:50.032464+0000","last_clean":"2024-06-10T00:32:50.032464+0000","last_became_active":"2024-06-10T00:28:37.654356+0000","last_became_peered":"2024-06-10T00:28:37.654356+0000","last_unstale":"2024-06-10T00:32:50.032464+0000","last_undegraded":"2024-06-10T00:32:50.032464+0000","last_fullsized":"2024-06-10T00:32:50.032464+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:07:12.950683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920789+0000","last_change":"2024-06-10T00:28:48.702686+0000","last_active":"2024-06-10T00:32:49.920789+0000","last_peered":"2024-06-10T00:32:49.920789+0000","last_clean":"2024-06-10T00:32:49.920789+0000","last_became_active":"2024-06-10T00:28:48.702273+0000","last_became_peered":"2024-06-10T00:28:48.702273+0000","last_unstale":"2024-06-10T00:32:49.920789+0000","last_undegraded":"2024-06-10T00:32:49.920789+0000","last_fullsized":"2024-06-10T00:32:49.920789+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:43:00.078232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"97'7","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921789+0000","last_change":"2024-06-10T00:28:34.447549+0000","last_active":"2024-06-10T00:32:49.921789+0000","last_peered":"2024-06-10T00:32:49.921789+0000","last_clean":"2024-06-10T00:32:49.921789+0000","last_became_active":"2024-06-10T00:28:34.447197+0000","last_became_peered":"2024-06-10T00:28:34.447197+0000","last_unstale":"2024-06-10T00:32:49.921789+0000","last_undegraded":"2024-06-10T00:32:49.921789+0000","last_fullsized":"2024-06-10T00:32:49.921789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:47.525153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.193483+0000","last_change":"2024-06-10T00:28:37.655585+0000","last_active":"2024-06-10T00:32:50.193483+0000","last_peered":"2024-06-10T00:32:50.193483+0000","last_clean":"2024-06-10T00:32:50.193483+0000","last_became_active":"2024-06-10T00:28:37.655225+0000","last_became_peered":"2024-06-10T00:28:37.655225+0000","last_unstale":"2024-06-10T00:32:50.193483+0000","last_undegraded":"2024-06-10T00:32:50.193483+0000","last_fullsized":"2024-06-10T00:32:50.193483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:57:25.184924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921606+0000","last_change":"2024-06-10T00:29:23.857447+0000","last_active":"2024-06-10T00:32:49.921606+0000","last_peered":"2024-06-10T00:32:49.921606+0000","last_clean":"2024-06-10T00:32:49.921606+0000","last_became_active":"2024-06-10T00:29:23.856943+0000","last_became_peered":"2024-06-10T00:29:23.856943+0000","last_unstale":"2024-06-10T00:32:49.921606+0000","last_undegraded":"2024-06-10T00:32:49.921606+0000","last_fullsized":"2024-06-10T00:32:49.921606+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:58:44.548818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"97'2","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188371+0000","last_change":"2024-06-10T00:28:34.445637+0000","last_active":"2024-06-10T00:32:50.188371+0000","last_peered":"2024-06-10T00:32:50.188371+0000","last_clean":"2024-06-10T00:32:50.188371+0000","last_became_active":"2024-06-10T00:28:34.445324+0000","last_became_peered":"2024-06-10T00:28:34.445324+0000","last_unstale":"2024-06-10T00:32:50.188371+0000","last_undegraded":"2024-06-10T00:32:50.188371+0000","last_fullsized":"2024-06-10T00:32:50.188371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:05.713557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"99'2","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031670+0000","last_change":"2024-06-10T00:28:37.659037+0000","last_active":"2024-06-10T00:32:50.031670+0000","last_peered":"2024-06-10T00:32:50.031670+0000","last_clean":"2024-06-10T00:32:50.031670+0000","last_became_active":"2024-06-10T00:28:37.658705+0000","last_became_peered":"2024-06-10T00:28:37.658705+0000","last_unstale":"2024-06-10T00:32:50.031670+0000","last_undegraded":"2024-06-10T00:32:50.031670+0000","last_fullsized":"2024-06-10T00:32:50.031670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:20:52.797237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.922067+0000","last_change":"2024-06-10T00:28:48.704130+0000","last_active":"2024-06-10T00:32:49.922067+0000","last_peered":"2024-06-10T00:32:49.922067+0000","last_clean":"2024-06-10T00:32:49.922067+0000","last_became_active":"2024-06-10T00:28:48.703738+0000","last_became_peered":"2024-06-10T00:28:48.703738+0000","last_unstale":"2024-06-10T00:32:49.922067+0000","last_undegraded":"2024-06-10T00:32:49.922067+0000","last_fullsized":"2024-06-10T00:32:49.922067+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:01:40.436710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"97'7","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032084+0000","last_change":"2024-06-10T00:28:34.386667+0000","last_active":"2024-06-10T00:32:50.032084+0000","last_peered":"2024-06-10T00:32:50.032084+0000","last_clean":"2024-06-10T00:32:50.032084+0000","last_became_active":"2024-06-10T00:28:34.386363+0000","last_became_peered":"2024-06-10T00:28:34.386363+0000","last_unstale":"2024-06-10T00:32:50.032084+0000","last_undegraded":"2024-06-10T00:32:50.032084+0000","last_fullsized":"2024-06-10T00:32:50.032084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:27:11.850162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198807+0000","last_change":"2024-06-10T00:28:37.664754+0000","last_active":"2024-06-10T00:32:50.198807+0000","last_peered":"2024-06-10T00:32:50.198807+0000","last_clean":"2024-06-10T00:32:50.198807+0000","last_became_active":"2024-06-10T00:28:37.658574+0000","last_became_peered":"2024-06-10T00:28:37.658574+0000","last_unstale":"2024-06-10T00:32:50.198807+0000","last_undegraded":"2024-06-10T00:32:50.198807+0000","last_fullsized":"2024-06-10T00:32:50.198807+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:16:47.437822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"97'1","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921563+0000","last_change":"2024-06-10T00:28:48.706560+0000","last_active":"2024-06-10T00:32:49.921563+0000","last_peered":"2024-06-10T00:32:49.921563+0000","last_clean":"2024-06-10T00:32:49.921563+0000","last_became_active":"2024-06-10T00:28:48.705986+0000","last_became_peered":"2024-06-10T00:28:48.705986+0000","last_unstale":"2024-06-10T00:32:49.921563+0000","last_undegraded":"2024-06-10T00:32:49.921563+0000","last_fullsized":"2024-06-10T00:32:49.921563+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:21:26.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'4","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.892250+0000","last_change":"2024-06-10T00:28:34.394509+0000","last_active":"2024-06-10T00:32:50.892250+0000","last_peered":"2024-06-10T00:32:50.892250+0000","last_clean":"2024-06-10T00:32:50.892250+0000","last_became_active":"2024-06-10T00:28:34.394105+0000","last_became_peered":"2024-06-10T00:28:34.394105+0000","last_unstale":"2024-06-10T00:32:50.892250+0000","last_undegraded":"2024-06-10T00:32:50.892250+0000","last_fullsized":"2024-06-10T00:32:50.892250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:03:13.415544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032324+0000","last_change":"2024-06-10T00:28:37.653192+0000","last_active":"2024-06-10T00:32:50.032324+0000","last_peered":"2024-06-10T00:32:50.032324+0000","last_clean":"2024-06-10T00:32:50.032324+0000","last_became_active":"2024-06-10T00:28:37.652868+0000","last_became_peered":"2024-06-10T00:28:37.652868+0000","last_unstale":"2024-06-10T00:32:50.032324+0000","last_undegraded":"2024-06-10T00:32:50.032324+0000","last_fullsized":"2024-06-10T00:32:50.032324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:21.614915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921695+0000","last_change":"2024-06-10T00:28:48.727615+0000","last_active":"2024-06-10T00:32:49.921695+0000","last_peered":"2024-06-10T00:32:49.921695+0000","last_clean":"2024-06-10T00:32:49.921695+0000","last_became_active":"2024-06-10T00:28:48.727190+0000","last_became_peered":"2024-06-10T00:28:48.727190+0000","last_unstale":"2024-06-10T00:32:49.921695+0000","last_undegraded":"2024-06-10T00:32:49.921695+0000","last_fullsized":"2024-06-10T00:32:49.921695+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:57.498183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"97'4","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.031777+0000","last_change":"2024-06-10T00:29:23.784131+0000","last_active":"2024-06-10T00:32:50.031777+0000","last_peered":"2024-06-10T00:32:50.031777+0000","last_clean":"2024-06-10T00:32:50.031777+0000","last_became_active":"2024-06-10T00:29:23.780909+0000","last_became_peered":"2024-06-10T00:29:23.780909+0000","last_unstale":"2024-06-10T00:32:50.031777+0000","last_undegraded":"2024-06-10T00:32:50.031777+0000","last_fullsized":"2024-06-10T00:32:50.031777+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:03.774217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"99'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134481+0000","last_change":"2024-06-10T00:28:37.660399+0000","last_active":"2024-06-10T00:32:50.134481+0000","last_peered":"2024-06-10T00:32:50.134481+0000","last_clean":"2024-06-10T00:32:50.134481+0000","last_became_active":"2024-06-10T00:28:37.660069+0000","last_became_peered":"2024-06-10T00:28:37.660069+0000","last_unstale":"2024-06-10T00:32:50.134481+0000","last_undegraded":"2024-06-10T00:32:50.134481+0000","last_fullsized":"2024-06-10T00:32:50.134481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:59.727883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188531+0000","last_change":"2024-06-10T00:28:48.713267+0000","last_active":"2024-06-10T00:32:50.188531+0000","last_peered":"2024-06-10T00:32:50.188531+0000","last_clean":"2024-06-10T00:32:50.188531+0000","last_became_active":"2024-06-10T00:28:48.712870+0000","last_became_peered":"2024-06-10T00:28:48.712870+0000","last_unstale":"2024-06-10T00:32:50.188531+0000","last_undegraded":"2024-06-10T00:32:50.188531+0000","last_fullsized":"2024-06-10T00:32:50.188531+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:15:59.210101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'6","reported_seq":261,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:09.191063+0000","last_change":"2024-06-10T00:28:34.448652+0000","last_active":"2024-06-10T00:33:09.191063+0000","last_peered":"2024-06-10T00:33:09.191063+0000","last_clean":"2024-06-10T00:33:09.191063+0000","last_became_active":"2024-06-10T00:28:34.448243+0000","last_became_peered":"2024-06-10T00:28:34.448243+0000","last_unstale":"2024-06-10T00:33:09.191063+0000","last_undegraded":"2024-06-10T00:33:09.191063+0000","last_fullsized":"2024-06-10T00:33:09.191063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:51:40.003429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"99'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.135081+0000","last_change":"2024-06-10T00:28:37.662307+0000","last_active":"2024-06-10T00:32:50.135081+0000","last_peered":"2024-06-10T00:32:50.135081+0000","last_clean":"2024-06-10T00:32:50.135081+0000","last_became_active":"2024-06-10T00:28:37.661727+0000","last_became_peered":"2024-06-10T00:28:37.661727+0000","last_unstale":"2024-06-10T00:32:50.135081+0000","last_undegraded":"2024-06-10T00:32:50.135081+0000","last_fullsized":"2024-06-10T00:32:50.135081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:09:23.989429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192376+0000","last_change":"2024-06-10T00:29:23.857626+0000","last_active":"2024-06-10T00:32:50.192376+0000","last_peered":"2024-06-10T00:32:50.192376+0000","last_clean":"2024-06-10T00:32:50.192376+0000","last_became_active":"2024-06-10T00:29:23.857057+0000","last_became_peered":"2024-06-10T00:29:23.857057+0000","last_unstale":"2024-06-10T00:32:50.192376+0000","last_undegraded":"2024-06-10T00:32:50.192376+0000","last_fullsized":"2024-06-10T00:32:50.192376+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:25:10.340053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"100'67","reported_seq":145,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:33:03.056001+0000","last_change":"2024-06-10T00:28:34.399253+0000","last_active":"2024-06-10T00:33:03.056001+0000","last_peered":"2024-06-10T00:33:03.056001+0000","last_clean":"2024-06-10T00:33:03.056001+0000","last_became_active":"2024-06-10T00:28:34.397763+0000","last_became_peered":"2024-06-10T00:28:34.397763+0000","last_unstale":"2024-06-10T00:33:03.056001+0000","last_undegraded":"2024-06-10T00:33:03.056001+0000","last_fullsized":"2024-06-10T00:33:03.056001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:05:57.371625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"99'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134749+0000","last_change":"2024-06-10T00:28:37.658027+0000","last_active":"2024-06-10T00:32:50.134749+0000","last_peered":"2024-06-10T00:32:50.134749+0000","last_clean":"2024-06-10T00:32:50.134749+0000","last_became_active":"2024-06-10T00:28:37.655513+0000","last_became_peered":"2024-06-10T00:28:37.655513+0000","last_unstale":"2024-06-10T00:32:50.134749+0000","last_undegraded":"2024-06-10T00:32:50.134749+0000","last_fullsized":"2024-06-10T00:32:50.134749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:50:23.525855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038618+0000","last_change":"2024-06-10T00:28:48.706997+0000","last_active":"2024-06-10T00:32:50.038618+0000","last_peered":"2024-06-10T00:32:50.038618+0000","last_clean":"2024-06-10T00:32:50.038618+0000","last_became_active":"2024-06-10T00:28:48.706597+0000","last_became_peered":"2024-06-10T00:28:48.706597+0000","last_unstale":"2024-06-10T00:32:50.038618+0000","last_undegraded":"2024-06-10T00:32:50.038618+0000","last_fullsized":"2024-06-10T00:32:50.038618+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:47.672123+0000","last_clean_scrub_stamp":"2024-06-10T00:28:47.672123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:24:16.385312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"97'5","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133955+0000","last_change":"2024-06-10T00:29:23.785774+0000","last_active":"2024-06-10T00:32:50.133955+0000","last_peered":"2024-06-10T00:32:50.133955+0000","last_clean":"2024-06-10T00:32:50.133955+0000","last_became_active":"2024-06-10T00:29:23.781768+0000","last_became_peered":"2024-06-10T00:29:23.781768+0000","last_unstale":"2024-06-10T00:32:50.133955+0000","last_undegraded":"2024-06-10T00:32:50.133955+0000","last_fullsized":"2024-06-10T00:32:50.133955+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:33.369965+0000","last_clean_scrub_stamp":"2024-06-10T00:28:33.369965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:25:27.348264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"99'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.134953+0000","last_change":"2024-06-10T00:28:37.657820+0000","last_active":"2024-06-10T00:32:50.134953+0000","last_peered":"2024-06-10T00:32:50.134953+0000","last_clean":"2024-06-10T00:32:50.134953+0000","last_became_active":"2024-06-10T00:28:37.657487+0000","last_became_peered":"2024-06-10T00:28:37.657487+0000","last_unstale":"2024-06-10T00:32:50.134953+0000","last_undegraded":"2024-06-10T00:32:50.134953+0000","last_fullsized":"2024-06-10T00:32:50.134953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:44:35.930401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"99'9","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.198953+0000","last_change":"2024-06-10T00:28:37.664543+0000","last_active":"2024-06-10T00:32:50.198953+0000","last_peered":"2024-06-10T00:32:50.198953+0000","last_clean":"2024-06-10T00:32:50.198953+0000","last_became_active":"2024-06-10T00:28:37.659015+0000","last_became_peered":"2024-06-10T00:28:37.659015+0000","last_unstale":"2024-06-10T00:32:50.198953+0000","last_undegraded":"2024-06-10T00:32:50.198953+0000","last_fullsized":"2024-06-10T00:32:50.198953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:51:06.555472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"99'12","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921072+0000","last_change":"2024-06-10T00:28:37.659389+0000","last_active":"2024-06-10T00:32:49.921072+0000","last_peered":"2024-06-10T00:32:49.921072+0000","last_clean":"2024-06-10T00:32:49.921072+0000","last_became_active":"2024-06-10T00:28:37.658990+0000","last_became_peered":"2024-06-10T00:28:37.658990+0000","last_unstale":"2024-06-10T00:32:49.921072+0000","last_undegraded":"2024-06-10T00:32:49.921072+0000","last_fullsized":"2024-06-10T00:32:49.921072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:29.532779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.920823+0000","last_change":"2024-06-10T00:28:37.655621+0000","last_active":"2024-06-10T00:32:49.920823+0000","last_peered":"2024-06-10T00:32:49.920823+0000","last_clean":"2024-06-10T00:32:49.920823+0000","last_became_active":"2024-06-10T00:28:37.653599+0000","last_became_peered":"2024-06-10T00:28:37.653599+0000","last_unstale":"2024-06-10T00:32:49.920823+0000","last_undegraded":"2024-06-10T00:32:49.920823+0000","last_fullsized":"2024-06-10T00:32:49.920823+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:22.165851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"99'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.192846+0000","last_change":"2024-06-10T00:29:23.782727+0000","last_active":"2024-06-10T00:32:50.192846+0000","last_peered":"2024-06-10T00:32:50.192846+0000","last_clean":"2024-06-10T00:32:50.192846+0000","last_became_active":"2024-06-10T00:29:23.782309+0000","last_became_peered":"2024-06-10T00:29:23.782309+0000","last_unstale":"2024-06-10T00:32:50.192846+0000","last_undegraded":"2024-06-10T00:32:50.192846+0000","last_fullsized":"2024-06-10T00:32:50.192846+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:31:20.585955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"99'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133931+0000","last_change":"2024-06-10T00:28:37.646750+0000","last_active":"2024-06-10T00:32:50.133931+0000","last_peered":"2024-06-10T00:32:50.133931+0000","last_clean":"2024-06-10T00:32:50.133931+0000","last_became_active":"2024-06-10T00:28:37.646372+0000","last_became_peered":"2024-06-10T00:28:37.646372+0000","last_unstale":"2024-06-10T00:32:50.133931+0000","last_undegraded":"2024-06-10T00:32:50.133931+0000","last_fullsized":"2024-06-10T00:32:50.133931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:23:45.762956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'6","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199378+0000","last_change":"2024-06-10T00:28:37.662278+0000","last_active":"2024-06-10T00:32:50.199378+0000","last_peered":"2024-06-10T00:32:50.199378+0000","last_clean":"2024-06-10T00:32:50.199378+0000","last_became_active":"2024-06-10T00:28:37.661855+0000","last_became_peered":"2024-06-10T00:28:37.661855+0000","last_unstale":"2024-06-10T00:32:50.199378+0000","last_undegraded":"2024-06-10T00:32:50.199378+0000","last_fullsized":"2024-06-10T00:32:50.199378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:48:20.074208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"99'15","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:49.921342+0000","last_change":"2024-06-10T00:28:37.656738+0000","last_active":"2024-06-10T00:32:49.921342+0000","last_peered":"2024-06-10T00:32:49.921342+0000","last_clean":"2024-06-10T00:32:49.921342+0000","last_became_active":"2024-06-10T00:28:37.656375+0000","last_became_peered":"2024-06-10T00:28:37.656375+0000","last_unstale":"2024-06-10T00:32:49.921342+0000","last_undegraded":"2024-06-10T00:32:49.921342+0000","last_fullsized":"2024-06-10T00:32:49.921342+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:49:17.779670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"99'2","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032601+0000","last_change":"2024-06-10T00:28:37.656849+0000","last_active":"2024-06-10T00:32:50.032601+0000","last_peered":"2024-06-10T00:32:50.032601+0000","last_clean":"2024-06-10T00:32:50.032601+0000","last_became_active":"2024-06-10T00:28:37.656261+0000","last_became_peered":"2024-06-10T00:28:37.656261+0000","last_unstale":"2024-06-10T00:32:50.032601+0000","last_undegraded":"2024-06-10T00:32:50.032601+0000","last_fullsized":"2024-06-10T00:32:50.032601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:13.152165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"99'6","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.038849+0000","last_change":"2024-06-10T00:28:37.648384+0000","last_active":"2024-06-10T00:32:50.038849+0000","last_peered":"2024-06-10T00:32:50.038849+0000","last_clean":"2024-06-10T00:32:50.038849+0000","last_became_active":"2024-06-10T00:28:37.648009+0000","last_became_peered":"2024-06-10T00:28:37.648009+0000","last_unstale":"2024-06-10T00:32:50.038849+0000","last_undegraded":"2024-06-10T00:32:50.038849+0000","last_fullsized":"2024-06-10T00:32:50.038849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:45:55.808932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.032228+0000","last_change":"2024-06-10T00:28:37.655828+0000","last_active":"2024-06-10T00:32:50.032228+0000","last_peered":"2024-06-10T00:32:50.032228+0000","last_clean":"2024-06-10T00:32:50.032228+0000","last_became_active":"2024-06-10T00:28:37.655269+0000","last_became_peered":"2024-06-10T00:28:37.655269+0000","last_unstale":"2024-06-10T00:32:50.032228+0000","last_undegraded":"2024-06-10T00:32:50.032228+0000","last_fullsized":"2024-06-10T00:32:50.032228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:32:56.618469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"99'10","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.188660+0000","last_change":"2024-06-10T00:28:37.663756+0000","last_active":"2024-06-10T00:32:50.188660+0000","last_peered":"2024-06-10T00:32:50.188660+0000","last_clean":"2024-06-10T00:32:50.188660+0000","last_became_active":"2024-06-10T00:28:37.663380+0000","last_became_peered":"2024-06-10T00:28:37.663380+0000","last_unstale":"2024-06-10T00:32:50.188660+0000","last_undegraded":"2024-06-10T00:32:50.188660+0000","last_fullsized":"2024-06-10T00:32:50.188660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:46:45.534648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"99'11","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.133177+0000","last_change":"2024-06-10T00:28:37.655246+0000","last_active":"2024-06-10T00:32:50.133177+0000","last_peered":"2024-06-10T00:32:50.133177+0000","last_clean":"2024-06-10T00:32:50.133177+0000","last_became_active":"2024-06-10T00:28:37.654605+0000","last_became_peered":"2024-06-10T00:28:37.654605+0000","last_unstale":"2024-06-10T00:32:50.133177+0000","last_undegraded":"2024-06-10T00:32:50.133177+0000","last_fullsized":"2024-06-10T00:32:50.133177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:44:15.592777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"99'8","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:32:50.199172+0000","last_change":"2024-06-10T00:28:37.662485+0000","last_active":"2024-06-10T00:32:50.199172+0000","last_peered":"2024-06-10T00:32:50.199172+0000","last_clean":"2024-06-10T00:32:50.199172+0000","last_became_active":"2024-06-10T00:28:37.661900+0000","last_became_peered":"2024-06-10T00:28:37.661900+0000","last_unstale":"2024-06-10T00:32:50.199172+0000","last_undegraded":"2024-06-10T00:32:50.199172+0000","last_fullsized":"2024-06-10T00:32:50.199172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:28:36.628727+0000","last_clean_scrub_stamp":"2024-06-10T00:28:36.628727+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:30:03.210634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800945,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2448,"num_read_kb":1450,"num_write":512,"num_write_kb":2392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2402835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":390,"ondisk_log_size":390,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383750,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1175552,"data_stored":581073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645386,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29280,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715872,"statfs":{"total":95995035648,"available":95965052928,"internally_reserved":0,"allocated":2195456,"data_stored":1565815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808911,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":977723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070550,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36860,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708292,"statfs":{"total":95995035648,"available":95957291008,"internally_reserved":0,"allocated":1560576,"data_stored":973221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332186,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28760,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716392,"statfs":{"total":95995035648,"available":95965585408,"internally_reserved":0,"allocated":1781760,"data_stored":1180215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495711,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":577003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36556,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708596,"statfs":{"total":95995035648,"available":95957602304,"internally_reserved":0,"allocated":1245184,"data_stored":648070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738493,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36500,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708652,"statfs":{"total":95995035648,"available":95957659648,"internally_reserved":0,"allocated":1187840,"data_stored":577761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36564,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708588,"statfs":{"total":95995035648,"available":95957594112,"internally_reserved":0,"allocated":1257472,"data_stored":648231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28992,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716160,"statfs":{"total":95995035648,"available":95965347840,"internally_reserved":0,"allocated":1900544,"data_stored":1297071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920873,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36484,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708668,"statfs":{"total":95995035648,"available":95957676032,"internally_reserved":0,"allocated":1171456,"data_stored":580910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757348,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29204,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715948,"statfs":{"total":95995035648,"available":95965130752,"internally_reserved":0,"allocated":2117632,"data_stored":1501930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":332273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":400254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:33:13.743 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:33:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:14.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:14 smithi062 ceph-mon[32390]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:33:14.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:14 smithi105 ceph-mon[36321]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:33:14.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:14 smithi138 ceph-mon[36549]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:33:15.453 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:15 smithi062 ceph-mon[32390]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-06-10T00:33:15.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:15 smithi105 ceph-mon[36321]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-06-10T00:33:15.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:15 smithi138 ceph-mon[36549]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-06-10T00:33:15.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.693+0000 7f30a7207700 1 -- 172.21.15.62:0/2043232268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a0102740 msgr2=0x7f30a0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:15.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.693+0000 7f30a7207700 1 --2- 172.21.15.62:0/2043232268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a0102740 0x7f30a0102b20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f30940099c0 tx=0x7f3094009cd0 comp rx=0 tx=0).stop 2024-06-10T00:33:15.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.693+0000 7f30a7207700 1 -- 172.21.15.62:0/2043232268 shutdown_connections 2024-06-10T00:33:15.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.693+0000 7f30a7207700 1 --2- 172.21.15.62:0/2043232268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a0102740 0x7f30a0102b20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:15.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.693+0000 7f30a7207700 1 --2- 172.21.15.62:0/2043232268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30a0107840 0x7f30a010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:15.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.693+0000 7f30a7207700 1 --2- 172.21.15.62:0/2043232268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30a01030f0 0x7f30a0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:15.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.693+0000 7f30a7207700 1 -- 172.21.15.62:0/2043232268 >> 172.21.15.62:0/2043232268 conn(0x7f30a00795f0 msgr2=0x7f30a0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:15.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.693+0000 7f30a7207700 1 -- 172.21.15.62:0/2043232268 shutdown_connections 2024-06-10T00:33:15.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.693+0000 7f30a7207700 1 -- 172.21.15.62:0/2043232268 wait complete. 2024-06-10T00:33:15.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.693+0000 7f30a7207700 1 Processor -- start 2024-06-10T00:33:15.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.693+0000 7f30a7207700 1 -- start start 2024-06-10T00:33:15.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.693+0000 7f30a7207700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30a0102740 0x7f30a0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:15.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f30a7207700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a01030f0 0x7f30a0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:15.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f30a7207700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30a0107840 0x7f30a0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:15.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f30a7207700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f30a0112760 con 0x7f30a01030f0 2024-06-10T00:33:15.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f30a7207700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f30a01125e0 con 0x7f30a0102740 2024-06-10T00:33:15.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f30a7207700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30a01128e0 con 0x7f30a0107840 2024-06-10T00:33:15.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f309ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a01030f0 0x7f30a0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:15.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f30a4fa3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30a0102740 0x7f30a0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f309ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a01030f0 0x7f30a0074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47820/0 (socket says 172.21.15.62:47820) 2024-06-10T00:33:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f309ffff700 1 -- 172.21.15.62:0/4111599073 learned_addr learned my addr 172.21.15.62:0/4111599073 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f30a57a4700 1 --2- 172.21.15.62:0/4111599073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30a0107840 0x7f30a0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f309ffff700 1 -- 172.21.15.62:0/4111599073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30a0107840 msgr2=0x7f30a0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f309ffff700 1 --2- 172.21.15.62:0/4111599073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30a0107840 0x7f30a0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f309ffff700 1 -- 172.21.15.62:0/4111599073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30a0102740 msgr2=0x7f30a0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f309ffff700 1 --2- 172.21.15.62:0/4111599073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30a0102740 0x7f30a0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f309ffff700 1 -- 172.21.15.62:0/4111599073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a01a7130 con 0x7f30a01030f0 2024-06-10T00:33:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f30a4fa3700 1 --2- 172.21.15.62:0/4111599073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30a0102740 0x7f30a0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f309ffff700 1 --2- 172.21.15.62:0/4111599073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a01030f0 0x7f30a0074cd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f308c00c920 tx=0x7f308c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:15.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f309dffb700 1 -- 172.21.15.62:0/4111599073 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c004a30 con 0x7f30a01030f0 2024-06-10T00:33:15.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.694+0000 7f309dffb700 1 -- 172.21.15.62:0/4111599073 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f308c005500 con 0x7f30a01030f0 2024-06-10T00:33:15.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.695+0000 7f30a7207700 1 -- 172.21.15.62:0/4111599073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a01a7420 con 0x7f30a01030f0 2024-06-10T00:33:15.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.695+0000 7f309dffb700 1 -- 172.21.15.62:0/4111599073 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c007d50 con 0x7f30a01030f0 2024-06-10T00:33:15.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.695+0000 7f30a7207700 1 -- 172.21.15.62:0/4111599073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a01a7960 con 0x7f30a01030f0 2024-06-10T00:33:15.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.696+0000 7f30a7207700 1 -- 172.21.15.62:0/4111599073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30a010aa80 con 0x7f30a01030f0 2024-06-10T00:33:15.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.697+0000 7f309dffb700 1 -- 172.21.15.62:0/4111599073 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f308c004d60 con 0x7f30a01030f0 2024-06-10T00:33:15.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.698+0000 7f309dffb700 1 --2- 172.21.15.62:0/4111599073 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f30900a2e20 0x7f30900a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:15.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.698+0000 7f309dffb700 1 -- 172.21.15.62:0/4111599073 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f308c0cf910 con 0x7f30a01030f0 2024-06-10T00:33:15.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.698+0000 7f30a4fa3700 1 --2- 172.21.15.62:0/4111599073 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f30900a2e20 0x7f30900a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:15.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.698+0000 7f30a4fa3700 1 --2- 172.21.15.62:0/4111599073 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f30900a2e20 0x7f30900a52e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3094000c00 tx=0x7f3094005fb0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:15.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.701+0000 7f309dffb700 1 -- 172.21.15.62:0/4111599073 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f308c014030 con 0x7f30a01030f0 2024-06-10T00:33:15.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.954+0000 7f30a7207700 1 -- 172.21.15.62:0/4111599073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30a0069180 con 0x7f30a01030f0 2024-06-10T00:33:15.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.954+0000 7f309dffb700 1 -- 172.21.15.62:0/4111599073 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f308c09c010 con 0x7f30a01030f0 2024-06-10T00:33:15.955 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:33:15.955 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:32:47.783154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:33:15.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.957+0000 7f30a7207700 1 -- 172.21.15.62:0/4111599073 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f30900a2e20 msgr2=0x7f30900a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:15.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.957+0000 7f30a7207700 1 --2- 172.21.15.62:0/4111599073 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f30900a2e20 0x7f30900a52e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3094000c00 tx=0x7f3094005fb0 comp rx=0 tx=0).stop 2024-06-10T00:33:15.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.957+0000 7f30a7207700 1 -- 172.21.15.62:0/4111599073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a01030f0 msgr2=0x7f30a0074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:15.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.957+0000 7f30a7207700 1 --2- 172.21.15.62:0/4111599073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a01030f0 0x7f30a0074cd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f308c00c920 tx=0x7f308c00cc30 comp rx=0 tx=0).stop 2024-06-10T00:33:15.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.957+0000 7f30a7207700 1 -- 172.21.15.62:0/4111599073 shutdown_connections 2024-06-10T00:33:15.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.957+0000 7f30a7207700 1 --2- 172.21.15.62:0/4111599073 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f30900a2e20 0x7f30900a52e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:15.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.957+0000 7f30a7207700 1 --2- 172.21.15.62:0/4111599073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30a0107840 0x7f30a0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:15.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.957+0000 7f30a7207700 1 --2- 172.21.15.62:0/4111599073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30a0102740 0x7f30a0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:15.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.957+0000 7f30a7207700 1 --2- 172.21.15.62:0/4111599073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30a01030f0 0x7f30a0074cd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:15.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.957+0000 7f30a7207700 1 -- 172.21.15.62:0/4111599073 >> 172.21.15.62:0/4111599073 conn(0x7f30a00795f0 msgr2=0x7f30a0108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:15.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.957+0000 7f30a7207700 1 -- 172.21.15.62:0/4111599073 shutdown_connections 2024-06-10T00:33:15.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:15.957+0000 7f30a7207700 1 -- 172.21.15.62:0/4111599073 wait complete. 2024-06-10T00:33:15.958 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-06-10T00:33:16.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:16 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/4111599073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:33:16.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:16 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/4111599073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:33:16.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:16 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/4111599073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:33:16.733 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:33:16.992 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:17.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:17 smithi062 ceph-mon[32390]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-06-10T00:33:17.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:17 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:33:17.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:17 smithi105 ceph-mon[36321]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-06-10T00:33:17.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:17 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:33:17.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:17 smithi138 ceph-mon[36549]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-06-10T00:33:17.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:17 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:33:18.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.642+0000 7ff38b43c700 1 -- 172.21.15.62:0/1506688617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff384100a80 msgr2=0x7ff384100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:18.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.642+0000 7ff38b43c700 1 --2- 172.21.15.62:0/1506688617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff384100a80 0x7ff384100f00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff378009990 tx=0x7ff378009ca0 comp rx=0 tx=0).stop 2024-06-10T00:33:18.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.642+0000 7ff38b43c700 1 -- 172.21.15.62:0/1506688617 shutdown_connections 2024-06-10T00:33:18.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.642+0000 7ff38b43c700 1 --2- 172.21.15.62:0/1506688617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff384101440 0x7ff38410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:18.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.642+0000 7ff38b43c700 1 --2- 172.21.15.62:0/1506688617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff384108510 0x7ff3841088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:18.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.642+0000 7ff38b43c700 1 --2- 172.21.15.62:0/1506688617 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff384100a80 0x7ff384100f00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:18.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.642+0000 7ff38b43c700 1 -- 172.21.15.62:0/1506688617 >> 172.21.15.62:0/1506688617 conn(0x7ff384077a00 msgr2=0x7ff3840fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:18.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.642+0000 7ff38b43c700 1 -- 172.21.15.62:0/1506688617 shutdown_connections 2024-06-10T00:33:18.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.642+0000 7ff38b43c700 1 -- 172.21.15.62:0/1506688617 wait complete. 2024-06-10T00:33:18.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.643+0000 7ff38b43c700 1 Processor -- start 2024-06-10T00:33:18.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.643+0000 7ff38b43c700 1 -- start start 2024-06-10T00:33:18.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.643+0000 7ff38b43c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff384100a80 0x7ff384101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:18.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.643+0000 7ff38b43c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff384101440 0x7ff3841021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:18.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.643+0000 7ff38b43c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff384108510 0x7ff3841026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:18.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.643+0000 7ff38b43c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff3841127f0 con 0x7ff384101440 2024-06-10T00:33:18.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.643+0000 7ff38b43c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff384112670 con 0x7ff384100a80 2024-06-10T00:33:18.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.643+0000 7ff38b43c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff384112970 con 0x7ff384108510 2024-06-10T00:33:18.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.643+0000 7ff3889d7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff384101440 0x7ff3841021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:18.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.643+0000 7ff3891d8700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff384100a80 0x7ff384101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:18.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.643+0000 7ff3889d7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff384101440 0x7ff3841021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59226/0 (socket says 172.21.15.62:59226) 2024-06-10T00:33:18.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.643+0000 7ff3889d7700 1 -- 172.21.15.62:0/2973541954 learned_addr learned my addr 172.21.15.62:0/2973541954 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:18.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.643+0000 7ff3899d9700 1 --2- 172.21.15.62:0/2973541954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff384108510 0x7ff3841026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:18.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.644+0000 7ff3891d8700 1 -- 172.21.15.62:0/2973541954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff384108510 msgr2=0x7ff3841026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:18.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.644+0000 7ff3891d8700 1 --2- 172.21.15.62:0/2973541954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff384108510 0x7ff3841026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:18.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.644+0000 7ff3891d8700 1 -- 172.21.15.62:0/2973541954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff384101440 msgr2=0x7ff3841021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:18.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.644+0000 7ff3891d8700 1 --2- 172.21.15.62:0/2973541954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff384101440 0x7ff3841021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:18.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.644+0000 7ff3891d8700 1 -- 172.21.15.62:0/2973541954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3841a71f0 con 0x7ff384100a80 2024-06-10T00:33:18.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.644+0000 7ff3889d7700 1 --2- 172.21.15.62:0/2973541954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff384101440 0x7ff3841021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:18.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.644+0000 7ff3891d8700 1 --2- 172.21.15.62:0/2973541954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff384100a80 0x7ff384101c60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff37400bd70 tx=0x7ff37400bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:18.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.644+0000 7ff3827fc700 1 -- 172.21.15.62:0/2973541954 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374019070 con 0x7ff384100a80 2024-06-10T00:33:18.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.644+0000 7ff3827fc700 1 -- 172.21.15.62:0/2973541954 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37400cc10 con 0x7ff384100a80 2024-06-10T00:33:18.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.644+0000 7ff3827fc700 1 -- 172.21.15.62:0/2973541954 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374012630 con 0x7ff384100a80 2024-06-10T00:33:18.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.644+0000 7ff38b43c700 1 -- 172.21.15.62:0/2973541954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3841a74e0 con 0x7ff384100a80 2024-06-10T00:33:18.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.644+0000 7ff38b43c700 1 -- 172.21.15.62:0/2973541954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3841a79d0 con 0x7ff384100a80 2024-06-10T00:33:18.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.645+0000 7ff38b43c700 1 -- 172.21.15.62:0/2973541954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff38410a9f0 con 0x7ff384100a80 2024-06-10T00:33:18.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.646+0000 7ff3827fc700 1 -- 172.21.15.62:0/2973541954 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7ff37401c070 con 0x7ff384100a80 2024-06-10T00:33:18.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.647+0000 7ff3827fc700 1 --2- 172.21.15.62:0/2973541954 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff36c0a2e20 0x7ff36c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:18.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.647+0000 7ff3889d7700 1 --2- 172.21.15.62:0/2973541954 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff36c0a2e20 0x7ff36c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:18.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.647+0000 7ff3827fc700 1 -- 172.21.15.62:0/2973541954 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7ff3740cf370 con 0x7ff384100a80 2024-06-10T00:33:18.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.648+0000 7ff3889d7700 1 --2- 172.21.15.62:0/2973541954 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff36c0a2e20 0x7ff36c0a52e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff384107c00 tx=0x7ff378009630 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:18.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.651+0000 7ff3827fc700 1 -- 172.21.15.62:0/2973541954 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff37409bb20 con 0x7ff384100a80 2024-06-10T00:33:18.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.902+0000 7ff38b43c700 1 -- 172.21.15.62:0/2973541954 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff384069180 con 0x7ff384100a80 2024-06-10T00:33:18.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.903+0000 7ff3827fc700 1 -- 172.21.15.62:0/2973541954 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7ff37409b420 con 0x7ff384100a80 2024-06-10T00:33:18.903 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:33:18.904 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:32:47.783154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:33:18.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.905+0000 7ff38b43c700 1 -- 172.21.15.62:0/2973541954 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff36c0a2e20 msgr2=0x7ff36c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:18.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.905+0000 7ff38b43c700 1 --2- 172.21.15.62:0/2973541954 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff36c0a2e20 0x7ff36c0a52e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff384107c00 tx=0x7ff378009630 comp rx=0 tx=0).stop 2024-06-10T00:33:18.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.905+0000 7ff38b43c700 1 -- 172.21.15.62:0/2973541954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff384100a80 msgr2=0x7ff384101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:18.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.905+0000 7ff38b43c700 1 --2- 172.21.15.62:0/2973541954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff384100a80 0x7ff384101c60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff37400bd70 tx=0x7ff37400bda0 comp rx=0 tx=0).stop 2024-06-10T00:33:18.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.905+0000 7ff38b43c700 1 -- 172.21.15.62:0/2973541954 shutdown_connections 2024-06-10T00:33:18.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.905+0000 7ff38b43c700 1 --2- 172.21.15.62:0/2973541954 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff36c0a2e20 0x7ff36c0a52e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:18.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.905+0000 7ff38b43c700 1 --2- 172.21.15.62:0/2973541954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff384108510 0x7ff3841026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:18.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.905+0000 7ff38b43c700 1 --2- 172.21.15.62:0/2973541954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff384100a80 0x7ff384101c60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:18.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.905+0000 7ff38b43c700 1 --2- 172.21.15.62:0/2973541954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff384101440 0x7ff3841021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:18.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.905+0000 7ff38b43c700 1 -- 172.21.15.62:0/2973541954 >> 172.21.15.62:0/2973541954 conn(0x7ff384077a00 msgr2=0x7ff3840ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:18.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.905+0000 7ff38b43c700 1 -- 172.21.15.62:0/2973541954 shutdown_connections 2024-06-10T00:33:18.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:18.905+0000 7ff38b43c700 1 -- 172.21.15.62:0/2973541954 wait complete. 2024-06-10T00:33:18.907 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-06-10T00:33:19.517 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:19 smithi062 ceph-mon[32390]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 41 op/s 2024-06-10T00:33:19.517 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:19 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2973541954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:33:19.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:19 smithi105 ceph-mon[36321]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 41 op/s 2024-06-10T00:33:19.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:19 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2973541954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:33:19.544 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:33:19.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:19 smithi138 ceph-mon[36549]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 41 op/s 2024-06-10T00:33:19.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:19 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2973541954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:33:19.792 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:21.441 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:21 smithi062 ceph-mon[32390]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:21.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:21 smithi105 ceph-mon[36321]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:21.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:21 smithi138 ceph-mon[36549]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.707+0000 7f1485ba8700 1 -- 172.21.15.62:0/435710283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101d50 msgr2=0x7f148010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.707+0000 7f1485ba8700 1 --2- 172.21.15.62:0/435710283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101d50 0x7f148010e240 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1478009990 tx=0x7f1478009ca0 comp rx=0 tx=0).stop 2024-06-10T00:33:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.707+0000 7f1485ba8700 1 -- 172.21.15.62:0/435710283 shutdown_connections 2024-06-10T00:33:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.707+0000 7f1485ba8700 1 --2- 172.21.15.62:0/435710283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101d50 0x7f148010e240 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.707+0000 7f1485ba8700 1 --2- 172.21.15.62:0/435710283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1480107400 0x7f14801077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.707+0000 7f1485ba8700 1 --2- 172.21.15.62:0/435710283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1480101390 0x7f1480101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.707+0000 7f1485ba8700 1 -- 172.21.15.62:0/435710283 >> 172.21.15.62:0/435710283 conn(0x7f1480078340 msgr2=0x7f1480078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.707+0000 7f1485ba8700 1 -- 172.21.15.62:0/435710283 shutdown_connections 2024-06-10T00:33:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.707+0000 7f1485ba8700 1 -- 172.21.15.62:0/435710283 wait complete. 2024-06-10T00:33:21.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.708+0000 7f1485ba8700 1 Processor -- start 2024-06-10T00:33:21.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.708+0000 7f1485ba8700 1 -- start start 2024-06-10T00:33:21.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.708+0000 7f1485ba8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1480101390 0x7f1480103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:21.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.708+0000 7f1485ba8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101d50 0x7f14801035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:21.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.708+0000 7f1485ba8700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1480107400 0x7f1480103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:21.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.708+0000 7f1485ba8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1480111670 con 0x7f1480101d50 2024-06-10T00:33:21.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.708+0000 7f1485ba8700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f14801114f0 con 0x7f1480107400 2024-06-10T00:33:21.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.708+0000 7f1485ba8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14801117f0 con 0x7f1480101390 2024-06-10T00:33:21.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f147effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101d50 0x7f14801035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:21.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f147effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101d50 0x7f14801035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59254/0 (socket says 172.21.15.62:59254) 2024-06-10T00:33:21.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f147effd700 1 -- 172.21.15.62:0/3994341818 learned_addr learned my addr 172.21.15.62:0/3994341818 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:21.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f147ffff700 1 --2- 172.21.15.62:0/3994341818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1480107400 0x7f1480103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:21.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f147f7fe700 1 --2- 172.21.15.62:0/3994341818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1480101390 0x7f1480103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:21.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f147effd700 1 -- 172.21.15.62:0/3994341818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1480101390 msgr2=0x7f1480103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:21.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f147effd700 1 --2- 172.21.15.62:0/3994341818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1480101390 0x7f1480103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:21.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f147effd700 1 -- 172.21.15.62:0/3994341818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1480107400 msgr2=0x7f1480103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:21.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f147effd700 1 --2- 172.21.15.62:0/3994341818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1480107400 0x7f1480103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:21.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f147effd700 1 -- 172.21.15.62:0/3994341818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14801a70c0 con 0x7f1480101d50 2024-06-10T00:33:21.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f147effd700 1 --2- 172.21.15.62:0/3994341818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101d50 0x7f14801035b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f146800d980 tx=0x7f146800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:21.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f147cff9700 1 -- 172.21.15.62:0/3994341818 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468016070 con 0x7f1480101d50 2024-06-10T00:33:21.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f1485ba8700 1 -- 172.21.15.62:0/3994341818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14801a73b0 con 0x7f1480101d50 2024-06-10T00:33:21.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f147cff9700 1 -- 172.21.15.62:0/3994341818 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1468004d10 con 0x7f1480101d50 2024-06-10T00:33:21.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f1485ba8700 1 -- 172.21.15.62:0/3994341818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14801a7940 con 0x7f1480101d50 2024-06-10T00:33:21.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.709+0000 7f147cff9700 1 -- 172.21.15.62:0/3994341818 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468005670 con 0x7f1480101d50 2024-06-10T00:33:21.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.710+0000 7f1485ba8700 1 -- 172.21.15.62:0/3994341818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1480069180 con 0x7f1480101d50 2024-06-10T00:33:21.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.712+0000 7f147cff9700 1 -- 172.21.15.62:0/3994341818 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f1468004750 con 0x7f1480101d50 2024-06-10T00:33:21.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.712+0000 7f147cff9700 1 --2- 172.21.15.62:0/3994341818 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f146c0a2e20 0x7f146c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:21.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.713+0000 7f147f7fe700 1 --2- 172.21.15.62:0/3994341818 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f146c0a2e20 0x7f146c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:21.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.713+0000 7f147cff9700 1 -- 172.21.15.62:0/3994341818 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f14680cfc50 con 0x7f1480101d50 2024-06-10T00:33:21.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.713+0000 7f147f7fe700 1 --2- 172.21.15.62:0/3994341818 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f146c0a2e20 0x7f146c0a52e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f147000f540 tx=0x7f147000b040 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:21.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.716+0000 7f147cff9700 1 -- 172.21.15.62:0/3994341818 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f146801f020 con 0x7f1480101d50 2024-06-10T00:33:21.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.920+0000 7f1485ba8700 1 -- 172.21.15.62:0/3994341818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14801a7c60 con 0x7f1480101d50 2024-06-10T00:33:21.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.922+0000 7f147cff9700 1 -- 172.21.15.62:0/3994341818 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+25218 (secure 0 0 0) 0x7f146809c350 con 0x7f1480101d50 2024-06-10T00:33:21.922 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:33:21.924 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":100,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:32:49.913791+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3602410963":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/1557827118":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/2350700567":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/3602410963":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/1086776708":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/2528347688":"2024-06-11T00:32:49.913741+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:33:21.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.924+0000 7f1485ba8700 1 -- 172.21.15.62:0/3994341818 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f146c0a2e20 msgr2=0x7f146c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:21.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.924+0000 7f1485ba8700 1 --2- 172.21.15.62:0/3994341818 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f146c0a2e20 0x7f146c0a52e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f147000f540 tx=0x7f147000b040 comp rx=0 tx=0).stop 2024-06-10T00:33:21.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.924+0000 7f1485ba8700 1 -- 172.21.15.62:0/3994341818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101d50 msgr2=0x7f14801035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:21.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.924+0000 7f1485ba8700 1 --2- 172.21.15.62:0/3994341818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101d50 0x7f14801035b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f146800d980 tx=0x7f146800dc90 comp rx=0 tx=0).stop 2024-06-10T00:33:21.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.924+0000 7f1485ba8700 1 -- 172.21.15.62:0/3994341818 shutdown_connections 2024-06-10T00:33:21.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.924+0000 7f1485ba8700 1 --2- 172.21.15.62:0/3994341818 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f146c0a2e20 0x7f146c0a52e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:21.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.924+0000 7f1485ba8700 1 --2- 172.21.15.62:0/3994341818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1480107400 0x7f1480103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:21.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.924+0000 7f1485ba8700 1 --2- 172.21.15.62:0/3994341818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1480101390 0x7f1480103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:21.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.924+0000 7f1485ba8700 1 --2- 172.21.15.62:0/3994341818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1480101d50 0x7f14801035b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:21.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.924+0000 7f1485ba8700 1 -- 172.21.15.62:0/3994341818 >> 172.21.15.62:0/3994341818 conn(0x7f1480078340 msgr2=0x7f14801005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:21.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.925+0000 7f1485ba8700 1 -- 172.21.15.62:0/3994341818 shutdown_connections 2024-06-10T00:33:21.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:21.925+0000 7f1485ba8700 1 -- 172.21.15.62:0/3994341818 wait complete. 2024-06-10T00:33:22.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:22 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3994341818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:33:22.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:22 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3994341818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:33:22.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:22 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3994341818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:33:22.711 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-06-10T00:33:22.965 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:23.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:23 smithi062 ceph-mon[32390]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-10T00:33:23.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:23 smithi105 ceph-mon[36321]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-10T00:33:23.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:23 smithi138 ceph-mon[36549]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-10T00:33:25.081 INFO:teuthology.orchestra.run.smithi062.stdout:{"run_dir":"/var/run/ceph"} 2024-06-10T00:33:25.082 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-06-10T00:33:25.082 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 178 2024-06-10T00:33:25.082 DEBUG:tasks.quiescer:Yielding 2024-06-10T00:33:25.083 INFO:teuthology.run_tasks:Running task workunit... 2024-06-10T00:33:25.097 INFO:tasks.workunit:Pulling workunits from ref 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb 2024-06-10T00:33:25.097 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-10T00:33:25.098 INFO:tasks.workunit:timeout=3h 2024-06-10T00:33:25.098 INFO:tasks.workunit:cleanup=True 2024-06-10T00:33:25.098 DEBUG:teuthology.orchestra.run.smithi062:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-10T00:33:25.099 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:33:25.116 INFO:teuthology.orchestra.run.smithi062.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-10T00:33:25.116 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-06-10T00:33:25.116 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-06-10T00:33:25.116 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:33:25.116 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-06-10T00:33:25.116 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-06-10 00:29:59.556049989 +0000 2024-06-10T00:33:25.117 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-06-10 00:29:59.556049989 +0000 2024-06-10T00:33:25.117 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-06-10 00:32:47.042252386 +0000 2024-06-10T00:33:25.117 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-06-10 00:29:59.556049989 +0000 2024-06-10T00:33:25.117 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-10T00:33:25.117 DEBUG:teuthology.orchestra.run.smithi062:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-10T00:33:25.193 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb 2024-06-10T00:33:25.250 INFO:tasks.workunit.client.0.smithi062.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-10T00:33:25.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:25 smithi062 ceph-mon[32390]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:25.358 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:25.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:25 smithi105 ceph-mon[36321]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:25.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:25 smithi138 ceph-mon[36549]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:27.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.155+0000 7f1539651700 1 -- 172.21.15.62:0/2733460634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f153406b830 msgr2=0x7f1534107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:27.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.155+0000 7f1539651700 1 --2- 172.21.15.62:0/2733460634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f153406b830 0x7f1534107120 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f151c0099c0 tx=0x7f151c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:33:27.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.155+0000 7f1539651700 1 -- 172.21.15.62:0/2733460634 shutdown_connections 2024-06-10T00:33:27.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.155+0000 7f1539651700 1 --2- 172.21.15.62:0/2733460634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1534107850 0x7f153410f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:27.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.155+0000 7f1539651700 1 --2- 172.21.15.62:0/2733460634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f153406ae80 0x7f153406b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:27.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.155+0000 7f1539651700 1 --2- 172.21.15.62:0/2733460634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f153406b830 0x7f1534107120 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:27.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.155+0000 7f1539651700 1 -- 172.21.15.62:0/2733460634 >> 172.21.15.62:0/2733460634 conn(0x7f1534078400 msgr2=0x7f1534078810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:27.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.156+0000 7f1539651700 1 -- 172.21.15.62:0/2733460634 shutdown_connections 2024-06-10T00:33:27.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.156+0000 7f1539651700 1 -- 172.21.15.62:0/2733460634 wait complete. 2024-06-10T00:33:27.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.156+0000 7f1539651700 1 Processor -- start 2024-06-10T00:33:27.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.156+0000 7f1539651700 1 -- start start 2024-06-10T00:33:27.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.156+0000 7f1539651700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f153406ae80 0x7f1534101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:27.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f1539651700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f153406b830 0x7f1534101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:27.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f1539651700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1534107850 0x7f1534101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:27.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f1539651700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1534112850 con 0x7f1534107850 2024-06-10T00:33:27.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f1539651700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f15341126d0 con 0x7f153406b830 2024-06-10T00:33:27.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f1539651700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15341129d0 con 0x7f153406ae80 2024-06-10T00:33:27.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f15337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1534107850 0x7f1534101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:27.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f15337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1534107850 0x7f1534101e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59266/0 (socket says 172.21.15.62:59266) 2024-06-10T00:33:27.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f15337fe700 1 -- 172.21.15.62:0/3231869645 learned_addr learned my addr 172.21.15.62:0/3231869645 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:27.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f1532ffd700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f153406ae80 0x7f1534101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:27.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f15327fc700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f153406b830 0x7f1534101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:27.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f15337fe700 1 -- 172.21.15.62:0/3231869645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f153406ae80 msgr2=0x7f1534101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:27.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f15337fe700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f153406ae80 0x7f1534101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:27.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f15337fe700 1 -- 172.21.15.62:0/3231869645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f153406b830 msgr2=0x7f1534101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:27.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f15337fe700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f153406b830 0x7f1534101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:27.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f15337fe700 1 -- 172.21.15.62:0/3231869645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15341a7180 con 0x7f1534107850 2024-06-10T00:33:27.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f1532ffd700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f153406ae80 0x7f1534101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:27.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f15337fe700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1534107850 0x7f1534101e80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f152c00ef30 tx=0x7f152c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f152bfff700 1 -- 172.21.15.62:0/3231869645 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c0092e0 con 0x7f1534107850 2024-06-10T00:33:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f152bfff700 1 -- 172.21.15.62:0/3231869645 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f152c004bb0 con 0x7f1534107850 2024-06-10T00:33:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f1539651700 1 -- 172.21.15.62:0/3231869645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15341a7410 con 0x7f1534107850 2024-06-10T00:33:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f152bfff700 1 -- 172.21.15.62:0/3231869645 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c004280 con 0x7f1534107850 2024-06-10T00:33:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.157+0000 7f1539651700 1 -- 172.21.15.62:0/3231869645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15341a78d0 con 0x7f1534107850 2024-06-10T00:33:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.158+0000 7f1539651700 1 -- 172.21.15.62:0/3231869645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1534069180 con 0x7f1534107850 2024-06-10T00:33:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.160+0000 7f152bfff700 1 -- 172.21.15.62:0/3231869645 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f152c01b030 con 0x7f1534107850 2024-06-10T00:33:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.160+0000 7f152bfff700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f15200a2e20 0x7f15200a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.160+0000 7f1532ffd700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f15200a2e20 0x7f15200a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.161+0000 7f152bfff700 1 -- 172.21.15.62:0/3231869645 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f152c0cffb0 con 0x7f1534107850 2024-06-10T00:33:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.161+0000 7f1532ffd700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f15200a2e20 0x7f15200a52e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f15240099d0 tx=0x7f1524008040 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.164+0000 7f152bfff700 1 -- 172.21.15.62:0/3231869645 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f152c022020 con 0x7f1534107850 2024-06-10T00:33:27.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.417+0000 7f1539651700 1 -- 172.21.15.62:0/3231869645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f153410aa90 con 0x7f1534107850 2024-06-10T00:33:27.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.418+0000 7f152bfff700 1 -- 172.21.15.62:0/3231869645 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f152c010040 con 0x7f1534107850 2024-06-10T00:33:27.418 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:33:27.418 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:32:47.783154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:33:27.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.420+0000 7f1539651700 1 -- 172.21.15.62:0/3231869645 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f15200a2e20 msgr2=0x7f15200a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:27.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.420+0000 7f1539651700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f15200a2e20 0x7f15200a52e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f15240099d0 tx=0x7f1524008040 comp rx=0 tx=0).stop 2024-06-10T00:33:27.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.420+0000 7f1539651700 1 -- 172.21.15.62:0/3231869645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1534107850 msgr2=0x7f1534101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:27.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.420+0000 7f1539651700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1534107850 0x7f1534101e80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f152c00ef30 tx=0x7f152c00c450 comp rx=0 tx=0).stop 2024-06-10T00:33:27.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.420+0000 7f1539651700 1 -- 172.21.15.62:0/3231869645 shutdown_connections 2024-06-10T00:33:27.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.420+0000 7f1539651700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f15200a2e20 0x7f15200a52e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:27.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.420+0000 7f1539651700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1534107850 0x7f1534101e80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:27.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.420+0000 7f1539651700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f153406ae80 0x7f1534101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:27.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.420+0000 7f1539651700 1 --2- 172.21.15.62:0/3231869645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f153406b830 0x7f1534101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:27.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.420+0000 7f1539651700 1 -- 172.21.15.62:0/3231869645 >> 172.21.15.62:0/3231869645 conn(0x7f1534078400 msgr2=0x7f15340ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:27.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.420+0000 7f1539651700 1 -- 172.21.15.62:0/3231869645 shutdown_connections 2024-06-10T00:33:27.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:27.420+0000 7f1539651700 1 -- 172.21.15.62:0/3231869645 wait complete. 2024-06-10T00:33:27.421 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-06-10T00:33:27.431 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:27 smithi062 ceph-mon[32390]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-06-10T00:33:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:27 smithi105 ceph-mon[36321]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-06-10T00:33:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:27 smithi138 ceph-mon[36549]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-06-10T00:33:28.195 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-10T00:28:39.478894+0000', 'modified': '2024-06-10T00:32:47.783154+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24464, 'mds_1': 24433, 'mds_2': 24431, 'mds_3': 24424, 'mds_4': 24418}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6839/4040406390', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 4040406390}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 4040406390}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6841/2460552589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6840', 'nonce': 2460552589}, {'type': 'v1', 'addr': '172.21.15.62:6841', 'nonce': 2460552589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24377': {'gid': 24377, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6833/1194987741', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6832', 'nonce': 1194987741}, {'type': 'v1', 'addr': '172.21.15.105:6833', 'nonce': 1194987741}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24382': {'gid': 24382, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6833/2673201423', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 2673201423}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 2673201423}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6835/588077925', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 588077925}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 588077925}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'f', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 27, 'addr': '172.21.15.138:6835/331298815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 331298815}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 331298815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.138:6837/3880084398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 3880084398}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 3880084398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.105:6837/4270590303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 4270590303}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 4270590303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.138:6839/339302206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 339302206}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 339302206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.105:6839/1632080154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 1632080154}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 1632080154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24464, 'qdb_cluster': [24464, 24433, 24418, 24431, 24424]} 2024-06-10T00:33:28.195 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-10T00:33:28.195 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-06-10T00:33:28.490 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:28.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:28 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3231869645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:33:28.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:28 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3231869645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:33:28.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:28 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3231869645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:33:29.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:29 smithi105 ceph-mon[36321]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-10T00:33:29.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:29 smithi062 ceph-mon[32390]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-10T00:33:29.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:29 smithi138 ceph-mon[36549]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-10T00:33:30.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.177+0000 7fb739f46700 1 -- 172.21.15.62:0/4263445078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb734107420 msgr2=0x7fb734107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:30.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.177+0000 7fb739f46700 1 --2- 172.21.15.62:0/4263445078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb734107420 0x7fb734107800 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb724009990 tx=0x7fb724009ca0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.177+0000 7fb739f46700 1 -- 172.21.15.62:0/4263445078 shutdown_connections 2024-06-10T00:33:30.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.177+0000 7fb739f46700 1 --2- 172.21.15.62:0/4263445078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb734107420 0x7fb734107800 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.177+0000 7fb739f46700 1 --2- 172.21.15.62:0/4263445078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb734101d30 0x7fb73410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.177+0000 7fb739f46700 1 --2- 172.21.15.62:0/4263445078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb734101370 0x7fb7341017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.177+0000 7fb739f46700 1 -- 172.21.15.62:0/4263445078 >> 172.21.15.62:0/4263445078 conn(0x7fb734078340 msgr2=0x7fb734078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:30.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.177+0000 7fb739f46700 1 -- 172.21.15.62:0/4263445078 shutdown_connections 2024-06-10T00:33:30.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.177+0000 7fb739f46700 1 -- 172.21.15.62:0/4263445078 wait complete. 2024-06-10T00:33:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.177+0000 7fb739f46700 1 Processor -- start 2024-06-10T00:33:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb739f46700 1 -- start start 2024-06-10T00:33:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb739f46700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb734101370 0x7fb73419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb739f46700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb734101d30 0x7fb73419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb739f46700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb734107420 0x7fb734197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb739f46700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb734111630 con 0x7fb734101370 2024-06-10T00:33:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb739f46700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb7341114b0 con 0x7fb734101d30 2024-06-10T00:33:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb739f46700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7341117b0 con 0x7fb734107420 2024-06-10T00:33:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb7337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb734101370 0x7fb73419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb7337fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb734101370 0x7fb73419d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55122/0 (socket says 172.21.15.62:55122) 2024-06-10T00:33:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb7337fe700 1 -- 172.21.15.62:0/3197471594 learned_addr learned my addr 172.21.15.62:0/3197471594 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb72bfff700 1 --2- 172.21.15.62:0/3197471594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb734101d30 0x7fb73419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb733fff700 1 --2- 172.21.15.62:0/3197471594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb734107420 0x7fb734197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb7337fe700 1 -- 172.21.15.62:0/3197471594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb734107420 msgr2=0x7fb734197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb7337fe700 1 --2- 172.21.15.62:0/3197471594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb734107420 0x7fb734197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb7337fe700 1 -- 172.21.15.62:0/3197471594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb734101d30 msgr2=0x7fb73419db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb7337fe700 1 --2- 172.21.15.62:0/3197471594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb734101d30 0x7fb73419db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.178+0000 7fb7337fe700 1 -- 172.21.15.62:0/3197471594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb734197f70 con 0x7fb734101370 2024-06-10T00:33:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.179+0000 7fb7337fe700 1 --2- 172.21.15.62:0/3197471594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb734101370 0x7fb73419d610 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb724000c00 tx=0x7fb724003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.179+0000 7fb7317fa700 1 -- 172.21.15.62:0/3197471594 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724029070 con 0x7fb734101370 2024-06-10T00:33:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.179+0000 7fb7317fa700 1 -- 172.21.15.62:0/3197471594 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb724019cf0 con 0x7fb734101370 2024-06-10T00:33:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.179+0000 7fb739f46700 1 -- 172.21.15.62:0/3197471594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb734198200 con 0x7fb734101370 2024-06-10T00:33:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.179+0000 7fb7317fa700 1 -- 172.21.15.62:0/3197471594 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724024470 con 0x7fb734101370 2024-06-10T00:33:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.179+0000 7fb739f46700 1 -- 172.21.15.62:0/3197471594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb73406abc0 con 0x7fb734101370 2024-06-10T00:33:30.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.181+0000 7fb7317fa700 1 -- 172.21.15.62:0/3197471594 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fb72402c070 con 0x7fb734101370 2024-06-10T00:33:30.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.182+0000 7fb7317fa700 1 --2- 172.21.15.62:0/3197471594 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb7200a2ea0 0x7fb7200a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.182+0000 7fb72bfff700 1 --2- 172.21.15.62:0/3197471594 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb7200a2ea0 0x7fb7200a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.182+0000 7fb7317fa700 1 -- 172.21.15.62:0/3197471594 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fb7240d29e0 con 0x7fb734101370 2024-06-10T00:33:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.183+0000 7fb72bfff700 1 --2- 172.21.15.62:0/3197471594 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb7200a2ea0 0x7fb7200a5360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb7340789b0 tx=0x7fb71c00a380 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.183+0000 7fb7447c9b80 1 Processor -- start 2024-06-10T00:33:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.183+0000 7fb7447c9b80 1 -- start start 2024-06-10T00:33:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.183+0000 7fb7447c9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2faff80 0x556dc2fb0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.183+0000 7fb7447c9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc2fb4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.183+0000 7fb7447c9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2fb4bb0 0x556dc2fb5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.183+0000 7fb7447c9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556dc2efd7c0 con 0x556dc2fb08a0 2024-06-10T00:33:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.183+0000 7fb7447c9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556dc2ea56d0 con 0x556dc2fb4bb0 2024-06-10T00:33:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.183+0000 7fb7447c9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556dc2ef9780 con 0x556dc2faff80 2024-06-10T00:33:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.183+0000 7fb72bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc2fb4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.183+0000 7fb72bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc2fb4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55124/0 (socket says 172.21.15.62:55124) 2024-06-10T00:33:30.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.183+0000 7fb72bfff700 1 -- 172.21.15.62:0/3729014682 learned_addr learned my addr 172.21.15.62:0/3729014682 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:30.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.183+0000 7fb7337fe700 1 --2- 172.21.15.62:0/3729014682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2faff80 0x556dc2fb0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.183+0000 7fb733fff700 1 --2- 172.21.15.62:0/3729014682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2fb4bb0 0x556dc2fb5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.184+0000 7fb72bfff700 1 -- 172.21.15.62:0/3729014682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2faff80 msgr2=0x556dc2fb0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:30.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.184+0000 7fb72bfff700 1 --2- 172.21.15.62:0/3729014682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2faff80 0x556dc2fb0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.184+0000 7fb72bfff700 1 -- 172.21.15.62:0/3729014682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2fb4bb0 msgr2=0x556dc2fb5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:30.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.184+0000 7fb72bfff700 1 --2- 172.21.15.62:0/3729014682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2fb4bb0 0x556dc2fb5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.184+0000 7fb72bfff700 1 -- 172.21.15.62:0/3729014682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dc2e9bf00 con 0x556dc2fb08a0 2024-06-10T00:33:30.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.184+0000 7fb7337fe700 1 --2- 172.21.15.62:0/3729014682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2faff80 0x556dc2fb0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:30.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.184+0000 7fb72bfff700 1 --2- 172.21.15.62:0/3729014682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc2fb4670 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00fea0 tx=0x7fb71c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c47d8f950cec00f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:30.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.184+0000 7fb729ffb700 1 -- 172.21.15.62:0/3729014682 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c016070 con 0x556dc2fb08a0 2024-06-10T00:33:30.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.184+0000 7fb729ffb700 1 -- 172.21.15.62:0/3729014682 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb71c014070 con 0x556dc2fb08a0 2024-06-10T00:33:30.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.184+0000 7fb729ffb700 1 -- 172.21.15.62:0/3729014682 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c010e10 con 0x556dc2fb08a0 2024-06-10T00:33:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.185+0000 7fb7447c9b80 1 -- 172.21.15.62:0/3729014682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 msgr2=0x556dc2fb4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.185+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/3729014682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc2fb4670 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00fea0 tx=0x7fb71c012000 comp rx=0 tx=0).stop 2024-06-10T00:33:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.185+0000 7fb7447c9b80 1 -- 172.21.15.62:0/3729014682 shutdown_connections 2024-06-10T00:33:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.185+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/3729014682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2fb4bb0 0x556dc2fb5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.185+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/3729014682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2faff80 0x556dc2fb0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.185+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/3729014682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc2fb4670 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.185+0000 7fb7447c9b80 1 -- 172.21.15.62:0/3729014682 >> 172.21.15.62:0/3729014682 conn(0x556dc2ed6c70 msgr2=0x556dc2fa6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.185+0000 7fb7447c9b80 1 -- 172.21.15.62:0/3729014682 shutdown_connections 2024-06-10T00:33:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.185+0000 7fb7447c9b80 1 -- 172.21.15.62:0/3729014682 wait complete. 2024-06-10T00:33:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.185+0000 7fb7447c9b80 1 -- start start 2024-06-10T00:33:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb7447c9b80 1 Processor -- start 2024-06-10T00:33:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb7447c9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 0x556dc2fbc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb7447c9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc2fb9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb7447c9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb4bb0 0x556dc2fb9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb7447c9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556dc2fbcd70 con 0x556dc2fb08a0 2024-06-10T00:33:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb7447c9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556dc2e9dba0 con 0x556dc2faff80 2024-06-10T00:33:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb7447c9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556dc2fbcef0 con 0x556dc2fb4bb0 2024-06-10T00:33:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb7447c9b80 10 client.0 _ll_register_callbacks cb 0x556dc2e978d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb72bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc2fb9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb72bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc2fb9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55126/0 (socket says 172.21.15.62:55126) 2024-06-10T00:33:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb72bfff700 1 -- 172.21.15.62:0/1994023477 learned_addr learned my addr 172.21.15.62:0/1994023477 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:30.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb7337fe700 1 --2- 172.21.15.62:0/1994023477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 0x556dc2fbc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb733fff700 1 --2- 172.21.15.62:0/1994023477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb4bb0 0x556dc2fb9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb733fff700 1 -- 172.21.15.62:0/1994023477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 msgr2=0x556dc2fbc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:30.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb733fff700 1 --2- 172.21.15.62:0/1994023477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 0x556dc2fbc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb733fff700 1 -- 172.21.15.62:0/1994023477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 msgr2=0x556dc2fb9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:30.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb733fff700 1 --2- 172.21.15.62:0/1994023477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc2fb9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb733fff700 1 -- 172.21.15.62:0/1994023477 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dc2e4c880 con 0x556dc2fb4bb0 2024-06-10T00:33:30.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.186+0000 7fb72bfff700 1 --2- 172.21.15.62:0/1994023477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc2fb9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:30.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.187+0000 7fb733fff700 1 --2- 172.21.15.62:0/1994023477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb4bb0 0x556dc2fb9990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb72c012da0 tx=0x7fb72c010520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdddcc9401cfbf3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:30.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.187+0000 7fb712ffd700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:33:30.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.187+0000 7fb712ffd700 1 -- 172.21.15.62:0/1994023477 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c01a070 con 0x556dc2fb4bb0 2024-06-10T00:33:30.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.187+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1994023477 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb71c014070 con 0x556dc2fb4bb0 2024-06-10T00:33:30.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.187+0000 7fb712ffd700 1 -- 172.21.15.62:0/1994023477 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fb72c004bb0 con 0x556dc2fb4bb0 2024-06-10T00:33:30.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.187+0000 7fb712ffd700 1 -- 172.21.15.62:0/1994023477 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb72c023050 con 0x556dc2fb4bb0 2024-06-10T00:33:30.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.187+0000 7fb7447c9b80 10 client.25177 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:30.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.187+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1994023477 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556dc2fc77b0 con 0x556dc2fb4bb0 2024-06-10T00:33:30.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.188+0000 7fb712ffd700 1 -- 172.21.15.62:0/1994023477 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb71c014070 con 0x556dc2fb4bb0 2024-06-10T00:33:30.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.188+0000 7fb712ffd700 1 -- 172.21.15.62:0/1994023477 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb72c026020 con 0x556dc2fb4bb0 2024-06-10T00:33:30.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.188+0000 7fb7447c9b80 10 client.25177 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:30.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.188+0000 7fb7447c9b80 10 client.25177 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-10T00:33:30.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.188+0000 7fb7447c9b80 20 client.25177 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:30.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.188+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/1994023477 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x556dc2fc9e40 0x556dc2fcb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.188+0000 7fb7447c9b80 4 client.25177 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:33:30.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.188+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1994023477 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dc2f0acc0 con 0x556dc2fc9e40 2024-06-10T00:33:30.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.188+0000 7fb7337fe700 1 --2- 172.21.15.62:0/1994023477 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x556dc2fc9e40 0x556dc2fcb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.189+0000 7fb7337fe700 1 --2- 172.21.15.62:0/1994023477 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x556dc2fc9e40 0x556dc2fcb2b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c3f70da4117e44b server_cookie=b3fb648e2b2280da in_seq=0 out_seq=0 2024-06-10T00:33:30.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.189+0000 7fb712ffd700 10 client.25177 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:33:30.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.189+0000 7fb712ffd700 1 -- 172.21.15.62:0/1994023477 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556dc2f0acc0 con 0x556dc2fc9e40 2024-06-10T00:33:30.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.190+0000 7fb712ffd700 10 client.25177 handle_command_reply: tid=0 2024-06-10T00:33:30.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.190+0000 7fb7447c9b80 1 client.25177 shutdown 2024-06-10T00:33:30.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.190+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1994023477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb4bb0 msgr2=0x556dc2fb9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:30.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.190+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/1994023477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb4bb0 0x556dc2fb9990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb72c012da0 tx=0x7fb72c010520 comp rx=0 tx=0).stop 2024-06-10T00:33:30.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.190+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1994023477 shutdown_connections 2024-06-10T00:33:30.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.190+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/1994023477 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x556dc2fc9e40 0x556dc2fcb2b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.190+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/1994023477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb4bb0 0x556dc2fb9990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.190+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/1994023477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 0x556dc2fbc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.190+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/1994023477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc2fb9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.190+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1994023477 >> 172.21.15.62:0/1994023477 conn(0x556dc2ed6c70 msgr2=0x556dc2faa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:30.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.190+0000 7fb712ffd700 0 client.25177 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:33:30.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.190+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1994023477 shutdown_connections 2024-06-10T00:33:30.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.190+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1994023477 wait complete. 2024-06-10T00:33:30.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.190+0000 7fb7447c9b80 20 client.25177 trim_cache size 0 max 16384 2024-06-10T00:33:30.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.211+0000 7fb7447c9b80 1 Processor -- start 2024-06-10T00:33:30.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb7447c9b80 1 -- start start 2024-06-10T00:33:30.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb7447c9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 0x556dc2fb6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb7447c9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb08a0 0x556dc2fb6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb7447c9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb4bb0 0x556dc2f9fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb7447c9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556dc2fc95f0 con 0x556dc2fb4bb0 2024-06-10T00:33:30.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb7447c9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556dc2fc9cc0 con 0x556dc2faff80 2024-06-10T00:33:30.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb7447c9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556dc2fca390 con 0x556dc2fb08a0 2024-06-10T00:33:30.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb733fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb4bb0 0x556dc2f9fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb733fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb4bb0 0x556dc2f9fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55138/0 (socket says 172.21.15.62:55138) 2024-06-10T00:33:30.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb733fff700 1 -- 172.21.15.62:0/237863194 learned_addr learned my addr 172.21.15.62:0/237863194 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:30.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb72bfff700 1 --2- 172.21.15.62:0/237863194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb08a0 0x556dc2fb6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb7337fe700 1 --2- 172.21.15.62:0/237863194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 0x556dc2fb6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb733fff700 1 -- 172.21.15.62:0/237863194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb08a0 msgr2=0x556dc2fb6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:30.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb733fff700 1 --2- 172.21.15.62:0/237863194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb08a0 0x556dc2fb6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb733fff700 1 -- 172.21.15.62:0/237863194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 msgr2=0x556dc2fb6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:30.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb733fff700 1 --2- 172.21.15.62:0/237863194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 0x556dc2fb6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.212+0000 7fb733fff700 1 -- 172.21.15.62:0/237863194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dc2fbc780 con 0x556dc2fb4bb0 2024-06-10T00:33:30.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.213+0000 7fb733fff700 1 --2- 172.21.15.62:0/237863194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb4bb0 0x556dc2f9fdb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00a040 tx=0x7fb72c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e24d1516dfd465 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:30.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.213+0000 7fb7297fa700 1 -- 172.21.15.62:0/237863194 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c02b020 con 0x556dc2fb4bb0 2024-06-10T00:33:30.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.213+0000 7fb7297fa700 1 -- 172.21.15.62:0/237863194 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb72c010c80 con 0x556dc2fb4bb0 2024-06-10T00:33:30.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.213+0000 7fb7297fa700 1 -- 172.21.15.62:0/237863194 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c01a050 con 0x556dc2fb4bb0 2024-06-10T00:33:30.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.213+0000 7fb7447c9b80 1 -- 172.21.15.62:0/237863194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb4bb0 msgr2=0x556dc2f9fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:30.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.213+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/237863194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb4bb0 0x556dc2f9fdb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00a040 tx=0x7fb72c005e70 comp rx=0 tx=0).stop 2024-06-10T00:33:30.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.213+0000 7fb7447c9b80 1 -- 172.21.15.62:0/237863194 shutdown_connections 2024-06-10T00:33:30.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.213+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/237863194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb4bb0 0x556dc2f9fdb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.213+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/237863194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 0x556dc2fb6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.213+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/237863194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb08a0 0x556dc2fb6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.213+0000 7fb7447c9b80 1 -- 172.21.15.62:0/237863194 >> 172.21.15.62:0/237863194 conn(0x556dc2ed6c70 msgr2=0x556dc2fab190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.214+0000 7fb7447c9b80 1 -- 172.21.15.62:0/237863194 shutdown_connections 2024-06-10T00:33:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.214+0000 7fb7447c9b80 1 -- 172.21.15.62:0/237863194 wait complete. 2024-06-10T00:33:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.214+0000 7fb7447c9b80 1 -- start start 2024-06-10T00:33:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.214+0000 7fb7447c9b80 1 Processor -- start 2024-06-10T00:33:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.214+0000 7fb7447c9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 0x556dc302d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.214+0000 7fb7447c9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc302ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.214+0000 7fb7447c9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb4bb0 0x556dc2fd4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.214+0000 7fb7447c9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556dc2fcb130 con 0x556dc2fb08a0 2024-06-10T00:33:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.214+0000 7fb7447c9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556dc2fcaa60 con 0x556dc2faff80 2024-06-10T00:33:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.214+0000 7fb7447c9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556dc2fcb800 con 0x556dc2fb4bb0 2024-06-10T00:33:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb7447c9b80 10 client.0 _ll_register_callbacks cb 0x556dc2e978d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:30.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb72bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc302ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb72bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc302ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55140/0 (socket says 172.21.15.62:55140) 2024-06-10T00:33:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb7337fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 0x556dc302d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb72bfff700 1 -- 172.21.15.62:0/1685036950 learned_addr learned my addr 172.21.15.62:0/1685036950 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb733fff700 1 --2- 172.21.15.62:0/1685036950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb4bb0 0x556dc2fd4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb72bfff700 1 -- 172.21.15.62:0/1685036950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb4bb0 msgr2=0x556dc2fd4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb72bfff700 1 --2- 172.21.15.62:0/1685036950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb4bb0 0x556dc2fd4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb72bfff700 1 -- 172.21.15.62:0/1685036950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 msgr2=0x556dc302d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb72bfff700 1 --2- 172.21.15.62:0/1685036950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 0x556dc302d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb72bfff700 1 -- 172.21.15.62:0/1685036950 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dc2fd4a60 con 0x556dc2fb08a0 2024-06-10T00:33:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb7337fe700 1 --2- 172.21.15.62:0/1685036950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 0x556dc302d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:30.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb72bfff700 1 --2- 172.21.15.62:0/1685036950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc302ddb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00fe40 tx=0x7fb71c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a38ab8c9441ed1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb712ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:33:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb712ffd700 1 -- 172.21.15.62:0/1685036950 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c035060 con 0x556dc2fb08a0 2024-06-10T00:33:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1685036950 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb72c010c80 con 0x556dc2fb08a0 2024-06-10T00:33:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.215+0000 7fb712ffd700 1 -- 172.21.15.62:0/1685036950 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fb71c003890 con 0x556dc2fb08a0 2024-06-10T00:33:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.216+0000 7fb712ffd700 1 -- 172.21.15.62:0/1685036950 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb71c03c050 con 0x556dc2fb08a0 2024-06-10T00:33:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.216+0000 7fb7447c9b80 10 client.15588 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.216+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1685036950 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556dc3035100 con 0x556dc2fb08a0 2024-06-10T00:33:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.216+0000 7fb712ffd700 1 -- 172.21.15.62:0/1685036950 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb72c010c80 con 0x556dc2fb08a0 2024-06-10T00:33:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.216+0000 7fb712ffd700 1 -- 172.21.15.62:0/1685036950 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb71c01d400 con 0x556dc2fb08a0 2024-06-10T00:33:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.216+0000 7fb7447c9b80 10 client.15588 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.216+0000 7fb7447c9b80 10 client.15588 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-10T00:33:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.216+0000 7fb7447c9b80 20 client.15588 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:30.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.216+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/1685036950 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x556dc3039920 0x556dc3038630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:30.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.216+0000 7fb7447c9b80 4 client.15588 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:33:30.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.216+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1685036950 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "status"}) -- 0x556dc2f05240 con 0x556dc3039920 2024-06-10T00:33:30.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb7337fe700 1 --2- 172.21.15.62:0/1685036950 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x556dc3039920 0x556dc3038630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:30.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb712ffd700 10 client.15588 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:33:30.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb7337fe700 1 --2- 172.21.15.62:0/1685036950 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x556dc3039920 0x556dc3038630 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f3a48d07a1bf66b8 server_cookie=e407279e135b09b5 in_seq=0 out_seq=0 2024-06-10T00:33:30.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb712ffd700 1 -- 172.21.15.62:0/1685036950 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x556dc2f05240 con 0x556dc3039920 2024-06-10T00:33:30.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb712ffd700 10 client.15588 handle_command_reply: tid=0 2024-06-10T00:33:30.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb7447c9b80 1 client.15588 shutdown 2024-06-10T00:33:30.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1685036950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 msgr2=0x556dc302ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:30.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/1685036950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc302ddb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00fe40 tx=0x7fb71c005720 comp rx=0 tx=0).stop 2024-06-10T00:33:30.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1685036950 shutdown_connections 2024-06-10T00:33:30.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/1685036950 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x556dc3039920 0x556dc3038630 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/1685036950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556dc2fb4bb0 0x556dc2fd4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/1685036950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556dc2faff80 0x556dc302d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb7447c9b80 1 --2- 172.21.15.62:0/1685036950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556dc2fb08a0 0x556dc302ddb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb712ffd700 0 client.15588 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:33:30.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.217+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1685036950 >> 172.21.15.62:0/1685036950 conn(0x556dc2ed6c70 msgr2=0x556dc2fb6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:30.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.218+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1685036950 shutdown_connections 2024-06-10T00:33:30.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.218+0000 7fb7447c9b80 1 -- 172.21.15.62:0/1685036950 wait complete. 2024-06-10T00:33:30.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.218+0000 7fb7447c9b80 20 client.15588 trim_cache size 0 max 16384 2024-06-10T00:33:30.225 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:33:30.225 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:33:30.225 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 4, 2024-06-10T00:33:30.226 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24418, 2024-06-10T00:33:30.226 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:33:30.226 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:33:30.226 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:33:30.226 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 225.34484338199999, 2024-06-10T00:33:30.226 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 30, 2024-06-10T00:33:30.226 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 100, 2024-06-10T00:33:30.226 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 99, 2024-06-10T00:33:30.226 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 327.80271470000002 2024-06-10T00:33:30.226 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:33:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.218+0000 7fb739f46700 1 -- 172.21.15.62:0/3197471594 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb7200a2ea0 msgr2=0x7fb7200a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.218+0000 7fb739f46700 1 --2- 172.21.15.62:0/3197471594 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb7200a2ea0 0x7fb7200a5360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb7340789b0 tx=0x7fb71c00a380 comp rx=0 tx=0).stop 2024-06-10T00:33:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.218+0000 7fb739f46700 1 -- 172.21.15.62:0/3197471594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb734101370 msgr2=0x7fb73419d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.218+0000 7fb739f46700 1 --2- 172.21.15.62:0/3197471594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb734101370 0x7fb73419d610 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb724000c00 tx=0x7fb724003680 comp rx=0 tx=0).stop 2024-06-10T00:33:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.218+0000 7fb739f46700 1 -- 172.21.15.62:0/3197471594 shutdown_connections 2024-06-10T00:33:30.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.218+0000 7fb739f46700 1 --2- 172.21.15.62:0/3197471594 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb7200a2ea0 0x7fb7200a5360 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.218+0000 7fb739f46700 1 --2- 172.21.15.62:0/3197471594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb734101370 0x7fb73419d610 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.218+0000 7fb739f46700 1 --2- 172.21.15.62:0/3197471594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb734107420 0x7fb734197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.218+0000 7fb739f46700 1 --2- 172.21.15.62:0/3197471594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb734101d30 0x7fb73419db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.218+0000 7fb739f46700 1 -- 172.21.15.62:0/3197471594 >> 172.21.15.62:0/3197471594 conn(0x7fb734078340 msgr2=0x7fb734100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.219+0000 7fb739f46700 1 -- 172.21.15.62:0/3197471594 shutdown_connections 2024-06-10T00:33:30.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:30.219+0000 7fb739f46700 1 -- 172.21.15.62:0/3197471594 wait complete. 2024-06-10T00:33:30.951 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-10T00:33:31.206 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:31.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:31 smithi105 ceph-mon[36321]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-10T00:33:31.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:31 smithi062 ceph-mon[32390]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-10T00:33:31.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:31 smithi138 ceph-mon[36549]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-10T00:33:33.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.003+0000 7f32c93a6700 1 -- 172.21.15.62:0/941864689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c406b3c0 msgr2=0x7f32c410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.003+0000 7f32c93a6700 1 --2- 172.21.15.62:0/941864689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c406b3c0 0x7f32c410f310 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009990 tx=0x7f32bc009ca0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.003+0000 7f32c93a6700 1 -- 172.21.15.62:0/941864689 shutdown_connections 2024-06-10T00:33:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.003+0000 7f32c93a6700 1 --2- 172.21.15.62:0/941864689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c406b3c0 0x7f32c410f310 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.003+0000 7f32c93a6700 1 --2- 172.21.15.62:0/941864689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c41084f0 0x7f32c41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.003+0000 7f32c93a6700 1 --2- 172.21.15.62:0/941864689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32c406aa00 0x7f32c406ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.003+0000 7f32c93a6700 1 -- 172.21.15.62:0/941864689 >> 172.21.15.62:0/941864689 conn(0x7f32c4078300 msgr2=0x7f32c4078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.003+0000 7f32c93a6700 1 -- 172.21.15.62:0/941864689 shutdown_connections 2024-06-10T00:33:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.003+0000 7f32c93a6700 1 -- 172.21.15.62:0/941864689 wait complete. 2024-06-10T00:33:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.004+0000 7f32c93a6700 1 Processor -- start 2024-06-10T00:33:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.004+0000 7f32c93a6700 1 -- start start 2024-06-10T00:33:33.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.004+0000 7f32c93a6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c406aa00 0x7f32c419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.004+0000 7f32c93a6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32c406b3c0 0x7f32c419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.004+0000 7f32c93a6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c41084f0 0x7f32c4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.004+0000 7f32c93a6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f32c4112750 con 0x7f32c41084f0 2024-06-10T00:33:33.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.004+0000 7f32c93a6700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32c41125d0 con 0x7f32c406aa00 2024-06-10T00:33:33.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.004+0000 7f32c93a6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32c41128d0 con 0x7f32c406b3c0 2024-06-10T00:33:33.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.004+0000 7f32c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c41084f0 0x7f32c4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.004+0000 7f32c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c41084f0 0x7f32c4198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55166/0 (socket says 172.21.15.62:55166) 2024-06-10T00:33:33.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.004+0000 7f32c37fe700 1 -- 172.21.15.62:0/1966795677 learned_addr learned my addr 172.21.15.62:0/1966795677 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:33.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.004+0000 7f32c27fc700 1 --2- 172.21.15.62:0/1966795677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32c406b3c0 0x7f32c419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.004+0000 7f32c2ffd700 1 --2- 172.21.15.62:0/1966795677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c406aa00 0x7f32c419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.005+0000 7f32c37fe700 1 -- 172.21.15.62:0/1966795677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32c406b3c0 msgr2=0x7f32c419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:33.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.005+0000 7f32c37fe700 1 --2- 172.21.15.62:0/1966795677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32c406b3c0 0x7f32c419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.005+0000 7f32c37fe700 1 -- 172.21.15.62:0/1966795677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c406aa00 msgr2=0x7f32c419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.005+0000 7f32c37fe700 1 --2- 172.21.15.62:0/1966795677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c406aa00 0x7f32c419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.005+0000 7f32c37fe700 1 -- 172.21.15.62:0/1966795677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c4199170 con 0x7f32c41084f0 2024-06-10T00:33:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.005+0000 7f32c37fe700 1 --2- 172.21.15.62:0/1966795677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c41084f0 0x7f32c4198a10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0038b0 tx=0x7f32bc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.005+0000 7f32bbfff700 1 -- 172.21.15.62:0/1966795677 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc027070 con 0x7f32c41084f0 2024-06-10T00:33:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.005+0000 7f32c93a6700 1 -- 172.21.15.62:0/1966795677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c4199400 con 0x7f32c41084f0 2024-06-10T00:33:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.005+0000 7f32bbfff700 1 -- 172.21.15.62:0/1966795677 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc003e40 con 0x7f32c41084f0 2024-06-10T00:33:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.005+0000 7f32bbfff700 1 -- 172.21.15.62:0/1966795677 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc022cc0 con 0x7f32c41084f0 2024-06-10T00:33:33.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.005+0000 7f32c93a6700 1 -- 172.21.15.62:0/1966795677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c41a56d0 con 0x7f32c41084f0 2024-06-10T00:33:33.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.008+0000 7f32bbfff700 1 -- 172.21.15.62:0/1966795677 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f32bc02a070 con 0x7f32c41084f0 2024-06-10T00:33:33.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.008+0000 7f32bbfff700 1 --2- 172.21.15.62:0/1966795677 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32b00a2ea0 0x7f32b00a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.008+0000 7f32c2ffd700 1 --2- 172.21.15.62:0/1966795677 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32b00a2ea0 0x7f32b00a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.008+0000 7f32bbfff700 1 -- 172.21.15.62:0/1966795677 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f32bc0d0570 con 0x7f32c41084f0 2024-06-10T00:33:33.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.009+0000 7f32c2ffd700 1 --2- 172.21.15.62:0/1966795677 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32b00a2ea0 0x7f32b00a5360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009550 tx=0x7f32b40091b0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:33.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.009+0000 7f32d3c29b80 1 Processor -- start 2024-06-10T00:33:33.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.009+0000 7f32d3c29b80 1 -- start start 2024-06-10T00:33:33.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32d3c29b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830a9f80 0x5622830aa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32d3c29b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830aa8a0 0x5622830ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32d3c29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32d3c29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562282ff77c0 con 0x5622830aebe0 2024-06-10T00:33:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32d3c29b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562282f9f6d0 con 0x5622830aa8a0 2024-06-10T00:33:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32d3c29b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562282ff3780 con 0x5622830a9f80 2024-06-10T00:33:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32c2ffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830a9f80 0x5622830aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830af040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55178/0 (socket says 172.21.15.62:55178) 2024-06-10T00:33:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32c37fe700 1 -- 172.21.15.62:0/471110636 learned_addr learned my addr 172.21.15.62:0/471110636 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32c27fc700 1 --2- 172.21.15.62:0/471110636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830aa8a0 0x5622830ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32c37fe700 1 -- 172.21.15.62:0/471110636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830a9f80 msgr2=0x5622830aa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32c37fe700 1 --2- 172.21.15.62:0/471110636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830a9f80 0x5622830aa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32c37fe700 1 -- 172.21.15.62:0/471110636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830aa8a0 msgr2=0x5622830ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32c37fe700 1 --2- 172.21.15.62:0/471110636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830aa8a0 0x5622830ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32c37fe700 1 -- 172.21.15.62:0/471110636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562282f95f00 con 0x5622830aebe0 2024-06-10T00:33:33.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32c27fc700 1 --2- 172.21.15.62:0/471110636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830aa8a0 0x5622830ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32c2ffd700 1 --2- 172.21.15.62:0/471110636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830a9f80 0x5622830aa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.010+0000 7f32c37fe700 1 --2- 172.21.15.62:0/471110636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830af040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f32bc09d2f0 tx=0x7f32bc009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d0c5287140805c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.011+0000 7f32b97fa700 1 -- 172.21.15.62:0/471110636 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc022e60 con 0x5622830aebe0 2024-06-10T00:33:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.011+0000 7f32b97fa700 1 -- 172.21.15.62:0/471110636 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc022470 con 0x5622830aebe0 2024-06-10T00:33:33.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.011+0000 7f32b97fa700 1 -- 172.21.15.62:0/471110636 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc0a61d0 con 0x5622830aebe0 2024-06-10T00:33:33.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.011+0000 7f32d3c29b80 1 -- 172.21.15.62:0/471110636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 msgr2=0x5622830af040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:33.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.011+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/471110636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830af040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f32bc09d2f0 tx=0x7f32bc009710 comp rx=0 tx=0).stop 2024-06-10T00:33:33.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.011+0000 7f32d3c29b80 1 -- 172.21.15.62:0/471110636 shutdown_connections 2024-06-10T00:33:33.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.011+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/471110636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830af040 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.011+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/471110636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830a9f80 0x5622830aa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.011+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/471110636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830aa8a0 0x5622830ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.012+0000 7f32d3c29b80 1 -- 172.21.15.62:0/471110636 >> 172.21.15.62:0/471110636 conn(0x562282fd0c70 msgr2=0x5622830a0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:33.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.012+0000 7f32d3c29b80 1 -- 172.21.15.62:0/471110636 shutdown_connections 2024-06-10T00:33:33.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.012+0000 7f32d3c29b80 1 -- 172.21.15.62:0/471110636 wait complete. 2024-06-10T00:33:33.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.012+0000 7f32d3c29b80 1 -- start start 2024-06-10T00:33:33.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.012+0000 7f32d3c29b80 1 Processor -- start 2024-06-10T00:33:33.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.012+0000 7f32d3c29b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 0x5622830b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.012+0000 7f32d3c29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aa8a0 0x5622830b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.012+0000 7f32d3c29b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aebe0 0x5622830b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.012+0000 7f32d3c29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5622830b6aa0 con 0x5622830aa8a0 2024-06-10T00:33:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.012+0000 7f32d3c29b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562282f97ba0 con 0x5622830a9f80 2024-06-10T00:33:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.012+0000 7f32d3c29b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5622830b6c20 con 0x5622830aebe0 2024-06-10T00:33:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.012+0000 7f32d3c29b80 10 client.0 _ll_register_callbacks cb 0x562282f918d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.012+0000 7f32c27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aa8a0 0x5622830b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.012+0000 7f32c27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aa8a0 0x5622830b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55182/0 (socket says 172.21.15.62:55182) 2024-06-10T00:33:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32c27fc700 1 -- 172.21.15.62:0/2370476466 learned_addr learned my addr 172.21.15.62:0/2370476466 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32c2ffd700 1 --2- 172.21.15.62:0/2370476466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 0x5622830b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32c37fe700 1 --2- 172.21.15.62:0/2370476466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aebe0 0x5622830b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32c27fc700 1 -- 172.21.15.62:0/2370476466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aebe0 msgr2=0x5622830b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:33.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32c27fc700 1 --2- 172.21.15.62:0/2370476466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aebe0 0x5622830b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32c27fc700 1 -- 172.21.15.62:0/2370476466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 msgr2=0x5622830b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32c27fc700 1 --2- 172.21.15.62:0/2370476466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 0x5622830b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32c27fc700 1 -- 172.21.15.62:0/2370476466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562282f46880 con 0x5622830aa8a0 2024-06-10T00:33:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32c27fc700 1 --2- 172.21.15.62:0/2370476466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aa8a0 0x5622830b34c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f32ac012af0 tx=0x7f32ac012e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfc1476df5a4290 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32aa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:33:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32aa7fc700 1 -- 172.21.15.62:0/2370476466 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac010cd0 con 0x5622830aa8a0 2024-06-10T00:33:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32aa7fc700 1 -- 172.21.15.62:0/2370476466 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f32ac004030 con 0x5622830aa8a0 2024-06-10T00:33:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32d3c29b80 1 -- 172.21.15.62:0/2370476466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562283004cc0 con 0x5622830aa8a0 2024-06-10T00:33:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32aa7fc700 1 -- 172.21.15.62:0/2370476466 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f32ac00ae50 con 0x5622830aa8a0 2024-06-10T00:33:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32d3c29b80 10 client.15600 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.013+0000 7f32d3c29b80 1 -- 172.21.15.62:0/2370476466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5622830c14e0 con 0x5622830aa8a0 2024-06-10T00:33:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.014+0000 7f32aa7fc700 1 -- 172.21.15.62:0/2370476466 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562283004cc0 con 0x5622830aa8a0 2024-06-10T00:33:33.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.014+0000 7f32aa7fc700 1 -- 172.21.15.62:0/2370476466 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f32ac01a430 con 0x5622830aa8a0 2024-06-10T00:33:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.014+0000 7f32d3c29b80 10 client.15600 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.014+0000 7f32d3c29b80 10 client.15600 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T00:33:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.014+0000 7f32d3c29b80 20 client.15600 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.014+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/2370476466 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5622830c3b70 0x5622830c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.014+0000 7f32d3c29b80 4 client.15600 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:33:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.014+0000 7f32d3c29b80 1 -- 172.21.15.62:0/2370476466 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562282fff240 con 0x5622830c3b70 2024-06-10T00:33:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.014+0000 7f32c2ffd700 1 --2- 172.21.15.62:0/2370476466 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5622830c3b70 0x5622830c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.014+0000 7f32c2ffd700 1 --2- 172.21.15.62:0/2370476466 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5622830c3b70 0x5622830c4fe0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=17e6e1060de377e7 server_cookie=20c3fc5dc2cc357 in_seq=0 out_seq=0 2024-06-10T00:33:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.014+0000 7f32aa7fc700 10 client.15600 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:33:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.015+0000 7f32aa7fc700 1 -- 172.21.15.62:0/2370476466 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562282fff240 con 0x5622830c3b70 2024-06-10T00:33:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.015+0000 7f32aa7fc700 10 client.15600 handle_command_reply: tid=0 2024-06-10T00:33:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.015+0000 7f32d3c29b80 1 client.15600 shutdown 2024-06-10T00:33:33.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.015+0000 7f32d3c29b80 1 -- 172.21.15.62:0/2370476466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aa8a0 msgr2=0x5622830b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.015+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/2370476466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aa8a0 0x5622830b34c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f32ac012af0 tx=0x7f32ac012e00 comp rx=0 tx=0).stop 2024-06-10T00:33:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.016+0000 7f32d3c29b80 1 -- 172.21.15.62:0/2370476466 shutdown_connections 2024-06-10T00:33:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.016+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/2370476466 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5622830c3b70 0x5622830c4fe0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.016+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/2370476466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aebe0 0x5622830b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.016+0000 7f32aa7fc700 0 client.15600 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:33:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.016+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/2370476466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 0x5622830b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.016+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/2370476466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aa8a0 0x5622830b34c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.016+0000 7f32d3c29b80 1 -- 172.21.15.62:0/2370476466 >> 172.21.15.62:0/2370476466 conn(0x562282fd0c70 msgr2=0x5622830a4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.016+0000 7f32d3c29b80 1 -- 172.21.15.62:0/2370476466 shutdown_connections 2024-06-10T00:33:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.016+0000 7f32d3c29b80 1 -- 172.21.15.62:0/2370476466 wait complete. 2024-06-10T00:33:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.016+0000 7f32d3c29b80 20 client.15600 trim_cache size 0 max 16384 2024-06-10T00:33:33.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.037+0000 7f32d3c29b80 1 Processor -- start 2024-06-10T00:33:33.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.037+0000 7f32d3c29b80 1 -- start start 2024-06-10T00:33:33.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32d3c29b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 0x5622830b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32d3c29b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aa8a0 0x5622830b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32d3c29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32d3c29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5622830c3320 con 0x5622830aebe0 2024-06-10T00:33:33.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32d3c29b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5622830c39f0 con 0x5622830a9f80 2024-06-10T00:33:33.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32d3c29b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5622830c40c0 con 0x5622830aa8a0 2024-06-10T00:33:33.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55198/0 (socket says 172.21.15.62:55198) 2024-06-10T00:33:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32c37fe700 1 -- 172.21.15.62:0/1199233201 learned_addr learned my addr 172.21.15.62:0/1199233201 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32c27fc700 1 --2- 172.21.15.62:0/1199233201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aa8a0 0x5622830b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32c2ffd700 1 --2- 172.21.15.62:0/1199233201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 0x5622830b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32c37fe700 1 -- 172.21.15.62:0/1199233201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aa8a0 msgr2=0x5622830b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32c37fe700 1 --2- 172.21.15.62:0/1199233201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aa8a0 0x5622830b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32c37fe700 1 -- 172.21.15.62:0/1199233201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 msgr2=0x5622830b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32c37fe700 1 --2- 172.21.15.62:0/1199233201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 0x5622830b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.038+0000 7f32c37fe700 1 -- 172.21.15.62:0/1199233201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622830ad4d0 con 0x5622830aebe0 2024-06-10T00:33:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.039+0000 7f32c37fe700 1 --2- 172.21.15.62:0/1199233201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830ce440 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0cdf10 tx=0x7f32bc022760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc5cf38283aa1bc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:33.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.039+0000 7f32b8ff9700 1 -- 172.21.15.62:0/1199233201 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc0ba070 con 0x5622830aebe0 2024-06-10T00:33:33.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.039+0000 7f32b8ff9700 1 -- 172.21.15.62:0/1199233201 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc09d5f0 con 0x5622830aebe0 2024-06-10T00:33:33.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.039+0000 7f32b8ff9700 1 -- 172.21.15.62:0/1199233201 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc09b8f0 con 0x5622830aebe0 2024-06-10T00:33:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.039+0000 7f32d3c29b80 1 -- 172.21.15.62:0/1199233201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 msgr2=0x5622830ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.039+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/1199233201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830ce440 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0cdf10 tx=0x7f32bc022760 comp rx=0 tx=0).stop 2024-06-10T00:33:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.039+0000 7f32d3c29b80 1 -- 172.21.15.62:0/1199233201 shutdown_connections 2024-06-10T00:33:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.039+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/1199233201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830ce440 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.039+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/1199233201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 0x5622830b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.039+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/1199233201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aa8a0 0x5622830b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.039+0000 7f32d3c29b80 1 -- 172.21.15.62:0/1199233201 >> 172.21.15.62:0/1199233201 conn(0x562282fd0c70 msgr2=0x5622830a4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.040+0000 7f32d3c29b80 1 -- 172.21.15.62:0/1199233201 shutdown_connections 2024-06-10T00:33:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.040+0000 7f32d3c29b80 1 -- 172.21.15.62:0/1199233201 wait complete. 2024-06-10T00:33:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.040+0000 7f32d3c29b80 1 -- start start 2024-06-10T00:33:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.040+0000 7f32d3c29b80 1 Processor -- start 2024-06-10T00:33:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.040+0000 7f32d3c29b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 0x5622830ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.040+0000 7f32d3c29b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aa8a0 0x5622830c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.040+0000 7f32d3c29b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.040+0000 7f32d3c29b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5622830c4e60 con 0x5622830aebe0 2024-06-10T00:33:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.040+0000 7f32d3c29b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5622830c4790 con 0x5622830a9f80 2024-06-10T00:33:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.040+0000 7f32d3c29b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5622830c5530 con 0x5622830aa8a0 2024-06-10T00:33:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.040+0000 7f32d3c29b80 10 client.0 _ll_register_callbacks cb 0x562282f918d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.040+0000 7f32c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.040+0000 7f32c37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55204/0 (socket says 172.21.15.62:55204) 2024-06-10T00:33:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.040+0000 7f32c37fe700 1 -- 172.21.15.62:0/1444253238 learned_addr learned my addr 172.21.15.62:0/1444253238 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32c2ffd700 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 0x5622830ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32c27fc700 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aa8a0 0x5622830c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32c37fe700 1 -- 172.21.15.62:0/1444253238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aa8a0 msgr2=0x5622830c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32c37fe700 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aa8a0 0x5622830c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32c37fe700 1 -- 172.21.15.62:0/1444253238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 msgr2=0x5622830ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32c37fe700 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 0x5622830ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32c37fe700 1 -- 172.21.15.62:0/1444253238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622830c8800 con 0x5622830aebe0 2024-06-10T00:33:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32c27fc700 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aa8a0 0x5622830c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32c2ffd700 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 0x5622830ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32c37fe700 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830c8210 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f32bc099fc0 tx=0x7f32bc0cd930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30db4ca0579afd4f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32aa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:33:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32aa7fc700 1 -- 172.21.15.62:0/1444253238 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc081060 con 0x5622830aebe0 2024-06-10T00:33:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32aa7fc700 1 -- 172.21.15.62:0/1444253238 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f32bc0d0160 con 0x5622830aebe0 2024-06-10T00:33:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32d3c29b80 1 -- 172.21.15.62:0/1444253238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562282e79200 con 0x5622830aebe0 2024-06-10T00:33:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32aa7fc700 1 -- 172.21.15.62:0/1444253238 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f32bc0ba050 con 0x5622830aebe0 2024-06-10T00:33:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32d3c29b80 10 client.15606 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.041+0000 7f32d3c29b80 1 -- 172.21.15.62:0/1444253238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5622830cd2b0 con 0x5622830aebe0 2024-06-10T00:33:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.042+0000 7f32aa7fc700 1 -- 172.21.15.62:0/1444253238 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562282e79200 con 0x5622830aebe0 2024-06-10T00:33:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.042+0000 7f32aa7fc700 1 -- 172.21.15.62:0/1444253238 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f32bc0a5d40 con 0x5622830aebe0 2024-06-10T00:33:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.042+0000 7f32d3c29b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.042+0000 7f32d3c29b80 10 client.15606 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T00:33:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.042+0000 7f32d3c29b80 20 client.15606 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.042+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5622830c3b70 0x5622831325a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.042+0000 7f32d3c29b80 4 client.15606 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:33:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.042+0000 7f32d3c29b80 1 -- 172.21.15.62:0/1444253238 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "status"}) -- 0x562282fc8b00 con 0x5622830c3b70 2024-06-10T00:33:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.042+0000 7f32c2ffd700 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5622830c3b70 0x5622831325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32c2ffd700 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5622830c3b70 0x5622831325a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9621ef9ec372d8b server_cookie=8f5136ecfc97860f in_seq=0 out_seq=0 2024-06-10T00:33:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32aa7fc700 10 client.15606 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:33:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32aa7fc700 1 -- 172.21.15.62:0/1444253238 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x562282fc8b00 con 0x5622830c3b70 2024-06-10T00:33:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32aa7fc700 10 client.15606 handle_command_reply: tid=0 2024-06-10T00:33:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32d3c29b80 1 client.15606 shutdown 2024-06-10T00:33:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32d3c29b80 1 -- 172.21.15.62:0/1444253238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 msgr2=0x5622830c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830c8210 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f32bc099fc0 tx=0x7f32bc0cd930 comp rx=0 tx=0).stop 2024-06-10T00:33:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32d3c29b80 1 -- 172.21.15.62:0/1444253238 shutdown_connections 2024-06-10T00:33:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5622830c3b70 0x5622831325a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5622830aebe0 0x5622830c8210 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622830a9f80 0x5622830ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32d3c29b80 1 --2- 172.21.15.62:0/1444253238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622830aa8a0 0x5622830c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32d3c29b80 1 -- 172.21.15.62:0/1444253238 >> 172.21.15.62:0/1444253238 conn(0x562282fd0c70 msgr2=0x562283099db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32aa7fc700 0 client.15606 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:33:33.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32d3c29b80 1 -- 172.21.15.62:0/1444253238 shutdown_connections 2024-06-10T00:33:33.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32d3c29b80 1 -- 172.21.15.62:0/1444253238 wait complete. 2024-06-10T00:33:33.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.043+0000 7f32d3c29b80 20 client.15606 trim_cache size 0 max 16384 2024-06-10T00:33:33.051 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:33:33.051 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:33:33.051 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 3, 2024-06-10T00:33:33.051 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24424, 2024-06-10T00:33:33.051 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:33:33.051 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:33:33.052 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:33:33.052 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 229.180647749, 2024-06-10T00:33:33.052 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 30, 2024-06-10T00:33:33.052 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 100, 2024-06-10T00:33:33.052 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 99, 2024-06-10T00:33:33.052 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 327.69682878399999 2024-06-10T00:33:33.052 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:33:33.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.044+0000 7f32c93a6700 1 -- 172.21.15.62:0/1966795677 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32b00a2ea0 msgr2=0x7f32b00a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:33.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.044+0000 7f32c93a6700 1 --2- 172.21.15.62:0/1966795677 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32b00a2ea0 0x7f32b00a5360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009550 tx=0x7f32b40091b0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.044+0000 7f32c93a6700 1 -- 172.21.15.62:0/1966795677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c41084f0 msgr2=0x7f32c4198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:33.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.044+0000 7f32c93a6700 1 --2- 172.21.15.62:0/1966795677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c41084f0 0x7f32c4198a10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0038b0 tx=0x7f32bc005e80 comp rx=0 tx=0).stop 2024-06-10T00:33:33.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.044+0000 7f32c93a6700 1 -- 172.21.15.62:0/1966795677 shutdown_connections 2024-06-10T00:33:33.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.044+0000 7f32c93a6700 1 --2- 172.21.15.62:0/1966795677 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32b00a2ea0 0x7f32b00a5360 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.044+0000 7f32c93a6700 1 --2- 172.21.15.62:0/1966795677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32c41084f0 0x7f32c4198a10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.044+0000 7f32c93a6700 1 --2- 172.21.15.62:0/1966795677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c406aa00 0x7f32c419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.044+0000 7f32c93a6700 1 --2- 172.21.15.62:0/1966795677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32c406b3c0 0x7f32c419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:33.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.044+0000 7f32c93a6700 1 -- 172.21.15.62:0/1966795677 >> 172.21.15.62:0/1966795677 conn(0x7f32c4078300 msgr2=0x7f32c40fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:33.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.044+0000 7f32c93a6700 1 -- 172.21.15.62:0/1966795677 shutdown_connections 2024-06-10T00:33:33.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:33.044+0000 7f32c93a6700 1 -- 172.21.15.62:0/1966795677 wait complete. 2024-06-10T00:33:33.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:33 smithi062 ceph-mon[32390]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-06-10T00:33:33.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:33 smithi105 ceph-mon[36321]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-06-10T00:33:33.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:33 smithi138 ceph-mon[36549]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-06-10T00:33:33.868 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-10T00:33:34.125 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:35.506 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:35 smithi062 ceph-mon[32390]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-10T00:33:35.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:35 smithi105 ceph-mon[36321]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-10T00:33:35.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:35 smithi138 ceph-mon[36549]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-10T00:33:35.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.848+0000 7f2eec70a700 1 -- 172.21.15.62:0/2753345528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee4107870 msgr2=0x7f2ee410f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:35.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.848+0000 7f2eec70a700 1 --2- 172.21.15.62:0/2753345528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee4107870 0x7f2ee410f3f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00b0d0 tx=0x7f2edc00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.848+0000 7f2eec70a700 1 -- 172.21.15.62:0/2753345528 shutdown_connections 2024-06-10T00:33:35.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.848+0000 7f2eec70a700 1 --2- 172.21.15.62:0/2753345528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee4107870 0x7f2ee410f3f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.848+0000 7f2eec70a700 1 --2- 172.21.15.62:0/2753345528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee41027a0 0x7f2ee4102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.848+0000 7f2eec70a700 1 --2- 172.21.15.62:0/2753345528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ee4103150 0x7f2ee4107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.848+0000 7f2eec70a700 1 -- 172.21.15.62:0/2753345528 >> 172.21.15.62:0/2753345528 conn(0x7f2ee4079520 msgr2=0x7f2ee4079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.849+0000 7f2eec70a700 1 -- 172.21.15.62:0/2753345528 shutdown_connections 2024-06-10T00:33:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.849+0000 7f2eec70a700 1 -- 172.21.15.62:0/2753345528 wait complete. 2024-06-10T00:33:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.849+0000 7f2eec70a700 1 Processor -- start 2024-06-10T00:33:35.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.849+0000 7f2eec70a700 1 -- start start 2024-06-10T00:33:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.849+0000 7f2eec70a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ee41027a0 0x7f2ee419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.849+0000 7f2eec70a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee4103150 0x7f2ee419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.849+0000 7f2eec70a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee4107870 0x7f2ee4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2eec70a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4112790 con 0x7f2ee4103150 2024-06-10T00:33:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2eec70a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4112610 con 0x7f2ee41027a0 2024-06-10T00:33:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2eec70a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4112910 con 0x7f2ee4107870 2024-06-10T00:33:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2ee9ca5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee4103150 0x7f2ee419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2ee9ca5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee4103150 0x7f2ee419ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55222/0 (socket says 172.21.15.62:55222) 2024-06-10T00:33:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2ee9ca5700 1 -- 172.21.15.62:0/2716699858 learned_addr learned my addr 172.21.15.62:0/2716699858 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2eea4a6700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ee41027a0 0x7f2ee419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2eeaca7700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee4107870 0x7f2ee4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2ee9ca5700 1 -- 172.21.15.62:0/2716699858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee4107870 msgr2=0x7f2ee4198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee4107870 0x7f2ee4198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2ee9ca5700 1 -- 172.21.15.62:0/2716699858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ee41027a0 msgr2=0x7f2ee419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ee41027a0 0x7f2ee419e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2ee9ca5700 1 -- 172.21.15.62:0/2716699858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee4199090 con 0x7f2ee4103150 2024-06-10T00:33:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2eeaca7700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee4107870 0x7f2ee4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee4103150 0x7f2ee419ecb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c920 tx=0x7f2ed800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2ed77fe700 1 -- 172.21.15.62:0/2716699858 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8007b00 con 0x7f2ee4103150 2024-06-10T00:33:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2eec70a700 1 -- 172.21.15.62:0/2716699858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee4199380 con 0x7f2ee4103150 2024-06-10T00:33:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.850+0000 7f2ed77fe700 1 -- 172.21.15.62:0/2716699858 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed8004d10 con 0x7f2ee4103150 2024-06-10T00:33:35.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.851+0000 7f2ed77fe700 1 -- 172.21.15.62:0/2716699858 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8005630 con 0x7f2ee4103150 2024-06-10T00:33:35.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.851+0000 7f2eec70a700 1 -- 172.21.15.62:0/2716699858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee40746c0 con 0x7f2ee4103150 2024-06-10T00:33:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.853+0000 7f2ed77fe700 1 -- 172.21.15.62:0/2716699858 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f2ed8004750 con 0x7f2ee4103150 2024-06-10T00:33:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.853+0000 7f2ed77fe700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2ed00a2e20 0x7f2ed00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.854+0000 7f2eea4a6700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2ed00a2e20 0x7f2ed00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.854+0000 7f2ed77fe700 1 -- 172.21.15.62:0/2716699858 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f2ed80cf490 con 0x7f2ee4103150 2024-06-10T00:33:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.854+0000 7f2ef6f8db80 1 Processor -- start 2024-06-10T00:33:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.854+0000 7f2eea4a6700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2ed00a2e20 0x7f2ed00a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4199a70 tx=0x7f2ee00073d0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.854+0000 7f2ef6f8db80 1 -- start start 2024-06-10T00:33:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.854+0000 7f2ef6f8db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bd39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.854+0000 7f2ef6f8db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bcd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.854+0000 7f2ef6f8db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bcded0 0x55b8f3bce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ef6f8db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b8f3b1f7c0 con 0x55b8f3bd3f10 2024-06-10T00:33:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ef6f8db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b8f3ac76d0 con 0x55b8f3bcded0 2024-06-10T00:33:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ef6f8db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b8f3b1b780 con 0x55b8f3bd35f0 2024-06-10T00:33:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ee9ca5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bcd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ee9ca5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bcd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55232/0 (socket says 172.21.15.62:55232) 2024-06-10T00:33:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ee9ca5700 1 -- 172.21.15.62:0/2649028881 learned_addr learned my addr 172.21.15.62:0/2649028881 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2eeaca7700 1 --2- 172.21.15.62:0/2649028881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bcded0 0x55b8f3bce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2eea4a6700 1 --2- 172.21.15.62:0/2649028881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bd39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ee9ca5700 1 -- 172.21.15.62:0/2649028881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd35f0 msgr2=0x55b8f3bd39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/2649028881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bd39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ee9ca5700 1 -- 172.21.15.62:0/2649028881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bcded0 msgr2=0x55b8f3bce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/2649028881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bcded0 0x55b8f3bce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ee9ca5700 1 -- 172.21.15.62:0/2649028881 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8f3abdf00 con 0x55b8f3bd3f10 2024-06-10T00:33:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2eeaca7700 1 --2- 172.21.15.62:0/2649028881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bcded0 0x55b8f3bce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/2649028881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bcd990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2ed809b190 tx=0x7f2ed809b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba8a5254b424b5d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ed4ff9700 1 -- 172.21.15.62:0/2649028881 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed801d040 con 0x55b8f3bd3f10 2024-06-10T00:33:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ed4ff9700 1 -- 172.21.15.62:0/2649028881 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed809b490 con 0x55b8f3bd3f10 2024-06-10T00:33:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.855+0000 7f2ed4ff9700 1 -- 172.21.15.62:0/2649028881 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed80cf1b0 con 0x55b8f3bd3f10 2024-06-10T00:33:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.856+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/2649028881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bd3f10 msgr2=0x55b8f3bcd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.856+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/2649028881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bcd990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2ed809b190 tx=0x7f2ed809b380 comp rx=0 tx=0).stop 2024-06-10T00:33:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.856+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/2649028881 shutdown_connections 2024-06-10T00:33:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.856+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/2649028881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bcded0 0x55b8f3bce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.856+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/2649028881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bd39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.856+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/2649028881 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bcd990 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.856+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/2649028881 >> 172.21.15.62:0/2649028881 conn(0x55b8f3af8c70 msgr2=0x55b8f3bc8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:35.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.856+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/2649028881 shutdown_connections 2024-06-10T00:33:35.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.856+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/2649028881 wait complete. 2024-06-10T00:33:35.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2ef6f8db80 1 -- start start 2024-06-10T00:33:35.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2ef6f8db80 1 Processor -- start 2024-06-10T00:33:35.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2ef6f8db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bcded0 0x55b8f3bd9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2ef6f8db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bd6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2ef6f8db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bd74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2ef6f8db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b8f3bda980 con 0x55b8f3bcded0 2024-06-10T00:33:35.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2ef6f8db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b8f3abfba0 con 0x55b8f3bd35f0 2024-06-10T00:33:35.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2ef6f8db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b8f3bdab00 con 0x55b8f3bd3f10 2024-06-10T00:33:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2ef6f8db80 10 client.0 _ll_register_callbacks cb 0x55b8f3ab98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2eea4a6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bcded0 0x55b8f3bd9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2eea4a6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bcded0 0x55b8f3bd9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55240/0 (socket says 172.21.15.62:55240) 2024-06-10T00:33:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2eea4a6700 1 -- 172.21.15.62:0/1188586732 learned_addr learned my addr 172.21.15.62:0/1188586732 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2eeaca7700 1 --2- 172.21.15.62:0/1188586732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bd74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/1188586732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bd6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2eea4a6700 1 -- 172.21.15.62:0/1188586732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd3f10 msgr2=0x55b8f3bd74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2eea4a6700 1 --2- 172.21.15.62:0/1188586732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bd74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2eea4a6700 1 -- 172.21.15.62:0/1188586732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd35f0 msgr2=0x55b8f3bd6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2eea4a6700 1 --2- 172.21.15.62:0/1188586732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bd6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.857+0000 7f2eea4a6700 1 -- 172.21.15.62:0/1188586732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8f3a6e880 con 0x55b8f3bcded0 2024-06-10T00:33:35.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.858+0000 7f2eea4a6700 1 --2- 172.21.15.62:0/1188586732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bcded0 0x55b8f3bd9c80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00040a0 tx=0x7f2ee00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb13346b876e914 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.858+0000 7f2ecdffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:33:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.858+0000 7f2ecdffb700 1 -- 172.21.15.62:0/1188586732 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0024070 con 0x55b8f3bcded0 2024-06-10T00:33:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.858+0000 7f2ecdffb700 1 -- 172.21.15.62:0/1188586732 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f2ee0003a90 con 0x55b8f3bcded0 2024-06-10T00:33:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.858+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/1188586732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed809b490 con 0x55b8f3bcded0 2024-06-10T00:33:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.858+0000 7f2ecdffb700 1 -- 172.21.15.62:0/1188586732 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ee0030050 con 0x55b8f3bcded0 2024-06-10T00:33:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.858+0000 7f2ef6f8db80 10 client.15618 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.858+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/1188586732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b8f3be52e0 con 0x55b8f3bcded0 2024-06-10T00:33:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.858+0000 7f2ecdffb700 1 -- 172.21.15.62:0/1188586732 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed809b490 con 0x55b8f3bcded0 2024-06-10T00:33:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.858+0000 7f2ecdffb700 1 -- 172.21.15.62:0/1188586732 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2ee0020410 con 0x55b8f3bcded0 2024-06-10T00:33:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.859+0000 7f2ef6f8db80 10 client.15618 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.859+0000 7f2ef6f8db80 10 client.15618 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-10T00:33:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.859+0000 7f2ef6f8db80 20 client.15618 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:35.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.859+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/1188586732 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55b8f3be78d0 0x55b8f3be8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.859+0000 7f2ef6f8db80 4 client.15618 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:33:35.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.859+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/1188586732 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8f3b2ccc0 con 0x55b8f3be78d0 2024-06-10T00:33:35.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.859+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/1188586732 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55b8f3be78d0 0x55b8f3be8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.859+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/1188586732 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55b8f3be78d0 0x55b8f3be8d40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3f9b1812c060e45 server_cookie=ad38602beb850f97 in_seq=0 out_seq=0 2024-06-10T00:33:35.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.859+0000 7f2ecdffb700 10 client.15618 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:33:35.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.860+0000 7f2ecdffb700 1 -- 172.21.15.62:0/1188586732 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b8f3b2ccc0 con 0x55b8f3be78d0 2024-06-10T00:33:35.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.860+0000 7f2ecdffb700 10 client.15618 handle_command_reply: tid=0 2024-06-10T00:33:35.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.860+0000 7f2ef6f8db80 1 client.15618 shutdown 2024-06-10T00:33:35.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.860+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/1188586732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bcded0 msgr2=0x55b8f3bd9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:35.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.860+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/1188586732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bcded0 0x55b8f3bd9c80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00040a0 tx=0x7f2ee00047c0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.860+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/1188586732 shutdown_connections 2024-06-10T00:33:35.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.860+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/1188586732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bcded0 0x55b8f3bd9c80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.860+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/1188586732 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55b8f3be78d0 0x55b8f3be8d40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.860+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/1188586732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bd74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.860+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/1188586732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bd6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.860+0000 7f2ecdffb700 0 client.15618 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:33:35.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.860+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/1188586732 >> 172.21.15.62:0/1188586732 conn(0x55b8f3af8c70 msgr2=0x55b8f3bcc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:35.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.860+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/1188586732 shutdown_connections 2024-06-10T00:33:35.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.860+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/1188586732 wait complete. 2024-06-10T00:33:35.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.861+0000 7f2ef6f8db80 20 client.15618 trim_cache size 0 max 16384 2024-06-10T00:33:35.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.882+0000 7f2ef6f8db80 1 Processor -- start 2024-06-10T00:33:35.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.882+0000 7f2ef6f8db80 1 -- start start 2024-06-10T00:33:35.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ef6f8db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bcded0 0x55b8f3bcd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ef6f8db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bf21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ef6f8db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bf26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ef6f8db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b8f3be7080 con 0x55b8f3bd35f0 2024-06-10T00:33:35.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ef6f8db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b8f3be7750 con 0x55b8f3bd3f10 2024-06-10T00:33:35.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ef6f8db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b8f3be7e20 con 0x55b8f3bcded0 2024-06-10T00:33:35.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ee9ca5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bf21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ee9ca5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bf21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55244/0 (socket says 172.21.15.62:55244) 2024-06-10T00:33:35.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ee9ca5700 1 -- 172.21.15.62:0/2744361843 learned_addr learned my addr 172.21.15.62:0/2744361843 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:35.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2eea4a6700 1 --2- 172.21.15.62:0/2744361843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bcded0 0x55b8f3bcd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ee9ca5700 1 -- 172.21.15.62:0/2744361843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bcded0 msgr2=0x55b8f3bcd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:35.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/2744361843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bcded0 0x55b8f3bcd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ee9ca5700 1 -- 172.21.15.62:0/2744361843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd3f10 msgr2=0x55b8f3bf26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:35.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/2744361843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bf26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ee9ca5700 1 -- 172.21.15.62:0/2744361843 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8f3bd5510 con 0x55b8f3bd35f0 2024-06-10T00:33:35.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/2744361843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bf21a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80cce30 tx=0x7f2ed8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd510e40d3bcc4e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:35.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ecffff700 1 -- 172.21.15.62:0/2744361843 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8064020 con 0x55b8f3bd35f0 2024-06-10T00:33:35.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ecffff700 1 -- 172.21.15.62:0/2744361843 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed809b490 con 0x55b8f3bd35f0 2024-06-10T00:33:35.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.883+0000 7f2ecffff700 1 -- 172.21.15.62:0/2744361843 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed801d040 con 0x55b8f3bd35f0 2024-06-10T00:33:35.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.884+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/2744361843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bd35f0 msgr2=0x55b8f3bf21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:35.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.884+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/2744361843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bf21a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80cce30 tx=0x7f2ed8000bc0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.884+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/2744361843 shutdown_connections 2024-06-10T00:33:35.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.884+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/2744361843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bf26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.884+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/2744361843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bcded0 0x55b8f3bcd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.884+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/2744361843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bf21a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.884+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/2744361843 >> 172.21.15.62:0/2744361843 conn(0x55b8f3af8c70 msgr2=0x55b8f3bd5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:35.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.884+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/2744361843 shutdown_connections 2024-06-10T00:33:35.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.884+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/2744361843 wait complete. 2024-06-10T00:33:35.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.884+0000 7f2ef6f8db80 1 -- start start 2024-06-10T00:33:35.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.884+0000 7f2ef6f8db80 1 Processor -- start 2024-06-10T00:33:35.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2ef6f8db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bcded0 0x55b8f3bccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2ef6f8db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bcd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2ef6f8db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bcd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2ef6f8db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b8f3be8bc0 con 0x55b8f3bcded0 2024-06-10T00:33:35.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2ef6f8db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b8f3be84f0 con 0x55b8f3bd35f0 2024-06-10T00:33:35.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2ef6f8db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b8f3be9290 con 0x55b8f3bd3f10 2024-06-10T00:33:35.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2ef6f8db80 10 client.0 _ll_register_callbacks cb 0x55b8f3ab98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:35.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2eea4a6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bcded0 0x55b8f3bccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2eea4a6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bcded0 0x55b8f3bccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55260/0 (socket says 172.21.15.62:55260) 2024-06-10T00:33:35.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2eea4a6700 1 -- 172.21.15.62:0/666864418 learned_addr learned my addr 172.21.15.62:0/666864418 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:35.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bcd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2eeaca7700 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bcd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2eea4a6700 1 -- 172.21.15.62:0/666864418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd3f10 msgr2=0x55b8f3bcd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:35.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2eea4a6700 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bcd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2eea4a6700 1 -- 172.21.15.62:0/666864418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd35f0 msgr2=0x55b8f3bcd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:35.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2eea4a6700 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bcd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2eea4a6700 1 -- 172.21.15.62:0/666864418 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8f3bf0700 con 0x55b8f3bcded0 2024-06-10T00:33:35.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bcd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:35.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2eeaca7700 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bcd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:35.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2eea4a6700 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bcded0 0x55b8f3bccd80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00040a0 tx=0x7f2ee00074d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7115637fa21abd98 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:35.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.885+0000 7f2ecdffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:33:35.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.886+0000 7f2ecdffb700 1 -- 172.21.15.62:0/666864418 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0024040 con 0x55b8f3bcded0 2024-06-10T00:33:35.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.886+0000 7f2ecdffb700 1 -- 172.21.15.62:0/666864418 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f2ee0020c60 con 0x55b8f3bcded0 2024-06-10T00:33:35.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.886+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/666864418 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed809b490 con 0x55b8f3bcded0 2024-06-10T00:33:35.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.886+0000 7f2ecdffb700 1 -- 172.21.15.62:0/666864418 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ee0043050 con 0x55b8f3bcded0 2024-06-10T00:33:35.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.886+0000 7f2ef6f8db80 10 client.15624 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:35.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.886+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/666864418 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b8f3c5bc30 con 0x55b8f3bcded0 2024-06-10T00:33:35.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.886+0000 7f2ecdffb700 1 -- 172.21.15.62:0/666864418 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed809b490 con 0x55b8f3bcded0 2024-06-10T00:33:35.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.886+0000 7f2ecdffb700 1 -- 172.21.15.62:0/666864418 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2ee0029740 con 0x55b8f3bcded0 2024-06-10T00:33:35.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.886+0000 7f2ef6f8db80 10 client.15624 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:35.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.886+0000 7f2ef6f8db80 10 client.15624 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-10T00:33:35.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.886+0000 7f2ef6f8db80 20 client.15624 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:35.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.886+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55b8f3c608c0 0x55b8f3c5f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:35.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.886+0000 7f2ef6f8db80 4 client.15624 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:33:35.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.886+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/666864418 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "status"}) -- 0x55b8f3b27240 con 0x55b8f3c608c0 2024-06-10T00:33:35.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55b8f3c608c0 0x55b8f3c5f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:35.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ecdffb700 10 client.15624 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:33:35.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ee9ca5700 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55b8f3c608c0 0x55b8f3c5f5d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e798b2c615b5775 server_cookie=7a9ca15417d6a5f1 in_seq=0 out_seq=0 2024-06-10T00:33:35.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ecdffb700 1 -- 172.21.15.62:0/666864418 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55b8f3b27240 con 0x55b8f3c608c0 2024-06-10T00:33:35.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ecdffb700 10 client.15624 handle_command_reply: tid=0 2024-06-10T00:33:35.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ef6f8db80 1 client.15624 shutdown 2024-06-10T00:33:35.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/666864418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bcded0 msgr2=0x55b8f3bccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:35.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bcded0 0x55b8f3bccd80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00040a0 tx=0x7f2ee00074d0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/666864418 shutdown_connections 2024-06-10T00:33:35.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b8f3bcded0 0x55b8f3bccd80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55b8f3c608c0 0x55b8f3c5f5d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b8f3bd3f10 0x55b8f3bcd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ef6f8db80 1 --2- 172.21.15.62:0/666864418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8f3bd35f0 0x55b8f3bcd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/666864418 >> 172.21.15.62:0/666864418 conn(0x55b8f3af8c70 msgr2=0x55b8f3bc1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:35.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ecdffb700 0 client.15624 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:33:35.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/666864418 shutdown_connections 2024-06-10T00:33:35.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.887+0000 7f2ef6f8db80 1 -- 172.21.15.62:0/666864418 wait complete. 2024-06-10T00:33:35.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.888+0000 7f2ef6f8db80 20 client.15624 trim_cache size 0 max 16384 2024-06-10T00:33:35.898 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:33:35.898 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:33:35.898 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2024-06-10T00:33:35.898 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24431, 2024-06-10T00:33:35.898 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:33:35.898 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:33:35.898 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:33:35.898 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 233.03561200600001, 2024-06-10T00:33:35.898 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 30, 2024-06-10T00:33:35.899 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 100, 2024-06-10T00:33:35.899 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 99, 2024-06-10T00:33:35.899 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 327.66678390800001 2024-06-10T00:33:35.899 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:33:35.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.888+0000 7f2eec70a700 1 -- 172.21.15.62:0/2716699858 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2ed00a2e20 msgr2=0x7f2ed00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:35.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.888+0000 7f2eec70a700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2ed00a2e20 0x7f2ed00a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4199a70 tx=0x7f2ee00073d0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.888+0000 7f2eec70a700 1 -- 172.21.15.62:0/2716699858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee4103150 msgr2=0x7f2ee419ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:35.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.888+0000 7f2eec70a700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee4103150 0x7f2ee419ecb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c920 tx=0x7f2ed800cc30 comp rx=0 tx=0).stop 2024-06-10T00:33:35.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.888+0000 7f2eec70a700 1 -- 172.21.15.62:0/2716699858 shutdown_connections 2024-06-10T00:33:35.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.888+0000 7f2eec70a700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2ed00a2e20 0x7f2ed00a52e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.888+0000 7f2eec70a700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee4107870 0x7f2ee4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.888+0000 7f2eec70a700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ee41027a0 0x7f2ee419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.888+0000 7f2eec70a700 1 --2- 172.21.15.62:0/2716699858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ee4103150 0x7f2ee419ecb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:35.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.888+0000 7f2eec70a700 1 -- 172.21.15.62:0/2716699858 >> 172.21.15.62:0/2716699858 conn(0x7f2ee4079520 msgr2=0x7f2ee4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:35.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.888+0000 7f2eec70a700 1 -- 172.21.15.62:0/2716699858 shutdown_connections 2024-06-10T00:33:35.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:35.888+0000 7f2eec70a700 1 -- 172.21.15.62:0/2716699858 wait complete. 2024-06-10T00:33:36.687 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T00:33:36.940 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:37.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:37 smithi062 ceph-mon[32390]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-06-10T00:33:37.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:37 smithi105 ceph-mon[36321]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-06-10T00:33:37.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:37 smithi138 ceph-mon[36549]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-06-10T00:33:38.293 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:33:38.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.663+0000 7f1072495700 1 -- 172.21.15.62:0/1066138795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f106c101390 msgr2=0x7f106c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.663+0000 7f1072495700 1 --2- 172.21.15.62:0/1066138795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f106c101390 0x7f106c101810 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1060009990 tx=0x7f1060009ca0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.663+0000 7f1072495700 1 -- 172.21.15.62:0/1066138795 shutdown_connections 2024-06-10T00:33:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.663+0000 7f1072495700 1 --2- 172.21.15.62:0/1066138795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f106c101d50 0x7f106c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.663+0000 7f1072495700 1 --2- 172.21.15.62:0/1066138795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106c107400 0x7f106c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.663+0000 7f1072495700 1 --2- 172.21.15.62:0/1066138795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f106c101390 0x7f106c101810 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.663+0000 7f1072495700 1 -- 172.21.15.62:0/1066138795 >> 172.21.15.62:0/1066138795 conn(0x7f106c078340 msgr2=0x7f106c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.663+0000 7f1072495700 1 -- 172.21.15.62:0/1066138795 shutdown_connections 2024-06-10T00:33:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.663+0000 7f1072495700 1 -- 172.21.15.62:0/1066138795 wait complete. 2024-06-10T00:33:38.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.664+0000 7f1072495700 1 Processor -- start 2024-06-10T00:33:38.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.664+0000 7f1072495700 1 -- start start 2024-06-10T00:33:38.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f1072495700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f106c101390 0x7f106c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f1072495700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f106c101d50 0x7f106c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f1072495700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106c107400 0x7f106c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f1072495700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f106c1116f0 con 0x7f106c101d50 2024-06-10T00:33:38.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f1072495700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f106c111570 con 0x7f106c107400 2024-06-10T00:33:38.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f1072495700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f106c111870 con 0x7f106c101390 2024-06-10T00:33:38.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f106b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f106c101d50 0x7f106c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f106b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f106c101d50 0x7f106c19edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55894/0 (socket says 172.21.15.62:55894) 2024-06-10T00:33:38.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f106b7fe700 1 -- 172.21.15.62:0/2029359514 learned_addr learned my addr 172.21.15.62:0/2029359514 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:38.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f106bfff700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f106c101390 0x7f106c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f1070a32700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106c107400 0x7f106c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f106b7fe700 1 -- 172.21.15.62:0/2029359514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f106c101390 msgr2=0x7f106c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:38.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f106b7fe700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f106c101390 0x7f106c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f106b7fe700 1 -- 172.21.15.62:0/2029359514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106c107400 msgr2=0x7f106c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:38.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f106b7fe700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106c107400 0x7f106c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f106b7fe700 1 -- 172.21.15.62:0/2029359514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106c1991a0 con 0x7f106c101d50 2024-06-10T00:33:38.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f106bfff700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f106c101390 0x7f106c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:38.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.665+0000 7f1070a32700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106c107400 0x7f106c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:38.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.666+0000 7f106b7fe700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f106c101d50 0x7f106c19edc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1060000c00 tx=0x7f1060009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:38.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.666+0000 7f10697fa700 1 -- 172.21.15.62:0/2029359514 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060027070 con 0x7f106c101d50 2024-06-10T00:33:38.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.666+0000 7f1072495700 1 -- 172.21.15.62:0/2029359514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106c199430 con 0x7f106c101d50 2024-06-10T00:33:38.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.666+0000 7f10697fa700 1 -- 172.21.15.62:0/2029359514 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1060004050 con 0x7f106c101d50 2024-06-10T00:33:38.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.666+0000 7f10697fa700 1 -- 172.21.15.62:0/2029359514 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060022e00 con 0x7f106c101d50 2024-06-10T00:33:38.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.666+0000 7f1072495700 1 -- 172.21.15.62:0/2029359514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f106c1a5750 con 0x7f106c101d50 2024-06-10T00:33:38.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.669+0000 7f10697fa700 1 -- 172.21.15.62:0/2029359514 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f106002a070 con 0x7f106c101d50 2024-06-10T00:33:38.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.670+0000 7f10697fa700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f10540a2ea0 0x7f10540a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.670+0000 7f107cd18b80 1 Processor -- start 2024-06-10T00:33:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.670+0000 7f107cd18b80 1 -- start start 2024-06-10T00:33:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.670+0000 7f10697fa700 1 -- 172.21.15.62:0/2029359514 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f10600d06e0 con 0x7f106c101d50 2024-06-10T00:33:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.670+0000 7f106bfff700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f10540a2ea0 0x7f10540a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.670+0000 7f107cd18b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 0x55c8ea4879d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.670+0000 7f107cd18b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea487f10 0x55c8ea481990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.670+0000 7f107cd18b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea4822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.670+0000 7f107cd18b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c8ea3d37c0 con 0x55c8ea481ed0 2024-06-10T00:33:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.670+0000 7f107cd18b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c8ea37b6d0 con 0x55c8ea4875f0 2024-06-10T00:33:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.670+0000 7f107cd18b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8ea3cf780 con 0x55c8ea487f10 2024-06-10T00:33:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.671+0000 7f1070a32700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea4822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.671+0000 7f1070a32700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea4822f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55906/0 (socket says 172.21.15.62:55906) 2024-06-10T00:33:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.671+0000 7f1070a32700 1 -- 172.21.15.62:0/1237452228 learned_addr learned my addr 172.21.15.62:0/1237452228 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.671+0000 7f106bfff700 1 --2- 172.21.15.62:0/1237452228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 0x55c8ea4879d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.671+0000 7f106bfff700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f10540a2ea0 0x7f10540a5360 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f106c199b80 tx=0x7f105c005d80 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.671+0000 7f106b7fe700 1 --2- 172.21.15.62:0/1237452228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea487f10 0x55c8ea481990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.671+0000 7f1070a32700 1 -- 172.21.15.62:0/1237452228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea487f10 msgr2=0x55c8ea481990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.671+0000 7f1070a32700 1 --2- 172.21.15.62:0/1237452228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea487f10 0x55c8ea481990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.671+0000 7f1070a32700 1 -- 172.21.15.62:0/1237452228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 msgr2=0x55c8ea4879d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.671+0000 7f1070a32700 1 --2- 172.21.15.62:0/1237452228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 0x55c8ea4879d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.671+0000 7f1070a32700 1 -- 172.21.15.62:0/1237452228 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8ea371f00 con 0x55c8ea481ed0 2024-06-10T00:33:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.671+0000 7f1070a32700 1 --2- 172.21.15.62:0/1237452228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea4822f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1064009e60 tx=0x7f10640077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40cc6bf1567f616 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.672+0000 7f10527fc700 1 -- 172.21.15.62:0/1237452228 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064014610 con 0x55c8ea481ed0 2024-06-10T00:33:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.672+0000 7f10527fc700 1 -- 172.21.15.62:0/1237452228 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1064019070 con 0x55c8ea481ed0 2024-06-10T00:33:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.672+0000 7f10527fc700 1 -- 172.21.15.62:0/1237452228 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064013ac0 con 0x55c8ea481ed0 2024-06-10T00:33:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.672+0000 7f107cd18b80 1 -- 172.21.15.62:0/1237452228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 msgr2=0x55c8ea4822f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.672+0000 7f107cd18b80 1 --2- 172.21.15.62:0/1237452228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea4822f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1064009e60 tx=0x7f10640077e0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.672+0000 7f107cd18b80 1 -- 172.21.15.62:0/1237452228 shutdown_connections 2024-06-10T00:33:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.672+0000 7f107cd18b80 1 --2- 172.21.15.62:0/1237452228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea4822f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.672+0000 7f107cd18b80 1 --2- 172.21.15.62:0/1237452228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 0x55c8ea4879d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.673+0000 7f107cd18b80 1 --2- 172.21.15.62:0/1237452228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea487f10 0x55c8ea481990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.673+0000 7f107cd18b80 1 -- 172.21.15.62:0/1237452228 >> 172.21.15.62:0/1237452228 conn(0x55c8ea3acc70 msgr2=0x55c8ea47c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.673+0000 7f107cd18b80 1 -- 172.21.15.62:0/1237452228 shutdown_connections 2024-06-10T00:33:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.673+0000 7f107cd18b80 1 -- 172.21.15.62:0/1237452228 wait complete. 2024-06-10T00:33:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.673+0000 7f107cd18b80 1 -- start start 2024-06-10T00:33:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.673+0000 7f107cd18b80 1 Processor -- start 2024-06-10T00:33:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f107cd18b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea481ed0 0x55c8ea48dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f107cd18b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea4875f0 0x55c8ea48afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f107cd18b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea487f10 0x55c8ea48b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f107cd18b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c8ea48e980 con 0x55c8ea487f10 2024-06-10T00:33:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f107cd18b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c8ea373ba0 con 0x55c8ea481ed0 2024-06-10T00:33:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f107cd18b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8ea48eb00 con 0x55c8ea4875f0 2024-06-10T00:33:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f107cd18b80 10 client.0 _ll_register_callbacks cb 0x55c8ea36d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f1070a32700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea487f10 0x55c8ea48b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f1070a32700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea487f10 0x55c8ea48b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55922/0 (socket says 172.21.15.62:55922) 2024-06-10T00:33:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f1070a32700 1 -- 172.21.15.62:0/2320687119 learned_addr learned my addr 172.21.15.62:0/2320687119 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:38.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f106bfff700 1 --2- 172.21.15.62:0/2320687119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea481ed0 0x55c8ea48dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f106b7fe700 1 --2- 172.21.15.62:0/2320687119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea4875f0 0x55c8ea48afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f1070a32700 1 -- 172.21.15.62:0/2320687119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea4875f0 msgr2=0x55c8ea48afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f1070a32700 1 --2- 172.21.15.62:0/2320687119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea4875f0 0x55c8ea48afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f1070a32700 1 -- 172.21.15.62:0/2320687119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea481ed0 msgr2=0x55c8ea48dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f1070a32700 1 --2- 172.21.15.62:0/2320687119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea481ed0 0x55c8ea48dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f1070a32700 1 -- 172.21.15.62:0/2320687119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8ea322880 con 0x55c8ea487f10 2024-06-10T00:33:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f1070a32700 1 --2- 172.21.15.62:0/2320687119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea487f10 0x55c8ea48b4f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1064009e30 tx=0x7f10640090b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3bc62a3daf79e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.674+0000 7f104b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:33:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.675+0000 7f104b7fe700 1 -- 172.21.15.62:0/2320687119 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064033070 con 0x55c8ea487f10 2024-06-10T00:33:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.675+0000 7f104b7fe700 1 -- 172.21.15.62:0/2320687119 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f1064004c90 con 0x55c8ea487f10 2024-06-10T00:33:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.675+0000 7f107cd18b80 1 -- 172.21.15.62:0/2320687119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1064019070 con 0x55c8ea487f10 2024-06-10T00:33:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.675+0000 7f104b7fe700 1 -- 172.21.15.62:0/2320687119 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1064041050 con 0x55c8ea487f10 2024-06-10T00:33:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.675+0000 7f107cd18b80 10 client.15636 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.675+0000 7f107cd18b80 1 -- 172.21.15.62:0/2320687119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8ea4a1af0 con 0x55c8ea487f10 2024-06-10T00:33:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.675+0000 7f104b7fe700 1 -- 172.21.15.62:0/2320687119 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1064038940 con 0x55c8ea487f10 2024-06-10T00:33:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.676+0000 7f104b7fe700 1 -- 172.21.15.62:0/2320687119 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1064005330 con 0x55c8ea487f10 2024-06-10T00:33:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.676+0000 7f107cd18b80 10 client.15636 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.676+0000 7f107cd18b80 10 client.15636 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T00:33:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.676+0000 7f107cd18b80 20 client.15636 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.676+0000 7f107cd18b80 1 --2- 172.21.15.62:0/2320687119 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c8ea4a2f50 0x55c8ea4a5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.676+0000 7f107cd18b80 4 client.15636 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:33:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.676+0000 7f107cd18b80 1 -- 172.21.15.62:0/2320687119 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1064019070 con 0x55c8ea4a2f50 2024-06-10T00:33:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.676+0000 7f106bfff700 1 --2- 172.21.15.62:0/2320687119 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c8ea4a2f50 0x55c8ea4a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.677+0000 7f106bfff700 1 --2- 172.21.15.62:0/2320687119 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c8ea4a2f50 0x55c8ea4a5140 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8196ebf9b08bb79 server_cookie=4b380c50ec82562b in_seq=0 out_seq=0 2024-06-10T00:33:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.677+0000 7f104b7fe700 10 client.15636 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:33:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.677+0000 7f104b7fe700 1 -- 172.21.15.62:0/2320687119 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f1064019070 con 0x55c8ea4a2f50 2024-06-10T00:33:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.677+0000 7f104b7fe700 10 client.15636 handle_command_reply: tid=0 2024-06-10T00:33:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.677+0000 7f107cd18b80 1 client.15636 shutdown 2024-06-10T00:33:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.678+0000 7f107cd18b80 1 -- 172.21.15.62:0/2320687119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea487f10 msgr2=0x55c8ea48b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.678+0000 7f107cd18b80 1 --2- 172.21.15.62:0/2320687119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea487f10 0x55c8ea48b4f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1064009e30 tx=0x7f10640090b0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.678+0000 7f107cd18b80 1 -- 172.21.15.62:0/2320687119 shutdown_connections 2024-06-10T00:33:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.678+0000 7f107cd18b80 1 --2- 172.21.15.62:0/2320687119 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c8ea4a2f50 0x55c8ea4a5140 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.678+0000 7f107cd18b80 1 --2- 172.21.15.62:0/2320687119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea487f10 0x55c8ea48b4f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.678+0000 7f107cd18b80 1 --2- 172.21.15.62:0/2320687119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea481ed0 0x55c8ea48dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.678+0000 7f104b7fe700 0 client.15636 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:33:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.678+0000 7f107cd18b80 1 --2- 172.21.15.62:0/2320687119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea4875f0 0x55c8ea48afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.678+0000 7f107cd18b80 1 -- 172.21.15.62:0/2320687119 >> 172.21.15.62:0/2320687119 conn(0x55c8ea3acc70 msgr2=0x55c8ea480980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.678+0000 7f107cd18b80 1 -- 172.21.15.62:0/2320687119 shutdown_connections 2024-06-10T00:33:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.678+0000 7f107cd18b80 1 -- 172.21.15.62:0/2320687119 wait complete. 2024-06-10T00:33:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.678+0000 7f107cd18b80 20 client.15636 trim_cache size 0 max 16384 2024-06-10T00:33:38.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.705+0000 7f107cd18b80 1 Processor -- start 2024-06-10T00:33:38.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.706+0000 7f107cd18b80 1 -- start start 2024-06-10T00:33:38.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.706+0000 7f107cd18b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea481990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.706+0000 7f107cd18b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 0x55c8ea485780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.706+0000 7f107cd18b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea487f10 0x55c8ea4ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.706+0000 7f107cd18b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c8ea4a3480 con 0x55c8ea481ed0 2024-06-10T00:33:38.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.706+0000 7f107cd18b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c8ea4a3b50 con 0x55c8ea4875f0 2024-06-10T00:33:38.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.706+0000 7f106bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea481990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.706+0000 7f107cd18b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8ea4a4220 con 0x55c8ea487f10 2024-06-10T00:33:38.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.706+0000 7f106bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea481990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55924/0 (socket says 172.21.15.62:55924) 2024-06-10T00:33:38.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.706+0000 7f106bfff700 1 -- 172.21.15.62:0/4148462921 learned_addr learned my addr 172.21.15.62:0/4148462921 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:38.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.706+0000 7f106b7fe700 1 --2- 172.21.15.62:0/4148462921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 0x55c8ea485780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.706+0000 7f1070a32700 1 --2- 172.21.15.62:0/4148462921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea487f10 0x55c8ea4ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.707+0000 7f106bfff700 1 -- 172.21.15.62:0/4148462921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea487f10 msgr2=0x55c8ea4ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:38.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.707+0000 7f106bfff700 1 --2- 172.21.15.62:0/4148462921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea487f10 0x55c8ea4ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.707+0000 7f106bfff700 1 -- 172.21.15.62:0/4148462921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 msgr2=0x55c8ea485780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:38.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.707+0000 7f106bfff700 1 --2- 172.21.15.62:0/4148462921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 0x55c8ea485780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.707+0000 7f106bfff700 1 -- 172.21.15.62:0/4148462921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8ea4a3970 con 0x55c8ea481ed0 2024-06-10T00:33:38.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.707+0000 7f106bfff700 1 --2- 172.21.15.62:0/4148462921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea481990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55c8ea4a58e0 tx=0x7f105c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b94947980ed7590 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:38.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.707+0000 7f1051ffb700 1 -- 172.21.15.62:0/4148462921 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c011bb0 con 0x55c8ea481ed0 2024-06-10T00:33:38.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.707+0000 7f1051ffb700 1 -- 172.21.15.62:0/4148462921 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f105c011d50 con 0x55c8ea481ed0 2024-06-10T00:33:38.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.707+0000 7f1051ffb700 1 -- 172.21.15.62:0/4148462921 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c033660 con 0x55c8ea481ed0 2024-06-10T00:33:38.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.708+0000 7f107cd18b80 1 -- 172.21.15.62:0/4148462921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 msgr2=0x55c8ea481990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:38.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.708+0000 7f107cd18b80 1 --2- 172.21.15.62:0/4148462921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea481990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55c8ea4a58e0 tx=0x7f105c012000 comp rx=0 tx=0).stop 2024-06-10T00:33:38.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.708+0000 7f107cd18b80 1 -- 172.21.15.62:0/4148462921 shutdown_connections 2024-06-10T00:33:38.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.708+0000 7f107cd18b80 1 --2- 172.21.15.62:0/4148462921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea481990 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.708+0000 7f107cd18b80 1 --2- 172.21.15.62:0/4148462921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea487f10 0x55c8ea4ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.708+0000 7f107cd18b80 1 --2- 172.21.15.62:0/4148462921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 0x55c8ea485780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.708+0000 7f107cd18b80 1 -- 172.21.15.62:0/4148462921 >> 172.21.15.62:0/4148462921 conn(0x55c8ea3acc70 msgr2=0x55c8ea485070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:38.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.708+0000 7f107cd18b80 1 -- 172.21.15.62:0/4148462921 shutdown_connections 2024-06-10T00:33:38.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.709+0000 7f107cd18b80 1 -- 172.21.15.62:0/4148462921 wait complete. 2024-06-10T00:33:38.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.709+0000 7f107cd18b80 1 -- start start 2024-06-10T00:33:38.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.709+0000 7f107cd18b80 1 Processor -- start 2024-06-10T00:33:38.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.709+0000 7f107cd18b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea4ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.709+0000 7f107cd18b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 0x55c8ea4aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f107cd18b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea487f10 0x55c8ea4a9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f106bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea4ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f106bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea4ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55934/0 (socket says 172.21.15.62:55934) 2024-06-10T00:33:38.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f106bfff700 1 -- 172.21.15.62:0/29919582 learned_addr learned my addr 172.21.15.62:0/29919582 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:38.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f107cd18b80 1 -- 172.21.15.62:0/29919582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c8ea4a4fc0 con 0x55c8ea481ed0 2024-06-10T00:33:38.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f107cd18b80 1 -- 172.21.15.62:0/29919582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c8ea4a48f0 con 0x55c8ea4875f0 2024-06-10T00:33:38.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f107cd18b80 1 -- 172.21.15.62:0/29919582 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8ea4a5690 con 0x55c8ea487f10 2024-06-10T00:33:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f107cd18b80 10 client.0 _ll_register_callbacks cb 0x55c8ea36d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f106b7fe700 1 --2- 172.21.15.62:0/29919582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 0x55c8ea4aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f106bfff700 1 -- 172.21.15.62:0/29919582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea487f10 msgr2=0x55c8ea4a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f106bfff700 1 --2- 172.21.15.62:0/29919582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea487f10 0x55c8ea4a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f106bfff700 1 -- 172.21.15.62:0/29919582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 msgr2=0x55c8ea4aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f106bfff700 1 --2- 172.21.15.62:0/29919582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 0x55c8ea4aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f106bfff700 1 -- 172.21.15.62:0/29919582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8ea4af1a0 con 0x55c8ea481ed0 2024-06-10T00:33:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f106bfff700 1 --2- 172.21.15.62:0/29919582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea4ae5c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f105c010010 tx=0x7f105c011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab69dde0b926c169 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.710+0000 7f104b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:33:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.711+0000 7f104b7fe700 1 -- 172.21.15.62:0/29919582 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c01f700 con 0x55c8ea481ed0 2024-06-10T00:33:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.711+0000 7f104b7fe700 1 -- 172.21.15.62:0/29919582 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f105c02f050 con 0x55c8ea481ed0 2024-06-10T00:33:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.711+0000 7f107cd18b80 1 -- 172.21.15.62:0/29919582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f105c011d50 con 0x55c8ea481ed0 2024-06-10T00:33:38.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.711+0000 7f104b7fe700 1 -- 172.21.15.62:0/29919582 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f105c01f8a0 con 0x55c8ea481ed0 2024-06-10T00:33:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.711+0000 7f107cd18b80 10 client.15642 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.711+0000 7f107cd18b80 1 -- 172.21.15.62:0/29919582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8ea50fb50 con 0x55c8ea481ed0 2024-06-10T00:33:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.711+0000 7f104b7fe700 1 -- 172.21.15.62:0/29919582 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f105c011d50 con 0x55c8ea481ed0 2024-06-10T00:33:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.712+0000 7f104b7fe700 1 -- 172.21.15.62:0/29919582 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f105c004750 con 0x55c8ea481ed0 2024-06-10T00:33:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.712+0000 7f107cd18b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.712+0000 7f107cd18b80 10 client.15642 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T00:33:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.712+0000 7f107cd18b80 20 client.15642 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.712+0000 7f107cd18b80 1 --2- 172.21.15.62:0/29919582 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c8ea5143f0 0x55c8ea513100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.712+0000 7f107cd18b80 4 client.15642 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:33:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.712+0000 7f107cd18b80 1 -- 172.21.15.62:0/29919582 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "status"}) -- 0x55c8ea3e0cc0 con 0x55c8ea5143f0 2024-06-10T00:33:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.713+0000 7f106b7fe700 1 --2- 172.21.15.62:0/29919582 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c8ea5143f0 0x55c8ea513100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.713+0000 7f104b7fe700 10 client.15642 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:33:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.713+0000 7f106b7fe700 1 --2- 172.21.15.62:0/29919582 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c8ea5143f0 0x55c8ea513100 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4c1c344c6f23c15 server_cookie=12f52f4d7214aa00 in_seq=0 out_seq=0 2024-06-10T00:33:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.713+0000 7f104b7fe700 1 -- 172.21.15.62:0/29919582 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55c8ea3e0cc0 con 0x55c8ea5143f0 2024-06-10T00:33:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.713+0000 7f104b7fe700 10 client.15642 handle_command_reply: tid=0 2024-06-10T00:33:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.713+0000 7f107cd18b80 1 client.15642 shutdown 2024-06-10T00:33:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.714+0000 7f107cd18b80 1 -- 172.21.15.62:0/29919582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 msgr2=0x55c8ea4ae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.714+0000 7f107cd18b80 1 --2- 172.21.15.62:0/29919582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea4ae5c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f105c010010 tx=0x7f105c011f90 comp rx=0 tx=0).stop 2024-06-10T00:33:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.714+0000 7f107cd18b80 1 -- 172.21.15.62:0/29919582 shutdown_connections 2024-06-10T00:33:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.714+0000 7f107cd18b80 1 --2- 172.21.15.62:0/29919582 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55c8ea5143f0 0x55c8ea513100 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.714+0000 7f107cd18b80 1 --2- 172.21.15.62:0/29919582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8ea481ed0 0x55c8ea4ae5c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.714+0000 7f104b7fe700 0 client.15642 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:33:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.714+0000 7f107cd18b80 1 --2- 172.21.15.62:0/29919582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8ea487f10 0x55c8ea4a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.714+0000 7f107cd18b80 1 --2- 172.21.15.62:0/29919582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8ea4875f0 0x55c8ea4aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.714+0000 7f107cd18b80 1 -- 172.21.15.62:0/29919582 >> 172.21.15.62:0/29919582 conn(0x55c8ea3acc70 msgr2=0x55c8ea475db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:38.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.714+0000 7f107cd18b80 1 -- 172.21.15.62:0/29919582 shutdown_connections 2024-06-10T00:33:38.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.714+0000 7f107cd18b80 1 -- 172.21.15.62:0/29919582 wait complete. 2024-06-10T00:33:38.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.714+0000 7f107cd18b80 20 client.15642 trim_cache size 0 max 16384 2024-06-10T00:33:38.719 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:33:38.719 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:33:38.719 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2024-06-10T00:33:38.719 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24433, 2024-06-10T00:33:38.719 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:33:38.719 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:33:38.719 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:33:38.719 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 236.87081956500001, 2024-06-10T00:33:38.720 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 30, 2024-06-10T00:33:38.720 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 100, 2024-06-10T00:33:38.720 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 99, 2024-06-10T00:33:38.720 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 327.75714798299998 2024-06-10T00:33:38.720 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:33:38.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.715+0000 7f1072495700 1 -- 172.21.15.62:0/2029359514 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f10540a2ea0 msgr2=0x7f10540a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:38.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.715+0000 7f1072495700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f10540a2ea0 0x7f10540a5360 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f106c199b80 tx=0x7f105c005d80 comp rx=0 tx=0).stop 2024-06-10T00:33:38.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.715+0000 7f1072495700 1 -- 172.21.15.62:0/2029359514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f106c101d50 msgr2=0x7f106c19edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:38.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.715+0000 7f1072495700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f106c101d50 0x7f106c19edc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1060000c00 tx=0x7f1060009f90 comp rx=0 tx=0).stop 2024-06-10T00:33:38.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.715+0000 7f1072495700 1 -- 172.21.15.62:0/2029359514 shutdown_connections 2024-06-10T00:33:38.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.715+0000 7f1072495700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f10540a2ea0 0x7f10540a5360 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.715+0000 7f1072495700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f106c107400 0x7f106c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.715+0000 7f1072495700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f106c101390 0x7f106c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.715+0000 7f1072495700 1 --2- 172.21.15.62:0/2029359514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f106c101d50 0x7f106c19edc0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:38.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.715+0000 7f1072495700 1 -- 172.21.15.62:0/2029359514 >> 172.21.15.62:0/2029359514 conn(0x7f106c078340 msgr2=0x7f106c1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:38.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.715+0000 7f1072495700 1 -- 172.21.15.62:0/2029359514 shutdown_connections 2024-06-10T00:33:38.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:38.715+0000 7f1072495700 1 -- 172.21.15.62:0/2029359514 wait complete. 2024-06-10T00:33:39.512 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:39 smithi062 ceph-mon[32390]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:33:39.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:39 smithi105 ceph-mon[36321]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:33:39.547 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-10T00:33:39.575 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:39.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:39 smithi138 ceph-mon[36549]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:33:40.327 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:41.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:41 smithi105 ceph-mon[36321]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:33:41.545 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:41 smithi062 ceph-mon[32390]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:33:41.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:41 smithi138 ceph-mon[36549]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:33:41.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.899+0000 7f22f33a0700 1 -- 172.21.15.62:0/1185479494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec108500 msgr2=0x7f22ec1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.899+0000 7f22f33a0700 1 --2- 172.21.15.62:0/1185479494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec108500 0x7f22ec1088e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009990 tx=0x7f22e8009ca0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.899+0000 7f22f33a0700 1 -- 172.21.15.62:0/1185479494 shutdown_connections 2024-06-10T00:33:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.899+0000 7f22f33a0700 1 --2- 172.21.15.62:0/1185479494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec108500 0x7f22ec1088e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.899+0000 7f22f33a0700 1 --2- 172.21.15.62:0/1185479494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22ec06ab30 0x7f22ec10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.899+0000 7f22f33a0700 1 --2- 172.21.15.62:0/1185479494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec06a170 0x7f22ec06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.899+0000 7f22f33a0700 1 -- 172.21.15.62:0/1185479494 >> 172.21.15.62:0/1185479494 conn(0x7f22ec078340 msgr2=0x7f22ec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.899+0000 7f22f33a0700 1 -- 172.21.15.62:0/1185479494 shutdown_connections 2024-06-10T00:33:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.899+0000 7f22f33a0700 1 -- 172.21.15.62:0/1185479494 wait complete. 2024-06-10T00:33:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.900+0000 7f22f33a0700 1 Processor -- start 2024-06-10T00:33:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.900+0000 7f22f33a0700 1 -- start start 2024-06-10T00:33:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.900+0000 7f22f33a0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec06a170 0x7f22ec19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.900+0000 7f22f33a0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec06ab30 0x7f22ec19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.900+0000 7f22f33a0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22ec108500 0x7f22ec1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.900+0000 7f22f33a0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec112780 con 0x7f22ec06a170 2024-06-10T00:33:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.900+0000 7f22f33a0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec112600 con 0x7f22ec06ab30 2024-06-10T00:33:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.900+0000 7f22f33a0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec112900 con 0x7f22ec108500 2024-06-10T00:33:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.900+0000 7f22f113c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec06a170 0x7f22ec19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.900+0000 7f22f113c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec06a170 0x7f22ec19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55938/0 (socket says 172.21.15.62:55938) 2024-06-10T00:33:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.901+0000 7f22f113c700 1 -- 172.21.15.62:0/1091489314 learned_addr learned my addr 172.21.15.62:0/1091489314 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.901+0000 7f22f093b700 1 --2- 172.21.15.62:0/1091489314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec06ab30 0x7f22ec19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.901+0000 7f22f193d700 1 --2- 172.21.15.62:0/1091489314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22ec108500 0x7f22ec1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.901+0000 7f22f113c700 1 -- 172.21.15.62:0/1091489314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22ec108500 msgr2=0x7f22ec1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.901+0000 7f22f113c700 1 --2- 172.21.15.62:0/1091489314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22ec108500 0x7f22ec1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.901+0000 7f22f113c700 1 -- 172.21.15.62:0/1091489314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec06ab30 msgr2=0x7f22ec19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.901+0000 7f22f113c700 1 --2- 172.21.15.62:0/1091489314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec06ab30 0x7f22ec19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.901+0000 7f22f113c700 1 -- 172.21.15.62:0/1091489314 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec199120 con 0x7f22ec06a170 2024-06-10T00:33:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.901+0000 7f22f113c700 1 --2- 172.21.15.62:0/1091489314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec06a170 0x7f22ec19e7b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f22e8000c00 tx=0x7f22e80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.901+0000 7f22e27fc700 1 -- 172.21.15.62:0/1091489314 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8027070 con 0x7f22ec06a170 2024-06-10T00:33:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.901+0000 7f22e27fc700 1 -- 172.21.15.62:0/1091489314 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e8003e40 con 0x7f22ec06a170 2024-06-10T00:33:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.901+0000 7f22e27fc700 1 -- 172.21.15.62:0/1091489314 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8022d10 con 0x7f22ec06a170 2024-06-10T00:33:41.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.901+0000 7f22f33a0700 1 -- 172.21.15.62:0/1091489314 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ec1993b0 con 0x7f22ec06a170 2024-06-10T00:33:41.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.901+0000 7f22f33a0700 1 -- 172.21.15.62:0/1091489314 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ec1a5680 con 0x7f22ec06a170 2024-06-10T00:33:41.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.904+0000 7f22e27fc700 1 -- 172.21.15.62:0/1091489314 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f22e802a070 con 0x7f22ec06a170 2024-06-10T00:33:41.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.904+0000 7f22e27fc700 1 --2- 172.21.15.62:0/1091489314 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f22d80a2e20 0x7f22d80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.905+0000 7f22f093b700 1 --2- 172.21.15.62:0/1091489314 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f22d80a2e20 0x7f22d80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.905+0000 7f22e27fc700 1 -- 172.21.15.62:0/1091489314 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f22e80d06a0 con 0x7f22ec06a170 2024-06-10T00:33:41.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.905+0000 7f22f093b700 1 --2- 172.21.15.62:0/1091489314 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f22d80a2e20 0x7f22d80a52e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f22ec078840 tx=0x7f22dc00a380 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:41.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.906+0000 7f22fdc23b80 1 Processor -- start 2024-06-10T00:33:41.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.906+0000 7f22fdc23b80 1 -- start start 2024-06-10T00:33:41.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.906+0000 7f22fdc23b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d855ecf60 0x560d855ed340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.906+0000 7f22fdc23b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855ed880 0x560d855e7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.906+0000 7f22fdc23b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855e77a0 0x560d855e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.906+0000 7f22fdc23b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560d854d9f60 con 0x560d855ecf60 2024-06-10T00:33:41.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.906+0000 7f22fdc23b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560d85538850 con 0x560d855ed880 2024-06-10T00:33:41.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.906+0000 7f22fdc23b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560d854a0460 con 0x560d855e77a0 2024-06-10T00:33:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.906+0000 7f22f113c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d855ecf60 0x560d855ed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.907+0000 7f22f113c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d855ecf60 0x560d855ed340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55946/0 (socket says 172.21.15.62:55946) 2024-06-10T00:33:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.907+0000 7f22f113c700 1 -- 172.21.15.62:0/3834379856 learned_addr learned my addr 172.21.15.62:0/3834379856 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.907+0000 7f22f093b700 1 --2- 172.21.15.62:0/3834379856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855ed880 0x560d855e7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.907+0000 7f22f193d700 1 --2- 172.21.15.62:0/3834379856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855e77a0 0x560d855e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.907+0000 7f22f113c700 1 -- 172.21.15.62:0/3834379856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855e77a0 msgr2=0x560d855e7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.907+0000 7f22f113c700 1 --2- 172.21.15.62:0/3834379856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855e77a0 0x560d855e7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.907+0000 7f22f113c700 1 -- 172.21.15.62:0/3834379856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855ed880 msgr2=0x560d855e7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.907+0000 7f22f113c700 1 --2- 172.21.15.62:0/3834379856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855ed880 0x560d855e7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.907+0000 7f22f113c700 1 -- 172.21.15.62:0/3834379856 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d854878d0 con 0x560d855ecf60 2024-06-10T00:33:41.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.907+0000 7f22f113c700 1 --2- 172.21.15.62:0/3834379856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d855ecf60 0x560d855ed340 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f22e809c6c0 tx=0x7f22e809c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb49c3a0e288d7d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:41.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.907+0000 7f22d77fe700 1 -- 172.21.15.62:0/3834379856 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e809c730 con 0x560d855ecf60 2024-06-10T00:33:41.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.907+0000 7f22d77fe700 1 -- 172.21.15.62:0/3834379856 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e80a0ae0 con 0x560d855ecf60 2024-06-10T00:33:41.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.907+0000 7f22d77fe700 1 -- 172.21.15.62:0/3834379856 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e80d0290 con 0x560d855ecf60 2024-06-10T00:33:41.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.908+0000 7f22fdc23b80 1 -- 172.21.15.62:0/3834379856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d855ecf60 msgr2=0x560d855ed340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:41.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.908+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/3834379856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d855ecf60 0x560d855ed340 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f22e809c6c0 tx=0x7f22e809c630 comp rx=0 tx=0).stop 2024-06-10T00:33:41.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.908+0000 7f22fdc23b80 1 -- 172.21.15.62:0/3834379856 shutdown_connections 2024-06-10T00:33:41.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.908+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/3834379856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d855ecf60 0x560d855ed340 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.908+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/3834379856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855e77a0 0x560d855e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.908+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/3834379856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855ed880 0x560d855e7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.908+0000 7f22fdc23b80 1 -- 172.21.15.62:0/3834379856 >> 172.21.15.62:0/3834379856 conn(0x560d85514080 msgr2=0x560d855e2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:41.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.908+0000 7f22fdc23b80 1 -- 172.21.15.62:0/3834379856 shutdown_connections 2024-06-10T00:33:41.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.908+0000 7f22fdc23b80 1 -- 172.21.15.62:0/3834379856 wait complete. 2024-06-10T00:33:41.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.909+0000 7f22fdc23b80 1 -- start start 2024-06-10T00:33:41.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.909+0000 7f22fdc23b80 1 Processor -- start 2024-06-10T00:33:41.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.909+0000 7f22fdc23b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855e77a0 0x560d855f3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.909+0000 7f22fdc23b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d855ecf60 0x560d855f0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.909+0000 7f22fdc23b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855ed880 0x560d855f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.909+0000 7f22fdc23b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560d854d8fc0 con 0x560d855ecf60 2024-06-10T00:33:41.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.909+0000 7f22fdc23b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560d85534810 con 0x560d855e77a0 2024-06-10T00:33:41.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.909+0000 7f22fdc23b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560d855f8420 con 0x560d855ed880 2024-06-10T00:33:41.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.909+0000 7f22fdc23b80 10 client.0 _ll_register_callbacks cb 0x560d854ce5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:41.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.909+0000 7f22f093b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d855ecf60 0x560d855f0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.910+0000 7f22f093b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d855ecf60 0x560d855f0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55950/0 (socket says 172.21.15.62:55950) 2024-06-10T00:33:41.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.910+0000 7f22f093b700 1 -- 172.21.15.62:0/2671918791 learned_addr learned my addr 172.21.15.62:0/2671918791 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:41.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.910+0000 7f22f113c700 1 --2- 172.21.15.62:0/2671918791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855e77a0 0x560d855f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.910+0000 7f22f193d700 1 --2- 172.21.15.62:0/2671918791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855ed880 0x560d855f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.910+0000 7f22f093b700 1 -- 172.21.15.62:0/2671918791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855ed880 msgr2=0x560d855f0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:41.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.910+0000 7f22f093b700 1 --2- 172.21.15.62:0/2671918791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855ed880 0x560d855f0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.910+0000 7f22f093b700 1 -- 172.21.15.62:0/2671918791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855e77a0 msgr2=0x560d855f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:41.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.910+0000 7f22f093b700 1 --2- 172.21.15.62:0/2671918791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855e77a0 0x560d855f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.910+0000 7f22f093b700 1 -- 172.21.15.62:0/2671918791 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d854fe750 con 0x560d855ecf60 2024-06-10T00:33:41.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.910+0000 7f22f093b700 1 --2- 172.21.15.62:0/2671918791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d855ecf60 0x560d855f0910 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f22dc003680 tx=0x7f22dc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63053366490bfeb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:41.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.910+0000 7f22d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:33:41.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.910+0000 7f22d4ff9700 1 -- 172.21.15.62:0/2671918791 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc016070 con 0x560d855ecf60 2024-06-10T00:33:41.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.910+0000 7f22d4ff9700 1 -- 172.21.15.62:0/2671918791 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f22dc010840 con 0x560d855ecf60 2024-06-10T00:33:41.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.910+0000 7f22fdc23b80 1 -- 172.21.15.62:0/2671918791 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e80a0ae0 con 0x560d855ecf60 2024-06-10T00:33:41.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.911+0000 7f22d4ff9700 1 -- 172.21.15.62:0/2671918791 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22dc030050 con 0x560d855ecf60 2024-06-10T00:33:41.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.911+0000 7f22fdc23b80 10 client.15654 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:41.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.911+0000 7f22fdc23b80 1 -- 172.21.15.62:0/2671918791 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d85602c00 con 0x560d855ecf60 2024-06-10T00:33:41.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.911+0000 7f22d4ff9700 1 -- 172.21.15.62:0/2671918791 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e80a0ae0 con 0x560d855ecf60 2024-06-10T00:33:41.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.911+0000 7f22d4ff9700 1 -- 172.21.15.62:0/2671918791 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f22dc027c60 con 0x560d855ecf60 2024-06-10T00:33:41.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.911+0000 7f22fdc23b80 10 client.15654 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:41.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.911+0000 7f22fdc23b80 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:33:41.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.911+0000 7f22fdc23b80 20 client.15654 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:41.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.911+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/2671918791 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d85605240 0x560d85607630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.911+0000 7f22fdc23b80 4 client.15654 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:33:41.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.911+0000 7f22fdc23b80 1 -- 172.21.15.62:0/2671918791 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d85545d50 con 0x560d85605240 2024-06-10T00:33:41.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.912+0000 7f22f113c700 1 --2- 172.21.15.62:0/2671918791 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d85605240 0x560d85607630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.912+0000 7f22f113c700 1 --2- 172.21.15.62:0/2671918791 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d85605240 0x560d85607630 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1125daffd402a13e server_cookie=bc4502a03c4db234 in_seq=0 out_seq=0 2024-06-10T00:33:41.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.912+0000 7f22d4ff9700 10 client.15654 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:41.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22d4ff9700 1 -- 172.21.15.62:0/2671918791 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560d85545d50 con 0x560d85605240 2024-06-10T00:33:41.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22d4ff9700 10 client.15654 handle_command_reply: tid=0 2024-06-10T00:33:41.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22fdc23b80 1 client.15654 shutdown 2024-06-10T00:33:41.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22fdc23b80 1 -- 172.21.15.62:0/2671918791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d855ecf60 msgr2=0x560d855f0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:41.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/2671918791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d855ecf60 0x560d855f0910 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f22dc003680 tx=0x7f22dc0047c0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22fdc23b80 1 -- 172.21.15.62:0/2671918791 shutdown_connections 2024-06-10T00:33:41.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/2671918791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855ed880 0x560d855f0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/2671918791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855e77a0 0x560d855f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/2671918791 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d85605240 0x560d85607630 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/2671918791 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d855ecf60 0x560d855f0910 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22d4ff9700 0 client.15654 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:41.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22fdc23b80 1 -- 172.21.15.62:0/2671918791 >> 172.21.15.62:0/2671918791 conn(0x560d85514080 msgr2=0x560d855e1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:41.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22fdc23b80 1 -- 172.21.15.62:0/2671918791 shutdown_connections 2024-06-10T00:33:41.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22fdc23b80 1 -- 172.21.15.62:0/2671918791 wait complete. 2024-06-10T00:33:41.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.913+0000 7f22fdc23b80 20 client.15654 trim_cache size 0 max 16384 2024-06-10T00:33:41.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.935+0000 7f22fdc23b80 1 Processor -- start 2024-06-10T00:33:41.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.935+0000 7f22fdc23b80 1 -- start start 2024-06-10T00:33:41.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.935+0000 7f22fdc23b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d85514080 0x560d855e7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.935+0000 7f22fdc23b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855e77a0 0x560d855ee370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.935+0000 7f22fdc23b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855ecf60 0x560d855e1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22fdc23b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec1ac180 con 0x560d85514080 2024-06-10T00:33:41.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22fdc23b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec1adcc0 con 0x560d855ecf60 2024-06-10T00:33:41.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22fdc23b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec1ac850 con 0x560d855e77a0 2024-06-10T00:33:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22f113c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d85514080 0x560d855e7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22f113c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d85514080 0x560d855e7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55960/0 (socket says 172.21.15.62:55960) 2024-06-10T00:33:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22f113c700 1 -- 172.21.15.62:0/1472535295 learned_addr learned my addr 172.21.15.62:0/1472535295 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22f093b700 1 --2- 172.21.15.62:0/1472535295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855e77a0 0x560d855ee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22f193d700 1 --2- 172.21.15.62:0/1472535295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855ecf60 0x560d855e1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22f113c700 1 -- 172.21.15.62:0/1472535295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855e77a0 msgr2=0x560d855ee370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22f113c700 1 --2- 172.21.15.62:0/1472535295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855e77a0 0x560d855ee370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22f113c700 1 -- 172.21.15.62:0/1472535295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855ecf60 msgr2=0x560d855e1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22f113c700 1 --2- 172.21.15.62:0/1472535295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855ecf60 0x560d855e1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22f113c700 1 -- 172.21.15.62:0/1472535295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d855f06e0 con 0x560d85514080 2024-06-10T00:33:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22f193d700 1 --2- 172.21.15.62:0/1472535295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855ecf60 0x560d855e1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:33:41.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22f113c700 1 --2- 172.21.15.62:0/1472535295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d85514080 0x560d855e7260 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f22e80cd9e0 tx=0x7f22e8087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d5b85cad21471f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22d6ffd700 1 -- 172.21.15.62:0/1472535295 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e809c730 con 0x560d85514080 2024-06-10T00:33:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22d6ffd700 1 -- 172.21.15.62:0/1472535295 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e8003ff0 con 0x560d85514080 2024-06-10T00:33:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.936+0000 7f22d6ffd700 1 -- 172.21.15.62:0/1472535295 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e80d04f0 con 0x560d85514080 2024-06-10T00:33:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.937+0000 7f22fdc23b80 1 -- 172.21.15.62:0/1472535295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d85514080 msgr2=0x560d855e7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.937+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/1472535295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d85514080 0x560d855e7260 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f22e80cd9e0 tx=0x7f22e8087040 comp rx=0 tx=0).stop 2024-06-10T00:33:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.937+0000 7f22fdc23b80 1 -- 172.21.15.62:0/1472535295 shutdown_connections 2024-06-10T00:33:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.937+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/1472535295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d85514080 0x560d855e7260 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.937+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/1472535295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855ecf60 0x560d855e1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.937+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/1472535295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855e77a0 0x560d855ee370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.937+0000 7f22fdc23b80 1 -- 172.21.15.62:0/1472535295 >> 172.21.15.62:0/1472535295 conn(0x560d855eb1e0 msgr2=0x560d855edc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.937+0000 7f22fdc23b80 1 -- 172.21.15.62:0/1472535295 shutdown_connections 2024-06-10T00:33:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.937+0000 7f22fdc23b80 1 -- 172.21.15.62:0/1472535295 wait complete. 2024-06-10T00:33:41.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.938+0000 7f22fdc23b80 1 -- start start 2024-06-10T00:33:41.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.938+0000 7f22fdc23b80 1 Processor -- start 2024-06-10T00:33:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.938+0000 7f22fdc23b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d85514080 0x560d855fd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.938+0000 7f22fdc23b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855e77a0 0x560d855fdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.938+0000 7f22fdc23b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855ecf60 0x560d855fa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.938+0000 7f22fdc23b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec1ad5f0 con 0x560d85514080 2024-06-10T00:33:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.938+0000 7f22fdc23b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec1acf20 con 0x560d855e77a0 2024-06-10T00:33:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.938+0000 7f22fdc23b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec1aa120 con 0x560d855ecf60 2024-06-10T00:33:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.938+0000 7f22fdc23b80 10 client.0 _ll_register_callbacks cb 0x560d854ce5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.938+0000 7f22f113c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d85514080 0x560d855fd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.938+0000 7f22f113c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d85514080 0x560d855fd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55970/0 (socket says 172.21.15.62:55970) 2024-06-10T00:33:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.938+0000 7f22f113c700 1 -- 172.21.15.62:0/3769300525 learned_addr learned my addr 172.21.15.62:0/3769300525 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22f193d700 1 --2- 172.21.15.62:0/3769300525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855ecf60 0x560d855fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22f093b700 1 --2- 172.21.15.62:0/3769300525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855e77a0 0x560d855fdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22f113c700 1 -- 172.21.15.62:0/3769300525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855ecf60 msgr2=0x560d855fa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22f113c700 1 --2- 172.21.15.62:0/3769300525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855ecf60 0x560d855fa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22f113c700 1 -- 172.21.15.62:0/3769300525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855e77a0 msgr2=0x560d855fdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22f113c700 1 --2- 172.21.15.62:0/3769300525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855e77a0 0x560d855fdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22f113c700 1 -- 172.21.15.62:0/3769300525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d855fab80 con 0x560d85514080 2024-06-10T00:33:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22f113c700 1 --2- 172.21.15.62:0/3769300525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d85514080 0x560d855fd670 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f22e809c430 tx=0x7f22e809cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9288295e0b56b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:33:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22d4ff9700 1 -- 172.21.15.62:0/3769300525 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e80cdc30 con 0x560d85514080 2024-06-10T00:33:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22d4ff9700 1 -- 172.21.15.62:0/3769300525 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f22e80cddd0 con 0x560d85514080 2024-06-10T00:33:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22fdc23b80 1 -- 172.21.15.62:0/3769300525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e8003ff0 con 0x560d85514080 2024-06-10T00:33:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22d4ff9700 1 -- 172.21.15.62:0/3769300525 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22e80d0290 con 0x560d85514080 2024-06-10T00:33:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22fdc23b80 10 client.15660 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.939+0000 7f22fdc23b80 1 -- 172.21.15.62:0/3769300525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d855fab80 con 0x560d85514080 2024-06-10T00:33:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.940+0000 7f22d4ff9700 1 -- 172.21.15.62:0/3769300525 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e8003ff0 con 0x560d85514080 2024-06-10T00:33:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.940+0000 7f22d4ff9700 1 -- 172.21.15.62:0/3769300525 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f22e80a4e80 con 0x560d85514080 2024-06-10T00:33:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.940+0000 7f22fdc23b80 10 client.15660 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.940+0000 7f22fdc23b80 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:33:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.940+0000 7f22fdc23b80 20 client.15660 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.940+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/3769300525 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d855ed880 0x560d85672740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:41.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.940+0000 7f22fdc23b80 4 client.15660 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:33:41.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.940+0000 7f22fdc23b80 1 -- 172.21.15.62:0/3769300525 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d855402d0 con 0x560d855ed880 2024-06-10T00:33:41.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.940+0000 7f22f093b700 1 --2- 172.21.15.62:0/3769300525 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d855ed880 0x560d85672740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:41.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22d4ff9700 10 client.15660 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:41.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22f093b700 1 --2- 172.21.15.62:0/3769300525 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d855ed880 0x560d85672740 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5f1e01cdd9d6dcc server_cookie=47b938c38397c32e in_seq=0 out_seq=0 2024-06-10T00:33:41.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22d4ff9700 1 -- 172.21.15.62:0/3769300525 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560d855402d0 con 0x560d855ed880 2024-06-10T00:33:41.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22d4ff9700 10 client.15660 handle_command_reply: tid=0 2024-06-10T00:33:41.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22fdc23b80 1 client.15660 shutdown 2024-06-10T00:33:41.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22fdc23b80 1 -- 172.21.15.62:0/3769300525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d85514080 msgr2=0x560d855fd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:41.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/3769300525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d85514080 0x560d855fd670 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f22e809c430 tx=0x7f22e809cf70 comp rx=0 tx=0).stop 2024-06-10T00:33:41.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22fdc23b80 1 -- 172.21.15.62:0/3769300525 shutdown_connections 2024-06-10T00:33:41.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/3769300525 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560d855ed880 0x560d85672740 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/3769300525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560d85514080 0x560d855fd670 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/3769300525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d855ecf60 0x560d855fa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22d4ff9700 0 client.15660 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:41.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22fdc23b80 1 --2- 172.21.15.62:0/3769300525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d855e77a0 0x560d855fdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22fdc23b80 1 -- 172.21.15.62:0/3769300525 >> 172.21.15.62:0/3769300525 conn(0x560d855eb1e0 msgr2=0x560d855eba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:41.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22fdc23b80 1 -- 172.21.15.62:0/3769300525 shutdown_connections 2024-06-10T00:33:41.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22fdc23b80 1 -- 172.21.15.62:0/3769300525 wait complete. 2024-06-10T00:33:41.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.941+0000 7f22fdc23b80 20 client.15660 trim_cache size 0 max 16384 2024-06-10T00:33:41.947 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:33:41.947 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-06-10T00:33:41.948 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-06-10T00:33:41.948 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:33:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.942+0000 7f22f33a0700 1 -- 172.21.15.62:0/1091489314 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f22d80a2e20 msgr2=0x7f22d80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.942+0000 7f22f33a0700 1 --2- 172.21.15.62:0/1091489314 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f22d80a2e20 0x7f22d80a52e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f22ec078840 tx=0x7f22dc00a380 comp rx=0 tx=0).stop 2024-06-10T00:33:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.942+0000 7f22f33a0700 1 -- 172.21.15.62:0/1091489314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec06a170 msgr2=0x7f22ec19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.942+0000 7f22f33a0700 1 --2- 172.21.15.62:0/1091489314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec06a170 0x7f22ec19e7b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f22e8000c00 tx=0x7f22e80047b0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.942+0000 7f22f33a0700 1 -- 172.21.15.62:0/1091489314 shutdown_connections 2024-06-10T00:33:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.942+0000 7f22f33a0700 1 --2- 172.21.15.62:0/1091489314 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f22d80a2e20 0x7f22d80a52e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.942+0000 7f22f33a0700 1 --2- 172.21.15.62:0/1091489314 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22ec06a170 0x7f22ec19e7b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.942+0000 7f22f33a0700 1 --2- 172.21.15.62:0/1091489314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22ec108500 0x7f22ec1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.942+0000 7f22f33a0700 1 --2- 172.21.15.62:0/1091489314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22ec06ab30 0x7f22ec19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.942+0000 7f22f33a0700 1 -- 172.21.15.62:0/1091489314 >> 172.21.15.62:0/1091489314 conn(0x7f22ec078340 msgr2=0x7f22ec0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.942+0000 7f22f33a0700 1 -- 172.21.15.62:0/1091489314 shutdown_connections 2024-06-10T00:33:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:41.942+0000 7f22f33a0700 1 -- 172.21.15.62:0/1091489314 wait complete. 2024-06-10T00:33:43.225 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T00:33:43.225 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:33:43.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:43 smithi062 ceph-mon[32390]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 12 op/s 2024-06-10T00:33:43.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:43 smithi105 ceph-mon[36321]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 12 op/s 2024-06-10T00:33:43.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:43 smithi138 ceph-mon[36549]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 12 op/s 2024-06-10T00:33:43.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.750+0000 7f293caa0700 1 -- 172.21.15.62:0/966365234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2938102510 msgr2=0x7f2938102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:43.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.750+0000 7f293caa0700 1 --2- 172.21.15.62:0/966365234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2938102510 0x7f2938102990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f292c009990 tx=0x7f292c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.750+0000 7f293caa0700 1 -- 172.21.15.62:0/966365234 shutdown_connections 2024-06-10T00:33:43.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.750+0000 7f293caa0700 1 --2- 172.21.15.62:0/966365234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2938102ed0 0x7f293810f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.750+0000 7f293caa0700 1 --2- 172.21.15.62:0/966365234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2938108520 0x7f2938108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.750+0000 7f293caa0700 1 --2- 172.21.15.62:0/966365234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2938102510 0x7f2938102990 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.750+0000 7f293caa0700 1 -- 172.21.15.62:0/966365234 >> 172.21.15.62:0/966365234 conn(0x7f2938079560 msgr2=0x7f2938079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:43.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.750+0000 7f293caa0700 1 -- 172.21.15.62:0/966365234 shutdown_connections 2024-06-10T00:33:43.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.750+0000 7f293caa0700 1 -- 172.21.15.62:0/966365234 wait complete. 2024-06-10T00:33:43.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.751+0000 7f293caa0700 1 Processor -- start 2024-06-10T00:33:43.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.751+0000 7f293caa0700 1 -- start start 2024-06-10T00:33:43.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.752+0000 7f293caa0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2938102510 0x7f293819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.752+0000 7f293caa0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2938102ed0 0x7f293819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.752+0000 7f293caa0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2938108520 0x7f29381989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.752+0000 7f293caa0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f29381127f0 con 0x7f2938102ed0 2024-06-10T00:33:43.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.752+0000 7f293caa0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2938112670 con 0x7f2938108520 2024-06-10T00:33:43.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.752+0000 7f293caa0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2938112970 con 0x7f2938102510 2024-06-10T00:33:43.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.752+0000 7f2935d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2938102ed0 0x7f293819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.752+0000 7f293659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2938102510 0x7f293819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.752+0000 7f293659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2938102510 0x7f293819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:37646/0 (socket says 172.21.15.62:37646) 2024-06-10T00:33:43.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.752+0000 7f293659c700 1 -- 172.21.15.62:0/3614193798 learned_addr learned my addr 172.21.15.62:0/3614193798 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:43.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.753+0000 7f2936d9d700 1 --2- 172.21.15.62:0/3614193798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2938108520 0x7f29381989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.753+0000 7f293659c700 1 -- 172.21.15.62:0/3614193798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2938108520 msgr2=0x7f29381989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:43.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.753+0000 7f293659c700 1 --2- 172.21.15.62:0/3614193798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2938108520 0x7f29381989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.753+0000 7f293659c700 1 -- 172.21.15.62:0/3614193798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2938102ed0 msgr2=0x7f293819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:43.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.753+0000 7f293659c700 1 --2- 172.21.15.62:0/3614193798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2938102ed0 0x7f293819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.753+0000 7f293659c700 1 -- 172.21.15.62:0/3614193798 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2938199150 con 0x7f2938102510 2024-06-10T00:33:43.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.753+0000 7f293659c700 1 --2- 172.21.15.62:0/3614193798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2938102510 0x7f293819e830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2928012db0 tx=0x7f292800c5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:43.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.753+0000 7f29277fe700 1 -- 172.21.15.62:0/3614193798 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292801f070 con 0x7f2938102510 2024-06-10T00:33:43.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.753+0000 7f29277fe700 1 -- 172.21.15.62:0/3614193798 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f292801a910 con 0x7f2938102510 2024-06-10T00:33:43.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.753+0000 7f293caa0700 1 -- 172.21.15.62:0/3614193798 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2938199440 con 0x7f2938102510 2024-06-10T00:33:43.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.753+0000 7f29277fe700 1 -- 172.21.15.62:0/3614193798 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29280189d0 con 0x7f2938102510 2024-06-10T00:33:43.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.754+0000 7f293caa0700 1 -- 172.21.15.62:0/3614193798 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29380746c0 con 0x7f2938102510 2024-06-10T00:33:43.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.757+0000 7f29277fe700 1 -- 172.21.15.62:0/3614193798 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f2928022070 con 0x7f2938102510 2024-06-10T00:33:43.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.758+0000 7f29277fe700 1 --2- 172.21.15.62:0/3614193798 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f29200a2ea0 0x7f29200a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.758+0000 7f29277fe700 1 -- 172.21.15.62:0/3614193798 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f29280d36c0 con 0x7f2938102510 2024-06-10T00:33:43.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.758+0000 7f2935d9b700 1 --2- 172.21.15.62:0/3614193798 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f29200a2ea0 0x7f29200a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.759+0000 7f2935d9b700 1 --2- 172.21.15.62:0/3614193798 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f29200a2ea0 0x7f29200a5360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2938079b60 tx=0x7f292c005fd0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:43.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.759+0000 7f2947323b80 1 Processor -- start 2024-06-10T00:33:43.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f2947323b80 1 -- start start 2024-06-10T00:33:43.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f2947323b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634816405f0 0x5634816409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f2947323b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x56348163a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f2947323b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56348163aed0 0x56348163b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f2947323b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56348158c7c0 con 0x563481640f10 2024-06-10T00:33:43.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f2947323b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5634815346d0 con 0x56348163aed0 2024-06-10T00:33:43.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f2947323b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563481588780 con 0x5634816405f0 2024-06-10T00:33:43.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f293659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634816405f0 0x5634816409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f293659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634816405f0 0x5634816409d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:37662/0 (socket says 172.21.15.62:37662) 2024-06-10T00:33:43.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f293659c700 1 -- 172.21.15.62:0/2599931645 learned_addr learned my addr 172.21.15.62:0/2599931645 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:43.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f2936d9d700 1 --2- 172.21.15.62:0/2599931645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56348163aed0 0x56348163b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f293659c700 1 -- 172.21.15.62:0/2599931645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56348163aed0 msgr2=0x56348163b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:43.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f293659c700 1 --2- 172.21.15.62:0/2599931645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56348163aed0 0x56348163b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f293659c700 1 -- 172.21.15.62:0/2599931645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 msgr2=0x56348163a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:43.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.760+0000 7f293659c700 1 --2- 172.21.15.62:0/2599931645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x56348163a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.761+0000 7f293659c700 1 -- 172.21.15.62:0/2599931645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56348152af00 con 0x5634816405f0 2024-06-10T00:33:43.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.761+0000 7f293659c700 1 --2- 172.21.15.62:0/2599931645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634816405f0 0x5634816409d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2928012d80 tx=0x7f292809ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ca74b1843cb58c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:43.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.761+0000 7f2924ff9700 1 -- 172.21.15.62:0/2599931645 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29280b8070 con 0x5634816405f0 2024-06-10T00:33:43.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.761+0000 7f2924ff9700 1 -- 172.21.15.62:0/2599931645 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f292809fd60 con 0x5634816405f0 2024-06-10T00:33:43.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.761+0000 7f2924ff9700 1 -- 172.21.15.62:0/2599931645 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292809f8c0 con 0x5634816405f0 2024-06-10T00:33:43.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.762+0000 7f2947323b80 1 -- 172.21.15.62:0/2599931645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634816405f0 msgr2=0x5634816409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:43.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.762+0000 7f2947323b80 1 --2- 172.21.15.62:0/2599931645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634816405f0 0x5634816409d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2928012d80 tx=0x7f292809ba30 comp rx=0 tx=0).stop 2024-06-10T00:33:43.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.762+0000 7f2947323b80 1 -- 172.21.15.62:0/2599931645 shutdown_connections 2024-06-10T00:33:43.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.762+0000 7f2947323b80 1 --2- 172.21.15.62:0/2599931645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56348163aed0 0x56348163b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.762+0000 7f2947323b80 1 --2- 172.21.15.62:0/2599931645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634816405f0 0x5634816409d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.762+0000 7f2947323b80 1 --2- 172.21.15.62:0/2599931645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x56348163a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.762+0000 7f2947323b80 1 -- 172.21.15.62:0/2599931645 >> 172.21.15.62:0/2599931645 conn(0x563481565c70 msgr2=0x563481635800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:43.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.763+0000 7f2947323b80 1 -- 172.21.15.62:0/2599931645 shutdown_connections 2024-06-10T00:33:43.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.763+0000 7f2947323b80 1 -- 172.21.15.62:0/2599931645 wait complete. 2024-06-10T00:33:43.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.763+0000 7f2947323b80 1 -- start start 2024-06-10T00:33:43.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.763+0000 7f2947323b80 1 Processor -- start 2024-06-10T00:33:43.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.764+0000 7f2947323b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 0x563481646d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.764+0000 7f2947323b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634816405f0 0x563481644050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.764+0000 7f2947323b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x563481644590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.764+0000 7f2947323b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563481647800 con 0x563481640f10 2024-06-10T00:33:43.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.764+0000 7f2936d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x563481644590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.764+0000 7f2936d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x563481644590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56018/0 (socket says 172.21.15.62:56018) 2024-06-10T00:33:43.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.764+0000 7f2936d9d700 1 -- 172.21.15.62:0/1091749212 learned_addr learned my addr 172.21.15.62:0/1091749212 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:43.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.764+0000 7f293659c700 1 --2- 172.21.15.62:0/1091749212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 0x563481646d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.764+0000 7f2935d9b700 1 --2- 172.21.15.62:0/1091749212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634816405f0 0x563481644050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.764+0000 7f2947323b80 1 -- 172.21.15.62:0/1091749212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56348152cba0 con 0x5634816405f0 2024-06-10T00:33:43.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.764+0000 7f2947323b80 1 -- 172.21.15.62:0/1091749212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563481647980 con 0x56348163aed0 2024-06-10T00:33:43.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.764+0000 7f2947323b80 10 client.0 _ll_register_callbacks cb 0x5634815268d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:43.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.765+0000 7f2936d9d700 1 -- 172.21.15.62:0/1091749212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 msgr2=0x563481646d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:43.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.765+0000 7f2936d9d700 1 --2- 172.21.15.62:0/1091749212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 0x563481646d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.765+0000 7f2936d9d700 1 -- 172.21.15.62:0/1091749212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634816405f0 msgr2=0x563481644050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:43.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.765+0000 7f2936d9d700 1 --2- 172.21.15.62:0/1091749212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634816405f0 0x563481644050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.765+0000 7f2936d9d700 1 -- 172.21.15.62:0/1091749212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634814db880 con 0x563481640f10 2024-06-10T00:33:43.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.765+0000 7f2936d9d700 1 --2- 172.21.15.62:0/1091749212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x563481644590 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2930007af0 tx=0x7f2930007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a5cea01cabebce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:43.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.765+0000 7f291dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:33:43.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.765+0000 7f291dffb700 1 -- 172.21.15.62:0/1091749212 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293001a070 con 0x563481640f10 2024-06-10T00:33:43.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.765+0000 7f2947323b80 1 -- 172.21.15.62:0/1091749212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f292809fd60 con 0x563481640f10 2024-06-10T00:33:43.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.766+0000 7f291dffb700 1 -- 172.21.15.62:0/1091749212 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f2930012b40 con 0x563481640f10 2024-06-10T00:33:43.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.766+0000 7f291dffb700 1 -- 172.21.15.62:0/1091749212 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2930026050 con 0x563481640f10 2024-06-10T00:33:43.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.766+0000 7f2947323b80 10 client.15669 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:43.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.766+0000 7f2947323b80 1 -- 172.21.15.62:0/1091749212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563481652200 con 0x563481640f10 2024-06-10T00:33:43.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.766+0000 7f291dffb700 1 -- 172.21.15.62:0/1091749212 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2930012d40 con 0x563481640f10 2024-06-10T00:33:43.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.766+0000 7f291dffb700 1 -- 172.21.15.62:0/1091749212 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f29300155c0 con 0x563481640f10 2024-06-10T00:33:43.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.766+0000 7f2947323b80 10 client.15669 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:43.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.766+0000 7f2947323b80 10 client.15669 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T00:33:43.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.766+0000 7f2947323b80 20 client.15669 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:43.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.766+0000 7f2947323b80 1 --2- 172.21.15.62:0/1091749212 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563481654c40 0x5634816560b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.767+0000 7f2947323b80 4 client.15669 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:33:43.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.767+0000 7f2947323b80 1 -- 172.21.15.62:0/1091749212 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f292809fd60 con 0x563481654c40 2024-06-10T00:33:43.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.767+0000 7f293659c700 1 --2- 172.21.15.62:0/1091749212 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563481654c40 0x5634816560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.767+0000 7f291dffb700 10 client.15669 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:43.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.767+0000 7f293659c700 1 --2- 172.21.15.62:0/1091749212 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563481654c40 0x5634816560b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb61181ed996350 server_cookie=598627a50712471f in_seq=0 out_seq=0 2024-06-10T00:33:43.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f291dffb700 1 -- 172.21.15.62:0/1091749212 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f292809fd60 con 0x563481654c40 2024-06-10T00:33:43.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f291dffb700 10 client.15669 handle_command_reply: tid=0 2024-06-10T00:33:43.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f2947323b80 1 client.15669 shutdown 2024-06-10T00:33:43.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f2947323b80 1 -- 172.21.15.62:0/1091749212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 msgr2=0x563481644590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:43.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f2947323b80 1 --2- 172.21.15.62:0/1091749212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x563481644590 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2930007af0 tx=0x7f2930007e00 comp rx=0 tx=0).stop 2024-06-10T00:33:43.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f2947323b80 1 -- 172.21.15.62:0/1091749212 shutdown_connections 2024-06-10T00:33:43.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f2947323b80 1 --2- 172.21.15.62:0/1091749212 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563481654c40 0x5634816560b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f2947323b80 1 --2- 172.21.15.62:0/1091749212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x563481644590 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f2947323b80 1 --2- 172.21.15.62:0/1091749212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 0x563481646d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f2947323b80 1 --2- 172.21.15.62:0/1091749212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634816405f0 0x563481644050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f2947323b80 1 -- 172.21.15.62:0/1091749212 >> 172.21.15.62:0/1091749212 conn(0x563481565c70 msgr2=0x5634816355a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:43.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f291dffb700 0 client.15669 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:43.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f2947323b80 1 -- 172.21.15.62:0/1091749212 shutdown_connections 2024-06-10T00:33:43.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f2947323b80 1 -- 172.21.15.62:0/1091749212 wait complete. 2024-06-10T00:33:43.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.769+0000 7f2947323b80 20 client.15669 trim_cache size 0 max 16384 2024-06-10T00:33:43.792 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:43.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.805+0000 7f2947323b80 1 Processor -- start 2024-06-10T00:33:43.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.805+0000 7f2947323b80 1 -- start start 2024-06-10T00:33:43.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.805+0000 7f2947323b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 0x56348163a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.805+0000 7f2947323b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634816405f0 0x56348162fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.805+0000 7f2947323b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x563481653d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.805+0000 7f2947323b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5634816543f0 con 0x563481640f10 2024-06-10T00:33:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.805+0000 7f2947323b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563481654ac0 con 0x5634816405f0 2024-06-10T00:33:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.805+0000 7f2947323b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563481655190 con 0x56348163aed0 2024-06-10T00:33:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.806+0000 7f2936d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x563481653d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.806+0000 7f2936d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x563481653d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56026/0 (socket says 172.21.15.62:56026) 2024-06-10T00:33:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.806+0000 7f2936d9d700 1 -- 172.21.15.62:0/1193575081 learned_addr learned my addr 172.21.15.62:0/1193575081 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.806+0000 7f2935d9b700 1 --2- 172.21.15.62:0/1193575081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634816405f0 0x56348162fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.806+0000 7f293659c700 1 --2- 172.21.15.62:0/1193575081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 0x56348163a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.806+0000 7f2936d9d700 1 -- 172.21.15.62:0/1193575081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 msgr2=0x56348163a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.806+0000 7f2936d9d700 1 --2- 172.21.15.62:0/1193575081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 0x56348163a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.806+0000 7f2936d9d700 1 -- 172.21.15.62:0/1193575081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634816405f0 msgr2=0x56348162fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.806+0000 7f2936d9d700 1 --2- 172.21.15.62:0/1193575081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634816405f0 0x56348162fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.806+0000 7f2936d9d700 1 -- 172.21.15.62:0/1193575081 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634816425c0 con 0x563481640f10 2024-06-10T00:33:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.807+0000 7f2936d9d700 1 --2- 172.21.15.62:0/1193575081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x563481653d80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2930000f80 tx=0x7f2930012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f0eeff49f43dce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:43.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.807+0000 7f291ffff700 1 -- 172.21.15.62:0/1193575081 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293001a040 con 0x563481640f10 2024-06-10T00:33:43.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.807+0000 7f291ffff700 1 -- 172.21.15.62:0/1193575081 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2930004a50 con 0x563481640f10 2024-06-10T00:33:43.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.807+0000 7f291ffff700 1 -- 172.21.15.62:0/1193575081 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293001f440 con 0x563481640f10 2024-06-10T00:33:43.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.808+0000 7f2947323b80 1 -- 172.21.15.62:0/1193575081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 msgr2=0x563481653d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:43.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.808+0000 7f2947323b80 1 --2- 172.21.15.62:0/1193575081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x563481653d80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2930000f80 tx=0x7f2930012520 comp rx=0 tx=0).stop 2024-06-10T00:33:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.812+0000 7f2947323b80 1 -- 172.21.15.62:0/1193575081 shutdown_connections 2024-06-10T00:33:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.812+0000 7f2947323b80 1 --2- 172.21.15.62:0/1193575081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563481640f10 0x563481653d80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.812+0000 7f2947323b80 1 --2- 172.21.15.62:0/1193575081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 0x56348163a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.812+0000 7f2947323b80 1 --2- 172.21.15.62:0/1193575081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5634816405f0 0x56348162fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.812+0000 7f2947323b80 1 -- 172.21.15.62:0/1193575081 >> 172.21.15.62:0/1193575081 conn(0x563481565c70 msgr2=0x563481653670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.812+0000 7f2947323b80 1 -- 172.21.15.62:0/1193575081 shutdown_connections 2024-06-10T00:33:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.812+0000 7f2947323b80 1 -- 172.21.15.62:0/1193575081 wait complete. 2024-06-10T00:33:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.812+0000 7f2947323b80 1 -- start start 2024-06-10T00:33:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.812+0000 7f2947323b80 1 Processor -- start 2024-06-10T00:33:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f2947323b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 0x56348165f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f2947323b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634816405f0 0x56348165fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f2947323b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563481640f10 0x56348165e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f2935d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634816405f0 0x56348165fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f2947323b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563481655f30 con 0x5634816405f0 2024-06-10T00:33:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f2947323b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563481655860 con 0x563481640f10 2024-06-10T00:33:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f2947323b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563481656600 con 0x56348163aed0 2024-06-10T00:33:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f2935d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634816405f0 0x56348165fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56032/0 (socket says 172.21.15.62:56032) 2024-06-10T00:33:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f2935d9b700 1 -- 172.21.15.62:0/3908632191 learned_addr learned my addr 172.21.15.62:0/3908632191 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f293659c700 1 --2- 172.21.15.62:0/3908632191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 0x56348165f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f2936d9d700 1 --2- 172.21.15.62:0/3908632191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563481640f10 0x56348165e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f2947323b80 10 client.0 _ll_register_callbacks cb 0x5634815268d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f293659c700 1 -- 172.21.15.62:0/3908632191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563481640f10 msgr2=0x56348165e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:43.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f293659c700 1 --2- 172.21.15.62:0/3908632191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563481640f10 0x56348165e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f293659c700 1 -- 172.21.15.62:0/3908632191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634816405f0 msgr2=0x56348165fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:43.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f293659c700 1 --2- 172.21.15.62:0/3908632191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634816405f0 0x56348165fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.813+0000 7f293659c700 1 -- 172.21.15.62:0/3908632191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56348165eff0 con 0x56348163aed0 2024-06-10T00:33:43.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.814+0000 7f2947323b80 1 -- 172.21.15.62:0/3908632191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2930004a50 con 0x56348163aed0 2024-06-10T00:33:43.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.814+0000 7f293659c700 1 --2- 172.21.15.62:0/3908632191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 0x56348165f860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f29280a8490 tx=0x7f292800cae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=739324c9b45dbf30 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:43.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.814+0000 7f291dffb700 10 client.25264 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:33:43.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.814+0000 7f291dffb700 1 -- 172.21.15.62:0/3908632191 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928084050 con 0x56348163aed0 2024-06-10T00:33:43.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.814+0000 7f291dffb700 1 -- 172.21.15.62:0/3908632191 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f29280d3250 con 0x56348163aed0 2024-06-10T00:33:43.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.814+0000 7f291dffb700 1 -- 172.21.15.62:0/3908632191 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f29280b8040 con 0x56348163aed0 2024-06-10T00:33:43.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.814+0000 7f2947323b80 10 client.25264 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:43.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.814+0000 7f2947323b80 1 -- 172.21.15.62:0/3908632191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5634816440e0 con 0x56348163aed0 2024-06-10T00:33:43.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.815+0000 7f291dffb700 1 -- 172.21.15.62:0/3908632191 <== mon.1 v2:172.21.15.138:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f292809f8c0 con 0x56348163aed0 2024-06-10T00:33:43.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.815+0000 7f2947323b80 10 client.25264 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:43.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.815+0000 7f2947323b80 10 client.25264 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T00:33:43.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.815+0000 7f2947323b80 20 client.25264 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:43.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.815+0000 7f2947323b80 1 --2- 172.21.15.62:0/3908632191 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5634816cd070 0x5634816cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:43.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.815+0000 7f2947323b80 4 client.25264 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:33:43.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.815+0000 7f2947323b80 1 -- 172.21.15.62:0/3908632191 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "status"}) -- 0x563481599cc0 con 0x5634816cd070 2024-06-10T00:33:43.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.815+0000 7f2935d9b700 1 --2- 172.21.15.62:0/3908632191 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5634816cd070 0x5634816cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:43.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.815+0000 7f2935d9b700 1 --2- 172.21.15.62:0/3908632191 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5634816cd070 0x5634816cbd80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56f4363b9a597f7d server_cookie=4161e4fafe79eb38 in_seq=0 out_seq=0 2024-06-10T00:33:43.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.815+0000 7f291dffb700 10 client.25264 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:43.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.816+0000 7f291dffb700 1 -- 172.21.15.62:0/3908632191 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x563481599cc0 con 0x5634816cd070 2024-06-10T00:33:43.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.816+0000 7f291dffb700 10 client.25264 handle_command_reply: tid=0 2024-06-10T00:33:43.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.816+0000 7f2947323b80 1 client.25264 shutdown 2024-06-10T00:33:43.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.816+0000 7f2947323b80 1 -- 172.21.15.62:0/3908632191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 msgr2=0x56348165f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:43.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.816+0000 7f2947323b80 1 --2- 172.21.15.62:0/3908632191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 0x56348165f860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f29280a8490 tx=0x7f292800cae0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.816+0000 7f2947323b80 1 -- 172.21.15.62:0/3908632191 shutdown_connections 2024-06-10T00:33:43.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.816+0000 7f2947323b80 1 --2- 172.21.15.62:0/3908632191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563481640f10 0x56348165e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.816+0000 7f2947323b80 1 --2- 172.21.15.62:0/3908632191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56348163aed0 0x56348165f860 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.816+0000 7f2947323b80 1 --2- 172.21.15.62:0/3908632191 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5634816cd070 0x5634816cbd80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.816+0000 7f2947323b80 1 --2- 172.21.15.62:0/3908632191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634816405f0 0x56348165fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.816+0000 7f291dffb700 0 client.25264 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:33:43.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.817+0000 7f2947323b80 1 -- 172.21.15.62:0/3908632191 >> 172.21.15.62:0/3908632191 conn(0x563481565c70 msgr2=0x56348162edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:43.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.817+0000 7f2947323b80 1 -- 172.21.15.62:0/3908632191 shutdown_connections 2024-06-10T00:33:43.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.817+0000 7f2947323b80 1 -- 172.21.15.62:0/3908632191 wait complete. 2024-06-10T00:33:43.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.817+0000 7f2947323b80 20 client.25264 trim_cache size 0 max 16384 2024-06-10T00:33:43.820 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:33:43.821 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "1353b2ce-26bf-11ef-bc9b-c7b262605968", 2024-06-10T00:33:43.821 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-06-10T00:33:43.821 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24464, 2024-06-10T00:33:43.821 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-06-10T00:33:43.821 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-06-10T00:33:43.821 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-06-10T00:33:43.821 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 304.17141677799998, 2024-06-10T00:33:43.821 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 30, 2024-06-10T00:33:43.821 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 100, 2024-06-10T00:33:43.821 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 99, 2024-06-10T00:33:43.821 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 321.89655196000001 2024-06-10T00:33:43.821 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:33:43.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.818+0000 7f293caa0700 1 -- 172.21.15.62:0/3614193798 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f29200a2ea0 msgr2=0x7f29200a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:43.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.818+0000 7f293caa0700 1 --2- 172.21.15.62:0/3614193798 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f29200a2ea0 0x7f29200a5360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2938079b60 tx=0x7f292c005fd0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.818+0000 7f293caa0700 1 -- 172.21.15.62:0/3614193798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2938102510 msgr2=0x7f293819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:43.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.818+0000 7f293caa0700 1 --2- 172.21.15.62:0/3614193798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2938102510 0x7f293819e830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2928012db0 tx=0x7f292800c5b0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.818+0000 7f293caa0700 1 -- 172.21.15.62:0/3614193798 shutdown_connections 2024-06-10T00:33:43.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.818+0000 7f293caa0700 1 --2- 172.21.15.62:0/3614193798 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f29200a2ea0 0x7f29200a5360 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.818+0000 7f293caa0700 1 --2- 172.21.15.62:0/3614193798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2938108520 0x7f29381989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.818+0000 7f293caa0700 1 --2- 172.21.15.62:0/3614193798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2938102510 0x7f293819e830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.818+0000 7f293caa0700 1 --2- 172.21.15.62:0/3614193798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2938102ed0 0x7f293819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:43.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.818+0000 7f293caa0700 1 -- 172.21.15.62:0/3614193798 >> 172.21.15.62:0/3614193798 conn(0x7f2938079560 msgr2=0x7f2938101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:43.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.818+0000 7f293caa0700 1 -- 172.21.15.62:0/3614193798 shutdown_connections 2024-06-10T00:33:43.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:43.818+0000 7f293caa0700 1 -- 172.21.15.62:0/3614193798 wait complete. 2024-06-10T00:33:45.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:45 smithi105 ceph-mon[36321]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:45.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:45 smithi062 ceph-mon[32390]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:45.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:45 smithi138 ceph-mon[36549]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:45.777 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-06-10T00:33:46.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.321+0000 7f1a50691700 1 -- 172.21.15.62:0/1833132961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a48107400 msgr2=0x7f1a481077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:46.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.321+0000 7f1a50691700 1 --2- 172.21.15.62:0/1833132961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a48107400 0x7f1a481077e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009990 tx=0x7f1a44009ca0 comp rx=0 tx=0).stop 2024-06-10T00:33:46.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.321+0000 7f1a50691700 1 -- 172.21.15.62:0/1833132961 shutdown_connections 2024-06-10T00:33:46.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.321+0000 7f1a50691700 1 --2- 172.21.15.62:0/1833132961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a48107400 0x7f1a481077e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:46.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.321+0000 7f1a50691700 1 --2- 172.21.15.62:0/1833132961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a48101d50 0x7f1a4810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:46.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.321+0000 7f1a50691700 1 --2- 172.21.15.62:0/1833132961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a48101390 0x7f1a48101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:46.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.321+0000 7f1a50691700 1 -- 172.21.15.62:0/1833132961 >> 172.21.15.62:0/1833132961 conn(0x7f1a48078340 msgr2=0x7f1a48078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:46.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.321+0000 7f1a50691700 1 -- 172.21.15.62:0/1833132961 shutdown_connections 2024-06-10T00:33:46.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.321+0000 7f1a50691700 1 -- 172.21.15.62:0/1833132961 wait complete. 2024-06-10T00:33:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.322+0000 7f1a50691700 1 Processor -- start 2024-06-10T00:33:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.322+0000 7f1a50691700 1 -- start start 2024-06-10T00:33:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.322+0000 7f1a50691700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a48101390 0x7f1a48197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.322+0000 7f1a50691700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a48101d50 0x7f1a48197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.322+0000 7f1a50691700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a48107400 0x7f1a481982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.322+0000 7f1a50691700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48111670 con 0x7f1a48101390 2024-06-10T00:33:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.322+0000 7f1a50691700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481114f0 con 0x7f1a48101d50 2024-06-10T00:33:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.322+0000 7f1a50691700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481117f0 con 0x7f1a48107400 2024-06-10T00:33:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.322+0000 7f1a4e42d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a48101390 0x7f1a48197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.322+0000 7f1a4e42d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a48101390 0x7f1a48197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56062/0 (socket says 172.21.15.62:56062) 2024-06-10T00:33:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.322+0000 7f1a4e42d700 1 -- 172.21.15.62:0/2290693026 learned_addr learned my addr 172.21.15.62:0/2290693026 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:46.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.322+0000 7f1a4dc2c700 1 --2- 172.21.15.62:0/2290693026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a48101d50 0x7f1a48197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.322+0000 7f1a4ec2e700 1 --2- 172.21.15.62:0/2290693026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a48107400 0x7f1a481982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.323+0000 7f1a4e42d700 1 -- 172.21.15.62:0/2290693026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a48107400 msgr2=0x7f1a481982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.323+0000 7f1a4e42d700 1 --2- 172.21.15.62:0/2290693026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a48107400 0x7f1a481982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.323+0000 7f1a4e42d700 1 -- 172.21.15.62:0/2290693026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a48101d50 msgr2=0x7f1a48197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.323+0000 7f1a4e42d700 1 --2- 172.21.15.62:0/2290693026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a48101d50 0x7f1a48197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.323+0000 7f1a4e42d700 1 -- 172.21.15.62:0/2290693026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a481a7060 con 0x7f1a48101390 2024-06-10T00:33:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.323+0000 7f1a4e42d700 1 --2- 172.21.15.62:0/2290693026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a48101390 0x7f1a48197820 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400bff0 tx=0x7f1a44005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.323+0000 7f1a3f7fe700 1 -- 172.21.15.62:0/2290693026 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44027070 con 0x7f1a48101390 2024-06-10T00:33:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.323+0000 7f1a50691700 1 -- 172.21.15.62:0/2290693026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a481a72f0 con 0x7f1a48101390 2024-06-10T00:33:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.323+0000 7f1a50691700 1 -- 172.21.15.62:0/2290693026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a481a7820 con 0x7f1a48101390 2024-06-10T00:33:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.323+0000 7f1a3f7fe700 1 -- 172.21.15.62:0/2290693026 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a44004150 con 0x7f1a48101390 2024-06-10T00:33:46.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.324+0000 7f1a3f7fe700 1 -- 172.21.15.62:0/2290693026 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4400f460 con 0x7f1a48101390 2024-06-10T00:33:46.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.324+0000 7f1a50691700 1 -- 172.21.15.62:0/2290693026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a48069180 con 0x7f1a48101390 2024-06-10T00:33:46.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.325+0000 7f1a3f7fe700 1 -- 172.21.15.62:0/2290693026 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f1a4402a070 con 0x7f1a48101390 2024-06-10T00:33:46.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.326+0000 7f1a3f7fe700 1 --2- 172.21.15.62:0/2290693026 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1a340a2ea0 0x7f1a340a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:46.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.326+0000 7f1a4dc2c700 1 --2- 172.21.15.62:0/2290693026 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1a340a2ea0 0x7f1a340a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:46.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.326+0000 7f1a3f7fe700 1 -- 172.21.15.62:0/2290693026 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f1a440d0870 con 0x7f1a48101390 2024-06-10T00:33:46.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.327+0000 7f1a4dc2c700 1 --2- 172.21.15.62:0/2290693026 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1a340a2ea0 0x7f1a340a5360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1a38005fd0 tx=0x7f1a3800a560 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:46.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.331+0000 7f1a3f7fe700 1 -- 172.21.15.62:0/2290693026 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1a4405e850 con 0x7f1a48101390 2024-06-10T00:33:46.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.596+0000 7f1a50691700 1 -- 172.21.15.62:0/2290693026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a4802dca0 con 0x7f1a48101390 2024-06-10T00:33:46.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.597+0000 7f1a3f7fe700 1 -- 172.21.15.62:0/2290693026 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f1a44013070 con 0x7f1a48101390 2024-06-10T00:33:46.597 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:33:46.597 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:32:47.783154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:33:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.600+0000 7f1a50691700 1 -- 172.21.15.62:0/2290693026 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1a340a2ea0 msgr2=0x7f1a340a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.600+0000 7f1a50691700 1 --2- 172.21.15.62:0/2290693026 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1a340a2ea0 0x7f1a340a5360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1a38005fd0 tx=0x7f1a3800a560 comp rx=0 tx=0).stop 2024-06-10T00:33:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.600+0000 7f1a50691700 1 -- 172.21.15.62:0/2290693026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a48101390 msgr2=0x7f1a48197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.600+0000 7f1a50691700 1 --2- 172.21.15.62:0/2290693026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a48101390 0x7f1a48197820 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400bff0 tx=0x7f1a44005640 comp rx=0 tx=0).stop 2024-06-10T00:33:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.600+0000 7f1a50691700 1 -- 172.21.15.62:0/2290693026 shutdown_connections 2024-06-10T00:33:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.600+0000 7f1a50691700 1 --2- 172.21.15.62:0/2290693026 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1a340a2ea0 0x7f1a340a5360 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.600+0000 7f1a50691700 1 --2- 172.21.15.62:0/2290693026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a48101390 0x7f1a48197820 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:46.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.600+0000 7f1a50691700 1 --2- 172.21.15.62:0/2290693026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a48107400 0x7f1a481982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:46.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.600+0000 7f1a50691700 1 --2- 172.21.15.62:0/2290693026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a48101d50 0x7f1a48197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:46.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.600+0000 7f1a50691700 1 -- 172.21.15.62:0/2290693026 >> 172.21.15.62:0/2290693026 conn(0x7f1a48078340 msgr2=0x7f1a480fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:46.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.600+0000 7f1a50691700 1 -- 172.21.15.62:0/2290693026 shutdown_connections 2024-06-10T00:33:46.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:46.600+0000 7f1a50691700 1 -- 172.21.15.62:0/2290693026 wait complete. 2024-06-10T00:33:46.601 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-06-10T00:33:47.359 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T00:33:47.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:47 smithi105 ceph-mon[36321]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-10T00:33:47.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:47 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2290693026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:33:47.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:47 smithi062 ceph-mon[32390]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-10T00:33:47.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:47 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2290693026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:33:47.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:47 smithi138 ceph-mon[36549]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-10T00:33:47.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:47 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2290693026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:33:47.632 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:49.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.413+0000 7fb5bbfee700 1 -- 172.21.15.62:0/784398370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b4102e90 msgr2=0x7fb5b410f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:49.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.413+0000 7fb5bbfee700 1 --2- 172.21.15.62:0/784398370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b4102e90 0x7fb5b410f350 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00b0d0 tx=0x7fb5ac00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.413+0000 7fb5bbfee700 1 -- 172.21.15.62:0/784398370 shutdown_connections 2024-06-10T00:33:49.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.413+0000 7fb5bbfee700 1 --2- 172.21.15.62:0/784398370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b4102e90 0x7fb5b410f350 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.413+0000 7fb5bbfee700 1 --2- 172.21.15.62:0/784398370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b4108580 0x7fb5b4108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.413+0000 7fb5bbfee700 1 --2- 172.21.15.62:0/784398370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b41024d0 0x7fb5b4102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.413+0000 7fb5bbfee700 1 -- 172.21.15.62:0/784398370 >> 172.21.15.62:0/784398370 conn(0x7fb5b40fdf60 msgr2=0x7fb5b4100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:49.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.413+0000 7fb5bbfee700 1 -- 172.21.15.62:0/784398370 shutdown_connections 2024-06-10T00:33:49.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.413+0000 7fb5bbfee700 1 -- 172.21.15.62:0/784398370 wait complete. 2024-06-10T00:33:49.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.414+0000 7fb5bbfee700 1 Processor -- start 2024-06-10T00:33:49.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.414+0000 7fb5bbfee700 1 -- start start 2024-06-10T00:33:49.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.414+0000 7fb5bbfee700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b41024d0 0x7fb5b419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.414+0000 7fb5bbfee700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b4102e90 0x7fb5b419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.414+0000 7fb5bbfee700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4108580 0x7fb5b4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.414+0000 7fb5bbfee700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41127f0 con 0x7fb5b41024d0 2024-06-10T00:33:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.414+0000 7fb5bbfee700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4112670 con 0x7fb5b4102e90 2024-06-10T00:33:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.414+0000 7fb5bbfee700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4112970 con 0x7fb5b4108580 2024-06-10T00:33:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.414+0000 7fb5b9d8a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b41024d0 0x7fb5b419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.414+0000 7fb5b9d8a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b41024d0 0x7fb5b419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44216/0 (socket says 172.21.15.62:44216) 2024-06-10T00:33:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.414+0000 7fb5b9d8a700 1 -- 172.21.15.62:0/1105168915 learned_addr learned my addr 172.21.15.62:0/1105168915 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.414+0000 7fb5ba58b700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4108580 0x7fb5b4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.414+0000 7fb5b9589700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b4102e90 0x7fb5b419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.414+0000 7fb5b9d8a700 1 -- 172.21.15.62:0/1105168915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4108580 msgr2=0x7fb5b4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.415+0000 7fb5b9d8a700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4108580 0x7fb5b4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.415+0000 7fb5b9d8a700 1 -- 172.21.15.62:0/1105168915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b4102e90 msgr2=0x7fb5b419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:49.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.415+0000 7fb5b9d8a700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b4102e90 0x7fb5b419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.415+0000 7fb5b9d8a700 1 -- 172.21.15.62:0/1105168915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b41990d0 con 0x7fb5b41024d0 2024-06-10T00:33:49.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.415+0000 7fb5b9589700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b4102e90 0x7fb5b419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:49.417 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:49 smithi062 ceph-mon[32390]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:49.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.415+0000 7fb5b9d8a700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b41024d0 0x7fb5b419e7b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000d950 tx=0x7fb5b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:49.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.415+0000 7fb5aaffd700 1 -- 172.21.15.62:0/1105168915 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0016070 con 0x7fb5b41024d0 2024-06-10T00:33:49.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.415+0000 7fb5bbfee700 1 -- 172.21.15.62:0/1105168915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b41993c0 con 0x7fb5b41024d0 2024-06-10T00:33:49.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.415+0000 7fb5aaffd700 1 -- 172.21.15.62:0/1105168915 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5b0009e20 con 0x7fb5b41024d0 2024-06-10T00:33:49.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.415+0000 7fb5aaffd700 1 -- 172.21.15.62:0/1105168915 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b000f620 con 0x7fb5b41024d0 2024-06-10T00:33:49.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.415+0000 7fb5bbfee700 1 -- 172.21.15.62:0/1105168915 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b41a5680 con 0x7fb5b41024d0 2024-06-10T00:33:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.418+0000 7fb5aaffd700 1 -- 172.21.15.62:0/1105168915 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fb5b0019070 con 0x7fb5b41024d0 2024-06-10T00:33:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.419+0000 7fb5aaffd700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb5a00a2e20 0x7fb5a00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.419+0000 7fb5aaffd700 1 -- 172.21.15.62:0/1105168915 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fb5b00cf570 con 0x7fb5b41024d0 2024-06-10T00:33:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.419+0000 7fb5b9589700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb5a00a2e20 0x7fb5a00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.420+0000 7fb5b9589700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb5a00a2e20 0x7fb5a00a52e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4107a90 tx=0x7fb5a400b2c0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.420+0000 7fb5c6871b80 1 Processor -- start 2024-06-10T00:33:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.420+0000 7fb5c6871b80 1 -- start start 2024-06-10T00:33:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.420+0000 7fb5c6871b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 0x561e2077c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.420+0000 7fb5c6871b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e2077c880 0x561e20776260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.420+0000 7fb5c6871b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e207767a0 0x561e20776bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.420+0000 7fb5c6871b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561e20668f60 con 0x561e207767a0 2024-06-10T00:33:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.420+0000 7fb5c6871b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561e206c7850 con 0x561e2077c880 2024-06-10T00:33:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.420+0000 7fb5c6871b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e2062f460 con 0x561e2077bf60 2024-06-10T00:33:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.421+0000 7fb5ba58b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e207767a0 0x561e20776bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.421+0000 7fb5ba58b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e207767a0 0x561e20776bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44218/0 (socket says 172.21.15.62:44218) 2024-06-10T00:33:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.421+0000 7fb5ba58b700 1 -- 172.21.15.62:0/1094880590 learned_addr learned my addr 172.21.15.62:0/1094880590 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.421+0000 7fb5b9589700 1 --2- 172.21.15.62:0/1094880590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e2077c880 0x561e20776260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.421+0000 7fb5b9d8a700 1 --2- 172.21.15.62:0/1094880590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 0x561e2077c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.421+0000 7fb5ba58b700 1 -- 172.21.15.62:0/1094880590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 msgr2=0x561e2077c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:49.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.421+0000 7fb5ba58b700 1 --2- 172.21.15.62:0/1094880590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 0x561e2077c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.421+0000 7fb5ba58b700 1 -- 172.21.15.62:0/1094880590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e2077c880 msgr2=0x561e20776260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:49.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.421+0000 7fb5ba58b700 1 --2- 172.21.15.62:0/1094880590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e2077c880 0x561e20776260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.421+0000 7fb5ba58b700 1 -- 172.21.15.62:0/1094880590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e206168d0 con 0x561e207767a0 2024-06-10T00:33:49.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.421+0000 7fb5ba58b700 1 --2- 172.21.15.62:0/1094880590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e207767a0 0x561e20776bc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009cb0 tx=0x7fb5ac006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba220a23b90a2686 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:49.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.421+0000 7fb59ffff700 1 -- 172.21.15.62:0/1094880590 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0092d0 con 0x561e207767a0 2024-06-10T00:33:49.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.421+0000 7fb59ffff700 1 -- 172.21.15.62:0/1094880590 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ac009470 con 0x561e207767a0 2024-06-10T00:33:49.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.421+0000 7fb59ffff700 1 -- 172.21.15.62:0/1094880590 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac005040 con 0x561e207767a0 2024-06-10T00:33:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.422+0000 7fb5c6871b80 1 -- 172.21.15.62:0/1094880590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e207767a0 msgr2=0x561e20776bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.422+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/1094880590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e207767a0 0x561e20776bc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009cb0 tx=0x7fb5ac006210 comp rx=0 tx=0).stop 2024-06-10T00:33:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.422+0000 7fb5c6871b80 1 -- 172.21.15.62:0/1094880590 shutdown_connections 2024-06-10T00:33:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.422+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/1094880590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e207767a0 0x561e20776bc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.422+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/1094880590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 0x561e2077c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.422+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/1094880590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e2077c880 0x561e20776260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.422+0000 7fb5c6871b80 1 -- 172.21.15.62:0/1094880590 >> 172.21.15.62:0/1094880590 conn(0x561e206a3080 msgr2=0x561e20771220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.422+0000 7fb5c6871b80 1 -- 172.21.15.62:0/1094880590 shutdown_connections 2024-06-10T00:33:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.423+0000 7fb5c6871b80 1 -- 172.21.15.62:0/1094880590 wait complete. 2024-06-10T00:33:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.423+0000 7fb5c6871b80 1 -- start start 2024-06-10T00:33:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.423+0000 7fb5c6871b80 1 Processor -- start 2024-06-10T00:33:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5c6871b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e207767a0 0x561e207825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5c6871b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 0x561e2077f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5c6871b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e2077c880 0x561e2077fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5c6871b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561e20667fc0 con 0x561e2077c880 2024-06-10T00:33:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5c6871b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561e206c3810 con 0x561e207767a0 2024-06-10T00:33:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5c6871b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e20787610 con 0x561e2077bf60 2024-06-10T00:33:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5ba58b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e2077c880 0x561e2077fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5ba58b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e2077c880 0x561e2077fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44228/0 (socket says 172.21.15.62:44228) 2024-06-10T00:33:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5ba58b700 1 -- 172.21.15.62:0/416636460 learned_addr learned my addr 172.21.15.62:0/416636460 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5b9589700 1 --2- 172.21.15.62:0/416636460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 0x561e2077f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5c6871b80 10 client.0 _ll_register_callbacks cb 0x561e2065d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5b9d8a700 1 --2- 172.21.15.62:0/416636460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e207767a0 0x561e207825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5b9589700 1 -- 172.21.15.62:0/416636460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e207767a0 msgr2=0x561e207825f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5b9589700 1 --2- 172.21.15.62:0/416636460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e207767a0 0x561e207825f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5b9589700 1 -- 172.21.15.62:0/416636460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e2077c880 msgr2=0x561e2077fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5b9589700 1 --2- 172.21.15.62:0/416636460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e2077c880 0x561e2077fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5b9589700 1 -- 172.21.15.62:0/416636460 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e2068d750 con 0x561e2077bf60 2024-06-10T00:33:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.424+0000 7fb5ba58b700 1 --2- 172.21.15.62:0/416636460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e2077c880 0x561e2077fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.425+0000 7fb5b9589700 1 --2- 172.21.15.62:0/416636460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 0x561e2077f8f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400b600 tx=0x7fb5a4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bfcd5e0588908f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.425+0000 7fb59d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:33:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.425+0000 7fb59d7fa700 1 -- 172.21.15.62:0/416636460 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4028670 con 0x561e2077bf60 2024-06-10T00:33:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.425+0000 7fb5c6871b80 1 -- 172.21.15.62:0/416636460 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5ac009470 con 0x561e2077bf60 2024-06-10T00:33:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.425+0000 7fb59d7fa700 1 -- 172.21.15.62:0/416636460 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fb5a4028810 con 0x561e2077bf60 2024-06-10T00:33:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.425+0000 7fb59d7fa700 1 -- 172.21.15.62:0/416636460 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb5a4031050 con 0x561e2077bf60 2024-06-10T00:33:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.425+0000 7fb5c6871b80 10 client.25279 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.425+0000 7fb5c6871b80 1 -- 172.21.15.62:0/416636460 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e20791e20 con 0x561e2077bf60 2024-06-10T00:33:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.425+0000 7fb59d7fa700 1 -- 172.21.15.62:0/416636460 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ac009470 con 0x561e2077bf60 2024-06-10T00:33:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.426+0000 7fb59d7fa700 1 -- 172.21.15.62:0/416636460 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb5a401e430 con 0x561e2077bf60 2024-06-10T00:33:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.426+0000 7fb5c6871b80 10 client.25279 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.426+0000 7fb5c6871b80 10 client.25279 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:33:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.426+0000 7fb5c6871b80 20 client.25279 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.426+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/416636460 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561e20794400 0x561e207967f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.426+0000 7fb5c6871b80 4 client.25279 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:33:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.426+0000 7fb5c6871b80 1 -- 172.21.15.62:0/416636460 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e206d4d50 con 0x561e20794400 2024-06-10T00:33:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.426+0000 7fb5b9d8a700 1 --2- 172.21.15.62:0/416636460 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561e20794400 0x561e207967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.426+0000 7fb59d7fa700 10 client.25279 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:33:49.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.426+0000 7fb5b9d8a700 1 --2- 172.21.15.62:0/416636460 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561e20794400 0x561e207967f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ff82c94fc8d5f95a server_cookie=2adf877946f0c340 in_seq=0 out_seq=0 2024-06-10T00:33:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.427+0000 7fb59d7fa700 1 -- 172.21.15.62:0/416636460 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561e206d4d50 con 0x561e20794400 2024-06-10T00:33:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.427+0000 7fb59d7fa700 10 client.25279 handle_command_reply: tid=0 2024-06-10T00:33:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.427+0000 7fb5c6871b80 1 client.25279 shutdown 2024-06-10T00:33:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.427+0000 7fb5c6871b80 1 -- 172.21.15.62:0/416636460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 msgr2=0x561e2077f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.427+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/416636460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 0x561e2077f8f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400b600 tx=0x7fb5a4004750 comp rx=0 tx=0).stop 2024-06-10T00:33:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.427+0000 7fb5c6871b80 1 -- 172.21.15.62:0/416636460 shutdown_connections 2024-06-10T00:33:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.427+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/416636460 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561e20794400 0x561e207967f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.427+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/416636460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e2077c880 0x561e2077fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.427+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/416636460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e207767a0 0x561e207825f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.427+0000 7fb59d7fa700 0 client.25279 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:33:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.427+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/416636460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 0x561e2077f8f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.427+0000 7fb5c6871b80 1 -- 172.21.15.62:0/416636460 >> 172.21.15.62:0/416636460 conn(0x561e206a3080 msgr2=0x561e20770700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.428+0000 7fb5c6871b80 1 -- 172.21.15.62:0/416636460 shutdown_connections 2024-06-10T00:33:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.428+0000 7fb5c6871b80 1 -- 172.21.15.62:0/416636460 wait complete. 2024-06-10T00:33:49.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.428+0000 7fb5c6871b80 20 client.25279 trim_cache size 0 max 16384 2024-06-10T00:33:49.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5c6871b80 1 Processor -- start 2024-06-10T00:33:49.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5c6871b80 1 -- start start 2024-06-10T00:33:49.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5c6871b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e207767a0 0x561e20776260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5c6871b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 0x561e2077d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5c6871b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e2077c880 0x561e20770e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5c6871b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41ac200 con 0x561e207767a0 2024-06-10T00:33:49.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5c6871b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41add40 con 0x561e2077c880 2024-06-10T00:33:49.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5c6871b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41ac8d0 con 0x561e2077bf60 2024-06-10T00:33:49.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5b9d8a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e207767a0 0x561e20776260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5b9d8a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e207767a0 0x561e20776260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44236/0 (socket says 172.21.15.62:44236) 2024-06-10T00:33:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5b9d8a700 1 -- 172.21.15.62:0/38705311 learned_addr learned my addr 172.21.15.62:0/38705311 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5b9589700 1 --2- 172.21.15.62:0/38705311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 0x561e2077d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5ba58b700 1 --2- 172.21.15.62:0/38705311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e2077c880 0x561e20770e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5b9d8a700 1 -- 172.21.15.62:0/38705311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 msgr2=0x561e2077d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5b9d8a700 1 --2- 172.21.15.62:0/38705311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 0x561e2077d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5b9d8a700 1 -- 172.21.15.62:0/38705311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e2077c880 msgr2=0x561e20770e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5b9d8a700 1 --2- 172.21.15.62:0/38705311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e2077c880 0x561e20770e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.444+0000 7fb5b9d8a700 1 -- 172.21.15.62:0/38705311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e2077c680 con 0x561e207767a0 2024-06-10T00:33:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.445+0000 7fb5b9d8a700 1 --2- 172.21.15.62:0/38705311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e207767a0 0x561e20776260 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00979d0 tx=0x7fb5b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79364b21d227ae0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.445+0000 7fb59f7fe700 1 -- 172.21.15.62:0/38705311 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00b3070 con 0x561e207767a0 2024-06-10T00:33:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.445+0000 7fb59f7fe700 1 -- 172.21.15.62:0/38705311 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5b00a41e0 con 0x561e207767a0 2024-06-10T00:33:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.445+0000 7fb59f7fe700 1 -- 172.21.15.62:0/38705311 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0020040 con 0x561e207767a0 2024-06-10T00:33:49.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.446+0000 7fb5c6871b80 1 -- 172.21.15.62:0/38705311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e207767a0 msgr2=0x561e20776260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:49.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.446+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/38705311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e207767a0 0x561e20776260 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00979d0 tx=0x7fb5b0000bc0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.446+0000 7fb5c6871b80 1 -- 172.21.15.62:0/38705311 shutdown_connections 2024-06-10T00:33:49.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.446+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/38705311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e207767a0 0x561e20776260 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.446+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/38705311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e2077c880 0x561e20770e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.446+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/38705311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077bf60 0x561e2077d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.446+0000 7fb5c6871b80 1 -- 172.21.15.62:0/38705311 >> 172.21.15.62:0/38705311 conn(0x561e206a3080 msgr2=0x561e207819b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:49.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.446+0000 7fb5c6871b80 1 -- 172.21.15.62:0/38705311 shutdown_connections 2024-06-10T00:33:49.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.446+0000 7fb5c6871b80 1 -- 172.21.15.62:0/38705311 wait complete. 2024-06-10T00:33:49.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.446+0000 7fb5c6871b80 1 -- start start 2024-06-10T00:33:49.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.446+0000 7fb5c6871b80 1 Processor -- start 2024-06-10T00:33:49.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5c6871b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e207767a0 0x561e2078d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5c6871b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e2077bf60 0x561e2079ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5c6871b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077c880 0x561e2079bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5c6871b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41ad670 con 0x561e2077bf60 2024-06-10T00:33:49.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5c6871b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41acfa0 con 0x561e207767a0 2024-06-10T00:33:49.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5c6871b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41aa1a0 con 0x561e2077c880 2024-06-10T00:33:49.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5c6871b80 10 client.0 _ll_register_callbacks cb 0x561e2065d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:49.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5b9589700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e2077bf60 0x561e2079ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5b9589700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e2077bf60 0x561e2079ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44246/0 (socket says 172.21.15.62:44246) 2024-06-10T00:33:49.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5b9589700 1 -- 172.21.15.62:0/2041854640 learned_addr learned my addr 172.21.15.62:0/2041854640 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:49.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5b9d8a700 1 --2- 172.21.15.62:0/2041854640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e207767a0 0x561e2078d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5ba58b700 1 --2- 172.21.15.62:0/2041854640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077c880 0x561e2079bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5b9589700 1 -- 172.21.15.62:0/2041854640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077c880 msgr2=0x561e2079bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:49.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5b9589700 1 --2- 172.21.15.62:0/2041854640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077c880 0x561e2079bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5b9589700 1 -- 172.21.15.62:0/2041854640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e207767a0 msgr2=0x561e2078d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:49.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5b9589700 1 --2- 172.21.15.62:0/2041854640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e207767a0 0x561e2078d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.447+0000 7fb5b9589700 1 -- 172.21.15.62:0/2041854640 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e2079c7b0 con 0x561e2077bf60 2024-06-10T00:33:49.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb5b9589700 1 --2- 172.21.15.62:0/2041854640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e2077bf60 0x561e2079ba30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x561e207728a0 tx=0x7fb5a4005b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459c5a76a17e17c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:49.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb59d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:33:49.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb59d7fa700 1 -- 172.21.15.62:0/2041854640 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4039020 con 0x561e2077bf60 2024-06-10T00:33:49.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb59d7fa700 1 -- 172.21.15.62:0/2041854640 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fb5a4005240 con 0x561e2077bf60 2024-06-10T00:33:49.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb5c6871b80 1 -- 172.21.15.62:0/2041854640 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5b00a41e0 con 0x561e2077bf60 2024-06-10T00:33:49.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb59d7fa700 1 -- 172.21.15.62:0/2041854640 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb5a4044050 con 0x561e2077bf60 2024-06-10T00:33:49.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb5c6871b80 10 client.15699 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:49.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb5c6871b80 1 -- 172.21.15.62:0/2041854640 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e207ff0c0 con 0x561e2077bf60 2024-06-10T00:33:49.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb59d7fa700 1 -- 172.21.15.62:0/2041854640 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5b00a41e0 con 0x561e2077bf60 2024-06-10T00:33:49.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb59d7fa700 1 -- 172.21.15.62:0/2041854640 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb5a4034040 con 0x561e2077bf60 2024-06-10T00:33:49.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb5c6871b80 10 client.15699 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:49.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb5c6871b80 10 client.15699 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:33:49.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb5c6871b80 20 client.15699 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:49.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/2041854640 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561e207ff820 0x561e20801c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:49.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.448+0000 7fb5c6871b80 4 client.15699 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:33:49.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.449+0000 7fb5c6871b80 1 -- 172.21.15.62:0/2041854640 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e206cf2d0 con 0x561e207ff820 2024-06-10T00:33:49.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.449+0000 7fb5b9d8a700 1 --2- 172.21.15.62:0/2041854640 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561e207ff820 0x561e20801c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:49.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.449+0000 7fb59d7fa700 10 client.15699 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:33:49.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.449+0000 7fb5b9d8a700 1 --2- 172.21.15.62:0/2041854640 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561e207ff820 0x561e20801c40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d2be524fcf2623c9 server_cookie=ff1af9768b77d936 in_seq=0 out_seq=0 2024-06-10T00:33:49.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.449+0000 7fb59d7fa700 1 -- 172.21.15.62:0/2041854640 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e206cf2d0 con 0x561e207ff820 2024-06-10T00:33:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.449+0000 7fb59d7fa700 10 client.15699 handle_command_reply: tid=0 2024-06-10T00:33:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.449+0000 7fb5c6871b80 1 client.15699 shutdown 2024-06-10T00:33:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.450+0000 7fb5c6871b80 1 -- 172.21.15.62:0/2041854640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e2077bf60 msgr2=0x561e2079ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.450+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/2041854640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e2077bf60 0x561e2079ba30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x561e207728a0 tx=0x7fb5a4005b80 comp rx=0 tx=0).stop 2024-06-10T00:33:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.450+0000 7fb5c6871b80 1 -- 172.21.15.62:0/2041854640 shutdown_connections 2024-06-10T00:33:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.450+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/2041854640 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561e207ff820 0x561e20801c40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.450+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/2041854640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e2077c880 0x561e2079bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.450+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/2041854640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e207767a0 0x561e2078d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.450+0000 7fb59d7fa700 0 client.15699 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:33:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.450+0000 7fb5c6871b80 1 --2- 172.21.15.62:0/2041854640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e2077bf60 0x561e2079ba30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.450+0000 7fb5c6871b80 1 -- 172.21.15.62:0/2041854640 >> 172.21.15.62:0/2041854640 conn(0x561e206a3080 msgr2=0x561e207819b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.450+0000 7fb5c6871b80 1 -- 172.21.15.62:0/2041854640 shutdown_connections 2024-06-10T00:33:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.450+0000 7fb5c6871b80 1 -- 172.21.15.62:0/2041854640 wait complete. 2024-06-10T00:33:49.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.450+0000 7fb5c6871b80 20 client.15699 trim_cache size 0 max 16384 2024-06-10T00:33:49.455 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:33:49.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.451+0000 7fb5bbfee700 1 -- 172.21.15.62:0/1105168915 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb5a00a2e20 msgr2=0x7fb5a00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:49.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.451+0000 7fb5bbfee700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb5a00a2e20 0x7fb5a00a52e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4107a90 tx=0x7fb5a400b2c0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.451+0000 7fb5bbfee700 1 -- 172.21.15.62:0/1105168915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b41024d0 msgr2=0x7fb5b419e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:49.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.451+0000 7fb5bbfee700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b41024d0 0x7fb5b419e7b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000d950 tx=0x7fb5b000dc60 comp rx=0 tx=0).stop 2024-06-10T00:33:49.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.451+0000 7fb5bbfee700 1 -- 172.21.15.62:0/1105168915 shutdown_connections 2024-06-10T00:33:49.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.451+0000 7fb5bbfee700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb5a00a2e20 0x7fb5a00a52e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.451+0000 7fb5bbfee700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5b41024d0 0x7fb5b419e7b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.451+0000 7fb5bbfee700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4108580 0x7fb5b4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.451+0000 7fb5bbfee700 1 --2- 172.21.15.62:0/1105168915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b4102e90 0x7fb5b419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:49.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.451+0000 7fb5bbfee700 1 -- 172.21.15.62:0/1105168915 >> 172.21.15.62:0/1105168915 conn(0x7fb5b40fdf60 msgr2=0x7fb5b40ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:49.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.451+0000 7fb5bbfee700 1 -- 172.21.15.62:0/1105168915 shutdown_connections 2024-06-10T00:33:49.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:49.451+0000 7fb5bbfee700 1 -- 172.21.15.62:0/1105168915 wait complete. 2024-06-10T00:33:49.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:49 smithi105 ceph-mon[36321]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:49.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:49 smithi138 ceph-mon[36549]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:50.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T00:33:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:51.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:51 smithi105 ceph-mon[36321]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:51.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:51 smithi062 ceph-mon[32390]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:51.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:51 smithi138 ceph-mon[36549]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:52.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.373+0000 7fd40b446700 1 -- 172.21.15.62:0/1675273564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd404100a80 msgr2=0x7fd404100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:52.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.373+0000 7fd40b446700 1 --2- 172.21.15.62:0/1675273564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd404100a80 0x7fd404100f00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009a10 tx=0x7fd3f4009d20 comp rx=0 tx=0).stop 2024-06-10T00:33:52.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.373+0000 7fd40b446700 1 -- 172.21.15.62:0/1675273564 shutdown_connections 2024-06-10T00:33:52.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.373+0000 7fd40b446700 1 --2- 172.21.15.62:0/1675273564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd404101440 0x7fd40410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.373+0000 7fd40b446700 1 --2- 172.21.15.62:0/1675273564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd404108510 0x7fd4041088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.373+0000 7fd40b446700 1 --2- 172.21.15.62:0/1675273564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd404100a80 0x7fd404100f00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.373+0000 7fd40b446700 1 -- 172.21.15.62:0/1675273564 >> 172.21.15.62:0/1675273564 conn(0x7fd404077a00 msgr2=0x7fd4040fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:52.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.373+0000 7fd40b446700 1 -- 172.21.15.62:0/1675273564 shutdown_connections 2024-06-10T00:33:52.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.373+0000 7fd40b446700 1 -- 172.21.15.62:0/1675273564 wait complete. 2024-06-10T00:33:52.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.374+0000 7fd40b446700 1 Processor -- start 2024-06-10T00:33:52.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.374+0000 7fd40b446700 1 -- start start 2024-06-10T00:33:52.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.374+0000 7fd40b446700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd404100a80 0x7fd40419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.374+0000 7fd40b446700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd404101440 0x7fd40419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.374+0000 7fd40b446700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd404108510 0x7fd404196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.374+0000 7fd40b446700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd404110590 con 0x7fd404100a80 2024-06-10T00:33:52.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.374+0000 7fd40b446700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd404110410 con 0x7fd404108510 2024-06-10T00:33:52.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.374+0000 7fd40b446700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd404110710 con 0x7fd404101440 2024-06-10T00:33:52.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.375+0000 7fd4091e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd404100a80 0x7fd40419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.375+0000 7fd4091e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd404100a80 0x7fd40419c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44256/0 (socket says 172.21.15.62:44256) 2024-06-10T00:33:52.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.375+0000 7fd4091e2700 1 -- 172.21.15.62:0/3869345413 learned_addr learned my addr 172.21.15.62:0/3869345413 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:52.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.375+0000 7fd4099e3700 1 --2- 172.21.15.62:0/3869345413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd404108510 0x7fd404196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.375+0000 7fd4091e2700 1 -- 172.21.15.62:0/3869345413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd404101440 msgr2=0x7fd40419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:52.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.375+0000 7fd4089e1700 1 --2- 172.21.15.62:0/3869345413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd404101440 0x7fd40419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.375+0000 7fd4091e2700 1 --2- 172.21.15.62:0/3869345413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd404101440 0x7fd40419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.375+0000 7fd4091e2700 1 -- 172.21.15.62:0/3869345413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd404108510 msgr2=0x7fd404196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:52.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.375+0000 7fd4091e2700 1 --2- 172.21.15.62:0/3869345413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd404108510 0x7fd404196890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.375+0000 7fd4091e2700 1 -- 172.21.15.62:0/3869345413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd404197070 con 0x7fd404100a80 2024-06-10T00:33:52.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.375+0000 7fd4091e2700 1 --2- 172.21.15.62:0/3869345413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd404100a80 0x7fd40419c6c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd40000d950 tx=0x7fd40000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:52.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.376+0000 7fd3fa7fc700 1 -- 172.21.15.62:0/3869345413 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400016070 con 0x7fd404100a80 2024-06-10T00:33:52.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.376+0000 7fd40b446700 1 -- 172.21.15.62:0/3869345413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd404197360 con 0x7fd404100a80 2024-06-10T00:33:52.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.376+0000 7fd40b446700 1 -- 172.21.15.62:0/3869345413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4041abd30 con 0x7fd404100a80 2024-06-10T00:33:52.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.376+0000 7fd3fa7fc700 1 -- 172.21.15.62:0/3869345413 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd400009b40 con 0x7fd404100a80 2024-06-10T00:33:52.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.377+0000 7fd3fa7fc700 1 -- 172.21.15.62:0/3869345413 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40000f620 con 0x7fd404100a80 2024-06-10T00:33:52.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.380+0000 7fd3fa7fc700 1 -- 172.21.15.62:0/3869345413 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fd400019070 con 0x7fd404100a80 2024-06-10T00:33:52.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.381+0000 7fd3fa7fc700 1 --2- 172.21.15.62:0/3869345413 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd3f00a2e20 0x7fd3f00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.381+0000 7fd415cc9b80 1 Processor -- start 2024-06-10T00:33:52.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.381+0000 7fd415cc9b80 1 -- start start 2024-06-10T00:33:52.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.381+0000 7fd4089e1700 1 --2- 172.21.15.62:0/3869345413 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd3f00a2e20 0x7fd3f00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.381+0000 7fd3fa7fc700 1 -- 172.21.15.62:0/3869345413 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fd4000cf940 con 0x7fd404100a80 2024-06-10T00:33:52.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd415cc9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c471ff340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd415cc9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471ff880 0x555c471f9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd415cc9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c471f9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd415cc9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555c470ebf60 con 0x555c471fef60 2024-06-10T00:33:52.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd415cc9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555c4714a850 con 0x555c471ff880 2024-06-10T00:33:52.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd415cc9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555c470b2460 con 0x555c471f97a0 2024-06-10T00:33:52.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd4089e1700 1 --2- 172.21.15.62:0/3869345413 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd3f00a2e20 0x7fd3f00a52e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4000c00 tx=0x7fd3f4010f90 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:52.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd4089e1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471ff880 0x555c471f9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd4099e3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c471f9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd4089e1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471ff880 0x555c471f9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:43248/0 (socket says 172.21.15.62:43248) 2024-06-10T00:33:52.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd4099e3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c471f9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:41524/0 (socket says 172.21.15.62:41524) 2024-06-10T00:33:52.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd4089e1700 1 -- 172.21.15.62:0/2997737383 learned_addr learned my addr 172.21.15.62:0/2997737383 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:52.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd4091e2700 1 --2- 172.21.15.62:0/2997737383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c471ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd4099e3700 1 -- 172.21.15.62:0/2997737383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471ff880 msgr2=0x555c471f9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:52.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd4099e3700 1 --2- 172.21.15.62:0/2997737383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471ff880 0x555c471f9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.382+0000 7fd4099e3700 1 -- 172.21.15.62:0/2997737383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 msgr2=0x555c471ff340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:52.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.383+0000 7fd4099e3700 1 --2- 172.21.15.62:0/2997737383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c471ff340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.383+0000 7fd4099e3700 1 -- 172.21.15.62:0/2997737383 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c470998d0 con 0x555c471f97a0 2024-06-10T00:33:52.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.383+0000 7fd4091e2700 1 --2- 172.21.15.62:0/2997737383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c471ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:33:52.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.383+0000 7fd4099e3700 1 --2- 172.21.15.62:0/2997737383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c471f9bc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009cd0 tx=0x7fd3fc007500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8dcf1f65fb1cf28 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:52.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.383+0000 7fd3ef7fe700 1 -- 172.21.15.62:0/2997737383 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc007d80 con 0x555c471f97a0 2024-06-10T00:33:52.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.383+0000 7fd3ef7fe700 1 -- 172.21.15.62:0/2997737383 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3fc004bb0 con 0x555c471f97a0 2024-06-10T00:33:52.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.383+0000 7fd3ef7fe700 1 -- 172.21.15.62:0/2997737383 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc014670 con 0x555c471f97a0 2024-06-10T00:33:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.384+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2997737383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 msgr2=0x555c471f9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.384+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/2997737383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c471f9bc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009cd0 tx=0x7fd3fc007500 comp rx=0 tx=0).stop 2024-06-10T00:33:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.384+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2997737383 shutdown_connections 2024-06-10T00:33:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.384+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/2997737383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c471ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.384+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/2997737383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c471f9bc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.384+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/2997737383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471ff880 0x555c471f9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.384+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2997737383 >> 172.21.15.62:0/2997737383 conn(0x555c47126080 msgr2=0x555c471f4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.384+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2997737383 shutdown_connections 2024-06-10T00:33:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.385+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2997737383 wait complete. 2024-06-10T00:33:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.385+0000 7fd415cc9b80 1 -- start start 2024-06-10T00:33:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.385+0000 7fd415cc9b80 1 Processor -- start 2024-06-10T00:33:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd415cc9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c47205550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd415cc9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c472028c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd415cc9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471ff880 0x555c47202e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd4089e1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c472028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd4089e1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c472028c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44288/0 (socket says 172.21.15.62:44288) 2024-06-10T00:33:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd4089e1700 1 -- 172.21.15.62:0/1249314733 learned_addr learned my addr 172.21.15.62:0/1249314733 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd4091e2700 1 --2- 172.21.15.62:0/1249314733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c47205550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd415cc9b80 1 -- 172.21.15.62:0/1249314733 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555c470eafc0 con 0x555c471fef60 2024-06-10T00:33:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd415cc9b80 1 -- 172.21.15.62:0/1249314733 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555c47146810 con 0x555c471ff880 2024-06-10T00:33:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd415cc9b80 1 -- 172.21.15.62:0/1249314733 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555c472062b0 con 0x555c471f97a0 2024-06-10T00:33:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd4099e3700 1 --2- 172.21.15.62:0/1249314733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471ff880 0x555c47202e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd415cc9b80 10 client.0 _ll_register_callbacks cb 0x555c470e05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd4099e3700 1 -- 172.21.15.62:0/1249314733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 msgr2=0x555c47205550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd4099e3700 1 --2- 172.21.15.62:0/1249314733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c47205550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd4099e3700 1 -- 172.21.15.62:0/1249314733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 msgr2=0x555c472028c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd4099e3700 1 --2- 172.21.15.62:0/1249314733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c472028c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.386+0000 7fd4099e3700 1 -- 172.21.15.62:0/1249314733 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c47110750 con 0x555c471ff880 2024-06-10T00:33:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.387+0000 7fd4089e1700 1 --2- 172.21.15.62:0/1249314733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c472028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.387+0000 7fd4099e3700 1 --2- 172.21.15.62:0/1249314733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471ff880 0x555c47202e00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc007900 tx=0x7fd3fc004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8984bc9a2ce3ab1e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:52.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.387+0000 7fd3ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:33:52.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.387+0000 7fd3ecff9700 1 -- 172.21.15.62:0/1249314733 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc032070 con 0x555c471ff880 2024-06-10T00:33:52.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.387+0000 7fd3ecff9700 1 -- 172.21.15.62:0/1249314733 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fd3fc005290 con 0x555c471ff880 2024-06-10T00:33:52.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.387+0000 7fd415cc9b80 1 -- 172.21.15.62:0/1249314733 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3fc004bb0 con 0x555c471ff880 2024-06-10T00:33:52.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.388+0000 7fd3ecff9700 1 -- 172.21.15.62:0/1249314733 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd3fc03e050 con 0x555c471ff880 2024-06-10T00:33:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.388+0000 7fd415cc9b80 10 client.25307 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.388+0000 7fd415cc9b80 1 -- 172.21.15.62:0/1249314733 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555c47210780 con 0x555c471ff880 2024-06-10T00:33:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.388+0000 7fd3ecff9700 1 -- 172.21.15.62:0/1249314733 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3fc02e930 con 0x555c471ff880 2024-06-10T00:33:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.388+0000 7fd3ecff9700 1 -- 172.21.15.62:0/1249314733 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd3fc037830 con 0x555c471ff880 2024-06-10T00:33:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.388+0000 7fd415cc9b80 10 client.25307 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.388+0000 7fd415cc9b80 10 client.25307 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:33:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.388+0000 7fd415cc9b80 20 client.25307 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.388+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/1249314733 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x555c47213050 0x555c47215470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.388+0000 7fd415cc9b80 4 client.25307 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:33:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.388+0000 7fd415cc9b80 1 -- 172.21.15.62:0/1249314733 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd3fc004bb0 con 0x555c47213050 2024-06-10T00:33:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.389+0000 7fd4091e2700 1 --2- 172.21.15.62:0/1249314733 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x555c47213050 0x555c47215470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.389+0000 7fd4091e2700 1 --2- 172.21.15.62:0/1249314733 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x555c47213050 0x555c47215470 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f15291ce027e5938 server_cookie=aef738b1157bb6d3 in_seq=0 out_seq=0 2024-06-10T00:33:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.389+0000 7fd3ecff9700 10 client.25307 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:33:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.390+0000 7fd3ecff9700 1 -- 172.21.15.62:0/1249314733 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fd3fc004bb0 con 0x555c47213050 2024-06-10T00:33:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.390+0000 7fd3ecff9700 10 client.25307 handle_command_reply: tid=0 2024-06-10T00:33:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.390+0000 7fd415cc9b80 1 client.25307 shutdown 2024-06-10T00:33:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.390+0000 7fd415cc9b80 1 -- 172.21.15.62:0/1249314733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471ff880 msgr2=0x555c47202e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.390+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/1249314733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471ff880 0x555c47202e00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc007900 tx=0x7fd3fc004f90 comp rx=0 tx=0).stop 2024-06-10T00:33:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.390+0000 7fd415cc9b80 1 -- 172.21.15.62:0/1249314733 shutdown_connections 2024-06-10T00:33:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.390+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/1249314733 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x555c47213050 0x555c47215470 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.390+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/1249314733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471ff880 0x555c47202e00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.390+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/1249314733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c47205550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.390+0000 7fd3ecff9700 0 client.25307 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:33:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.390+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/1249314733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c472028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.390+0000 7fd415cc9b80 1 -- 172.21.15.62:0/1249314733 >> 172.21.15.62:0/1249314733 conn(0x555c47126080 msgr2=0x555c471f3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.391+0000 7fd415cc9b80 1 -- 172.21.15.62:0/1249314733 shutdown_connections 2024-06-10T00:33:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.391+0000 7fd415cc9b80 1 -- 172.21.15.62:0/1249314733 wait complete. 2024-06-10T00:33:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.391+0000 7fd415cc9b80 20 client.25307 trim_cache size 0 max 16384 2024-06-10T00:33:52.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.421+0000 7fd415cc9b80 1 Processor -- start 2024-06-10T00:33:52.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.421+0000 7fd415cc9b80 1 -- start start 2024-06-10T00:33:52.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.421+0000 7fd415cc9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c471f9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.421+0000 7fd415cc9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471fef60 0x555c471f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.421+0000 7fd415cc9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471ff880 0x555c471f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd415cc9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4041ac530 con 0x555c471ff880 2024-06-10T00:33:52.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd415cc9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd4041ae070 con 0x555c471fef60 2024-06-10T00:33:52.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd415cc9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4041acc00 con 0x555c471f97a0 2024-06-10T00:33:52.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd4099e3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471ff880 0x555c471f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd4091e2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c471f9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd4091e2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c471f9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:41546/0 (socket says 172.21.15.62:41546) 2024-06-10T00:33:52.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd4091e2700 1 -- 172.21.15.62:0/2184822125 learned_addr learned my addr 172.21.15.62:0/2184822125 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:52.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd4089e1700 1 --2- 172.21.15.62:0/2184822125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471fef60 0x555c471f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd4091e2700 1 -- 172.21.15.62:0/2184822125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471fef60 msgr2=0x555c471f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:52.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd4091e2700 1 --2- 172.21.15.62:0/2184822125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471fef60 0x555c471f3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd4091e2700 1 -- 172.21.15.62:0/2184822125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471ff880 msgr2=0x555c471f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:52.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd4091e2700 1 --2- 172.21.15.62:0/2184822125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471ff880 0x555c471f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd4091e2700 1 -- 172.21.15.62:0/2184822125 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c47200e30 con 0x555c471f97a0 2024-06-10T00:33:52.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd4099e3700 1 --2- 172.21.15.62:0/2184822125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471ff880 0x555c471f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:52.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.422+0000 7fd4091e2700 1 --2- 172.21.15.62:0/2184822125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c471f9260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd40009b8d0 tx=0x7fd400010ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da7c04741e2457c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:52.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.423+0000 7fd3eeffd700 1 -- 172.21.15.62:0/2184822125 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4000afe60 con 0x555c471f97a0 2024-06-10T00:33:52.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.423+0000 7fd3eeffd700 1 -- 172.21.15.62:0/2184822125 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4000a45b0 con 0x555c471f97a0 2024-06-10T00:33:52.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.423+0000 7fd3eeffd700 1 -- 172.21.15.62:0/2184822125 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400020040 con 0x555c471f97a0 2024-06-10T00:33:52.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.423+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2184822125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 msgr2=0x555c471f9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:52.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.423+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/2184822125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c471f9260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd40009b8d0 tx=0x7fd400010ed0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.424+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2184822125 shutdown_connections 2024-06-10T00:33:52.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.424+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/2184822125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471ff880 0x555c471f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.424+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/2184822125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471f97a0 0x555c471f9260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.424+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/2184822125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471fef60 0x555c471f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.424+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2184822125 >> 172.21.15.62:0/2184822125 conn(0x555c47126080 msgr2=0x555c47202880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:52.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.424+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2184822125 shutdown_connections 2024-06-10T00:33:52.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.424+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2184822125 wait complete. 2024-06-10T00:33:52.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.424+0000 7fd415cc9b80 1 -- start start 2024-06-10T00:33:52.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.424+0000 7fd415cc9b80 1 Processor -- start 2024-06-10T00:33:52.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.425+0000 7fd415cc9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471f97a0 0x555c4720b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.425+0000 7fd415cc9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c4720bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.425+0000 7fd415cc9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471ff880 0x555c47219db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.425+0000 7fd415cc9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4041ad9a0 con 0x555c471fef60 2024-06-10T00:33:52.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.425+0000 7fd4089e1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c4720bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.425+0000 7fd415cc9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd4041ad2d0 con 0x555c471f97a0 2024-06-10T00:33:52.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.425+0000 7fd415cc9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd404072bd0 con 0x555c471ff880 2024-06-10T00:33:52.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.425+0000 7fd4089e1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c4720bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44314/0 (socket says 172.21.15.62:44314) 2024-06-10T00:33:52.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.425+0000 7fd4089e1700 1 -- 172.21.15.62:0/2407302655 learned_addr learned my addr 172.21.15.62:0/2407302655 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:52.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.425+0000 7fd4091e2700 1 --2- 172.21.15.62:0/2407302655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471f97a0 0x555c4720b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.425+0000 7fd415cc9b80 10 client.0 _ll_register_callbacks cb 0x555c470e05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:52.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.425+0000 7fd4099e3700 1 --2- 172.21.15.62:0/2407302655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471ff880 0x555c47219db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.426+0000 7fd4091e2700 1 -- 172.21.15.62:0/2407302655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471ff880 msgr2=0x555c47219db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:52.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.426+0000 7fd4091e2700 1 --2- 172.21.15.62:0/2407302655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471ff880 0x555c47219db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.426+0000 7fd4091e2700 1 -- 172.21.15.62:0/2407302655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 msgr2=0x555c4720bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:52.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.426+0000 7fd4091e2700 1 --2- 172.21.15.62:0/2407302655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c4720bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.426+0000 7fd4091e2700 1 -- 172.21.15.62:0/2407302655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c4721a2f0 con 0x555c471f97a0 2024-06-10T00:33:52.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.426+0000 7fd4089e1700 1 --2- 172.21.15.62:0/2407302655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c4720bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:52.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.426+0000 7fd4091e2700 1 --2- 172.21.15.62:0/2407302655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471f97a0 0x555c4720b5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x555c470b24b0 tx=0x7fd40009fdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bec80f07859ab7f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:52.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.426+0000 7fd3ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:33:52.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.426+0000 7fd3ecff9700 1 -- 172.21.15.62:0/2407302655 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4000b3070 con 0x555c471f97a0 2024-06-10T00:33:52.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.426+0000 7fd3ecff9700 1 -- 172.21.15.62:0/2407302655 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fd4000cced0 con 0x555c471f97a0 2024-06-10T00:33:52.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.426+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2407302655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4000a45b0 con 0x555c471f97a0 2024-06-10T00:33:52.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.426+0000 7fd3ecff9700 1 -- 172.21.15.62:0/2407302655 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd400020050 con 0x555c471f97a0 2024-06-10T00:33:52.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.426+0000 7fd415cc9b80 10 client.25313 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:52.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.426+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2407302655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555c4727d4d0 con 0x555c471f97a0 2024-06-10T00:33:52.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.427+0000 7fd3ecff9700 1 -- 172.21.15.62:0/2407302655 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd40009b940 con 0x555c471f97a0 2024-06-10T00:33:52.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.427+0000 7fd3ecff9700 1 -- 172.21.15.62:0/2407302655 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd4000860b0 con 0x555c471f97a0 2024-06-10T00:33:52.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.427+0000 7fd415cc9b80 10 client.25313 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:52.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.427+0000 7fd415cc9b80 10 client.25313 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:33:52.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.427+0000 7fd415cc9b80 20 client.25313 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:52.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.427+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/2407302655 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x555c4727dbb0 0x555c4727ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:52.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.427+0000 7fd415cc9b80 4 client.25313 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:33:52.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.427+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2407302655 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd4000a45b0 con 0x555c4727dbb0 2024-06-10T00:33:52.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.427+0000 7fd4089e1700 1 --2- 172.21.15.62:0/2407302655 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x555c4727dbb0 0x555c4727ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:52.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.428+0000 7fd4089e1700 1 --2- 172.21.15.62:0/2407302655 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x555c4727dbb0 0x555c4727ffd0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8ca1125992f5762a server_cookie=174029ad0ddfec82 in_seq=0 out_seq=0 2024-06-10T00:33:52.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.428+0000 7fd3ecff9700 10 client.25313 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:33:52.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.428+0000 7fd3ecff9700 1 -- 172.21.15.62:0/2407302655 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd4000a45b0 con 0x555c4727dbb0 2024-06-10T00:33:52.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.428+0000 7fd3ecff9700 10 client.25313 handle_command_reply: tid=0 2024-06-10T00:33:52.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.428+0000 7fd415cc9b80 1 client.25313 shutdown 2024-06-10T00:33:52.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.428+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2407302655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471f97a0 msgr2=0x555c4720b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:52.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.428+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/2407302655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471f97a0 0x555c4720b5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x555c470b24b0 tx=0x7fd40009fdb0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.428+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2407302655 shutdown_connections 2024-06-10T00:33:52.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.428+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/2407302655 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x555c4727dbb0 0x555c4727ffd0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.428+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/2407302655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c471ff880 0x555c47219db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.428+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/2407302655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c471f97a0 0x555c4720b5a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.428+0000 7fd415cc9b80 1 --2- 172.21.15.62:0/2407302655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c471fef60 0x555c4720bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.428+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2407302655 >> 172.21.15.62:0/2407302655 conn(0x555c47126080 msgr2=0x555c471ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:52.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.428+0000 7fd3ecff9700 0 client.25313 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:33:52.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.429+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2407302655 shutdown_connections 2024-06-10T00:33:52.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.429+0000 7fd415cc9b80 1 -- 172.21.15.62:0/2407302655 wait complete. 2024-06-10T00:33:52.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.429+0000 7fd415cc9b80 20 client.25313 trim_cache size 0 max 16384 2024-06-10T00:33:52.435 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:33:52.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.429+0000 7fd40b446700 1 -- 172.21.15.62:0/3869345413 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd3f00a2e20 msgr2=0x7fd3f00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:52.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.429+0000 7fd40b446700 1 --2- 172.21.15.62:0/3869345413 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd3f00a2e20 0x7fd3f00a52e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4000c00 tx=0x7fd3f4010f90 comp rx=0 tx=0).stop 2024-06-10T00:33:52.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.429+0000 7fd40b446700 1 -- 172.21.15.62:0/3869345413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd404100a80 msgr2=0x7fd40419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:52.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.429+0000 7fd40b446700 1 --2- 172.21.15.62:0/3869345413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd404100a80 0x7fd40419c6c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd40000d950 tx=0x7fd40000dd10 comp rx=0 tx=0).stop 2024-06-10T00:33:52.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.429+0000 7fd40b446700 1 -- 172.21.15.62:0/3869345413 shutdown_connections 2024-06-10T00:33:52.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.429+0000 7fd40b446700 1 --2- 172.21.15.62:0/3869345413 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd3f00a2e20 0x7fd3f00a52e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.429+0000 7fd40b446700 1 --2- 172.21.15.62:0/3869345413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd404100a80 0x7fd40419c6c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.429+0000 7fd40b446700 1 --2- 172.21.15.62:0/3869345413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd404108510 0x7fd404196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.429+0000 7fd40b446700 1 --2- 172.21.15.62:0/3869345413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd404101440 0x7fd40419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:52.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.429+0000 7fd40b446700 1 -- 172.21.15.62:0/3869345413 >> 172.21.15.62:0/3869345413 conn(0x7fd404077a00 msgr2=0x7fd4040ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:52.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.430+0000 7fd40b446700 1 -- 172.21.15.62:0/3869345413 shutdown_connections 2024-06-10T00:33:52.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:52.430+0000 7fd40b446700 1 -- 172.21.15.62:0/3869345413 wait complete. 2024-06-10T00:33:53.207 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T00:33:53.530 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:53.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:53 smithi105 ceph-mon[36321]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-10T00:33:53.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:53 smithi062 ceph-mon[32390]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-10T00:33:53.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:53 smithi138 ceph-mon[36549]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-10T00:33:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:55 smithi062 ceph-mon[32390]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:55.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.421+0000 7f27a7c20700 1 -- 172.21.15.62:0/2234116706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a0101d50 msgr2=0x7f27a010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:55.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.421+0000 7f27a7c20700 1 --2- 172.21.15.62:0/2234116706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a0101d50 0x7f27a010e240 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f279800b0a0 tx=0x7f279800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.422+0000 7f27a7c20700 1 -- 172.21.15.62:0/2234116706 shutdown_connections 2024-06-10T00:33:55.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.422+0000 7f27a7c20700 1 --2- 172.21.15.62:0/2234116706 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a0101d50 0x7f27a010e240 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.422+0000 7f27a7c20700 1 --2- 172.21.15.62:0/2234116706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27a0107400 0x7f27a01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.422+0000 7f27a7c20700 1 --2- 172.21.15.62:0/2234116706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27a0101390 0x7f27a0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.422+0000 7f27a7c20700 1 -- 172.21.15.62:0/2234116706 >> 172.21.15.62:0/2234116706 conn(0x7f27a0078340 msgr2=0x7f27a0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:55.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.422+0000 7f27a7c20700 1 -- 172.21.15.62:0/2234116706 shutdown_connections 2024-06-10T00:33:55.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.422+0000 7f27a7c20700 1 -- 172.21.15.62:0/2234116706 wait complete. 2024-06-10T00:33:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.423+0000 7f27a7c20700 1 Processor -- start 2024-06-10T00:33:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.423+0000 7f27a7c20700 1 -- start start 2024-06-10T00:33:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.423+0000 7f27a7c20700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27a0101390 0x7f27a0103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.423+0000 7f27a7c20700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27a0101d50 0x7f27a01035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.423+0000 7f27a7c20700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a0107400 0x7f27a0103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.423+0000 7f27a7c20700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01116a0 con 0x7f27a0107400 2024-06-10T00:33:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.423+0000 7f27a7c20700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0111520 con 0x7f27a0101390 2024-06-10T00:33:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.423+0000 7f27a7c20700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0111820 con 0x7f27a0101d50 2024-06-10T00:33:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.424+0000 7f27a61bd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a0107400 0x7f27a0103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.424+0000 7f27a61bd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a0107400 0x7f27a0103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44332/0 (socket says 172.21.15.62:44332) 2024-06-10T00:33:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.424+0000 7f27a61bd700 1 -- 172.21.15.62:0/2558985233 learned_addr learned my addr 172.21.15.62:0/2558985233 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.424+0000 7f27a51bb700 1 --2- 172.21.15.62:0/2558985233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27a0101d50 0x7f27a01035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.424+0000 7f27a59bc700 1 --2- 172.21.15.62:0/2558985233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27a0101390 0x7f27a0103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.424+0000 7f27a61bd700 1 -- 172.21.15.62:0/2558985233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27a0101d50 msgr2=0x7f27a01035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.424+0000 7f27a61bd700 1 --2- 172.21.15.62:0/2558985233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27a0101d50 0x7f27a01035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.424+0000 7f27a61bd700 1 -- 172.21.15.62:0/2558985233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27a0101390 msgr2=0x7f27a0103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.424+0000 7f27a61bd700 1 --2- 172.21.15.62:0/2558985233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27a0101390 0x7f27a0103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.424+0000 7f27a61bd700 1 -- 172.21.15.62:0/2558985233 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a01a70f0 con 0x7f27a0107400 2024-06-10T00:33:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.424+0000 7f27a61bd700 1 --2- 172.21.15.62:0/2558985233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a0107400 0x7f27a0103af0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f279800b780 tx=0x7f2798004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.425+0000 7f2796ffd700 1 -- 172.21.15.62:0/2558985233 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798021070 con 0x7f27a0107400 2024-06-10T00:33:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.425+0000 7f2796ffd700 1 -- 172.21.15.62:0/2558985233 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2798007b80 con 0x7f27a0107400 2024-06-10T00:33:55.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.425+0000 7f2796ffd700 1 -- 172.21.15.62:0/2558985233 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27980052a0 con 0x7f27a0107400 2024-06-10T00:33:55.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.425+0000 7f27a7c20700 1 -- 172.21.15.62:0/2558985233 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a01a7380 con 0x7f27a0107400 2024-06-10T00:33:55.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.425+0000 7f27a7c20700 1 -- 172.21.15.62:0/2558985233 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a01a77e0 con 0x7f27a0107400 2024-06-10T00:33:55.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.427+0000 7f2796ffd700 1 -- 172.21.15.62:0/2558985233 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f2798024070 con 0x7f27a0107400 2024-06-10T00:33:55.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.428+0000 7f2796ffd700 1 --2- 172.21.15.62:0/2558985233 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f278c0a2ea0 0x7f278c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.428+0000 7f27a59bc700 1 --2- 172.21.15.62:0/2558985233 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f278c0a2ea0 0x7f278c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.428+0000 7f2796ffd700 1 -- 172.21.15.62:0/2558985233 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f279800f020 con 0x7f27a0107400 2024-06-10T00:33:55.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.429+0000 7f27a59bc700 1 --2- 172.21.15.62:0/2558985233 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f278c0a2ea0 0x7f278c0a5360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f279c009be0 tx=0x7f279c009260 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:55.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.430+0000 7f27b24a3b80 1 Processor -- start 2024-06-10T00:33:55.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.430+0000 7f27b24a3b80 1 -- start start 2024-06-10T00:33:55.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.430+0000 7f27b24a3b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c76640 0x55c049c76a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.430+0000 7f27b24a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.430+0000 7f27b24a3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c70f20 0x55c049c71340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.430+0000 7f27b24a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c049bc2810 con 0x55c049c76f60 2024-06-10T00:33:55.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.430+0000 7f27b24a3b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c049b6a720 con 0x55c049c76640 2024-06-10T00:33:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.430+0000 7f27b24a3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c049bbe7d0 con 0x55c049c70f20 2024-06-10T00:33:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.431+0000 7f27a51bb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.431+0000 7f27a59bc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c76640 0x55c049c76a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.431+0000 7f27a51bb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44338/0 (socket says 172.21.15.62:44338) 2024-06-10T00:33:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.431+0000 7f27a51bb700 1 -- 172.21.15.62:0/2422741340 learned_addr learned my addr 172.21.15.62:0/2422741340 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.431+0000 7f27a61bd700 1 --2- 172.21.15.62:0/2422741340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c70f20 0x55c049c71340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.431+0000 7f27a59bc700 1 -- 172.21.15.62:0/2422741340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c70f20 msgr2=0x55c049c71340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.431+0000 7f27a59bc700 1 --2- 172.21.15.62:0/2422741340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c70f20 0x55c049c71340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.431+0000 7f27a59bc700 1 -- 172.21.15.62:0/2422741340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 msgr2=0x55c049c709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.431+0000 7f27a59bc700 1 --2- 172.21.15.62:0/2422741340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.431+0000 7f27a59bc700 1 -- 172.21.15.62:0/2422741340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c049b60f50 con 0x55c049c76640 2024-06-10T00:33:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.431+0000 7f27a51bb700 1 --2- 172.21.15.62:0/2422741340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:55.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.431+0000 7f27a61bd700 1 --2- 172.21.15.62:0/2422741340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c70f20 0x55c049c71340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:33:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.432+0000 7f27a59bc700 1 --2- 172.21.15.62:0/2422741340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c76640 0x55c049c76a20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f279c0117d0 tx=0x7f279c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9056e0862e40a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.432+0000 7f278bfff700 1 -- 172.21.15.62:0/2422741340 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c004020 con 0x55c049c76640 2024-06-10T00:33:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.432+0000 7f278bfff700 1 -- 172.21.15.62:0/2422741340 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f279c00e840 con 0x55c049c76640 2024-06-10T00:33:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.432+0000 7f278bfff700 1 -- 172.21.15.62:0/2422741340 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c003c20 con 0x55c049c76640 2024-06-10T00:33:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.433+0000 7f27b24a3b80 1 -- 172.21.15.62:0/2422741340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c76640 msgr2=0x55c049c76a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.433+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/2422741340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c76640 0x55c049c76a20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f279c0117d0 tx=0x7f279c014000 comp rx=0 tx=0).stop 2024-06-10T00:33:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.433+0000 7f27b24a3b80 1 -- 172.21.15.62:0/2422741340 shutdown_connections 2024-06-10T00:33:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.433+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/2422741340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c70f20 0x55c049c71340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.433+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/2422741340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c76640 0x55c049c76a20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.433+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/2422741340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.433+0000 7f27b24a3b80 1 -- 172.21.15.62:0/2422741340 >> 172.21.15.62:0/2422741340 conn(0x55c049b9bcc0 msgr2=0x55c049c6b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.433+0000 7f27b24a3b80 1 -- 172.21.15.62:0/2422741340 shutdown_connections 2024-06-10T00:33:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.433+0000 7f27b24a3b80 1 -- 172.21.15.62:0/2422741340 wait complete. 2024-06-10T00:33:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.434+0000 7f27b24a3b80 1 -- start start 2024-06-10T00:33:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.434+0000 7f27b24a3b80 1 Processor -- start 2024-06-10T00:33:55.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.434+0000 7f27b24a3b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c70f20 0x55c049c7cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.434+0000 7f27b24a3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76640 0x55c049c7a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.434+0000 7f27b24a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c7a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.434+0000 7f27b24a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c049c7d9d0 con 0x55c049c76f60 2024-06-10T00:33:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.434+0000 7f27b24a3b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c049b62bf0 con 0x55c049c70f20 2024-06-10T00:33:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27b24a3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c049c7db50 con 0x55c049c76640 2024-06-10T00:33:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27a61bd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c7a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27a61bd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c7a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44348/0 (socket says 172.21.15.62:44348) 2024-06-10T00:33:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27b24a3b80 10 client.0 _ll_register_callbacks cb 0x55c049b5c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27a61bd700 1 -- 172.21.15.62:0/488449532 learned_addr learned my addr 172.21.15.62:0/488449532 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27a51bb700 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76640 0x55c049c7a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27a59bc700 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c70f20 0x55c049c7cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27a51bb700 1 -- 172.21.15.62:0/488449532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c70f20 msgr2=0x55c049c7cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27a51bb700 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c70f20 0x55c049c7cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27a51bb700 1 -- 172.21.15.62:0/488449532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 msgr2=0x55c049c7a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27a51bb700 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c7a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27a51bb700 1 -- 172.21.15.62:0/488449532 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c049b118d0 con 0x55c049c76640 2024-06-10T00:33:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27a61bd700 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c7a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27a59bc700 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c70f20 0x55c049c7cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:33:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27a51bb700 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76640 0x55c049c7a010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f279000f8d0 tx=0x7f279000fbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64a13c7a188f98bb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.435+0000 7f27897fa700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:33:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.436+0000 7f27897fa700 1 -- 172.21.15.62:0/488449532 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790004320 con 0x55c049c76640 2024-06-10T00:33:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.436+0000 7f27897fa700 1 -- 172.21.15.62:0/488449532 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f27900044c0 con 0x55c049c76640 2024-06-10T00:33:55.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.436+0000 7f27b24a3b80 1 -- 172.21.15.62:0/488449532 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f279c00e840 con 0x55c049c76640 2024-06-10T00:33:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.436+0000 7f27897fa700 1 -- 172.21.15.62:0/488449532 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f279002d050 con 0x55c049c76640 2024-06-10T00:33:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.436+0000 7f27b24a3b80 10 client.25312 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.436+0000 7f27b24a3b80 1 -- 172.21.15.62:0/488449532 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c049c88380 con 0x55c049c76640 2024-06-10T00:33:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.436+0000 7f27897fa700 1 -- 172.21.15.62:0/488449532 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f279c00e840 con 0x55c049c76640 2024-06-10T00:33:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.437+0000 7f27897fa700 1 -- 172.21.15.62:0/488449532 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f279000ebf0 con 0x55c049c76640 2024-06-10T00:33:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.437+0000 7f27b24a3b80 10 client.25312 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.437+0000 7f27b24a3b80 10 client.25312 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:33:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.437+0000 7f27b24a3b80 20 client.25312 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.437+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55c049c8adf0 0x55c049c8d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.437+0000 7f27b24a3b80 4 client.25312 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:33:55.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.437+0000 7f27b24a3b80 1 -- 172.21.15.62:0/488449532 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c049bcfd10 con 0x55c049c8adf0 2024-06-10T00:33:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.437+0000 7f27a59bc700 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55c049c8adf0 0x55c049c8d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.438+0000 7f27897fa700 10 client.25312 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:33:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.438+0000 7f27a59bc700 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55c049c8adf0 0x55c049c8d290 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=effce0f9463c7ced server_cookie=3d01a14077fd6005 in_seq=0 out_seq=0 2024-06-10T00:33:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.438+0000 7f27897fa700 1 -- 172.21.15.62:0/488449532 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c049bcfd10 con 0x55c049c8adf0 2024-06-10T00:33:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.438+0000 7f27897fa700 10 client.25312 handle_command_reply: tid=0 2024-06-10T00:33:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.438+0000 7f27b24a3b80 1 client.25312 shutdown 2024-06-10T00:33:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.439+0000 7f27b24a3b80 1 -- 172.21.15.62:0/488449532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76640 msgr2=0x55c049c7a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.439+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76640 0x55c049c7a010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f279000f8d0 tx=0x7f279000fbe0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.439+0000 7f27b24a3b80 1 -- 172.21.15.62:0/488449532 shutdown_connections 2024-06-10T00:33:55.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.439+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c7a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.439+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55c049c8adf0 0x55c049c8d290 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.439+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c70f20 0x55c049c7cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.439+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/488449532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76640 0x55c049c7a010 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.439+0000 7f27b24a3b80 1 -- 172.21.15.62:0/488449532 >> 172.21.15.62:0/488449532 conn(0x55c049b9bcc0 msgr2=0x55c049c6f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:55.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.439+0000 7f27897fa700 0 client.25312 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:33:55.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.439+0000 7f27b24a3b80 1 -- 172.21.15.62:0/488449532 shutdown_connections 2024-06-10T00:33:55.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.440+0000 7f27b24a3b80 1 -- 172.21.15.62:0/488449532 wait complete. 2024-06-10T00:33:55.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.440+0000 7f27b24a3b80 20 client.25312 trim_cache size 0 max 16384 2024-06-10T00:33:55.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.463+0000 7f27b24a3b80 1 Processor -- start 2024-06-10T00:33:55.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.463+0000 7f27b24a3b80 1 -- start start 2024-06-10T00:33:55.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27b24a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c70f20 0x55c049c709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27b24a3b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c76640 0x55c049c79fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27b24a3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76f60 0x55c049c7a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27b24a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01ac270 con 0x55c049c70f20 2024-06-10T00:33:55.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27b24a3b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01addb0 con 0x55c049c76640 2024-06-10T00:33:55.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27b24a3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01ac940 con 0x55c049c76f60 2024-06-10T00:33:55.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27a59bc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c70f20 0x55c049c709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27a59bc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c70f20 0x55c049c709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44356/0 (socket says 172.21.15.62:44356) 2024-06-10T00:33:55.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27a59bc700 1 -- 172.21.15.62:0/911389239 learned_addr learned my addr 172.21.15.62:0/911389239 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:55.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27a61bd700 1 --2- 172.21.15.62:0/911389239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76f60 0x55c049c7a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27a51bb700 1 --2- 172.21.15.62:0/911389239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c76640 0x55c049c79fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27a61bd700 1 -- 172.21.15.62:0/911389239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c76640 msgr2=0x55c049c79fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27a61bd700 1 --2- 172.21.15.62:0/911389239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c76640 0x55c049c79fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27a61bd700 1 -- 172.21.15.62:0/911389239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c70f20 msgr2=0x55c049c709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27a61bd700 1 --2- 172.21.15.62:0/911389239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c70f20 0x55c049c709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27a61bd700 1 -- 172.21.15.62:0/911389239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c049c78580 con 0x55c049c76f60 2024-06-10T00:33:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27a51bb700 1 --2- 172.21.15.62:0/911389239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c76640 0x55c049c79fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.464+0000 7f27a59bc700 1 --2- 172.21.15.62:0/911389239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c70f20 0x55c049c709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.465+0000 7f27a61bd700 1 --2- 172.21.15.62:0/911389239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76f60 0x55c049c7a510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f279809cd50 tx=0x7f2798009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1d6a30b5d7d197f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.465+0000 7f278b7fe700 1 -- 172.21.15.62:0/911389239 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798007d30 con 0x55c049c76f60 2024-06-10T00:33:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.465+0000 7f278b7fe700 1 -- 172.21.15.62:0/911389239 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f279809d4c0 con 0x55c049c76f60 2024-06-10T00:33:55.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.465+0000 7f278b7fe700 1 -- 172.21.15.62:0/911389239 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279809b1a0 con 0x55c049c76f60 2024-06-10T00:33:55.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.466+0000 7f27b24a3b80 1 -- 172.21.15.62:0/911389239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76f60 msgr2=0x55c049c7a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:55.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.466+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/911389239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76f60 0x55c049c7a510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f279809cd50 tx=0x7f2798009040 comp rx=0 tx=0).stop 2024-06-10T00:33:55.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.466+0000 7f27b24a3b80 1 -- 172.21.15.62:0/911389239 shutdown_connections 2024-06-10T00:33:55.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.466+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/911389239 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c70f20 0x55c049c709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.466+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/911389239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76f60 0x55c049c7a510 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.466+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/911389239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c76640 0x55c049c79fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.466+0000 7f27b24a3b80 1 -- 172.21.15.62:0/911389239 >> 172.21.15.62:0/911389239 conn(0x55c049b9bcc0 msgr2=0x55c049c78900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:55.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.466+0000 7f27b24a3b80 1 -- 172.21.15.62:0/911389239 shutdown_connections 2024-06-10T00:33:55.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.466+0000 7f27b24a3b80 1 -- 172.21.15.62:0/911389239 wait complete. 2024-06-10T00:33:55.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.467+0000 7f27b24a3b80 1 -- start start 2024-06-10T00:33:55.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.467+0000 7f27b24a3b80 1 Processor -- start 2024-06-10T00:33:55.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.467+0000 7f27b24a3b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c70f20 0x55c049c933b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.467+0000 7f27b24a3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76640 0x55c049c82e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.467+0000 7f27b24a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c83470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.467+0000 7f27b24a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01ad6e0 con 0x55c049c76f60 2024-06-10T00:33:55.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.467+0000 7f27b24a3b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01ad010 con 0x55c049c70f20 2024-06-10T00:33:55.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.467+0000 7f27b24a3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01aa210 con 0x55c049c76640 2024-06-10T00:33:55.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.467+0000 7f27b24a3b80 10 client.0 _ll_register_callbacks cb 0x55c049b5c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:55.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.468+0000 7f27a61bd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c83470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.468+0000 7f27a59bc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c70f20 0x55c049c933b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.468+0000 7f27a61bd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c83470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44358/0 (socket says 172.21.15.62:44358) 2024-06-10T00:33:55.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.468+0000 7f27a61bd700 1 -- 172.21.15.62:0/50740579 learned_addr learned my addr 172.21.15.62:0/50740579 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:55.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.468+0000 7f27a51bb700 1 --2- 172.21.15.62:0/50740579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76640 0x55c049c82e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.468+0000 7f27a59bc700 1 -- 172.21.15.62:0/50740579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76640 msgr2=0x55c049c82e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:55.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.468+0000 7f27a59bc700 1 --2- 172.21.15.62:0/50740579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76640 0x55c049c82e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.468+0000 7f27a59bc700 1 -- 172.21.15.62:0/50740579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 msgr2=0x55c049c83470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:55.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.468+0000 7f27a59bc700 1 --2- 172.21.15.62:0/50740579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c83470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.468+0000 7f27a59bc700 1 -- 172.21.15.62:0/50740579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c049c91b90 con 0x55c049c70f20 2024-06-10T00:33:55.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.468+0000 7f27a61bd700 1 --2- 172.21.15.62:0/50740579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c83470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:55.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.468+0000 7f27a59bc700 1 --2- 172.21.15.62:0/50740579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c70f20 0x55c049c933b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f279c000c00 tx=0x7f279c005180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b908d96703e5586 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:55.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.468+0000 7f27897fa700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:33:55.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.468+0000 7f27897fa700 1 -- 172.21.15.62:0/50740579 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c039060 con 0x55c049c70f20 2024-06-10T00:33:55.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.469+0000 7f27897fa700 1 -- 172.21.15.62:0/50740579 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f279c00e7c0 con 0x55c049c70f20 2024-06-10T00:33:55.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.469+0000 7f27b24a3b80 1 -- 172.21.15.62:0/50740579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f279809d4c0 con 0x55c049c70f20 2024-06-10T00:33:55.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.469+0000 7f27897fa700 1 -- 172.21.15.62:0/50740579 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f279c00e9c0 con 0x55c049c70f20 2024-06-10T00:33:55.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.469+0000 7f27b24a3b80 10 client.25328 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:55.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.469+0000 7f27b24a3b80 1 -- 172.21.15.62:0/50740579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c049cf4df0 con 0x55c049c70f20 2024-06-10T00:33:55.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.470+0000 7f27897fa700 1 -- 172.21.15.62:0/50740579 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f279809d4c0 con 0x55c049c70f20 2024-06-10T00:33:55.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.470+0000 7f27897fa700 1 -- 172.21.15.62:0/50740579 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f279c032440 con 0x55c049c70f20 2024-06-10T00:33:55.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.470+0000 7f27b24a3b80 10 client.25328 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:55.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.470+0000 7f27b24a3b80 10 client.25328 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:33:55.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.470+0000 7f27b24a3b80 20 client.25328 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:55.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.470+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/50740579 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55c049cf5180 0x55c049cf75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:55.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.470+0000 7f27b24a3b80 4 client.25328 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:33:55.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.470+0000 7f27b24a3b80 1 -- 172.21.15.62:0/50740579 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c049bca290 con 0x55c049cf5180 2024-06-10T00:33:55.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.470+0000 7f27a51bb700 1 --2- 172.21.15.62:0/50740579 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55c049cf5180 0x55c049cf75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:55.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.471+0000 7f27a51bb700 1 --2- 172.21.15.62:0/50740579 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55c049cf5180 0x55c049cf75a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=784706e51d10f0d0 server_cookie=923c5837269ad8a in_seq=0 out_seq=0 2024-06-10T00:33:55.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.471+0000 7f27897fa700 10 client.25328 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:33:55.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.471+0000 7f27897fa700 1 -- 172.21.15.62:0/50740579 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c049bca290 con 0x55c049cf5180 2024-06-10T00:33:55.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.471+0000 7f27897fa700 10 client.25328 handle_command_reply: tid=0 2024-06-10T00:33:55.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.471+0000 7f27b24a3b80 1 client.25328 shutdown 2024-06-10T00:33:55.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.471+0000 7f27b24a3b80 1 -- 172.21.15.62:0/50740579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c70f20 msgr2=0x55c049c933b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:55.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.471+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/50740579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c70f20 0x55c049c933b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f279c000c00 tx=0x7f279c005180 comp rx=0 tx=0).stop 2024-06-10T00:33:55.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.471+0000 7f27b24a3b80 1 -- 172.21.15.62:0/50740579 shutdown_connections 2024-06-10T00:33:55.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.471+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/50740579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c049c76f60 0x55c049c83470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.472+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/50740579 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55c049cf5180 0x55c049cf75a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.472+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/50740579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c049c70f20 0x55c049c933b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.472+0000 7f27897fa700 0 client.25328 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:33:55.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.472+0000 7f27b24a3b80 1 --2- 172.21.15.62:0/50740579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c049c76640 0x55c049c82e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.472+0000 7f27b24a3b80 1 -- 172.21.15.62:0/50740579 >> 172.21.15.62:0/50740579 conn(0x55c049b9bcc0 msgr2=0x55c049c78840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:55.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.472+0000 7f27b24a3b80 1 -- 172.21.15.62:0/50740579 shutdown_connections 2024-06-10T00:33:55.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.472+0000 7f27b24a3b80 1 -- 172.21.15.62:0/50740579 wait complete. 2024-06-10T00:33:55.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.472+0000 7f27b24a3b80 20 client.25328 trim_cache size 0 max 16384 2024-06-10T00:33:55.475 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:33:55.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.473+0000 7f27a7c20700 1 -- 172.21.15.62:0/2558985233 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f278c0a2ea0 msgr2=0x7f278c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:55.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.473+0000 7f27a7c20700 1 --2- 172.21.15.62:0/2558985233 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f278c0a2ea0 0x7f278c0a5360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f279c009be0 tx=0x7f279c009260 comp rx=0 tx=0).stop 2024-06-10T00:33:55.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.473+0000 7f27a7c20700 1 -- 172.21.15.62:0/2558985233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a0107400 msgr2=0x7f27a0103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:55.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.473+0000 7f27a7c20700 1 --2- 172.21.15.62:0/2558985233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a0107400 0x7f27a0103af0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f279800b780 tx=0x7f2798004d00 comp rx=0 tx=0).stop 2024-06-10T00:33:55.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.473+0000 7f27a7c20700 1 -- 172.21.15.62:0/2558985233 shutdown_connections 2024-06-10T00:33:55.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.473+0000 7f27a7c20700 1 --2- 172.21.15.62:0/2558985233 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f278c0a2ea0 0x7f278c0a5360 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.473+0000 7f27a7c20700 1 --2- 172.21.15.62:0/2558985233 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27a0107400 0x7f27a0103af0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.473+0000 7f27a7c20700 1 --2- 172.21.15.62:0/2558985233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27a0101390 0x7f27a0103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.473+0000 7f27a7c20700 1 --2- 172.21.15.62:0/2558985233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27a0101d50 0x7f27a01035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:55.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.473+0000 7f27a7c20700 1 -- 172.21.15.62:0/2558985233 >> 172.21.15.62:0/2558985233 conn(0x7f27a0078340 msgr2=0x7f27a01005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:55.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.474+0000 7f27a7c20700 1 -- 172.21.15.62:0/2558985233 shutdown_connections 2024-06-10T00:33:55.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:55.474+0000 7f27a7c20700 1 -- 172.21.15.62:0/2558985233 wait complete. 2024-06-10T00:33:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:55 smithi105 ceph-mon[36321]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:55 smithi138 ceph-mon[36549]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:56.389 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T00:33:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:57 smithi105 ceph-mon[36321]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-10T00:33:57.542 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:57 smithi062 ceph-mon[32390]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-10T00:33:57.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:57 smithi138 ceph-mon[36549]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-10T00:33:58.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.513+0000 7f74c69ac700 1 -- 172.21.15.62:0/3355743295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006ab60 msgr2=0x7f74c006afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:58.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.513+0000 7f74c69ac700 1 --2- 172.21.15.62:0/3355743295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006ab60 0x7f74c006afe0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009990 tx=0x7f74b4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.513+0000 7f74c69ac700 1 -- 172.21.15.62:0/3355743295 shutdown_connections 2024-06-10T00:33:58.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.513+0000 7f74c69ac700 1 --2- 172.21.15.62:0/3355743295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74c006b520 0x7f74c010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.513+0000 7f74c69ac700 1 --2- 172.21.15.62:0/3355743295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74c0108530 0x7f74c0108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.513+0000 7f74c69ac700 1 --2- 172.21.15.62:0/3355743295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006ab60 0x7f74c006afe0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.514+0000 7f74c69ac700 1 -- 172.21.15.62:0/3355743295 >> 172.21.15.62:0/3355743295 conn(0x7f74c00783d0 msgr2=0x7f74c00787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.514+0000 7f74c69ac700 1 -- 172.21.15.62:0/3355743295 shutdown_connections 2024-06-10T00:33:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.514+0000 7f74c69ac700 1 -- 172.21.15.62:0/3355743295 wait complete. 2024-06-10T00:33:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.515+0000 7f74c69ac700 1 Processor -- start 2024-06-10T00:33:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.515+0000 7f74c69ac700 1 -- start start 2024-06-10T00:33:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.515+0000 7f74c69ac700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006ab60 0x7f74c019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.515+0000 7f74c69ac700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74c006b520 0x7f74c019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.515+0000 7f74c69ac700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74c0108530 0x7f74c0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.515+0000 7f74c69ac700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0112790 con 0x7f74c006ab60 2024-06-10T00:33:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.516+0000 7f74c69ac700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0112610 con 0x7f74c0108530 2024-06-10T00:33:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.516+0000 7f74c69ac700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0112910 con 0x7f74c006b520 2024-06-10T00:33:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.516+0000 7f74bffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006ab60 0x7f74c019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.516+0000 7f74bffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006ab60 0x7f74c019e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35116/0 (socket says 172.21.15.62:35116) 2024-06-10T00:33:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.516+0000 7f74bffff700 1 -- 172.21.15.62:0/1143776170 learned_addr learned my addr 172.21.15.62:0/1143776170 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.516+0000 7f74bf7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74c006b520 0x7f74c019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.516+0000 7f74c4f49700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74c0108530 0x7f74c0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.516+0000 7f74bf7fe700 1 -- 172.21.15.62:0/1143776170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74c0108530 msgr2=0x7f74c0198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.516+0000 7f74bf7fe700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74c0108530 0x7f74c0198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.516+0000 7f74bf7fe700 1 -- 172.21.15.62:0/1143776170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006ab60 msgr2=0x7f74c019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.516+0000 7f74bf7fe700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006ab60 0x7f74c019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.516+0000 7f74bf7fe700 1 -- 172.21.15.62:0/1143776170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c0199230 con 0x7f74c006b520 2024-06-10T00:33:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.516+0000 7f74c4f49700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74c0108530 0x7f74c0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.516+0000 7f74bffff700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006ab60 0x7f74c019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.517+0000 7f74bf7fe700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74c006b520 0x7f74c019ee50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f74b4006010 tx=0x7f74b4003890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.517+0000 7f74bd7fa700 1 -- 172.21.15.62:0/1143776170 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4027070 con 0x7f74c006b520 2024-06-10T00:33:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.517+0000 7f74bd7fa700 1 -- 172.21.15.62:0/1143776170 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74b40056d0 con 0x7f74c006b520 2024-06-10T00:33:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.517+0000 7f74c69ac700 1 -- 172.21.15.62:0/1143776170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c01994c0 con 0x7f74c006b520 2024-06-10T00:33:58.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.517+0000 7f74bd7fa700 1 -- 172.21.15.62:0/1143776170 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4022bd0 con 0x7f74c006b520 2024-06-10T00:33:58.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.517+0000 7f74c69ac700 1 -- 172.21.15.62:0/1143776170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c01a57e0 con 0x7f74c006b520 2024-06-10T00:33:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.520+0000 7f74bd7fa700 1 -- 172.21.15.62:0/1143776170 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f74b402a070 con 0x7f74c006b520 2024-06-10T00:33:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.521+0000 7f74bd7fa700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f74a80a2e20 0x7f74a80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.522+0000 7f74bd7fa700 1 -- 172.21.15.62:0/1143776170 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f74b40d0cc0 con 0x7f74c006b520 2024-06-10T00:33:58.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.522+0000 7f74bffff700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f74a80a2e20 0x7f74a80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.522+0000 7f74d122fb80 1 Processor -- start 2024-06-10T00:33:58.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.522+0000 7f74d122fb80 1 -- start start 2024-06-10T00:33:58.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.522+0000 7f74d122fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31792f60 0x559d31793340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.522+0000 7f74bffff700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f74a80a2e20 0x7f74a80a52e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f74b0010420 tx=0x7f74b0007480 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:58.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.522+0000 7f74d122fb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31793880 0x559d3178d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.522+0000 7f74d122fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d3178d7a0 0x559d3178dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.522+0000 7f74d122fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559d3167ff60 con 0x559d3178d7a0 2024-06-10T00:33:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.522+0000 7f74d122fb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559d316de850 con 0x559d31792f60 2024-06-10T00:33:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.523+0000 7f74d122fb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559d31646460 con 0x559d31793880 2024-06-10T00:33:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.523+0000 7f74bffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31792f60 0x559d31793340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.523+0000 7f74bffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31792f60 0x559d31793340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:49276/0 (socket says 172.21.15.62:49276) 2024-06-10T00:33:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.523+0000 7f74bffff700 1 -- 172.21.15.62:0/1890451854 learned_addr learned my addr 172.21.15.62:0/1890451854 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.523+0000 7f74c4f49700 1 --2- 172.21.15.62:0/1890451854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d3178d7a0 0x559d3178dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.523+0000 7f74bffff700 1 -- 172.21.15.62:0/1890451854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31793880 msgr2=0x559d3178d260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:33:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.523+0000 7f74bffff700 1 --2- 172.21.15.62:0/1890451854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31793880 0x559d3178d260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.523+0000 7f74bffff700 1 -- 172.21.15.62:0/1890451854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d3178d7a0 msgr2=0x559d3178dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.523+0000 7f74bffff700 1 --2- 172.21.15.62:0/1890451854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d3178d7a0 0x559d3178dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.523+0000 7f74bffff700 1 -- 172.21.15.62:0/1890451854 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d3162d8d0 con 0x559d31792f60 2024-06-10T00:33:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.523+0000 7f74c4f49700 1 --2- 172.21.15.62:0/1890451854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d3178d7a0 0x559d3178dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.524+0000 7f74bffff700 1 --2- 172.21.15.62:0/1890451854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31792f60 0x559d31793340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f74b0011760 tx=0x7f74b0013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae405df94eb4968d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:58.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.524+0000 7f74a67fc700 1 -- 172.21.15.62:0/1890451854 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b001e420 con 0x559d31792f60 2024-06-10T00:33:58.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.524+0000 7f74a67fc700 1 -- 172.21.15.62:0/1890451854 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74b001eaa0 con 0x559d31792f60 2024-06-10T00:33:58.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.524+0000 7f74a67fc700 1 -- 172.21.15.62:0/1890451854 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b001ccb0 con 0x559d31792f60 2024-06-10T00:33:58.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.525+0000 7f74d122fb80 1 -- 172.21.15.62:0/1890451854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31792f60 msgr2=0x559d31793340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:58.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.525+0000 7f74d122fb80 1 --2- 172.21.15.62:0/1890451854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31792f60 0x559d31793340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f74b0011760 tx=0x7f74b0013000 comp rx=0 tx=0).stop 2024-06-10T00:33:58.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.525+0000 7f74d122fb80 1 -- 172.21.15.62:0/1890451854 shutdown_connections 2024-06-10T00:33:58.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.525+0000 7f74d122fb80 1 --2- 172.21.15.62:0/1890451854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d3178d7a0 0x559d3178dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.525+0000 7f74d122fb80 1 --2- 172.21.15.62:0/1890451854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31792f60 0x559d31793340 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.525+0000 7f74d122fb80 1 --2- 172.21.15.62:0/1890451854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31793880 0x559d3178d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.525+0000 7f74d122fb80 1 -- 172.21.15.62:0/1890451854 >> 172.21.15.62:0/1890451854 conn(0x559d316ba080 msgr2=0x559d31788220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:58.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.526+0000 7f74d122fb80 1 -- 172.21.15.62:0/1890451854 shutdown_connections 2024-06-10T00:33:58.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.526+0000 7f74d122fb80 1 -- 172.21.15.62:0/1890451854 wait complete. 2024-06-10T00:33:58.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.526+0000 7f74d122fb80 1 -- start start 2024-06-10T00:33:58.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.526+0000 7f74d122fb80 1 Processor -- start 2024-06-10T00:33:58.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.527+0000 7f74d122fb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d3178d7a0 0x559d317996f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.527+0000 7f74d122fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31792f60 0x559d317969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.527+0000 7f74d122fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d31793880 0x559d31796f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.527+0000 7f74bf7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31792f60 0x559d317969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.527+0000 7f74bf7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31792f60 0x559d317969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:49288/0 (socket says 172.21.15.62:49288) 2024-06-10T00:33:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.527+0000 7f74c4f49700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d31793880 0x559d31796f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.527+0000 7f74c4f49700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d31793880 0x559d31796f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35130/0 (socket says 172.21.15.62:35130) 2024-06-10T00:33:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.527+0000 7f74bffff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d3178d7a0 0x559d317996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.527+0000 7f74bffff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d3178d7a0 0x559d317996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:53084/0 (socket says 172.21.15.62:53084) 2024-06-10T00:33:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.527+0000 7f74bf7fe700 1 -- 172.21.15.62:0/4228506872 learned_addr learned my addr 172.21.15.62:0/4228506872 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.527+0000 7f74d122fb80 1 -- 172.21.15.62:0/4228506872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559d3167efc0 con 0x559d31793880 2024-06-10T00:33:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.527+0000 7f74d122fb80 1 -- 172.21.15.62:0/4228506872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559d316da810 con 0x559d31792f60 2024-06-10T00:33:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.527+0000 7f74d122fb80 1 -- 172.21.15.62:0/4228506872 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559d3179e670 con 0x559d3178d7a0 2024-06-10T00:33:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.528+0000 7f74bf7fe700 1 -- 172.21.15.62:0/4228506872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d3178d7a0 msgr2=0x559d317996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.528+0000 7f74bf7fe700 1 --2- 172.21.15.62:0/4228506872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d3178d7a0 0x559d317996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.528+0000 7f74bf7fe700 1 -- 172.21.15.62:0/4228506872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d31793880 msgr2=0x559d31796f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:58.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.528+0000 7f74bf7fe700 1 --2- 172.21.15.62:0/4228506872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d31793880 0x559d31796f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.528+0000 7f74bf7fe700 1 -- 172.21.15.62:0/4228506872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d316a4750 con 0x559d31792f60 2024-06-10T00:33:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.528+0000 7f74c4f49700 1 --2- 172.21.15.62:0/4228506872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d31793880 0x559d31796f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.528+0000 7f74bffff700 1 --2- 172.21.15.62:0/4228506872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d3178d7a0 0x559d317996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:33:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.528+0000 7f74d122fb80 10 client.0 _ll_register_callbacks cb 0x559d316745b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.528+0000 7f74bf7fe700 1 --2- 172.21.15.62:0/4228506872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31792f60 0x559d317969f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f74b4000c00 tx=0x7f74b40ce4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee1d85da700b470b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.528+0000 7f74d122fb80 1 -- 172.21.15.62:0/4228506872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b001eaa0 con 0x559d31792f60 2024-06-10T00:33:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.528+0000 7f749f7fe700 10 client.25340 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:33:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.528+0000 7f749f7fe700 1 -- 172.21.15.62:0/4228506872 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b40b5e60 con 0x559d31792f60 2024-06-10T00:33:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.529+0000 7f749f7fe700 1 -- 172.21.15.62:0/4228506872 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f74b409c140 con 0x559d31792f60 2024-06-10T00:33:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.529+0000 7f749f7fe700 1 -- 172.21.15.62:0/4228506872 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f74b409c340 con 0x559d31792f60 2024-06-10T00:33:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.529+0000 7f74d122fb80 10 client.25340 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.529+0000 7f74d122fb80 1 -- 172.21.15.62:0/4228506872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=30}) -- 0x559d317a88d0 con 0x559d31792f60 2024-06-10T00:33:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.529+0000 7f749f7fe700 1 -- 172.21.15.62:0/4228506872 <== mon.2 v2:172.21.15.105:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f74b407bb10 con 0x559d31792f60 2024-06-10T00:33:58.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.529+0000 7f74d122fb80 10 client.25340 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:58.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.529+0000 7f74d122fb80 10 client.25340 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:33:58.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.529+0000 7f74d122fb80 20 client.25340 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:58.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.529+0000 7f74d122fb80 1 --2- 172.21.15.62:0/4228506872 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x559d317aaff0 0x559d317ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.529+0000 7f74d122fb80 4 client.25340 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:33:58.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.529+0000 7f74d122fb80 1 -- 172.21.15.62:0/4228506872 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d316ebd50 con 0x559d317aaff0 2024-06-10T00:33:58.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.530+0000 7f74bffff700 1 --2- 172.21.15.62:0/4228506872 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x559d317aaff0 0x559d317ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.530+0000 7f749f7fe700 10 client.25340 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:33:58.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.530+0000 7f74bffff700 1 --2- 172.21.15.62:0/4228506872 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x559d317aaff0 0x559d317ad4b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b796cfaf3de1a0bd server_cookie=6cb298b84fc84693 in_seq=0 out_seq=0 2024-06-10T00:33:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.531+0000 7f749f7fe700 1 -- 172.21.15.62:0/4228506872 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559d316ebd50 con 0x559d317aaff0 2024-06-10T00:33:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.531+0000 7f749f7fe700 10 client.25340 handle_command_reply: tid=0 2024-06-10T00:33:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.531+0000 7f74d122fb80 1 client.25340 shutdown 2024-06-10T00:33:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.531+0000 7f74d122fb80 1 -- 172.21.15.62:0/4228506872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31792f60 msgr2=0x559d317969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.531+0000 7f74d122fb80 1 --2- 172.21.15.62:0/4228506872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31792f60 0x559d317969f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f74b4000c00 tx=0x7f74b40ce4d0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.532+0000 7f74d122fb80 1 -- 172.21.15.62:0/4228506872 shutdown_connections 2024-06-10T00:33:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.532+0000 7f74d122fb80 1 --2- 172.21.15.62:0/4228506872 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x559d317aaff0 0x559d317ad4b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.532+0000 7f749f7fe700 0 client.25340 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:33:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.532+0000 7f74d122fb80 1 --2- 172.21.15.62:0/4228506872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d31793880 0x559d31796f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.532+0000 7f74d122fb80 1 --2- 172.21.15.62:0/4228506872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d3178d7a0 0x559d317996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.532+0000 7f74d122fb80 1 --2- 172.21.15.62:0/4228506872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31792f60 0x559d317969f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.532+0000 7f74d122fb80 1 -- 172.21.15.62:0/4228506872 >> 172.21.15.62:0/4228506872 conn(0x559d316ba080 msgr2=0x559d31787700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.532+0000 7f74d122fb80 1 -- 172.21.15.62:0/4228506872 shutdown_connections 2024-06-10T00:33:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.532+0000 7f74d122fb80 1 -- 172.21.15.62:0/4228506872 wait complete. 2024-06-10T00:33:58.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.532+0000 7f74d122fb80 20 client.25340 trim_cache size 0 max 16384 2024-06-10T00:33:58.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.548+0000 7f74d122fb80 1 Processor -- start 2024-06-10T00:33:58.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.548+0000 7f74d122fb80 1 -- start start 2024-06-10T00:33:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.548+0000 7f74d122fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d3178d7a0 0x559d3178d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.548+0000 7f74d122fb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31792f60 0x559d317943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.548+0000 7f74d122fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31793880 0x559d31799230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.548+0000 7f74d122fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01ac280 con 0x559d3178d7a0 2024-06-10T00:33:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.548+0000 7f74d122fb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01addc0 con 0x559d31793880 2024-06-10T00:33:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.548+0000 7f74d122fb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01ac950 con 0x559d31792f60 2024-06-10T00:33:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.549+0000 7f74bffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d3178d7a0 0x559d3178d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.549+0000 7f74bffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d3178d7a0 0x559d3178d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35142/0 (socket says 172.21.15.62:35142) 2024-06-10T00:33:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.549+0000 7f74bffff700 1 -- 172.21.15.62:0/2335157181 learned_addr learned my addr 172.21.15.62:0/2335157181 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.549+0000 7f74bf7fe700 1 --2- 172.21.15.62:0/2335157181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31792f60 0x559d317943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.549+0000 7f74c4f49700 1 --2- 172.21.15.62:0/2335157181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31793880 0x559d31799230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.549+0000 7f74bf7fe700 1 -- 172.21.15.62:0/2335157181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31793880 msgr2=0x559d31799230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:58.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.549+0000 7f74bf7fe700 1 --2- 172.21.15.62:0/2335157181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31793880 0x559d31799230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.549+0000 7f74bf7fe700 1 -- 172.21.15.62:0/2335157181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d3178d7a0 msgr2=0x559d3178d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.549+0000 7f74bf7fe700 1 --2- 172.21.15.62:0/2335157181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d3178d7a0 0x559d3178d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.549+0000 7f74bf7fe700 1 -- 172.21.15.62:0/2335157181 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d31784810 con 0x559d31792f60 2024-06-10T00:33:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.549+0000 7f74bffff700 1 --2- 172.21.15.62:0/2335157181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d3178d7a0 0x559d3178d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.549+0000 7f74bf7fe700 1 --2- 172.21.15.62:0/2335157181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31792f60 0x559d317943d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f74b409cc80 tx=0x7f74b401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82748ca5d9aba9b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.550+0000 7f74a5ffb700 1 -- 172.21.15.62:0/2335157181 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b40b5e60 con 0x559d31792f60 2024-06-10T00:33:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.550+0000 7f74a5ffb700 1 -- 172.21.15.62:0/2335157181 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74b001eaa0 con 0x559d31792f60 2024-06-10T00:33:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.550+0000 7f74a5ffb700 1 -- 172.21.15.62:0/2335157181 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b40a10a0 con 0x559d31792f60 2024-06-10T00:33:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.551+0000 7f74d122fb80 1 -- 172.21.15.62:0/2335157181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31792f60 msgr2=0x559d317943d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.551+0000 7f74d122fb80 1 --2- 172.21.15.62:0/2335157181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31792f60 0x559d317943d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f74b409cc80 tx=0x7f74b401a000 comp rx=0 tx=0).stop 2024-06-10T00:33:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.551+0000 7f74d122fb80 1 -- 172.21.15.62:0/2335157181 shutdown_connections 2024-06-10T00:33:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.551+0000 7f74d122fb80 1 --2- 172.21.15.62:0/2335157181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d3178d7a0 0x559d3178d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.551+0000 7f74d122fb80 1 --2- 172.21.15.62:0/2335157181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d31793880 0x559d31799230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.551+0000 7f74d122fb80 1 --2- 172.21.15.62:0/2335157181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31792f60 0x559d317943d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.551+0000 7f74d122fb80 1 -- 172.21.15.62:0/2335157181 >> 172.21.15.62:0/2335157181 conn(0x559d316ba080 msgr2=0x559d31793cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:58.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.551+0000 7f74d122fb80 1 -- 172.21.15.62:0/2335157181 shutdown_connections 2024-06-10T00:33:58.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.551+0000 7f74d122fb80 1 -- 172.21.15.62:0/2335157181 wait complete. 2024-06-10T00:33:58.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.552+0000 7f74d122fb80 1 -- start start 2024-06-10T00:33:58.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.552+0000 7f74d122fb80 1 Processor -- start 2024-06-10T00:33:58.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.552+0000 7f74d122fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d3178d7a0 0x559d317a3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.552+0000 7f74d122fb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31792f60 0x559d317a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.552+0000 7f74d122fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d31793880 0x559d317b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74bffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d3178d7a0 0x559d317a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74c4f49700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d31793880 0x559d317b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74bf7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31792f60 0x559d317a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74bffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d3178d7a0 0x559d317a3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:49304/0 (socket says 172.21.15.62:49304) 2024-06-10T00:33:58.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74c4f49700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d31793880 0x559d317b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35152/0 (socket says 172.21.15.62:35152) 2024-06-10T00:33:58.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74bffff700 1 -- 172.21.15.62:0/3847154025 learned_addr learned my addr 172.21.15.62:0/3847154025 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:33:58.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74d122fb80 1 -- 172.21.15.62:0/3847154025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01ad6f0 con 0x559d31793880 2024-06-10T00:33:58.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74d122fb80 1 -- 172.21.15.62:0/3847154025 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01ad020 con 0x559d3178d7a0 2024-06-10T00:33:58.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74d122fb80 1 -- 172.21.15.62:0/3847154025 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01aa220 con 0x559d31792f60 2024-06-10T00:33:58.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74c4f49700 1 -- 172.21.15.62:0/3847154025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31792f60 msgr2=0x559d317a3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:58.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74c4f49700 1 --2- 172.21.15.62:0/3847154025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31792f60 0x559d317a3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74c4f49700 1 -- 172.21.15.62:0/3847154025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d3178d7a0 msgr2=0x559d317a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:33:58.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74c4f49700 1 --2- 172.21.15.62:0/3847154025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d3178d7a0 0x559d317a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74c4f49700 1 -- 172.21.15.62:0/3847154025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d317b22f0 con 0x559d31793880 2024-06-10T00:33:58.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.553+0000 7f74bf7fe700 1 --2- 172.21.15.62:0/3847154025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31792f60 0x559d317a3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:33:58.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.554+0000 7f74c4f49700 1 --2- 172.21.15.62:0/3847154025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d31793880 0x559d317b1db0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x559d31782a10 tx=0x7f74b8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cd1f142c333804 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:33:58.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.554+0000 7f749f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:33:58.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.554+0000 7f74d122fb80 10 client.0 _ll_register_callbacks cb 0x559d316745b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:33:58.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.554+0000 7f74d122fb80 1 -- 172.21.15.62:0/3847154025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b001eaa0 con 0x559d31793880 2024-06-10T00:33:58.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.554+0000 7f749f7fe700 1 -- 172.21.15.62:0/3847154025 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8004bb0 con 0x559d31793880 2024-06-10T00:33:58.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.554+0000 7f749f7fe700 1 -- 172.21.15.62:0/3847154025 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f74b8004d50 con 0x559d31793880 2024-06-10T00:33:58.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.554+0000 7f749f7fe700 1 -- 172.21.15.62:0/3847154025 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f74b801dba0 con 0x559d31793880 2024-06-10T00:33:58.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.554+0000 7f74d122fb80 10 client.15750 fetch_fsmap learned FSMap version 30 2024-06-10T00:33:58.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.554+0000 7f74d122fb80 1 -- 172.21.15.62:0/3847154025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({fsmap=30}) -- 0x559d3179a380 con 0x559d31793880 2024-06-10T00:33:58.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.555+0000 7f749f7fe700 1 -- 172.21.15.62:0/3847154025 <== mon.0 v2:172.21.15.62:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f74b80043b0 con 0x559d31793880 2024-06-10T00:33:58.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.555+0000 7f74d122fb80 10 client.15750 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:33:58.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.555+0000 7f74d122fb80 10 client.15750 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:33:58.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.555+0000 7f74d122fb80 20 client.15750 populate_metadata read hostname 'smithi062' 2024-06-10T00:33:58.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.555+0000 7f74d122fb80 1 --2- 172.21.15.62:0/3847154025 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x559d317aaff0 0x559d3179ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:33:58.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.555+0000 7f74d122fb80 4 client.15750 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:33:58.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.555+0000 7f74d122fb80 1 -- 172.21.15.62:0/3847154025 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d316e62d0 con 0x559d317aaff0 2024-06-10T00:33:58.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.555+0000 7f74bffff700 1 --2- 172.21.15.62:0/3847154025 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x559d317aaff0 0x559d3179ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:33:58.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.556+0000 7f74bffff700 1 --2- 172.21.15.62:0/3847154025 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x559d317aaff0 0x559d3179ca70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2db2f01ec699b67d server_cookie=3f93554c453d9220 in_seq=0 out_seq=0 2024-06-10T00:33:58.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.556+0000 7f749f7fe700 10 client.15750 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:33:58.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.556+0000 7f749f7fe700 1 -- 172.21.15.62:0/3847154025 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d316e62d0 con 0x559d317aaff0 2024-06-10T00:33:58.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.556+0000 7f749f7fe700 10 client.15750 handle_command_reply: tid=0 2024-06-10T00:33:58.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.556+0000 7f74d122fb80 1 client.15750 shutdown 2024-06-10T00:33:58.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.556+0000 7f74d122fb80 1 -- 172.21.15.62:0/3847154025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d31793880 msgr2=0x559d317b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:58.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.556+0000 7f74d122fb80 1 --2- 172.21.15.62:0/3847154025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d31793880 0x559d317b1db0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x559d31782a10 tx=0x7f74b8007e00 comp rx=0 tx=0).stop 2024-06-10T00:33:58.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.556+0000 7f74d122fb80 1 -- 172.21.15.62:0/3847154025 shutdown_connections 2024-06-10T00:33:58.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.556+0000 7f74d122fb80 1 --2- 172.21.15.62:0/3847154025 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x559d317aaff0 0x559d3179ca70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.556+0000 7f749f7fe700 0 client.15750 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:33:58.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.556+0000 7f74d122fb80 1 --2- 172.21.15.62:0/3847154025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559d31793880 0x559d317b1db0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.556+0000 7f74d122fb80 1 --2- 172.21.15.62:0/3847154025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559d3178d7a0 0x559d317a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.556+0000 7f74d122fb80 1 --2- 172.21.15.62:0/3847154025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d31792f60 0x559d317a3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.556+0000 7f74d122fb80 1 -- 172.21.15.62:0/3847154025 >> 172.21.15.62:0/3847154025 conn(0x559d316ba080 msgr2=0x559d317944f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:58.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.557+0000 7f74d122fb80 1 -- 172.21.15.62:0/3847154025 shutdown_connections 2024-06-10T00:33:58.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.557+0000 7f74d122fb80 1 -- 172.21.15.62:0/3847154025 wait complete. 2024-06-10T00:33:58.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.557+0000 7f74d122fb80 20 client.15750 trim_cache size 0 max 16384 2024-06-10T00:33:58.561 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:33:58.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.557+0000 7f74c69ac700 1 -- 172.21.15.62:0/1143776170 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f74a80a2e20 msgr2=0x7f74a80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:58.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.557+0000 7f74c69ac700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f74a80a2e20 0x7f74a80a52e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f74b0010420 tx=0x7f74b0007480 comp rx=0 tx=0).stop 2024-06-10T00:33:58.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.558+0000 7f74c69ac700 1 -- 172.21.15.62:0/1143776170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74c006b520 msgr2=0x7f74c019ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:33:58.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.558+0000 7f74c69ac700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74c006b520 0x7f74c019ee50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f74b4006010 tx=0x7f74b4003890 comp rx=0 tx=0).stop 2024-06-10T00:33:58.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.558+0000 7f74c69ac700 1 -- 172.21.15.62:0/1143776170 shutdown_connections 2024-06-10T00:33:58.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.558+0000 7f74c69ac700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f74a80a2e20 0x7f74a80a52e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.558+0000 7f74c69ac700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006ab60 0x7f74c019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.558+0000 7f74c69ac700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74c0108530 0x7f74c0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.558+0000 7f74c69ac700 1 --2- 172.21.15.62:0/1143776170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74c006b520 0x7f74c019ee50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:33:58.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.558+0000 7f74c69ac700 1 -- 172.21.15.62:0/1143776170 >> 172.21.15.62:0/1143776170 conn(0x7f74c00783d0 msgr2=0x7f74c0100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:33:58.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.558+0000 7f74c69ac700 1 -- 172.21.15.62:0/1143776170 shutdown_connections 2024-06-10T00:33:58.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:33:58.558+0000 7f74c69ac700 1 -- 172.21.15.62:0/1143776170 wait complete. 2024-06-10T00:33:59.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:33:59 smithi062 ceph-mon[32390]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:59.456 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T00:33:59.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:33:59 smithi138 ceph-mon[36549]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:33:59.748 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:33:59.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:33:59 smithi105 ceph-mon[36321]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:34:00.606 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:34:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:34:00.366+0000 7f466c991700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:34:00.606 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:34:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:34:00.366+0000 7f466c991700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:34:00.607 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:34:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:34:00.366+0000 7f466c991700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:34:00.607 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:34:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:34:00.366+0000 7f466c991700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:34:00.609 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:34:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:34:00.366+0000 7f466c991700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:34:00.779 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-10T00:34:00.779 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-10T00:34:00.779 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:34:01.561 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:01 smithi062 ceph-mon[32390]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:34:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:01.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.613+0000 7fc37191f700 1 -- 172.21.15.62:0/289697019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc36c107400 msgr2=0x7fc36c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:01.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.613+0000 7fc37191f700 1 --2- 172.21.15.62:0/289697019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc36c107400 0x7fc36c1077e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009990 tx=0x7fc35c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.613+0000 7fc37191f700 1 -- 172.21.15.62:0/289697019 shutdown_connections 2024-06-10T00:34:01.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.613+0000 7fc37191f700 1 --2- 172.21.15.62:0/289697019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc36c107400 0x7fc36c1077e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.613+0000 7fc37191f700 1 --2- 172.21.15.62:0/289697019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc36c101d50 0x7fc36c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.613+0000 7fc37191f700 1 --2- 172.21.15.62:0/289697019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc36c101390 0x7fc36c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.613+0000 7fc37191f700 1 -- 172.21.15.62:0/289697019 >> 172.21.15.62:0/289697019 conn(0x7fc36c078340 msgr2=0x7fc36c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:01.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.613+0000 7fc37191f700 1 -- 172.21.15.62:0/289697019 shutdown_connections 2024-06-10T00:34:01.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.614+0000 7fc37191f700 1 -- 172.21.15.62:0/289697019 wait complete. 2024-06-10T00:34:01.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.614+0000 7fc37191f700 1 Processor -- start 2024-06-10T00:34:01.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.615+0000 7fc37191f700 1 -- start start 2024-06-10T00:34:01.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.615+0000 7fc37191f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc36c101390 0x7fc36c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.615+0000 7fc37191f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc36c101d50 0x7fc36c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.615+0000 7fc37191f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc36c107400 0x7fc36c1a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.616+0000 7fc37191f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c111670 con 0x7fc36c101d50 2024-06-10T00:34:01.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.616+0000 7fc36affd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc36c101390 0x7fc36c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.616+0000 7fc36affd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc36c101390 0x7fc36c199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:53122/0 (socket says 172.21.15.62:53122) 2024-06-10T00:34:01.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.616+0000 7fc36affd700 1 -- 172.21.15.62:0/649758278 learned_addr learned my addr 172.21.15.62:0/649758278 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:01.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.616+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/649758278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc36c101d50 0x7fc36c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.616+0000 7fc36b7fe700 1 --2- 172.21.15.62:0/649758278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc36c107400 0x7fc36c1a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.616+0000 7fc37191f700 1 -- 172.21.15.62:0/649758278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c1114f0 con 0x7fc36c107400 2024-06-10T00:34:01.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.616+0000 7fc37191f700 1 -- 172.21.15.62:0/649758278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c1117f0 con 0x7fc36c101390 2024-06-10T00:34:01.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.616+0000 7fc36a7fc700 1 -- 172.21.15.62:0/649758278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc36c101390 msgr2=0x7fc36c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:01.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.616+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/649758278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc36c101390 0x7fc36c199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.616+0000 7fc36a7fc700 1 -- 172.21.15.62:0/649758278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc36c107400 msgr2=0x7fc36c1a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:01.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.616+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/649758278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc36c107400 0x7fc36c1a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.616+0000 7fc36a7fc700 1 -- 172.21.15.62:0/649758278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36c1a50f0 con 0x7fc36c101d50 2024-06-10T00:34:01.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.617+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/649758278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc36c101d50 0x7fc36c19a070 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc36000be10 tx=0x7fc36000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:01.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.617+0000 7fc37091d700 1 -- 172.21.15.62:0/649758278 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360018070 con 0x7fc36c101d50 2024-06-10T00:34:01.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.617+0000 7fc37091d700 1 -- 172.21.15.62:0/649758278 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc360012c90 con 0x7fc36c101d50 2024-06-10T00:34:01.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.617+0000 7fc37091d700 1 -- 172.21.15.62:0/649758278 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36001b710 con 0x7fc36c101d50 2024-06-10T00:34:01.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.617+0000 7fc37191f700 1 -- 172.21.15.62:0/649758278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36c197980 con 0x7fc36c101d50 2024-06-10T00:34:01.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.618+0000 7fc37191f700 1 -- 172.21.15.62:0/649758278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36c197e70 con 0x7fc36c101d50 2024-06-10T00:34:01.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.623+0000 7fc37c1a2b80 1 Processor -- start 2024-06-10T00:34:01.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.623+0000 7fc37c1a2b80 1 -- start start 2024-06-10T00:34:01.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:01 smithi138 ceph-mon[36549]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:34:01.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.623+0000 7fc37c1a2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d63a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.623+0000 7fc36b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d63a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.624+0000 7fc36b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d63a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35206/0 (socket says 172.21.15.62:35206) 2024-06-10T00:34:01.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.623+0000 7fc37091d700 1 -- 172.21.15.62:0/649758278 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fc36001b8b0 con 0x7fc36c101d50 2024-06-10T00:34:01.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.623+0000 7fc37c1a2b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 0x5600b3d5d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.624+0000 7fc36b7fe700 1 -- 172.21.15.62:0/3013010121 learned_addr learned my addr 172.21.15.62:0/3013010121 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:01.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.624+0000 7fc36affd700 1 --2- 172.21.15.62:0/3013010121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 0x5600b3d5d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.624+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/3013010121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 0x5600b3d5e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.624+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/3013010121 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5600b3caf810 con 0x5600b3d63640 2024-06-10T00:34:01.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.625+0000 7fc37091d700 1 --2- 172.21.15.62:0/649758278 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fc3540a2ff0 0x7fc3540a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.625+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/3013010121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 0x5600b3d5e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.625+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/3013010121 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5600b3c57720 con 0x5600b3d5df20 2024-06-10T00:34:01.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.625+0000 7fc37091d700 1 -- 172.21.15.62:0/649758278 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fc3600d30b0 con 0x7fc36c101d50 2024-06-10T00:34:01.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.625+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/3013010121 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5600b3cab7d0 con 0x5600b3d63f60 2024-06-10T00:34:01.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.626+0000 7fc36b7fe700 1 --2- 172.21.15.62:0/649758278 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fc3540a2ff0 0x7fc3540a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.626+0000 7fc36affd700 1 -- 172.21.15.62:0/3013010121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 msgr2=0x5600b3d5e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:01.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.626+0000 7fc36affd700 1 --2- 172.21.15.62:0/3013010121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 0x5600b3d5e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.626+0000 7fc36affd700 1 -- 172.21.15.62:0/3013010121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 msgr2=0x5600b3d63a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:01.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.626+0000 7fc36affd700 1 --2- 172.21.15.62:0/3013010121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d63a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.626+0000 7fc36affd700 1 -- 172.21.15.62:0/3013010121 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600b3c4df50 con 0x5600b3d63f60 2024-06-10T00:34:01.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.626+0000 7fc36affd700 1 --2- 172.21.15.62:0/3013010121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 0x5600b3d5d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc35c006010 tx=0x7fc35c005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee8a736ccd19eb54 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:01.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.626+0000 7fc351ffb700 1 -- 172.21.15.62:0/3013010121 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c028070 con 0x5600b3d63f60 2024-06-10T00:34:01.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.626+0000 7fc351ffb700 1 -- 172.21.15.62:0/3013010121 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35c0055b0 con 0x5600b3d63f60 2024-06-10T00:34:01.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.626+0000 7fc351ffb700 1 -- 172.21.15.62:0/3013010121 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00bbb0 con 0x5600b3d63f60 2024-06-10T00:34:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.626+0000 7fc36b7fe700 1 --2- 172.21.15.62:0/3013010121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d63a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.627+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/3013010121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 msgr2=0x5600b3d5d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.627+0000 7fc36b7fe700 1 --2- 172.21.15.62:0/649758278 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fc3540a2ff0 0x7fc3540a54b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc364009d40 tx=0x7fc364004b40 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.627+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/3013010121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 0x5600b3d5d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc35c006010 tx=0x7fc35c005c70 comp rx=0 tx=0).stop 2024-06-10T00:34:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.627+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/3013010121 shutdown_connections 2024-06-10T00:34:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.627+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/3013010121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d63a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.627+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/3013010121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 0x5600b3d5e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.627+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/3013010121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 0x5600b3d5d9e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.627+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/3013010121 >> 172.21.15.62:0/3013010121 conn(0x5600b3c88cc0 msgr2=0x5600b3d58850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.627+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/3013010121 shutdown_connections 2024-06-10T00:34:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.627+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/3013010121 wait complete. 2024-06-10T00:34:01.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.628+0000 7fc37c1a2b80 1 -- start start 2024-06-10T00:34:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.628+0000 7fc37c1a2b80 1 Processor -- start 2024-06-10T00:34:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.628+0000 7fc37c1a2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 0x5600b3d69d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.628+0000 7fc37c1a2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d67010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc37c1a2b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 0x5600b3d67550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc37c1a2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5600b3d730a0 con 0x5600b3d63640 2024-06-10T00:34:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc37c1a2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5600b3c4fbf0 con 0x5600b3d5df20 2024-06-10T00:34:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc37c1a2b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5600b3d73220 con 0x5600b3d63f60 2024-06-10T00:34:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc36a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d67010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc36a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d67010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35220/0 (socket says 172.21.15.62:35220) 2024-06-10T00:34:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc36a7fc700 1 -- 172.21.15.62:0/4173511014 learned_addr learned my addr 172.21.15.62:0/4173511014 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc36affd700 1 --2- 172.21.15.62:0/4173511014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 0x5600b3d69d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc36b7fe700 1 --2- 172.21.15.62:0/4173511014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 0x5600b3d67550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc37c1a2b80 10 client.0 _ll_register_callbacks cb 0x5600b3c49920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc36a7fc700 1 -- 172.21.15.62:0/4173511014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 msgr2=0x5600b3d67550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/4173511014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 0x5600b3d67550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc36a7fc700 1 -- 172.21.15.62:0/4173511014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 msgr2=0x5600b3d69d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:01.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/4173511014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 0x5600b3d69d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc36a7fc700 1 -- 172.21.15.62:0/4173511014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600b3bfe8d0 con 0x5600b3d63640 2024-06-10T00:34:01.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/4173511014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc35c0055b0 con 0x5600b3d63640 2024-06-10T00:34:01.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/4173511014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d67010 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc36001cf10 tx=0x7fc36009ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfc32c9aee4ce3f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:01.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.629+0000 7fc34affd700 10 client.15768 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:01.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.630+0000 7fc34affd700 1 -- 172.21.15.62:0/4173511014 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3600d52e0 con 0x5600b3d63640 2024-06-10T00:34:01.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.630+0000 7fc34affd700 1 -- 172.21.15.62:0/4173511014 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fc3600d05e0 con 0x5600b3d63640 2024-06-10T00:34:01.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.630+0000 7fc34affd700 1 -- 172.21.15.62:0/4173511014 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc3600d07e0 con 0x5600b3d63640 2024-06-10T00:34:01.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.630+0000 7fc37c1a2b80 10 client.15768 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:01.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.630+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/4173511014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5600b3d728c0 con 0x5600b3d63640 2024-06-10T00:34:01.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.630+0000 7fc34affd700 1 -- 172.21.15.62:0/4173511014 <== mon.0 v2:172.21.15.62:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc3600a3120 con 0x5600b3d63640 2024-06-10T00:34:01.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.630+0000 7fc37c1a2b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:01.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.630+0000 7fc37c1a2b80 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:34:01.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.630+0000 7fc37c1a2b80 20 client.15768 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:01.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.630+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/4173511014 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5600b3d88900 0x5600b3d8acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.630+0000 7fc37c1a2b80 4 client.15768 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:34:01.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.630+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/4173511014 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600b3cbcd10 con 0x5600b3d88900 2024-06-10T00:34:01.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.631+0000 7fc36affd700 1 --2- 172.21.15.62:0/4173511014 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5600b3d88900 0x5600b3d8acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.631+0000 7fc34affd700 10 client.15768 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:01.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.631+0000 7fc36affd700 1 --2- 172.21.15.62:0/4173511014 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5600b3d88900 0x5600b3d8acf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cd62df26f6ecc04 server_cookie=60968517719ee8f7 in_seq=0 out_seq=0 2024-06-10T00:34:01.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.632+0000 7fc34affd700 1 -- 172.21.15.62:0/4173511014 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5600b3cbcd10 con 0x5600b3d88900 2024-06-10T00:34:01.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.632+0000 7fc34affd700 10 client.15768 handle_command_reply: tid=0 2024-06-10T00:34:01.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.632+0000 7fc37c1a2b80 1 client.15768 shutdown 2024-06-10T00:34:01.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.633+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/4173511014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 msgr2=0x5600b3d67010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:01.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.633+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/4173511014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d67010 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc36001cf10 tx=0x7fc36009ebf0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.633+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/4173511014 shutdown_connections 2024-06-10T00:34:01.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.633+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/4173511014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 0x5600b3d67550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.633+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/4173511014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 0x5600b3d69d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.633+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/4173511014 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5600b3d88900 0x5600b3d8acf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.633+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/4173511014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d67010 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.633+0000 7fc34affd700 0 client.15768 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:01.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.633+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/4173511014 >> 172.21.15.62:0/4173511014 conn(0x5600b3c88cc0 msgr2=0x5600b3d5c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:01.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.633+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/4173511014 shutdown_connections 2024-06-10T00:34:01.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.633+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/4173511014 wait complete. 2024-06-10T00:34:01.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.633+0000 7fc37c1a2b80 20 client.15768 trim_cache size 0 max 16384 2024-06-10T00:34:01.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.663+0000 7fc37c1a2b80 1 Processor -- start 2024-06-10T00:34:01.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.663+0000 7fc37c1a2b80 1 -- start start 2024-06-10T00:34:01.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.663+0000 7fc37c1a2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 0x5600b3d5d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.663+0000 7fc37c1a2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.663+0000 7fc37c1a2b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 0x5600b3d61600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.663+0000 7fc37c1a2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc35800eda0 con 0x5600b3d63640 2024-06-10T00:34:01.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.663+0000 7fc37c1a2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc3580108e0 con 0x5600b3d5df20 2024-06-10T00:34:01.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.663+0000 7fc37c1a2b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc35800f470 con 0x5600b3d63f60 2024-06-10T00:34:01.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.663+0000 7fc36a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.663+0000 7fc36a7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35234/0 (socket says 172.21.15.62:35234) 2024-06-10T00:34:01.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.663+0000 7fc36a7fc700 1 -- 172.21.15.62:0/2327679008 learned_addr learned my addr 172.21.15.62:0/2327679008 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:01.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.663+0000 7fc36affd700 1 --2- 172.21.15.62:0/2327679008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 0x5600b3d5d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.664+0000 7fc36b7fe700 1 --2- 172.21.15.62:0/2327679008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 0x5600b3d61600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.664+0000 7fc36a7fc700 1 -- 172.21.15.62:0/2327679008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 msgr2=0x5600b3d61600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:01.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.664+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/2327679008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 0x5600b3d61600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.664+0000 7fc36a7fc700 1 -- 172.21.15.62:0/2327679008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 msgr2=0x5600b3d5d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:01.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.664+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/2327679008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 0x5600b3d5d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.664+0000 7fc36a7fc700 1 -- 172.21.15.62:0/2327679008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600b3d72b70 con 0x5600b3d63640 2024-06-10T00:34:01.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.664+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/2327679008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d610c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc360005950 tx=0x7fc3600d5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab011a7afee15527 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:01.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.664+0000 7fc3517fa700 1 -- 172.21.15.62:0/2327679008 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3600a3510 con 0x5600b3d63640 2024-06-10T00:34:01.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.664+0000 7fc3517fa700 1 -- 172.21.15.62:0/2327679008 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35c0055b0 con 0x5600b3d63640 2024-06-10T00:34:01.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.664+0000 7fc3517fa700 1 -- 172.21.15.62:0/2327679008 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36009f620 con 0x5600b3d63640 2024-06-10T00:34:01.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.665+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/2327679008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 msgr2=0x5600b3d610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:01.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.665+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/2327679008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d610c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc360005950 tx=0x7fc3600d5000 comp rx=0 tx=0).stop 2024-06-10T00:34:01.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.665+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/2327679008 shutdown_connections 2024-06-10T00:34:01.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.665+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/2327679008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d63f60 0x5600b3d61600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.665+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/2327679008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d5df20 0x5600b3d5d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.665+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/2327679008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d610c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.665+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/2327679008 >> 172.21.15.62:0/2327679008 conn(0x5600b3c88cc0 msgr2=0x5600b3d66fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:01.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.665+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/2327679008 shutdown_connections 2024-06-10T00:34:01.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.665+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/2327679008 wait complete. 2024-06-10T00:34:01.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.666+0000 7fc37c1a2b80 1 -- start start 2024-06-10T00:34:01.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.666+0000 7fc37c1a2b80 1 Processor -- start 2024-06-10T00:34:01.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.666+0000 7fc37c1a2b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d5df20 0x5600b3d710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.666+0000 7fc37c1a2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d715e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.666+0000 7fc37c1a2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d63f60 0x5600b3d6e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.666+0000 7fc37c1a2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc358010210 con 0x5600b3d63640 2024-06-10T00:34:01.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.666+0000 7fc37c1a2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc35800fb40 con 0x5600b3d63f60 2024-06-10T00:34:01.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.666+0000 7fc37c1a2b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc35800cd40 con 0x5600b3d5df20 2024-06-10T00:34:01.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.666+0000 7fc37c1a2b80 10 client.0 _ll_register_callbacks cb 0x5600b3c49920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.666+0000 7fc36b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d63f60 0x5600b3d6e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.667+0000 7fc36b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d63f60 0x5600b3d6e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:49404/0 (socket says 172.21.15.62:49404) 2024-06-10T00:34:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.667+0000 7fc36b7fe700 1 -- 172.21.15.62:0/1466623918 learned_addr learned my addr 172.21.15.62:0/1466623918 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.667+0000 7fc36affd700 1 --2- 172.21.15.62:0/1466623918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d5df20 0x5600b3d710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.667+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/1466623918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.667+0000 7fc36a7fc700 1 -- 172.21.15.62:0/1466623918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d5df20 msgr2=0x5600b3d710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.667+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/1466623918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d5df20 0x5600b3d710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.667+0000 7fc36a7fc700 1 -- 172.21.15.62:0/1466623918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d63f60 msgr2=0x5600b3d6e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.667+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/1466623918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d63f60 0x5600b3d6e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.667+0000 7fc36a7fc700 1 -- 172.21.15.62:0/1466623918 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600b3d6eed0 con 0x5600b3d63640 2024-06-10T00:34:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.667+0000 7fc36b7fe700 1 --2- 172.21.15.62:0/1466623918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d63f60 0x5600b3d6e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:34:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.667+0000 7fc36a7fc700 1 --2- 172.21.15.62:0/1466623918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d715e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x5600b3d54ba0 tx=0x7fc36009b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d83f5910ed9127 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.667+0000 7fc34affd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.668+0000 7fc34affd700 1 -- 172.21.15.62:0/1466623918 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3600a3510 con 0x5600b3d63640 2024-06-10T00:34:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.668+0000 7fc34affd700 1 -- 172.21.15.62:0/1466623918 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fc3600a7940 con 0x5600b3d63640 2024-06-10T00:34:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.668+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/1466623918 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc35c0055b0 con 0x5600b3d63640 2024-06-10T00:34:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.668+0000 7fc34affd700 1 -- 172.21.15.62:0/1466623918 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc3600a7b40 con 0x5600b3d63640 2024-06-10T00:34:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.668+0000 7fc37c1a2b80 10 client.15774 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.668+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/1466623918 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5600b3df2e80 con 0x5600b3d63640 2024-06-10T00:34:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.668+0000 7fc34affd700 1 -- 172.21.15.62:0/1466623918 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35c0055b0 con 0x5600b3d63640 2024-06-10T00:34:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.668+0000 7fc34affd700 1 -- 172.21.15.62:0/1466623918 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc36009e5b0 con 0x5600b3d63640 2024-06-10T00:34:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.669+0000 7fc37c1a2b80 10 client.15774 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.669+0000 7fc37c1a2b80 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:34:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.669+0000 7fc37c1a2b80 20 client.15774 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.669+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/1466623918 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5600b3df3320 0x5600b3df5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.669+0000 7fc37c1a2b80 4 client.15774 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:34:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.669+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/1466623918 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600b3cb7290 con 0x5600b3df3320 2024-06-10T00:34:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.669+0000 7fc36affd700 1 --2- 172.21.15.62:0/1466623918 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5600b3df3320 0x5600b3df5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.669+0000 7fc36affd700 1 --2- 172.21.15.62:0/1466623918 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5600b3df3320 0x5600b3df5740 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbf076c4e0709db server_cookie=ff39f209ed9eecd3 in_seq=0 out_seq=0 2024-06-10T00:34:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.669+0000 7fc34affd700 10 client.15774 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.670+0000 7fc34affd700 1 -- 172.21.15.62:0/1466623918 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600b3cb7290 con 0x5600b3df3320 2024-06-10T00:34:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.670+0000 7fc34affd700 10 client.15774 handle_command_reply: tid=0 2024-06-10T00:34:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.670+0000 7fc37c1a2b80 1 client.15774 shutdown 2024-06-10T00:34:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.670+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/1466623918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 msgr2=0x5600b3d715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.670+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/1466623918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d715e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x5600b3d54ba0 tx=0x7fc36009b4d0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.670+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/1466623918 shutdown_connections 2024-06-10T00:34:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.670+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/1466623918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600b3d63f60 0x5600b3d6e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.670+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/1466623918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600b3d5df20 0x5600b3d710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.670+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/1466623918 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5600b3df3320 0x5600b3df5740 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.670+0000 7fc34affd700 0 client.15774 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:01.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.670+0000 7fc37c1a2b80 1 --2- 172.21.15.62:0/1466623918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5600b3d63640 0x5600b3d715e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.670+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/1466623918 >> 172.21.15.62:0/1466623918 conn(0x5600b3c88cc0 msgr2=0x5600b3d66f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:01.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.670+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/1466623918 shutdown_connections 2024-06-10T00:34:01.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.670+0000 7fc37c1a2b80 1 -- 172.21.15.62:0/1466623918 wait complete. 2024-06-10T00:34:01.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.671+0000 7fc37c1a2b80 20 client.15774 trim_cache size 0 max 16384 2024-06-10T00:34:01.677 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:34:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.671+0000 7fc37191f700 1 -- 172.21.15.62:0/649758278 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fc3540a2ff0 msgr2=0x7fc3540a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.671+0000 7fc37191f700 1 --2- 172.21.15.62:0/649758278 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fc3540a2ff0 0x7fc3540a54b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc364009d40 tx=0x7fc364004b40 comp rx=0 tx=0).stop 2024-06-10T00:34:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.671+0000 7fc37191f700 1 -- 172.21.15.62:0/649758278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc36c101d50 msgr2=0x7fc36c19a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.671+0000 7fc37191f700 1 --2- 172.21.15.62:0/649758278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc36c101d50 0x7fc36c19a070 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc36000be10 tx=0x7fc36000dc30 comp rx=0 tx=0).stop 2024-06-10T00:34:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.671+0000 7fc37191f700 1 -- 172.21.15.62:0/649758278 shutdown_connections 2024-06-10T00:34:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.671+0000 7fc37191f700 1 --2- 172.21.15.62:0/649758278 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fc3540a2ff0 0x7fc3540a54b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.671+0000 7fc37191f700 1 --2- 172.21.15.62:0/649758278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc36c107400 0x7fc36c1a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.672+0000 7fc37191f700 1 --2- 172.21.15.62:0/649758278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc36c101390 0x7fc36c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.672+0000 7fc37191f700 1 --2- 172.21.15.62:0/649758278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc36c101d50 0x7fc36c19a070 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.672+0000 7fc37191f700 1 -- 172.21.15.62:0/649758278 >> 172.21.15.62:0/649758278 conn(0x7fc36c078340 msgr2=0x7fc36c1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:01.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.672+0000 7fc37191f700 1 -- 172.21.15.62:0/649758278 shutdown_connections 2024-06-10T00:34:01.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:01.672+0000 7fc37191f700 1 -- 172.21.15.62:0/649758278 wait complete. 2024-06-10T00:34:01.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:01 smithi105 ceph-mon[36321]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:34:02.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:02 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:34:02.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:02 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:34:02.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:02 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:34:03.592 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:03 smithi062 ceph-mon[32390]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-10T00:34:03.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:03 smithi062 ceph-mon[32390]: mgrmap e25: x(active, since 72s), standbys: z, y 2024-06-10T00:34:03.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:03 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:34:03.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:03 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:34:03.593 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:03 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:34:03.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:03 smithi138 ceph-mon[36549]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-10T00:34:03.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:03 smithi138 ceph-mon[36549]: mgrmap e25: x(active, since 72s), standbys: z, y 2024-06-10T00:34:03.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:03 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:34:03.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:03 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:34:03.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:03 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:34:03.637 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T00:34:03.637 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T00:34:03.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:03 smithi105 ceph-mon[36321]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-10T00:34:03.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:03 smithi105 ceph-mon[36321]: mgrmap e25: x(active, since 72s), standbys: z, y 2024-06-10T00:34:03.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:03 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:34:03.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:03 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:34:03.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:03 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:34:04.263 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:04.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.272+0000 7fe9af727700 1 -- 172.21.15.62:0/122200241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a8101380 msgr2=0x7fe9a8101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:04.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.272+0000 7fe9af727700 1 --2- 172.21.15.62:0/122200241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a8101380 0x7fe9a8101800 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe998009990 tx=0x7fe998009ca0 comp rx=0 tx=0).stop 2024-06-10T00:34:04.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.272+0000 7fe9af727700 1 -- 172.21.15.62:0/122200241 shutdown_connections 2024-06-10T00:34:04.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.272+0000 7fe9af727700 1 --2- 172.21.15.62:0/122200241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9a8101d40 0x7fe9a810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:04.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.272+0000 7fe9af727700 1 --2- 172.21.15.62:0/122200241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9a81073f0 0x7fe9a81077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:04.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.272+0000 7fe9af727700 1 --2- 172.21.15.62:0/122200241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a8101380 0x7fe9a8101800 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:04.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.272+0000 7fe9af727700 1 -- 172.21.15.62:0/122200241 >> 172.21.15.62:0/122200241 conn(0x7fe9a8078300 msgr2=0x7fe9a8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:04.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.273+0000 7fe9af727700 1 -- 172.21.15.62:0/122200241 shutdown_connections 2024-06-10T00:34:04.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.273+0000 7fe9af727700 1 -- 172.21.15.62:0/122200241 wait complete. 2024-06-10T00:34:04.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.274+0000 7fe9af727700 1 Processor -- start 2024-06-10T00:34:04.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.274+0000 7fe9af727700 1 -- start start 2024-06-10T00:34:04.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.274+0000 7fe9af727700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a8101d40 0x7fe9a819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:04.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.274+0000 7fe9af727700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9a81073f0 0x7fe9a819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:04.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.274+0000 7fe9af727700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9a8196af0 0x7fe9a8196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:04.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.274+0000 7fe9af727700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8110590 con 0x7fe9a8101d40 2024-06-10T00:34:04.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.274+0000 7fe9af727700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8110410 con 0x7fe9a81073f0 2024-06-10T00:34:04.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.274+0000 7fe9af727700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8110710 con 0x7fe9a8196af0 2024-06-10T00:34:04.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.274+0000 7fe9ad4c3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a8101d40 0x7fe9a819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:04.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.275+0000 7fe9ad4c3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a8101d40 0x7fe9a819c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35260/0 (socket says 172.21.15.62:35260) 2024-06-10T00:34:04.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.275+0000 7fe9adcc4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9a8196af0 0x7fe9a8196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:04.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.275+0000 7fe9ad4c3700 1 -- 172.21.15.62:0/1540841837 learned_addr learned my addr 172.21.15.62:0/1540841837 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:04.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.275+0000 7fe9accc2700 1 --2- 172.21.15.62:0/1540841837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9a81073f0 0x7fe9a819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:04.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.275+0000 7fe9ad4c3700 1 -- 172.21.15.62:0/1540841837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9a8196af0 msgr2=0x7fe9a8196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:04.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.275+0000 7fe9ad4c3700 1 --2- 172.21.15.62:0/1540841837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9a8196af0 0x7fe9a8196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:04.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.275+0000 7fe9ad4c3700 1 -- 172.21.15.62:0/1540841837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9a81073f0 msgr2=0x7fe9a819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:04.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.275+0000 7fe9ad4c3700 1 --2- 172.21.15.62:0/1540841837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9a81073f0 0x7fe9a819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:04.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.275+0000 7fe9ad4c3700 1 -- 172.21.15.62:0/1540841837 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a8197670 con 0x7fe9a8101d40 2024-06-10T00:34:04.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.276+0000 7fe9ad4c3700 1 --2- 172.21.15.62:0/1540841837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a8101d40 0x7fe9a819c930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e9c0 tx=0x7fe9a400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:04.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.277+0000 7fe99e7fc700 1 -- 172.21.15.62:0/1540841837 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a400cbd0 con 0x7fe9a8101d40 2024-06-10T00:34:04.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.277+0000 7fe99e7fc700 1 -- 172.21.15.62:0/1540841837 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9a400cd70 con 0x7fe9a8101d40 2024-06-10T00:34:04.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.277+0000 7fe99e7fc700 1 -- 172.21.15.62:0/1540841837 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4010430 con 0x7fe9a8101d40 2024-06-10T00:34:04.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.277+0000 7fe9af727700 1 -- 172.21.15.62:0/1540841837 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a81a7c10 con 0x7fe9a8101d40 2024-06-10T00:34:04.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.277+0000 7fe9af727700 1 -- 172.21.15.62:0/1540841837 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a81a39d0 con 0x7fe9a8101d40 2024-06-10T00:34:04.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.278+0000 7fe993fff700 1 -- 172.21.15.62:0/1540841837 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a8102aa0 con 0x7fe9a8101d40 2024-06-10T00:34:04.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.281+0000 7fe99e7fc700 1 -- 172.21.15.62:0/1540841837 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7fe9a4004750 con 0x7fe9a8101d40 2024-06-10T00:34:04.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.282+0000 7fe99e7fc700 1 --2- 172.21.15.62:0/1540841837 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe9940ab720 0x7fe9940adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:04.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.283+0000 7fe99e7fc700 1 -- 172.21.15.62:0/1540841837 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fe9a40cfee0 con 0x7fe9a8101d40 2024-06-10T00:34:04.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.284+0000 7fe9accc2700 1 --2- 172.21.15.62:0/1540841837 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe9940ab720 0x7fe9940adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:04.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.284+0000 7fe9accc2700 1 --2- 172.21.15.62:0/1540841837 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe9940ab720 0x7fe9940adbe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9980098e0 tx=0x7fe998005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:04.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.287+0000 7fe99e7fc700 1 -- 172.21.15.62:0/1540841837 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe9a4053850 con 0x7fe9a8101d40 2024-06-10T00:34:04.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.560+0000 7fe993fff700 1 -- 172.21.15.62:0/1540841837 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9a8197fe0 con 0x7fe9a8101d40 2024-06-10T00:34:04.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.561+0000 7fe99e7fc700 1 -- 172.21.15.62:0/1540841837 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7fe9a409c4a0 con 0x7fe9a8101d40 2024-06-10T00:34:04.562 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:34:04.562 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:32:47.783154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:34:04.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.565+0000 7fe993fff700 1 -- 172.21.15.62:0/1540841837 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe9940ab720 msgr2=0x7fe9940adbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:04.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.565+0000 7fe993fff700 1 --2- 172.21.15.62:0/1540841837 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe9940ab720 0x7fe9940adbe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9980098e0 tx=0x7fe998005960 comp rx=0 tx=0).stop 2024-06-10T00:34:04.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.565+0000 7fe993fff700 1 -- 172.21.15.62:0/1540841837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a8101d40 msgr2=0x7fe9a819c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:04.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.565+0000 7fe993fff700 1 --2- 172.21.15.62:0/1540841837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a8101d40 0x7fe9a819c930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e9c0 tx=0x7fe9a400ecd0 comp rx=0 tx=0).stop 2024-06-10T00:34:04.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.566+0000 7fe993fff700 1 -- 172.21.15.62:0/1540841837 shutdown_connections 2024-06-10T00:34:04.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.566+0000 7fe993fff700 1 --2- 172.21.15.62:0/1540841837 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe9940ab720 0x7fe9940adbe0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:04.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.566+0000 7fe993fff700 1 --2- 172.21.15.62:0/1540841837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a8101d40 0x7fe9a819c930 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:04.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.566+0000 7fe993fff700 1 --2- 172.21.15.62:0/1540841837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9a8196af0 0x7fe9a8196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:04.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.566+0000 7fe993fff700 1 --2- 172.21.15.62:0/1540841837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9a81073f0 0x7fe9a819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:04.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.566+0000 7fe993fff700 1 -- 172.21.15.62:0/1540841837 >> 172.21.15.62:0/1540841837 conn(0x7fe9a8078300 msgr2=0x7fe9a81001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:04.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.566+0000 7fe993fff700 1 -- 172.21.15.62:0/1540841837 shutdown_connections 2024-06-10T00:34:04.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:04.566+0000 7fe993fff700 1 -- 172.21.15.62:0/1540841837 wait complete. 2024-06-10T00:34:04.568 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-06-10T00:34:05.583 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:05 smithi105 ceph-mon[36321]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-10T00:34:05.583 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:05 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1540841837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:05.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:05 smithi062 ceph-mon[32390]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-10T00:34:05.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:05 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1540841837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:05.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:05 smithi138 ceph-mon[36549]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-10T00:34:05.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:05 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1540841837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:05.819 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-10T00:34:05.819 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24464 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-10T00:34:06.589 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:06.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.849+0000 7f3dfbbf0700 1 -- 172.21.15.62:0/2894132293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df4107550 msgr2=0x7f3df4107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:06.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.849+0000 7f3dfbbf0700 1 --2- 172.21.15.62:0/2894132293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df4107550 0x7f3df4107930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009990 tx=0x7f3df0009ca0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.849+0000 7f3dfbbf0700 1 -- 172.21.15.62:0/2894132293 shutdown_connections 2024-06-10T00:34:06.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.849+0000 7f3dfbbf0700 1 --2- 172.21.15.62:0/2894132293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df4107550 0x7f3df4107930 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.849+0000 7f3dfbbf0700 1 --2- 172.21.15.62:0/2894132293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df4102130 0x7f3df410e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.849+0000 7f3dfbbf0700 1 --2- 172.21.15.62:0/2894132293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df4101770 0x7f3df4101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.849+0000 7f3dfbbf0700 1 -- 172.21.15.62:0/2894132293 >> 172.21.15.62:0/2894132293 conn(0x7f3df4078340 msgr2=0x7f3df4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:06.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.849+0000 7f3dfbbf0700 1 -- 172.21.15.62:0/2894132293 shutdown_connections 2024-06-10T00:34:06.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.849+0000 7f3dfbbf0700 1 -- 172.21.15.62:0/2894132293 wait complete. 2024-06-10T00:34:06.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.850+0000 7f3dfbbf0700 1 Processor -- start 2024-06-10T00:34:06.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.850+0000 7f3dfbbf0700 1 -- start start 2024-06-10T00:34:06.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.851+0000 7f3dfbbf0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df4101770 0x7f3df419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.851+0000 7f3dfbbf0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df4102130 0x7f3df419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.851+0000 7f3dfbbf0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df4107550 0x7f3df4196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.851+0000 7f3dfbbf0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3df4110640 con 0x7f3df4101770 2024-06-10T00:34:06.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.851+0000 7f3dfbbf0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41104c0 con 0x7f3df4107550 2024-06-10T00:34:06.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.851+0000 7f3dfbbf0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41107c0 con 0x7f3df4102130 2024-06-10T00:34:06.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.851+0000 7f3df918b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df4102130 0x7f3df419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.851+0000 7f3df918b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df4102130 0x7f3df419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:53192/0 (socket says 172.21.15.62:53192) 2024-06-10T00:34:06.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.851+0000 7f3df918b700 1 -- 172.21.15.62:0/2991950405 learned_addr learned my addr 172.21.15.62:0/2991950405 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:06.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.851+0000 7f3df998c700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df4101770 0x7f3df419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.851+0000 7f3dfa18d700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df4107550 0x7f3df4196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.852+0000 7f3df998c700 1 -- 172.21.15.62:0/2991950405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df4102130 msgr2=0x7f3df419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:06.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.852+0000 7f3df998c700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df4102130 0x7f3df419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.852+0000 7f3df998c700 1 -- 172.21.15.62:0/2991950405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df4107550 msgr2=0x7f3df4196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:06.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.852+0000 7f3df998c700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df4107550 0x7f3df4196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.852+0000 7f3df998c700 1 -- 172.21.15.62:0/2991950405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df4197020 con 0x7f3df4101770 2024-06-10T00:34:06.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.852+0000 7f3dfa18d700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df4107550 0x7f3df4196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:06.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.852+0000 7f3df998c700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df4101770 0x7f3df419c670 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3df00059d0 tx=0x7f3df000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:06.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.852+0000 7f3deaffd700 1 -- 172.21.15.62:0/2991950405 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0027070 con 0x7f3df4101770 2024-06-10T00:34:06.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.852+0000 7f3deaffd700 1 -- 172.21.15.62:0/2991950405 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df0003d70 con 0x7f3df4101770 2024-06-10T00:34:06.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.852+0000 7f3dfbbf0700 1 -- 172.21.15.62:0/2991950405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df41972b0 con 0x7f3df4101770 2024-06-10T00:34:06.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.852+0000 7f3dfbbf0700 1 -- 172.21.15.62:0/2991950405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df41a78e0 con 0x7f3df4101770 2024-06-10T00:34:06.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.853+0000 7f3deaffd700 1 -- 172.21.15.62:0/2991950405 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df000f460 con 0x7f3df4101770 2024-06-10T00:34:06.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.856+0000 7f3deaffd700 1 -- 172.21.15.62:0/2991950405 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f3df002a070 con 0x7f3df4101770 2024-06-10T00:34:06.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.856+0000 7f3e06473b80 1 Processor -- start 2024-06-10T00:34:06.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.856+0000 7f3e06473b80 1 -- start start 2024-06-10T00:34:06.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.856+0000 7f3e06473b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bb9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.856+0000 7f3e06473b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 0x556df4bb3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3e06473b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb3fa0 0x556df4bb43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3e06473b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556df4b05890 con 0x556df4bb9fe0 2024-06-10T00:34:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3e06473b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556df4aad7a0 con 0x556df4bb96c0 2024-06-10T00:34:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3e06473b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556df4b01850 con 0x556df4bb3fa0 2024-06-10T00:34:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3df918b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb3fa0 0x556df4bb43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3dfa18d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bb9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3dfa18d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bb9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:49450/0 (socket says 172.21.15.62:49450) 2024-06-10T00:34:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3dfa18d700 1 -- 172.21.15.62:0/2834198942 learned_addr learned my addr 172.21.15.62:0/2834198942 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3df998c700 1 --2- 172.21.15.62:0/2834198942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 0x556df4bb3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3deaffd700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3de00a2f10 0x7f3de00a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3df918b700 1 -- 172.21.15.62:0/2834198942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 msgr2=0x556df4bb9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3df918b700 1 --2- 172.21.15.62:0/2834198942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bb9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3df918b700 1 -- 172.21.15.62:0/2834198942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 msgr2=0x556df4bb3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:06.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3df918b700 1 --2- 172.21.15.62:0/2834198942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 0x556df4bb3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.857+0000 7f3df918b700 1 -- 172.21.15.62:0/2834198942 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556df4aa3fd0 con 0x556df4bb3fa0 2024-06-10T00:34:06.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.858+0000 7f3dfa18d700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3de00a2f10 0x7f3de00a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.858+0000 7f3deaffd700 1 -- 172.21.15.62:0/2991950405 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f3df00d09a0 con 0x7f3df4101770 2024-06-10T00:34:06.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.858+0000 7f3df918b700 1 --2- 172.21.15.62:0/2834198942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb3fa0 0x556df4bb43c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3de4005950 tx=0x7f3de40106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d4601d8892f143b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:06.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.858+0000 7f3dfa18d700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3de00a2f10 0x7f3de00a53d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3dec0075c0 tx=0x7f3dec004ad0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:06.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.858+0000 7f3ddffff700 1 -- 172.21.15.62:0/2834198942 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4010e10 con 0x556df4bb3fa0 2024-06-10T00:34:06.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.859+0000 7f3ddffff700 1 -- 172.21.15.62:0/2834198942 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3de4004510 con 0x556df4bb3fa0 2024-06-10T00:34:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.859+0000 7f3ddffff700 1 -- 172.21.15.62:0/2834198942 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de40116a0 con 0x556df4bb3fa0 2024-06-10T00:34:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.860+0000 7f3e06473b80 1 -- 172.21.15.62:0/2834198942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb3fa0 msgr2=0x556df4bb43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.860+0000 7f3e06473b80 1 --2- 172.21.15.62:0/2834198942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb3fa0 0x556df4bb43c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3de4005950 tx=0x7f3de40106b0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.860+0000 7f3e06473b80 1 -- 172.21.15.62:0/2834198942 shutdown_connections 2024-06-10T00:34:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.860+0000 7f3e06473b80 1 --2- 172.21.15.62:0/2834198942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb3fa0 0x556df4bb43c0 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3de4005950 tx=0x7f3de40106b0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.860+0000 7f3e06473b80 1 --2- 172.21.15.62:0/2834198942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bb9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.860+0000 7f3e06473b80 1 --2- 172.21.15.62:0/2834198942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 0x556df4bb3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.860+0000 7f3e06473b80 1 -- 172.21.15.62:0/2834198942 >> 172.21.15.62:0/2834198942 conn(0x556df4aded40 msgr2=0x556df4bae8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.860+0000 7f3e06473b80 1 -- 172.21.15.62:0/2834198942 shutdown_connections 2024-06-10T00:34:06.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.860+0000 7f3e06473b80 1 -- 172.21.15.62:0/2834198942 wait complete. 2024-06-10T00:34:06.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.861+0000 7f3e06473b80 1 -- start start 2024-06-10T00:34:06.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.861+0000 7f3e06473b80 1 Processor -- start 2024-06-10T00:34:06.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.861+0000 7f3e06473b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb96c0 0x556df4bc0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.861+0000 7f3e06473b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 0x556df4bbd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3df918b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 0x556df4bbd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3df918b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 0x556df4bbd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35304/0 (socket says 172.21.15.62:35304) 2024-06-10T00:34:06.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3df918b700 1 -- 172.21.15.62:0/3007878250 learned_addr learned my addr 172.21.15.62:0/3007878250 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:06.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3df998c700 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb96c0 0x556df4bc0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3e06473b80 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bbd9a0 0x556df4bd2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3e06473b80 1 -- 172.21.15.62:0/3007878250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556df4bc0a50 con 0x556df4bb9fe0 2024-06-10T00:34:06.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3e06473b80 1 -- 172.21.15.62:0/3007878250 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556df4aa5c70 con 0x556df4bbd9a0 2024-06-10T00:34:06.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3e06473b80 1 -- 172.21.15.62:0/3007878250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556df4bc0bd0 con 0x556df4bb96c0 2024-06-10T00:34:06.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3dfa18d700 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bbd9a0 0x556df4bd2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3df918b700 1 -- 172.21.15.62:0/3007878250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb96c0 msgr2=0x556df4bc0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:06.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3df918b700 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb96c0 0x556df4bc0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3df918b700 1 -- 172.21.15.62:0/3007878250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bbd9a0 msgr2=0x556df4bd2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:06.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3df918b700 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bbd9a0 0x556df4bd2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3df918b700 1 -- 172.21.15.62:0/3007878250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556df4a54950 con 0x556df4bb9fe0 2024-06-10T00:34:06.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.862+0000 7f3dfa18d700 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bbd9a0 0x556df4bd2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:34:06.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.863+0000 7f3e06473b80 10 client.0 _ll_register_callbacks cb 0x556df4a9f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:06.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.863+0000 7f3e06473b80 1 -- 172.21.15.62:0/3007878250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3de4004510 con 0x556df4bb9fe0 2024-06-10T00:34:06.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.863+0000 7f3df918b700 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 0x556df4bbd300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3de4000c00 tx=0x7f3de4003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3aea88c30c01826 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:06.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.863+0000 7f3ddd7fa700 10 client.15792 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:06.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.863+0000 7f3ddd7fa700 1 -- 172.21.15.62:0/3007878250 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4010c10 con 0x556df4bb9fe0 2024-06-10T00:34:06.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.864+0000 7f3ddd7fa700 1 -- 172.21.15.62:0/3007878250 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f3de4010db0 con 0x556df4bb9fe0 2024-06-10T00:34:06.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.864+0000 7f3ddd7fa700 1 -- 172.21.15.62:0/3007878250 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3de4003cc0 con 0x556df4bb9fe0 2024-06-10T00:34:06.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.864+0000 7f3e06473b80 10 client.15792 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:06.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.864+0000 7f3e06473b80 1 -- 172.21.15.62:0/3007878250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({fsmap=30}) -- 0x556df4bd3f60 con 0x556df4bb9fe0 2024-06-10T00:34:06.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.864+0000 7f3ddd7fa700 1 -- 172.21.15.62:0/3007878250 <== mon.0 v2:172.21.15.62:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3de4003e60 con 0x556df4bb9fe0 2024-06-10T00:34:06.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.864+0000 7f3e06473b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:06.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.864+0000 7f3e06473b80 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:34:06.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.864+0000 7f3e06473b80 20 client.15792 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:06.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.864+0000 7f3e06473b80 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556df4bd6430 0x556df4bd8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.864+0000 7f3e06473b80 4 client.15792 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:34:06.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.864+0000 7f3e06473b80 1 -- 172.21.15.62:0/3007878250 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556df4b12d90 con 0x556df4bd6430 2024-06-10T00:34:06.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.865+0000 7f3df998c700 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556df4bd6430 0x556df4bd8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.865+0000 7f3ddd7fa700 10 client.15792 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:06.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.865+0000 7f3df998c700 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556df4bd6430 0x556df4bd8820 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e24709a6427c7639 server_cookie=60aecc2c65033422 in_seq=0 out_seq=0 2024-06-10T00:34:06.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.866+0000 7f3ddd7fa700 1 -- 172.21.15.62:0/3007878250 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556df4b12d90 con 0x556df4bd6430 2024-06-10T00:34:06.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.866+0000 7f3ddd7fa700 10 client.15792 handle_command_reply: tid=0 2024-06-10T00:34:06.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.866+0000 7f3e06473b80 1 client.15792 shutdown 2024-06-10T00:34:06.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.866+0000 7f3e06473b80 1 -- 172.21.15.62:0/3007878250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 msgr2=0x556df4bbd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:06.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.866+0000 7f3e06473b80 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 0x556df4bbd300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3de4000c00 tx=0x7f3de4003aa0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.866+0000 7f3e06473b80 1 -- 172.21.15.62:0/3007878250 shutdown_connections 2024-06-10T00:34:06.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.866+0000 7f3e06473b80 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bbd9a0 0x556df4bd2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.866+0000 7f3e06473b80 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb96c0 0x556df4bc0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.866+0000 7f3e06473b80 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556df4bd6430 0x556df4bd8820 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.866+0000 7f3e06473b80 1 --2- 172.21.15.62:0/3007878250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 0x556df4bbd300 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.866+0000 7f3e06473b80 1 -- 172.21.15.62:0/3007878250 >> 172.21.15.62:0/3007878250 conn(0x556df4aded40 msgr2=0x556df4bb2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:06.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.866+0000 7f3ddd7fa700 0 client.15792 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:06.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.867+0000 7f3e06473b80 1 -- 172.21.15.62:0/3007878250 shutdown_connections 2024-06-10T00:34:06.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.867+0000 7f3e06473b80 1 -- 172.21.15.62:0/3007878250 wait complete. 2024-06-10T00:34:06.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.867+0000 7f3e06473b80 20 client.15792 trim_cache size 0 max 16384 2024-06-10T00:34:06.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3e06473b80 1 Processor -- start 2024-06-10T00:34:06.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3e06473b80 1 -- start start 2024-06-10T00:34:06.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3e06473b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bb3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3e06473b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 0x556df4ba8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3e06473b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bbd9a0 0x556df4bd25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3e06473b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41ac340 con 0x556df4bb9fe0 2024-06-10T00:34:06.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3e06473b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41ade80 con 0x556df4bb96c0 2024-06-10T00:34:06.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3e06473b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41aca10 con 0x556df4bbd9a0 2024-06-10T00:34:06.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3df998c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bb3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3df998c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bb3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:49462/0 (socket says 172.21.15.62:49462) 2024-06-10T00:34:06.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3df998c700 1 -- 172.21.15.62:0/4183011965 learned_addr learned my addr 172.21.15.62:0/4183011965 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:06.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3df918b700 1 --2- 172.21.15.62:0/4183011965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 0x556df4ba8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3df998c700 1 -- 172.21.15.62:0/4183011965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bbd9a0 msgr2=0x556df4bd25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:06.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3df998c700 1 --2- 172.21.15.62:0/4183011965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bbd9a0 0x556df4bd25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3df998c700 1 -- 172.21.15.62:0/4183011965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 msgr2=0x556df4ba8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:06.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3df998c700 1 --2- 172.21.15.62:0/4183011965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 0x556df4ba8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.889+0000 7f3df998c700 1 -- 172.21.15.62:0/4183011965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556df4baf580 con 0x556df4bb96c0 2024-06-10T00:34:06.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.890+0000 7f3df998c700 1 --2- 172.21.15.62:0/4183011965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bb3a60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3df0099430 tx=0x7f3df00d6510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=734c412d5966de2a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:06.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.890+0000 7f3ddf7fe700 1 -- 172.21.15.62:0/4183011965 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df009c5a0 con 0x556df4bb96c0 2024-06-10T00:34:06.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.890+0000 7f3ddf7fe700 1 -- 172.21.15.62:0/4183011965 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df00d0e50 con 0x556df4bb96c0 2024-06-10T00:34:06.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.890+0000 7f3ddf7fe700 1 -- 172.21.15.62:0/4183011965 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df009c3c0 con 0x556df4bb96c0 2024-06-10T00:34:06.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.891+0000 7f3e06473b80 1 -- 172.21.15.62:0/4183011965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 msgr2=0x556df4bb3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:06.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.891+0000 7f3e06473b80 1 --2- 172.21.15.62:0/4183011965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bb3a60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3df0099430 tx=0x7f3df00d6510 comp rx=0 tx=0).stop 2024-06-10T00:34:06.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.891+0000 7f3e06473b80 1 -- 172.21.15.62:0/4183011965 shutdown_connections 2024-06-10T00:34:06.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.891+0000 7f3e06473b80 1 --2- 172.21.15.62:0/4183011965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bbd9a0 0x556df4bd25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.891+0000 7f3e06473b80 1 --2- 172.21.15.62:0/4183011965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bb3a60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.891+0000 7f3e06473b80 1 --2- 172.21.15.62:0/4183011965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bb9fe0 0x556df4ba8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.891+0000 7f3e06473b80 1 -- 172.21.15.62:0/4183011965 >> 172.21.15.62:0/4183011965 conn(0x556df4aded40 msgr2=0x556df4ba7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:06.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.891+0000 7f3e06473b80 1 -- 172.21.15.62:0/4183011965 shutdown_connections 2024-06-10T00:34:06.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.891+0000 7f3e06473b80 1 -- 172.21.15.62:0/4183011965 wait complete. 2024-06-10T00:34:06.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.891+0000 7f3e06473b80 1 -- start start 2024-06-10T00:34:06.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.892+0000 7f3e06473b80 1 Processor -- start 2024-06-10T00:34:06.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.892+0000 7f3e06473b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bddd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.892+0000 7f3e06473b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb9fe0 0x556df4bde2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.892+0000 7f3e06473b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bbd9a0 0x556df4bde810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.892+0000 7f3e06473b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41ad7b0 con 0x556df4bbd9a0 2024-06-10T00:34:06.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.892+0000 7f3e06473b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41ad0e0 con 0x556df4bb96c0 2024-06-10T00:34:06.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.892+0000 7f3e06473b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41aa2e0 con 0x556df4bb9fe0 2024-06-10T00:34:06.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.892+0000 7f3dfa18d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bbd9a0 0x556df4bde810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.893+0000 7f3df998c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bddd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.893+0000 7f3dfa18d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bbd9a0 0x556df4bde810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35324/0 (socket says 172.21.15.62:35324) 2024-06-10T00:34:06.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.893+0000 7f3dfa18d700 1 -- 172.21.15.62:0/1153376462 learned_addr learned my addr 172.21.15.62:0/1153376462 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:06.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.893+0000 7f3e06473b80 10 client.0 _ll_register_callbacks cb 0x556df4a9f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:06.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.893+0000 7f3df918b700 1 --2- 172.21.15.62:0/1153376462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb9fe0 0x556df4bde2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.893+0000 7f3dfa18d700 1 -- 172.21.15.62:0/1153376462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb9fe0 msgr2=0x556df4bde2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:06.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.893+0000 7f3dfa18d700 1 --2- 172.21.15.62:0/1153376462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb9fe0 0x556df4bde2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.893+0000 7f3dfa18d700 1 -- 172.21.15.62:0/1153376462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 msgr2=0x556df4bddd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:06.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.893+0000 7f3dfa18d700 1 --2- 172.21.15.62:0/1153376462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bddd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.893+0000 7f3dfa18d700 1 -- 172.21.15.62:0/1153376462 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556df4c400f0 con 0x556df4bbd9a0 2024-06-10T00:34:06.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.893+0000 7f3df998c700 1 --2- 172.21.15.62:0/1153376462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bddd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:06.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.893+0000 7f3df918b700 1 --2- 172.21.15.62:0/1153376462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb9fe0 0x556df4bde2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:06.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.893+0000 7f3dfa18d700 1 --2- 172.21.15.62:0/1153376462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bbd9a0 0x556df4bde810 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3dec008f40 tx=0x7f3dec009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08f01f8375c4293 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:06.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.893+0000 7f3ddd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:06.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.894+0000 7f3ddd7fa700 1 -- 172.21.15.62:0/1153376462 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec024600 con 0x556df4bbd9a0 2024-06-10T00:34:06.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.894+0000 7f3ddd7fa700 1 -- 172.21.15.62:0/1153376462 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f3dec0247a0 con 0x556df4bbd9a0 2024-06-10T00:34:06.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.894+0000 7f3e06473b80 1 -- 172.21.15.62:0/1153376462 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3df00d0e50 con 0x556df4bbd9a0 2024-06-10T00:34:06.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.894+0000 7f3ddd7fa700 1 -- 172.21.15.62:0/1153376462 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3dec016e50 con 0x556df4bbd9a0 2024-06-10T00:34:06.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.894+0000 7f3e06473b80 10 client.15798 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:06.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.894+0000 7f3e06473b80 1 -- 172.21.15.62:0/1153376462 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556df4c41430 con 0x556df4bbd9a0 2024-06-10T00:34:06.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.895+0000 7f3ddd7fa700 1 -- 172.21.15.62:0/1153376462 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df00d0e50 con 0x556df4bbd9a0 2024-06-10T00:34:06.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.895+0000 7f3ddd7fa700 1 -- 172.21.15.62:0/1153376462 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3dec023040 con 0x556df4bbd9a0 2024-06-10T00:34:06.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.895+0000 7f3e06473b80 10 client.15798 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:06.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.895+0000 7f3e06473b80 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:34:06.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.895+0000 7f3e06473b80 20 client.15798 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:06.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.895+0000 7f3e06473b80 1 --2- 172.21.15.62:0/1153376462 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556df4c41b30 0x556df4c43f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:06.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.895+0000 7f3e06473b80 4 client.15798 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T00:34:06.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.895+0000 7f3e06473b80 1 -- 172.21.15.62:0/1153376462 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556df4b0d310 con 0x556df4c41b30 2024-06-10T00:34:06.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.895+0000 7f3df998c700 1 --2- 172.21.15.62:0/1153376462 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556df4c41b30 0x556df4c43f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:06.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.896+0000 7f3ddd7fa700 10 client.15798 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:06.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.896+0000 7f3df998c700 1 --2- 172.21.15.62:0/1153376462 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556df4c41b30 0x556df4c43f50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7de2546b34c9342 server_cookie=13a5a703f259a8f6 in_seq=0 out_seq=0 2024-06-10T00:34:06.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.897+0000 7f3ddd7fa700 1 -- 172.21.15.62:0/1153376462 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556df4b0d310 con 0x556df4c41b30 2024-06-10T00:34:06.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.897+0000 7f3ddd7fa700 10 client.15798 handle_command_reply: tid=0 2024-06-10T00:34:06.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.897+0000 7f3e06473b80 1 client.15798 shutdown 2024-06-10T00:34:06.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.898+0000 7f3e06473b80 1 -- 172.21.15.62:0/1153376462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bbd9a0 msgr2=0x556df4bde810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:06.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.898+0000 7f3e06473b80 1 --2- 172.21.15.62:0/1153376462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bbd9a0 0x556df4bde810 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3dec008f40 tx=0x7f3dec009920 comp rx=0 tx=0).stop 2024-06-10T00:34:06.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.898+0000 7f3e06473b80 1 -- 172.21.15.62:0/1153376462 shutdown_connections 2024-06-10T00:34:06.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.898+0000 7f3e06473b80 1 --2- 172.21.15.62:0/1153376462 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556df4c41b30 0x556df4c43f50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.898+0000 7f3e06473b80 1 --2- 172.21.15.62:0/1153376462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556df4bbd9a0 0x556df4bde810 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.898+0000 7f3e06473b80 1 --2- 172.21.15.62:0/1153376462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556df4bb96c0 0x556df4bddd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.898+0000 7f3e06473b80 1 --2- 172.21.15.62:0/1153376462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556df4bb9fe0 0x556df4bde2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.898+0000 7f3e06473b80 1 -- 172.21.15.62:0/1153376462 >> 172.21.15.62:0/1153376462 conn(0x556df4aded40 msgr2=0x556df4bb8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:06.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.898+0000 7f3ddd7fa700 0 client.15798 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:06.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.899+0000 7f3e06473b80 1 -- 172.21.15.62:0/1153376462 shutdown_connections 2024-06-10T00:34:06.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.899+0000 7f3e06473b80 1 -- 172.21.15.62:0/1153376462 wait complete. 2024-06-10T00:34:06.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.899+0000 7f3e06473b80 20 client.15798 trim_cache size 0 max 16384 2024-06-10T00:34:06.901 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:34:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-06-10T00:34:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "00eb3ccb-3eb5-4cbf-a802-bbcb8fc90164", 2024-06-10T00:34:06.901 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-06-10T00:34:06.901 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:34:06.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.899+0000 7f3dfbbf0700 1 -- 172.21.15.62:0/2991950405 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3de00a2f10 msgr2=0x7f3de00a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:06.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.900+0000 7f3dfbbf0700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3de00a2f10 0x7f3de00a53d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3dec0075c0 tx=0x7f3dec004ad0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.900+0000 7f3dfbbf0700 1 -- 172.21.15.62:0/2991950405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df4101770 msgr2=0x7f3df419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:06.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.900+0000 7f3dfbbf0700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df4101770 0x7f3df419c670 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3df00059d0 tx=0x7f3df000bab0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.900+0000 7f3dfbbf0700 1 -- 172.21.15.62:0/2991950405 shutdown_connections 2024-06-10T00:34:06.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.900+0000 7f3dfbbf0700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3de00a2f10 0x7f3de00a53d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.900+0000 7f3dfbbf0700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3df4101770 0x7f3df419c670 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.900+0000 7f3dfbbf0700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3df4107550 0x7f3df4196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.900+0000 7f3dfbbf0700 1 --2- 172.21.15.62:0/2991950405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df4102130 0x7f3df419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:06.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.900+0000 7f3dfbbf0700 1 -- 172.21.15.62:0/2991950405 >> 172.21.15.62:0/2991950405 conn(0x7f3df4078340 msgr2=0x7f3df4100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:06.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.900+0000 7f3dfbbf0700 1 -- 172.21.15.62:0/2991950405 shutdown_connections 2024-06-10T00:34:06.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:06.900+0000 7f3dfbbf0700 1 -- 172.21.15.62:0/2991950405 wait complete. 2024-06-10T00:34:07.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:07 smithi062 ceph-mon[32390]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-10T00:34:07.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:07 smithi062 ceph-mon[32390]: scrub queued for path: / 2024-06-10T00:34:07.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:07 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:34:07.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:07 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:34:07.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:07 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:34:07.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:07 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:34:07.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:07 smithi138 ceph-mon[36549]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-10T00:34:07.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:07 smithi138 ceph-mon[36549]: scrub queued for path: / 2024-06-10T00:34:07.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:07 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:34:07.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:07 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:34:07.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:07 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:34:07.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:07 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:34:07.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:07 smithi105 ceph-mon[36321]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-10T00:34:07.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:07 smithi105 ceph-mon[36321]: scrub queued for path: / 2024-06-10T00:34:07.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:07 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:34:07.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:07 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:34:07.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:07 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:34:07.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:07 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:34:07.900 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:34:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:09 smithi062 ceph-mon[32390]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-10T00:34:09.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:09 smithi138 ceph-mon[36549]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-10T00:34:09.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:09 smithi105 ceph-mon[36321]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-10T00:34:11.603 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:11 smithi062 ceph-mon[32390]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-10T00:34:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:11 smithi138 ceph-mon[36549]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-10T00:34:11.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:11 smithi105 ceph-mon[36321]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-10T00:34:13.494 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:13 smithi062 ceph-mon[32390]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-06-10T00:34:13.704 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:13 smithi138 ceph-mon[36549]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-06-10T00:34:13.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:13 smithi105 ceph-mon[36321]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-06-10T00:34:13.788 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:13.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.833+0000 7f7e38f7a700 1 -- 172.21.15.62:0/1751295334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e34101440 msgr2=0x7f7e34105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:13.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.833+0000 7f7e38f7a700 1 --2- 172.21.15.62:0/1751295334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e34101440 0x7f7e34105fc0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00b0a0 tx=0x7f7e2c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.833+0000 7f7e38f7a700 1 -- 172.21.15.62:0/1751295334 shutdown_connections 2024-06-10T00:34:13.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.833+0000 7f7e38f7a700 1 --2- 172.21.15.62:0/1751295334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e34101440 0x7f7e34105fc0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.833+0000 7f7e38f7a700 1 --2- 172.21.15.62:0/1751295334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e34107430 0x7f7e34107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.833+0000 7f7e38f7a700 1 --2- 172.21.15.62:0/1751295334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e34100a80 0x7f7e34100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.833+0000 7f7e38f7a700 1 -- 172.21.15.62:0/1751295334 >> 172.21.15.62:0/1751295334 conn(0x7f7e34077a30 msgr2=0x7f7e340fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:13.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.833+0000 7f7e38f7a700 1 -- 172.21.15.62:0/1751295334 shutdown_connections 2024-06-10T00:34:13.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.833+0000 7f7e38f7a700 1 -- 172.21.15.62:0/1751295334 wait complete. 2024-06-10T00:34:13.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.834+0000 7f7e38f7a700 1 Processor -- start 2024-06-10T00:34:13.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.835+0000 7f7e38f7a700 1 -- start start 2024-06-10T00:34:13.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.835+0000 7f7e38f7a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e34100a80 0x7f7e34072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.835+0000 7f7e38f7a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e34101440 0x7f7e34073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.835+0000 7f7e38f7a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e34107430 0x7f7e3410c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.835+0000 7f7e38f7a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7e34118e30 con 0x7f7e34100a80 2024-06-10T00:34:13.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.835+0000 7f7e38f7a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7e34118cb0 con 0x7f7e34101440 2024-06-10T00:34:13.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.835+0000 7f7e38f7a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e34118fb0 con 0x7f7e34107430 2024-06-10T00:34:13.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.836+0000 7f7e3259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e34100a80 0x7f7e34072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.836+0000 7f7e3259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e34100a80 0x7f7e34072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42480/0 (socket says 172.21.15.62:42480) 2024-06-10T00:34:13.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.836+0000 7f7e3259c700 1 -- 172.21.15.62:0/2984359063 learned_addr learned my addr 172.21.15.62:0/2984359063 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:13.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.836+0000 7f7e31d9b700 1 --2- 172.21.15.62:0/2984359063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e34101440 0x7f7e34073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.837+0000 7f7e3259c700 1 -- 172.21.15.62:0/2984359063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e34107430 msgr2=0x7f7e3410c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:13.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.837+0000 7f7e3259c700 1 --2- 172.21.15.62:0/2984359063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e34107430 0x7f7e3410c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.837+0000 7f7e3259c700 1 -- 172.21.15.62:0/2984359063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e34101440 msgr2=0x7f7e34073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:13.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.837+0000 7f7e3259c700 1 --2- 172.21.15.62:0/2984359063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e34101440 0x7f7e34073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.837+0000 7f7e3259c700 1 -- 172.21.15.62:0/2984359063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3410c9a0 con 0x7f7e34100a80 2024-06-10T00:34:13.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.837+0000 7f7e3259c700 1 --2- 172.21.15.62:0/2984359063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e34100a80 0x7f7e34072d10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00ed60 tx=0x7f7e1c00c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:13.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.838+0000 7f7e2b7fe700 1 -- 172.21.15.62:0/2984359063 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c00cbc0 con 0x7f7e34100a80 2024-06-10T00:34:13.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.838+0000 7f7e2b7fe700 1 -- 172.21.15.62:0/2984359063 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e1c00cd60 con 0x7f7e34100a80 2024-06-10T00:34:13.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.838+0000 7f7e38f7a700 1 -- 172.21.15.62:0/2984359063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3410cc90 con 0x7f7e34100a80 2024-06-10T00:34:13.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.838+0000 7f7e38f7a700 1 -- 172.21.15.62:0/2984359063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e34113330 con 0x7f7e34100a80 2024-06-10T00:34:13.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.841+0000 7f7e2b7fe700 1 -- 172.21.15.62:0/2984359063 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c018820 con 0x7f7e34100a80 2024-06-10T00:34:13.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.844+0000 7f7e2b7fe700 1 -- 172.21.15.62:0/2984359063 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f7e1c018a80 con 0x7f7e34100a80 2024-06-10T00:34:13.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.845+0000 7f7e437fdb80 1 Processor -- start 2024-06-10T00:34:13.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.845+0000 7f7e437fdb80 1 -- start start 2024-06-10T00:34:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e437fdb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 0x56301692f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e437fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692f5f0 0x563016928f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e437fdb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x5630169298a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e437fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56301687a9a0 con 0x56301692f5f0 2024-06-10T00:34:13.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e437fdb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5630168228b0 con 0x563016929460 2024-06-10T00:34:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e437fdb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563016876960 con 0x56301692ecd0 2024-06-10T00:34:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e31d9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x5630169298a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e31d9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x5630169298a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:43146/0 (socket says 172.21.15.62:43146) 2024-06-10T00:34:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e31d9b700 1 -- 172.21.15.62:0/4172612909 learned_addr learned my addr 172.21.15.62:0/4172612909 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e3259c700 1 --2- 172.21.15.62:0/4172612909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692f5f0 0x563016928f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e32d9d700 1 --2- 172.21.15.62:0/4172612909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 0x56301692f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e31d9b700 1 -- 172.21.15.62:0/4172612909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 msgr2=0x56301692f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e31d9b700 1 --2- 172.21.15.62:0/4172612909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 0x56301692f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e31d9b700 1 -- 172.21.15.62:0/4172612909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692f5f0 msgr2=0x563016928f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.846+0000 7f7e31d9b700 1 --2- 172.21.15.62:0/4172612909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692f5f0 0x563016928f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.847+0000 7f7e31d9b700 1 -- 172.21.15.62:0/4172612909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630168190e0 con 0x563016929460 2024-06-10T00:34:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.847+0000 7f7e3259c700 1 --2- 172.21.15.62:0/4172612909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692f5f0 0x563016928f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:34:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.847+0000 7f7e31d9b700 1 --2- 172.21.15.62:0/4172612909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x5630169298a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009ca0 tx=0x7f7e240113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=597cfa55523b7f9b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.847+0000 7f7e2b7fe700 1 --2- 172.21.15.62:0/2984359063 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7e200a3130 0x7f7e200a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.847+0000 7f7e28ff9700 1 -- 172.21.15.62:0/4172612909 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24011c60 con 0x563016929460 2024-06-10T00:34:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.847+0000 7f7e28ff9700 1 -- 172.21.15.62:0/4172612909 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e24011e00 con 0x563016929460 2024-06-10T00:34:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.847+0000 7f7e28ff9700 1 -- 172.21.15.62:0/4172612909 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2400f840 con 0x563016929460 2024-06-10T00:34:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.847+0000 7f7e32d9d700 1 --2- 172.21.15.62:0/2984359063 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7e200a3130 0x7f7e200a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.848+0000 7f7e32d9d700 1 --2- 172.21.15.62:0/2984359063 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7e200a3130 0x7f7e200a55f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00b780 tx=0x7f7e2c009780 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.849+0000 7f7e437fdb80 1 -- 172.21.15.62:0/4172612909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 msgr2=0x5630169298a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.849+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/4172612909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x5630169298a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009ca0 tx=0x7f7e240113e0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.849+0000 7f7e437fdb80 1 -- 172.21.15.62:0/4172612909 shutdown_connections 2024-06-10T00:34:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.849+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/4172612909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x5630169298a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.849+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/4172612909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 0x56301692f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.849+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/4172612909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692f5f0 0x563016928f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.849+0000 7f7e437fdb80 1 -- 172.21.15.62:0/4172612909 >> 172.21.15.62:0/4172612909 conn(0x563016853e50 msgr2=0x5630169241b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.849+0000 7f7e437fdb80 1 -- 172.21.15.62:0/4172612909 shutdown_connections 2024-06-10T00:34:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.849+0000 7f7e437fdb80 1 -- 172.21.15.62:0/4172612909 wait complete. 2024-06-10T00:34:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.850+0000 7f7e437fdb80 1 -- start start 2024-06-10T00:34:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.850+0000 7f7e437fdb80 1 Processor -- start 2024-06-10T00:34:13.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.850+0000 7f7e437fdb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x5630169352a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.850+0000 7f7e437fdb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 0x5630169325a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.850+0000 7f7e437fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692f5f0 0x563016932ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.850+0000 7f7e437fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563016936060 con 0x56301692f5f0 2024-06-10T00:34:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.850+0000 7f7e32d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692f5f0 0x563016932ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.850+0000 7f7e32d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692f5f0 0x563016932ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42502/0 (socket says 172.21.15.62:42502) 2024-06-10T00:34:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.850+0000 7f7e32d9d700 1 -- 172.21.15.62:0/2549527150 learned_addr learned my addr 172.21.15.62:0/2549527150 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.850+0000 7f7e437fdb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56301681ad80 con 0x563016929460 2024-06-10T00:34:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.850+0000 7f7e437fdb80 1 -- 172.21.15.62:0/2549527150 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5630169361e0 con 0x56301692ecd0 2024-06-10T00:34:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.851+0000 7f7e31d9b700 1 --2- 172.21.15.62:0/2549527150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 0x5630169325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.851+0000 7f7e437fdb80 10 client.0 _ll_register_callbacks cb 0x563016814ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.851+0000 7f7e32d9d700 1 -- 172.21.15.62:0/2549527150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 msgr2=0x5630169325a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.851+0000 7f7e32d9d700 1 --2- 172.21.15.62:0/2549527150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 0x5630169325a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.851+0000 7f7e32d9d700 1 -- 172.21.15.62:0/2549527150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 msgr2=0x5630169352a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.851+0000 7f7e2b7fe700 1 -- 172.21.15.62:0/2984359063 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f7e1c010430 con 0x7f7e34100a80 2024-06-10T00:34:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.851+0000 7f7e3259c700 1 --2- 172.21.15.62:0/2549527150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x5630169352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.851+0000 7f7e32d9d700 1 --2- 172.21.15.62:0/2549527150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x5630169352a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.851+0000 7f7e32d9d700 1 -- 172.21.15.62:0/2549527150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630167c9a60 con 0x56301692f5f0 2024-06-10T00:34:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.851+0000 7f7e32d9d700 1 --2- 172.21.15.62:0/2549527150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692f5f0 0x563016932ae0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c015000 tx=0x7f7e2c006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3eb23fb35648ce2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.851+0000 7f7e19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.852+0000 7f7e19ffb700 1 -- 172.21.15.62:0/2549527150 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c02d020 con 0x56301692f5f0 2024-06-10T00:34:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.852+0000 7f7e19ffb700 1 -- 172.21.15.62:0/2549527150 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f7e2c007970 con 0x56301692f5f0 2024-06-10T00:34:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.852+0000 7f7e437fdb80 1 -- 172.21.15.62:0/2549527150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e24011e00 con 0x56301692f5f0 2024-06-10T00:34:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.852+0000 7f7e19ffb700 1 -- 172.21.15.62:0/2549527150 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7e2c007bb0 con 0x56301692f5f0 2024-06-10T00:34:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.852+0000 7f7e437fdb80 10 client.15810 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.852+0000 7f7e437fdb80 1 -- 172.21.15.62:0/2549527150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5630169408f0 con 0x56301692f5f0 2024-06-10T00:34:13.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.853+0000 7f7e19ffb700 1 -- 172.21.15.62:0/2549527150 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e24011e00 con 0x56301692f5f0 2024-06-10T00:34:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.853+0000 7f7e19ffb700 1 -- 172.21.15.62:0/2549527150 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7e2c011070 con 0x56301692f5f0 2024-06-10T00:34:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.853+0000 7f7e437fdb80 10 client.15810 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.853+0000 7f7e437fdb80 10 client.15810 resolve_mds: validated gid 24464 aka daemon mds.b 2024-06-10T00:34:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.853+0000 7f7e437fdb80 20 client.15810 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.853+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/2549527150 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563016942f80 0x5630169453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.853+0000 7f7e437fdb80 4 client.15810 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:34:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.853+0000 7f7e437fdb80 1 -- 172.21.15.62:0/2549527150 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563016887ea0 con 0x563016942f80 2024-06-10T00:34:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.853+0000 7f7e3259c700 1 --2- 172.21.15.62:0/2549527150 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563016942f80 0x5630169453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.854+0000 7f7e19ffb700 10 client.15810 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.854+0000 7f7e3259c700 1 --2- 172.21.15.62:0/2549527150 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563016942f80 0x5630169453f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e86e477f9e92c4e server_cookie=d6b18769e79fcf5a in_seq=0 out_seq=0 2024-06-10T00:34:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.855+0000 7f7e19ffb700 1 -- 172.21.15.62:0/2549527150 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563016887ea0 con 0x563016942f80 2024-06-10T00:34:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.855+0000 7f7e19ffb700 10 client.15810 handle_command_reply: tid=0 2024-06-10T00:34:13.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.855+0000 7f7e437fdb80 1 client.15810 shutdown 2024-06-10T00:34:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.855+0000 7f7e437fdb80 1 -- 172.21.15.62:0/2549527150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692f5f0 msgr2=0x563016932ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.855+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/2549527150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692f5f0 0x563016932ae0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c015000 tx=0x7f7e2c006000 comp rx=0 tx=0).stop 2024-06-10T00:34:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.855+0000 7f7e437fdb80 1 -- 172.21.15.62:0/2549527150 shutdown_connections 2024-06-10T00:34:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.855+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/2549527150 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563016942f80 0x5630169453f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.855+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/2549527150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692f5f0 0x563016932ae0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.855+0000 7f7e19ffb700 0 client.15810 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.855+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/2549527150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x5630169352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.855+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/2549527150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 0x5630169325a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.855+0000 7f7e437fdb80 1 -- 172.21.15.62:0/2549527150 >> 172.21.15.62:0/2549527150 conn(0x563016853e50 msgr2=0x563016923da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.856+0000 7f7e437fdb80 1 -- 172.21.15.62:0/2549527150 shutdown_connections 2024-06-10T00:34:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.856+0000 7f7e437fdb80 1 -- 172.21.15.62:0/2549527150 wait complete. 2024-06-10T00:34:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.856+0000 7f7e437fdb80 20 client.15810 trim_cache size 0 max 16384 2024-06-10T00:34:13.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.897+0000 7f7e437fdb80 1 Processor -- start 2024-06-10T00:34:13.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.898+0000 7f7e437fdb80 1 -- start start 2024-06-10T00:34:13.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.898+0000 7f7e437fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563016929460 0x563016928f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.898+0000 7f7e437fdb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 0x56301692cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.898+0000 7f7e437fdb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56301692f5f0 0x56301692f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.898+0000 7f7e437fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1400edb0 con 0x563016929460 2024-06-10T00:34:13.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.898+0000 7f7e437fdb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7e140108f0 con 0x56301692f5f0 2024-06-10T00:34:13.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.898+0000 7f7e437fdb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1400f480 con 0x56301692ecd0 2024-06-10T00:34:13.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.898+0000 7f7e3259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563016929460 0x563016928f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.898+0000 7f7e3259c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563016929460 0x563016928f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42514/0 (socket says 172.21.15.62:42514) 2024-06-10T00:34:13.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.898+0000 7f7e3259c700 1 -- 172.21.15.62:0/2907518585 learned_addr learned my addr 172.21.15.62:0/2907518585 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:13.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.898+0000 7f7e31d9b700 1 --2- 172.21.15.62:0/2907518585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 0x56301692cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.898+0000 7f7e32d9d700 1 --2- 172.21.15.62:0/2907518585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56301692f5f0 0x56301692f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.899+0000 7f7e3259c700 1 -- 172.21.15.62:0/2907518585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 msgr2=0x56301692cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:13.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.899+0000 7f7e3259c700 1 --2- 172.21.15.62:0/2907518585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 0x56301692cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.899+0000 7f7e3259c700 1 -- 172.21.15.62:0/2907518585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56301692f5f0 msgr2=0x56301692f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:13.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.899+0000 7f7e3259c700 1 --2- 172.21.15.62:0/2907518585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56301692f5f0 0x56301692f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.899+0000 7f7e3259c700 1 -- 172.21.15.62:0/2907518585 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563016933b40 con 0x563016929460 2024-06-10T00:34:13.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.899+0000 7f7e3259c700 1 --2- 172.21.15.62:0/2907518585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563016929460 0x563016928f20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00c9b0 tx=0x7f7e1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51053de99a33bbb3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:13.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.899+0000 7f7e1bfff700 1 -- 172.21.15.62:0/2907518585 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c09c7b0 con 0x563016929460 2024-06-10T00:34:13.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.899+0000 7f7e1bfff700 1 -- 172.21.15.62:0/2907518585 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e1c09d370 con 0x563016929460 2024-06-10T00:34:13.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.899+0000 7f7e1bfff700 1 -- 172.21.15.62:0/2907518585 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c09c5d0 con 0x563016929460 2024-06-10T00:34:13.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.900+0000 7f7e437fdb80 1 -- 172.21.15.62:0/2907518585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563016929460 msgr2=0x563016928f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:13.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.900+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/2907518585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563016929460 0x563016928f20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00c9b0 tx=0x7f7e1c017000 comp rx=0 tx=0).stop 2024-06-10T00:34:13.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.900+0000 7f7e437fdb80 1 -- 172.21.15.62:0/2907518585 shutdown_connections 2024-06-10T00:34:13.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.900+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/2907518585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563016929460 0x563016928f20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.900+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/2907518585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56301692f5f0 0x56301692f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.900+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/2907518585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692ecd0 0x56301692cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.900+0000 7f7e437fdb80 1 -- 172.21.15.62:0/2907518585 >> 172.21.15.62:0/2907518585 conn(0x563016853e50 msgr2=0x56301692c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:13.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.901+0000 7f7e437fdb80 1 -- 172.21.15.62:0/2907518585 shutdown_connections 2024-06-10T00:34:13.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.901+0000 7f7e437fdb80 1 -- 172.21.15.62:0/2907518585 wait complete. 2024-06-10T00:34:13.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.901+0000 7f7e437fdb80 1 -- start start 2024-06-10T00:34:13.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.901+0000 7f7e437fdb80 1 Processor -- start 2024-06-10T00:34:13.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.902+0000 7f7e437fdb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x56301693bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.902+0000 7f7e437fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692ecd0 0x5630169384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.902+0000 7f7e31d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692ecd0 0x5630169384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.902+0000 7f7e3259c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x56301693bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.902+0000 7f7e3259c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x56301693bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:43172/0 (socket says 172.21.15.62:43172) 2024-06-10T00:34:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.902+0000 7f7e31d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692ecd0 0x5630169384d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42524/0 (socket says 172.21.15.62:42524) 2024-06-10T00:34:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.902+0000 7f7e3259c700 1 -- 172.21.15.62:0/3857413196 learned_addr learned my addr 172.21.15.62:0/3857413196 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.902+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/3857413196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692f5f0 0x563016938a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.902+0000 7f7e437fdb80 1 -- 172.21.15.62:0/3857413196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7e14010220 con 0x56301692ecd0 2024-06-10T00:34:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.902+0000 7f7e437fdb80 1 -- 172.21.15.62:0/3857413196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1400fb50 con 0x563016929460 2024-06-10T00:34:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.902+0000 7f7e437fdb80 1 -- 172.21.15.62:0/3857413196 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1400cd50 con 0x56301692f5f0 2024-06-10T00:34:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.903+0000 7f7e32d9d700 1 --2- 172.21.15.62:0/3857413196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692f5f0 0x563016938a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.903+0000 7f7e3259c700 1 -- 172.21.15.62:0/3857413196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692f5f0 msgr2=0x563016938a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.903+0000 7f7e3259c700 1 --2- 172.21.15.62:0/3857413196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692f5f0 0x563016938a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.903+0000 7f7e3259c700 1 -- 172.21.15.62:0/3857413196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692ecd0 msgr2=0x5630169384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.903+0000 7f7e3259c700 1 --2- 172.21.15.62:0/3857413196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692ecd0 0x5630169384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.903+0000 7f7e3259c700 1 -- 172.21.15.62:0/3857413196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563016939140 con 0x563016929460 2024-06-10T00:34:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.903+0000 7f7e31d9b700 1 --2- 172.21.15.62:0/3857413196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692ecd0 0x5630169384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:34:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.903+0000 7f7e3259c700 1 --2- 172.21.15.62:0/3857413196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x56301693bb40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c099640 tx=0x7f7e1c0cdf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0ab676e277ef5a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.903+0000 7f7e19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:34:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.903+0000 7f7e437fdb80 10 client.0 _ll_register_callbacks cb 0x563016814ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.903+0000 7f7e437fdb80 1 -- 172.21.15.62:0/3857413196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e1c09d370 con 0x563016929460 2024-06-10T00:34:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.904+0000 7f7e19ffb700 1 -- 172.21.15.62:0/3857413196 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c09c7b0 con 0x563016929460 2024-06-10T00:34:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.904+0000 7f7e19ffb700 1 -- 172.21.15.62:0/3857413196 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f7e1c0ce250 con 0x563016929460 2024-06-10T00:34:13.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.904+0000 7f7e19ffb700 1 -- 172.21.15.62:0/3857413196 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7e1c0ce4d0 con 0x563016929460 2024-06-10T00:34:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.904+0000 7f7e437fdb80 10 client.25403 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.904+0000 7f7e437fdb80 1 -- 172.21.15.62:0/3857413196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5630169ae9a0 con 0x563016929460 2024-06-10T00:34:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.905+0000 7f7e19ffb700 1 -- 172.21.15.62:0/3857413196 <== mon.2 v2:172.21.15.105:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7e1c09c0c0 con 0x563016929460 2024-06-10T00:34:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.905+0000 7f7e437fdb80 10 client.25403 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.905+0000 7f7e437fdb80 10 client.25403 resolve_mds: validated gid 24464 aka daemon mds.b 2024-06-10T00:34:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.905+0000 7f7e437fdb80 20 client.25403 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.905+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/3857413196 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563016942f80 0x5630169b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.905+0000 7f7e437fdb80 4 client.25403 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-10T00:34:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.905+0000 7f7e437fdb80 1 -- 172.21.15.62:0/3857413196 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7f7e1c09d370 con 0x563016942f80 2024-06-10T00:34:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.906+0000 7f7e31d9b700 1 --2- 172.21.15.62:0/3857413196 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563016942f80 0x5630169b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.906+0000 7f7e19ffb700 10 client.25403 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:13.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.906+0000 7f7e31d9b700 1 --2- 172.21.15.62:0/3857413196 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563016942f80 0x5630169b0fd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e87fdeb7a3f2082a server_cookie=560c9a62d2a6a87b in_seq=0 out_seq=0 2024-06-10T00:34:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.915+0000 7f7e19ffb700 1 -- 172.21.15.62:0/3857413196 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f7e1c09d370 con 0x563016942f80 2024-06-10T00:34:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.915+0000 7f7e19ffb700 10 client.25403 handle_command_reply: tid=0 2024-06-10T00:34:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.915+0000 7f7e437fdb80 1 client.25403 shutdown 2024-06-10T00:34:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.915+0000 7f7e437fdb80 1 -- 172.21.15.62:0/3857413196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 msgr2=0x56301693bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:13.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.915+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/3857413196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x56301693bb40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c099640 tx=0x7f7e1c0cdf50 comp rx=0 tx=0).stop 2024-06-10T00:34:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.916+0000 7f7e437fdb80 1 -- 172.21.15.62:0/3857413196 shutdown_connections 2024-06-10T00:34:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.916+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/3857413196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56301692f5f0 0x563016938a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.916+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/3857413196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563016929460 0x56301693bb40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.916+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/3857413196 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563016942f80 0x5630169b0fd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.916+0000 7f7e19ffb700 0 client.25403 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.916+0000 7f7e437fdb80 1 --2- 172.21.15.62:0/3857413196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56301692ecd0 0x5630169384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.916+0000 7f7e437fdb80 1 -- 172.21.15.62:0/3857413196 >> 172.21.15.62:0/3857413196 conn(0x563016853e50 msgr2=0x56301692f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:13.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.916+0000 7f7e437fdb80 1 -- 172.21.15.62:0/3857413196 shutdown_connections 2024-06-10T00:34:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.916+0000 7f7e437fdb80 1 -- 172.21.15.62:0/3857413196 wait complete. 2024-06-10T00:34:13.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.917+0000 7f7e437fdb80 20 client.25403 trim_cache size 0 max 16384 2024-06-10T00:34:13.919 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:34:13.919 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 30, 2024-06-10T00:34:13.919 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24464, 2024-06-10T00:34:13.919 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 2, 2024-06-10T00:34:13.919 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-06-10T00:34:13.919 INFO:teuthology.orchestra.run.smithi062.stdout: "8081428b": { 2024-06-10T00:34:13.919 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 2, 2024-06-10T00:34:13.919 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-06-10T00:34:13.919 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-06-10T00:34:13.919 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-06-10T00:34:13.919 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-06-10T00:34:13.919 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:34:13.920 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-06-10T00:34:13.920 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 150.0, 2024-06-10T00:34:13.920 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-06-10T00:34:13.920 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-06-10T00:34:13.920 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-06-10T00:34:13.920 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-06-10T00:34:13.920 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-06-10T00:34:13.920 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-06-10T00:34:13.920 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:13.920 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:13.920 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:13.920 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:13.920 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:13.920 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:34:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.918+0000 7f7e38f7a700 1 -- 172.21.15.62:0/2984359063 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7e200a3130 msgr2=0x7f7e200a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.918+0000 7f7e38f7a700 1 --2- 172.21.15.62:0/2984359063 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7e200a3130 0x7f7e200a55f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00b780 tx=0x7f7e2c009780 comp rx=0 tx=0).stop 2024-06-10T00:34:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.918+0000 7f7e38f7a700 1 -- 172.21.15.62:0/2984359063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e34100a80 msgr2=0x7f7e34072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.918+0000 7f7e38f7a700 1 --2- 172.21.15.62:0/2984359063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e34100a80 0x7f7e34072d10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00ed60 tx=0x7f7e1c00c5b0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.919+0000 7f7e38f7a700 1 -- 172.21.15.62:0/2984359063 shutdown_connections 2024-06-10T00:34:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.919+0000 7f7e38f7a700 1 --2- 172.21.15.62:0/2984359063 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7e200a3130 0x7f7e200a55f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.919+0000 7f7e38f7a700 1 --2- 172.21.15.62:0/2984359063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e34100a80 0x7f7e34072d10 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.919+0000 7f7e38f7a700 1 --2- 172.21.15.62:0/2984359063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e34107430 0x7f7e3410c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.920+0000 7f7e38f7a700 1 --2- 172.21.15.62:0/2984359063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e34101440 0x7f7e34073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.920+0000 7f7e38f7a700 1 -- 172.21.15.62:0/2984359063 >> 172.21.15.62:0/2984359063 conn(0x7f7e34077a30 msgr2=0x7f7e34103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.920+0000 7f7e38f7a700 1 -- 172.21.15.62:0/2984359063 shutdown_connections 2024-06-10T00:34:13.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:13.920+0000 7f7e38f7a700 1 -- 172.21.15.62:0/2984359063 wait complete. 2024-06-10T00:34:15.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:15 smithi062 ceph-mon[32390]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-10T00:34:15.408 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8081428b', quiesce took 14.6 seconds. Will release after: 15.4 2024-06-10T00:34:15.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:15 smithi138 ceph-mon[36549]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-10T00:34:15.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:15 smithi105 ceph-mon[36321]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-10T00:34:16.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.497+0000 7f07e556d700 1 -- 172.21.15.62:0/2101066808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07e006b3c0 msgr2=0x7f07e010f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:16.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.497+0000 7f07e556d700 1 --2- 172.21.15.62:0/2101066808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07e006b3c0 0x7f07e010f310 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f07d800b0a0 tx=0x7f07d800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.497+0000 7f07e556d700 1 -- 172.21.15.62:0/2101066808 shutdown_connections 2024-06-10T00:34:16.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.497+0000 7f07e556d700 1 --2- 172.21.15.62:0/2101066808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07e006b3c0 0x7f07e010f310 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.497+0000 7f07e556d700 1 --2- 172.21.15.62:0/2101066808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07e01084f0 0x7f07e01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.497+0000 7f07e556d700 1 --2- 172.21.15.62:0/2101066808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07e006aa00 0x7f07e006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.497+0000 7f07e556d700 1 -- 172.21.15.62:0/2101066808 >> 172.21.15.62:0/2101066808 conn(0x7f07e0078300 msgr2=0x7f07e0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:16.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.497+0000 7f07e556d700 1 -- 172.21.15.62:0/2101066808 shutdown_connections 2024-06-10T00:34:16.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.497+0000 7f07e556d700 1 -- 172.21.15.62:0/2101066808 wait complete. 2024-06-10T00:34:16.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.498+0000 7f07e556d700 1 Processor -- start 2024-06-10T00:34:16.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.498+0000 7f07e556d700 1 -- start start 2024-06-10T00:34:16.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.498+0000 7f07e556d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07e006aa00 0x7f07e0101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.498+0000 7f07e556d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07e006b3c0 0x7f07e0101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.498+0000 7f07e556d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07e01084f0 0x7f07e0101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.498+0000 7f07e556d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f07e0112750 con 0x7f07e01084f0 2024-06-10T00:34:16.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.498+0000 7f07e556d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01125d0 con 0x7f07e006aa00 2024-06-10T00:34:16.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.498+0000 7f07e556d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01128d0 con 0x7f07e006b3c0 2024-06-10T00:34:16.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.499+0000 7f07de7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07e006b3c0 0x7f07e0101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.499+0000 7f07de7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07e006b3c0 0x7f07e0101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:45392/0 (socket says 172.21.15.62:45392) 2024-06-10T00:34:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.499+0000 7f07de7fc700 1 -- 172.21.15.62:0/167611502 learned_addr learned my addr 172.21.15.62:0/167611502 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.499+0000 7f07deffd700 1 --2- 172.21.15.62:0/167611502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07e006aa00 0x7f07e0101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.499+0000 7f07de7fc700 1 -- 172.21.15.62:0/167611502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07e006aa00 msgr2=0x7f07e0101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.499+0000 7f07de7fc700 1 --2- 172.21.15.62:0/167611502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07e006aa00 0x7f07e0101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.499+0000 7f07de7fc700 1 -- 172.21.15.62:0/167611502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07e01084f0 msgr2=0x7f07e0101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.499+0000 7f07de7fc700 1 --2- 172.21.15.62:0/167611502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07e01084f0 0x7f07e0101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.499+0000 7f07de7fc700 1 -- 172.21.15.62:0/167611502 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e01ab530 con 0x7f07e006b3c0 2024-06-10T00:34:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.499+0000 7f07de7fc700 1 --2- 172.21.15.62:0/167611502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07e006b3c0 0x7f07e0101990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f07d400d980 tx=0x7f07d400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.499+0000 7f07c7fff700 1 -- 172.21.15.62:0/167611502 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4016070 con 0x7f07e006b3c0 2024-06-10T00:34:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.499+0000 7f07e556d700 1 -- 172.21.15.62:0/167611502 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e01ab7c0 con 0x7f07e006b3c0 2024-06-10T00:34:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.499+0000 7f07c7fff700 1 -- 172.21.15.62:0/167611502 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d4004d10 con 0x7f07e006b3c0 2024-06-10T00:34:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.499+0000 7f07c7fff700 1 -- 172.21.15.62:0/167611502 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4005680 con 0x7f07e006b3c0 2024-06-10T00:34:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.500+0000 7f07e556d700 1 -- 172.21.15.62:0/167611502 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e01abd50 con 0x7f07e006b3c0 2024-06-10T00:34:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.503+0000 7f07c7fff700 1 -- 172.21.15.62:0/167611502 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f07d4004750 con 0x7f07e006b3c0 2024-06-10T00:34:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.504+0000 7f07c7fff700 1 --2- 172.21.15.62:0/167611502 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f07c80a2f30 0x7f07c80a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.504+0000 7f07c7fff700 1 -- 172.21.15.62:0/167611502 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f07d40cf880 con 0x7f07e006b3c0 2024-06-10T00:34:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.504+0000 7f07efdf0b80 1 Processor -- start 2024-06-10T00:34:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.504+0000 7f07efdf0b80 1 -- start start 2024-06-10T00:34:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.505+0000 7f07efdf0b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c337a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.505+0000 7f07deffd700 1 --2- 172.21.15.62:0/167611502 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f07c80a2f30 0x7f07c80a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.505+0000 7f07efdf0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c337f60 0x55963c3319e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.505+0000 7f07efdf0b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c331f20 0x55963c332340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.505+0000 7f07efdf0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55963c283810 con 0x55963c337f60 2024-06-10T00:34:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.505+0000 7f07efdf0b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55963c22b720 con 0x55963c331f20 2024-06-10T00:34:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.505+0000 7f07efdf0b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55963c27f7d0 con 0x55963c337640 2024-06-10T00:34:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.505+0000 7f07deffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c337a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.505+0000 7f07deffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c337a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:45406/0 (socket says 172.21.15.62:45406) 2024-06-10T00:34:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.505+0000 7f07deffd700 1 -- 172.21.15.62:0/4257869760 learned_addr learned my addr 172.21.15.62:0/4257869760 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.505+0000 7f07deffd700 1 --2- 172.21.15.62:0/167611502 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f07c80a2f30 0x7f07c80a53f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07d0013570 tx=0x7f07d0013930 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.506+0000 7f07deffd700 1 -- 172.21.15.62:0/4257869760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c331f20 msgr2=0x55963c332340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.506+0000 7f07deffd700 1 --2- 172.21.15.62:0/4257869760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c331f20 0x55963c332340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.506+0000 7f07deffd700 1 -- 172.21.15.62:0/4257869760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c337f60 msgr2=0x55963c3319e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.506+0000 7f07deffd700 1 --2- 172.21.15.62:0/4257869760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c337f60 0x55963c3319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.506+0000 7f07deffd700 1 -- 172.21.15.62:0/4257869760 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55963c221f50 con 0x55963c337640 2024-06-10T00:34:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.506+0000 7f07deffd700 1 --2- 172.21.15.62:0/4257869760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c337a20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f07d000f760 tx=0x7f07d001e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc33718f39fe1038 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.506+0000 7f07c57fa700 1 -- 172.21.15.62:0/4257869760 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d000fe30 con 0x55963c337640 2024-06-10T00:34:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.506+0000 7f07c57fa700 1 -- 172.21.15.62:0/4257869760 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d0023020 con 0x55963c337640 2024-06-10T00:34:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.506+0000 7f07c57fa700 1 -- 172.21.15.62:0/4257869760 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0010430 con 0x55963c337640 2024-06-10T00:34:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.507+0000 7f07efdf0b80 1 -- 172.21.15.62:0/4257869760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 msgr2=0x55963c337a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.507+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/4257869760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c337a20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f07d000f760 tx=0x7f07d001e000 comp rx=0 tx=0).stop 2024-06-10T00:34:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.507+0000 7f07efdf0b80 1 -- 172.21.15.62:0/4257869760 shutdown_connections 2024-06-10T00:34:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.507+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/4257869760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c331f20 0x55963c332340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.507+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/4257869760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c337a20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.507+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/4257869760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c337f60 0x55963c3319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.507+0000 7f07efdf0b80 1 -- 172.21.15.62:0/4257869760 >> 172.21.15.62:0/4257869760 conn(0x55963c25ccc0 msgr2=0x55963c32c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.507+0000 7f07efdf0b80 1 -- 172.21.15.62:0/4257869760 shutdown_connections 2024-06-10T00:34:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.508+0000 7f07efdf0b80 1 -- 172.21.15.62:0/4257869760 wait complete. 2024-06-10T00:34:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.508+0000 7f07efdf0b80 1 -- start start 2024-06-10T00:34:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.508+0000 7f07efdf0b80 1 Processor -- start 2024-06-10T00:34:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.508+0000 7f07efdf0b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c331f20 0x55963c33dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.508+0000 7f07efdf0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c337640 0x55963c33b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.508+0000 7f07efdf0b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 0x55963c33b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.508+0000 7f07efdf0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55963c33e970 con 0x55963c337640 2024-06-10T00:34:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.508+0000 7f07efdf0b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55963c223bf0 con 0x55963c337f60 2024-06-10T00:34:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.508+0000 7f07efdf0b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55963c33eaf0 con 0x55963c331f20 2024-06-10T00:34:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.508+0000 7f07efdf0b80 10 client.0 _ll_register_callbacks cb 0x55963c21d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.509+0000 7f07de7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c337640 0x55963c33b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.509+0000 7f07de7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c337640 0x55963c33b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42556/0 (socket says 172.21.15.62:42556) 2024-06-10T00:34:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.509+0000 7f07de7fc700 1 -- 172.21.15.62:0/1818634141 learned_addr learned my addr 172.21.15.62:0/1818634141 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.509+0000 7f07df7fe700 1 --2- 172.21.15.62:0/1818634141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 0x55963c33b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.509+0000 7f07de7fc700 1 -- 172.21.15.62:0/1818634141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c331f20 msgr2=0x55963c33dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.509+0000 7f07deffd700 1 --2- 172.21.15.62:0/1818634141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c331f20 0x55963c33dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.509+0000 7f07de7fc700 1 --2- 172.21.15.62:0/1818634141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c331f20 0x55963c33dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.509+0000 7f07de7fc700 1 -- 172.21.15.62:0/1818634141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 msgr2=0x55963c33b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.509+0000 7f07de7fc700 1 --2- 172.21.15.62:0/1818634141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 0x55963c33b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.509+0000 7f07de7fc700 1 -- 172.21.15.62:0/1818634141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55963c1d28d0 con 0x55963c337640 2024-06-10T00:34:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.509+0000 7f07df7fe700 1 --2- 172.21.15.62:0/1818634141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 0x55963c33b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:16.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.510+0000 7f07de7fc700 1 --2- 172.21.15.62:0/1818634141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c337640 0x55963c33b070 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f07d409b440 tx=0x7f07d409b1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c1eb29236807cc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:16.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.510+0000 7f07c2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:16.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.510+0000 7f07c2ff5700 1 -- 172.21.15.62:0/1818634141 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d40a43b0 con 0x55963c337640 2024-06-10T00:34:16.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.510+0000 7f07c2ff5700 1 -- 172.21.15.62:0/1818634141 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f07d4004390 con 0x55963c337640 2024-06-10T00:34:16.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.510+0000 7f07efdf0b80 1 -- 172.21.15.62:0/1818634141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d0023020 con 0x55963c337640 2024-06-10T00:34:16.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.510+0000 7f07c2ff5700 1 -- 172.21.15.62:0/1818634141 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f07d4004a40 con 0x55963c337640 2024-06-10T00:34:16.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.510+0000 7f07efdf0b80 10 client.15822 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:16.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.510+0000 7f07efdf0b80 1 -- 172.21.15.62:0/1818634141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55963c349360 con 0x55963c337640 2024-06-10T00:34:16.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.511+0000 7f07c2ff5700 1 -- 172.21.15.62:0/1818634141 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d40b6e50 con 0x55963c337640 2024-06-10T00:34:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.511+0000 7f07c2ff5700 1 -- 172.21.15.62:0/1818634141 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f07d40cf3f0 con 0x55963c337640 2024-06-10T00:34:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.511+0000 7f07efdf0b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.511+0000 7f07efdf0b80 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:34:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.511+0000 7f07efdf0b80 20 client.15822 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.511+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/1818634141 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55963c34bb80 0x55963c34df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.511+0000 7f07efdf0b80 4 client.15822 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:34:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.511+0000 7f07efdf0b80 1 -- 172.21.15.62:0/1818634141 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f07d0023020 con 0x55963c34bb80 2024-06-10T00:34:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.511+0000 7f07deffd700 1 --2- 172.21.15.62:0/1818634141 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55963c34bb80 0x55963c34df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.512+0000 7f07c2ff5700 10 client.15822 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.512+0000 7f07deffd700 1 --2- 172.21.15.62:0/1818634141 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55963c34bb80 0x55963c34df70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=668f84e4967f08ff server_cookie=27a2a99fb1d2528f in_seq=0 out_seq=0 2024-06-10T00:34:16.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.512+0000 7f07c2ff5700 1 -- 172.21.15.62:0/1818634141 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f07d0023020 con 0x55963c34bb80 2024-06-10T00:34:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.512+0000 7f07c2ff5700 10 client.15822 handle_command_reply: tid=0 2024-06-10T00:34:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.512+0000 7f07efdf0b80 1 client.15822 shutdown 2024-06-10T00:34:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.513+0000 7f07efdf0b80 1 -- 172.21.15.62:0/1818634141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c337640 msgr2=0x55963c33b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.513+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/1818634141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c337640 0x55963c33b070 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f07d409b440 tx=0x7f07d409b1d0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.513+0000 7f07efdf0b80 1 -- 172.21.15.62:0/1818634141 shutdown_connections 2024-06-10T00:34:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.513+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/1818634141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 0x55963c33b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.513+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/1818634141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c331f20 0x55963c33dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.513+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/1818634141 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55963c34bb80 0x55963c34df70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.513+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/1818634141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c337640 0x55963c33b070 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.513+0000 7f07efdf0b80 1 -- 172.21.15.62:0/1818634141 >> 172.21.15.62:0/1818634141 conn(0x55963c25ccc0 msgr2=0x55963c3309d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.513+0000 7f07c2ff5700 0 client.15822 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.513+0000 7f07efdf0b80 1 -- 172.21.15.62:0/1818634141 shutdown_connections 2024-06-10T00:34:16.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.513+0000 7f07efdf0b80 1 -- 172.21.15.62:0/1818634141 wait complete. 2024-06-10T00:34:16.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.513+0000 7f07efdf0b80 20 client.15822 trim_cache size 0 max 16384 2024-06-10T00:34:16.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.534+0000 7f07efdf0b80 1 Processor -- start 2024-06-10T00:34:16.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.534+0000 7f07efdf0b80 1 -- start start 2024-06-10T00:34:16.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07efdf0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c331f20 0x55963c3319e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07efdf0b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c326690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07efdf0b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 0x55963c339f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07efdf0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01b4c50 con 0x55963c331f20 2024-06-10T00:34:16.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07efdf0b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01b6790 con 0x55963c337f60 2024-06-10T00:34:16.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07efdf0b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01b5320 con 0x55963c337640 2024-06-10T00:34:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07deffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c331f20 0x55963c3319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07deffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c331f20 0x55963c3319e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42572/0 (socket says 172.21.15.62:42572) 2024-06-10T00:34:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07deffd700 1 -- 172.21.15.62:0/2716018327 learned_addr learned my addr 172.21.15.62:0/2716018327 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:16.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07de7fc700 1 --2- 172.21.15.62:0/2716018327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c326690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:16.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07de7fc700 1 -- 172.21.15.62:0/2716018327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 msgr2=0x55963c339f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:34:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07de7fc700 1 --2- 172.21.15.62:0/2716018327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 0x55963c339f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07de7fc700 1 -- 172.21.15.62:0/2716018327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c331f20 msgr2=0x55963c3319e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07de7fc700 1 --2- 172.21.15.62:0/2716018327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c331f20 0x55963c3319e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07de7fc700 1 -- 172.21.15.62:0/2716018327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55963c32e080 con 0x55963c337640 2024-06-10T00:34:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.535+0000 7f07deffd700 1 --2- 172.21.15.62:0/2716018327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c331f20 0x55963c3319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:34:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.536+0000 7f07de7fc700 1 --2- 172.21.15.62:0/2716018327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c326690 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07d400d950 tx=0x7f07d4005120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f59667f974b288ab server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.536+0000 7f07c4ff9700 1 -- 172.21.15.62:0/2716018327 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4004390 con 0x55963c337640 2024-06-10T00:34:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.536+0000 7f07c4ff9700 1 -- 172.21.15.62:0/2716018327 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d409b9a0 con 0x55963c337640 2024-06-10T00:34:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.536+0000 7f07c4ff9700 1 -- 172.21.15.62:0/2716018327 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d40ccd20 con 0x55963c337640 2024-06-10T00:34:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.537+0000 7f07efdf0b80 1 -- 172.21.15.62:0/2716018327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 msgr2=0x55963c326690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.537+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/2716018327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c326690 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07d400d950 tx=0x7f07d4005120 comp rx=0 tx=0).stop 2024-06-10T00:34:16.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.537+0000 7f07efdf0b80 1 -- 172.21.15.62:0/2716018327 shutdown_connections 2024-06-10T00:34:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.537+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/2716018327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c331f20 0x55963c3319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.537+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/2716018327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 0x55963c339f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.537+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/2716018327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c326690 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.537+0000 7f07efdf0b80 1 -- 172.21.15.62:0/2716018327 >> 172.21.15.62:0/2716018327 conn(0x55963c25ccc0 msgr2=0x55963c339830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.537+0000 7f07efdf0b80 1 -- 172.21.15.62:0/2716018327 shutdown_connections 2024-06-10T00:34:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.537+0000 7f07efdf0b80 1 -- 172.21.15.62:0/2716018327 wait complete. 2024-06-10T00:34:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.537+0000 7f07efdf0b80 1 -- start start 2024-06-10T00:34:16.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.538+0000 7f07efdf0b80 1 Processor -- start 2024-06-10T00:34:16.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.538+0000 7f07efdf0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c331f20 0x55963c353230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.538+0000 7f07efdf0b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c353770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.538+0000 7f07efdf0b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 0x55963c353cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.538+0000 7f07efdf0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01b60c0 con 0x55963c331f20 2024-06-10T00:34:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.538+0000 7f07efdf0b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01b59f0 con 0x55963c337f60 2024-06-10T00:34:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.538+0000 7f07efdf0b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01b2bf0 con 0x55963c337640 2024-06-10T00:34:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.538+0000 7f07efdf0b80 10 client.0 _ll_register_callbacks cb 0x55963c21d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.538+0000 7f07de7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c353770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.538+0000 7f07df7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 0x55963c353cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.538+0000 7f07de7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c353770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:45428/0 (socket says 172.21.15.62:45428) 2024-06-10T00:34:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.538+0000 7f07de7fc700 1 -- 172.21.15.62:0/3803650663 learned_addr learned my addr 172.21.15.62:0/3803650663 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.539+0000 7f07deffd700 1 --2- 172.21.15.62:0/3803650663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c331f20 0x55963c353230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.539+0000 7f07de7fc700 1 -- 172.21.15.62:0/3803650663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 msgr2=0x55963c353cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.539+0000 7f07de7fc700 1 --2- 172.21.15.62:0/3803650663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 0x55963c353cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.539+0000 7f07de7fc700 1 -- 172.21.15.62:0/3803650663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c331f20 msgr2=0x55963c353230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:16.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.539+0000 7f07de7fc700 1 --2- 172.21.15.62:0/3803650663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c331f20 0x55963c353230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.539+0000 7f07de7fc700 1 -- 172.21.15.62:0/3803650663 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55963c3b5660 con 0x55963c337640 2024-06-10T00:34:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.539+0000 7f07df7fe700 1 --2- 172.21.15.62:0/3803650663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 0x55963c353cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.539+0000 7f07deffd700 1 --2- 172.21.15.62:0/3803650663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c331f20 0x55963c353230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.539+0000 7f07de7fc700 1 --2- 172.21.15.62:0/3803650663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c353770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f07d4097ba0 tx=0x7f07d40a3ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd072c374fe04da server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.539+0000 7f07c2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:34:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.539+0000 7f07c2ff5700 1 -- 172.21.15.62:0/3803650663 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d40ccf20 con 0x55963c337640 2024-06-10T00:34:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.539+0000 7f07efdf0b80 1 -- 172.21.15.62:0/3803650663 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d409b9a0 con 0x55963c337640 2024-06-10T00:34:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.539+0000 7f07c2ff5700 1 -- 172.21.15.62:0/3803650663 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f07d409ab90 con 0x55963c337640 2024-06-10T00:34:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.540+0000 7f07c2ff5700 1 -- 172.21.15.62:0/3803650663 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f07d409bb50 con 0x55963c337640 2024-06-10T00:34:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.540+0000 7f07efdf0b80 10 client.25402 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.540+0000 7f07efdf0b80 1 -- 172.21.15.62:0/3803650663 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55963c3b68b0 con 0x55963c337640 2024-06-10T00:34:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.540+0000 7f07c2ff5700 1 -- 172.21.15.62:0/3803650663 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d409b9a0 con 0x55963c337640 2024-06-10T00:34:16.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.540+0000 7f07c2ff5700 1 -- 172.21.15.62:0/3803650663 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f07d409a600 con 0x55963c337640 2024-06-10T00:34:16.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.540+0000 7f07efdf0b80 10 client.25402 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:16.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.540+0000 7f07efdf0b80 10 client.25402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:34:16.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.540+0000 7f07efdf0b80 20 client.25402 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:16.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.540+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/3803650663 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55963c3b6cc0 0x55963c3b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:16.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.540+0000 7f07efdf0b80 4 client.25402 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:34:16.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.540+0000 7f07efdf0b80 1 -- 172.21.15.62:0/3803650663 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55963c290d10 con 0x55963c3b6cc0 2024-06-10T00:34:16.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.541+0000 7f07deffd700 1 --2- 172.21.15.62:0/3803650663 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55963c3b6cc0 0x55963c3b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:16.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.541+0000 7f07c2ff5700 10 client.25402 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:16.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.541+0000 7f07deffd700 1 --2- 172.21.15.62:0/3803650663 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55963c3b6cc0 0x55963c3b90e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf2b6d8b1b9f302c server_cookie=3287fe5832774491 in_seq=0 out_seq=0 2024-06-10T00:34:16.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.541+0000 7f07c2ff5700 1 -- 172.21.15.62:0/3803650663 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55963c290d10 con 0x55963c3b6cc0 2024-06-10T00:34:16.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.541+0000 7f07c2ff5700 10 client.25402 handle_command_reply: tid=0 2024-06-10T00:34:16.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.541+0000 7f07efdf0b80 1 client.25402 shutdown 2024-06-10T00:34:16.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.544+0000 7f07efdf0b80 1 -- 172.21.15.62:0/3803650663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 msgr2=0x55963c353770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:16.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.544+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/3803650663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c353770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f07d4097ba0 tx=0x7f07d40a3ff0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.544+0000 7f07efdf0b80 1 -- 172.21.15.62:0/3803650663 shutdown_connections 2024-06-10T00:34:16.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.544+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/3803650663 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55963c3b6cc0 0x55963c3b90e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.544+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/3803650663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55963c331f20 0x55963c353230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.544+0000 7f07c2ff5700 0 client.25402 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:16.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.544+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/3803650663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55963c337f60 0x55963c353cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.544+0000 7f07efdf0b80 1 --2- 172.21.15.62:0/3803650663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55963c337640 0x55963c353770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.544+0000 7f07efdf0b80 1 -- 172.21.15.62:0/3803650663 >> 172.21.15.62:0/3803650663 conn(0x55963c25ccc0 msgr2=0x55963c33a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:16.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.544+0000 7f07efdf0b80 1 -- 172.21.15.62:0/3803650663 shutdown_connections 2024-06-10T00:34:16.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.544+0000 7f07efdf0b80 1 -- 172.21.15.62:0/3803650663 wait complete. 2024-06-10T00:34:16.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.544+0000 7f07efdf0b80 20 client.25402 trim_cache size 0 max 16384 2024-06-10T00:34:16.548 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:34:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-06-10T00:34:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-06-10T00:34:16.548 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:34:16.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.545+0000 7f07e556d700 1 -- 172.21.15.62:0/167611502 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f07c80a2f30 msgr2=0x7f07c80a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:16.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.545+0000 7f07e556d700 1 --2- 172.21.15.62:0/167611502 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f07c80a2f30 0x7f07c80a53f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07d0013570 tx=0x7f07d0013930 comp rx=0 tx=0).stop 2024-06-10T00:34:16.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.545+0000 7f07e556d700 1 -- 172.21.15.62:0/167611502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07e006b3c0 msgr2=0x7f07e0101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:16.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.545+0000 7f07e556d700 1 --2- 172.21.15.62:0/167611502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07e006b3c0 0x7f07e0101990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f07d400d980 tx=0x7f07d400dc90 comp rx=0 tx=0).stop 2024-06-10T00:34:16.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.545+0000 7f07e556d700 1 -- 172.21.15.62:0/167611502 shutdown_connections 2024-06-10T00:34:16.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.545+0000 7f07e556d700 1 --2- 172.21.15.62:0/167611502 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f07c80a2f30 0x7f07c80a53f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.545+0000 7f07e556d700 1 --2- 172.21.15.62:0/167611502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07e01084f0 0x7f07e0101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.545+0000 7f07e556d700 1 --2- 172.21.15.62:0/167611502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07e006aa00 0x7f07e0101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.545+0000 7f07e556d700 1 --2- 172.21.15.62:0/167611502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07e006b3c0 0x7f07e0101990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:16.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.545+0000 7f07e556d700 1 -- 172.21.15.62:0/167611502 >> 172.21.15.62:0/167611502 conn(0x7f07e0078300 msgr2=0x7f07e00ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:16.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.546+0000 7f07e556d700 1 -- 172.21.15.62:0/167611502 shutdown_connections 2024-06-10T00:34:16.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:16.546+0000 7f07e556d700 1 -- 172.21.15.62:0/167611502 wait complete. 2024-06-10T00:34:17.251 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T00:34:17.251 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:34:17.527 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:17.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:17 smithi062 ceph-mon[32390]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-06-10T00:34:17.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:17 smithi138 ceph-mon[36549]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-06-10T00:34:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:17 smithi105 ceph-mon[36321]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-06-10T00:34:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.500+0000 7fb2383c2700 1 -- 172.21.15.62:0/938467534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb230101390 msgr2=0x7fb230101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.500+0000 7fb2383c2700 1 --2- 172.21.15.62:0/938467534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb230101390 0x7fb230101810 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb220009990 tx=0x7fb220009ca0 comp rx=0 tx=0).stop 2024-06-10T00:34:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.501+0000 7fb2383c2700 1 -- 172.21.15.62:0/938467534 shutdown_connections 2024-06-10T00:34:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.501+0000 7fb2383c2700 1 --2- 172.21.15.62:0/938467534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb230101d50 0x7fb23010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.501+0000 7fb2383c2700 1 --2- 172.21.15.62:0/938467534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb230107400 0x7fb2301077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.501+0000 7fb2383c2700 1 --2- 172.21.15.62:0/938467534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb230101390 0x7fb230101810 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.501+0000 7fb2383c2700 1 -- 172.21.15.62:0/938467534 >> 172.21.15.62:0/938467534 conn(0x7fb230078340 msgr2=0x7fb230078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:19.503 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:19 smithi062 ceph-mon[32390]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T00:34:19.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.501+0000 7fb2383c2700 1 -- 172.21.15.62:0/938467534 shutdown_connections 2024-06-10T00:34:19.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.501+0000 7fb2383c2700 1 -- 172.21.15.62:0/938467534 wait complete. 2024-06-10T00:34:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.502+0000 7fb2383c2700 1 Processor -- start 2024-06-10T00:34:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.502+0000 7fb2383c2700 1 -- start start 2024-06-10T00:34:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.502+0000 7fb2383c2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb230101390 0x7fb230103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.502+0000 7fb2383c2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb230101d50 0x7fb2301035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.502+0000 7fb2383c2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb230107400 0x7fb230103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.502+0000 7fb2383c2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb230111670 con 0x7fb230101d50 2024-06-10T00:34:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.502+0000 7fb2383c2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb2301114f0 con 0x7fb230107400 2024-06-10T00:34:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.502+0000 7fb2383c2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2301117f0 con 0x7fb230101390 2024-06-10T00:34:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.502+0000 7fb23595d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb230101d50 0x7fb2301035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.502+0000 7fb23595d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb230101d50 0x7fb2301035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46746/0 (socket says 172.21.15.62:46746) 2024-06-10T00:34:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.503+0000 7fb23595d700 1 -- 172.21.15.62:0/3906645978 learned_addr learned my addr 172.21.15.62:0/3906645978 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.503+0000 7fb23695f700 1 --2- 172.21.15.62:0/3906645978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb230107400 0x7fb230103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.503+0000 7fb23615e700 1 --2- 172.21.15.62:0/3906645978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb230101390 0x7fb230103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.503+0000 7fb23595d700 1 -- 172.21.15.62:0/3906645978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb230101390 msgr2=0x7fb230103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.503+0000 7fb23595d700 1 --2- 172.21.15.62:0/3906645978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb230101390 0x7fb230103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.503+0000 7fb23595d700 1 -- 172.21.15.62:0/3906645978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb230107400 msgr2=0x7fb230103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.503+0000 7fb23595d700 1 --2- 172.21.15.62:0/3906645978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb230107400 0x7fb230103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.503+0000 7fb23595d700 1 -- 172.21.15.62:0/3906645978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2301a7120 con 0x7fb230101d50 2024-06-10T00:34:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.503+0000 7fb23595d700 1 --2- 172.21.15.62:0/3906645978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb230101d50 0x7fb2301035b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb220000c00 tx=0x7fb2200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.503+0000 7fb2277fe700 1 -- 172.21.15.62:0/3906645978 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb220027070 con 0x7fb230101d50 2024-06-10T00:34:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.503+0000 7fb2277fe700 1 -- 172.21.15.62:0/3906645978 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb220003e40 con 0x7fb230101d50 2024-06-10T00:34:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.503+0000 7fb2383c2700 1 -- 172.21.15.62:0/3906645978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2301a73b0 con 0x7fb230101d50 2024-06-10T00:34:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.503+0000 7fb2277fe700 1 -- 172.21.15.62:0/3906645978 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb220022d10 con 0x7fb230101d50 2024-06-10T00:34:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.503+0000 7fb2383c2700 1 -- 172.21.15.62:0/3906645978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2301a7860 con 0x7fb230101d50 2024-06-10T00:34:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.505+0000 7fb2383c2700 1 -- 172.21.15.62:0/3906645978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb230069180 con 0x7fb230101d50 2024-06-10T00:34:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.506+0000 7fb2277fe700 1 -- 172.21.15.62:0/3906645978 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7fb22002a070 con 0x7fb230101d50 2024-06-10T00:34:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.507+0000 7fb2277fe700 1 --2- 172.21.15.62:0/3906645978 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb21c0a2f30 0x7fb21c0a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.507+0000 7fb23615e700 1 --2- 172.21.15.62:0/3906645978 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb21c0a2f30 0x7fb21c0a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.507+0000 7fb2277fe700 1 -- 172.21.15.62:0/3906645978 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fb2200d0f30 con 0x7fb230101d50 2024-06-10T00:34:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.508+0000 7fb23615e700 1 --2- 172.21.15.62:0/3906645978 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb21c0a2f30 0x7fb21c0a53f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb22c005950 tx=0x7fb22c00b410 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:19.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.510+0000 7fb2277fe700 1 -- 172.21.15.62:0/3906645978 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb220031070 con 0x7fb230101d50 2024-06-10T00:34:19.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:19 smithi138 ceph-mon[36549]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T00:34:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.762+0000 7fb2383c2700 1 -- 172.21.15.62:0/3906645978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2301a7b80 con 0x7fb230101d50 2024-06-10T00:34:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.762+0000 7fb2277fe700 1 -- 172.21.15.62:0/3906645978 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7fb220013070 con 0x7fb230101d50 2024-06-10T00:34:19.764 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:34:19.764 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:32:47.783154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:34:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.765+0000 7fb2383c2700 1 -- 172.21.15.62:0/3906645978 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb21c0a2f30 msgr2=0x7fb21c0a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.765+0000 7fb2383c2700 1 --2- 172.21.15.62:0/3906645978 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb21c0a2f30 0x7fb21c0a53f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb22c005950 tx=0x7fb22c00b410 comp rx=0 tx=0).stop 2024-06-10T00:34:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.765+0000 7fb2383c2700 1 -- 172.21.15.62:0/3906645978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb230101d50 msgr2=0x7fb2301035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.765+0000 7fb2383c2700 1 --2- 172.21.15.62:0/3906645978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb230101d50 0x7fb2301035b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb220000c00 tx=0x7fb2200047b0 comp rx=0 tx=0).stop 2024-06-10T00:34:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.765+0000 7fb2383c2700 1 -- 172.21.15.62:0/3906645978 shutdown_connections 2024-06-10T00:34:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.765+0000 7fb2383c2700 1 --2- 172.21.15.62:0/3906645978 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb21c0a2f30 0x7fb21c0a53f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.765+0000 7fb2383c2700 1 --2- 172.21.15.62:0/3906645978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb230107400 0x7fb230103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:19.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.765+0000 7fb2383c2700 1 --2- 172.21.15.62:0/3906645978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb230101390 0x7fb230103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:19.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.765+0000 7fb2383c2700 1 --2- 172.21.15.62:0/3906645978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb230101d50 0x7fb2301035b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:19.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.765+0000 7fb2383c2700 1 -- 172.21.15.62:0/3906645978 >> 172.21.15.62:0/3906645978 conn(0x7fb230078340 msgr2=0x7fb2301005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:19.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.765+0000 7fb2383c2700 1 -- 172.21.15.62:0/3906645978 shutdown_connections 2024-06-10T00:34:19.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:19.765+0000 7fb2383c2700 1 -- 172.21.15.62:0/3906645978 wait complete. 2024-06-10T00:34:19.767 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-06-10T00:34:19.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:19 smithi105 ceph-mon[36321]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T00:34:20.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:20 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3906645978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:20.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:20 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3906645978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:20.764 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T00:34:20.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:20 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3906645978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:21.017 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:21.444 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:21 smithi062 ceph-mon[32390]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T00:34:21.582 INFO:tasks.workunit.client.0.smithi062.stderr:Updating files: 91% (11537/12576) Updating files: 92% (11570/12576) Updating files: 93% (11696/12576) Updating files: 94% (11822/12576) Updating files: 95% (11948/12576) Updating files: 96% (12073/12576) Updating files: 97% (12199/12576) Updating files: 98% (12325/12576) Updating files: 99% (12451/12576) Updating files: 100% (12576/12576) Updating files: 100% (12576/12576), done. 2024-06-10T00:34:21.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:21 smithi138 ceph-mon[36549]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T00:34:21.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:21 smithi105 ceph-mon[36321]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T00:34:22.276 INFO:tasks.workunit.client.0.smithi062.stderr:Note: switching to '7b7a5d375cf4e4afab96b9d4d229b699ae8304bb'. 2024-06-10T00:34:22.276 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-06-10T00:34:22.276 INFO:tasks.workunit.client.0.smithi062.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-10T00:34:22.276 INFO:tasks.workunit.client.0.smithi062.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-10T00:34:22.276 INFO:tasks.workunit.client.0.smithi062.stderr:state without impacting any branches by switching back to a branch. 2024-06-10T00:34:22.277 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-06-10T00:34:22.277 INFO:tasks.workunit.client.0.smithi062.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-10T00:34:22.277 INFO:tasks.workunit.client.0.smithi062.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-10T00:34:22.277 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-06-10T00:34:22.277 INFO:tasks.workunit.client.0.smithi062.stderr: git switch -c 2024-06-10T00:34:22.277 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-06-10T00:34:22.277 INFO:tasks.workunit.client.0.smithi062.stderr:Or undo this operation with: 2024-06-10T00:34:22.277 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-06-10T00:34:22.277 INFO:tasks.workunit.client.0.smithi062.stderr: git switch - 2024-06-10T00:34:22.277 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-06-10T00:34:22.277 INFO:tasks.workunit.client.0.smithi062.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-10T00:34:22.277 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-06-10T00:34:22.277 INFO:tasks.workunit.client.0.smithi062.stderr:HEAD is now at 7b7a5d375cf Merge pull request #57941 from zdover23/wip-doc-2024-06-09-backport-57939-to-squid 2024-06-10T00:34:22.281 DEBUG:teuthology.orchestra.run.smithi062:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-10T00:34:22.332 INFO:tasks.workunit.client.0.smithi062.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-10T00:34:22.334 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-10T00:34:22.334 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-10T00:34:23.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.975+0000 7f517e216700 1 -- 172.21.15.62:0/3188495374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51781010b0 msgr2=0x7f5178101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:23.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.975+0000 7f517e216700 1 --2- 172.21.15.62:0/3188495374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51781010b0 0x7f5178101490 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5168009990 tx=0x7f5168009ca0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.975+0000 7f517e216700 1 -- 172.21.15.62:0/3188495374 shutdown_connections 2024-06-10T00:34:23.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.975+0000 7f517e216700 1 --2- 172.21.15.62:0/3188495374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51781010b0 0x7f5178101490 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.975+0000 7f517e216700 1 --2- 172.21.15.62:0/3188495374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5178105f50 0x7f517810da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.975+0000 7f517e216700 1 --2- 172.21.15.62:0/3188495374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5178101a60 0x7f5178105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.975+0000 7f517e216700 1 -- 172.21.15.62:0/3188495374 >> 172.21.15.62:0/3188495374 conn(0x7f51780766b0 msgr2=0x7f5178076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.975+0000 7f517e216700 1 -- 172.21.15.62:0/3188495374 shutdown_connections 2024-06-10T00:34:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.975+0000 7f517e216700 1 -- 172.21.15.62:0/3188495374 wait complete. 2024-06-10T00:34:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.976+0000 7f517e216700 1 Processor -- start 2024-06-10T00:34:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.976+0000 7f517e216700 1 -- start start 2024-06-10T00:34:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.976+0000 7f517e216700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51781010b0 0x7f517819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.976+0000 7f517e216700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5178101a60 0x7f517819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.976+0000 7f517e216700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5178105f50 0x7f5178198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.976+0000 7f517e216700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5178110f60 con 0x7f51781010b0 2024-06-10T00:34:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.976+0000 7f517e216700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5178110de0 con 0x7f5178101a60 2024-06-10T00:34:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.976+0000 7f517e216700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51781110e0 con 0x7f5178105f50 2024-06-10T00:34:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.976+0000 7f51777fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51781010b0 0x7f517819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.976+0000 7f51777fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51781010b0 0x7f517819e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46756/0 (socket says 172.21.15.62:46756) 2024-06-10T00:34:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.976+0000 7f51777fe700 1 -- 172.21.15.62:0/1884271583 learned_addr learned my addr 172.21.15.62:0/1884271583 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.977+0000 7f5176ffd700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5178101a60 0x7f517819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.977+0000 7f5177fff700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5178105f50 0x7f5178198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.977+0000 7f51777fe700 1 -- 172.21.15.62:0/1884271583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5178105f50 msgr2=0x7f5178198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.977+0000 7f51777fe700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5178105f50 0x7f5178198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.977+0000 7f51777fe700 1 -- 172.21.15.62:0/1884271583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5178101a60 msgr2=0x7f517819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.977+0000 7f51777fe700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5178101a60 0x7f517819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.977+0000 7f51777fe700 1 -- 172.21.15.62:0/1884271583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51781991d0 con 0x7f51781010b0 2024-06-10T00:34:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.977+0000 7f5176ffd700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5178101a60 0x7f517819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.977+0000 7f51777fe700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51781010b0 0x7f517819e8b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f51680098e0 tx=0x7f5168004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.977+0000 7f5174ff9700 1 -- 172.21.15.62:0/1884271583 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168027070 con 0x7f51781010b0 2024-06-10T00:34:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.977+0000 7f517e216700 1 -- 172.21.15.62:0/1884271583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5178199460 con 0x7f51781010b0 2024-06-10T00:34:23.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.977+0000 7f5174ff9700 1 -- 172.21.15.62:0/1884271583 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51680041a0 con 0x7f51781010b0 2024-06-10T00:34:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.977+0000 7f5174ff9700 1 -- 172.21.15.62:0/1884271583 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516800f460 con 0x7f51781010b0 2024-06-10T00:34:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.977+0000 7f517e216700 1 -- 172.21.15.62:0/1884271583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51781a5780 con 0x7f51781010b0 2024-06-10T00:34:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.979+0000 7f5174ff9700 1 -- 172.21.15.62:0/1884271583 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f516802a070 con 0x7f51781010b0 2024-06-10T00:34:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.980+0000 7f5174ff9700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f51600a2d90 0x7f51600a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.981+0000 7f5176ffd700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f51600a2d90 0x7f51600a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.981+0000 7f5174ff9700 1 -- 172.21.15.62:0/1884271583 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f51680d08e0 con 0x7f51781010b0 2024-06-10T00:34:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.981+0000 7f5176ffd700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f51600a2d90 0x7f51600a5250 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f51780fde70 tx=0x7f516c008040 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.981+0000 7f5188a99b80 1 Processor -- start 2024-06-10T00:34:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.981+0000 7f5188a99b80 1 -- start start 2024-06-10T00:34:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.981+0000 7f5188a99b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 0x561af6f76c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.981+0000 7f5188a99b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f7b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5188a99b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f7b550 0x561af6f7b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5188a99b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561af6e64f60 con 0x561af6f77170 2024-06-10T00:34:23.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5188a99b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561af6ec3850 con 0x561af6f7b550 2024-06-10T00:34:23.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5188a99b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561af6e2b460 con 0x561af6f76850 2024-06-10T00:34:23.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5176ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f7b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5176ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f7b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46762/0 (socket says 172.21.15.62:46762) 2024-06-10T00:34:23.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5176ffd700 1 -- 172.21.15.62:0/756809673 learned_addr learned my addr 172.21.15.62:0/756809673 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:23.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f51777fe700 1 --2- 172.21.15.62:0/756809673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 0x561af6f76c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5177fff700 1 --2- 172.21.15.62:0/756809673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f7b550 0x561af6f7b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5176ffd700 1 -- 172.21.15.62:0/756809673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 msgr2=0x561af6f76c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:23.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5176ffd700 1 --2- 172.21.15.62:0/756809673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 0x561af6f76c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5176ffd700 1 -- 172.21.15.62:0/756809673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f7b550 msgr2=0x561af6f7b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:23.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5176ffd700 1 --2- 172.21.15.62:0/756809673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f7b550 0x561af6f7b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5176ffd700 1 -- 172.21.15.62:0/756809673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af6e128d0 con 0x561af6f77170 2024-06-10T00:34:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5177fff700 1 --2- 172.21.15.62:0/756809673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f7b550 0x561af6f7b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f5176ffd700 1 --2- 172.21.15.62:0/756809673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f7b010 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f516c00d720 tx=0x7f516c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977c0b3243987b07 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f515dffb700 1 -- 172.21.15.62:0/756809673 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c024070 con 0x561af6f77170 2024-06-10T00:34:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f515dffb700 1 -- 172.21.15.62:0/756809673 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f516c014b20 con 0x561af6f77170 2024-06-10T00:34:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.982+0000 7f515dffb700 1 -- 172.21.15.62:0/756809673 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c01fad0 con 0x561af6f77170 2024-06-10T00:34:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.983+0000 7f5188a99b80 1 -- 172.21.15.62:0/756809673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 msgr2=0x561af6f7b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.983+0000 7f5188a99b80 1 --2- 172.21.15.62:0/756809673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f7b010 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f516c00d720 tx=0x7f516c010000 comp rx=0 tx=0).stop 2024-06-10T00:34:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.983+0000 7f5188a99b80 1 -- 172.21.15.62:0/756809673 shutdown_connections 2024-06-10T00:34:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.983+0000 7f5188a99b80 1 --2- 172.21.15.62:0/756809673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f7b550 0x561af6f7b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.983+0000 7f5188a99b80 1 --2- 172.21.15.62:0/756809673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 0x561af6f76c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.983+0000 7f5188a99b80 1 --2- 172.21.15.62:0/756809673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f7b010 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.983+0000 7f5188a99b80 1 -- 172.21.15.62:0/756809673 >> 172.21.15.62:0/756809673 conn(0x561af6e9f080 msgr2=0x561af6f6d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:23.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.983+0000 7f5188a99b80 1 -- 172.21.15.62:0/756809673 shutdown_connections 2024-06-10T00:34:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.983+0000 7f5188a99b80 1 -- 172.21.15.62:0/756809673 wait complete. 2024-06-10T00:34:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.984+0000 7f5188a99b80 1 -- start start 2024-06-10T00:34:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.984+0000 7f5188a99b80 1 Processor -- start 2024-06-10T00:34:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.984+0000 7f5188a99b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 0x561af6f82af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.984+0000 7f5188a99b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f7fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.984+0000 7f5188a99b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f7b550 0x561af6f80330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.984+0000 7f5188a99b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561af6e63fc0 con 0x561af6f77170 2024-06-10T00:34:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.984+0000 7f5188a99b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561af6ebf810 con 0x561af6f7b550 2024-06-10T00:34:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.984+0000 7f5188a99b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561af6f83680 con 0x561af6f76850 2024-06-10T00:34:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.984+0000 7f5188a99b80 10 client.0 _ll_register_callbacks cb 0x561af6e595b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:23.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.984+0000 7f5176ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f7fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.984+0000 7f5176ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f7fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46778/0 (socket says 172.21.15.62:46778) 2024-06-10T00:34:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.984+0000 7f5176ffd700 1 -- 172.21.15.62:0/2639275078 learned_addr learned my addr 172.21.15.62:0/2639275078 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.984+0000 7f51777fe700 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 0x561af6f82af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.984+0000 7f5177fff700 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f7b550 0x561af6f80330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5176ffd700 1 -- 172.21.15.62:0/2639275078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 msgr2=0x561af6f82af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5176ffd700 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 0x561af6f82af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5176ffd700 1 -- 172.21.15.62:0/2639275078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f7b550 msgr2=0x561af6f80330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5176ffd700 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f7b550 0x561af6f80330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5176ffd700 1 -- 172.21.15.62:0/2639275078 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af6e89750 con 0x561af6f77170 2024-06-10T00:34:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f51777fe700 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 0x561af6f82af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5177fff700 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f7b550 0x561af6f80330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5176ffd700 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f7fdf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f516c022040 tx=0x7f516c014920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e3fd5a288fa684 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5156ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5156ffd700 1 -- 172.21.15.62:0/2639275078 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c024040 con 0x561af6f77170 2024-06-10T00:34:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5156ffd700 1 -- 172.21.15.62:0/2639275078 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f516c01f750 con 0x561af6f77170 2024-06-10T00:34:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5188a99b80 1 -- 172.21.15.62:0/2639275078 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f516c014b20 con 0x561af6f77170 2024-06-10T00:34:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5156ffd700 1 -- 172.21.15.62:0/2639275078 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f516c037050 con 0x561af6f77170 2024-06-10T00:34:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5188a99b80 10 client.15843 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5188a99b80 1 -- 172.21.15.62:0/2639275078 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561af6f8de30 con 0x561af6f77170 2024-06-10T00:34:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.985+0000 7f5156ffd700 1 -- 172.21.15.62:0/2639275078 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f516c014b20 con 0x561af6f77170 2024-06-10T00:34:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.986+0000 7f5156ffd700 1 -- 172.21.15.62:0/2639275078 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f516c032030 con 0x561af6f77170 2024-06-10T00:34:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.986+0000 7f5188a99b80 10 client.15843 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.986+0000 7f5188a99b80 10 client.15843 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:34:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.986+0000 7f5188a99b80 20 client.15843 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.986+0000 7f5188a99b80 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561af6f903b0 0x561af6f927a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.986+0000 7f5188a99b80 4 client.15843 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:34:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.986+0000 7f5188a99b80 1 -- 172.21.15.62:0/2639275078 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561af6ed0d50 con 0x561af6f903b0 2024-06-10T00:34:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.986+0000 7f51777fe700 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561af6f903b0 0x561af6f927a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.986+0000 7f5156ffd700 10 client.15843 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:34:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.986+0000 7f51777fe700 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561af6f903b0 0x561af6f927a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6e47528b5fdcf830 server_cookie=90313a20defc0ffd in_seq=0 out_seq=0 2024-06-10T00:34:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.987+0000 7f5156ffd700 1 -- 172.21.15.62:0/2639275078 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561af6ed0d50 con 0x561af6f903b0 2024-06-10T00:34:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.987+0000 7f5156ffd700 10 client.15843 handle_command_reply: tid=0 2024-06-10T00:34:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.987+0000 7f5188a99b80 1 client.15843 shutdown 2024-06-10T00:34:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.987+0000 7f5188a99b80 1 -- 172.21.15.62:0/2639275078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 msgr2=0x561af6f7fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.987+0000 7f5188a99b80 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f7fdf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f516c022040 tx=0x7f516c014920 comp rx=0 tx=0).stop 2024-06-10T00:34:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.987+0000 7f5188a99b80 1 -- 172.21.15.62:0/2639275078 shutdown_connections 2024-06-10T00:34:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.987+0000 7f5188a99b80 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561af6f903b0 0x561af6f927a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.987+0000 7f5188a99b80 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f7b550 0x561af6f80330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.987+0000 7f5188a99b80 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 0x561af6f82af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.987+0000 7f5188a99b80 1 --2- 172.21.15.62:0/2639275078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f7fdf0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.987+0000 7f5156ffd700 0 client.15843 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:34:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.987+0000 7f5188a99b80 1 -- 172.21.15.62:0/2639275078 >> 172.21.15.62:0/2639275078 conn(0x561af6e9f080 msgr2=0x561af6f7ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.987+0000 7f5188a99b80 1 -- 172.21.15.62:0/2639275078 shutdown_connections 2024-06-10T00:34:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.987+0000 7f5188a99b80 1 -- 172.21.15.62:0/2639275078 wait complete. 2024-06-10T00:34:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:22.988+0000 7f5188a99b80 20 client.15843 trim_cache size 0 max 16384 2024-06-10T00:34:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5188a99b80 1 Processor -- start 2024-06-10T00:34:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5188a99b80 1 -- start start 2024-06-10T00:34:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5188a99b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f76850 0x561af6f81d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5188a99b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f97f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5188a99b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f7b550 0x561af6f98470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5188a99b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51781ac350 con 0x561af6f77170 2024-06-10T00:34:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5188a99b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f51781ade90 con 0x561af6f76850 2024-06-10T00:34:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5188a99b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51781aca20 con 0x561af6f7b550 2024-06-10T00:34:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5176ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f97f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5176ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f97f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46788/0 (socket says 172.21.15.62:46788) 2024-06-10T00:34:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5176ffd700 1 -- 172.21.15.62:0/300388993 learned_addr learned my addr 172.21.15.62:0/300388993 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f51777fe700 1 --2- 172.21.15.62:0/300388993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f76850 0x561af6f81d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5177fff700 1 --2- 172.21.15.62:0/300388993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f7b550 0x561af6f98470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5176ffd700 1 -- 172.21.15.62:0/300388993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f7b550 msgr2=0x561af6f98470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5176ffd700 1 --2- 172.21.15.62:0/300388993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f7b550 0x561af6f98470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5176ffd700 1 -- 172.21.15.62:0/300388993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f76850 msgr2=0x561af6f81d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5176ffd700 1 --2- 172.21.15.62:0/300388993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f76850 0x561af6f81d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5176ffd700 1 -- 172.21.15.62:0/300388993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af6f80d30 con 0x561af6f77170 2024-06-10T00:34:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.003+0000 7f5177fff700 1 --2- 172.21.15.62:0/300388993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f7b550 0x561af6f98470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.004+0000 7f5176ffd700 1 --2- 172.21.15.62:0/300388993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f97f30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f516c0048a0 tx=0x7f516c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964b81e995506fb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.004+0000 7f515d7fa700 1 -- 172.21.15.62:0/300388993 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c03f020 con 0x561af6f77170 2024-06-10T00:34:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.004+0000 7f515d7fa700 1 -- 172.21.15.62:0/300388993 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f516c0095c0 con 0x561af6f77170 2024-06-10T00:34:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.004+0000 7f515d7fa700 1 -- 172.21.15.62:0/300388993 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c024040 con 0x561af6f77170 2024-06-10T00:34:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.004+0000 7f5188a99b80 1 -- 172.21.15.62:0/300388993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 msgr2=0x561af6f97f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.004+0000 7f5188a99b80 1 --2- 172.21.15.62:0/300388993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f97f30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f516c0048a0 tx=0x7f516c00dd10 comp rx=0 tx=0).stop 2024-06-10T00:34:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.004+0000 7f5188a99b80 1 -- 172.21.15.62:0/300388993 shutdown_connections 2024-06-10T00:34:23.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.004+0000 7f5188a99b80 1 --2- 172.21.15.62:0/300388993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f7b550 0x561af6f98470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.041 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:34:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.004+0000 7f5188a99b80 1 --2- 172.21.15.62:0/300388993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f76850 0x561af6f81d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.004+0000 7f5188a99b80 1 --2- 172.21.15.62:0/300388993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f77170 0x561af6f97f30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.004+0000 7f5188a99b80 1 -- 172.21.15.62:0/300388993 >> 172.21.15.62:0/300388993 conn(0x561af6e9f080 msgr2=0x561af6f81620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.005+0000 7f5188a99b80 1 -- 172.21.15.62:0/300388993 shutdown_connections 2024-06-10T00:34:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.005+0000 7f5188a99b80 1 -- 172.21.15.62:0/300388993 wait complete. 2024-06-10T00:34:23.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.005+0000 7f5188a99b80 1 -- start start 2024-06-10T00:34:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.005+0000 7f5188a99b80 1 Processor -- start 2024-06-10T00:34:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.005+0000 7f5188a99b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 0x561af6f86870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.005+0000 7f5188a99b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f77170 0x561af6f86db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.005+0000 7f5188a99b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f7b550 0x561af6f872f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.005+0000 7f5188a99b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51781ad7c0 con 0x561af6f7b550 2024-06-10T00:34:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.005+0000 7f5188a99b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f51781ad0f0 con 0x561af6f77170 2024-06-10T00:34:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.005+0000 7f5188a99b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51781aa2f0 con 0x561af6f76850 2024-06-10T00:34:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.005+0000 7f5188a99b80 10 client.0 _ll_register_callbacks cb 0x561af6e595b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.005+0000 7f5177fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f7b550 0x561af6f872f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5177fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f7b550 0x561af6f872f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46792/0 (socket says 172.21.15.62:46792) 2024-06-10T00:34:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5177fff700 1 -- 172.21.15.62:0/2708260511 learned_addr learned my addr 172.21.15.62:0/2708260511 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5176ffd700 1 --2- 172.21.15.62:0/2708260511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f77170 0x561af6f86db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f51777fe700 1 --2- 172.21.15.62:0/2708260511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 0x561af6f86870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5177fff700 1 -- 172.21.15.62:0/2708260511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 msgr2=0x561af6f86870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5177fff700 1 --2- 172.21.15.62:0/2708260511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 0x561af6f86870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5177fff700 1 -- 172.21.15.62:0/2708260511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f77170 msgr2=0x561af6f86db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5177fff700 1 --2- 172.21.15.62:0/2708260511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f77170 0x561af6f86db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5177fff700 1 -- 172.21.15.62:0/2708260511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af6ffa0d0 con 0x561af6f7b550 2024-06-10T00:34:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5177fff700 1 --2- 172.21.15.62:0/2708260511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f7b550 0x561af6f872f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f517000a970 tx=0x7f517000a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1281e9b5d45806c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5156ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5156ffd700 1 -- 172.21.15.62:0/2708260511 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5170015760 con 0x561af6f7b550 2024-06-10T00:34:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5156ffd700 1 -- 172.21.15.62:0/2708260511 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f5170015900 con 0x561af6f7b550 2024-06-10T00:34:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5188a99b80 1 -- 172.21.15.62:0/2708260511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f516c0095c0 con 0x561af6f7b550 2024-06-10T00:34:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5156ffd700 1 -- 172.21.15.62:0/2708260511 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5170024050 con 0x561af6f7b550 2024-06-10T00:34:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5188a99b80 10 client.15849 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:23.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.006+0000 7f5188a99b80 1 -- 172.21.15.62:0/2708260511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561af6ffb380 con 0x561af6f7b550 2024-06-10T00:34:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.007+0000 7f5156ffd700 1 -- 172.21.15.62:0/2708260511 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f516c0095c0 con 0x561af6f7b550 2024-06-10T00:34:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.007+0000 7f5156ffd700 1 -- 172.21.15.62:0/2708260511 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f517001f3b0 con 0x561af6f7b550 2024-06-10T00:34:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.007+0000 7f5188a99b80 10 client.15849 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.007+0000 7f5188a99b80 10 client.15849 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:34:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.007+0000 7f5188a99b80 20 client.15849 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.007+0000 7f5188a99b80 1 --2- 172.21.15.62:0/2708260511 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561af6ffbb10 0x561af6ffdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.007+0000 7f5188a99b80 4 client.15849 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:34:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.007+0000 7f5188a99b80 1 -- 172.21.15.62:0/2708260511 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561af6ecb2d0 con 0x561af6ffbb10 2024-06-10T00:34:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.007+0000 7f51777fe700 1 --2- 172.21.15.62:0/2708260511 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561af6ffbb10 0x561af6ffdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f51777fe700 1 --2- 172.21.15.62:0/2708260511 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561af6ffbb10 0x561af6ffdf30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=448e268f0ed39f41 server_cookie=b9d6239ecc2374c6 in_seq=0 out_seq=0 2024-06-10T00:34:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5156ffd700 10 client.15849 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:34:23.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5156ffd700 1 -- 172.21.15.62:0/2708260511 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561af6ecb2d0 con 0x561af6ffbb10 2024-06-10T00:34:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5156ffd700 10 client.15849 handle_command_reply: tid=0 2024-06-10T00:34:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5188a99b80 1 client.15849 shutdown 2024-06-10T00:34:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5188a99b80 1 -- 172.21.15.62:0/2708260511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f7b550 msgr2=0x561af6f872f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5188a99b80 1 --2- 172.21.15.62:0/2708260511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f7b550 0x561af6f872f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f517000a970 tx=0x7f517000a9a0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5188a99b80 1 -- 172.21.15.62:0/2708260511 shutdown_connections 2024-06-10T00:34:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5188a99b80 1 --2- 172.21.15.62:0/2708260511 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x561af6ffbb10 0x561af6ffdf30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5188a99b80 1 --2- 172.21.15.62:0/2708260511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561af6f7b550 0x561af6f872f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5188a99b80 1 --2- 172.21.15.62:0/2708260511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af6f76850 0x561af6f86870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5188a99b80 1 --2- 172.21.15.62:0/2708260511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561af6f77170 0x561af6f86db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5188a99b80 1 -- 172.21.15.62:0/2708260511 >> 172.21.15.62:0/2708260511 conn(0x561af6e9f080 msgr2=0x561af6f81eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5156ffd700 0 client.15849 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:34:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5188a99b80 1 -- 172.21.15.62:0/2708260511 shutdown_connections 2024-06-10T00:34:23.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5188a99b80 1 -- 172.21.15.62:0/2708260511 wait complete. 2024-06-10T00:34:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.008+0000 7f5188a99b80 20 client.15849 trim_cache size 0 max 16384 2024-06-10T00:34:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.009+0000 7f517e216700 1 -- 172.21.15.62:0/1884271583 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f51600a2d90 msgr2=0x7f51600a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.009+0000 7f517e216700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f51600a2d90 0x7f51600a5250 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f51780fde70 tx=0x7f516c008040 comp rx=0 tx=0).stop 2024-06-10T00:34:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.009+0000 7f517e216700 1 -- 172.21.15.62:0/1884271583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51781010b0 msgr2=0x7f517819e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.009+0000 7f517e216700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51781010b0 0x7f517819e8b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f51680098e0 tx=0x7f5168004630 comp rx=0 tx=0).stop 2024-06-10T00:34:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.009+0000 7f517e216700 1 -- 172.21.15.62:0/1884271583 shutdown_connections 2024-06-10T00:34:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.009+0000 7f517e216700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f51600a2d90 0x7f51600a5250 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.009+0000 7f517e216700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51781010b0 0x7f517819e8b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.009+0000 7f517e216700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5178105f50 0x7f5178198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.009+0000 7f517e216700 1 --2- 172.21.15.62:0/1884271583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5178101a60 0x7f517819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.009+0000 7f517e216700 1 -- 172.21.15.62:0/1884271583 >> 172.21.15.62:0/1884271583 conn(0x7f51780766b0 msgr2=0x7f5178104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:23.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.009+0000 7f517e216700 1 -- 172.21.15.62:0/1884271583 shutdown_connections 2024-06-10T00:34:23.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:23.009+0000 7f517e216700 1 -- 172.21.15.62:0/1884271583 wait complete. 2024-06-10T00:34:23.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:23 smithi062 ceph-mon[32390]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-06-10T00:34:23.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:23 smithi062 ceph-mon[32390]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-06-10T00:34:23.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:23 smithi138 ceph-mon[36549]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-06-10T00:34:23.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:23 smithi138 ceph-mon[36549]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-06-10T00:34:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:23 smithi105 ceph-mon[36321]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-06-10T00:34:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:23 smithi105 ceph-mon[36321]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-06-10T00:34:23.961 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T00:34:24.134 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-10T00:34:24.194 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-10T00:34:24.219 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:24.239 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-10T00:34:24.241 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-10T00:34:24.241 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-10T00:34:24.281 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-10T00:34:24.285 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:34:24.285 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-10T00:34:24.301 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-06-10T00:34:24.303 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-06-10T00:34:24.303 DEBUG:teuthology.orchestra.run.smithi062:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-06-10T00:34:25.594 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:25 smithi062 ceph-mon[32390]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-10T00:34:25.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:25 smithi138 ceph-mon[36549]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-10T00:34:25.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:25 smithi105 ceph-mon[36321]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-10T00:34:25.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.966+0000 7fad61eea700 1 -- 172.21.15.62:0/777933430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c101d50 msgr2=0x7fad5c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:25.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.966+0000 7fad61eea700 1 --2- 172.21.15.62:0/777933430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c101d50 0x7fad5c10e240 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fad5400b0a0 tx=0x7fad5400b3b0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.966+0000 7fad61eea700 1 -- 172.21.15.62:0/777933430 shutdown_connections 2024-06-10T00:34:25.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.966+0000 7fad61eea700 1 --2- 172.21.15.62:0/777933430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c101d50 0x7fad5c10e240 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.966+0000 7fad61eea700 1 --2- 172.21.15.62:0/777933430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad5c107400 0x7fad5c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.966+0000 7fad61eea700 1 --2- 172.21.15.62:0/777933430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad5c101390 0x7fad5c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.966+0000 7fad61eea700 1 -- 172.21.15.62:0/777933430 >> 172.21.15.62:0/777933430 conn(0x7fad5c078340 msgr2=0x7fad5c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:25.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.966+0000 7fad61eea700 1 -- 172.21.15.62:0/777933430 shutdown_connections 2024-06-10T00:34:25.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.966+0000 7fad61eea700 1 -- 172.21.15.62:0/777933430 wait complete. 2024-06-10T00:34:25.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.967+0000 7fad61eea700 1 Processor -- start 2024-06-10T00:34:25.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.967+0000 7fad61eea700 1 -- start start 2024-06-10T00:34:25.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.967+0000 7fad61eea700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad5c101390 0x7fad5c1031c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:25.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.967+0000 7fad61eea700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad5c101d50 0x7fad5c103700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:25.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.967+0000 7fad61eea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c107400 0x7fad5c103c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:25.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.967+0000 7fad61eea700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c1116f0 con 0x7fad5c107400 2024-06-10T00:34:25.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.967+0000 7fad61eea700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c111570 con 0x7fad5c101d50 2024-06-10T00:34:25.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.967+0000 7fad61eea700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c111870 con 0x7fad5c101390 2024-06-10T00:34:25.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.967+0000 7fad5bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c107400 0x7fad5c103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:25.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.967+0000 7fad5bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c107400 0x7fad5c103c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46824/0 (socket says 172.21.15.62:46824) 2024-06-10T00:34:25.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.967+0000 7fad5bfff700 1 -- 172.21.15.62:0/3195037766 learned_addr learned my addr 172.21.15.62:0/3195037766 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:25.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.967+0000 7fad5b7fe700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad5c101390 0x7fad5c1031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:25.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.968+0000 7fad5affd700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad5c101d50 0x7fad5c103700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:25.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.968+0000 7fad5bfff700 1 -- 172.21.15.62:0/3195037766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad5c101390 msgr2=0x7fad5c1031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:25.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.968+0000 7fad5bfff700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad5c101390 0x7fad5c1031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.968+0000 7fad5bfff700 1 -- 172.21.15.62:0/3195037766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad5c101d50 msgr2=0x7fad5c103700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:25.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.968+0000 7fad5bfff700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad5c101d50 0x7fad5c103700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.968+0000 7fad5bfff700 1 -- 172.21.15.62:0/3195037766 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c1a70a0 con 0x7fad5c107400 2024-06-10T00:34:25.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.968+0000 7fad5affd700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad5c101d50 0x7fad5c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:25.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.968+0000 7fad5bfff700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c107400 0x7fad5c103c40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fad5400d010 tx=0x7fad54004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:25.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.968+0000 7fad58ff9700 1 -- 172.21.15.62:0/3195037766 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad54021070 con 0x7fad5c107400 2024-06-10T00:34:25.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.968+0000 7fad58ff9700 1 -- 172.21.15.62:0/3195037766 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad54007c00 con 0x7fad5c107400 2024-06-10T00:34:25.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.968+0000 7fad61eea700 1 -- 172.21.15.62:0/3195037766 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5c1a7330 con 0x7fad5c107400 2024-06-10T00:34:25.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.968+0000 7fad61eea700 1 -- 172.21.15.62:0/3195037766 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5c1a77e0 con 0x7fad5c107400 2024-06-10T00:34:25.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.968+0000 7fad58ff9700 1 -- 172.21.15.62:0/3195037766 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5401d450 con 0x7fad5c107400 2024-06-10T00:34:25.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.971+0000 7fad58ff9700 1 -- 172.21.15.62:0/3195037766 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fad54004020 con 0x7fad5c107400 2024-06-10T00:34:25.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.972+0000 7fad58ff9700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fad440a2d90 0x7fad440a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:25.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.972+0000 7fad5b7fe700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fad440a2d90 0x7fad440a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:25.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.972+0000 7fad58ff9700 1 -- 172.21.15.62:0/3195037766 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fad5400f020 con 0x7fad5c107400 2024-06-10T00:34:25.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.972+0000 7fad5b7fe700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fad440a2d90 0x7fad440a5250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00abd0 tx=0x7fad4c00a380 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:25.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.972+0000 7fad6c76db80 1 Processor -- start 2024-06-10T00:34:25.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.972+0000 7fad6c76db80 1 -- start start 2024-06-10T00:34:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad6c76db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 0x5651dcac6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad6c76db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcac0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad6c76db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac07a0 0x5651dcac0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad6c76db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5651dc9b2f60 con 0x5651dcac6880 2024-06-10T00:34:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad6c76db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5651dca11850 con 0x5651dcac07a0 2024-06-10T00:34:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad6c76db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5651dc979460 con 0x5651dcac5f60 2024-06-10T00:34:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad5affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcac0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad5affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcac0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46826/0 (socket says 172.21.15.62:46826) 2024-06-10T00:34:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad5affd700 1 -- 172.21.15.62:0/1777792054 learned_addr learned my addr 172.21.15.62:0/1777792054 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad5b7fe700 1 --2- 172.21.15.62:0/1777792054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 0x5651dcac6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad5bfff700 1 --2- 172.21.15.62:0/1777792054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac07a0 0x5651dcac0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad5affd700 1 -- 172.21.15.62:0/1777792054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 msgr2=0x5651dcac6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad5affd700 1 --2- 172.21.15.62:0/1777792054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 0x5651dcac6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad5affd700 1 -- 172.21.15.62:0/1777792054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac07a0 msgr2=0x5651dcac0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:25.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad5affd700 1 --2- 172.21.15.62:0/1777792054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac07a0 0x5651dcac0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad5affd700 1 -- 172.21.15.62:0/1777792054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651dc9608d0 con 0x5651dcac6880 2024-06-10T00:34:25.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad5affd700 1 --2- 172.21.15.62:0/1777792054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcac0260 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fad50009c70 tx=0x7fad50009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4622602fdd296ac4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:25.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.973+0000 7fad41ffb700 1 -- 172.21.15.62:0/1777792054 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50016070 con 0x5651dcac6880 2024-06-10T00:34:25.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.974+0000 7fad41ffb700 1 -- 172.21.15.62:0/1777792054 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad50011470 con 0x5651dcac6880 2024-06-10T00:34:25.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.974+0000 7fad41ffb700 1 -- 172.21.15.62:0/1777792054 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50010610 con 0x5651dcac6880 2024-06-10T00:34:25.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.974+0000 7fad6c76db80 1 -- 172.21.15.62:0/1777792054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 msgr2=0x5651dcac0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:25.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.974+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1777792054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcac0260 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fad50009c70 tx=0x7fad50009f80 comp rx=0 tx=0).stop 2024-06-10T00:34:25.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.974+0000 7fad6c76db80 1 -- 172.21.15.62:0/1777792054 shutdown_connections 2024-06-10T00:34:25.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.974+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1777792054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac07a0 0x5651dcac0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.974+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1777792054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 0x5651dcac6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.974+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1777792054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcac0260 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.974+0000 7fad6c76db80 1 -- 172.21.15.62:0/1777792054 >> 172.21.15.62:0/1777792054 conn(0x5651dc9ed080 msgr2=0x5651dcabb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:25.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.974+0000 7fad6c76db80 1 -- 172.21.15.62:0/1777792054 shutdown_connections 2024-06-10T00:34:25.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.974+0000 7fad6c76db80 1 -- 172.21.15.62:0/1777792054 wait complete. 2024-06-10T00:34:25.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.975+0000 7fad6c76db80 1 -- start start 2024-06-10T00:34:25.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.975+0000 7fad6c76db80 1 Processor -- start 2024-06-10T00:34:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.975+0000 7fad6c76db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac07a0 0x5651dcacc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.975+0000 7fad6c76db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 0x5651dcac9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.975+0000 7fad6c76db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcac9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.975+0000 7fad6c76db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5651dc9b1fc0 con 0x5651dcac6880 2024-06-10T00:34:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.975+0000 7fad6c76db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5651dca0d810 con 0x5651dcac07a0 2024-06-10T00:34:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.975+0000 7fad6c76db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5651dcacd260 con 0x5651dcac5f60 2024-06-10T00:34:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.975+0000 7fad6c76db80 10 client.0 _ll_register_callbacks cb 0x5651dc9a75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.975+0000 7fad5bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcac9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.975+0000 7fad5bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcac9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46836/0 (socket says 172.21.15.62:46836) 2024-06-10T00:34:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.975+0000 7fad5bfff700 1 -- 172.21.15.62:0/1007605032 learned_addr learned my addr 172.21.15.62:0/1007605032 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad5b7fe700 1 --2- 172.21.15.62:0/1007605032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac07a0 0x5651dcacc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad5affd700 1 --2- 172.21.15.62:0/1007605032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 0x5651dcac9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad5bfff700 1 -- 172.21.15.62:0/1007605032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 msgr2=0x5651dcac9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad5bfff700 1 --2- 172.21.15.62:0/1007605032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 0x5651dcac9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad5bfff700 1 -- 172.21.15.62:0/1007605032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac07a0 msgr2=0x5651dcacc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad5bfff700 1 --2- 172.21.15.62:0/1007605032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac07a0 0x5651dcacc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad5bfff700 1 -- 172.21.15.62:0/1007605032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651dc9d7750 con 0x5651dcac6880 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad5b7fe700 1 --2- 172.21.15.62:0/1007605032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac07a0 0x5651dcacc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad5bfff700 1 --2- 172.21.15.62:0/1007605032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcac9d90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fad54009fd0 tx=0x7fad5409cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cba2e3bf39d81f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad3affd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad3affd700 1 -- 172.21.15.62:0/1007605032 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5409d3c0 con 0x5651dcac6880 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad3affd700 1 -- 172.21.15.62:0/1007605032 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fad540a56b0 con 0x5651dcac6880 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad6c76db80 1 -- 172.21.15.62:0/1007605032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad50011470 con 0x5651dcac6880 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad3affd700 1 -- 172.21.15.62:0/1007605032 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fad5409cf60 con 0x5651dcac6880 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad6c76db80 10 client.15861 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad6c76db80 1 -- 172.21.15.62:0/1007605032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5651dcad7980 con 0x5651dcac6880 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.976+0000 7fad3affd700 1 -- 172.21.15.62:0/1007605032 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad50011470 con 0x5651dcac6880 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.977+0000 7fad3affd700 1 -- 172.21.15.62:0/1007605032 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fad540a0070 con 0x5651dcac6880 2024-06-10T00:34:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.977+0000 7fad6c76db80 10 client.15861 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.977+0000 7fad6c76db80 10 client.15861 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:34:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.977+0000 7fad6c76db80 20 client.15861 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.977+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1007605032 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5651dcad9f90 0x5651dcadc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.977+0000 7fad6c76db80 4 client.15861 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:34:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.977+0000 7fad6c76db80 1 -- 172.21.15.62:0/1007605032 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651dca1ed50 con 0x5651dcad9f90 2024-06-10T00:34:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.977+0000 7fad5b7fe700 1 --2- 172.21.15.62:0/1007605032 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5651dcad9f90 0x5651dcadc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.977+0000 7fad5b7fe700 1 --2- 172.21.15.62:0/1007605032 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5651dcad9f90 0x5651dcadc3a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a1f540c4d10bf99e server_cookie=c124bf18a26c1d30 in_seq=0 out_seq=0 2024-06-10T00:34:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.977+0000 7fad3affd700 10 client.15861 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:34:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.978+0000 7fad3affd700 1 -- 172.21.15.62:0/1007605032 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5651dca1ed50 con 0x5651dcad9f90 2024-06-10T00:34:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.978+0000 7fad3affd700 10 client.15861 handle_command_reply: tid=0 2024-06-10T00:34:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.978+0000 7fad6c76db80 1 client.15861 shutdown 2024-06-10T00:34:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.978+0000 7fad6c76db80 1 -- 172.21.15.62:0/1007605032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 msgr2=0x5651dcac9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:25.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.978+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1007605032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcac9d90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fad54009fd0 tx=0x7fad5409cbd0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.978+0000 7fad6c76db80 1 -- 172.21.15.62:0/1007605032 shutdown_connections 2024-06-10T00:34:25.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.978+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1007605032 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5651dcad9f90 0x5651dcadc3a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.978+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1007605032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcac9d90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.978+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1007605032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac07a0 0x5651dcacc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.978+0000 7fad3affd700 0 client.15861 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:34:25.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.978+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1007605032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 0x5651dcac9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.978+0000 7fad6c76db80 1 -- 172.21.15.62:0/1007605032 >> 172.21.15.62:0/1007605032 conn(0x5651dc9ed080 msgr2=0x5651dcaba700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:25.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.979+0000 7fad6c76db80 1 -- 172.21.15.62:0/1007605032 shutdown_connections 2024-06-10T00:34:25.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.979+0000 7fad6c76db80 1 -- 172.21.15.62:0/1007605032 wait complete. 2024-06-10T00:34:25.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.979+0000 7fad6c76db80 20 client.15861 trim_cache size 0 max 16384 2024-06-10T00:34:25.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.994+0000 7fad6c76db80 1 Processor -- start 2024-06-10T00:34:25.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.994+0000 7fad6c76db80 1 -- start start 2024-06-10T00:34:25.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.994+0000 7fad6c76db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac07a0 0x5651dcac0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:25.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.994+0000 7fad6c76db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 0x5651dcabae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:25.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.994+0000 7fad6c76db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac6880 0x5651dcabb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:25.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.994+0000 7fad6c76db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c1b0590 con 0x5651dcac07a0 2024-06-10T00:34:25.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.994+0000 7fad6c76db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c1b20d0 con 0x5651dcac6880 2024-06-10T00:34:25.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.994+0000 7fad6c76db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c1b0c60 con 0x5651dcac5f60 2024-06-10T00:34:25.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.994+0000 7fad5b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac07a0 0x5651dcac0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:25.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.994+0000 7fad5b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac07a0 0x5651dcac0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46848/0 (socket says 172.21.15.62:46848) 2024-06-10T00:34:25.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.994+0000 7fad5b7fe700 1 -- 172.21.15.62:0/153910494 learned_addr learned my addr 172.21.15.62:0/153910494 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:25.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.994+0000 7fad5bfff700 1 --2- 172.21.15.62:0/153910494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac6880 0x5651dcabb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:25.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.994+0000 7fad5affd700 1 --2- 172.21.15.62:0/153910494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 0x5651dcabae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:25.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.995+0000 7fad5b7fe700 1 -- 172.21.15.62:0/153910494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 msgr2=0x5651dcabae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:25.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.995+0000 7fad5b7fe700 1 --2- 172.21.15.62:0/153910494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 0x5651dcabae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.995+0000 7fad5b7fe700 1 -- 172.21.15.62:0/153910494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac6880 msgr2=0x5651dcabb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:25.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.995+0000 7fad5b7fe700 1 --2- 172.21.15.62:0/153910494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac6880 0x5651dcabb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:25.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.995+0000 7fad5b7fe700 1 -- 172.21.15.62:0/153910494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651dcabcd20 con 0x5651dcac07a0 2024-06-10T00:34:25.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.995+0000 7fad5affd700 1 --2- 172.21.15.62:0/153910494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 0x5651dcabae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:25.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.995+0000 7fad5bfff700 1 --2- 172.21.15.62:0/153910494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac6880 0x5651dcabb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:25.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.995+0000 7fad5b7fe700 1 --2- 172.21.15.62:0/153910494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac07a0 0x5651dcac0b80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00a9b0 tx=0x7fad4c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76eeac1e83131e8e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:25.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.995+0000 7fad417fa700 1 -- 172.21.15.62:0/153910494 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c016070 con 0x5651dcac07a0 2024-06-10T00:34:25.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.995+0000 7fad417fa700 1 -- 172.21.15.62:0/153910494 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad4c02c070 con 0x5651dcac07a0 2024-06-10T00:34:25.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.995+0000 7fad417fa700 1 -- 172.21.15.62:0/153910494 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c010690 con 0x5651dcac07a0 2024-06-10T00:34:26.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.996+0000 7fad6c76db80 1 -- 172.21.15.62:0/153910494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac07a0 msgr2=0x5651dcac0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:26.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.996+0000 7fad6c76db80 1 --2- 172.21.15.62:0/153910494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac07a0 0x5651dcac0b80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00a9b0 tx=0x7fad4c00af90 comp rx=0 tx=0).stop 2024-06-10T00:34:26.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.996+0000 7fad6c76db80 1 -- 172.21.15.62:0/153910494 shutdown_connections 2024-06-10T00:34:26.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.996+0000 7fad6c76db80 1 --2- 172.21.15.62:0/153910494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac07a0 0x5651dcac0b80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:26.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.996+0000 7fad6c76db80 1 --2- 172.21.15.62:0/153910494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac6880 0x5651dcabb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:26.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.996+0000 7fad6c76db80 1 --2- 172.21.15.62:0/153910494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac5f60 0x5651dcabae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:26.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.996+0000 7fad6c76db80 1 -- 172.21.15.62:0/153910494 >> 172.21.15.62:0/153910494 conn(0x5651dc9ed080 msgr2=0x5651dcac8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:26.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.996+0000 7fad6c76db80 1 -- 172.21.15.62:0/153910494 shutdown_connections 2024-06-10T00:34:26.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.996+0000 7fad6c76db80 1 -- 172.21.15.62:0/153910494 wait complete. 2024-06-10T00:34:26.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.996+0000 7fad6c76db80 1 -- start start 2024-06-10T00:34:26.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.996+0000 7fad6c76db80 1 Processor -- start 2024-06-10T00:34:26.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad6c76db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac07a0 0x5651dcad2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:26.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad6c76db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac5f60 0x5651dcad2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:26.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad6c76db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcae1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:26.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad6c76db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c1b1a00 con 0x5651dcac6880 2024-06-10T00:34:26.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad6c76db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c1b1330 con 0x5651dcac5f60 2024-06-10T00:34:26.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad6c76db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c1ae530 con 0x5651dcac07a0 2024-06-10T00:34:26.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad6c76db80 10 client.0 _ll_register_callbacks cb 0x5651dc9a75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:26.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad5bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcae1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:26.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad5b7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac07a0 0x5651dcad2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:26.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad5bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcae1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46864/0 (socket says 172.21.15.62:46864) 2024-06-10T00:34:26.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad5bfff700 1 -- 172.21.15.62:0/1398367938 learned_addr learned my addr 172.21.15.62:0/1398367938 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:26.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad5affd700 1 --2- 172.21.15.62:0/1398367938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac5f60 0x5651dcad2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:26.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad5bfff700 1 -- 172.21.15.62:0/1398367938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac07a0 msgr2=0x5651dcad2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:26.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad5bfff700 1 --2- 172.21.15.62:0/1398367938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac07a0 0x5651dcad2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:26.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad5bfff700 1 -- 172.21.15.62:0/1398367938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac5f60 msgr2=0x5651dcad2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:26.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad5bfff700 1 --2- 172.21.15.62:0/1398367938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac5f60 0x5651dcad2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:26.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad5bfff700 1 -- 172.21.15.62:0/1398367938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651dcae1b30 con 0x5651dcac6880 2024-06-10T00:34:26.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad5bfff700 1 --2- 172.21.15.62:0/1398367938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcae1350 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fad5402de90 tx=0x7fad5409d4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2dd9719188bc7f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:26.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad3affd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:26.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad3affd700 1 -- 172.21.15.62:0/1398367938 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad540a1310 con 0x5651dcac6880 2024-06-10T00:34:26.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad3affd700 1 -- 172.21.15.62:0/1398367938 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fad540bbb10 con 0x5651dcac6880 2024-06-10T00:34:26.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.997+0000 7fad6c76db80 1 -- 172.21.15.62:0/1398367938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad4c02c070 con 0x5651dcac6880 2024-06-10T00:34:26.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.998+0000 7fad3affd700 1 -- 172.21.15.62:0/1398367938 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fad5402dec0 con 0x5651dcac6880 2024-06-10T00:34:26.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.998+0000 7fad6c76db80 10 client.15867 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:26.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.998+0000 7fad6c76db80 1 -- 172.21.15.62:0/1398367938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5651dcb44770 con 0x5651dcac6880 2024-06-10T00:34:26.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.998+0000 7fad3affd700 1 -- 172.21.15.62:0/1398367938 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad4c02c070 con 0x5651dcac6880 2024-06-10T00:34:26.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.998+0000 7fad3affd700 1 -- 172.21.15.62:0/1398367938 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fad54080070 con 0x5651dcac6880 2024-06-10T00:34:26.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.998+0000 7fad6c76db80 10 client.15867 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:26.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.998+0000 7fad6c76db80 10 client.15867 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:34:26.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.998+0000 7fad6c76db80 20 client.15867 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:26.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.998+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1398367938 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5651dcb44c60 0x5651dcb47080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:26.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.998+0000 7fad6c76db80 4 client.15867 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:34:26.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.998+0000 7fad6c76db80 1 -- 172.21.15.62:0/1398367938 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5651dca192d0 con 0x5651dcb44c60 2024-06-10T00:34:26.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.998+0000 7fad5b7fe700 1 --2- 172.21.15.62:0/1398367938 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5651dcb44c60 0x5651dcb47080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:26.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad3affd700 10 client.15867 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:34:26.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad5b7fe700 1 --2- 172.21.15.62:0/1398367938 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5651dcb44c60 0x5651dcb47080 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=851e1da963e461f0 server_cookie=a89508ad7e8804c4 in_seq=0 out_seq=0 2024-06-10T00:34:26.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad3affd700 1 -- 172.21.15.62:0/1398367938 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651dca192d0 con 0x5651dcb44c60 2024-06-10T00:34:26.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad3affd700 10 client.15867 handle_command_reply: tid=0 2024-06-10T00:34:26.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad6c76db80 1 client.15867 shutdown 2024-06-10T00:34:26.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad6c76db80 1 -- 172.21.15.62:0/1398367938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 msgr2=0x5651dcae1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:26.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1398367938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcae1350 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fad5402de90 tx=0x7fad5409d4f0 comp rx=0 tx=0).stop 2024-06-10T00:34:26.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad6c76db80 1 -- 172.21.15.62:0/1398367938 shutdown_connections 2024-06-10T00:34:26.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1398367938 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x5651dcb44c60 0x5651dcb47080 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:26.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad3affd700 0 client.15867 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:34:26.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1398367938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5651dcac6880 0x5651dcae1350 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:26.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1398367938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651dcac07a0 0x5651dcad2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:26.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad6c76db80 1 --2- 172.21.15.62:0/1398367938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651dcac5f60 0x5651dcad2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:26.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad6c76db80 1 -- 172.21.15.62:0/1398367938 >> 172.21.15.62:0/1398367938 conn(0x5651dc9ed080 msgr2=0x5651dcac6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:26.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad6c76db80 1 -- 172.21.15.62:0/1398367938 shutdown_connections 2024-06-10T00:34:26.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:25.999+0000 7fad6c76db80 1 -- 172.21.15.62:0/1398367938 wait complete. 2024-06-10T00:34:26.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:26.000+0000 7fad6c76db80 20 client.15867 trim_cache size 0 max 16384 2024-06-10T00:34:26.005 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:34:26.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:26.000+0000 7fad61eea700 1 -- 172.21.15.62:0/3195037766 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fad440a2d90 msgr2=0x7fad440a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:26.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:26.000+0000 7fad61eea700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fad440a2d90 0x7fad440a5250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00abd0 tx=0x7fad4c00a380 comp rx=0 tx=0).stop 2024-06-10T00:34:26.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:26.000+0000 7fad61eea700 1 -- 172.21.15.62:0/3195037766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c107400 msgr2=0x7fad5c103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:26.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:26.000+0000 7fad61eea700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c107400 0x7fad5c103c40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fad5400d010 tx=0x7fad54004d50 comp rx=0 tx=0).stop 2024-06-10T00:34:26.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:26.000+0000 7fad61eea700 1 -- 172.21.15.62:0/3195037766 shutdown_connections 2024-06-10T00:34:26.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:26.000+0000 7fad61eea700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fad440a2d90 0x7fad440a5250 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:26.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:26.000+0000 7fad61eea700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad5c107400 0x7fad5c103c40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:26.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:26.000+0000 7fad61eea700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad5c101390 0x7fad5c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:26.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:26.000+0000 7fad61eea700 1 --2- 172.21.15.62:0/3195037766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad5c101d50 0x7fad5c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:26.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:26.000+0000 7fad61eea700 1 -- 172.21.15.62:0/3195037766 >> 172.21.15.62:0/3195037766 conn(0x7fad5c078340 msgr2=0x7fad5c105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:26.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:26.000+0000 7fad61eea700 1 -- 172.21.15.62:0/3195037766 shutdown_connections 2024-06-10T00:34:26.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:26.000+0000 7fad61eea700 1 -- 172.21.15.62:0/3195037766 wait complete. 2024-06-10T00:34:26.716 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T00:34:26.966 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:27.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:27 smithi062 ceph-mon[32390]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-06-10T00:34:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:27 smithi138 ceph-mon[36549]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-06-10T00:34:27.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:27 smithi105 ceph-mon[36321]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-06-10T00:34:28.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.654+0000 7f116d8e9700 1 -- 172.21.15.62:0/2888570220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11681066d0 msgr2=0x7f116810e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:28.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.654+0000 7f116d8e9700 1 --2- 172.21.15.62:0/2888570220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11681066d0 0x7f116810e2a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f116000b0d0 tx=0x7f116000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.654+0000 7f116d8e9700 1 -- 172.21.15.62:0/2888570220 shutdown_connections 2024-06-10T00:34:28.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.654+0000 7f116d8e9700 1 --2- 172.21.15.62:0/2888570220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11681066d0 0x7f116810e2a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.654+0000 7f116d8e9700 1 --2- 172.21.15.62:0/2888570220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1168101680 0x7f1168101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.654+0000 7f116d8e9700 1 --2- 172.21.15.62:0/2888570220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1168102030 0x7f1168105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.654+0000 7f116d8e9700 1 -- 172.21.15.62:0/2888570220 >> 172.21.15.62:0/2888570220 conn(0x7f11680773e0 msgr2=0x7f11680777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:28.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.654+0000 7f116d8e9700 1 -- 172.21.15.62:0/2888570220 shutdown_connections 2024-06-10T00:34:28.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.654+0000 7f116d8e9700 1 -- 172.21.15.62:0/2888570220 wait complete. 2024-06-10T00:34:28.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.655+0000 7f116d8e9700 1 Processor -- start 2024-06-10T00:34:28.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.655+0000 7f116d8e9700 1 -- start start 2024-06-10T00:34:28.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.655+0000 7f116d8e9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1168101680 0x7f1168199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.655+0000 7f116d8e9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1168102030 0x7f1168199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.655+0000 7f116d8e9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11681066d0 0x7f11681a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.655+0000 7f116d8e9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f11681116c0 con 0x7f11681066d0 2024-06-10T00:34:28.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.655+0000 7f116d8e9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1168111540 con 0x7f1168101680 2024-06-10T00:34:28.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.655+0000 7f116d8e9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1168111840 con 0x7f1168102030 2024-06-10T00:34:28.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.656+0000 7f11677fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11681066d0 0x7f11681a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.656+0000 7f11667fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1168102030 0x7f1168199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.656+0000 7f11677fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11681066d0 0x7f11681a4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48566/0 (socket says 172.21.15.62:48566) 2024-06-10T00:34:28.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.656+0000 7f11677fe700 1 -- 172.21.15.62:0/1548841312 learned_addr learned my addr 172.21.15.62:0/1548841312 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:28.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.656+0000 7f1166ffd700 1 --2- 172.21.15.62:0/1548841312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1168101680 0x7f1168199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.656+0000 7f11677fe700 1 -- 172.21.15.62:0/1548841312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1168102030 msgr2=0x7f1168199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:28.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.656+0000 7f11677fe700 1 --2- 172.21.15.62:0/1548841312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1168102030 0x7f1168199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.656+0000 7f11677fe700 1 -- 172.21.15.62:0/1548841312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1168101680 msgr2=0x7f1168199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:28.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.656+0000 7f11677fe700 1 --2- 172.21.15.62:0/1548841312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1168101680 0x7f1168199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.656+0000 7f11677fe700 1 -- 172.21.15.62:0/1548841312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11681a5060 con 0x7f11681066d0 2024-06-10T00:34:28.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.656+0000 7f11677fe700 1 --2- 172.21.15.62:0/1548841312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11681066d0 0x7f11681a4b20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1160006070 tx=0x7f1160004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:28.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.656+0000 7f116c8e7700 1 -- 172.21.15.62:0/1548841312 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160021070 con 0x7f11681066d0 2024-06-10T00:34:28.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.657+0000 7f116d8e9700 1 -- 172.21.15.62:0/1548841312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11681a5230 con 0x7f11681066d0 2024-06-10T00:34:28.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.657+0000 7f116c8e7700 1 -- 172.21.15.62:0/1548841312 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1160007b80 con 0x7f11681066d0 2024-06-10T00:34:28.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.657+0000 7f116c8e7700 1 -- 172.21.15.62:0/1548841312 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11600052a0 con 0x7f11681066d0 2024-06-10T00:34:28.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.657+0000 7f116d8e9700 1 -- 172.21.15.62:0/1548841312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11681a56e0 con 0x7f11681066d0 2024-06-10T00:34:28.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.659+0000 7f116c8e7700 1 -- 172.21.15.62:0/1548841312 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1160024070 con 0x7f11681066d0 2024-06-10T00:34:28.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.660+0000 7f116c8e7700 1 --2- 172.21.15.62:0/1548841312 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f11500a2e10 0x7f11500a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.660+0000 7f1166ffd700 1 --2- 172.21.15.62:0/1548841312 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f11500a2e10 0x7f11500a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.660+0000 7f116c8e7700 1 -- 172.21.15.62:0/1548841312 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f116000f020 con 0x7f11681066d0 2024-06-10T00:34:28.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.660+0000 7f1166ffd700 1 --2- 172.21.15.62:0/1548841312 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f11500a2e10 0x7f11500a52d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1158005950 tx=0x7f115800b410 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:28.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f117816cb80 1 Processor -- start 2024-06-10T00:34:28.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f117816cb80 1 -- start start 2024-06-10T00:34:28.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f117816cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 0x55f2b5761850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f117816cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b575baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f117816cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 0x55f2b5768210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f117816cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f2b5649f60 con 0x55f2b575b670 2024-06-10T00:34:28.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f117816cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f2b56a8850 con 0x55f2b5761470 2024-06-10T00:34:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f117816cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f2b5610460 con 0x55f2b575c030 2024-06-10T00:34:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f11667fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b575baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f11667fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b575baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48578/0 (socket says 172.21.15.62:48578) 2024-06-10T00:34:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f11667fc700 1 -- 172.21.15.62:0/2782716179 learned_addr learned my addr 172.21.15.62:0/2782716179 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f11677fe700 1 --2- 172.21.15.62:0/2782716179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 0x55f2b5768210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f1166ffd700 1 --2- 172.21.15.62:0/2782716179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 0x55f2b5761850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f11667fc700 1 -- 172.21.15.62:0/2782716179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 msgr2=0x55f2b5768210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f11667fc700 1 --2- 172.21.15.62:0/2782716179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 0x55f2b5768210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f11667fc700 1 -- 172.21.15.62:0/2782716179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 msgr2=0x55f2b5761850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f11667fc700 1 --2- 172.21.15.62:0/2782716179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 0x55f2b5761850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f11667fc700 1 -- 172.21.15.62:0/2782716179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b55f78d0 con 0x55f2b575b670 2024-06-10T00:34:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f11677fe700 1 --2- 172.21.15.62:0/2782716179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 0x55f2b5768210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.661+0000 7f1166ffd700 1 --2- 172.21.15.62:0/2782716179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 0x55f2b5761850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:28.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.662+0000 7f11667fc700 1 --2- 172.21.15.62:0/2782716179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b575baf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f115c00bc30 tx=0x7f115c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7ca37bf9b17cc1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:28.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.662+0000 7f114dffb700 1 -- 172.21.15.62:0/2782716179 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c010d50 con 0x55f2b575b670 2024-06-10T00:34:28.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.662+0000 7f114dffb700 1 -- 172.21.15.62:0/2782716179 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f115c004510 con 0x55f2b575b670 2024-06-10T00:34:28.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.662+0000 7f114dffb700 1 -- 172.21.15.62:0/2782716179 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c0052d0 con 0x55f2b575b670 2024-06-10T00:34:28.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.662+0000 7f117816cb80 1 -- 172.21.15.62:0/2782716179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 msgr2=0x55f2b575baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:28.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.662+0000 7f117816cb80 1 --2- 172.21.15.62:0/2782716179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b575baf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f115c00bc30 tx=0x7f115c00bf40 comp rx=0 tx=0).stop 2024-06-10T00:34:28.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.662+0000 7f117816cb80 1 -- 172.21.15.62:0/2782716179 shutdown_connections 2024-06-10T00:34:28.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.662+0000 7f117816cb80 1 --2- 172.21.15.62:0/2782716179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 0x55f2b5768210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.662+0000 7f117816cb80 1 --2- 172.21.15.62:0/2782716179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 0x55f2b5761850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.662+0000 7f117816cb80 1 --2- 172.21.15.62:0/2782716179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b575baf0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.663+0000 7f117816cb80 1 -- 172.21.15.62:0/2782716179 >> 172.21.15.62:0/2782716179 conn(0x55f2b5684080 msgr2=0x55f2b5752220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:28.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.663+0000 7f117816cb80 1 -- 172.21.15.62:0/2782716179 shutdown_connections 2024-06-10T00:34:28.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.663+0000 7f117816cb80 1 -- 172.21.15.62:0/2782716179 wait complete. 2024-06-10T00:34:28.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.663+0000 7f117816cb80 1 -- start start 2024-06-10T00:34:28.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.663+0000 7f117816cb80 1 Processor -- start 2024-06-10T00:34:28.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.663+0000 7f117816cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b5767df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.663+0000 7f117816cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b575c030 0x55f2b57650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.663+0000 7f117816cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b5761470 0x55f2b5765630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.663+0000 7f117816cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f2b5648fc0 con 0x55f2b575b670 2024-06-10T00:34:28.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.663+0000 7f117816cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f2b56a4810 con 0x55f2b575c030 2024-06-10T00:34:28.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.663+0000 7f117816cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f2b5768750 con 0x55f2b5761470 2024-06-10T00:34:28.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.663+0000 7f117816cb80 10 client.0 _ll_register_callbacks cb 0x55f2b563e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:28.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f1166ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b5767df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f1166ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b5767df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48592/0 (socket says 172.21.15.62:48592) 2024-06-10T00:34:28.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f1166ffd700 1 -- 172.21.15.62:0/1549172168 learned_addr learned my addr 172.21.15.62:0/1549172168 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:28.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f11677fe700 1 --2- 172.21.15.62:0/1549172168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b5761470 0x55f2b5765630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f11667fc700 1 --2- 172.21.15.62:0/1549172168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b575c030 0x55f2b57650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f1166ffd700 1 -- 172.21.15.62:0/1549172168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b5761470 msgr2=0x55f2b5765630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:28.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f1166ffd700 1 --2- 172.21.15.62:0/1549172168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b5761470 0x55f2b5765630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f1166ffd700 1 -- 172.21.15.62:0/1549172168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b575c030 msgr2=0x55f2b57650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:28.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f1166ffd700 1 --2- 172.21.15.62:0/1549172168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b575c030 0x55f2b57650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f1166ffd700 1 -- 172.21.15.62:0/1549172168 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b566e750 con 0x55f2b575b670 2024-06-10T00:34:28.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f11667fc700 1 --2- 172.21.15.62:0/1549172168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b575c030 0x55f2b57650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:28.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f1166ffd700 1 --2- 172.21.15.62:0/1549172168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b5767df0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f11580116c0 tx=0x7f11580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b3e0a839f6ef4a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:28.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f1146ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:28.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f1146ffd700 1 -- 172.21.15.62:0/1549172168 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115801e830 con 0x55f2b575b670 2024-06-10T00:34:28.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f1146ffd700 1 -- 172.21.15.62:0/1549172168 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f115801e9d0 con 0x55f2b575b670 2024-06-10T00:34:28.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f117816cb80 1 -- 172.21.15.62:0/1549172168 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f115c004510 con 0x55f2b575b670 2024-06-10T00:34:28.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f1146ffd700 1 -- 172.21.15.62:0/1549172168 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f115801ebd0 con 0x55f2b575b670 2024-06-10T00:34:28.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f117816cb80 10 client.15879 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:28.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.664+0000 7f117816cb80 1 -- 172.21.15.62:0/1549172168 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f2b5772cf0 con 0x55f2b575b670 2024-06-10T00:34:28.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.665+0000 7f1146ffd700 1 -- 172.21.15.62:0/1549172168 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f115c004510 con 0x55f2b575b670 2024-06-10T00:34:28.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.665+0000 7f1146ffd700 1 -- 172.21.15.62:0/1549172168 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f115801b410 con 0x55f2b575b670 2024-06-10T00:34:28.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.665+0000 7f117816cb80 10 client.15879 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:28.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.665+0000 7f117816cb80 10 client.15879 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.665+0000 7f117816cb80 20 client.15879 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.665+0000 7f117816cb80 1 --2- 172.21.15.62:0/1549172168 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f2b5775380 0x55f2b57777d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.665+0000 7f117816cb80 4 client.15879 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.665+0000 7f117816cb80 1 -- 172.21.15.62:0/1549172168 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2b56b5d50 con 0x55f2b5775380 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.665+0000 7f11667fc700 1 --2- 172.21.15.62:0/1549172168 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f2b5775380 0x55f2b57777d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.666+0000 7f11667fc700 1 --2- 172.21.15.62:0/1549172168 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f2b5775380 0x55f2b57777d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=973bc1c7cb71c27c server_cookie=1b39c60cef0c4456 in_seq=0 out_seq=0 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.666+0000 7f1146ffd700 10 client.15879 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.666+0000 7f1146ffd700 1 -- 172.21.15.62:0/1549172168 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f2b56b5d50 con 0x55f2b5775380 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.667+0000 7f1146ffd700 10 client.15879 handle_command_reply: tid=0 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.667+0000 7f117816cb80 1 client.15879 shutdown 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.667+0000 7f117816cb80 1 -- 172.21.15.62:0/1549172168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 msgr2=0x55f2b5767df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.667+0000 7f117816cb80 1 --2- 172.21.15.62:0/1549172168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b5767df0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f11580116c0 tx=0x7f11580047c0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.667+0000 7f117816cb80 1 -- 172.21.15.62:0/1549172168 shutdown_connections 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.667+0000 7f117816cb80 1 --2- 172.21.15.62:0/1549172168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b5767df0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.667+0000 7f117816cb80 1 --2- 172.21.15.62:0/1549172168 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f2b5775380 0x55f2b57777d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.667+0000 7f1146ffd700 0 client.15879 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:34:28.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.667+0000 7f117816cb80 1 --2- 172.21.15.62:0/1549172168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b5761470 0x55f2b5765630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.667+0000 7f117816cb80 1 --2- 172.21.15.62:0/1549172168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b575c030 0x55f2b57650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.667+0000 7f117816cb80 1 -- 172.21.15.62:0/1549172168 >> 172.21.15.62:0/1549172168 conn(0x55f2b5684080 msgr2=0x55f2b5751700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:28.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.667+0000 7f117816cb80 1 -- 172.21.15.62:0/1549172168 shutdown_connections 2024-06-10T00:34:28.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.667+0000 7f117816cb80 1 -- 172.21.15.62:0/1549172168 wait complete. 2024-06-10T00:34:28.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.667+0000 7f117816cb80 20 client.15879 trim_cache size 0 max 16384 2024-06-10T00:34:28.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.682+0000 7f117816cb80 1 Processor -- start 2024-06-10T00:34:28.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.682+0000 7f117816cb80 1 -- start start 2024-06-10T00:34:28.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f117816cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b5767030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f117816cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 0x55f2b57717a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f117816cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 0x55f2b5751e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f117816cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f11681ac3b0 con 0x55f2b575b670 2024-06-10T00:34:28.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f117816cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f11681adef0 con 0x55f2b5761470 2024-06-10T00:34:28.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f117816cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11681aca80 con 0x55f2b575c030 2024-06-10T00:34:28.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f1166ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b5767030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f1166ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b5767030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48606/0 (socket says 172.21.15.62:48606) 2024-06-10T00:34:28.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f1166ffd700 1 -- 172.21.15.62:0/3872210038 learned_addr learned my addr 172.21.15.62:0/3872210038 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:28.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f11677fe700 1 --2- 172.21.15.62:0/3872210038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 0x55f2b5751e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f11667fc700 1 --2- 172.21.15.62:0/3872210038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 0x55f2b57717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f1166ffd700 1 -- 172.21.15.62:0/3872210038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 msgr2=0x55f2b57717a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:28.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f1166ffd700 1 --2- 172.21.15.62:0/3872210038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 0x55f2b57717a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f1166ffd700 1 -- 172.21.15.62:0/3872210038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 msgr2=0x55f2b5751e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:28.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f1166ffd700 1 --2- 172.21.15.62:0/3872210038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 0x55f2b5751e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f1166ffd700 1 -- 172.21.15.62:0/3872210038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b5752e40 con 0x55f2b575b670 2024-06-10T00:34:28.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f1166ffd700 1 --2- 172.21.15.62:0/3872210038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b5767030 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1158010ac0 tx=0x7f1158004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5481012df0e63e93 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:28.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f114d7fa700 1 -- 172.21.15.62:0/3872210038 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115801b610 con 0x55f2b575b670 2024-06-10T00:34:28.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f114d7fa700 1 -- 172.21.15.62:0/3872210038 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1158022070 con 0x55f2b575b670 2024-06-10T00:34:28.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.683+0000 7f114d7fa700 1 -- 172.21.15.62:0/3872210038 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11580277f0 con 0x55f2b575b670 2024-06-10T00:34:28.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.684+0000 7f117816cb80 1 -- 172.21.15.62:0/3872210038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 msgr2=0x55f2b5767030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:28.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.684+0000 7f117816cb80 1 --2- 172.21.15.62:0/3872210038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b5767030 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1158010ac0 tx=0x7f1158004690 comp rx=0 tx=0).stop 2024-06-10T00:34:28.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.684+0000 7f117816cb80 1 -- 172.21.15.62:0/3872210038 shutdown_connections 2024-06-10T00:34:28.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.684+0000 7f117816cb80 1 --2- 172.21.15.62:0/3872210038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b5767030 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.684+0000 7f117816cb80 1 --2- 172.21.15.62:0/3872210038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 0x55f2b5751e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.684+0000 7f117816cb80 1 --2- 172.21.15.62:0/3872210038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 0x55f2b57717a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.684+0000 7f117816cb80 1 -- 172.21.15.62:0/3872210038 >> 172.21.15.62:0/3872210038 conn(0x55f2b5684080 msgr2=0x55f2b5766920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:28.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.684+0000 7f117816cb80 1 -- 172.21.15.62:0/3872210038 shutdown_connections 2024-06-10T00:34:28.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.684+0000 7f117816cb80 1 -- 172.21.15.62:0/3872210038 wait complete. 2024-06-10T00:34:28.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.684+0000 7f117816cb80 1 -- start start 2024-06-10T00:34:28.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f117816cb80 1 Processor -- start 2024-06-10T00:34:28.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f117816cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b576de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f117816cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 0x55f2b577c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f117816cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 0x55f2b577cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f117816cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f11681ad820 con 0x55f2b575b670 2024-06-10T00:34:28.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f117816cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f11681ad150 con 0x55f2b5761470 2024-06-10T00:34:28.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f117816cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11681aa350 con 0x55f2b575c030 2024-06-10T00:34:28.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f117816cb80 10 client.0 _ll_register_callbacks cb 0x55f2b5764560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:28.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f1166ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b576de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f1166ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b576de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48616/0 (socket says 172.21.15.62:48616) 2024-06-10T00:34:28.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f1166ffd700 1 -- 172.21.15.62:0/4291653493 learned_addr learned my addr 172.21.15.62:0/4291653493 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:28.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f11677fe700 1 --2- 172.21.15.62:0/4291653493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 0x55f2b577cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f11667fc700 1 --2- 172.21.15.62:0/4291653493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 0x55f2b577c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f1166ffd700 1 -- 172.21.15.62:0/4291653493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 msgr2=0x55f2b577c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:28.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f1166ffd700 1 --2- 172.21.15.62:0/4291653493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 0x55f2b577c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f1166ffd700 1 -- 172.21.15.62:0/4291653493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 msgr2=0x55f2b577cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:28.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f1166ffd700 1 --2- 172.21.15.62:0/4291653493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 0x55f2b577cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.685+0000 7f1166ffd700 1 -- 172.21.15.62:0/4291653493 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b577d2f0 con 0x55f2b575b670 2024-06-10T00:34:28.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f1166ffd700 1 --2- 172.21.15.62:0/4291653493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b576de10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1158000c00 tx=0x7f115801e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396c8c3ca5247ed6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:28.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f1146ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f1146ffd700 1 -- 172.21.15.62:0/4291653493 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115801be30 con 0x55f2b575b670 2024-06-10T00:34:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f117816cb80 1 -- 172.21.15.62:0/4291653493 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1158022070 con 0x55f2b575b670 2024-06-10T00:34:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f1146ffd700 1 -- 172.21.15.62:0/4291653493 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f1158005040 con 0x55f2b575b670 2024-06-10T00:34:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f1146ffd700 1 -- 172.21.15.62:0/4291653493 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f115804e050 con 0x55f2b575b670 2024-06-10T00:34:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f117816cb80 10 client.15885 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f117816cb80 1 -- 172.21.15.62:0/4291653493 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f2b576c600 con 0x55f2b575b670 2024-06-10T00:34:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f1146ffd700 1 -- 172.21.15.62:0/4291653493 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1158022070 con 0x55f2b575b670 2024-06-10T00:34:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f1146ffd700 1 -- 172.21.15.62:0/4291653493 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f11580484b0 con 0x55f2b575b670 2024-06-10T00:34:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f117816cb80 10 client.15885 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f117816cb80 10 client.15885 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:34:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f117816cb80 20 client.15885 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f117816cb80 1 --2- 172.21.15.62:0/4291653493 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f2b5775380 0x55f2b57e23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f117816cb80 4 client.15885 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:34:28.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.686+0000 7f117816cb80 1 -- 172.21.15.62:0/4291653493 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2b56b02d0 con 0x55f2b5775380 2024-06-10T00:34:28.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.687+0000 7f11667fc700 1 --2- 172.21.15.62:0/4291653493 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f2b5775380 0x55f2b57e23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:28.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.687+0000 7f11667fc700 1 --2- 172.21.15.62:0/4291653493 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f2b5775380 0x55f2b57e23e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45ceb252f3015332 server_cookie=9d42d953aa8b3b23 in_seq=0 out_seq=0 2024-06-10T00:34:28.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.687+0000 7f1146ffd700 10 client.15885 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:34:28.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.687+0000 7f1146ffd700 1 -- 172.21.15.62:0/4291653493 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2b56b02d0 con 0x55f2b5775380 2024-06-10T00:34:28.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.687+0000 7f1146ffd700 10 client.15885 handle_command_reply: tid=0 2024-06-10T00:34:28.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.687+0000 7f117816cb80 1 client.15885 shutdown 2024-06-10T00:34:28.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.687+0000 7f117816cb80 1 -- 172.21.15.62:0/4291653493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 msgr2=0x55f2b576de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:28.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.687+0000 7f117816cb80 1 --2- 172.21.15.62:0/4291653493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b576de10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1158000c00 tx=0x7f115801e910 comp rx=0 tx=0).stop 2024-06-10T00:34:28.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.687+0000 7f117816cb80 1 -- 172.21.15.62:0/4291653493 shutdown_connections 2024-06-10T00:34:28.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.687+0000 7f117816cb80 1 --2- 172.21.15.62:0/4291653493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2b575b670 0x55f2b576de10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.687+0000 7f117816cb80 1 --2- 172.21.15.62:0/4291653493 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f2b5775380 0x55f2b57e23e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f117816cb80 1 --2- 172.21.15.62:0/4291653493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f2b5761470 0x55f2b577cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f117816cb80 1 --2- 172.21.15.62:0/4291653493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2b575c030 0x55f2b577c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f1146ffd700 0 client.15885 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:34:28.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f117816cb80 1 -- 172.21.15.62:0/4291653493 >> 172.21.15.62:0/4291653493 conn(0x55f2b5684080 msgr2=0x55f2b5756ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:28.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f117816cb80 1 -- 172.21.15.62:0/4291653493 shutdown_connections 2024-06-10T00:34:28.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f117816cb80 1 -- 172.21.15.62:0/4291653493 wait complete. 2024-06-10T00:34:28.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f117816cb80 20 client.15885 trim_cache size 0 max 16384 2024-06-10T00:34:28.693 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:34:28.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f116d8e9700 1 -- 172.21.15.62:0/1548841312 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f11500a2e10 msgr2=0x7f11500a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:28.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f116d8e9700 1 --2- 172.21.15.62:0/1548841312 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f11500a2e10 0x7f11500a52d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1158005950 tx=0x7f115800b410 comp rx=0 tx=0).stop 2024-06-10T00:34:28.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f116d8e9700 1 -- 172.21.15.62:0/1548841312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11681066d0 msgr2=0x7f11681a4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:28.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f116d8e9700 1 --2- 172.21.15.62:0/1548841312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11681066d0 0x7f11681a4b20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1160006070 tx=0x7f1160004d00 comp rx=0 tx=0).stop 2024-06-10T00:34:28.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f116d8e9700 1 -- 172.21.15.62:0/1548841312 shutdown_connections 2024-06-10T00:34:28.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f116d8e9700 1 --2- 172.21.15.62:0/1548841312 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f11500a2e10 0x7f11500a52d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f116d8e9700 1 --2- 172.21.15.62:0/1548841312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11681066d0 0x7f11681a4b20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f116d8e9700 1 --2- 172.21.15.62:0/1548841312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1168101680 0x7f1168199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f116d8e9700 1 --2- 172.21.15.62:0/1548841312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1168102030 0x7f1168199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:28.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.688+0000 7f116d8e9700 1 -- 172.21.15.62:0/1548841312 >> 172.21.15.62:0/1548841312 conn(0x7f11680773e0 msgr2=0x7f11680ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:28.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.689+0000 7f116d8e9700 1 -- 172.21.15.62:0/1548841312 shutdown_connections 2024-06-10T00:34:28.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:28.689+0000 7f116d8e9700 1 -- 172.21.15.62:0/1548841312 wait complete. 2024-06-10T00:34:29.423 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T00:34:29.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:29 smithi062 ceph-mon[32390]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T00:34:29.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:29 smithi138 ceph-mon[36549]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T00:34:29.675 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:29.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:29 smithi105 ceph-mon[36321]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T00:34:30.809 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:34:31.348 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:31 smithi062 ceph-mon[32390]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T00:34:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.455+0000 7feb26a05700 1 -- 172.21.15.62:0/334163713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb2006b810 msgr2=0x7feb201070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.455+0000 7feb26a05700 1 --2- 172.21.15.62:0/334163713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb2006b810 0x7feb201070d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7feb080099c0 tx=0x7feb08009cd0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.456+0000 7feb26a05700 1 -- 172.21.15.62:0/334163713 shutdown_connections 2024-06-10T00:34:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.456+0000 7feb26a05700 1 --2- 172.21.15.62:0/334163713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb201077b0 0x7feb2010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.456+0000 7feb26a05700 1 --2- 172.21.15.62:0/334163713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2006ae60 0x7feb2006b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.456+0000 7feb26a05700 1 --2- 172.21.15.62:0/334163713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb2006b810 0x7feb201070d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.456+0000 7feb26a05700 1 -- 172.21.15.62:0/334163713 >> 172.21.15.62:0/334163713 conn(0x7feb200773e0 msgr2=0x7feb200777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.456+0000 7feb26a05700 1 -- 172.21.15.62:0/334163713 shutdown_connections 2024-06-10T00:34:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.456+0000 7feb26a05700 1 -- 172.21.15.62:0/334163713 wait complete. 2024-06-10T00:34:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.456+0000 7feb26a05700 1 Processor -- start 2024-06-10T00:34:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.457+0000 7feb26a05700 1 -- start start 2024-06-10T00:34:31.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.457+0000 7feb26a05700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb2006ae60 0x7feb201014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.457+0000 7feb26a05700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2006b810 0x7feb20101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.457+0000 7feb26a05700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb201077b0 0x7feb20101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.457+0000 7feb26a05700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feb20112880 con 0x7feb2006ae60 2024-06-10T00:34:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.457+0000 7feb26a05700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb20112700 con 0x7feb201077b0 2024-06-10T00:34:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.457+0000 7feb26a05700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb20112a00 con 0x7feb2006b810 2024-06-10T00:34:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.457+0000 7feb1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb2006ae60 0x7feb201014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.457+0000 7feb1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb2006ae60 0x7feb201014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48634/0 (socket says 172.21.15.62:48634) 2024-06-10T00:34:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.457+0000 7feb1ffff700 1 -- 172.21.15.62:0/3778844631 learned_addr learned my addr 172.21.15.62:0/3778844631 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.458+0000 7feb17fff700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2006b810 0x7feb20101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.458+0000 7feb24fa2700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb201077b0 0x7feb20101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.458+0000 7feb1ffff700 1 -- 172.21.15.62:0/3778844631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2006b810 msgr2=0x7feb20101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.458+0000 7feb1ffff700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2006b810 0x7feb20101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.458+0000 7feb1ffff700 1 -- 172.21.15.62:0/3778844631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb201077b0 msgr2=0x7feb20101f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.458+0000 7feb1ffff700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb201077b0 0x7feb20101f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.458+0000 7feb1ffff700 1 -- 172.21.15.62:0/3778844631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb201a7200 con 0x7feb2006ae60 2024-06-10T00:34:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.458+0000 7feb24fa2700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb201077b0 0x7feb20101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.458+0000 7feb1ffff700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb2006ae60 0x7feb201014f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7feb1000b530 tx=0x7feb1000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.458+0000 7feb1dffb700 1 -- 172.21.15.62:0/3778844631 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10015020 con 0x7feb2006ae60 2024-06-10T00:34:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.458+0000 7feb26a05700 1 -- 172.21.15.62:0/3778844631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb201a74f0 con 0x7feb2006ae60 2024-06-10T00:34:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.458+0000 7feb1dffb700 1 -- 172.21.15.62:0/3778844631 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb10011d10 con 0x7feb2006ae60 2024-06-10T00:34:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.458+0000 7feb1dffb700 1 -- 172.21.15.62:0/3778844631 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1000f2e0 con 0x7feb2006ae60 2024-06-10T00:34:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.458+0000 7feb26a05700 1 -- 172.21.15.62:0/3778844631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb201a7a50 con 0x7feb2006ae60 2024-06-10T00:34:31.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.461+0000 7feb1dffb700 1 -- 172.21.15.62:0/3778844631 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7feb10018020 con 0x7feb2006ae60 2024-06-10T00:34:31.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.461+0000 7feb1dffb700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7feb0c0a2d90 0x7feb0c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.462+0000 7feb17fff700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7feb0c0a2d90 0x7feb0c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.462+0000 7feb1dffb700 1 -- 172.21.15.62:0/3778844631 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7feb100cef50 con 0x7feb2006ae60 2024-06-10T00:34:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.462+0000 7feb17fff700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7feb0c0a2d90 0x7feb0c0a5250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb0800b580 tx=0x7feb08005fb0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.462+0000 7feb31288b80 1 Processor -- start 2024-06-10T00:34:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.462+0000 7feb31288b80 1 -- start start 2024-06-10T00:34:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.462+0000 7feb31288b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc455f60 0x5556fc456340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.462+0000 7feb31288b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc456880 0x5556fc450260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.462+0000 7feb31288b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc4507a0 0x5556fc450bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.462+0000 7feb31288b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5556fc342f60 con 0x5556fc4507a0 2024-06-10T00:34:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.462+0000 7feb31288b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5556fc3a1850 con 0x5556fc456880 2024-06-10T00:34:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.462+0000 7feb31288b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5556fc309460 con 0x5556fc455f60 2024-06-10T00:34:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.463+0000 7feb24fa2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc4507a0 0x5556fc450bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.463+0000 7feb24fa2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc4507a0 0x5556fc450bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48638/0 (socket says 172.21.15.62:48638) 2024-06-10T00:34:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.463+0000 7feb24fa2700 1 -- 172.21.15.62:0/2112519908 learned_addr learned my addr 172.21.15.62:0/2112519908 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.463+0000 7feb17fff700 1 --2- 172.21.15.62:0/2112519908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc456880 0x5556fc450260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.463+0000 7feb1ffff700 1 --2- 172.21.15.62:0/2112519908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc455f60 0x5556fc456340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.463+0000 7feb24fa2700 1 -- 172.21.15.62:0/2112519908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc455f60 msgr2=0x5556fc456340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.463+0000 7feb24fa2700 1 --2- 172.21.15.62:0/2112519908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc455f60 0x5556fc456340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.463+0000 7feb24fa2700 1 -- 172.21.15.62:0/2112519908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc456880 msgr2=0x5556fc450260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.463+0000 7feb24fa2700 1 --2- 172.21.15.62:0/2112519908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc456880 0x5556fc450260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.463+0000 7feb24fa2700 1 -- 172.21.15.62:0/2112519908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556fc2f08d0 con 0x5556fc4507a0 2024-06-10T00:34:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.463+0000 7feb24fa2700 1 --2- 172.21.15.62:0/2112519908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc4507a0 0x5556fc450bc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7feb1800e260 tx=0x7feb1800e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4494cadff6d491b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.463+0000 7feb167fc700 1 -- 172.21.15.62:0/2112519908 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1800ee00 con 0x5556fc4507a0 2024-06-10T00:34:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.463+0000 7feb167fc700 1 -- 172.21.15.62:0/2112519908 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb18004bb0 con 0x5556fc4507a0 2024-06-10T00:34:31.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.463+0000 7feb167fc700 1 -- 172.21.15.62:0/2112519908 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb18005020 con 0x5556fc4507a0 2024-06-10T00:34:31.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.464+0000 7feb31288b80 1 -- 172.21.15.62:0/2112519908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc4507a0 msgr2=0x5556fc450bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:31.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.464+0000 7feb31288b80 1 --2- 172.21.15.62:0/2112519908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc4507a0 0x5556fc450bc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7feb1800e260 tx=0x7feb1800e570 comp rx=0 tx=0).stop 2024-06-10T00:34:31.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.464+0000 7feb31288b80 1 -- 172.21.15.62:0/2112519908 shutdown_connections 2024-06-10T00:34:31.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.464+0000 7feb31288b80 1 --2- 172.21.15.62:0/2112519908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc4507a0 0x5556fc450bc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.464+0000 7feb31288b80 1 --2- 172.21.15.62:0/2112519908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc455f60 0x5556fc456340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.464+0000 7feb31288b80 1 --2- 172.21.15.62:0/2112519908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc456880 0x5556fc450260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.464+0000 7feb31288b80 1 -- 172.21.15.62:0/2112519908 >> 172.21.15.62:0/2112519908 conn(0x5556fc37d080 msgr2=0x5556fc44b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:31.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.464+0000 7feb31288b80 1 -- 172.21.15.62:0/2112519908 shutdown_connections 2024-06-10T00:34:31.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.464+0000 7feb31288b80 1 -- 172.21.15.62:0/2112519908 wait complete. 2024-06-10T00:34:31.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.465+0000 7feb31288b80 1 -- start start 2024-06-10T00:34:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.465+0000 7feb31288b80 1 Processor -- start 2024-06-10T00:34:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.465+0000 7feb31288b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc4507a0 0x5556fc45c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.465+0000 7feb31288b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc455f60 0x5556fc459830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.465+0000 7feb31288b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc456880 0x5556fc459d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.465+0000 7feb31288b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5556fc341fc0 con 0x5556fc456880 2024-06-10T00:34:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.465+0000 7feb31288b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5556fc39d810 con 0x5556fc455f60 2024-06-10T00:34:31.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.465+0000 7feb31288b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5556fc45d260 con 0x5556fc4507a0 2024-06-10T00:34:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.465+0000 7feb31288b80 10 client.0 _ll_register_callbacks cb 0x5556fc3375b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.465+0000 7feb24fa2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc456880 0x5556fc459d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.465+0000 7feb24fa2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc456880 0x5556fc459d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48640/0 (socket says 172.21.15.62:48640) 2024-06-10T00:34:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.465+0000 7feb24fa2700 1 -- 172.21.15.62:0/465040400 learned_addr learned my addr 172.21.15.62:0/465040400 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.465+0000 7feb17fff700 1 --2- 172.21.15.62:0/465040400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc455f60 0x5556fc459830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.465+0000 7feb1ffff700 1 --2- 172.21.15.62:0/465040400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc4507a0 0x5556fc45c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.466+0000 7feb24fa2700 1 -- 172.21.15.62:0/465040400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc4507a0 msgr2=0x5556fc45c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.466+0000 7feb24fa2700 1 --2- 172.21.15.62:0/465040400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc4507a0 0x5556fc45c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.466+0000 7feb24fa2700 1 -- 172.21.15.62:0/465040400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc455f60 msgr2=0x5556fc459830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.466+0000 7feb24fa2700 1 --2- 172.21.15.62:0/465040400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc455f60 0x5556fc459830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.466+0000 7feb24fa2700 1 -- 172.21.15.62:0/465040400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556fc367750 con 0x5556fc456880 2024-06-10T00:34:31.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.466+0000 7feb24fa2700 1 --2- 172.21.15.62:0/465040400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc456880 0x5556fc459d70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7feb18007740 tx=0x7feb18007d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1532d1d12244896 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.466+0000 7feaff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.466+0000 7feaff7fe700 1 -- 172.21.15.62:0/465040400 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb18022030 con 0x5556fc456880 2024-06-10T00:34:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.466+0000 7feb31288b80 1 -- 172.21.15.62:0/465040400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb18004bb0 con 0x5556fc456880 2024-06-10T00:34:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.466+0000 7feaff7fe700 1 -- 172.21.15.62:0/465040400 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7feb18010070 con 0x5556fc456880 2024-06-10T00:34:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.466+0000 7feaff7fe700 1 -- 172.21.15.62:0/465040400 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feb1802f050 con 0x5556fc456880 2024-06-10T00:34:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.466+0000 7feb31288b80 10 client.15897 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.466+0000 7feb31288b80 1 -- 172.21.15.62:0/465040400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5556fc467960 con 0x5556fc456880 2024-06-10T00:34:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.466+0000 7feaff7fe700 1 -- 172.21.15.62:0/465040400 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb18004bb0 con 0x5556fc456880 2024-06-10T00:34:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.467+0000 7feaff7fe700 1 -- 172.21.15.62:0/465040400 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7feb180042a0 con 0x5556fc456880 2024-06-10T00:34:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.467+0000 7feb31288b80 10 client.15897 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.467+0000 7feb31288b80 10 client.15897 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:34:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.467+0000 7feb31288b80 20 client.15897 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.467+0000 7feb31288b80 1 --2- 172.21.15.62:0/465040400 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5556fc469e30 0x5556fc46c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.467+0000 7feb31288b80 4 client.15897 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:34:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.467+0000 7feb31288b80 1 -- 172.21.15.62:0/465040400 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556fc3aed50 con 0x5556fc469e30 2024-06-10T00:34:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.467+0000 7feb1ffff700 1 --2- 172.21.15.62:0/465040400 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5556fc469e30 0x5556fc46c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.467+0000 7feaff7fe700 10 client.15897 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:34:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.467+0000 7feb1ffff700 1 --2- 172.21.15.62:0/465040400 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5556fc469e30 0x5556fc46c220 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d3e7965ec2fafba server_cookie=dba39b22b5b132e7 in_seq=0 out_seq=0 2024-06-10T00:34:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.468+0000 7feaff7fe700 1 -- 172.21.15.62:0/465040400 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5556fc3aed50 con 0x5556fc469e30 2024-06-10T00:34:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.468+0000 7feaff7fe700 10 client.15897 handle_command_reply: tid=0 2024-06-10T00:34:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.468+0000 7feb31288b80 1 client.15897 shutdown 2024-06-10T00:34:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.468+0000 7feb31288b80 1 -- 172.21.15.62:0/465040400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc456880 msgr2=0x5556fc459d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.468+0000 7feb31288b80 1 --2- 172.21.15.62:0/465040400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc456880 0x5556fc459d70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7feb18007740 tx=0x7feb18007d00 comp rx=0 tx=0).stop 2024-06-10T00:34:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.468+0000 7feb31288b80 1 -- 172.21.15.62:0/465040400 shutdown_connections 2024-06-10T00:34:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.468+0000 7feb31288b80 1 --2- 172.21.15.62:0/465040400 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5556fc469e30 0x5556fc46c220 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.468+0000 7feb31288b80 1 --2- 172.21.15.62:0/465040400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc456880 0x5556fc459d70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.468+0000 7feb31288b80 1 --2- 172.21.15.62:0/465040400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc4507a0 0x5556fc45c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.468+0000 7feb31288b80 1 --2- 172.21.15.62:0/465040400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc455f60 0x5556fc459830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.468+0000 7feb31288b80 1 -- 172.21.15.62:0/465040400 >> 172.21.15.62:0/465040400 conn(0x5556fc37d080 msgr2=0x5556fc44a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:31.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.468+0000 7feaff7fe700 0 client.15897 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:34:31.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.469+0000 7feb31288b80 1 -- 172.21.15.62:0/465040400 shutdown_connections 2024-06-10T00:34:31.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.469+0000 7feb31288b80 1 -- 172.21.15.62:0/465040400 wait complete. 2024-06-10T00:34:31.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.469+0000 7feb31288b80 20 client.15897 trim_cache size 0 max 16384 2024-06-10T00:34:31.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb31288b80 1 Processor -- start 2024-06-10T00:34:31.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb31288b80 1 -- start start 2024-06-10T00:34:31.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb31288b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc4507a0 0x5556fc450260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb31288b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc455f60 0x5556fc45b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb31288b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc456880 0x5556fc44ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb31288b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feb201b08f0 con 0x5556fc455f60 2024-06-10T00:34:31.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb31288b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb201b2430 con 0x5556fc4507a0 2024-06-10T00:34:31.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb31288b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb201b0fc0 con 0x5556fc456880 2024-06-10T00:34:31.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb17fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc455f60 0x5556fc45b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb17fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc455f60 0x5556fc45b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48652/0 (socket says 172.21.15.62:48652) 2024-06-10T00:34:31.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb17fff700 1 -- 172.21.15.62:0/1545445001 learned_addr learned my addr 172.21.15.62:0/1545445001 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb1ffff700 1 --2- 172.21.15.62:0/1545445001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc4507a0 0x5556fc450260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb24fa2700 1 --2- 172.21.15.62:0/1545445001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc456880 0x5556fc44ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb17fff700 1 -- 172.21.15.62:0/1545445001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc456880 msgr2=0x5556fc44ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb17fff700 1 --2- 172.21.15.62:0/1545445001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc456880 0x5556fc44ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb17fff700 1 -- 172.21.15.62:0/1545445001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc4507a0 msgr2=0x5556fc450260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb17fff700 1 --2- 172.21.15.62:0/1545445001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc4507a0 0x5556fc450260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.484+0000 7feb17fff700 1 -- 172.21.15.62:0/1545445001 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556fc466560 con 0x5556fc455f60 2024-06-10T00:34:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.485+0000 7feb17fff700 1 --2- 172.21.15.62:0/1545445001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc455f60 0x5556fc45b8f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feb080099c0 tx=0x7feb08020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888339fa9c9d502 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.485+0000 7feb15ffb700 1 -- 172.21.15.62:0/1545445001 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08037070 con 0x5556fc455f60 2024-06-10T00:34:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.485+0000 7feb15ffb700 1 -- 172.21.15.62:0/1545445001 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb0800be90 con 0x5556fc455f60 2024-06-10T00:34:31.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.485+0000 7feb15ffb700 1 -- 172.21.15.62:0/1545445001 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0800fdc0 con 0x5556fc455f60 2024-06-10T00:34:31.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.485+0000 7feb31288b80 1 -- 172.21.15.62:0/1545445001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc455f60 msgr2=0x5556fc45b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:31.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.485+0000 7feb31288b80 1 --2- 172.21.15.62:0/1545445001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc455f60 0x5556fc45b8f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feb080099c0 tx=0x7feb08020000 comp rx=0 tx=0).stop 2024-06-10T00:34:31.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 -- 172.21.15.62:0/1545445001 shutdown_connections 2024-06-10T00:34:31.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 --2- 172.21.15.62:0/1545445001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc456880 0x5556fc44ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 --2- 172.21.15.62:0/1545445001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc4507a0 0x5556fc450260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 --2- 172.21.15.62:0/1545445001 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc455f60 0x5556fc45b8f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 -- 172.21.15.62:0/1545445001 >> 172.21.15.62:0/1545445001 conn(0x5556fc37d080 msgr2=0x5556fc4572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:31.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 -- 172.21.15.62:0/1545445001 shutdown_connections 2024-06-10T00:34:31.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 -- 172.21.15.62:0/1545445001 wait complete. 2024-06-10T00:34:31.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 -- start start 2024-06-10T00:34:31.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 Processor -- start 2024-06-10T00:34:31.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc4507a0 0x5556fc4cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc455f60 0x5556fc4ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc456880 0x5556fc46c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feb201b1d60 con 0x5556fc4507a0 2024-06-10T00:34:31.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb201b1690 con 0x5556fc456880 2024-06-10T00:34:31.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb201ae890 con 0x5556fc455f60 2024-06-10T00:34:31.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.486+0000 7feb31288b80 10 client.0 _ll_register_callbacks cb 0x5556fc3375b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:31.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feb1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc4507a0 0x5556fc4cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feb24fa2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc456880 0x5556fc46c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feb1ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc4507a0 0x5556fc4cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48654/0 (socket says 172.21.15.62:48654) 2024-06-10T00:34:31.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feb1ffff700 1 -- 172.21.15.62:0/3628565807 learned_addr learned my addr 172.21.15.62:0/3628565807 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:31.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feb24fa2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc456880 0x5556fc46c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:34038/0 (socket says 172.21.15.62:34038) 2024-06-10T00:34:31.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feb17fff700 1 --2- 172.21.15.62:0/3628565807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc455f60 0x5556fc4ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feb1ffff700 1 -- 172.21.15.62:0/3628565807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc455f60 msgr2=0x5556fc4ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:31.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feb1ffff700 1 --2- 172.21.15.62:0/3628565807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc455f60 0x5556fc4ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feb1ffff700 1 -- 172.21.15.62:0/3628565807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc456880 msgr2=0x5556fc46c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feb1ffff700 1 --2- 172.21.15.62:0/3628565807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc456880 0x5556fc46c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feb1ffff700 1 -- 172.21.15.62:0/3628565807 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556fc46cf20 con 0x5556fc4507a0 2024-06-10T00:34:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feb24fa2700 1 --2- 172.21.15.62:0/3628565807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc456880 0x5556fc46c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:34:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feb1ffff700 1 --2- 172.21.15.62:0/3628565807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc4507a0 0x5556fc4cca50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7feb10097ba0 tx=0x7feb100ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf73e4ad05bd3c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feaff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feaff7fe700 1 -- 172.21.15.62:0/3628565807 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb100b3070 con 0x5556fc4507a0 2024-06-10T00:34:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feaff7fe700 1 -- 172.21.15.62:0/3628565807 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7feb10010610 con 0x5556fc4507a0 2024-06-10T00:34:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feb31288b80 1 -- 172.21.15.62:0/3628565807 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0800be90 con 0x5556fc4507a0 2024-06-10T00:34:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.487+0000 7feaff7fe700 1 -- 172.21.15.62:0/3628565807 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feb100a3e40 con 0x5556fc4507a0 2024-06-10T00:34:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.488+0000 7feb31288b80 10 client.15903 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.488+0000 7feb31288b80 1 -- 172.21.15.62:0/3628565807 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5556fc4d4240 con 0x5556fc4507a0 2024-06-10T00:34:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.488+0000 7feaff7fe700 1 -- 172.21.15.62:0/3628565807 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb0800be90 con 0x5556fc4507a0 2024-06-10T00:34:31.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.488+0000 7feaff7fe700 1 -- 172.21.15.62:0/3628565807 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7feb100172c0 con 0x5556fc4507a0 2024-06-10T00:34:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.488+0000 7feb31288b80 10 client.15903 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.488+0000 7feb31288b80 10 client.15903 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:34:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.488+0000 7feb31288b80 20 client.15903 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.488+0000 7feb31288b80 1 --2- 172.21.15.62:0/3628565807 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5556fc4d4970 0x5556fc4d6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.488+0000 7feb31288b80 4 client.15903 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:34:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.488+0000 7feb31288b80 1 -- 172.21.15.62:0/3628565807 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556fc3a92d0 con 0x5556fc4d4970 2024-06-10T00:34:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.488+0000 7feb17fff700 1 --2- 172.21.15.62:0/3628565807 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5556fc4d4970 0x5556fc4d6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feb17fff700 1 --2- 172.21.15.62:0/3628565807 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5556fc4d4970 0x5556fc4d6d90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5dfcd53f8c36cb82 server_cookie=e97b45c012572b21 in_seq=0 out_seq=0 2024-06-10T00:34:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feaff7fe700 10 client.15903 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:34:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feaff7fe700 1 -- 172.21.15.62:0/3628565807 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556fc3a92d0 con 0x5556fc4d4970 2024-06-10T00:34:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feaff7fe700 10 client.15903 handle_command_reply: tid=0 2024-06-10T00:34:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feb31288b80 1 client.15903 shutdown 2024-06-10T00:34:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feb31288b80 1 -- 172.21.15.62:0/3628565807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc4507a0 msgr2=0x5556fc4cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:31.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feb31288b80 1 --2- 172.21.15.62:0/3628565807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc4507a0 0x5556fc4cca50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7feb10097ba0 tx=0x7feb100ce040 comp rx=0 tx=0).stop 2024-06-10T00:34:31.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feb31288b80 1 -- 172.21.15.62:0/3628565807 shutdown_connections 2024-06-10T00:34:31.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feb31288b80 1 --2- 172.21.15.62:0/3628565807 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5556fc4d4970 0x5556fc4d6d90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feb31288b80 1 --2- 172.21.15.62:0/3628565807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5556fc4507a0 0x5556fc4cca50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feb31288b80 1 --2- 172.21.15.62:0/3628565807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556fc456880 0x5556fc46c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feb31288b80 1 --2- 172.21.15.62:0/3628565807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5556fc455f60 0x5556fc4ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feb31288b80 1 -- 172.21.15.62:0/3628565807 >> 172.21.15.62:0/3628565807 conn(0x5556fc37d080 msgr2=0x5556fc465de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:31.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feaff7fe700 0 client.15903 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:34:31.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feb31288b80 1 -- 172.21.15.62:0/3628565807 shutdown_connections 2024-06-10T00:34:31.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feb31288b80 1 -- 172.21.15.62:0/3628565807 wait complete. 2024-06-10T00:34:31.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.489+0000 7feb31288b80 20 client.15903 trim_cache size 0 max 16384 2024-06-10T00:34:31.495 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:34:31.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.490+0000 7feb26a05700 1 -- 172.21.15.62:0/3778844631 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7feb0c0a2d90 msgr2=0x7feb0c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:31.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.490+0000 7feb26a05700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7feb0c0a2d90 0x7feb0c0a5250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb0800b580 tx=0x7feb08005fb0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.490+0000 7feb26a05700 1 -- 172.21.15.62:0/3778844631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb2006ae60 msgr2=0x7feb201014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:31.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.490+0000 7feb26a05700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb2006ae60 0x7feb201014f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7feb1000b530 tx=0x7feb1000b840 comp rx=0 tx=0).stop 2024-06-10T00:34:31.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.490+0000 7feb26a05700 1 -- 172.21.15.62:0/3778844631 shutdown_connections 2024-06-10T00:34:31.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.490+0000 7feb26a05700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7feb0c0a2d90 0x7feb0c0a5250 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.490+0000 7feb26a05700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb2006ae60 0x7feb201014f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.490+0000 7feb26a05700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb201077b0 0x7feb20101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.490+0000 7feb26a05700 1 --2- 172.21.15.62:0/3778844631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2006b810 0x7feb20101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:31.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.490+0000 7feb26a05700 1 -- 172.21.15.62:0/3778844631 >> 172.21.15.62:0/3778844631 conn(0x7feb200773e0 msgr2=0x7feb20100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:31.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.490+0000 7feb26a05700 1 -- 172.21.15.62:0/3778844631 shutdown_connections 2024-06-10T00:34:31.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:31.490+0000 7feb26a05700 1 -- 172.21.15.62:0/3778844631 wait complete. 2024-06-10T00:34:31.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:31 smithi138 ceph-mon[36549]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T00:34:31.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:31 smithi105 ceph-mon[36321]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-06-10T00:34:32.384 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T00:34:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:33.091 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:33.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:33 smithi062 ceph-mon[32390]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-06-10T00:34:33.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:33 smithi138 ceph-mon[36549]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-06-10T00:34:33.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:33 smithi105 ceph-mon[36321]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-06-10T00:34:35.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.396+0000 7f4b497d1700 1 -- 172.21.15.62:0/13520551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b44101390 msgr2=0x7f4b44101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:35.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.396+0000 7f4b497d1700 1 --2- 172.21.15.62:0/13520551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b44101390 0x7f4b44101810 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009990 tx=0x7f4b2c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.396+0000 7f4b497d1700 1 -- 172.21.15.62:0/13520551 shutdown_connections 2024-06-10T00:34:35.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.396+0000 7f4b497d1700 1 --2- 172.21.15.62:0/13520551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b44101d50 0x7f4b4410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.396+0000 7f4b497d1700 1 --2- 172.21.15.62:0/13520551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b44107400 0x7f4b441077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.396+0000 7f4b497d1700 1 --2- 172.21.15.62:0/13520551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b44101390 0x7f4b44101810 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.396+0000 7f4b497d1700 1 -- 172.21.15.62:0/13520551 >> 172.21.15.62:0/13520551 conn(0x7f4b44078340 msgr2=0x7f4b44078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:35.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.396+0000 7f4b497d1700 1 -- 172.21.15.62:0/13520551 shutdown_connections 2024-06-10T00:34:35.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.396+0000 7f4b497d1700 1 -- 172.21.15.62:0/13520551 wait complete. 2024-06-10T00:34:35.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.397+0000 7f4b497d1700 1 Processor -- start 2024-06-10T00:34:35.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.397+0000 7f4b497d1700 1 -- start start 2024-06-10T00:34:35.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b497d1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b44101390 0x7f4b4419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b497d1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b44101d50 0x7f4b4419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b497d1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b44107400 0x7f4b441989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b497d1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4b441116a0 con 0x7f4b44101390 2024-06-10T00:34:35.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b497d1700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44111520 con 0x7f4b44101d50 2024-06-10T00:34:35.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b497d1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44111820 con 0x7f4b44107400 2024-06-10T00:34:35.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b42ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b44101390 0x7f4b4419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b42ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b44101390 0x7f4b4419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48678/0 (socket says 172.21.15.62:48678) 2024-06-10T00:34:35.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b42ffd700 1 -- 172.21.15.62:0/2742096032 learned_addr learned my addr 172.21.15.62:0/2742096032 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:35.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b3a7fc700 1 --2- 172.21.15.62:0/2742096032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b44101d50 0x7f4b4419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b437fe700 1 --2- 172.21.15.62:0/2742096032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b44107400 0x7f4b441989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b42ffd700 1 -- 172.21.15.62:0/2742096032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b44107400 msgr2=0x7f4b441989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:35.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b42ffd700 1 --2- 172.21.15.62:0/2742096032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b44107400 0x7f4b441989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b42ffd700 1 -- 172.21.15.62:0/2742096032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b44101d50 msgr2=0x7f4b4419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:35.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b42ffd700 1 --2- 172.21.15.62:0/2742096032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b44101d50 0x7f4b4419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b42ffd700 1 -- 172.21.15.62:0/2742096032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b44199120 con 0x7f4b44101390 2024-06-10T00:34:35.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b42ffd700 1 --2- 172.21.15.62:0/2742096032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b44101390 0x7f4b4419e800 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4b34007e20 tx=0x7f4b3400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:35.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.398+0000 7f4b40ff9700 1 -- 172.21.15.62:0/2742096032 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34016020 con 0x7f4b44101390 2024-06-10T00:34:35.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.399+0000 7f4b40ff9700 1 -- 172.21.15.62:0/2742096032 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b3400ac90 con 0x7f4b44101390 2024-06-10T00:34:35.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.399+0000 7f4b497d1700 1 -- 172.21.15.62:0/2742096032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b44199410 con 0x7f4b44101390 2024-06-10T00:34:35.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.399+0000 7f4b40ff9700 1 -- 172.21.15.62:0/2742096032 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34011620 con 0x7f4b44101390 2024-06-10T00:34:35.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.399+0000 7f4b497d1700 1 -- 172.21.15.62:0/2742096032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b441a56d0 con 0x7f4b44101390 2024-06-10T00:34:35.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.400+0000 7f4b497d1700 1 -- 172.21.15.62:0/2742096032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b44069180 con 0x7f4b44101390 2024-06-10T00:34:35.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.401+0000 7f4b40ff9700 1 -- 172.21.15.62:0/2742096032 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4b34019020 con 0x7f4b44101390 2024-06-10T00:34:35.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.402+0000 7f4b40ff9700 1 --2- 172.21.15.62:0/2742096032 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4b300a2d90 0x7f4b300a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.402+0000 7f4b3a7fc700 1 --2- 172.21.15.62:0/2742096032 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4b300a2d90 0x7f4b300a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.402+0000 7f4b40ff9700 1 -- 172.21.15.62:0/2742096032 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f4b340cf300 con 0x7f4b44101390 2024-06-10T00:34:35.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.402+0000 7f4b3a7fc700 1 --2- 172.21.15.62:0/2742096032 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4b300a2d90 0x7f4b300a5250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00b5c0 tx=0x7f4b2c005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.407+0000 7f4b40ff9700 1 -- 172.21.15.62:0/2742096032 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4b3409bc80 con 0x7f4b44101390 2024-06-10T00:34:35.543 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:35 smithi062 ceph-mon[32390]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T00:34:35.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:35 smithi138 ceph-mon[36549]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T00:34:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.662+0000 7f4b497d1700 1 -- 172.21.15.62:0/2742096032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b44199ef0 con 0x7f4b44101390 2024-06-10T00:34:35.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.663+0000 7f4b40ff9700 1 -- 172.21.15.62:0/2742096032 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f4b3409b580 con 0x7f4b44101390 2024-06-10T00:34:35.663 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:34:35.664 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:32:47.783154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:34:35.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.665+0000 7f4b497d1700 1 -- 172.21.15.62:0/2742096032 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4b300a2d90 msgr2=0x7f4b300a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:35.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.665+0000 7f4b497d1700 1 --2- 172.21.15.62:0/2742096032 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4b300a2d90 0x7f4b300a5250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00b5c0 tx=0x7f4b2c005960 comp rx=0 tx=0).stop 2024-06-10T00:34:35.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.665+0000 7f4b497d1700 1 -- 172.21.15.62:0/2742096032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b44101390 msgr2=0x7f4b4419e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:35.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.665+0000 7f4b497d1700 1 --2- 172.21.15.62:0/2742096032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b44101390 0x7f4b4419e800 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4b34007e20 tx=0x7f4b3400a340 comp rx=0 tx=0).stop 2024-06-10T00:34:35.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.665+0000 7f4b497d1700 1 -- 172.21.15.62:0/2742096032 shutdown_connections 2024-06-10T00:34:35.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.665+0000 7f4b497d1700 1 --2- 172.21.15.62:0/2742096032 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4b300a2d90 0x7f4b300a5250 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.665+0000 7f4b497d1700 1 --2- 172.21.15.62:0/2742096032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b44101390 0x7f4b4419e800 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.665+0000 7f4b497d1700 1 --2- 172.21.15.62:0/2742096032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b44107400 0x7f4b441989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.665+0000 7f4b497d1700 1 --2- 172.21.15.62:0/2742096032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b44101d50 0x7f4b4419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.665+0000 7f4b497d1700 1 -- 172.21.15.62:0/2742096032 >> 172.21.15.62:0/2742096032 conn(0x7f4b44078340 msgr2=0x7f4b44100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:35.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.665+0000 7f4b497d1700 1 -- 172.21.15.62:0/2742096032 shutdown_connections 2024-06-10T00:34:35.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.665+0000 7f4b497d1700 1 -- 172.21.15.62:0/2742096032 wait complete. 2024-06-10T00:34:35.667 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-06-10T00:34:35.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.760+0000 7f405c9f7700 1 -- 172.21.15.62:0/673629985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4058107870 msgr2=0x7f405810f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:35.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.760+0000 7f405c9f7700 1 --2- 172.21.15.62:0/673629985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4058107870 0x7f405810f3f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f405000b0d0 tx=0x7f405000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.760+0000 7f405c9f7700 1 -- 172.21.15.62:0/673629985 shutdown_connections 2024-06-10T00:34:35.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.760+0000 7f405c9f7700 1 --2- 172.21.15.62:0/673629985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4058107870 0x7f405810f3f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.760+0000 7f405c9f7700 1 --2- 172.21.15.62:0/673629985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40581027a0 0x7f4058102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.760+0000 7f405c9f7700 1 --2- 172.21.15.62:0/673629985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4058103150 0x7f4058107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.760+0000 7f405c9f7700 1 -- 172.21.15.62:0/673629985 >> 172.21.15.62:0/673629985 conn(0x7f4058079520 msgr2=0x7f4058079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:35.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.760+0000 7f405c9f7700 1 -- 172.21.15.62:0/673629985 shutdown_connections 2024-06-10T00:34:35.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.760+0000 7f405c9f7700 1 -- 172.21.15.62:0/673629985 wait complete. 2024-06-10T00:34:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.761+0000 7f405c9f7700 1 Processor -- start 2024-06-10T00:34:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.761+0000 7f405c9f7700 1 -- start start 2024-06-10T00:34:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.761+0000 7f405c9f7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40581027a0 0x7f405819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.761+0000 7f405c9f7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4058103150 0x7f405819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.761+0000 7f405c9f7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4058107870 0x7f4058198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.761+0000 7f405c9f7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4058112790 con 0x7f4058107870 2024-06-10T00:34:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.761+0000 7f405c9f7700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4058112610 con 0x7f40581027a0 2024-06-10T00:34:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.761+0000 7f405c9f7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4058112910 con 0x7f4058103150 2024-06-10T00:34:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f4056d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4058107870 0x7f4058198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f4056d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4058107870 0x7f4058198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48694/0 (socket says 172.21.15.62:48694) 2024-06-10T00:34:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f4056d9d700 1 -- 172.21.15.62:0/1889081164 learned_addr learned my addr 172.21.15.62:0/1889081164 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f404dd9b700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4058103150 0x7f405819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f405659c700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40581027a0 0x7f405819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f4056d9d700 1 -- 172.21.15.62:0/1889081164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4058103150 msgr2=0x7f405819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f4056d9d700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4058103150 0x7f405819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f4056d9d700 1 -- 172.21.15.62:0/1889081164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40581027a0 msgr2=0x7f405819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f4056d9d700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40581027a0 0x7f405819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f4056d9d700 1 -- 172.21.15.62:0/1889081164 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40581990f0 con 0x7f4058107870 2024-06-10T00:34:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f404dd9b700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4058103150 0x7f405819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:34:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f4056d9d700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4058107870 0x7f4058198990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4050006070 tx=0x7f4050007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f404ffff700 1 -- 172.21.15.62:0/1889081164 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050003c00 con 0x7f4058107870 2024-06-10T00:34:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f405c9f7700 1 -- 172.21.15.62:0/1889081164 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4058199380 con 0x7f4058107870 2024-06-10T00:34:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f404ffff700 1 -- 172.21.15.62:0/1889081164 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4050003da0 con 0x7f4058107870 2024-06-10T00:34:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f404ffff700 1 -- 172.21.15.62:0/1889081164 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405001b880 con 0x7f4058107870 2024-06-10T00:34:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.762+0000 7f405c9f7700 1 -- 172.21.15.62:0/1889081164 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40581a5650 con 0x7f4058107870 2024-06-10T00:34:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.765+0000 7f404ffff700 1 -- 172.21.15.62:0/1889081164 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f405001ba20 con 0x7f4058107870 2024-06-10T00:34:35.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.766+0000 7f404ffff700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f40440a2d90 0x7f40440a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.766+0000 7f405659c700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f40440a2d90 0x7f40440a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.766+0000 7f404ffff700 1 -- 172.21.15.62:0/1889081164 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f405000f020 con 0x7f4058107870 2024-06-10T00:34:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.766+0000 7f406727ab80 1 Processor -- start 2024-06-10T00:34:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f405659c700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f40440a2d90 0x7f40440a5250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f40581073b0 tx=0x7f404800b410 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f406727ab80 1 -- start start 2024-06-10T00:34:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f406727ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d0af60 0x559c01d0b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f406727ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0b880 0x559c01d05260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f406727ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d057a0 0x559c01d05bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f406727ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559c01bf7f60 con 0x559c01d0af60 2024-06-10T00:34:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f406727ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559c01c56850 con 0x559c01d057a0 2024-06-10T00:34:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f406727ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559c01bbe460 con 0x559c01d0b880 2024-06-10T00:34:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f405659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d0af60 0x559c01d0b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f405659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d0af60 0x559c01d0b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48698/0 (socket says 172.21.15.62:48698) 2024-06-10T00:34:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f405659c700 1 -- 172.21.15.62:0/3146193359 learned_addr learned my addr 172.21.15.62:0/3146193359 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f404dd9b700 1 --2- 172.21.15.62:0/3146193359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0b880 0x559c01d05260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f4056d9d700 1 --2- 172.21.15.62:0/3146193359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d057a0 0x559c01d05bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f405659c700 1 -- 172.21.15.62:0/3146193359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0b880 msgr2=0x559c01d05260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f405659c700 1 --2- 172.21.15.62:0/3146193359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0b880 0x559c01d05260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f405659c700 1 -- 172.21.15.62:0/3146193359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d057a0 msgr2=0x559c01d05bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f405659c700 1 --2- 172.21.15.62:0/3146193359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d057a0 0x559c01d05bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.767+0000 7f405659c700 1 -- 172.21.15.62:0/3146193359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c01ba58d0 con 0x559c01d0af60 2024-06-10T00:34:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.768+0000 7f405659c700 1 --2- 172.21.15.62:0/3146193359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d0af60 0x559c01d0b340 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4048010f70 tx=0x7f4048013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bae0f74923abb76 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:35.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.768+0000 7f404cd99700 1 -- 172.21.15.62:0/3146193359 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404800be60 con 0x559c01d0af60 2024-06-10T00:34:35.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.768+0000 7f404cd99700 1 -- 172.21.15.62:0/3146193359 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f404801eb00 con 0x559c01d0af60 2024-06-10T00:34:35.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.768+0000 7f404cd99700 1 -- 172.21.15.62:0/3146193359 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404801cbb0 con 0x559c01d0af60 2024-06-10T00:34:35.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.769+0000 7f406727ab80 1 -- 172.21.15.62:0/3146193359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d0af60 msgr2=0x559c01d0b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:35.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.769+0000 7f406727ab80 1 --2- 172.21.15.62:0/3146193359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d0af60 0x559c01d0b340 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4048010f70 tx=0x7f4048013000 comp rx=0 tx=0).stop 2024-06-10T00:34:35.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.769+0000 7f406727ab80 1 -- 172.21.15.62:0/3146193359 shutdown_connections 2024-06-10T00:34:35.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.769+0000 7f406727ab80 1 --2- 172.21.15.62:0/3146193359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d0af60 0x559c01d0b340 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.769+0000 7f406727ab80 1 --2- 172.21.15.62:0/3146193359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d057a0 0x559c01d05bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.769+0000 7f406727ab80 1 --2- 172.21.15.62:0/3146193359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0b880 0x559c01d05260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.769+0000 7f406727ab80 1 -- 172.21.15.62:0/3146193359 >> 172.21.15.62:0/3146193359 conn(0x559c01c32080 msgr2=0x559c01d00220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:35.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.769+0000 7f406727ab80 1 -- 172.21.15.62:0/3146193359 shutdown_connections 2024-06-10T00:34:35.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.769+0000 7f406727ab80 1 -- 172.21.15.62:0/3146193359 wait complete. 2024-06-10T00:34:35.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.769+0000 7f406727ab80 1 -- start start 2024-06-10T00:34:35.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.769+0000 7f406727ab80 1 Processor -- start 2024-06-10T00:34:35.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f406727ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d11580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f406727ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0af60 0x559c01d0e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f406727ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0b880 0x559c01d0edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f406727ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559c01bf6fc0 con 0x559c01d057a0 2024-06-10T00:34:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f406727ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559c01c52810 con 0x559c01d0b880 2024-06-10T00:34:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f406727ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559c01d12260 con 0x559c01d0af60 2024-06-10T00:34:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f406727ab80 10 client.0 _ll_register_callbacks cb 0x559c01bec5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f405659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d11580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f405659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d11580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48700/0 (socket says 172.21.15.62:48700) 2024-06-10T00:34:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f405659c700 1 -- 172.21.15.62:0/866317024 learned_addr learned my addr 172.21.15.62:0/866317024 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f4056d9d700 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0b880 0x559c01d0edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f404dd9b700 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0af60 0x559c01d0e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f405659c700 1 -- 172.21.15.62:0/866317024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0af60 msgr2=0x559c01d0e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f405659c700 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0af60 0x559c01d0e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f405659c700 1 -- 172.21.15.62:0/866317024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0b880 msgr2=0x559c01d0edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f405659c700 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0b880 0x559c01d0edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f405659c700 1 -- 172.21.15.62:0/866317024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c01c1c750 con 0x559c01d057a0 2024-06-10T00:34:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f4056d9d700 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0b880 0x559c01d0edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:34:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f404dd9b700 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0af60 0x559c01d0e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f405659c700 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d11580 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f404801c5e0 tx=0x7f404801c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ee86052787f393 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.770+0000 7f403dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.771+0000 7f403dffb700 1 -- 172.21.15.62:0/866317024 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048035060 con 0x559c01d057a0 2024-06-10T00:34:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.771+0000 7f403dffb700 1 -- 172.21.15.62:0/866317024 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f404801e7f0 con 0x559c01d057a0 2024-06-10T00:34:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.771+0000 7f406727ab80 1 -- 172.21.15.62:0/866317024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f404801eb00 con 0x559c01d057a0 2024-06-10T00:34:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.771+0000 7f403dffb700 1 -- 172.21.15.62:0/866317024 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f404803c050 con 0x559c01d057a0 2024-06-10T00:34:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.771+0000 7f406727ab80 10 client.15921 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.771+0000 7f406727ab80 1 -- 172.21.15.62:0/866317024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559c01d1c9a0 con 0x559c01d057a0 2024-06-10T00:34:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.771+0000 7f403dffb700 1 -- 172.21.15.62:0/866317024 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f404801eb00 con 0x559c01d057a0 2024-06-10T00:34:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.771+0000 7f403dffb700 1 -- 172.21.15.62:0/866317024 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4048004690 con 0x559c01d057a0 2024-06-10T00:34:35.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.771+0000 7f406727ab80 10 client.15921 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:35.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.771+0000 7f406727ab80 10 client.15921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:34:35.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.771+0000 7f406727ab80 20 client.15921 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:35.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.771+0000 7f406727ab80 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x559c01d1ef20 0x559c01d21310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.771+0000 7f406727ab80 4 client.15921 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:34:35.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.771+0000 7f406727ab80 1 -- 172.21.15.62:0/866317024 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c01c63d50 con 0x559c01d1ef20 2024-06-10T00:34:35.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.772+0000 7f404dd9b700 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x559c01d1ef20 0x559c01d21310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.772+0000 7f404dd9b700 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x559c01d1ef20 0x559c01d21310 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6712a7a1572d8982 server_cookie=4bf7e6db50cc6279 in_seq=0 out_seq=0 2024-06-10T00:34:35.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.772+0000 7f403dffb700 10 client.15921 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:35.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f403dffb700 1 -- 172.21.15.62:0/866317024 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559c01c63d50 con 0x559c01d1ef20 2024-06-10T00:34:35.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f403dffb700 10 client.15921 handle_command_reply: tid=0 2024-06-10T00:34:35.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f406727ab80 1 client.15921 shutdown 2024-06-10T00:34:35.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f406727ab80 1 -- 172.21.15.62:0/866317024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 msgr2=0x559c01d11580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:35.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f406727ab80 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d11580 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f404801c5e0 tx=0x7f404801c610 comp rx=0 tx=0).stop 2024-06-10T00:34:35.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f406727ab80 1 -- 172.21.15.62:0/866317024 shutdown_connections 2024-06-10T00:34:35.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f406727ab80 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x559c01d1ef20 0x559c01d21310 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f403dffb700 0 client.15921 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:35.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f406727ab80 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d11580 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f406727ab80 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0b880 0x559c01d0edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f406727ab80 1 --2- 172.21.15.62:0/866317024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0af60 0x559c01d0e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f406727ab80 1 -- 172.21.15.62:0/866317024 >> 172.21.15.62:0/866317024 conn(0x559c01c32080 msgr2=0x559c01cff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:35.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f406727ab80 1 -- 172.21.15.62:0/866317024 shutdown_connections 2024-06-10T00:34:35.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f406727ab80 1 -- 172.21.15.62:0/866317024 wait complete. 2024-06-10T00:34:35.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.773+0000 7f406727ab80 20 client.15921 trim_cache size 0 max 16384 2024-06-10T00:34:35.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:35 smithi105 ceph-mon[36321]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T00:34:35.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.789+0000 7f406727ab80 1 Processor -- start 2024-06-10T00:34:35.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.789+0000 7f406727ab80 1 -- start start 2024-06-10T00:34:35.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.789+0000 7f406727ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d0c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.789+0000 7f406727ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0af60 0x559c01d107c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.789+0000 7f406727ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0b880 0x559c01cffe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.789+0000 7f406727ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f40581ae460 con 0x559c01d057a0 2024-06-10T00:34:35.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.789+0000 7f406727ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f40581affa0 con 0x559c01d0af60 2024-06-10T00:34:35.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.789+0000 7f406727ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40581aeb30 con 0x559c01d0b880 2024-06-10T00:34:35.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f405659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d0c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f405659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d0c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48706/0 (socket says 172.21.15.62:48706) 2024-06-10T00:34:35.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f405659c700 1 -- 172.21.15.62:0/3240403369 learned_addr learned my addr 172.21.15.62:0/3240403369 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:35.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f4056d9d700 1 --2- 172.21.15.62:0/3240403369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0b880 0x559c01cffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f405659c700 1 -- 172.21.15.62:0/3240403369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0b880 msgr2=0x559c01cffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:35.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f404dd9b700 1 --2- 172.21.15.62:0/3240403369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0af60 0x559c01d107c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f405659c700 1 --2- 172.21.15.62:0/3240403369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0b880 0x559c01cffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f405659c700 1 -- 172.21.15.62:0/3240403369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0af60 msgr2=0x559c01d107c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:35.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f405659c700 1 --2- 172.21.15.62:0/3240403369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0af60 0x559c01d107c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f405659c700 1 -- 172.21.15.62:0/3240403369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c01d0cdf0 con 0x559c01d057a0 2024-06-10T00:34:35.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f4056d9d700 1 --2- 172.21.15.62:0/3240403369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0b880 0x559c01cffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:35.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f404dd9b700 1 --2- 172.21.15.62:0/3240403369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0af60 0x559c01d107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:35.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f405659c700 1 --2- 172.21.15.62:0/3240403369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d0c4f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4048004a60 tx=0x7f404800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c43dce0a5a6c3c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:35.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f403ffff700 1 -- 172.21.15.62:0/3240403369 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048035030 con 0x559c01d057a0 2024-06-10T00:34:35.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f403ffff700 1 -- 172.21.15.62:0/3240403369 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f404801cd00 con 0x559c01d057a0 2024-06-10T00:34:35.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.790+0000 7f403ffff700 1 -- 172.21.15.62:0/3240403369 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048004cd0 con 0x559c01d057a0 2024-06-10T00:34:35.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.791+0000 7f406727ab80 1 -- 172.21.15.62:0/3240403369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 msgr2=0x559c01d0c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:35.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.791+0000 7f406727ab80 1 --2- 172.21.15.62:0/3240403369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d0c4f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4048004a60 tx=0x7f404800bd50 comp rx=0 tx=0).stop 2024-06-10T00:34:35.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.791+0000 7f406727ab80 1 -- 172.21.15.62:0/3240403369 shutdown_connections 2024-06-10T00:34:35.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.791+0000 7f406727ab80 1 --2- 172.21.15.62:0/3240403369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d0c4f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.791+0000 7f406727ab80 1 --2- 172.21.15.62:0/3240403369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0b880 0x559c01cffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.791+0000 7f406727ab80 1 --2- 172.21.15.62:0/3240403369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0af60 0x559c01d107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.791+0000 7f406727ab80 1 -- 172.21.15.62:0/3240403369 >> 172.21.15.62:0/3240403369 conn(0x559c01c32080 msgr2=0x559c01d100b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:35.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.791+0000 7f406727ab80 1 -- 172.21.15.62:0/3240403369 shutdown_connections 2024-06-10T00:34:35.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.791+0000 7f406727ab80 1 -- 172.21.15.62:0/3240403369 wait complete. 2024-06-10T00:34:35.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.791+0000 7f406727ab80 1 -- start start 2024-06-10T00:34:35.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.791+0000 7f406727ab80 1 Processor -- start 2024-06-10T00:34:35.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f406727ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d24830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f406727ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0af60 0x559c01d21bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f406727ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0b880 0x559c01d22110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f406727ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f40581af8d0 con 0x559c01d057a0 2024-06-10T00:34:35.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f406727ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f40581af200 con 0x559c01d0b880 2024-06-10T00:34:35.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f406727ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40581ac400 con 0x559c01d0af60 2024-06-10T00:34:35.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f406727ab80 10 client.0 _ll_register_callbacks cb 0x559c01bec5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:35.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f405659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d24830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f405659c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d24830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48722/0 (socket says 172.21.15.62:48722) 2024-06-10T00:34:35.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f405659c700 1 -- 172.21.15.62:0/780519262 learned_addr learned my addr 172.21.15.62:0/780519262 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:35.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f404dd9b700 1 --2- 172.21.15.62:0/780519262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0af60 0x559c01d21bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f4056d9d700 1 --2- 172.21.15.62:0/780519262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0b880 0x559c01d22110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f405659c700 1 -- 172.21.15.62:0/780519262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0af60 msgr2=0x559c01d21bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:35.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f405659c700 1 --2- 172.21.15.62:0/780519262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0af60 0x559c01d21bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f405659c700 1 -- 172.21.15.62:0/780519262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0b880 msgr2=0x559c01d22110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:35.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f405659c700 1 --2- 172.21.15.62:0/780519262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0b880 0x559c01d22110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f405659c700 1 -- 172.21.15.62:0/780519262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c01d226d0 con 0x559c01d057a0 2024-06-10T00:34:35.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f405659c700 1 --2- 172.21.15.62:0/780519262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d24830 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4048005810 tx=0x7f404801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a8cc9fab8216c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:35.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f403dffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:35.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.792+0000 7f403dffb700 1 -- 172.21.15.62:0/780519262 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048035070 con 0x559c01d057a0 2024-06-10T00:34:35.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.793+0000 7f403dffb700 1 -- 172.21.15.62:0/780519262 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f4048045aa0 con 0x559c01d057a0 2024-06-10T00:34:35.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.793+0000 7f406727ab80 1 -- 172.21.15.62:0/780519262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f404801cd00 con 0x559c01d057a0 2024-06-10T00:34:35.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.793+0000 7f403dffb700 1 -- 172.21.15.62:0/780519262 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4048053050 con 0x559c01d057a0 2024-06-10T00:34:35.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.793+0000 7f406727ab80 10 client.15927 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:35.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.793+0000 7f406727ab80 1 -- 172.21.15.62:0/780519262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559c01d895a0 con 0x559c01d057a0 2024-06-10T00:34:35.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.793+0000 7f403dffb700 1 -- 172.21.15.62:0/780519262 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f404801cd00 con 0x559c01d057a0 2024-06-10T00:34:35.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.793+0000 7f403dffb700 1 -- 172.21.15.62:0/780519262 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f404803f070 con 0x559c01d057a0 2024-06-10T00:34:35.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.793+0000 7f406727ab80 10 client.15927 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:35.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.793+0000 7f406727ab80 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:34:35.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.793+0000 7f406727ab80 20 client.15927 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:35.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.793+0000 7f406727ab80 1 --2- 172.21.15.62:0/780519262 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x559c01d1ef20 0x559c01d8bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:35.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.793+0000 7f406727ab80 4 client.15927 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:34:35.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.793+0000 7f406727ab80 1 -- 172.21.15.62:0/780519262 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c01c5e2d0 con 0x559c01d1ef20 2024-06-10T00:34:35.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f404dd9b700 1 --2- 172.21.15.62:0/780519262 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x559c01d1ef20 0x559c01d8bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:35.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f404dd9b700 1 --2- 172.21.15.62:0/780519262 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x559c01d1ef20 0x559c01d8bbc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfbacc6e229ec75d server_cookie=45fcf3e7abef45ce in_seq=0 out_seq=0 2024-06-10T00:34:35.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f403dffb700 10 client.15927 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:35.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f403dffb700 1 -- 172.21.15.62:0/780519262 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c01c5e2d0 con 0x559c01d1ef20 2024-06-10T00:34:35.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f403dffb700 10 client.15927 handle_command_reply: tid=0 2024-06-10T00:34:35.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f406727ab80 1 client.15927 shutdown 2024-06-10T00:34:35.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f406727ab80 1 -- 172.21.15.62:0/780519262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 msgr2=0x559c01d24830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:35.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f406727ab80 1 --2- 172.21.15.62:0/780519262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d24830 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4048005810 tx=0x7f404801ca50 comp rx=0 tx=0).stop 2024-06-10T00:34:35.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f406727ab80 1 -- 172.21.15.62:0/780519262 shutdown_connections 2024-06-10T00:34:35.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f406727ab80 1 --2- 172.21.15.62:0/780519262 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x559c01d1ef20 0x559c01d8bbc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f406727ab80 1 --2- 172.21.15.62:0/780519262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c01d057a0 0x559c01d24830 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f406727ab80 1 --2- 172.21.15.62:0/780519262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c01d0b880 0x559c01d22110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f406727ab80 1 --2- 172.21.15.62:0/780519262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c01d0af60 0x559c01d21bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f406727ab80 1 -- 172.21.15.62:0/780519262 >> 172.21.15.62:0/780519262 conn(0x559c01c32080 msgr2=0x559c01d0c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:35.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f403dffb700 0 client.15927 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:35.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f406727ab80 1 -- 172.21.15.62:0/780519262 shutdown_connections 2024-06-10T00:34:35.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.794+0000 7f406727ab80 1 -- 172.21.15.62:0/780519262 wait complete. 2024-06-10T00:34:35.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.795+0000 7f406727ab80 20 client.15927 trim_cache size 0 max 16384 2024-06-10T00:34:35.801 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:34:35.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.795+0000 7f405c9f7700 1 -- 172.21.15.62:0/1889081164 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f40440a2d90 msgr2=0x7f40440a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:35.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.795+0000 7f405c9f7700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f40440a2d90 0x7f40440a5250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f40581073b0 tx=0x7f404800b410 comp rx=0 tx=0).stop 2024-06-10T00:34:35.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.795+0000 7f405c9f7700 1 -- 172.21.15.62:0/1889081164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4058107870 msgr2=0x7f4058198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:35.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.795+0000 7f405c9f7700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4058107870 0x7f4058198990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4050006070 tx=0x7f4050007a20 comp rx=0 tx=0).stop 2024-06-10T00:34:35.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.795+0000 7f405c9f7700 1 -- 172.21.15.62:0/1889081164 shutdown_connections 2024-06-10T00:34:35.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.795+0000 7f405c9f7700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f40440a2d90 0x7f40440a5250 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.795+0000 7f405c9f7700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4058107870 0x7f4058198990 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.795+0000 7f405c9f7700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40581027a0 0x7f405819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.795+0000 7f405c9f7700 1 --2- 172.21.15.62:0/1889081164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4058103150 0x7f405819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:35.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.795+0000 7f405c9f7700 1 -- 172.21.15.62:0/1889081164 >> 172.21.15.62:0/1889081164 conn(0x7f4058079520 msgr2=0x7f40581004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:35.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.795+0000 7f405c9f7700 1 -- 172.21.15.62:0/1889081164 shutdown_connections 2024-06-10T00:34:35.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:35.795+0000 7f405c9f7700 1 -- 172.21.15.62:0/1889081164 wait complete. 2024-06-10T00:34:36.551 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:36 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2742096032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:36.584 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db --set-id 8081428b --release --await' 2024-06-10T00:34:36.584 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24464 quiesce db --set-id 8081428b --release --await 2024-06-10T00:34:36.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:36 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2742096032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:36.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:36 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2742096032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:37.412 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:37.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:37 smithi062 ceph-mon[32390]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T00:34:37.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:37 smithi138 ceph-mon[36549]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T00:34:37.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:37 smithi105 ceph-mon[36321]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T00:34:38.293 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T00:34:38.294 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T00:34:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.285+0000 7fbc9f8f7700 1 -- 172.21.15.62:0/4022693564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc981084f0 msgr2=0x7fbc981088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.285+0000 7fbc9f8f7700 1 --2- 172.21.15.62:0/4022693564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc981084f0 0x7fbc981088d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009990 tx=0x7fbc94009ca0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.286+0000 7fbc9f8f7700 1 -- 172.21.15.62:0/4022693564 shutdown_connections 2024-06-10T00:34:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.286+0000 7fbc9f8f7700 1 --2- 172.21.15.62:0/4022693564 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc981084f0 0x7fbc981088d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.286+0000 7fbc9f8f7700 1 --2- 172.21.15.62:0/4022693564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc98102f10 0x7fbc9810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.286+0000 7fbc9f8f7700 1 --2- 172.21.15.62:0/4022693564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc98102550 0x7fbc981029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.286+0000 7fbc9f8f7700 1 -- 172.21.15.62:0/4022693564 >> 172.21.15.62:0/4022693564 conn(0x7fbc98079650 msgr2=0x7fbc98079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.286+0000 7fbc9f8f7700 1 -- 172.21.15.62:0/4022693564 shutdown_connections 2024-06-10T00:34:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.286+0000 7fbc9f8f7700 1 -- 172.21.15.62:0/4022693564 wait complete. 2024-06-10T00:34:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.286+0000 7fbc9f8f7700 1 Processor -- start 2024-06-10T00:34:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.286+0000 7fbc9f8f7700 1 -- start start 2024-06-10T00:34:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.286+0000 7fbc9f8f7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc98102550 0x7fbc98074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.286+0000 7fbc9f8f7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc98102f10 0x7fbc98074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.286+0000 7fbc9f8f7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc981084f0 0x7fbc98075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9f8f7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981127a0 con 0x7fbc98102f10 2024-06-10T00:34:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9f8f7700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc98112620 con 0x7fbc981084f0 2024-06-10T00:34:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9f8f7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc98112920 con 0x7fbc98102550 2024-06-10T00:34:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9ce92700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc98102f10 0x7fbc98074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9ce92700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc98102f10 0x7fbc98074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37320/0 (socket says 172.21.15.62:37320) 2024-06-10T00:34:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9ce92700 1 -- 172.21.15.62:0/851299508 learned_addr learned my addr 172.21.15.62:0/851299508 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9de94700 1 --2- 172.21.15.62:0/851299508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc981084f0 0x7fbc98075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9d693700 1 --2- 172.21.15.62:0/851299508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc98102550 0x7fbc98074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9ce92700 1 -- 172.21.15.62:0/851299508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc98102550 msgr2=0x7fbc98074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9ce92700 1 --2- 172.21.15.62:0/851299508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc98102550 0x7fbc98074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9ce92700 1 -- 172.21.15.62:0/851299508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc981084f0 msgr2=0x7fbc98075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9ce92700 1 --2- 172.21.15.62:0/851299508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc981084f0 0x7fbc98075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9ce92700 1 -- 172.21.15.62:0/851299508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc981a7100 con 0x7fbc98102f10 2024-06-10T00:34:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9ce92700 1 --2- 172.21.15.62:0/851299508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc98102f10 0x7fbc98074cd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00c9b0 tx=0x7fbc8c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc8a7fc700 1 -- 172.21.15.62:0/851299508 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c007b00 con 0x7fbc98102f10 2024-06-10T00:34:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9f8f7700 1 -- 172.21.15.62:0/851299508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc981a73f0 con 0x7fbc98102f10 2024-06-10T00:34:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc8a7fc700 1 -- 172.21.15.62:0/851299508 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc8c00ce50 con 0x7fbc98102f10 2024-06-10T00:34:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc8a7fc700 1 -- 172.21.15.62:0/851299508 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c018660 con 0x7fbc98102f10 2024-06-10T00:34:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.287+0000 7fbc9f8f7700 1 -- 172.21.15.62:0/851299508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc981a7930 con 0x7fbc98102f10 2024-06-10T00:34:39.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.290+0000 7fbc8a7fc700 1 -- 172.21.15.62:0/851299508 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbc8c00f450 con 0x7fbc98102f10 2024-06-10T00:34:39.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.290+0000 7fbc8a7fc700 1 --2- 172.21.15.62:0/851299508 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbc840a2d90 0x7fbc840a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.291+0000 7fbc8a7fc700 1 -- 172.21.15.62:0/851299508 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fbc8c0cf730 con 0x7fbc98102f10 2024-06-10T00:34:39.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.291+0000 7fbc9d693700 1 --2- 172.21.15.62:0/851299508 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbc840a2d90 0x7fbc840a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.291+0000 7fbc9d693700 1 --2- 172.21.15.62:0/851299508 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbc840a2d90 0x7fbc840a5250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400b5c0 tx=0x7fbc94005fd0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.291+0000 7fbcaa17ab80 1 Processor -- start 2024-06-10T00:34:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.291+0000 7fbcaa17ab80 1 -- start start 2024-06-10T00:34:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbcaa17ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614500 0x55c9786148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbcaa17ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c978614e20 0x55c97860e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbcaa17ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c97860ec50 0x55c97860f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbcaa17ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c978501060 con 0x55c978614500 2024-06-10T00:34:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbcaa17ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c97855f950 con 0x55c97860ec50 2024-06-10T00:34:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbcaa17ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c9784c7560 con 0x55c978614e20 2024-06-10T00:34:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbc9d693700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614500 0x55c9786148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbc9d693700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614500 0x55c9786148e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37328/0 (socket says 172.21.15.62:37328) 2024-06-10T00:34:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbc9d693700 1 -- 172.21.15.62:0/3412461441 learned_addr learned my addr 172.21.15.62:0/3412461441 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbc9ce92700 1 --2- 172.21.15.62:0/3412461441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c978614e20 0x55c97860e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbc9de94700 1 --2- 172.21.15.62:0/3412461441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c97860ec50 0x55c97860f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbc9d693700 1 -- 172.21.15.62:0/3412461441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c978614e20 msgr2=0x55c97860e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbc9d693700 1 --2- 172.21.15.62:0/3412461441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c978614e20 0x55c97860e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbc9d693700 1 -- 172.21.15.62:0/3412461441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c97860ec50 msgr2=0x55c97860f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbc9d693700 1 --2- 172.21.15.62:0/3412461441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c97860ec50 0x55c97860f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbc9d693700 1 -- 172.21.15.62:0/3412461441 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9784ae9d0 con 0x55c978614500 2024-06-10T00:34:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.292+0000 7fbc9d693700 1 --2- 172.21.15.62:0/3412461441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614500 0x55c9786148e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400ba90 tx=0x7fbc94020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b83ce3666a0e3b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:39.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.293+0000 7fbc837fe700 1 -- 172.21.15.62:0/3412461441 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9400be50 con 0x55c978614500 2024-06-10T00:34:39.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.293+0000 7fbc837fe700 1 -- 172.21.15.62:0/3412461441 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9400fd50 con 0x55c978614500 2024-06-10T00:34:39.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.293+0000 7fbc837fe700 1 -- 172.21.15.62:0/3412461441 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94024e20 con 0x55c978614500 2024-06-10T00:34:39.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.293+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/3412461441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614500 msgr2=0x55c9786148e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:39.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.293+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/3412461441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614500 0x55c9786148e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400ba90 tx=0x7fbc94020000 comp rx=0 tx=0).stop 2024-06-10T00:34:39.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.293+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/3412461441 shutdown_connections 2024-06-10T00:34:39.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.293+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/3412461441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614500 0x55c9786148e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.293+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/3412461441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c97860ec50 0x55c97860f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.293+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/3412461441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c978614e20 0x55c97860e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.293+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/3412461441 >> 172.21.15.62:0/3412461441 conn(0x55c97853b180 msgr2=0x55c978609a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:39.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.293+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/3412461441 shutdown_connections 2024-06-10T00:34:39.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.293+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/3412461441 wait complete. 2024-06-10T00:34:39.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.294+0000 7fbcaa17ab80 1 -- start start 2024-06-10T00:34:39.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.294+0000 7fbcaa17ab80 1 Processor -- start 2024-06-10T00:34:39.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.294+0000 7fbcaa17ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c97860ec50 0x55c97861abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.294+0000 7fbcaa17ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614500 0x55c978617eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.294+0000 7fbcaa17ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614e20 0x55c9786183f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.294+0000 7fbcaa17ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c9785000c0 con 0x55c978614e20 2024-06-10T00:34:39.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.294+0000 7fbcaa17ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c97855b910 con 0x55c978614500 2024-06-10T00:34:39.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.294+0000 7fbcaa17ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c97861b740 con 0x55c97860ec50 2024-06-10T00:34:39.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.294+0000 7fbcaa17ab80 10 client.0 _ll_register_callbacks cb 0x55c9784f56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:39.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc9de94700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614e20 0x55c9786183f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc9de94700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614e20 0x55c9786183f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37334/0 (socket says 172.21.15.62:37334) 2024-06-10T00:34:39.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc9de94700 1 -- 172.21.15.62:0/790555580 learned_addr learned my addr 172.21.15.62:0/790555580 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:39.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc9d693700 1 --2- 172.21.15.62:0/790555580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c97860ec50 0x55c97861abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc9ce92700 1 --2- 172.21.15.62:0/790555580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614500 0x55c978617eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc9de94700 1 -- 172.21.15.62:0/790555580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c97860ec50 msgr2=0x55c97861abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:39.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc9de94700 1 --2- 172.21.15.62:0/790555580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c97860ec50 0x55c97861abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc9de94700 1 -- 172.21.15.62:0/790555580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614500 msgr2=0x55c978617eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:39.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc9de94700 1 --2- 172.21.15.62:0/790555580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614500 0x55c978617eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc9de94700 1 -- 172.21.15.62:0/790555580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c978525850 con 0x55c978614e20 2024-06-10T00:34:39.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc9de94700 1 --2- 172.21.15.62:0/790555580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614e20 0x55c9786183f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbc90004070 tx=0x7fbc90004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8ec8a26ff2f18f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:39.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:39.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc80ff9700 1 -- 172.21.15.62:0/790555580 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9001a020 con 0x55c978614e20 2024-06-10T00:34:39.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc80ff9700 1 -- 172.21.15.62:0/790555580 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fbc900079d0 con 0x55c978614e20 2024-06-10T00:34:39.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/790555580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc9400fd50 con 0x55c978614e20 2024-06-10T00:34:39.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbc80ff9700 1 -- 172.21.15.62:0/790555580 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbc90025050 con 0x55c978614e20 2024-06-10T00:34:39.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbcaa17ab80 10 client.15939 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:39.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.295+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/790555580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c978625f50 con 0x55c978614e20 2024-06-10T00:34:39.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.296+0000 7fbc80ff9700 1 -- 172.21.15.62:0/790555580 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9400fd50 con 0x55c978614e20 2024-06-10T00:34:39.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.296+0000 7fbc80ff9700 1 -- 172.21.15.62:0/790555580 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fbc900153c0 con 0x55c978614e20 2024-06-10T00:34:39.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.296+0000 7fbcaa17ab80 10 client.15939 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:39.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.296+0000 7fbcaa17ab80 10 client.15939 resolve_mds: validated gid 24464 aka daemon mds.b 2024-06-10T00:34:39.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.296+0000 7fbcaa17ab80 20 client.15939 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:39.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.296+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/790555580 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c9786285e0 0x55c97862a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.296+0000 7fbcaa17ab80 4 client.15939 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:34:39.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.296+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/790555580 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c97856ce50 con 0x55c9786285e0 2024-06-10T00:34:39.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.296+0000 7fbc9d693700 1 --2- 172.21.15.62:0/790555580 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c9786285e0 0x55c97862a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.297+0000 7fbc80ff9700 10 client.15939 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:39.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.297+0000 7fbc9d693700 1 --2- 172.21.15.62:0/790555580 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c9786285e0 0x55c97862a9d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b19b6a65de859db server_cookie=77cad7cfed13953f in_seq=0 out_seq=0 2024-06-10T00:34:39.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.297+0000 7fbc80ff9700 1 -- 172.21.15.62:0/790555580 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c97856ce50 con 0x55c9786285e0 2024-06-10T00:34:39.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.298+0000 7fbc80ff9700 10 client.15939 handle_command_reply: tid=0 2024-06-10T00:34:39.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.298+0000 7fbcaa17ab80 1 client.15939 shutdown 2024-06-10T00:34:39.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.298+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/790555580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614e20 msgr2=0x55c9786183f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:39.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.298+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/790555580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614e20 0x55c9786183f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbc90004070 tx=0x7fbc90004380 comp rx=0 tx=0).stop 2024-06-10T00:34:39.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.298+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/790555580 shutdown_connections 2024-06-10T00:34:39.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.298+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/790555580 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c9786285e0 0x55c97862a9d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.298+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/790555580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614e20 0x55c9786183f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.298+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/790555580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c97860ec50 0x55c97861abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.298+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/790555580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614500 0x55c978617eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.298+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/790555580 >> 172.21.15.62:0/790555580 conn(0x55c97853b180 msgr2=0x55c978608320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:39.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.298+0000 7fbc80ff9700 0 client.15939 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:39.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.298+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/790555580 shutdown_connections 2024-06-10T00:34:39.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.298+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/790555580 wait complete. 2024-06-10T00:34:39.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.298+0000 7fbcaa17ab80 20 client.15939 trim_cache size 0 max 16384 2024-06-10T00:34:39.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.319+0000 7fbcaa17ab80 1 Processor -- start 2024-06-10T00:34:39.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.319+0000 7fbcaa17ab80 1 -- start start 2024-06-10T00:34:39.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.319+0000 7fbcaa17ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c97860ec50 0x55c97860e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.319+0000 7fbcaa17ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614500 0x55c978612650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.319+0000 7fbcaa17ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614e20 0x55c9786184e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.319+0000 7fbcaa17ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981b0780 con 0x55c978614e20 2024-06-10T00:34:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.319+0000 7fbcaa17ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981b22c0 con 0x55c978614500 2024-06-10T00:34:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.319+0000 7fbcaa17ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981b0e50 con 0x55c97860ec50 2024-06-10T00:34:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.319+0000 7fbc9de94700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614e20 0x55c9786184e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.319+0000 7fbc9de94700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614e20 0x55c9786184e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37342/0 (socket says 172.21.15.62:37342) 2024-06-10T00:34:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.319+0000 7fbc9de94700 1 -- 172.21.15.62:0/489133902 learned_addr learned my addr 172.21.15.62:0/489133902 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.319+0000 7fbc9d693700 1 --2- 172.21.15.62:0/489133902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c97860ec50 0x55c97860e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.320+0000 7fbc9ce92700 1 --2- 172.21.15.62:0/489133902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614500 0x55c978612650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.320+0000 7fbc9de94700 1 -- 172.21.15.62:0/489133902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c97860ec50 msgr2=0x55c97860e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.320+0000 7fbc9de94700 1 --2- 172.21.15.62:0/489133902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c97860ec50 0x55c97860e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.320+0000 7fbc9de94700 1 -- 172.21.15.62:0/489133902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614500 msgr2=0x55c978612650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.320+0000 7fbc9de94700 1 --2- 172.21.15.62:0/489133902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614500 0x55c978612650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.320+0000 7fbc9de94700 1 -- 172.21.15.62:0/489133902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c97862ad10 con 0x55c978614e20 2024-06-10T00:34:39.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.320+0000 7fbc9de94700 1 --2- 172.21.15.62:0/489133902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614e20 0x55c9786184e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbc900095c0 tx=0x7fbc900095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3553815d26cb10 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:39.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.320+0000 7fbc82ffd700 1 -- 172.21.15.62:0/489133902 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9001a070 con 0x55c978614e20 2024-06-10T00:34:39.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.320+0000 7fbc82ffd700 1 -- 172.21.15.62:0/489133902 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc900153c0 con 0x55c978614e20 2024-06-10T00:34:39.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.320+0000 7fbc82ffd700 1 -- 172.21.15.62:0/489133902 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90007a30 con 0x55c978614e20 2024-06-10T00:34:39.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/489133902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614e20 msgr2=0x55c9786184e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:39.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/489133902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614e20 0x55c9786184e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbc900095c0 tx=0x7fbc900095f0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/489133902 shutdown_connections 2024-06-10T00:34:39.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/489133902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c978614e20 0x55c9786184e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/489133902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c97860ec50 0x55c97860e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/489133902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614500 0x55c978612650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/489133902 >> 172.21.15.62:0/489133902 conn(0x55c97853b180 msgr2=0x55c9786196e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:39.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/489133902 shutdown_connections 2024-06-10T00:34:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/489133902 wait complete. 2024-06-10T00:34:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 -- start start 2024-06-10T00:34:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 Processor -- start 2024-06-10T00:34:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c97860ec50 0x55c978620da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c978614500 0x55c9786212e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614e20 0x55c97862f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981b1bf0 con 0x55c97860ec50 2024-06-10T00:34:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981b1520 con 0x55c978614e20 2024-06-10T00:34:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.321+0000 7fbcaa17ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc981ae720 con 0x55c978614500 2024-06-10T00:34:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbcaa17ab80 10 client.0 _ll_register_callbacks cb 0x55c9784f56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc9d693700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c97860ec50 0x55c978620da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc9d693700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c97860ec50 0x55c978620da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37344/0 (socket says 172.21.15.62:37344) 2024-06-10T00:34:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc9d693700 1 -- 172.21.15.62:0/227661544 learned_addr learned my addr 172.21.15.62:0/227661544 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc9ce92700 1 --2- 172.21.15.62:0/227661544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c978614500 0x55c9786212e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc9de94700 1 --2- 172.21.15.62:0/227661544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614e20 0x55c97862f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc9d693700 1 -- 172.21.15.62:0/227661544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c978614500 msgr2=0x55c9786212e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc9d693700 1 --2- 172.21.15.62:0/227661544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c978614500 0x55c9786212e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc9d693700 1 -- 172.21.15.62:0/227661544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614e20 msgr2=0x55c97862f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc9d693700 1 --2- 172.21.15.62:0/227661544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614e20 0x55c97862f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc9d693700 1 -- 172.21.15.62:0/227661544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c978630230 con 0x55c97860ec50 2024-06-10T00:34:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc9de94700 1 --2- 172.21.15.62:0/227661544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614e20 0x55c97862f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc9d693700 1 --2- 172.21.15.62:0/227661544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c97860ec50 0x55c978620da0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbc94031040 tx=0x7fbc94009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0148bb4a971259a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc80ff9700 1 -- 172.21.15.62:0/227661544 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9400f7f0 con 0x55c97860ec50 2024-06-10T00:34:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbc80ff9700 1 -- 172.21.15.62:0/227661544 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fbc9400f990 con 0x55c97860ec50 2024-06-10T00:34:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.322+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/227661544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc900153c0 con 0x55c97860ec50 2024-06-10T00:34:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.323+0000 7fbc80ff9700 1 -- 172.21.15.62:0/227661544 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbc9404a050 con 0x55c97860ec50 2024-06-10T00:34:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.323+0000 7fbcaa17ab80 10 client.15945 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.323+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/227661544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c978692ee0 con 0x55c97860ec50 2024-06-10T00:34:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.323+0000 7fbc80ff9700 1 -- 172.21.15.62:0/227661544 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc900153c0 con 0x55c97860ec50 2024-06-10T00:34:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.323+0000 7fbc80ff9700 1 -- 172.21.15.62:0/227661544 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fbc94039640 con 0x55c97860ec50 2024-06-10T00:34:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.323+0000 7fbcaa17ab80 10 client.15945 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.323+0000 7fbcaa17ab80 10 client.15945 resolve_mds: validated gid 24464 aka daemon mds.b 2024-06-10T00:34:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.323+0000 7fbcaa17ab80 20 client.15945 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.323+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/227661544 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c978693170 0x55c978695560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.323+0000 7fbcaa17ab80 4 client.15945 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8081428b", "release": true, "await": true}] 2024-06-10T00:34:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.323+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/227661544 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8081428b", "release": true, "await": true}) -- 0x55c9785673d0 con 0x55c978693170 2024-06-10T00:34:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.323+0000 7fbc9ce92700 1 --2- 172.21.15.62:0/227661544 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c978693170 0x55c978695560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:39.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.324+0000 7fbc80ff9700 10 client.15945 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:39.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.324+0000 7fbc9ce92700 1 --2- 172.21.15.62:0/227661544 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c978693170 0x55c978695560 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a14712d7fd35c0e server_cookie=a5efe70c3a96a4de in_seq=0 out_seq=0 2024-06-10T00:34:39.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.329+0000 7fbc80ff9700 1 -- 172.21.15.62:0/227661544 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55c9785673d0 con 0x55c978693170 2024-06-10T00:34:39.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.329+0000 7fbc80ff9700 10 client.15945 handle_command_reply: tid=0 2024-06-10T00:34:39.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.329+0000 7fbcaa17ab80 1 client.15945 shutdown 2024-06-10T00:34:39.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.329+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/227661544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c97860ec50 msgr2=0x55c978620da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:39.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.329+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/227661544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c97860ec50 0x55c978620da0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbc94031040 tx=0x7fbc94009ee0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.329+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/227661544 shutdown_connections 2024-06-10T00:34:39.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.329+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/227661544 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c978693170 0x55c978695560 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.329+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/227661544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c97860ec50 0x55c978620da0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.329+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/227661544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c978614e20 0x55c97862f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.329+0000 7fbc80ff9700 0 client.15945 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:39.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.329+0000 7fbcaa17ab80 1 --2- 172.21.15.62:0/227661544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c978614500 0x55c9786212e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.330+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/227661544 >> 172.21.15.62:0/227661544 conn(0x55c97853b180 msgr2=0x55c978619f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:39.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.330+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/227661544 shutdown_connections 2024-06-10T00:34:39.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.330+0000 7fbcaa17ab80 1 -- 172.21.15.62:0/227661544 wait complete. 2024-06-10T00:34:39.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.330+0000 7fbcaa17ab80 20 client.15945 trim_cache size 0 max 16384 2024-06-10T00:34:39.331 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:34:39.331 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 30, 2024-06-10T00:34:39.331 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24464, 2024-06-10T00:34:39.331 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 4, 2024-06-10T00:34:39.332 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-06-10T00:34:39.332 INFO:teuthology.orchestra.run.smithi062.stdout: "8081428b": { 2024-06-10T00:34:39.332 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 4, 2024-06-10T00:34:39.332 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-06-10T00:34:39.332 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-06-10T00:34:39.332 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-06-10T00:34:39.332 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-06-10T00:34:39.332 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:34:39.332 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-06-10T00:34:39.332 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 150.0, 2024-06-10T00:34:39.332 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-06-10T00:34:39.332 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-06-10T00:34:39.333 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-06-10T00:34:39.333 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-06-10T00:34:39.333 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-06-10T00:34:39.333 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-06-10T00:34:39.333 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:39.333 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:39.333 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:39.333 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:39.333 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:39.333 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:34:39.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.330+0000 7fbc9f8f7700 1 -- 172.21.15.62:0/851299508 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbc840a2d90 msgr2=0x7fbc840a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:39.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.330+0000 7fbc9f8f7700 1 --2- 172.21.15.62:0/851299508 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbc840a2d90 0x7fbc840a5250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400b5c0 tx=0x7fbc94005fd0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.330+0000 7fbc9f8f7700 1 -- 172.21.15.62:0/851299508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc98102f10 msgr2=0x7fbc98074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:39.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.330+0000 7fbc9f8f7700 1 --2- 172.21.15.62:0/851299508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc98102f10 0x7fbc98074cd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00c9b0 tx=0x7fbc8c00ccc0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.331+0000 7fbc9f8f7700 1 -- 172.21.15.62:0/851299508 shutdown_connections 2024-06-10T00:34:39.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.331+0000 7fbc9f8f7700 1 --2- 172.21.15.62:0/851299508 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbc840a2d90 0x7fbc840a5250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.331+0000 7fbc9f8f7700 1 --2- 172.21.15.62:0/851299508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc981084f0 0x7fbc98075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.331+0000 7fbc9f8f7700 1 --2- 172.21.15.62:0/851299508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc98102550 0x7fbc98074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.331+0000 7fbc9f8f7700 1 --2- 172.21.15.62:0/851299508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc98102f10 0x7fbc98074cd0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:39.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.331+0000 7fbc9f8f7700 1 -- 172.21.15.62:0/851299508 >> 172.21.15.62:0/851299508 conn(0x7fbc98079650 msgr2=0x7fbc98100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:39.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.331+0000 7fbc9f8f7700 1 -- 172.21.15.62:0/851299508 shutdown_connections 2024-06-10T00:34:39.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:39.331+0000 7fbc9f8f7700 1 -- 172.21.15.62:0/851299508 wait complete. 2024-06-10T00:34:39.335 INFO:tasks.workunit.client.0.smithi062.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-06-10T00:34:39.336 INFO:tasks.workunit.client.0.smithi062.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-06-10T00:34:39.336 INFO:tasks.workunit.client.0.smithi062.stderr:+ trap -- retry EXIT 2024-06-10T00:34:39.336 INFO:tasks.workunit.client.0.smithi062.stderr:+ rm -rf ffsb 2024-06-10T00:34:39.336 INFO:tasks.workunit.client.0.smithi062.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-06-10T00:34:39.339 INFO:tasks.workunit.client.0.smithi062.stderr:Cloning into 'ffsb'... 2024-06-10T00:34:39.576 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:39 smithi062 ceph-mon[32390]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:34:39.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:39 smithi138 ceph-mon[36549]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:34:39.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:39 smithi105 ceph-mon[36321]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:34:40.432 INFO:tasks.workunit.client.0.smithi062.stderr:+ trap - EXIT 2024-06-10T00:34:40.432 INFO:tasks.workunit.client.0.smithi062.stderr:+ cd ffsb 2024-06-10T00:34:40.432 INFO:tasks.workunit.client.0.smithi062.stderr:+ ./configure 2024-06-10T00:34:40.527 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8081428b', total seconds elapsed: 39.7 2024-06-10T00:34:40.527 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-10T00:34:40.527 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:34:40.587 INFO:tasks.workunit.client.0.smithi062.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-06-10T00:34:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:41.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.510+0000 7fbacfa20700 1 -- 172.21.15.62:0/2552029497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac8101d40 msgr2=0x7fbac810e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:41.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.510+0000 7fbacfa20700 1 --2- 172.21.15.62:0/2552029497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac8101d40 0x7fbac810e230 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbac000b0a0 tx=0x7fbac000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.510+0000 7fbacfa20700 1 -- 172.21.15.62:0/2552029497 shutdown_connections 2024-06-10T00:34:41.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.510+0000 7fbacfa20700 1 --2- 172.21.15.62:0/2552029497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac8101d40 0x7fbac810e230 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.510+0000 7fbacfa20700 1 --2- 172.21.15.62:0/2552029497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbac81073f0 0x7fbac81077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.510+0000 7fbacfa20700 1 --2- 172.21.15.62:0/2552029497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbac8101380 0x7fbac8101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.510+0000 7fbacfa20700 1 -- 172.21.15.62:0/2552029497 >> 172.21.15.62:0/2552029497 conn(0x7fbac8078300 msgr2=0x7fbac8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:41.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.510+0000 7fbacfa20700 1 -- 172.21.15.62:0/2552029497 shutdown_connections 2024-06-10T00:34:41.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.510+0000 7fbacfa20700 1 -- 172.21.15.62:0/2552029497 wait complete. 2024-06-10T00:34:41.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.511+0000 7fbacfa20700 1 Processor -- start 2024-06-10T00:34:41.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.511+0000 7fbacfa20700 1 -- start start 2024-06-10T00:34:41.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.511+0000 7fbacfa20700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbac8101380 0x7fbac819d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.511+0000 7fbacfa20700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac8101d40 0x7fbac819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.511+0000 7fbacfa20700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbac81073f0 0x7fbac81979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.511+0000 7fbacfa20700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbac8111660 con 0x7fbac8101d40 2024-06-10T00:34:41.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.511+0000 7fbacfa20700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81114e0 con 0x7fbac8101380 2024-06-10T00:34:41.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.511+0000 7fbacfa20700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81117e0 con 0x7fbac81073f0 2024-06-10T00:34:41.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.512+0000 7fbaccfbb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac8101d40 0x7fbac819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.512+0000 7fbaccfbb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac8101d40 0x7fbac819dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37352/0 (socket says 172.21.15.62:37352) 2024-06-10T00:34:41.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.512+0000 7fbaccfbb700 1 -- 172.21.15.62:0/1252810668 learned_addr learned my addr 172.21.15.62:0/1252810668 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:41.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.512+0000 7fbacd7bc700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbac8101380 0x7fbac819d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.512+0000 7fbacdfbd700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbac81073f0 0x7fbac81979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.512+0000 7fbaccfbb700 1 -- 172.21.15.62:0/1252810668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbac81073f0 msgr2=0x7fbac81979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:41.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.512+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbac81073f0 0x7fbac81979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.512+0000 7fbaccfbb700 1 -- 172.21.15.62:0/1252810668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbac8101380 msgr2=0x7fbac819d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:41.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.512+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbac8101380 0x7fbac819d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.512+0000 7fbaccfbb700 1 -- 172.21.15.62:0/1252810668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac8198120 con 0x7fbac8101d40 2024-06-10T00:34:41.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.512+0000 7fbacdfbd700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbac81073f0 0x7fbac81979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:41.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.512+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac8101d40 0x7fbac819dd40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009970 tx=0x7fbab8009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:41.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.512+0000 7fbabe7fc700 1 -- 172.21.15.62:0/1252810668 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab801c070 con 0x7fbac8101d40 2024-06-10T00:34:41.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.513+0000 7fbabe7fc700 1 -- 172.21.15.62:0/1252810668 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbab800d260 con 0x7fbac8101d40 2024-06-10T00:34:41.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.513+0000 7fbacfa20700 1 -- 172.21.15.62:0/1252810668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac8198410 con 0x7fbac8101d40 2024-06-10T00:34:41.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.513+0000 7fbabe7fc700 1 -- 172.21.15.62:0/1252810668 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab8017ad0 con 0x7fbac8101d40 2024-06-10T00:34:41.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.513+0000 7fbacfa20700 1 -- 172.21.15.62:0/1252810668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac806aa00 con 0x7fbac8101d40 2024-06-10T00:34:41.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.515+0000 7fbabe7fc700 1 -- 172.21.15.62:0/1252810668 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbab801f070 con 0x7fbac8101d40 2024-06-10T00:34:41.517 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:41 smithi062 ceph-mon[32390]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:34:41.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.516+0000 7fbabe7fc700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbab40a2e10 0x7fbab40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.516+0000 7fbabe7fc700 1 -- 172.21.15.62:0/1252810668 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fbab80d08a0 con 0x7fbac8101d40 2024-06-10T00:34:41.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.516+0000 7fbacd7bc700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbab40a2e10 0x7fbab40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.517+0000 7fbacd7bc700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbab40a2e10 0x7fbab40a52d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbac8198b00 tx=0x7fbac400b410 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:41.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.517+0000 7fbada2a3b80 1 Processor -- start 2024-06-10T00:34:41.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.517+0000 7fbada2a3b80 1 -- start start 2024-06-10T00:34:41.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.518+0000 7fbada2a3b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f050 0x563a0dc1f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.518+0000 7fbada2a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f970 0x563a0dc23770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.518+0000 7fbada2a3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc23cb0 0x563a0dc24110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.518+0000 7fbaccfbb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f970 0x563a0dc23770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.518+0000 7fbaccfbb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f970 0x563a0dc23770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37366/0 (socket says 172.21.15.62:37366) 2024-06-10T00:34:41.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.518+0000 7fbacd7bc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f050 0x563a0dc1f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.518+0000 7fbacd7bc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f050 0x563a0dc1f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:35048/0 (socket says 172.21.15.62:35048) 2024-06-10T00:34:41.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.518+0000 7fbaccfbb700 1 -- 172.21.15.62:0/3400900470 learned_addr learned my addr 172.21.15.62:0/3400900470 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:41.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.518+0000 7fbada2a3b80 1 -- 172.21.15.62:0/3400900470 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563a0db6c890 con 0x563a0dc1f970 2024-06-10T00:34:41.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.519+0000 7fbada2a3b80 1 -- 172.21.15.62:0/3400900470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563a0db147a0 con 0x563a0dc1f050 2024-06-10T00:34:41.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.519+0000 7fbada2a3b80 1 -- 172.21.15.62:0/3400900470 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a0db68850 con 0x563a0dc23cb0 2024-06-10T00:34:41.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.519+0000 7fbacdfbd700 1 --2- 172.21.15.62:0/3400900470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc23cb0 0x563a0dc24110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.519+0000 7fbaccfbb700 1 -- 172.21.15.62:0/3400900470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc23cb0 msgr2=0x563a0dc24110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:41.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.519+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/3400900470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc23cb0 0x563a0dc24110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.519+0000 7fbaccfbb700 1 -- 172.21.15.62:0/3400900470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f050 msgr2=0x563a0dc1f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:41.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.519+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/3400900470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f050 0x563a0dc1f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.519+0000 7fbaccfbb700 1 -- 172.21.15.62:0/3400900470 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a0db0afd0 con 0x563a0dc1f970 2024-06-10T00:34:41.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.519+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/3400900470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f970 0x563a0dc23770 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbab80993e0 tx=0x7fbab809c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761a627fa40978be server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:41.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.519+0000 7fbab37fe700 1 -- 172.21.15.62:0/3400900470 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab809ca10 con 0x563a0dc1f970 2024-06-10T00:34:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.519+0000 7fbab37fe700 1 -- 172.21.15.62:0/3400900470 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbab80a0dc0 con 0x563a0dc1f970 2024-06-10T00:34:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.519+0000 7fbab37fe700 1 -- 172.21.15.62:0/3400900470 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab80a5450 con 0x563a0dc1f970 2024-06-10T00:34:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.520+0000 7fbada2a3b80 1 -- 172.21.15.62:0/3400900470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f970 msgr2=0x563a0dc23770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.520+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/3400900470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f970 0x563a0dc23770 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbab80993e0 tx=0x7fbab809c370 comp rx=0 tx=0).stop 2024-06-10T00:34:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.521+0000 7fbada2a3b80 1 -- 172.21.15.62:0/3400900470 shutdown_connections 2024-06-10T00:34:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.521+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/3400900470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc23cb0 0x563a0dc24110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.521+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/3400900470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f050 0x563a0dc1f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.521+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/3400900470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f970 0x563a0dc23770 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.521+0000 7fbada2a3b80 1 -- 172.21.15.62:0/3400900470 >> 172.21.15.62:0/3400900470 conn(0x563a0db45d40 msgr2=0x563a0dc158d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.521+0000 7fbada2a3b80 1 -- 172.21.15.62:0/3400900470 shutdown_connections 2024-06-10T00:34:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.521+0000 7fbada2a3b80 1 -- 172.21.15.62:0/3400900470 wait complete. 2024-06-10T00:34:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.521+0000 7fbada2a3b80 1 -- start start 2024-06-10T00:34:41.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.521+0000 7fbada2a3b80 1 Processor -- start 2024-06-10T00:34:41.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.522+0000 7fbada2a3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc1f050 0x563a0dc2b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.522+0000 7fbada2a3b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f970 0x563a0dc285f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.522+0000 7fbada2a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc23cb0 0x563a0dc28b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.522+0000 7fbada2a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563a0dc2be50 con 0x563a0dc23cb0 2024-06-10T00:34:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.522+0000 7fbada2a3b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563a0db0cc70 con 0x563a0dc1f970 2024-06-10T00:34:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.522+0000 7fbada2a3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a0dc2bfd0 con 0x563a0dc1f050 2024-06-10T00:34:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.522+0000 7fbada2a3b80 10 client.0 _ll_register_callbacks cb 0x563a0db069a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.522+0000 7fbaccfbb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f970 0x563a0dc285f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.522+0000 7fbacdfbd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc23cb0 0x563a0dc28b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.522+0000 7fbaccfbb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f970 0x563a0dc285f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:35052/0 (socket says 172.21.15.62:35052) 2024-06-10T00:34:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.522+0000 7fbaccfbb700 1 -- 172.21.15.62:0/1882010200 learned_addr learned my addr 172.21.15.62:0/1882010200 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.522+0000 7fbacdfbd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc23cb0 0x563a0dc28b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37378/0 (socket says 172.21.15.62:37378) 2024-06-10T00:34:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.522+0000 7fbacd7bc700 1 --2- 172.21.15.62:0/1882010200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc1f050 0x563a0dc2b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.523+0000 7fbacd7bc700 1 -- 172.21.15.62:0/1882010200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f970 msgr2=0x563a0dc285f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:41.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.523+0000 7fbacd7bc700 1 --2- 172.21.15.62:0/1882010200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f970 0x563a0dc285f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.523+0000 7fbacd7bc700 1 -- 172.21.15.62:0/1882010200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc23cb0 msgr2=0x563a0dc28b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.523+0000 7fbacd7bc700 1 --2- 172.21.15.62:0/1882010200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc23cb0 0x563a0dc28b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.523+0000 7fbacd7bc700 1 -- 172.21.15.62:0/1882010200 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a0dabb950 con 0x563a0dc1f050 2024-06-10T00:34:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.523+0000 7fbacdfbd700 1 --2- 172.21.15.62:0/1882010200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc23cb0 0x563a0dc28b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:34:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.523+0000 7fbacd7bc700 1 --2- 172.21.15.62:0/1882010200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc1f050 0x563a0dc2b2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x563a0dc1fea0 tx=0x7fbac40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a24ecfd93f26d367 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.523+0000 7fbab0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:34:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.523+0000 7fbab0ff9700 1 -- 172.21.15.62:0/1882010200 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4028470 con 0x563a0dc1f050 2024-06-10T00:34:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.523+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1882010200 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbab80a0dc0 con 0x563a0dc1f050 2024-06-10T00:34:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.523+0000 7fbab0ff9700 1 -- 172.21.15.62:0/1882010200 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fbac4028610 con 0x563a0dc1f050 2024-06-10T00:34:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.524+0000 7fbab0ff9700 1 -- 172.21.15.62:0/1882010200 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbac4031050 con 0x563a0dc1f050 2024-06-10T00:34:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.524+0000 7fbada2a3b80 10 client.25519 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.524+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1882010200 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a0dc36690 con 0x563a0dc1f050 2024-06-10T00:34:41.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.524+0000 7fbab0ff9700 1 -- 172.21.15.62:0/1882010200 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbab80a0dc0 con 0x563a0dc1f050 2024-06-10T00:34:41.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.524+0000 7fbab0ff9700 1 -- 172.21.15.62:0/1882010200 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fbac401f800 con 0x563a0dc1f050 2024-06-10T00:34:41.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.524+0000 7fbada2a3b80 10 client.25519 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:41.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.524+0000 7fbada2a3b80 10 client.25519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:34:41.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.524+0000 7fbada2a3b80 20 client.25519 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:41.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.524+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/1882010200 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563a0dc38cd0 0x563a0dc3b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.524+0000 7fbada2a3b80 4 client.25519 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:34:41.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.524+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1882010200 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a0db79d90 con 0x563a0dc38cd0 2024-06-10T00:34:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.525+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/1882010200 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563a0dc38cd0 0x563a0dc3b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.525+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/1882010200 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563a0dc38cd0 0x563a0dc3b100 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361b5d9724ce0d72 server_cookie=416ba6ea7f8988fd in_seq=0 out_seq=0 2024-06-10T00:34:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.525+0000 7fbab0ff9700 10 client.25519 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbab0ff9700 1 -- 172.21.15.62:0/1882010200 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563a0db79d90 con 0x563a0dc38cd0 2024-06-10T00:34:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbab0ff9700 10 client.25519 handle_command_reply: tid=0 2024-06-10T00:34:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbada2a3b80 1 client.25519 shutdown 2024-06-10T00:34:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1882010200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc1f050 msgr2=0x563a0dc2b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/1882010200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc1f050 0x563a0dc2b2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x563a0dc1fea0 tx=0x7fbac40047c0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1882010200 shutdown_connections 2024-06-10T00:34:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/1882010200 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563a0dc38cd0 0x563a0dc3b100 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/1882010200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc23cb0 0x563a0dc28b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/1882010200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc1f050 0x563a0dc2b2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/1882010200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f970 0x563a0dc285f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1882010200 >> 172.21.15.62:0/1882010200 conn(0x563a0db45d40 msgr2=0x563a0dc19a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbab0ff9700 0 client.25519 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1882010200 shutdown_connections 2024-06-10T00:34:41.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1882010200 wait complete. 2024-06-10T00:34:41.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.526+0000 7fbada2a3b80 20 client.25519 trim_cache size 0 max 16384 2024-06-10T00:34:41.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.548+0000 7fbada2a3b80 1 Processor -- start 2024-06-10T00:34:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.548+0000 7fbada2a3b80 1 -- start start 2024-06-10T00:34:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.548+0000 7fbada2a3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc1f050 0x563a0dc350f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.548+0000 7fbada2a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f970 0x563a0dc40a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.548+0000 7fbada2a3b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc23cb0 0x563a0dc40f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.548+0000 7fbada2a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81ac4a0 con 0x563a0dc1f970 2024-06-10T00:34:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.548+0000 7fbada2a3b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81adfe0 con 0x563a0dc23cb0 2024-06-10T00:34:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.548+0000 7fbada2a3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81acb70 con 0x563a0dc1f050 2024-06-10T00:34:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.548+0000 7fbaccfbb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f970 0x563a0dc40a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.548+0000 7fbaccfbb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f970 0x563a0dc40a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37382/0 (socket says 172.21.15.62:37382) 2024-06-10T00:34:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.548+0000 7fbaccfbb700 1 -- 172.21.15.62:0/549974157 learned_addr learned my addr 172.21.15.62:0/549974157 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:41.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.548+0000 7fbacd7bc700 1 --2- 172.21.15.62:0/549974157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc1f050 0x563a0dc350f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.548+0000 7fbacdfbd700 1 --2- 172.21.15.62:0/549974157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc23cb0 0x563a0dc40f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.549+0000 7fbaccfbb700 1 -- 172.21.15.62:0/549974157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc1f050 msgr2=0x563a0dc350f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:41.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.549+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/549974157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc1f050 0x563a0dc350f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.549+0000 7fbaccfbb700 1 -- 172.21.15.62:0/549974157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc23cb0 msgr2=0x563a0dc40f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:41.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.549+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/549974157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc23cb0 0x563a0dc40f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.549+0000 7fbaccfbb700 1 -- 172.21.15.62:0/549974157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a0dc26b60 con 0x563a0dc1f970 2024-06-10T00:34:41.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.549+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/549974157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f970 0x563a0dc40a10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbab809acc0 tx=0x7fbab809d250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07d70f5b5141048 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:41.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.549+0000 7fbab2ffd700 1 -- 172.21.15.62:0/549974157 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab809cc70 con 0x563a0dc1f970 2024-06-10T00:34:41.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.549+0000 7fbab2ffd700 1 -- 172.21.15.62:0/549974157 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbab809ce10 con 0x563a0dc1f970 2024-06-10T00:34:41.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.549+0000 7fbab2ffd700 1 -- 172.21.15.62:0/549974157 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab809c0c0 con 0x563a0dc1f970 2024-06-10T00:34:41.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.550+0000 7fbada2a3b80 1 -- 172.21.15.62:0/549974157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f970 msgr2=0x563a0dc40a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:41.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.550+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/549974157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f970 0x563a0dc40a10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbab809acc0 tx=0x7fbab809d250 comp rx=0 tx=0).stop 2024-06-10T00:34:41.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.550+0000 7fbada2a3b80 1 -- 172.21.15.62:0/549974157 shutdown_connections 2024-06-10T00:34:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.550+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/549974157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc23cb0 0x563a0dc40f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.550+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/549974157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc1f050 0x563a0dc350f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.550+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/549974157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f970 0x563a0dc40a10 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.550+0000 7fbada2a3b80 1 -- 172.21.15.62:0/549974157 >> 172.21.15.62:0/549974157 conn(0x563a0db45d40 msgr2=0x563a0dc234e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.550+0000 7fbada2a3b80 1 -- 172.21.15.62:0/549974157 shutdown_connections 2024-06-10T00:34:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.550+0000 7fbada2a3b80 1 -- 172.21.15.62:0/549974157 wait complete. 2024-06-10T00:34:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.550+0000 7fbada2a3b80 1 -- start start 2024-06-10T00:34:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.550+0000 7fbada2a3b80 1 Processor -- start 2024-06-10T00:34:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbada2a3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f050 0x563a0dc2e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbada2a3b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f970 0x563a0dc2e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbada2a3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc23cb0 0x563a0dc2ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbada2a3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81ad910 con 0x563a0dc1f050 2024-06-10T00:34:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbada2a3b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81ad240 con 0x563a0dc1f970 2024-06-10T00:34:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbada2a3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81aa440 con 0x563a0dc23cb0 2024-06-10T00:34:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbada2a3b80 10 client.0 _ll_register_callbacks cb 0x563a0db069a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbacd7bc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f050 0x563a0dc2e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbacd7bc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f050 0x563a0dc2e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37396/0 (socket says 172.21.15.62:37396) 2024-06-10T00:34:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbacd7bc700 1 -- 172.21.15.62:0/1748557009 learned_addr learned my addr 172.21.15.62:0/1748557009 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f970 0x563a0dc2e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbacdfbd700 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc23cb0 0x563a0dc2ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbacd7bc700 1 -- 172.21.15.62:0/1748557009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc23cb0 msgr2=0x563a0dc2ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbacd7bc700 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc23cb0 0x563a0dc2ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbacd7bc700 1 -- 172.21.15.62:0/1748557009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f970 msgr2=0x563a0dc2e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:41.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbacd7bc700 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f970 0x563a0dc2e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbacd7bc700 1 -- 172.21.15.62:0/1748557009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a0dca3c50 con 0x563a0dc1f050 2024-06-10T00:34:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbacdfbd700 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc23cb0 0x563a0dc2ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f970 0x563a0dc2e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbacd7bc700 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f050 0x563a0dc2e230 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbac4011010 tx=0x7fbac401f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd24b055b31462c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.551+0000 7fbab0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.552+0000 7fbab0ff9700 1 -- 172.21.15.62:0/1748557009 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac40109a0 con 0x563a0dc1f050 2024-06-10T00:34:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.552+0000 7fbab0ff9700 1 -- 172.21.15.62:0/1748557009 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fbac4028680 con 0x563a0dc1f050 2024-06-10T00:34:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.552+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1748557009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbab809ce10 con 0x563a0dc1f050 2024-06-10T00:34:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.552+0000 7fbab0ff9700 1 -- 172.21.15.62:0/1748557009 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbac4046050 con 0x563a0dc1f050 2024-06-10T00:34:41.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.552+0000 7fbada2a3b80 10 client.15963 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:41.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.552+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1748557009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a0dca4f50 con 0x563a0dc1f050 2024-06-10T00:34:41.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.552+0000 7fbab0ff9700 1 -- 172.21.15.62:0/1748557009 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbab809ce10 con 0x563a0dc1f050 2024-06-10T00:34:41.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.552+0000 7fbab0ff9700 1 -- 172.21.15.62:0/1748557009 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fbac403d5d0 con 0x563a0dc1f050 2024-06-10T00:34:41.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.552+0000 7fbada2a3b80 10 client.15963 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:41.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.552+0000 7fbada2a3b80 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:34:41.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.552+0000 7fbada2a3b80 20 client.15963 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:41.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.552+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563a0dca5660 0x563a0dca7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:41.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.552+0000 7fbada2a3b80 4 client.15963 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T00:34:41.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.552+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1748557009 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563a0db74310 con 0x563a0dca5660 2024-06-10T00:34:41.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.553+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563a0dca5660 0x563a0dca7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:41.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.553+0000 7fbaccfbb700 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563a0dca5660 0x563a0dca7a80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b61959499c242e0 server_cookie=7ce581e70bd28a21 in_seq=0 out_seq=0 2024-06-10T00:34:41.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.553+0000 7fbab0ff9700 10 client.15963 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:41.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.554+0000 7fbab0ff9700 1 -- 172.21.15.62:0/1748557009 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563a0db74310 con 0x563a0dca5660 2024-06-10T00:34:41.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.554+0000 7fbab0ff9700 10 client.15963 handle_command_reply: tid=0 2024-06-10T00:34:41.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.554+0000 7fbada2a3b80 1 client.15963 shutdown 2024-06-10T00:34:41.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.554+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1748557009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f050 msgr2=0x563a0dc2e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:41.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.554+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f050 0x563a0dc2e230 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbac4011010 tx=0x7fbac401f790 comp rx=0 tx=0).stop 2024-06-10T00:34:41.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.554+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1748557009 shutdown_connections 2024-06-10T00:34:41.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.554+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563a0dca5660 0x563a0dca7a80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.554+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a0dc1f050 0x563a0dc2e230 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.554+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a0dc23cb0 0x563a0dc2ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.554+0000 7fbada2a3b80 1 --2- 172.21.15.62:0/1748557009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a0dc1f970 0x563a0dc2e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.554+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1748557009 >> 172.21.15.62:0/1748557009 conn(0x563a0db45d40 msgr2=0x563a0dc29e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:41.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.555+0000 7fbab0ff9700 0 client.15963 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:41.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.555+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1748557009 shutdown_connections 2024-06-10T00:34:41.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.555+0000 7fbada2a3b80 1 -- 172.21.15.62:0/1748557009 wait complete. 2024-06-10T00:34:41.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.555+0000 7fbada2a3b80 20 client.15963 trim_cache size 0 max 16384 2024-06-10T00:34:41.561 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:34:41.561 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-06-10T00:34:41.561 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "6d995127-a626-45b8-a01d-767addb17c40", 2024-06-10T00:34:41.562 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-06-10T00:34:41.562 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:34:41.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.555+0000 7fbacfa20700 1 -- 172.21.15.62:0/1252810668 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbab40a2e10 msgr2=0x7fbab40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:41.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.555+0000 7fbacfa20700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbab40a2e10 0x7fbab40a52d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbac8198b00 tx=0x7fbac400b410 comp rx=0 tx=0).stop 2024-06-10T00:34:41.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.555+0000 7fbacfa20700 1 -- 172.21.15.62:0/1252810668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac8101d40 msgr2=0x7fbac819dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:41.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.555+0000 7fbacfa20700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac8101d40 0x7fbac819dd40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009970 tx=0x7fbab8009c80 comp rx=0 tx=0).stop 2024-06-10T00:34:41.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.555+0000 7fbacfa20700 1 -- 172.21.15.62:0/1252810668 shutdown_connections 2024-06-10T00:34:41.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.555+0000 7fbacfa20700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbab40a2e10 0x7fbab40a52d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.555+0000 7fbacfa20700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbac81073f0 0x7fbac81979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.555+0000 7fbacfa20700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbac8101380 0x7fbac819d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.555+0000 7fbacfa20700 1 --2- 172.21.15.62:0/1252810668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac8101d40 0x7fbac819dd40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:41.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.555+0000 7fbacfa20700 1 -- 172.21.15.62:0/1252810668 >> 172.21.15.62:0/1252810668 conn(0x7fbac8078300 msgr2=0x7fbac80ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:41.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.556+0000 7fbacfa20700 1 -- 172.21.15.62:0/1252810668 shutdown_connections 2024-06-10T00:34:41.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:41.556+0000 7fbacfa20700 1 -- 172.21.15.62:0/1252810668 wait complete. 2024-06-10T00:34:41.594 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether build environment is sane... yes 2024-06-10T00:34:41.604 INFO:tasks.workunit.client.0.smithi062.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-06-10T00:34:41.604 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gawk... gawk 2024-06-10T00:34:41.620 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether make sets $(MAKE)... yes 2024-06-10T00:34:41.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:41 smithi138 ceph-mon[36549]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:34:41.649 INFO:tasks.workunit.client.0.smithi062.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-06-10T00:34:41.649 INFO:tasks.workunit.client.0.smithi062.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-06-10T00:34:41.661 INFO:tasks.workunit.client.0.smithi062.stdout:checking for style of include used by make... GNU 2024-06-10T00:34:41.666 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc... gcc 2024-06-10T00:34:41.725 INFO:tasks.workunit.client.0.smithi062.stdout:checking for C compiler default output file name... a.out 2024-06-10T00:34:41.726 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether the C compiler works... yes 2024-06-10T00:34:41.729 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether we are cross compiling... no 2024-06-10T00:34:41.769 INFO:tasks.workunit.client.0.smithi062.stdout:checking for suffix of executables... 2024-06-10T00:34:41.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:41 smithi105 ceph-mon[36321]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-10T00:34:41.805 INFO:tasks.workunit.client.0.smithi062.stdout:checking for suffix of object files... o 2024-06-10T00:34:41.849 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether we are using the GNU C compiler... yes 2024-06-10T00:34:41.883 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether gcc accepts -g... yes 2024-06-10T00:34:42.014 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc option to accept ISO C89... none needed 2024-06-10T00:34:42.123 INFO:tasks.workunit.client.0.smithi062.stdout:checking dependency style of gcc... gcc3 2024-06-10T00:34:42.290 INFO:tasks.workunit.client.0.smithi062.stdout:checking how to run the C preprocessor... gcc -E 2024-06-10T00:34:42.339 INFO:tasks.workunit.client.0.smithi062.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-06-10T00:34:42.340 INFO:tasks.workunit.client.0.smithi062.stdout:checking for egrep... /usr/bin/grep -E 2024-06-10T00:34:42.349 INFO:tasks.workunit.client.0.smithi062.stdout:checking for AIX... no 2024-06-10T00:34:42.372 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc... (cached) gcc 2024-06-10T00:34:42.379 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-06-10T00:34:42.381 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether gcc accepts -g... (cached) yes 2024-06-10T00:34:42.381 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-06-10T00:34:42.381 INFO:tasks.workunit.client.0.smithi062.stdout:checking dependency style of gcc... (cached) gcc3 2024-06-10T00:34:42.447 INFO:tasks.workunit.client.0.smithi062.stdout:checking for main in -lm... yes 2024-06-10T00:34:42.504 INFO:tasks.workunit.client.0.smithi062.stdout:checking for main in -lpthread... yes 2024-06-10T00:34:42.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:42 smithi138 ceph-mon[36549]: scrub queued for path: / 2024-06-10T00:34:42.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:42 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:34:42.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:42 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:34:42.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:42 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:34:42.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:42 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:34:42.670 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:42 smithi062 ceph-mon[32390]: scrub queued for path: / 2024-06-10T00:34:42.670 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:42 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:34:42.670 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:42 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:34:42.670 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:42 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:34:42.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:42 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:34:42.717 INFO:tasks.workunit.client.0.smithi062.stdout:checking for ANSI C header files... yes 2024-06-10T00:34:42.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:42 smithi105 ceph-mon[36321]: scrub queued for path: / 2024-06-10T00:34:42.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:42 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:34:42.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:42 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:34:42.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:42 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:34:42.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:42 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:34:42.936 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-06-10T00:34:42.989 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/types.h... yes 2024-06-10T00:34:43.041 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:34:43.041 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/stat.h... yes 2024-06-10T00:34:43.088 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stdlib.h... yes 2024-06-10T00:34:43.138 INFO:tasks.workunit.client.0.smithi062.stdout:checking for string.h... yes 2024-06-10T00:34:43.196 INFO:tasks.workunit.client.0.smithi062.stdout:checking for memory.h... yes 2024-06-10T00:34:43.255 INFO:tasks.workunit.client.0.smithi062.stdout:checking for strings.h... yes 2024-06-10T00:34:43.304 INFO:tasks.workunit.client.0.smithi062.stdout:checking for inttypes.h... yes 2024-06-10T00:34:43.373 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stdint.h... yes 2024-06-10T00:34:43.428 INFO:tasks.workunit.client.0.smithi062.stdout:checking for unistd.h... yes 2024-06-10T00:34:43.550 INFO:tasks.workunit.client.0.smithi062.stdout:checking pthread.h usability... yes 2024-06-10T00:34:43.573 INFO:tasks.workunit.client.0.smithi062.stdout:checking pthread.h presence... yes 2024-06-10T00:34:43.574 INFO:tasks.workunit.client.0.smithi062.stdout:checking for pthread.h... yes 2024-06-10T00:34:43.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:43 smithi138 ceph-mon[36549]: pgmap v59: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 127 KiB/s wr, 19 op/s 2024-06-10T00:34:43.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:43 smithi062 ceph-mon[32390]: pgmap v59: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 127 KiB/s wr, 19 op/s 2024-06-10T00:34:43.634 INFO:tasks.workunit.client.0.smithi062.stdout:checking fcntl.h usability... yes 2024-06-10T00:34:43.655 INFO:tasks.workunit.client.0.smithi062.stdout:checking fcntl.h presence... yes 2024-06-10T00:34:43.656 INFO:tasks.workunit.client.0.smithi062.stdout:checking for fcntl.h... yes 2024-06-10T00:34:43.723 INFO:tasks.workunit.client.0.smithi062.stdout:checking limits.h usability... yes 2024-06-10T00:34:43.745 INFO:tasks.workunit.client.0.smithi062.stdout:checking limits.h presence... yes 2024-06-10T00:34:43.745 INFO:tasks.workunit.client.0.smithi062.stdout:checking for limits.h... yes 2024-06-10T00:34:43.760 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stdint.h... (cached) yes 2024-06-10T00:34:43.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:43 smithi105 ceph-mon[36321]: pgmap v59: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 127 KiB/s wr, 19 op/s 2024-06-10T00:34:43.830 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/time.h usability... yes 2024-06-10T00:34:43.853 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/time.h presence... yes 2024-06-10T00:34:43.853 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/time.h... yes 2024-06-10T00:34:43.871 INFO:tasks.workunit.client.0.smithi062.stdout:checking for unistd.h... (cached) yes 2024-06-10T00:34:43.953 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/vfs.h usability... yes 2024-06-10T00:34:43.975 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/vfs.h presence... yes 2024-06-10T00:34:43.975 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/vfs.h... yes 2024-06-10T00:34:44.026 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/limits.h usability... no 2024-06-10T00:34:44.047 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/limits.h presence... no 2024-06-10T00:34:44.047 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/limits.h... no 2024-06-10T00:34:44.082 INFO:tasks.workunit.client.0.smithi062.stdout:checking for an ANSI C-conforming const... yes 2024-06-10T00:34:44.118 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-06-10T00:34:44.119 INFO:tasks.workunit.client.0.smithi062.stdout:checking for function prototypes... yes 2024-06-10T00:34:44.149 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether setvbuf arguments are reversed... no 2024-06-10T00:34:44.235 INFO:tasks.workunit.client.0.smithi062.stdout:checking for system... yes 2024-06-10T00:34:44.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.247+0000 7f503d5c9700 1 -- 172.21.15.62:0/3874823497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503806aa00 msgr2=0x7f503806ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:44.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.247+0000 7f503d5c9700 1 --2- 172.21.15.62:0/3874823497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503806aa00 0x7f503806ae80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f502c009990 tx=0x7f502c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:34:44.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.247+0000 7f503d5c9700 1 -- 172.21.15.62:0/3874823497 shutdown_connections 2024-06-10T00:34:44.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.247+0000 7f503d5c9700 1 --2- 172.21.15.62:0/3874823497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f503806b3c0 0x7f503810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:44.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.247+0000 7f503d5c9700 1 --2- 172.21.15.62:0/3874823497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50381084f0 0x7f50381088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:44.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.247+0000 7f503d5c9700 1 --2- 172.21.15.62:0/3874823497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503806aa00 0x7f503806ae80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.247+0000 7f503d5c9700 1 -- 172.21.15.62:0/3874823497 >> 172.21.15.62:0/3874823497 conn(0x7f5038078300 msgr2=0x7f5038078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.248+0000 7f503d5c9700 1 -- 172.21.15.62:0/3874823497 shutdown_connections 2024-06-10T00:34:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.248+0000 7f503d5c9700 1 -- 172.21.15.62:0/3874823497 wait complete. 2024-06-10T00:34:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.248+0000 7f503d5c9700 1 Processor -- start 2024-06-10T00:34:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f503d5c9700 1 -- start start 2024-06-10T00:34:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f503d5c9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f503806aa00 0x7f5038101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f503d5c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f503806b3c0 0x7f5038101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f503d5c9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50381084f0 0x7f5038101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f503d5c9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5038112750 con 0x7f50381084f0 2024-06-10T00:34:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f503d5c9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f50381125d0 con 0x7f503806aa00 2024-06-10T00:34:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f503d5c9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f50381128d0 con 0x7f503806b3c0 2024-06-10T00:34:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f50377fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50381084f0 0x7f5038101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f50367fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f503806b3c0 0x7f5038101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f5036ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f503806aa00 0x7f5038101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f50377fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50381084f0 0x7f5038101ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37410/0 (socket says 172.21.15.62:37410) 2024-06-10T00:34:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f50377fe700 1 -- 172.21.15.62:0/4219681263 learned_addr learned my addr 172.21.15.62:0/4219681263 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f5036ffd700 1 -- 172.21.15.62:0/4219681263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f503806b3c0 msgr2=0x7f5038101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f5036ffd700 1 --2- 172.21.15.62:0/4219681263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f503806b3c0 0x7f5038101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f5036ffd700 1 -- 172.21.15.62:0/4219681263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50381084f0 msgr2=0x7f5038101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f5036ffd700 1 --2- 172.21.15.62:0/4219681263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50381084f0 0x7f5038101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f5036ffd700 1 -- 172.21.15.62:0/4219681263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50381a7190 con 0x7f503806aa00 2024-06-10T00:34:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.249+0000 7f50377fe700 1 --2- 172.21.15.62:0/4219681263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50381084f0 0x7f5038101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:34:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.250+0000 7f5036ffd700 1 --2- 172.21.15.62:0/4219681263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f503806aa00 0x7f5038101450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f502800e9f0 tx=0x7f502800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.250+0000 7f501ffff700 1 -- 172.21.15.62:0/4219681263 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502800cb70 con 0x7f503806aa00 2024-06-10T00:34:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.250+0000 7f501ffff700 1 -- 172.21.15.62:0/4219681263 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502800ee90 con 0x7f503806aa00 2024-06-10T00:34:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.250+0000 7f503d5c9700 1 -- 172.21.15.62:0/4219681263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50381a7480 con 0x7f503806aa00 2024-06-10T00:34:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.250+0000 7f501ffff700 1 -- 172.21.15.62:0/4219681263 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028003dc0 con 0x7f503806aa00 2024-06-10T00:34:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.250+0000 7f503d5c9700 1 -- 172.21.15.62:0/4219681263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50381a7990 con 0x7f503806aa00 2024-06-10T00:34:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.251+0000 7f503d5c9700 1 -- 172.21.15.62:0/4219681263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5038069180 con 0x7f503806aa00 2024-06-10T00:34:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.253+0000 7f501ffff700 1 -- 172.21.15.62:0/4219681263 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5028004000 con 0x7f503806aa00 2024-06-10T00:34:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.254+0000 7f501ffff700 1 --2- 172.21.15.62:0/4219681263 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f50200a2d90 0x7f50200a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.254+0000 7f50367fc700 1 --2- 172.21.15.62:0/4219681263 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f50200a2d90 0x7f50200a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.254+0000 7f501ffff700 1 -- 172.21.15.62:0/4219681263 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f50280cf950 con 0x7f503806aa00 2024-06-10T00:34:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.254+0000 7f50367fc700 1 --2- 172.21.15.62:0/4219681263 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f50200a2d90 0x7f50200a5250 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f502c00b2e0 tx=0x7f502c005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.258+0000 7f501ffff700 1 -- 172.21.15.62:0/4219681263 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f502809c160 con 0x7f503806aa00 2024-06-10T00:34:44.301 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gettimeofday... yes 2024-06-10T00:34:44.371 INFO:tasks.workunit.client.0.smithi062.stdout:checking for mkdir... yes 2024-06-10T00:34:44.378 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:44.509 INFO:tasks.workunit.client.0.smithi062.stdout:checking for strerror... yes 2024-06-10T00:34:44.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.579+0000 7f503d5c9700 1 -- 172.21.15.62:0/4219681263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5038102980 con 0x7f503806aa00 2024-06-10T00:34:44.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.580+0000 7f501ffff700 1 -- 172.21.15.62:0/4219681263 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f502809ba60 con 0x7f503806aa00 2024-06-10T00:34:44.581 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:34:44.581 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:32:47.783154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:34:44.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.583+0000 7f503d5c9700 1 -- 172.21.15.62:0/4219681263 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f50200a2d90 msgr2=0x7f50200a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:44.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.583+0000 7f503d5c9700 1 --2- 172.21.15.62:0/4219681263 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f50200a2d90 0x7f50200a5250 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f502c00b2e0 tx=0x7f502c005960 comp rx=0 tx=0).stop 2024-06-10T00:34:44.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.583+0000 7f503d5c9700 1 -- 172.21.15.62:0/4219681263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f503806aa00 msgr2=0x7f5038101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:44.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.583+0000 7f503d5c9700 1 --2- 172.21.15.62:0/4219681263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f503806aa00 0x7f5038101450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f502800e9f0 tx=0x7f502800ed00 comp rx=0 tx=0).stop 2024-06-10T00:34:44.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.583+0000 7f503d5c9700 1 -- 172.21.15.62:0/4219681263 shutdown_connections 2024-06-10T00:34:44.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.583+0000 7f503d5c9700 1 --2- 172.21.15.62:0/4219681263 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f50200a2d90 0x7f50200a5250 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:44.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.583+0000 7f503d5c9700 1 --2- 172.21.15.62:0/4219681263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50381084f0 0x7f5038101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:44.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.583+0000 7f503d5c9700 1 --2- 172.21.15.62:0/4219681263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f503806aa00 0x7f5038101450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:44.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.583+0000 7f503d5c9700 1 --2- 172.21.15.62:0/4219681263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f503806b3c0 0x7f5038101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:44.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.583+0000 7f503d5c9700 1 -- 172.21.15.62:0/4219681263 >> 172.21.15.62:0/4219681263 conn(0x7f5038078300 msgr2=0x7f5038100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:44.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.583+0000 7f503d5c9700 1 -- 172.21.15.62:0/4219681263 shutdown_connections 2024-06-10T00:34:44.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:44.583+0000 7f503d5c9700 1 -- 172.21.15.62:0/4219681263 wait complete. 2024-06-10T00:34:44.585 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-06-10T00:34:44.595 INFO:tasks.workunit.client.0.smithi062.stdout:checking for open64... yes 2024-06-10T00:34:44.663 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stat64... yes 2024-06-10T00:34:44.729 INFO:tasks.workunit.client.0.smithi062.stdout:checking for fseeko64... yes 2024-06-10T00:34:44.797 INFO:tasks.workunit.client.0.smithi062.stdout:checking for lrand48_r... yes 2024-06-10T00:34:44.883 INFO:tasks.workunit.client.0.smithi062.stdout:checking for srand48_r... yes 2024-06-10T00:34:44.924 INFO:tasks.workunit.client.0.smithi062.stdout:configure: creating ./config.status 2024-06-10T00:34:45.023 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: creating Makefile 2024-06-10T00:34:45.037 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: creating config.h 2024-06-10T00:34:45.054 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: executing depfiles commands 2024-06-10T00:34:45.100 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: executing default commands 2024-06-10T00:34:45.154 INFO:tasks.workunit.client.0.smithi062.stderr:+ make 2024-06-10T00:34:45.162 INFO:tasks.workunit.client.0.smithi062.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-06-10T00:34:45.165 INFO:tasks.workunit.client.0.smithi062.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-06-10T00:34:45.166 INFO:tasks.workunit.client.0.smithi062.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-06-10T00:34:45.166 INFO:tasks.workunit.client.0.smithi062.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-06-10T00:34:45.166 INFO:tasks.workunit.client.0.smithi062.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-06-10T00:34:45.166 INFO:tasks.workunit.client.0.smithi062.stderr: any GNU archive site. 2024-06-10T00:34:45.168 INFO:tasks.workunit.client.0.smithi062.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-06-10T00:34:45.169 INFO:tasks.workunit.client.0.smithi062.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-06-10T00:34:45.170 INFO:tasks.workunit.client.0.smithi062.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-06-10T00:34:45.170 INFO:tasks.workunit.client.0.smithi062.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-06-10T00:34:45.170 INFO:tasks.workunit.client.0.smithi062.stderr: You might want to install the `Automake' and `Perl' packages. 2024-06-10T00:34:45.170 INFO:tasks.workunit.client.0.smithi062.stderr: Grab them from any GNU archive site. 2024-06-10T00:34:45.241 INFO:tasks.workunit.client.0.smithi062.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-06-10T00:34:45.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:45 smithi138 ceph-mon[36549]: pgmap v60: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 127 KiB/s wr, 15 op/s 2024-06-10T00:34:45.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:45 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/4219681263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:45.725 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:45 smithi062 ceph-mon[32390]: pgmap v60: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 127 KiB/s wr, 15 op/s 2024-06-10T00:34:45.725 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:45 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/4219681263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:45.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:45 smithi105 ceph-mon[36321]: pgmap v60: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 127 KiB/s wr, 15 op/s 2024-06-10T00:34:45.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:45 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/4219681263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:46.085 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-10T00:34:46.085 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24464 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-10T00:34:46.282 INFO:tasks.workunit.client.0.smithi062.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-06-10T00:34:46.283 INFO:tasks.workunit.client.0.smithi062.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-06-10T00:34:46.283 INFO:tasks.workunit.client.0.smithi062.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-06-10T00:34:46.283 INFO:tasks.workunit.client.0.smithi062.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-06-10T00:34:46.430 INFO:tasks.workunit.client.0.smithi062.stdout:/bin/sh ./config.status --recheck 2024-06-10T00:34:46.486 INFO:tasks.workunit.client.0.smithi062.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-06-10T00:34:46.599 INFO:tasks.workunit.client.0.smithi062.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-06-10T00:34:46.713 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:47.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.205+0000 7f2f1fc82700 1 -- 172.21.15.62:0/3889292693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f18107840 msgr2=0x7f2f1810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:47.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.205+0000 7f2f1fc82700 1 --2- 172.21.15.62:0/3889292693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f18107840 0x7f2f1810f3c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000b0d0 tx=0x7f2f1000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.206+0000 7f2f1fc82700 1 -- 172.21.15.62:0/3889292693 shutdown_connections 2024-06-10T00:34:47.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.206+0000 7f2f1fc82700 1 --2- 172.21.15.62:0/3889292693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f18107840 0x7f2f1810f3c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.206+0000 7f2f1fc82700 1 --2- 172.21.15.62:0/3889292693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f18102740 0x7f2f18102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.206+0000 7f2f1fc82700 1 --2- 172.21.15.62:0/3889292693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f181030f0 0x7f2f18107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.206+0000 7f2f1fc82700 1 -- 172.21.15.62:0/3889292693 >> 172.21.15.62:0/3889292693 conn(0x7f2f180795f0 msgr2=0x7f2f18079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:47.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.206+0000 7f2f1fc82700 1 -- 172.21.15.62:0/3889292693 shutdown_connections 2024-06-10T00:34:47.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.206+0000 7f2f1fc82700 1 -- 172.21.15.62:0/3889292693 wait complete. 2024-06-10T00:34:47.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1fc82700 1 Processor -- start 2024-06-10T00:34:47.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1fc82700 1 -- start start 2024-06-10T00:34:47.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1fc82700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f18102740 0x7f2f18074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1fc82700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f181030f0 0x7f2f18074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1fc82700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f18107840 0x7f2f18075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1fc82700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2f18112760 con 0x7f2f18107840 2024-06-10T00:34:47.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1fc82700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181125e0 con 0x7f2f18102740 2024-06-10T00:34:47.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1fc82700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181128e0 con 0x7f2f181030f0 2024-06-10T00:34:47.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1e21f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f18107840 0x7f2f18075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1e21f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f18107840 0x7f2f18075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37424/0 (socket says 172.21.15.62:37424) 2024-06-10T00:34:47.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1e21f700 1 -- 172.21.15.62:0/2348047694 learned_addr learned my addr 172.21.15.62:0/2348047694 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:47.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/2348047694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f181030f0 0x7f2f18074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1da1e700 1 --2- 172.21.15.62:0/2348047694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f18102740 0x7f2f18074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1e21f700 1 -- 172.21.15.62:0/2348047694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f181030f0 msgr2=0x7f2f18074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:47.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1e21f700 1 --2- 172.21.15.62:0/2348047694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f181030f0 0x7f2f18074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1e21f700 1 -- 172.21.15.62:0/2348047694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f18102740 msgr2=0x7f2f18074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:47.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1e21f700 1 --2- 172.21.15.62:0/2348047694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f18102740 0x7f2f18074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.207+0000 7f2f1e21f700 1 -- 172.21.15.62:0/2348047694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f181a70e0 con 0x7f2f18107840 2024-06-10T00:34:47.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.208+0000 7f2f1e21f700 1 --2- 172.21.15.62:0/2348047694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f18107840 0x7f2f18075210 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2f10006070 tx=0x7f2f10004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:47.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.208+0000 7f2f0effd700 1 -- 172.21.15.62:0/2348047694 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10021070 con 0x7f2f18107840 2024-06-10T00:34:47.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.208+0000 7f2f1fc82700 1 -- 172.21.15.62:0/2348047694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f181a7370 con 0x7f2f18107840 2024-06-10T00:34:47.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.208+0000 7f2f0effd700 1 -- 172.21.15.62:0/2348047694 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f10007b80 con 0x7f2f18107840 2024-06-10T00:34:47.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.208+0000 7f2f0effd700 1 -- 172.21.15.62:0/2348047694 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10005330 con 0x7f2f18107840 2024-06-10T00:34:47.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.208+0000 7f2f1fc82700 1 -- 172.21.15.62:0/2348047694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f181a7820 con 0x7f2f18107840 2024-06-10T00:34:47.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.211+0000 7f2f0effd700 1 -- 172.21.15.62:0/2348047694 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2f10024070 con 0x7f2f18107840 2024-06-10T00:34:47.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.211+0000 7f2f2a505b80 1 Processor -- start 2024-06-10T00:34:47.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.212+0000 7f2f2a505b80 1 -- start start 2024-06-10T00:34:47.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.212+0000 7f2f2a505b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bc640 0x55dfb07bca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.212+0000 7f2f2a505b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07bcf60 0x55dfb07b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.212+0000 7f2f2a505b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07b6f20 0x55dfb07b7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.212+0000 7f2f2a505b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55dfb0708810 con 0x55dfb07bc640 2024-06-10T00:34:47.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.212+0000 7f2f2a505b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dfb06b0720 con 0x55dfb07b6f20 2024-06-10T00:34:47.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.212+0000 7f2f2a505b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dfb07047d0 con 0x55dfb07bcf60 2024-06-10T00:34:47.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.212+0000 7f2f0effd700 1 --2- 172.21.15.62:0/2348047694 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2f040a2d90 0x7f2f040a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.212+0000 7f2f1d21d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bc640 0x55dfb07bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.212+0000 7f2f1d21d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bc640 0x55dfb07bca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37436/0 (socket says 172.21.15.62:37436) 2024-06-10T00:34:47.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.212+0000 7f2f1d21d700 1 -- 172.21.15.62:0/1919052495 learned_addr learned my addr 172.21.15.62:0/1919052495 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:47.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.212+0000 7f2f1e21f700 1 --2- 172.21.15.62:0/2348047694 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2f040a2d90 0x7f2f040a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.212+0000 7f2f1da1e700 1 --2- 172.21.15.62:0/1919052495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07bcf60 0x55dfb07b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.213+0000 7f2f1d21d700 1 -- 172.21.15.62:0/1919052495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07bcf60 msgr2=0x55dfb07b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:47.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.213+0000 7f2f0effd700 1 -- 172.21.15.62:0/2348047694 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f2f100d0920 con 0x7f2f18107840 2024-06-10T00:34:47.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.213+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/1919052495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07bcf60 0x55dfb07b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.213+0000 7f2f1d21d700 1 -- 172.21.15.62:0/1919052495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07b6f20 msgr2=0x55dfb07b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:47.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.213+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/1919052495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07b6f20 0x55dfb07b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.213+0000 7f2f1d21d700 1 -- 172.21.15.62:0/1919052495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb06a6f50 con 0x55dfb07bc640 2024-06-10T00:34:47.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.213+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/1919052495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bc640 0x55dfb07bca20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2f08007c90 tx=0x7f2f080123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8faa9af7d4a01299 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:47.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.213+0000 7f2f1e21f700 1 --2- 172.21.15.62:0/2348047694 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2f040a2d90 0x7f2f040a5250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2f10099490 tx=0x7f2f10009040 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:47.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.213+0000 7f2f03fff700 1 -- 172.21.15.62:0/1919052495 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08012ac0 con 0x55dfb07bc640 2024-06-10T00:34:47.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.213+0000 7f2f03fff700 1 -- 172.21.15.62:0/1919052495 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f08004510 con 0x55dfb07bc640 2024-06-10T00:34:47.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.213+0000 7f2f03fff700 1 -- 172.21.15.62:0/1919052495 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08003df0 con 0x55dfb07bc640 2024-06-10T00:34:47.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.214+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1919052495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bc640 msgr2=0x55dfb07bca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:47.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.214+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1919052495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bc640 0x55dfb07bca20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2f08007c90 tx=0x7f2f080123f0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.214+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1919052495 shutdown_connections 2024-06-10T00:34:47.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.214+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1919052495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bc640 0x55dfb07bca20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.214+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1919052495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07b6f20 0x55dfb07b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.214+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1919052495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07bcf60 0x55dfb07b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.214+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1919052495 >> 172.21.15.62:0/1919052495 conn(0x55dfb06e1cc0 msgr2=0x55dfb07b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:47.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.214+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1919052495 shutdown_connections 2024-06-10T00:34:47.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.214+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1919052495 wait complete. 2024-06-10T00:34:47.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.214+0000 7f2f2a505b80 1 -- start start 2024-06-10T00:34:47.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f2a505b80 1 Processor -- start 2024-06-10T00:34:47.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f2a505b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07b6f20 0x55dfb07c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f2a505b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07bc640 0x55dfb07bffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f2a505b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bcf60 0x55dfb07c0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f2a505b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55dfb07c39d0 con 0x55dfb07b6f20 2024-06-10T00:34:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f2a505b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dfb06a8bf0 con 0x55dfb07bcf60 2024-06-10T00:34:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f2a505b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dfb07c3b50 con 0x55dfb07bc640 2024-06-10T00:34:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f2a505b80 10 client.0 _ll_register_callbacks cb 0x55dfb06a2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f1d21d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07b6f20 0x55dfb07c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f1d21d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07b6f20 0x55dfb07c2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37442/0 (socket says 172.21.15.62:37442) 2024-06-10T00:34:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f1d21d700 1 -- 172.21.15.62:0/1083233999 learned_addr learned my addr 172.21.15.62:0/1083233999 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f1da1e700 1 --2- 172.21.15.62:0/1083233999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07bc640 0x55dfb07bffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/1083233999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bcf60 0x55dfb07c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f1d21d700 1 -- 172.21.15.62:0/1083233999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07bc640 msgr2=0x55dfb07bffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.215+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/1083233999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07bc640 0x55dfb07bffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.216+0000 7f2f1d21d700 1 -- 172.21.15.62:0/1083233999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bcf60 msgr2=0x55dfb07c0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:47.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.216+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/1083233999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bcf60 0x55dfb07c0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.216+0000 7f2f1d21d700 1 -- 172.21.15.62:0/1083233999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb06578d0 con 0x55dfb07b6f20 2024-06-10T00:34:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.216+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/1083233999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07b6f20 0x55dfb07c2ce0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2f08007430 tx=0x7f2f08007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39cf9a9df2605a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.216+0000 7f2f017fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.216+0000 7f2f017fa700 1 -- 172.21.15.62:0/1083233999 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08003a70 con 0x55dfb07b6f20 2024-06-10T00:34:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.216+0000 7f2f017fa700 1 -- 172.21.15.62:0/1083233999 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f2f08003c10 con 0x55dfb07b6f20 2024-06-10T00:34:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.216+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1083233999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f08004510 con 0x55dfb07b6f20 2024-06-10T00:34:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.216+0000 7f2f017fa700 1 -- 172.21.15.62:0/1083233999 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2f08003e10 con 0x55dfb07b6f20 2024-06-10T00:34:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.216+0000 7f2f2a505b80 10 client.15981 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.216+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1083233999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dfb07ce2e0 con 0x55dfb07b6f20 2024-06-10T00:34:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.216+0000 7f2f017fa700 1 -- 172.21.15.62:0/1083233999 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f08004510 con 0x55dfb07b6f20 2024-06-10T00:34:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.217+0000 7f2f017fa700 1 -- 172.21.15.62:0/1083233999 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2f08010b30 con 0x55dfb07b6f20 2024-06-10T00:34:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.217+0000 7f2f2a505b80 10 client.15981 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:47.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.217+0000 7f2f2a505b80 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:34:47.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.217+0000 7f2f2a505b80 20 client.15981 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:47.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.217+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1083233999 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfb07d0810 0x55dfb07d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.217+0000 7f2f2a505b80 4 client.15981 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:34:47.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.217+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1083233999 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfb0715d10 con 0x55dfb07d0810 2024-06-10T00:34:47.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.217+0000 7f2f1da1e700 1 --2- 172.21.15.62:0/1083233999 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfb07d0810 0x55dfb07d2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.217+0000 7f2f1da1e700 1 --2- 172.21.15.62:0/1083233999 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfb07d0810 0x55dfb07d2c00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94ff242ca1459ebd server_cookie=5e1ddb227bc5669c in_seq=0 out_seq=0 2024-06-10T00:34:47.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.217+0000 7f2f017fa700 10 client.15981 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:47.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.218+0000 7f2f017fa700 1 -- 172.21.15.62:0/1083233999 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dfb0715d10 con 0x55dfb07d0810 2024-06-10T00:34:47.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.218+0000 7f2f017fa700 10 client.15981 handle_command_reply: tid=0 2024-06-10T00:34:47.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.218+0000 7f2f2a505b80 1 client.15981 shutdown 2024-06-10T00:34:47.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.218+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1083233999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07b6f20 msgr2=0x55dfb07c2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:47.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.218+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1083233999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07b6f20 0x55dfb07c2ce0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2f08007430 tx=0x7f2f08007830 comp rx=0 tx=0).stop 2024-06-10T00:34:47.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.218+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1083233999 shutdown_connections 2024-06-10T00:34:47.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.218+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1083233999 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfb07d0810 0x55dfb07d2c00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.218+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1083233999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07b6f20 0x55dfb07c2ce0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.218+0000 7f2f017fa700 0 client.15981 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:47.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.218+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1083233999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bcf60 0x55dfb07c0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.218+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1083233999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07bc640 0x55dfb07bffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.219+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1083233999 >> 172.21.15.62:0/1083233999 conn(0x55dfb06e1cc0 msgr2=0x55dfb07b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:47.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.219+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1083233999 shutdown_connections 2024-06-10T00:34:47.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.219+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1083233999 wait complete. 2024-06-10T00:34:47.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.219+0000 7f2f2a505b80 20 client.15981 trim_cache size 0 max 16384 2024-06-10T00:34:47.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.239+0000 7f2f2a505b80 1 Processor -- start 2024-06-10T00:34:47.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.239+0000 7f2f2a505b80 1 -- start start 2024-06-10T00:34:47.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.239+0000 7f2f2a505b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07b6f20 0x55dfb07b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.239+0000 7f2f2a505b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bc640 0x55dfb07ab690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f2a505b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bcf60 0x55dfb07bb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f2a505b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181b0730 con 0x55dfb07bcf60 2024-06-10T00:34:47.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f2a505b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181b2270 con 0x55dfb07bc640 2024-06-10T00:34:47.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f2a505b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181b0e00 con 0x55dfb07b6f20 2024-06-10T00:34:47.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f1d21d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bcf60 0x55dfb07bb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f1d21d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bcf60 0x55dfb07bb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37446/0 (socket says 172.21.15.62:37446) 2024-06-10T00:34:47.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f1d21d700 1 -- 172.21.15.62:0/2777506754 learned_addr learned my addr 172.21.15.62:0/2777506754 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:47.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/2777506754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07b6f20 0x55dfb07b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f1da1e700 1 --2- 172.21.15.62:0/2777506754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bc640 0x55dfb07ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f1d21d700 1 -- 172.21.15.62:0/2777506754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07b6f20 msgr2=0x55dfb07b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:47.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/2777506754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07b6f20 0x55dfb07b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f1d21d700 1 -- 172.21.15.62:0/2777506754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bc640 msgr2=0x55dfb07ab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:47.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/2777506754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bc640 0x55dfb07ab690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f1d21d700 1 -- 172.21.15.62:0/2777506754 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb07c0f10 con 0x55dfb07bcf60 2024-06-10T00:34:47.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/2777506754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bcf60 0x55dfb07bb130 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2f08007400 tx=0x7f2f0801b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799c26a48eeb1e09 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:47.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f037fe700 1 -- 172.21.15.62:0/2777506754 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f080127f0 con 0x55dfb07bcf60 2024-06-10T00:34:47.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f037fe700 1 -- 172.21.15.62:0/2777506754 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f08026040 con 0x55dfb07bcf60 2024-06-10T00:34:47.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.240+0000 7f2f037fe700 1 -- 172.21.15.62:0/2777506754 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0801bbf0 con 0x55dfb07bcf60 2024-06-10T00:34:47.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.241+0000 7f2f2a505b80 1 -- 172.21.15.62:0/2777506754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bcf60 msgr2=0x55dfb07bb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:47.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.241+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/2777506754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bcf60 0x55dfb07bb130 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2f08007400 tx=0x7f2f0801b7d0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.241+0000 7f2f2a505b80 1 -- 172.21.15.62:0/2777506754 shutdown_connections 2024-06-10T00:34:47.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.241+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/2777506754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bcf60 0x55dfb07bb130 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.241+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/2777506754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07b6f20 0x55dfb07b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.241+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/2777506754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bc640 0x55dfb07ab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.241+0000 7f2f2a505b80 1 -- 172.21.15.62:0/2777506754 >> 172.21.15.62:0/2777506754 conn(0x55dfb06e1cc0 msgr2=0x55dfb07bd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:47.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.241+0000 7f2f2a505b80 1 -- 172.21.15.62:0/2777506754 shutdown_connections 2024-06-10T00:34:47.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.241+0000 7f2f2a505b80 1 -- 172.21.15.62:0/2777506754 wait complete. 2024-06-10T00:34:47.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.241+0000 7f2f2a505b80 1 -- start start 2024-06-10T00:34:47.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f2a505b80 1 Processor -- start 2024-06-10T00:34:47.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f2a505b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07b6f20 0x55dfb07c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f2a505b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bc640 0x55dfb07d7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f2a505b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bcf60 0x55dfb07d8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f2a505b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181b1ba0 con 0x55dfb07bc640 2024-06-10T00:34:47.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f2a505b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181b14d0 con 0x55dfb07bcf60 2024-06-10T00:34:47.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f2a505b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181ae6d0 con 0x55dfb07b6f20 2024-06-10T00:34:47.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f2a505b80 10 client.0 _ll_register_callbacks cb 0x55dfb06a2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:47.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f1da1e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bc640 0x55dfb07d7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f1da1e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bc640 0x55dfb07d7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37454/0 (socket says 172.21.15.62:37454) 2024-06-10T00:34:47.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f1da1e700 1 -- 172.21.15.62:0/1525160751 learned_addr learned my addr 172.21.15.62:0/1525160751 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:47.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/1525160751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07b6f20 0x55dfb07c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/1525160751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bcf60 0x55dfb07d8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f1da1e700 1 -- 172.21.15.62:0/1525160751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07b6f20 msgr2=0x55dfb07c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:47.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f1da1e700 1 --2- 172.21.15.62:0/1525160751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07b6f20 0x55dfb07c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f1da1e700 1 -- 172.21.15.62:0/1525160751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bcf60 msgr2=0x55dfb07d8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f1da1e700 1 --2- 172.21.15.62:0/1525160751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bcf60 0x55dfb07d8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f1da1e700 1 -- 172.21.15.62:0/1525160751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb083a1a0 con 0x55dfb07bc640 2024-06-10T00:34:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.242+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/1525160751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bcf60 0x55dfb07d8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.243+0000 7f2f1da1e700 1 --2- 172.21.15.62:0/1525160751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bc640 0x55dfb07d7c30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55dfb07d3140 tx=0x7f2f140127e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e4f00a9b036cb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.243+0000 7f2f017fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.243+0000 7f2f017fa700 1 -- 172.21.15.62:0/1525160751 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14004240 con 0x55dfb07bc640 2024-06-10T00:34:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.243+0000 7f2f017fa700 1 -- 172.21.15.62:0/1525160751 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f2f140043e0 con 0x55dfb07bc640 2024-06-10T00:34:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.243+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1525160751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f08026040 con 0x55dfb07bc640 2024-06-10T00:34:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.243+0000 7f2f017fa700 1 -- 172.21.15.62:0/1525160751 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2f14003680 con 0x55dfb07bc640 2024-06-10T00:34:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.243+0000 7f2f2a505b80 10 client.15987 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.243+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1525160751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dfb083b380 con 0x55dfb07bc640 2024-06-10T00:34:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.243+0000 7f2f017fa700 1 -- 172.21.15.62:0/1525160751 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f08026040 con 0x55dfb07bc640 2024-06-10T00:34:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.244+0000 7f2f017fa700 1 -- 172.21.15.62:0/1525160751 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2f1401e8d0 con 0x55dfb07bc640 2024-06-10T00:34:47.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.244+0000 7f2f2a505b80 10 client.15987 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:47.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.244+0000 7f2f2a505b80 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:34:47.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.244+0000 7f2f2a505b80 20 client.15987 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:47.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.244+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1525160751 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfb07d0810 0x55dfb083da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:47.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.244+0000 7f2f2a505b80 4 client.15987 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:34:47.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.244+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1525160751 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dfb07bab00 con 0x55dfb07d0810 2024-06-10T00:34:47.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.244+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/1525160751 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfb07d0810 0x55dfb083da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:47.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f1d21d700 1 --2- 172.21.15.62:0/1525160751 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfb07d0810 0x55dfb083da70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9995fd5c29868c server_cookie=b4c03a0e1d48b8a4 in_seq=0 out_seq=0 2024-06-10T00:34:47.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f017fa700 10 client.15987 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:47.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f017fa700 1 -- 172.21.15.62:0/1525160751 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dfb07bab00 con 0x55dfb07d0810 2024-06-10T00:34:47.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f017fa700 10 client.15987 handle_command_reply: tid=0 2024-06-10T00:34:47.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f2a505b80 1 client.15987 shutdown 2024-06-10T00:34:47.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1525160751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bc640 msgr2=0x55dfb07d7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1525160751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bc640 0x55dfb07d7c30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55dfb07d3140 tx=0x7f2f140127e0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1525160751 shutdown_connections 2024-06-10T00:34:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1525160751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dfb07bcf60 0x55dfb07d8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1525160751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfb07b6f20 0x55dfb07c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1525160751 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dfb07d0810 0x55dfb083da70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f2a505b80 1 --2- 172.21.15.62:0/1525160751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dfb07bc640 0x55dfb07d7c30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f017fa700 0 client.15987 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1525160751 >> 172.21.15.62:0/1525160751 conn(0x55dfb06e1cc0 msgr2=0x55dfb07c1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1525160751 shutdown_connections 2024-06-10T00:34:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f2a505b80 1 -- 172.21.15.62:0/1525160751 wait complete. 2024-06-10T00:34:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.245+0000 7f2f2a505b80 20 client.15987 trim_cache size 0 max 16384 2024-06-10T00:34:47.252 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:34:47.252 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-06-10T00:34:47.252 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-06-10T00:34:47.252 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:34:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.246+0000 7f2f1fc82700 1 -- 172.21.15.62:0/2348047694 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2f040a2d90 msgr2=0x7f2f040a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.246+0000 7f2f1fc82700 1 --2- 172.21.15.62:0/2348047694 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2f040a2d90 0x7f2f040a5250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2f10099490 tx=0x7f2f10009040 comp rx=0 tx=0).stop 2024-06-10T00:34:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.246+0000 7f2f1fc82700 1 -- 172.21.15.62:0/2348047694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f18107840 msgr2=0x7f2f18075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.246+0000 7f2f1fc82700 1 --2- 172.21.15.62:0/2348047694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f18107840 0x7f2f18075210 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2f10006070 tx=0x7f2f10004d00 comp rx=0 tx=0).stop 2024-06-10T00:34:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.246+0000 7f2f1fc82700 1 -- 172.21.15.62:0/2348047694 shutdown_connections 2024-06-10T00:34:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.246+0000 7f2f1fc82700 1 --2- 172.21.15.62:0/2348047694 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f2f040a2d90 0x7f2f040a5250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.246+0000 7f2f1fc82700 1 --2- 172.21.15.62:0/2348047694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f18107840 0x7f2f18075210 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.246+0000 7f2f1fc82700 1 --2- 172.21.15.62:0/2348047694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f18102740 0x7f2f18074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.246+0000 7f2f1fc82700 1 --2- 172.21.15.62:0/2348047694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f181030f0 0x7f2f18074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.246+0000 7f2f1fc82700 1 -- 172.21.15.62:0/2348047694 >> 172.21.15.62:0/2348047694 conn(0x7f2f180795f0 msgr2=0x7f2f18100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.246+0000 7f2f1fc82700 1 -- 172.21.15.62:0/2348047694 shutdown_connections 2024-06-10T00:34:47.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:47.246+0000 7f2f1fc82700 1 -- 172.21.15.62:0/2348047694 wait complete. 2024-06-10T00:34:47.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:47 smithi062 ceph-mon[32390]: pgmap v61: 129 pgs: 129 active+clean; 5.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 394 KiB/s wr, 52 op/s 2024-06-10T00:34:47.606 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether build environment is sane... yes 2024-06-10T00:34:47.611 INFO:tasks.workunit.client.0.smithi062.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-06-10T00:34:47.611 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gawk... gawk 2024-06-10T00:34:47.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:47 smithi138 ceph-mon[36549]: pgmap v61: 129 pgs: 129 active+clean; 5.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 394 KiB/s wr, 52 op/s 2024-06-10T00:34:47.628 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether make sets $(MAKE)... yes 2024-06-10T00:34:47.658 INFO:tasks.workunit.client.0.smithi062.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-06-10T00:34:47.658 INFO:tasks.workunit.client.0.smithi062.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-06-10T00:34:47.671 INFO:tasks.workunit.client.0.smithi062.stdout:checking for style of include used by make... GNU 2024-06-10T00:34:47.677 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc... gcc 2024-06-10T00:34:47.770 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether the C compiler works... yes 2024-06-10T00:34:47.770 INFO:tasks.workunit.client.0.smithi062.stdout:checking for C compiler default output file name... a.out 2024-06-10T00:34:47.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:47 smithi105 ceph-mon[36321]: pgmap v61: 129 pgs: 129 active+clean; 5.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 394 KiB/s wr, 52 op/s 2024-06-10T00:34:47.811 INFO:tasks.workunit.client.0.smithi062.stdout:checking for suffix of executables... 2024-06-10T00:34:47.850 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether we are cross compiling... no 2024-06-10T00:34:47.894 INFO:tasks.workunit.client.0.smithi062.stdout:checking for suffix of object files... o 2024-06-10T00:34:47.919 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether we are using the GNU C compiler... yes 2024-06-10T00:34:47.944 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether gcc accepts -g... yes 2024-06-10T00:34:48.005 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc option to accept ISO C89... none needed 2024-06-10T00:34:48.117 INFO:tasks.workunit.client.0.smithi062.stdout:checking dependency style of gcc... gcc3 2024-06-10T00:34:48.167 INFO:tasks.workunit.client.0.smithi062.stdout:checking how to run the C preprocessor... gcc -E 2024-06-10T00:34:48.176 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T00:34:48.177 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:34:48.219 INFO:tasks.workunit.client.0.smithi062.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-06-10T00:34:48.220 INFO:tasks.workunit.client.0.smithi062.stdout:checking for egrep... /usr/bin/grep -E 2024-06-10T00:34:48.351 INFO:tasks.workunit.client.0.smithi062.stdout:checking for ANSI C header files... yes 2024-06-10T00:34:48.391 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/types.h... yes 2024-06-10T00:34:48.434 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/stat.h... yes 2024-06-10T00:34:48.476 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stdlib.h... yes 2024-06-10T00:34:48.519 INFO:tasks.workunit.client.0.smithi062.stdout:checking for string.h... yes 2024-06-10T00:34:48.566 INFO:tasks.workunit.client.0.smithi062.stdout:checking for memory.h... yes 2024-06-10T00:34:48.610 INFO:tasks.workunit.client.0.smithi062.stdout:checking for strings.h... yes 2024-06-10T00:34:48.654 INFO:tasks.workunit.client.0.smithi062.stdout:checking for inttypes.h... yes 2024-06-10T00:34:48.698 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stdint.h... yes 2024-06-10T00:34:48.744 INFO:tasks.workunit.client.0.smithi062.stdout:checking for unistd.h... yes 2024-06-10T00:34:48.795 INFO:tasks.workunit.client.0.smithi062.stdout:checking minix/config.h usability... no 2024-06-10T00:34:48.827 INFO:tasks.workunit.client.0.smithi062.stdout:checking minix/config.h presence... no 2024-06-10T00:34:48.827 INFO:tasks.workunit.client.0.smithi062.stdout:checking for minix/config.h... no 2024-06-10T00:34:48.878 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-06-10T00:34:48.879 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc... (cached) gcc 2024-06-10T00:34:48.914 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-06-10T00:34:48.914 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether gcc accepts -g... (cached) yes 2024-06-10T00:34:48.914 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-06-10T00:34:48.914 INFO:tasks.workunit.client.0.smithi062.stdout:checking dependency style of gcc... (cached) gcc3 2024-06-10T00:34:48.957 INFO:tasks.workunit.client.0.smithi062.stdout:checking for main in -lm... yes 2024-06-10T00:34:49.002 INFO:tasks.workunit.client.0.smithi062.stdout:checking for main in -lpthread... yes 2024-06-10T00:34:49.003 INFO:tasks.workunit.client.0.smithi062.stdout:checking for ANSI C header files... (cached) yes 2024-06-10T00:34:49.246 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-06-10T00:34:49.358 INFO:tasks.workunit.client.0.smithi062.stdout:checking pthread.h usability... yes 2024-06-10T00:34:49.377 INFO:tasks.workunit.client.0.smithi062.stdout:checking pthread.h presence... yes 2024-06-10T00:34:49.377 INFO:tasks.workunit.client.0.smithi062.stdout:checking for pthread.h... yes 2024-06-10T00:34:49.448 INFO:tasks.workunit.client.0.smithi062.stdout:checking fcntl.h usability... yes 2024-06-10T00:34:49.465 INFO:tasks.workunit.client.0.smithi062.stdout:checking fcntl.h presence... yes 2024-06-10T00:34:49.465 INFO:tasks.workunit.client.0.smithi062.stdout:checking for fcntl.h... yes 2024-06-10T00:34:49.516 INFO:tasks.workunit.client.0.smithi062.stdout:checking limits.h usability... yes 2024-06-10T00:34:49.535 INFO:tasks.workunit.client.0.smithi062.stdout:checking limits.h presence... yes 2024-06-10T00:34:49.535 INFO:tasks.workunit.client.0.smithi062.stdout:checking for limits.h... yes 2024-06-10T00:34:49.550 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stdint.h... (cached) yes 2024-06-10T00:34:49.591 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/time.h usability... yes 2024-06-10T00:34:49.610 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/time.h presence... yes 2024-06-10T00:34:49.610 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/time.h... yes 2024-06-10T00:34:49.625 INFO:tasks.workunit.client.0.smithi062.stdout:checking for unistd.h... (cached) yes 2024-06-10T00:34:49.636 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:49 smithi062 ceph-mon[32390]: pgmap v62: 129 pgs: 129 active+clean; 5.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 394 KiB/s wr, 48 op/s 2024-06-10T00:34:49.666 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/vfs.h usability... yes 2024-06-10T00:34:49.684 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/vfs.h presence... yes 2024-06-10T00:34:49.684 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/vfs.h... yes 2024-06-10T00:34:49.743 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/limits.h usability... no 2024-06-10T00:34:49.773 INFO:tasks.workunit.client.0.smithi062.stdout:checking sys/limits.h presence... no 2024-06-10T00:34:49.773 INFO:tasks.workunit.client.0.smithi062.stdout:checking for sys/limits.h... no 2024-06-10T00:34:49.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:49 smithi105 ceph-mon[36321]: pgmap v62: 129 pgs: 129 active+clean; 5.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 394 KiB/s wr, 48 op/s 2024-06-10T00:34:49.800 INFO:tasks.workunit.client.0.smithi062.stdout:checking for an ANSI C-conforming const... yes 2024-06-10T00:34:49.830 INFO:tasks.workunit.client.0.smithi062.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-06-10T00:34:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:49 smithi138 ceph-mon[36549]: pgmap v62: 129 pgs: 129 active+clean; 5.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 394 KiB/s wr, 48 op/s 2024-06-10T00:34:49.889 INFO:tasks.workunit.client.0.smithi062.stdout:checking for system... yes 2024-06-10T00:34:49.946 INFO:tasks.workunit.client.0.smithi062.stdout:checking for gettimeofday... yes 2024-06-10T00:34:50.004 INFO:tasks.workunit.client.0.smithi062.stdout:checking for mkdir... yes 2024-06-10T00:34:50.061 INFO:tasks.workunit.client.0.smithi062.stdout:checking for strerror... yes 2024-06-10T00:34:50.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.086+0000 7fbc6e8b2700 1 -- 172.21.15.62:0/3722069252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc6806b520 msgr2=0x7fbc6810f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:50.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.086+0000 7fbc6e8b2700 1 --2- 172.21.15.62:0/3722069252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc6806b520 0x7fbc6810f280 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000b0d0 tx=0x7fbc6000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.086+0000 7fbc6e8b2700 1 -- 172.21.15.62:0/3722069252 shutdown_connections 2024-06-10T00:34:50.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.086+0000 7fbc6e8b2700 1 --2- 172.21.15.62:0/3722069252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc6806b520 0x7fbc6810f280 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.086+0000 7fbc6e8b2700 1 --2- 172.21.15.62:0/3722069252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc68107600 0x7fbc681079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.086+0000 7fbc6e8b2700 1 --2- 172.21.15.62:0/3722069252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6806ab60 0x7fbc6806afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.086+0000 7fbc6e8b2700 1 -- 172.21.15.62:0/3722069252 >> 172.21.15.62:0/3722069252 conn(0x7fbc68078340 msgr2=0x7fbc68078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:50.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.086+0000 7fbc6e8b2700 1 -- 172.21.15.62:0/3722069252 shutdown_connections 2024-06-10T00:34:50.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.086+0000 7fbc6e8b2700 1 -- 172.21.15.62:0/3722069252 wait complete. 2024-06-10T00:34:50.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.087+0000 7fbc6e8b2700 1 Processor -- start 2024-06-10T00:34:50.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.087+0000 7fbc6e8b2700 1 -- start start 2024-06-10T00:34:50.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.087+0000 7fbc6e8b2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6806ab60 0x7fbc6819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.087+0000 7fbc6e8b2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc6806b520 0x7fbc6819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.087+0000 7fbc6e8b2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc68107600 0x7fbc68198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.087+0000 7fbc6e8b2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc68112790 con 0x7fbc68107600 2024-06-10T00:34:50.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.087+0000 7fbc6e8b2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc68112610 con 0x7fbc6806b520 2024-06-10T00:34:50.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.087+0000 7fbc6e8b2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc68112910 con 0x7fbc6806ab60 2024-06-10T00:34:50.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.087+0000 7fbc6ce4f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc68107600 0x7fbc68198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.087+0000 7fbc6ce4f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc68107600 0x7fbc68198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35600/0 (socket says 172.21.15.62:35600) 2024-06-10T00:34:50.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.087+0000 7fbc6ce4f700 1 -- 172.21.15.62:0/1838362803 learned_addr learned my addr 172.21.15.62:0/1838362803 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:50.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.087+0000 7fbc677fe700 1 --2- 172.21.15.62:0/1838362803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc6806b520 0x7fbc6819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.087+0000 7fbc67fff700 1 --2- 172.21.15.62:0/1838362803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6806ab60 0x7fbc6819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.088+0000 7fbc6ce4f700 1 -- 172.21.15.62:0/1838362803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6806ab60 msgr2=0x7fbc6819e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.088+0000 7fbc6ce4f700 1 --2- 172.21.15.62:0/1838362803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6806ab60 0x7fbc6819e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.088+0000 7fbc6ce4f700 1 -- 172.21.15.62:0/1838362803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc6806b520 msgr2=0x7fbc6819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.088+0000 7fbc6ce4f700 1 --2- 172.21.15.62:0/1838362803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc6806b520 0x7fbc6819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.088+0000 7fbc6ce4f700 1 -- 172.21.15.62:0/1838362803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc68199010 con 0x7fbc68107600 2024-06-10T00:34:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.088+0000 7fbc6ce4f700 1 --2- 172.21.15.62:0/1838362803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc68107600 0x7fbc68198830 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009850 tx=0x7fbc60007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.088+0000 7fbc657fa700 1 -- 172.21.15.62:0/1838362803 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60021020 con 0x7fbc68107600 2024-06-10T00:34:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.088+0000 7fbc657fa700 1 -- 172.21.15.62:0/1838362803 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc60004c50 con 0x7fbc68107600 2024-06-10T00:34:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.088+0000 7fbc6e8b2700 1 -- 172.21.15.62:0/1838362803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc681992a0 con 0x7fbc68107600 2024-06-10T00:34:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.088+0000 7fbc657fa700 1 -- 172.21.15.62:0/1838362803 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc600052b0 con 0x7fbc68107600 2024-06-10T00:34:50.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.088+0000 7fbc6e8b2700 1 -- 172.21.15.62:0/1838362803 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc681a5510 con 0x7fbc68107600 2024-06-10T00:34:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.090+0000 7fbc657fa700 1 -- 172.21.15.62:0/1838362803 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbc60024020 con 0x7fbc68107600 2024-06-10T00:34:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.091+0000 7fbc657fa700 1 --2- 172.21.15.62:0/1838362803 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbc500a2eb0 0x7fbc500a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.091+0000 7fbc657fa700 1 -- 172.21.15.62:0/1838362803 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fbc600d0970 con 0x7fbc68107600 2024-06-10T00:34:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.091+0000 7fbc67fff700 1 --2- 172.21.15.62:0/1838362803 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbc500a2eb0 0x7fbc500a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.092+0000 7fbc67fff700 1 --2- 172.21.15.62:0/1838362803 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbc500a2eb0 0x7fbc500a5370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009c60 tx=0x7fbc58009260 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.092+0000 7fbc79135b80 1 Processor -- start 2024-06-10T00:34:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.092+0000 7fbc79135b80 1 -- start start 2024-06-10T00:34:50.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc79135b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f230590 0x55a44f230970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc79135b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 0x55a44f22a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc79135b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f22ace0 0x55a44f22b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc79135b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a44f11d0f0 con 0x55a44f230eb0 2024-06-10T00:34:50.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc79135b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a44f17b9e0 con 0x55a44f22ace0 2024-06-10T00:34:50.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc79135b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a44f0e35f0 con 0x55a44f230590 2024-06-10T00:34:50.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc677fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 0x55a44f22a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc677fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 0x55a44f22a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35602/0 (socket says 172.21.15.62:35602) 2024-06-10T00:34:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc677fe700 1 -- 172.21.15.62:0/2199834878 learned_addr learned my addr 172.21.15.62:0/2199834878 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc67fff700 1 --2- 172.21.15.62:0/2199834878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f230590 0x55a44f230970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc6ce4f700 1 --2- 172.21.15.62:0/2199834878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f22ace0 0x55a44f22b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc6ce4f700 1 -- 172.21.15.62:0/2199834878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f230590 msgr2=0x55a44f230970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc6ce4f700 1 --2- 172.21.15.62:0/2199834878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f230590 0x55a44f230970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc6ce4f700 1 -- 172.21.15.62:0/2199834878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 msgr2=0x55a44f22a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc6ce4f700 1 --2- 172.21.15.62:0/2199834878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 0x55a44f22a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc6ce4f700 1 -- 172.21.15.62:0/2199834878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a44f0caa60 con 0x55a44f22ace0 2024-06-10T00:34:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc6ce4f700 1 --2- 172.21.15.62:0/2199834878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f22ace0 0x55a44f22b100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc600cdcb0 tx=0x7fbc6009ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3edf1c69d809386e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.093+0000 7fbc4e7fc700 1 -- 172.21.15.62:0/2199834878 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6009cb10 con 0x55a44f22ace0 2024-06-10T00:34:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.094+0000 7fbc4e7fc700 1 -- 172.21.15.62:0/2199834878 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc600a0ec0 con 0x55a44f22ace0 2024-06-10T00:34:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.094+0000 7fbc4e7fc700 1 -- 172.21.15.62:0/2199834878 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6009cd70 con 0x55a44f22ace0 2024-06-10T00:34:50.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.094+0000 7fbc79135b80 1 -- 172.21.15.62:0/2199834878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f22ace0 msgr2=0x55a44f22b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:50.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.094+0000 7fbc79135b80 1 --2- 172.21.15.62:0/2199834878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f22ace0 0x55a44f22b100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc600cdcb0 tx=0x7fbc6009ca00 comp rx=0 tx=0).stop 2024-06-10T00:34:50.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.094+0000 7fbc79135b80 1 -- 172.21.15.62:0/2199834878 shutdown_connections 2024-06-10T00:34:50.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.094+0000 7fbc79135b80 1 --2- 172.21.15.62:0/2199834878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f22ace0 0x55a44f22b100 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.094+0000 7fbc79135b80 1 --2- 172.21.15.62:0/2199834878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f230590 0x55a44f230970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.094+0000 7fbc79135b80 1 --2- 172.21.15.62:0/2199834878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 0x55a44f22a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.094+0000 7fbc79135b80 1 -- 172.21.15.62:0/2199834878 >> 172.21.15.62:0/2199834878 conn(0x55a44f157210 msgr2=0x55a44f225a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.094+0000 7fbc79135b80 1 -- 172.21.15.62:0/2199834878 shutdown_connections 2024-06-10T00:34:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.094+0000 7fbc79135b80 1 -- 172.21.15.62:0/2199834878 wait complete. 2024-06-10T00:34:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.095+0000 7fbc79135b80 1 -- start start 2024-06-10T00:34:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.095+0000 7fbc79135b80 1 Processor -- start 2024-06-10T00:34:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.095+0000 7fbc79135b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f22ace0 0x55a44f236d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.095+0000 7fbc79135b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230590 0x55a44f234020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.095+0000 7fbc79135b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f230eb0 0x55a44f234560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.095+0000 7fbc79135b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a44f11c150 con 0x55a44f230590 2024-06-10T00:34:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc79135b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a44f1779a0 con 0x55a44f230eb0 2024-06-10T00:34:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc79135b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a44f237640 con 0x55a44f22ace0 2024-06-10T00:34:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc79135b80 10 client.0 _ll_register_callbacks cb 0x55a44f111740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc677fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230590 0x55a44f234020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc6ce4f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f230eb0 0x55a44f234560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc6ce4f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f230eb0 0x55a44f234560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:42826/0 (socket says 172.21.15.62:42826) 2024-06-10T00:34:50.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc6ce4f700 1 -- 172.21.15.62:0/3125454303 learned_addr learned my addr 172.21.15.62:0/3125454303 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:50.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc677fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230590 0x55a44f234020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35604/0 (socket says 172.21.15.62:35604) 2024-06-10T00:34:50.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc67fff700 1 --2- 172.21.15.62:0/3125454303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f22ace0 0x55a44f236d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc6ce4f700 1 -- 172.21.15.62:0/3125454303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f22ace0 msgr2=0x55a44f236d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:50.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc6ce4f700 1 --2- 172.21.15.62:0/3125454303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f22ace0 0x55a44f236d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc6ce4f700 1 -- 172.21.15.62:0/3125454303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230590 msgr2=0x55a44f234020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:50.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc6ce4f700 1 --2- 172.21.15.62:0/3125454303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230590 0x55a44f234020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc6ce4f700 1 -- 172.21.15.62:0/3125454303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a44f1418e0 con 0x55a44f230eb0 2024-06-10T00:34:50.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc67fff700 1 --2- 172.21.15.62:0/3125454303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f22ace0 0x55a44f236d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:50.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc6ce4f700 1 --2- 172.21.15.62:0/3125454303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f230eb0 0x55a44f234560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbc60099490 tx=0x7fbc60009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11dd34935bd9011d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:50.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc477fe700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:34:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc477fe700 1 -- 172.21.15.62:0/3125454303 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6007f020 con 0x55a44f230eb0 2024-06-10T00:34:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc477fe700 1 -- 172.21.15.62:0/3125454303 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fbc600074e0 con 0x55a44f230eb0 2024-06-10T00:34:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.096+0000 7fbc79135b80 1 -- 172.21.15.62:0/3125454303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc600a0ec0 con 0x55a44f230eb0 2024-06-10T00:34:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.097+0000 7fbc477fe700 1 -- 172.21.15.62:0/3125454303 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbc600aa050 con 0x55a44f230eb0 2024-06-10T00:34:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.097+0000 7fbc79135b80 10 client.25538 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.097+0000 7fbc79135b80 1 -- 172.21.15.62:0/3125454303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a44f241d60 con 0x55a44f230eb0 2024-06-10T00:34:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.097+0000 7fbc477fe700 1 -- 172.21.15.62:0/3125454303 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc600a0ec0 con 0x55a44f230eb0 2024-06-10T00:34:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.097+0000 7fbc477fe700 1 -- 172.21.15.62:0/3125454303 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fbc60011070 con 0x55a44f230eb0 2024-06-10T00:34:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.097+0000 7fbc79135b80 10 client.25538 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.097+0000 7fbc79135b80 10 client.25538 resolve_mds: validated gid 24464 aka daemon mds.b 2024-06-10T00:34:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.097+0000 7fbc79135b80 20 client.25538 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.097+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3125454303 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a44f244230 0x55a44f246620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.097+0000 7fbc79135b80 4 client.25538 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:34:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.097+0000 7fbc79135b80 1 -- 172.21.15.62:0/3125454303 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a44f188ee0 con 0x55a44f244230 2024-06-10T00:34:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.097+0000 7fbc67fff700 1 --2- 172.21.15.62:0/3125454303 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a44f244230 0x55a44f246620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.098+0000 7fbc67fff700 1 --2- 172.21.15.62:0/3125454303 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a44f244230 0x55a44f246620 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=935ca26cffe71d35 server_cookie=b8deaeb163c61ea1 in_seq=0 out_seq=0 2024-06-10T00:34:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.098+0000 7fbc477fe700 10 client.25538 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.098+0000 7fbc477fe700 1 -- 172.21.15.62:0/3125454303 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a44f188ee0 con 0x55a44f244230 2024-06-10T00:34:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.098+0000 7fbc477fe700 10 client.25538 handle_command_reply: tid=0 2024-06-10T00:34:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.098+0000 7fbc79135b80 1 client.25538 shutdown 2024-06-10T00:34:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.099+0000 7fbc79135b80 1 -- 172.21.15.62:0/3125454303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f230eb0 msgr2=0x55a44f234560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.099+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3125454303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f230eb0 0x55a44f234560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbc60099490 tx=0x7fbc60009650 comp rx=0 tx=0).stop 2024-06-10T00:34:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.099+0000 7fbc79135b80 1 -- 172.21.15.62:0/3125454303 shutdown_connections 2024-06-10T00:34:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.099+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3125454303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f230eb0 0x55a44f234560 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.099+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3125454303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f22ace0 0x55a44f236d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.099+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3125454303 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a44f244230 0x55a44f246620 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.100+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3125454303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230590 0x55a44f234020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.100+0000 7fbc477fe700 0 client.25538 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.100+0000 7fbc79135b80 1 -- 172.21.15.62:0/3125454303 >> 172.21.15.62:0/3125454303 conn(0x55a44f157210 msgr2=0x55a44f224c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.100+0000 7fbc79135b80 1 -- 172.21.15.62:0/3125454303 shutdown_connections 2024-06-10T00:34:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.100+0000 7fbc79135b80 1 -- 172.21.15.62:0/3125454303 wait complete. 2024-06-10T00:34:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.100+0000 7fbc79135b80 20 client.25538 trim_cache size 0 max 16384 2024-06-10T00:34:50.107 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:50.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.121+0000 7fbc79135b80 1 Processor -- start 2024-06-10T00:34:50.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.121+0000 7fbc79135b80 1 -- start start 2024-06-10T00:34:50.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.121+0000 7fbc79135b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f22ace0 0x55a44f236860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.121+0000 7fbc79135b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f230590 0x55a44f2253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.121+0000 7fbc79135b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 0x55a44f225900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.121+0000 7fbc79135b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc681ac130 con 0x55a44f230eb0 2024-06-10T00:34:50.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.121+0000 7fbc79135b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc681adc70 con 0x55a44f22ace0 2024-06-10T00:34:50.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.121+0000 7fbc79135b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc681ac800 con 0x55a44f230590 2024-06-10T00:34:50.122 INFO:tasks.workunit.client.0.smithi062.stdout:checking for open64... yes 2024-06-10T00:34:50.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.121+0000 7fbc6ce4f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 0x55a44f225900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.121+0000 7fbc67fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f22ace0 0x55a44f236860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.121+0000 7fbc6ce4f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 0x55a44f225900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35616/0 (socket says 172.21.15.62:35616) 2024-06-10T00:34:50.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.121+0000 7fbc67fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f22ace0 0x55a44f236860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:42840/0 (socket says 172.21.15.62:42840) 2024-06-10T00:34:50.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.121+0000 7fbc6ce4f700 1 -- 172.21.15.62:0/3295276841 learned_addr learned my addr 172.21.15.62:0/3295276841 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:50.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.121+0000 7fbc677fe700 1 --2- 172.21.15.62:0/3295276841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f230590 0x55a44f2253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.122+0000 7fbc67fff700 1 -- 172.21.15.62:0/3295276841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f230590 msgr2=0x55a44f2253c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:50.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.122+0000 7fbc67fff700 1 --2- 172.21.15.62:0/3295276841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f230590 0x55a44f2253c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.122+0000 7fbc67fff700 1 -- 172.21.15.62:0/3295276841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 msgr2=0x55a44f225900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:50.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.122+0000 7fbc67fff700 1 --2- 172.21.15.62:0/3295276841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 0x55a44f225900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.122+0000 7fbc67fff700 1 -- 172.21.15.62:0/3295276841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a44f233df0 con 0x55a44f22ace0 2024-06-10T00:34:50.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.122+0000 7fbc67fff700 1 --2- 172.21.15.62:0/3295276841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f22ace0 0x55a44f236860 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc6009c7c0 tx=0x7fbc58014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f99dca09e61f4ac server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:50.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.122+0000 7fbc4dffb700 1 -- 172.21.15.62:0/3295276841 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58003f40 con 0x55a44f22ace0 2024-06-10T00:34:50.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.122+0000 7fbc4dffb700 1 -- 172.21.15.62:0/3295276841 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc5801d8f0 con 0x55a44f22ace0 2024-06-10T00:34:50.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.122+0000 7fbc4dffb700 1 -- 172.21.15.62:0/3295276841 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc580297a0 con 0x55a44f22ace0 2024-06-10T00:34:50.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.123+0000 7fbc79135b80 1 -- 172.21.15.62:0/3295276841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f22ace0 msgr2=0x55a44f236860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:50.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.123+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3295276841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f22ace0 0x55a44f236860 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc6009c7c0 tx=0x7fbc58014000 comp rx=0 tx=0).stop 2024-06-10T00:34:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.123+0000 7fbc79135b80 1 -- 172.21.15.62:0/3295276841 shutdown_connections 2024-06-10T00:34:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.123+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3295276841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 0x55a44f225900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.123+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3295276841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f22ace0 0x55a44f236860 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.123+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3295276841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f230590 0x55a44f2253c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.123+0000 7fbc79135b80 1 -- 172.21.15.62:0/3295276841 >> 172.21.15.62:0/3295276841 conn(0x55a44f157210 msgr2=0x55a44f22f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:50.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.123+0000 7fbc79135b80 1 -- 172.21.15.62:0/3295276841 shutdown_connections 2024-06-10T00:34:50.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.123+0000 7fbc79135b80 1 -- 172.21.15.62:0/3295276841 wait complete. 2024-06-10T00:34:50.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.124+0000 7fbc79135b80 1 -- start start 2024-06-10T00:34:50.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.124+0000 7fbc79135b80 1 Processor -- start 2024-06-10T00:34:50.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc79135b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f22ace0 0x55a44f23c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc79135b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f230590 0x55a44f23ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc79135b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 0x55a44f239760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc79135b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc681ad5a0 con 0x55a44f230eb0 2024-06-10T00:34:50.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc79135b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc681aced0 con 0x55a44f230590 2024-06-10T00:34:50.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc79135b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc681aa0d0 con 0x55a44f22ace0 2024-06-10T00:34:50.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc79135b80 10 client.0 _ll_register_callbacks cb 0x55a44f233490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:50.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc67fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f22ace0 0x55a44f23c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc677fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f230590 0x55a44f23ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc67fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f22ace0 0x55a44f23c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:39752/0 (socket says 172.21.15.62:39752) 2024-06-10T00:34:50.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc67fff700 1 -- 172.21.15.62:0/3408403950 learned_addr learned my addr 172.21.15.62:0/3408403950 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:50.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc6ce4f700 1 --2- 172.21.15.62:0/3408403950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 0x55a44f239760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc67fff700 1 -- 172.21.15.62:0/3408403950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f230590 msgr2=0x55a44f23ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:50.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc67fff700 1 --2- 172.21.15.62:0/3408403950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f230590 0x55a44f23ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc67fff700 1 -- 172.21.15.62:0/3408403950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 msgr2=0x55a44f239760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:50.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc67fff700 1 --2- 172.21.15.62:0/3408403950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 0x55a44f239760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:50.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc67fff700 1 -- 172.21.15.62:0/3408403950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a44f239ca0 con 0x55a44f22ace0 2024-06-10T00:34:50.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc67fff700 1 --2- 172.21.15.62:0/3408403950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f22ace0 0x55a44f23c790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009fd0 tx=0x7fbc58003d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e54bdd2b1a1eacc4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:50.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.125+0000 7fbc477fe700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:34:50.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.126+0000 7fbc477fe700 1 -- 172.21.15.62:0/3408403950 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5801d410 con 0x55a44f22ace0 2024-06-10T00:34:50.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.126+0000 7fbc79135b80 1 -- 172.21.15.62:0/3408403950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc5801d8f0 con 0x55a44f22ace0 2024-06-10T00:34:50.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.126+0000 7fbc477fe700 1 -- 172.21.15.62:0/3408403950 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fbc5801d5b0 con 0x55a44f22ace0 2024-06-10T00:34:50.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.126+0000 7fbc477fe700 1 -- 172.21.15.62:0/3408403950 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbc5801daa0 con 0x55a44f22ace0 2024-06-10T00:34:50.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.126+0000 7fbc79135b80 10 client.25561 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:50.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.126+0000 7fbc79135b80 1 -- 172.21.15.62:0/3408403950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a44f2af660 con 0x55a44f22ace0 2024-06-10T00:34:50.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.126+0000 7fbc477fe700 1 -- 172.21.15.62:0/3408403950 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc5801d8f0 con 0x55a44f22ace0 2024-06-10T00:34:50.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.126+0000 7fbc477fe700 1 -- 172.21.15.62:0/3408403950 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fbc58030070 con 0x55a44f22ace0 2024-06-10T00:34:50.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.126+0000 7fbc79135b80 10 client.25561 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:50.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.126+0000 7fbc79135b80 10 client.25561 resolve_mds: validated gid 24464 aka daemon mds.b 2024-06-10T00:34:50.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.126+0000 7fbc79135b80 20 client.25561 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:50.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.126+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3408403950 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a44f244230 0x55a44f2b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:50.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.126+0000 7fbc79135b80 4 client.25561 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-10T00:34:50.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.126+0000 7fbc79135b80 1 -- 172.21.15.62:0/3408403950 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55a44f183460 con 0x55a44f244230 2024-06-10T00:34:50.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.127+0000 7fbc677fe700 1 --2- 172.21.15.62:0/3408403950 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a44f244230 0x55a44f2b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:50.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.127+0000 7fbc477fe700 10 client.25561 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:50.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:50.127+0000 7fbc677fe700 1 --2- 172.21.15.62:0/3408403950 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a44f244230 0x55a44f2b1d50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62db24c9372d2abf server_cookie=f40a0d4f2fbb124b in_seq=0 out_seq=0 2024-06-10T00:34:51.711 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:51 smithi062 ceph-mon[32390]: pgmap v63: 129 pgs: 129 active+clean; 5.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 394 KiB/s wr, 48 op/s 2024-06-10T00:34:51.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:51 smithi105 ceph-mon[36321]: pgmap v63: 129 pgs: 129 active+clean; 5.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 394 KiB/s wr, 48 op/s 2024-06-10T00:34:51.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:51 smithi138 ceph-mon[36549]: pgmap v63: 129 pgs: 129 active+clean; 5.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 394 KiB/s wr, 48 op/s 2024-06-10T00:34:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.059+0000 7f71254dc700 1 -- 172.21.15.62:0/3986470422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7120100a80 msgr2=0x7f7120100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.059+0000 7f71254dc700 1 --2- 172.21.15.62:0/3986470422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7120100a80 0x7f7120100f00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7108009990 tx=0x7f7108009ca0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.059+0000 7f71254dc700 1 -- 172.21.15.62:0/3986470422 shutdown_connections 2024-06-10T00:34:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.059+0000 7f71254dc700 1 --2- 172.21.15.62:0/3986470422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7120101440 0x7f712010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.059+0000 7f71254dc700 1 --2- 172.21.15.62:0/3986470422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7120108510 0x7f71201088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.059+0000 7f71254dc700 1 --2- 172.21.15.62:0/3986470422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7120100a80 0x7f7120100f00 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.059+0000 7f71254dc700 1 -- 172.21.15.62:0/3986470422 >> 172.21.15.62:0/3986470422 conn(0x7f7120077a00 msgr2=0x7f71200fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:52.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.059+0000 7f71254dc700 1 -- 172.21.15.62:0/3986470422 shutdown_connections 2024-06-10T00:34:52.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.059+0000 7f71254dc700 1 -- 172.21.15.62:0/3986470422 wait complete. 2024-06-10T00:34:52.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.060+0000 7f71254dc700 1 Processor -- start 2024-06-10T00:34:52.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.060+0000 7f71254dc700 1 -- start start 2024-06-10T00:34:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.060+0000 7f71254dc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7120100a80 0x7f7120101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.060+0000 7f71254dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7120101440 0x7f71201021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.060+0000 7f71254dc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7120108510 0x7f71201026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.060+0000 7f71254dc700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7120112790 con 0x7f7120100a80 2024-06-10T00:34:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.060+0000 7f71254dc700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7120112610 con 0x7f7120108510 2024-06-10T00:34:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.060+0000 7f71254dc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7120112910 con 0x7f7120101440 2024-06-10T00:34:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f711effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7120100a80 0x7f7120101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f71167fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7120101440 0x7f71201021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f711effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7120100a80 0x7f7120101c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35634/0 (socket says 172.21.15.62:35634) 2024-06-10T00:34:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f711effd700 1 -- 172.21.15.62:0/1270102370 learned_addr learned my addr 172.21.15.62:0/1270102370 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f711f7fe700 1 --2- 172.21.15.62:0/1270102370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7120108510 0x7f71201026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:52.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f711effd700 1 -- 172.21.15.62:0/1270102370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7120101440 msgr2=0x7f71201021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:52.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f711effd700 1 --2- 172.21.15.62:0/1270102370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7120101440 0x7f71201021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f711effd700 1 -- 172.21.15.62:0/1270102370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7120108510 msgr2=0x7f71201026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:52.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f711effd700 1 --2- 172.21.15.62:0/1270102370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7120108510 0x7f71201026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f711effd700 1 -- 172.21.15.62:0/1270102370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71201a7120 con 0x7f7120100a80 2024-06-10T00:34:52.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f711effd700 1 --2- 172.21.15.62:0/1270102370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7120100a80 0x7f7120101c60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f711000ef30 tx=0x7f711000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:52.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f711cff9700 1 -- 172.21.15.62:0/1270102370 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71100092e0 con 0x7f7120100a80 2024-06-10T00:34:52.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f711cff9700 1 -- 172.21.15.62:0/1270102370 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7110004bb0 con 0x7f7120100a80 2024-06-10T00:34:52.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f71254dc700 1 -- 172.21.15.62:0/1270102370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71201a73b0 con 0x7f7120100a80 2024-06-10T00:34:52.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f711cff9700 1 -- 172.21.15.62:0/1270102370 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110004280 con 0x7f7120100a80 2024-06-10T00:34:52.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.061+0000 7f71254dc700 1 -- 172.21.15.62:0/1270102370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71201a7870 con 0x7f7120100a80 2024-06-10T00:34:52.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.062+0000 7f71254dc700 1 -- 172.21.15.62:0/1270102370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f712010a9f0 con 0x7f7120100a80 2024-06-10T00:34:52.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.064+0000 7f711cff9700 1 -- 172.21.15.62:0/1270102370 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f711001b030 con 0x7f7120100a80 2024-06-10T00:34:52.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.065+0000 7f711cff9700 1 --2- 172.21.15.62:0/1270102370 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f710c0a2d90 0x7f710c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:52.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.065+0000 7f71167fc700 1 --2- 172.21.15.62:0/1270102370 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f710c0a2d90 0x7f710c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:52.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.065+0000 7f711cff9700 1 -- 172.21.15.62:0/1270102370 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f71100cfc70 con 0x7f7120100a80 2024-06-10T00:34:52.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.065+0000 7f71167fc700 1 --2- 172.21.15.62:0/1270102370 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f710c0a2d90 0x7f710c0a5250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f710800b2e0 tx=0x7f7108005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:52.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.069+0000 7f711cff9700 1 -- 172.21.15.62:0/1270102370 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7110022020 con 0x7f7120100a80 2024-06-10T00:34:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.290+0000 7fbc477fe700 1 -- 172.21.15.62:0/3408403950 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a44f183460 con 0x55a44f244230 2024-06-10T00:34:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.290+0000 7fbc477fe700 10 client.25561 handle_command_reply: tid=0 2024-06-10T00:34:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.290+0000 7fbc79135b80 1 client.25561 shutdown 2024-06-10T00:34:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.290+0000 7fbc79135b80 1 -- 172.21.15.62:0/3408403950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f22ace0 msgr2=0x55a44f23c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.290+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3408403950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f22ace0 0x55a44f23c790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009fd0 tx=0x7fbc58003d40 comp rx=0 tx=0).stop 2024-06-10T00:34:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.290+0000 7fbc79135b80 1 -- 172.21.15.62:0/3408403950 shutdown_connections 2024-06-10T00:34:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.290+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3408403950 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a44f244230 0x55a44f2b1d50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.290+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3408403950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a44f230eb0 0x55a44f239760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.290+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3408403950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a44f22ace0 0x55a44f23c790 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.290+0000 7fbc79135b80 1 --2- 172.21.15.62:0/3408403950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a44f230590 0x55a44f23ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.291+0000 7fbc477fe700 0 client.25561 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:34:52.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.291+0000 7fbc79135b80 1 -- 172.21.15.62:0/3408403950 >> 172.21.15.62:0/3408403950 conn(0x55a44f157210 msgr2=0x55a44f229790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:52.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.291+0000 7fbc79135b80 1 -- 172.21.15.62:0/3408403950 shutdown_connections 2024-06-10T00:34:52.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.291+0000 7fbc79135b80 1 -- 172.21.15.62:0/3408403950 wait complete. 2024-06-10T00:34:52.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.291+0000 7fbc79135b80 20 client.25561 trim_cache size 0 max 16384 2024-06-10T00:34:52.292 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:34:52.293 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 30, 2024-06-10T00:34:52.293 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24464, 2024-06-10T00:34:52.293 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 6, 2024-06-10T00:34:52.293 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-06-10T00:34:52.293 INFO:teuthology.orchestra.run.smithi062.stdout: "547e97b4": { 2024-06-10T00:34:52.293 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 6, 2024-06-10T00:34:52.293 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-06-10T00:34:52.293 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-06-10T00:34:52.293 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-06-10T00:34:52.293 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-06-10T00:34:52.294 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:34:52.294 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-06-10T00:34:52.294 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 150.0, 2024-06-10T00:34:52.294 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-06-10T00:34:52.294 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-06-10T00:34:52.294 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-06-10T00:34:52.294 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-06-10T00:34:52.294 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-06-10T00:34:52.294 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-06-10T00:34:52.294 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:52.294 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:52.294 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:52.294 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:52.294 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:34:52.295 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:34:52.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.291+0000 7fbc6e8b2700 1 -- 172.21.15.62:0/1838362803 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbc500a2eb0 msgr2=0x7fbc500a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.291+0000 7fbc6e8b2700 1 --2- 172.21.15.62:0/1838362803 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbc500a2eb0 0x7fbc500a5370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009c60 tx=0x7fbc58009260 comp rx=0 tx=0).stop 2024-06-10T00:34:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.292+0000 7fbc6e8b2700 1 -- 172.21.15.62:0/1838362803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc68107600 msgr2=0x7fbc68198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.292+0000 7fbc6e8b2700 1 --2- 172.21.15.62:0/1838362803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc68107600 0x7fbc68198830 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009850 tx=0x7fbc60007a20 comp rx=0 tx=0).stop 2024-06-10T00:34:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.292+0000 7fbc6e8b2700 1 -- 172.21.15.62:0/1838362803 shutdown_connections 2024-06-10T00:34:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.292+0000 7fbc6e8b2700 1 --2- 172.21.15.62:0/1838362803 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbc500a2eb0 0x7fbc500a5370 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.292+0000 7fbc6e8b2700 1 --2- 172.21.15.62:0/1838362803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc68107600 0x7fbc68198830 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.292+0000 7fbc6e8b2700 1 --2- 172.21.15.62:0/1838362803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6806ab60 0x7fbc6819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.292+0000 7fbc6e8b2700 1 --2- 172.21.15.62:0/1838362803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc6806b520 0x7fbc6819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.292+0000 7fbc6e8b2700 1 -- 172.21.15.62:0/1838362803 >> 172.21.15.62:0/1838362803 conn(0x7fbc68078340 msgr2=0x7fbc68100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.292+0000 7fbc6e8b2700 1 -- 172.21.15.62:0/1838362803 shutdown_connections 2024-06-10T00:34:52.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.292+0000 7fbc6e8b2700 1 -- 172.21.15.62:0/1838362803 wait complete. 2024-06-10T00:34:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.341+0000 7f71254dc700 1 -- 172.21.15.62:0/1270102370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7120103140 con 0x7f7120100a80 2024-06-10T00:34:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.342+0000 7f711cff9700 1 -- 172.21.15.62:0/1270102370 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f7110010040 con 0x7f7120100a80 2024-06-10T00:34:52.342 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:34:52.342 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:32:47.783154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:34:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.344+0000 7f71254dc700 1 -- 172.21.15.62:0/1270102370 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f710c0a2d90 msgr2=0x7f710c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.344+0000 7f71254dc700 1 --2- 172.21.15.62:0/1270102370 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f710c0a2d90 0x7f710c0a5250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f710800b2e0 tx=0x7f7108005960 comp rx=0 tx=0).stop 2024-06-10T00:34:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.344+0000 7f71254dc700 1 -- 172.21.15.62:0/1270102370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7120100a80 msgr2=0x7f7120101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.344+0000 7f71254dc700 1 --2- 172.21.15.62:0/1270102370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7120100a80 0x7f7120101c60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f711000ef30 tx=0x7f711000c3a0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.344+0000 7f71254dc700 1 -- 172.21.15.62:0/1270102370 shutdown_connections 2024-06-10T00:34:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.344+0000 7f71254dc700 1 --2- 172.21.15.62:0/1270102370 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f710c0a2d90 0x7f710c0a5250 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.344+0000 7f71254dc700 1 --2- 172.21.15.62:0/1270102370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7120100a80 0x7f7120101c60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.344+0000 7f71254dc700 1 --2- 172.21.15.62:0/1270102370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7120108510 0x7f71201026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.344+0000 7f71254dc700 1 --2- 172.21.15.62:0/1270102370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7120101440 0x7f71201021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.344+0000 7f71254dc700 1 -- 172.21.15.62:0/1270102370 >> 172.21.15.62:0/1270102370 conn(0x7f7120077a00 msgr2=0x7f71200ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.345+0000 7f71254dc700 1 -- 172.21.15.62:0/1270102370 shutdown_connections 2024-06-10T00:34:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:52.345+0000 7f71254dc700 1 -- 172.21.15.62:0/1270102370 wait complete. 2024-06-10T00:34:52.346 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-06-10T00:34:52.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:52 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1270102370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:52.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:52 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1270102370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:52.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:52 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1270102370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:34:53.347 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T00:34:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:53 smithi062 ceph-mon[32390]: pgmap v64: 129 pgs: 129 active+clean; 6.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 526 KiB/s wr, 63 op/s 2024-06-10T00:34:53.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:53 smithi105 ceph-mon[36321]: pgmap v64: 129 pgs: 129 active+clean; 6.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 526 KiB/s wr, 63 op/s 2024-06-10T00:34:53.813 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '547e97b4', quiesce took 13.3 seconds. Will release after: 16.7 2024-06-10T00:34:53.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:53 smithi138 ceph-mon[36549]: pgmap v64: 129 pgs: 129 active+clean; 6.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 526 KiB/s wr, 63 op/s 2024-06-10T00:34:53.882 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:55.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:55 smithi062 ceph-mon[32390]: pgmap v65: 129 pgs: 129 active+clean; 6.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 398 KiB/s wr, 52 op/s 2024-06-10T00:34:55.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:55 smithi105 ceph-mon[36321]: pgmap v65: 129 pgs: 129 active+clean; 6.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 398 KiB/s wr, 52 op/s 2024-06-10T00:34:55.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:55 smithi138 ceph-mon[36549]: pgmap v65: 129 pgs: 129 active+clean; 6.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 398 KiB/s wr, 52 op/s 2024-06-10T00:34:57.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:57 smithi062 ceph-mon[32390]: pgmap v66: 129 pgs: 129 active+clean; 7.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 466 KiB/s wr, 73 op/s 2024-06-10T00:34:57.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:57 smithi105 ceph-mon[36321]: pgmap v66: 129 pgs: 129 active+clean; 7.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 466 KiB/s wr, 73 op/s 2024-06-10T00:34:57.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:57 smithi138 ceph-mon[36549]: pgmap v66: 129 pgs: 129 active+clean; 7.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 466 KiB/s wr, 73 op/s 2024-06-10T00:34:58.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.352+0000 7f0bdd431700 1 -- 172.21.15.62:0/2536141063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd8107400 msgr2=0x7f0bd81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:58.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.352+0000 7f0bdd431700 1 --2- 172.21.15.62:0/2536141063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd8107400 0x7f0bd81077e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009990 tx=0x7f0bc8009ca0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.352+0000 7f0bdd431700 1 -- 172.21.15.62:0/2536141063 shutdown_connections 2024-06-10T00:34:58.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.352+0000 7f0bdd431700 1 --2- 172.21.15.62:0/2536141063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd8107400 0x7f0bd81077e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.352+0000 7f0bdd431700 1 --2- 172.21.15.62:0/2536141063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd8101d50 0x7f0bd810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.352+0000 7f0bdd431700 1 --2- 172.21.15.62:0/2536141063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bd8101390 0x7f0bd8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.352+0000 7f0bdd431700 1 -- 172.21.15.62:0/2536141063 >> 172.21.15.62:0/2536141063 conn(0x7f0bd8078340 msgr2=0x7f0bd8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:58.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.352+0000 7f0bdd431700 1 -- 172.21.15.62:0/2536141063 shutdown_connections 2024-06-10T00:34:58.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.352+0000 7f0bdd431700 1 -- 172.21.15.62:0/2536141063 wait complete. 2024-06-10T00:34:58.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.352+0000 7f0bdd431700 1 Processor -- start 2024-06-10T00:34:58.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.353+0000 7f0bdd431700 1 -- start start 2024-06-10T00:34:58.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.353+0000 7f0bdd431700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bd8101390 0x7f0bd819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.353+0000 7f0bdd431700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd8101d50 0x7f0bd819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.353+0000 7f0bdd431700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd8107400 0x7f0bd8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.353+0000 7f0bdd431700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81116f0 con 0x7f0bd8101d50 2024-06-10T00:34:58.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.353+0000 7f0bdd431700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd8111570 con 0x7f0bd8101390 2024-06-10T00:34:58.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.353+0000 7f0bdd431700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd8111870 con 0x7f0bd8107400 2024-06-10T00:34:58.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.353+0000 7f0bce7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd8101d50 0x7f0bd819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.353+0000 7f0bce7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd8101d50 0x7f0bd819edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60380/0 (socket says 172.21.15.62:60380) 2024-06-10T00:34:58.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.353+0000 7f0bce7fc700 1 -- 172.21.15.62:0/3663798732 learned_addr learned my addr 172.21.15.62:0/3663798732 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:58.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.353+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bd8101390 0x7f0bd819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.353+0000 7f0bd77fe700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd8107400 0x7f0bd8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.354+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/3663798732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd8107400 msgr2=0x7f0bd8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:58.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.354+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd8107400 0x7f0bd8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.354+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/3663798732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd8101d50 msgr2=0x7f0bd819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:58.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.354+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd8101d50 0x7f0bd819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.354+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/3663798732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd8199190 con 0x7f0bd8101390 2024-06-10T00:34:58.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.354+0000 7f0bce7fc700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd8101d50 0x7f0bd819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:34:58.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.354+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bd8101390 0x7f0bd819e870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80059d0 tx=0x7f0bc80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:58.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.354+0000 7f0bd4ff9700 1 -- 172.21.15.62:0/3663798732 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8027070 con 0x7f0bd8101390 2024-06-10T00:34:58.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.354+0000 7f0bd4ff9700 1 -- 172.21.15.62:0/3663798732 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc8003740 con 0x7f0bd8101390 2024-06-10T00:34:58.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.354+0000 7f0bdd431700 1 -- 172.21.15.62:0/3663798732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd8199420 con 0x7f0bd8101390 2024-06-10T00:34:58.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.354+0000 7f0bd4ff9700 1 -- 172.21.15.62:0/3663798732 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8022bc0 con 0x7f0bd8101390 2024-06-10T00:34:58.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.354+0000 7f0bdd431700 1 -- 172.21.15.62:0/3663798732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd81a5740 con 0x7f0bd8101390 2024-06-10T00:34:58.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.357+0000 7f0bd4ff9700 1 -- 172.21.15.62:0/3663798732 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0bc802a070 con 0x7f0bd8101390 2024-06-10T00:34:58.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.358+0000 7f0bd4ff9700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f0bc40a2d90 0x7f0bc40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.358+0000 7f0bd4ff9700 1 -- 172.21.15.62:0/3663798732 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f0bc80d0650 con 0x7f0bd8101390 2024-06-10T00:34:58.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.358+0000 7f0bce7fc700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f0bc40a2d90 0x7f0bc40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.358+0000 7f0be7cb4b80 1 Processor -- start 2024-06-10T00:34:58.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.358+0000 7f0be7cb4b80 1 -- start start 2024-06-10T00:34:58.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0bce7fc700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f0bc40a2d90 0x7f0bc40a5250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80ffb70 tx=0x7f0bc000b500 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:58.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0be7cb4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3ea640 0x563a6f3eaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0be7cb4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3eaf60 0x563a6f3e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0be7cb4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3e4f20 0x563a6f3e5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0be7cb4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563a6f336810 con 0x563a6f3ea640 2024-06-10T00:34:58.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0be7cb4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563a6f2de720 con 0x563a6f3e4f20 2024-06-10T00:34:58.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0be7cb4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a6f3327d0 con 0x563a6f3eaf60 2024-06-10T00:34:58.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0bd6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3ea640 0x563a6f3eaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0bd6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3ea640 0x563a6f3eaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60396/0 (socket says 172.21.15.62:60396) 2024-06-10T00:34:58.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/4265808626 learned_addr learned my addr 172.21.15.62:0/4265808626 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:58.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0bce7fc700 1 --2- 172.21.15.62:0/4265808626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3eaf60 0x563a6f3e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0bd77fe700 1 --2- 172.21.15.62:0/4265808626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3e4f20 0x563a6f3e5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/4265808626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3eaf60 msgr2=0x563a6f3e49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:58.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/4265808626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3eaf60 0x563a6f3e49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/4265808626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3e4f20 msgr2=0x563a6f3e5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:58.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/4265808626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3e4f20 0x563a6f3e5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/4265808626 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a6f2d4f50 con 0x563a6f3ea640 2024-06-10T00:34:58.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.359+0000 7f0bd77fe700 1 --2- 172.21.15.62:0/4265808626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3e4f20 0x563a6f3e5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:34:58.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.360+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/4265808626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3ea640 0x563a6f3eaa20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8099220 tx=0x7f0bc8022650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ecc94dad759176 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:58.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.360+0000 7f0bcd7fa700 1 -- 172.21.15.62:0/4265808626 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8022470 con 0x563a6f3ea640 2024-06-10T00:34:58.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.360+0000 7f0bcd7fa700 1 -- 172.21.15.62:0/4265808626 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc809cf50 con 0x563a6f3ea640 2024-06-10T00:34:58.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.360+0000 7f0bcd7fa700 1 -- 172.21.15.62:0/4265808626 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80d0280 con 0x563a6f3ea640 2024-06-10T00:34:58.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.360+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/4265808626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3ea640 msgr2=0x563a6f3eaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:58.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.360+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/4265808626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3ea640 0x563a6f3eaa20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8099220 tx=0x7f0bc8022650 comp rx=0 tx=0).stop 2024-06-10T00:34:58.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/4265808626 shutdown_connections 2024-06-10T00:34:58.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/4265808626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3ea640 0x563a6f3eaa20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/4265808626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3e4f20 0x563a6f3e5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/4265808626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3eaf60 0x563a6f3e49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/4265808626 >> 172.21.15.62:0/4265808626 conn(0x563a6f30fcc0 msgr2=0x563a6f3df850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:58.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/4265808626 shutdown_connections 2024-06-10T00:34:58.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/4265808626 wait complete. 2024-06-10T00:34:58.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 -- start start 2024-06-10T00:34:58.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 Processor -- start 2024-06-10T00:34:58.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3e4f20 0x563a6f3f0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3ea640 0x563a6f3ee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3eaf60 0x563a6f3ee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563a6f3f16f0 con 0x563a6f3e4f20 2024-06-10T00:34:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563a6f2d6bf0 con 0x563a6f3eaf60 2024-06-10T00:34:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.361+0000 7f0be7cb4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a6f3f1870 con 0x563a6f3ea640 2024-06-10T00:34:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0be7cb4b80 10 client.0 _ll_register_callbacks cb 0x563a6f2d0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bd6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3e4f20 0x563a6f3f0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bd6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3e4f20 0x563a6f3f0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60404/0 (socket says 172.21.15.62:60404) 2024-06-10T00:34:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/2156644876 learned_addr learned my addr 172.21.15.62:0/2156644876 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bce7fc700 1 --2- 172.21.15.62:0/2156644876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3ea640 0x563a6f3ee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bd77fe700 1 --2- 172.21.15.62:0/2156644876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3eaf60 0x563a6f3ee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/2156644876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3ea640 msgr2=0x563a6f3ee060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/2156644876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3ea640 0x563a6f3ee060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/2156644876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3eaf60 msgr2=0x563a6f3ee5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:58.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/2156644876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3eaf60 0x563a6f3ee5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/2156644876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a6f2858d0 con 0x563a6f3e4f20 2024-06-10T00:34:58.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/2156644876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3e4f20 0x563a6f3f0d60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80cdf70 tx=0x7f0bc80cda10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7bff4a997eb0c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:58.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bbe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:58.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bbe7fc700 1 -- 172.21.15.62:0/2156644876 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8022470 con 0x563a6f3e4f20 2024-06-10T00:34:58.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bbe7fc700 1 -- 172.21.15.62:0/2156644876 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f0bc80870e0 con 0x563a6f3e4f20 2024-06-10T00:34:58.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/2156644876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc809cf50 con 0x563a6f3e4f20 2024-06-10T00:34:58.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0bbe7fc700 1 -- 172.21.15.62:0/2156644876 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0bc80a5430 con 0x563a6f3e4f20 2024-06-10T00:34:58.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0be7cb4b80 10 client.16023 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:58.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.362+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/2156644876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a6f3f16f0 con 0x563a6f3e4f20 2024-06-10T00:34:58.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.363+0000 7f0bbe7fc700 1 -- 172.21.15.62:0/2156644876 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc809cf50 con 0x563a6f3e4f20 2024-06-10T00:34:58.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.363+0000 7f0bbe7fc700 1 -- 172.21.15.62:0/2156644876 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0bc80b0ce0 con 0x563a6f3e4f20 2024-06-10T00:34:58.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.363+0000 7f0be7cb4b80 10 client.16023 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:58.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.363+0000 7f0be7cb4b80 10 client.16023 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:34:58.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.363+0000 7f0be7cb4b80 20 client.16023 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:58.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.363+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/2156644876 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x563a6f3fe3e0 0x563a6f400800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.363+0000 7f0be7cb4b80 4 client.16023 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:34:58.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.363+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/2156644876 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a6f343d10 con 0x563a6f3fe3e0 2024-06-10T00:34:58.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.363+0000 7f0bce7fc700 1 --2- 172.21.15.62:0/2156644876 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x563a6f3fe3e0 0x563a6f400800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.364+0000 7f0bce7fc700 1 --2- 172.21.15.62:0/2156644876 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x563a6f3fe3e0 0x563a6f400800 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=318f7f8d41c0fd8a server_cookie=c65feffc6f8c2b9 in_seq=0 out_seq=0 2024-06-10T00:34:58.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.364+0000 7f0bbe7fc700 10 client.16023 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:34:58.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.364+0000 7f0bbe7fc700 1 -- 172.21.15.62:0/2156644876 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563a6f343d10 con 0x563a6f3fe3e0 2024-06-10T00:34:58.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.364+0000 7f0bbe7fc700 10 client.16023 handle_command_reply: tid=0 2024-06-10T00:34:58.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.365+0000 7f0be7cb4b80 1 client.16023 shutdown 2024-06-10T00:34:58.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.365+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/2156644876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3e4f20 msgr2=0x563a6f3f0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:58.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.365+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/2156644876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3e4f20 0x563a6f3f0d60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80cdf70 tx=0x7f0bc80cda10 comp rx=0 tx=0).stop 2024-06-10T00:34:58.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.365+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/2156644876 shutdown_connections 2024-06-10T00:34:58.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.365+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/2156644876 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x563a6f3fe3e0 0x563a6f400800 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.365+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/2156644876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3e4f20 0x563a6f3f0d60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.365+0000 7f0bbe7fc700 0 client.16023 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:34:58.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.365+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/2156644876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3eaf60 0x563a6f3ee5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.365+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/2156644876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3ea640 0x563a6f3ee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.365+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/2156644876 >> 172.21.15.62:0/2156644876 conn(0x563a6f30fcc0 msgr2=0x563a6f3e39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:58.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.365+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/2156644876 shutdown_connections 2024-06-10T00:34:58.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.365+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/2156644876 wait complete. 2024-06-10T00:34:58.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.365+0000 7f0be7cb4b80 20 client.16023 trim_cache size 0 max 16384 2024-06-10T00:34:58.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.380+0000 7f0be7cb4b80 1 Processor -- start 2024-06-10T00:34:58.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.380+0000 7f0be7cb4b80 1 -- start start 2024-06-10T00:34:58.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0be7cb4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3e4f20 0x563a6f3e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0be7cb4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3ea640 0x563a6f3ebbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0be7cb4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3eaf60 0x563a6f3e91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0be7cb4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81ae510 con 0x563a6f3ea640 2024-06-10T00:34:58.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0be7cb4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81b0050 con 0x563a6f3e4f20 2024-06-10T00:34:58.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0be7cb4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81aebe0 con 0x563a6f3eaf60 2024-06-10T00:34:58.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0bce7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3ea640 0x563a6f3ebbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0bce7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3ea640 0x563a6f3ebbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60418/0 (socket says 172.21.15.62:60418) 2024-06-10T00:34:58.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0bce7fc700 1 -- 172.21.15.62:0/265067171 learned_addr learned my addr 172.21.15.62:0/265067171 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:58.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0bd77fe700 1 --2- 172.21.15.62:0/265067171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3eaf60 0x563a6f3e91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/265067171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3e4f20 0x563a6f3e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0bce7fc700 1 -- 172.21.15.62:0/265067171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3eaf60 msgr2=0x563a6f3e91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:58.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0bce7fc700 1 --2- 172.21.15.62:0/265067171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3eaf60 0x563a6f3e91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0bce7fc700 1 -- 172.21.15.62:0/265067171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3e4f20 msgr2=0x563a6f3e49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:58.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0bce7fc700 1 --2- 172.21.15.62:0/265067171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3e4f20 0x563a6f3e49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0bce7fc700 1 -- 172.21.15.62:0/265067171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a6f3eaa20 con 0x563a6f3ea640 2024-06-10T00:34:58.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/265067171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3e4f20 0x563a6f3e49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:34:58.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.381+0000 7f0bce7fc700 1 --2- 172.21.15.62:0/265067171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3ea640 0x563a6f3ebbd0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x563a6f3ef630 tx=0x7f0bc0013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fa2a2effe693b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:58.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.382+0000 7f0bccff9700 1 -- 172.21.15.62:0/265067171 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0004220 con 0x563a6f3ea640 2024-06-10T00:34:58.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.382+0000 7f0bccff9700 1 -- 172.21.15.62:0/265067171 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc00043c0 con 0x563a6f3ea640 2024-06-10T00:34:58.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.382+0000 7f0bccff9700 1 -- 172.21.15.62:0/265067171 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc002a9d0 con 0x563a6f3ea640 2024-06-10T00:34:58.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.382+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/265067171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3ea640 msgr2=0x563a6f3ebbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:58.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.382+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/265067171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3ea640 0x563a6f3ebbd0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x563a6f3ef630 tx=0x7f0bc0013070 comp rx=0 tx=0).stop 2024-06-10T00:34:58.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.382+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/265067171 shutdown_connections 2024-06-10T00:34:58.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.382+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/265067171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3eaf60 0x563a6f3e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.382+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/265067171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3e4f20 0x563a6f3e49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.382+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/265067171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3ea640 0x563a6f3ebbd0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.382+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/265067171 >> 172.21.15.62:0/265067171 conn(0x563a6f30fcc0 msgr2=0x563a6f3d9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:58.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/265067171 shutdown_connections 2024-06-10T00:34:58.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/265067171 wait complete. 2024-06-10T00:34:58.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0be7cb4b80 1 -- start start 2024-06-10T00:34:58.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0be7cb4b80 1 Processor -- start 2024-06-10T00:34:58.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0be7cb4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3e4f20 0x563a6f405b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0be7cb4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3ea640 0x563a6f4060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0be7cb4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3eaf60 0x563a6f406600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0be7cb4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81af980 con 0x563a6f3e4f20 2024-06-10T00:34:58.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0be7cb4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81af2b0 con 0x563a6f3eaf60 2024-06-10T00:34:58.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0be7cb4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81ac4b0 con 0x563a6f3ea640 2024-06-10T00:34:58.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0be7cb4b80 10 client.0 _ll_register_callbacks cb 0x563a6f2d0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:34:58.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0bd6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3e4f20 0x563a6f405b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0bce7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3ea640 0x563a6f4060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0bd6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3e4f20 0x563a6f405b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60424/0 (socket says 172.21.15.62:60424) 2024-06-10T00:34:58.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.383+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/3416061824 learned_addr learned my addr 172.21.15.62:0/3416061824 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:34:58.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0bd77fe700 1 --2- 172.21.15.62:0/3416061824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3eaf60 0x563a6f406600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/3416061824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3ea640 msgr2=0x563a6f4060c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:58.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/3416061824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3ea640 0x563a6f4060c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/3416061824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3eaf60 msgr2=0x563a6f406600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:34:58.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/3416061824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3eaf60 0x563a6f406600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0bd6ffd700 1 -- 172.21.15.62:0/3416061824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a6f468060 con 0x563a6f3e4f20 2024-06-10T00:34:58.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0bd6ffd700 1 --2- 172.21.15.62:0/3416061824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3e4f20 0x563a6f405b80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8005b40 tx=0x7f0bc80bb310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c96348880677fb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:34:58.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0bbe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:34:58.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0bbe7fc700 1 -- 172.21.15.62:0/3416061824 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc809c1b0 con 0x563a6f3e4f20 2024-06-10T00:34:58.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0bbe7fc700 1 -- 172.21.15.62:0/3416061824 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f0bc809ccb0 con 0x563a6f3e4f20 2024-06-10T00:34:58.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/3416061824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc00043c0 con 0x563a6f3e4f20 2024-06-10T00:34:58.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0bbe7fc700 1 -- 172.21.15.62:0/3416061824 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0bc8022470 con 0x563a6f3e4f20 2024-06-10T00:34:58.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0be7cb4b80 10 client.16029 fetch_fsmap learned FSMap version 30 2024-06-10T00:34:58.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/3416061824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a6f468060 con 0x563a6f3e4f20 2024-06-10T00:34:58.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.384+0000 7f0bbe7fc700 1 -- 172.21.15.62:0/3416061824 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc00043c0 con 0x563a6f3e4f20 2024-06-10T00:34:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.385+0000 7f0bbe7fc700 1 -- 172.21.15.62:0/3416061824 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0bc80b0ce0 con 0x563a6f3e4f20 2024-06-10T00:34:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.385+0000 7f0be7cb4b80 10 client.16029 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:34:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.385+0000 7f0be7cb4b80 10 client.16029 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:34:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.385+0000 7f0be7cb4b80 20 client.16029 populate_metadata read hostname 'smithi062' 2024-06-10T00:34:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.385+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/3416061824 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x563a6f4694a0 0x563a6f46b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:34:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.385+0000 7f0be7cb4b80 4 client.16029 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:34:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.385+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/3416061824 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a6f33e290 con 0x563a6f4694a0 2024-06-10T00:34:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.385+0000 7f0bce7fc700 1 --2- 172.21.15.62:0/3416061824 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x563a6f4694a0 0x563a6f46b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:34:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.385+0000 7f0bce7fc700 1 --2- 172.21.15.62:0/3416061824 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x563a6f4694a0 0x563a6f46b890 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cea22579b40f6610 server_cookie=8aaf7e45645a6a64 in_seq=0 out_seq=0 2024-06-10T00:34:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.385+0000 7f0bbe7fc700 10 client.16029 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:34:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0bbe7fc700 1 -- 172.21.15.62:0/3416061824 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a6f33e290 con 0x563a6f4694a0 2024-06-10T00:34:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0bbe7fc700 10 client.16029 handle_command_reply: tid=0 2024-06-10T00:34:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0be7cb4b80 1 client.16029 shutdown 2024-06-10T00:34:58.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/3416061824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3e4f20 msgr2=0x563a6f405b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:58.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/3416061824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3e4f20 0x563a6f405b80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8005b40 tx=0x7f0bc80bb310 comp rx=0 tx=0).stop 2024-06-10T00:34:58.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/3416061824 shutdown_connections 2024-06-10T00:34:58.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/3416061824 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x563a6f4694a0 0x563a6f46b890 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/3416061824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563a6f3e4f20 0x563a6f405b80 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/3416061824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a6f3eaf60 0x563a6f406600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0bbe7fc700 0 client.16029 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:34:58.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0be7cb4b80 1 --2- 172.21.15.62:0/3416061824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a6f3ea640 0x563a6f4060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/3416061824 >> 172.21.15.62:0/3416061824 conn(0x563a6f30fcc0 msgr2=0x563a6f3ec840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:58.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/3416061824 shutdown_connections 2024-06-10T00:34:58.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0be7cb4b80 1 -- 172.21.15.62:0/3416061824 wait complete. 2024-06-10T00:34:58.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0be7cb4b80 20 client.16029 trim_cache size 0 max 16384 2024-06-10T00:34:58.392 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:34:58.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0bdd431700 1 -- 172.21.15.62:0/3663798732 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f0bc40a2d90 msgr2=0x7f0bc40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:58.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0bdd431700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f0bc40a2d90 0x7f0bc40a5250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80ffb70 tx=0x7f0bc000b500 comp rx=0 tx=0).stop 2024-06-10T00:34:58.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0bdd431700 1 -- 172.21.15.62:0/3663798732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bd8101390 msgr2=0x7f0bd819e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:34:58.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.386+0000 7f0bdd431700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bd8101390 0x7f0bd819e870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80059d0 tx=0x7f0bc80047b0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.387+0000 7f0bdd431700 1 -- 172.21.15.62:0/3663798732 shutdown_connections 2024-06-10T00:34:58.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.387+0000 7f0bdd431700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f0bc40a2d90 0x7f0bc40a5250 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.387+0000 7f0bdd431700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bd8107400 0x7f0bd8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.387+0000 7f0bdd431700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bd8101390 0x7f0bd819e870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.387+0000 7f0bdd431700 1 --2- 172.21.15.62:0/3663798732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bd8101d50 0x7f0bd819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:34:58.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.387+0000 7f0bdd431700 1 -- 172.21.15.62:0/3663798732 >> 172.21.15.62:0/3663798732 conn(0x7f0bd8078340 msgr2=0x7f0bd81005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:34:58.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.387+0000 7f0bdd431700 1 -- 172.21.15.62:0/3663798732 shutdown_connections 2024-06-10T00:34:58.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:34:58.387+0000 7f0bdd431700 1 -- 172.21.15.62:0/3663798732 wait complete. 2024-06-10T00:34:59.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T00:34:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:34:59.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:34:59 smithi062 ceph-mon[32390]: pgmap v67: 129 pgs: 129 active+clean; 7.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 200 KiB/s wr, 35 op/s 2024-06-10T00:34:59.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:34:59 smithi105 ceph-mon[36321]: pgmap v67: 129 pgs: 129 active+clean; 7.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 200 KiB/s wr, 35 op/s 2024-06-10T00:34:59.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:34:59 smithi138 ceph-mon[36549]: pgmap v67: 129 pgs: 129 active+clean; 7.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 200 KiB/s wr, 35 op/s 2024-06-10T00:35:01.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.271+0000 7f1bcd8f7700 1 -- 172.21.15.62:0/65164567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc8102900 msgr2=0x7f1bc810e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:01.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.271+0000 7f1bcd8f7700 1 --2- 172.21.15.62:0/65164567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc8102900 0x7f1bc810e310 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000b0d0 tx=0x7f1bc000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.271+0000 7f1bcd8f7700 1 -- 172.21.15.62:0/65164567 shutdown_connections 2024-06-10T00:35:01.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.271+0000 7f1bcd8f7700 1 --2- 172.21.15.62:0/65164567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc8102900 0x7f1bc810e310 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.271+0000 7f1bcd8f7700 1 --2- 172.21.15.62:0/65164567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bc8107660 0x7f1bc8107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.271+0000 7f1bcd8f7700 1 --2- 172.21.15.62:0/65164567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc8101f40 0x7f1bc81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.271+0000 7f1bcd8f7700 1 -- 172.21.15.62:0/65164567 >> 172.21.15.62:0/65164567 conn(0x7f1bc80766b0 msgr2=0x7f1bc8076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:01.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.271+0000 7f1bcd8f7700 1 -- 172.21.15.62:0/65164567 shutdown_connections 2024-06-10T00:35:01.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.271+0000 7f1bcd8f7700 1 -- 172.21.15.62:0/65164567 wait complete. 2024-06-10T00:35:01.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.271+0000 7f1bcd8f7700 1 Processor -- start 2024-06-10T00:35:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bcd8f7700 1 -- start start 2024-06-10T00:35:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bcd8f7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc8101f40 0x7f1bc8199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bcd8f7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bc8102900 0x7f1bc819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bcd8f7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc8107660 0x7f1bc81a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bcd8f7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc81117b0 con 0x7f1bc8107660 2024-06-10T00:35:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bcd8f7700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc8111630 con 0x7f1bc8102900 2024-06-10T00:35:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bcd8f7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc8111930 con 0x7f1bc8101f40 2024-06-10T00:35:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bc77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc8107660 0x7f1bc81a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bc77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc8107660 0x7f1bc81a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60440/0 (socket says 172.21.15.62:60440) 2024-06-10T00:35:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bc77fe700 1 -- 172.21.15.62:0/2732062490 learned_addr learned my addr 172.21.15.62:0/2732062490 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/2732062490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc8101f40 0x7f1bc8199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bc67fc700 1 --2- 172.21.15.62:0/2732062490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bc8102900 0x7f1bc819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bc77fe700 1 -- 172.21.15.62:0/2732062490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc8101f40 msgr2=0x7f1bc8199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bc77fe700 1 --2- 172.21.15.62:0/2732062490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc8101f40 0x7f1bc8199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bc77fe700 1 -- 172.21.15.62:0/2732062490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bc8102900 msgr2=0x7f1bc819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bc77fe700 1 --2- 172.21.15.62:0/2732062490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bc8102900 0x7f1bc819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.272+0000 7f1bc77fe700 1 -- 172.21.15.62:0/2732062490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc81a50f0 con 0x7f1bc8107660 2024-06-10T00:35:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.273+0000 7f1bc77fe700 1 --2- 172.21.15.62:0/2732062490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc8107660 0x7f1bc81a4bb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000d010 tx=0x7f1bc0004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.273+0000 7f1bcc8f5700 1 -- 172.21.15.62:0/2732062490 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0021020 con 0x7f1bc8107660 2024-06-10T00:35:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.273+0000 7f1bcd8f7700 1 -- 172.21.15.62:0/2732062490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc81a52c0 con 0x7f1bc8107660 2024-06-10T00:35:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.273+0000 7f1bcc8f5700 1 -- 172.21.15.62:0/2732062490 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bc0007b80 con 0x7f1bc8107660 2024-06-10T00:35:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.273+0000 7f1bcc8f5700 1 -- 172.21.15.62:0/2732062490 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc00052a0 con 0x7f1bc8107660 2024-06-10T00:35:01.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.273+0000 7f1bcd8f7700 1 -- 172.21.15.62:0/2732062490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc81a56c0 con 0x7f1bc8107660 2024-06-10T00:35:01.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.276+0000 7f1bcc8f5700 1 -- 172.21.15.62:0/2732062490 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1bc0024020 con 0x7f1bc8107660 2024-06-10T00:35:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.277+0000 7f1bcc8f5700 1 --2- 172.21.15.62:0/2732062490 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1bb40a2d90 0x7f1bb40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.277+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/2732062490 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1bb40a2d90 0x7f1bb40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.277+0000 7f1bcc8f5700 1 -- 172.21.15.62:0/2732062490 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f1bc00d0aa0 con 0x7f1bc8107660 2024-06-10T00:35:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.277+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/2732062490 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1bb40a2d90 0x7f1bb40a5250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8005950 tx=0x7f1bb800b410 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.277+0000 7f1bd817ab80 1 Processor -- start 2024-06-10T00:35:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.277+0000 7f1bd817ab80 1 -- start start 2024-06-10T00:35:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.277+0000 7f1bd817ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x557910516340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.277+0000 7f1bd817ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 0x557910510260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.277+0000 7f1bd817ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579105107a0 0x557910510bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bd817ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557910402f60 con 0x557910515f60 2024-06-10T00:35:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bd817ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557910461850 con 0x5579105107a0 2024-06-10T00:35:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bd817ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5579103c9460 con 0x557910516880 2024-06-10T00:35:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x557910516340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x557910516340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60442/0 (socket says 172.21.15.62:60442) 2024-06-10T00:35:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bc6ffd700 1 -- 172.21.15.62:0/4205503929 learned_addr learned my addr 172.21.15.62:0/4205503929 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bc77fe700 1 --2- 172.21.15.62:0/4205503929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579105107a0 0x557910510bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bc67fc700 1 --2- 172.21.15.62:0/4205503929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 0x557910510260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bc6ffd700 1 -- 172.21.15.62:0/4205503929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 msgr2=0x557910510260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/4205503929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 0x557910510260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bc6ffd700 1 -- 172.21.15.62:0/4205503929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579105107a0 msgr2=0x557910510bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/4205503929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579105107a0 0x557910510bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bc6ffd700 1 -- 172.21.15.62:0/4205503929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579103b08d0 con 0x557910515f60 2024-06-10T00:35:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/4205503929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x557910516340 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8010e60 tx=0x7f1bb8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34bfaa8b53ebb8d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.278+0000 7f1bbdffb700 1 -- 172.21.15.62:0/4205503929 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8011db0 con 0x557910515f60 2024-06-10T00:35:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.279+0000 7f1bbdffb700 1 -- 172.21.15.62:0/4205503929 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bb8010e90 con 0x557910515f60 2024-06-10T00:35:01.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.279+0000 7f1bbdffb700 1 -- 172.21.15.62:0/4205503929 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb801c790 con 0x557910515f60 2024-06-10T00:35:01.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.279+0000 7f1bd817ab80 1 -- 172.21.15.62:0/4205503929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 msgr2=0x557910516340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:01.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.279+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/4205503929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x557910516340 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8010e60 tx=0x7f1bb8013000 comp rx=0 tx=0).stop 2024-06-10T00:35:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.279+0000 7f1bd817ab80 1 -- 172.21.15.62:0/4205503929 shutdown_connections 2024-06-10T00:35:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.279+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/4205503929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x557910516340 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.279+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/4205503929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579105107a0 0x557910510bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.279+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/4205503929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 0x557910510260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.279+0000 7f1bd817ab80 1 -- 172.21.15.62:0/4205503929 >> 172.21.15.62:0/4205503929 conn(0x55791043d080 msgr2=0x55791050b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.279+0000 7f1bd817ab80 1 -- 172.21.15.62:0/4205503929 shutdown_connections 2024-06-10T00:35:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.280+0000 7f1bd817ab80 1 -- 172.21.15.62:0/4205503929 wait complete. 2024-06-10T00:35:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.280+0000 7f1bd817ab80 1 -- start start 2024-06-10T00:35:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.280+0000 7f1bd817ab80 1 Processor -- start 2024-06-10T00:35:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.280+0000 7f1bd817ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579105107a0 0x55791051c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.280+0000 7f1bd817ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x5579105198c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.280+0000 7f1bd817ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 0x557910519e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.280+0000 7f1bd817ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557910401fc0 con 0x557910515f60 2024-06-10T00:35:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.280+0000 7f1bd817ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55791045d810 con 0x5579105107a0 2024-06-10T00:35:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.280+0000 7f1bd817ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55791051d260 con 0x557910516880 2024-06-10T00:35:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1bd817ab80 10 client.0 _ll_register_callbacks cb 0x5579103f75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1bc67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x5579105198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1bc67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x5579105198c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60450/0 (socket says 172.21.15.62:60450) 2024-06-10T00:35:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1bc67fc700 1 -- 172.21.15.62:0/4117189050 learned_addr learned my addr 172.21.15.62:0/4117189050 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/4117189050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579105107a0 0x55791051c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1bc77fe700 1 --2- 172.21.15.62:0/4117189050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 0x557910519e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1bc67fc700 1 -- 172.21.15.62:0/4117189050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 msgr2=0x557910519e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1bc67fc700 1 --2- 172.21.15.62:0/4117189050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 0x557910519e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1bc67fc700 1 -- 172.21.15.62:0/4117189050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579105107a0 msgr2=0x55791051c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1bc67fc700 1 --2- 172.21.15.62:0/4117189050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579105107a0 0x55791051c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1bc67fc700 1 -- 172.21.15.62:0/4117189050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557910427750 con 0x557910515f60 2024-06-10T00:35:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/4117189050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579105107a0 0x55791051c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:35:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1bc67fc700 1 --2- 172.21.15.62:0/4117189050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x5579105198c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000bb70 tx=0x7f1bb000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce56ddf498b3aa51 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1ba6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1ba6ffd700 1 -- 172.21.15.62:0/4117189050 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0003ca0 con 0x557910515f60 2024-06-10T00:35:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1ba6ffd700 1 -- 172.21.15.62:0/4117189050 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f1bb0004830 con 0x557910515f60 2024-06-10T00:35:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.281+0000 7f1bd817ab80 1 -- 172.21.15.62:0/4117189050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bb8010e90 con 0x557910515f60 2024-06-10T00:35:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.282+0000 7f1ba6ffd700 1 -- 172.21.15.62:0/4117189050 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1bb0024050 con 0x557910515f60 2024-06-10T00:35:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.282+0000 7f1bd817ab80 10 client.16041 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.282+0000 7f1bd817ab80 1 -- 172.21.15.62:0/4117189050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557910527a20 con 0x557910515f60 2024-06-10T00:35:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.282+0000 7f1ba6ffd700 1 -- 172.21.15.62:0/4117189050 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bb8010e90 con 0x557910515f60 2024-06-10T00:35:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.282+0000 7f1ba6ffd700 1 -- 172.21.15.62:0/4117189050 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1bb0011560 con 0x557910515f60 2024-06-10T00:35:01.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.282+0000 7f1bd817ab80 10 client.16041 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.282+0000 7f1bd817ab80 10 client.16041 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:35:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.282+0000 7f1bd817ab80 20 client.16041 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.282+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/4117189050 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55791052a060 0x55791052c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.282+0000 7f1bd817ab80 4 client.16041 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:35:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.282+0000 7f1bd817ab80 1 -- 172.21.15.62:0/4117189050 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55791046ed50 con 0x55791052a060 2024-06-10T00:35:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.282+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/4117189050 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55791052a060 0x55791052c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.283+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/4117189050 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55791052a060 0x55791052c450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=62685716d6bb5d30 server_cookie=ce6fcfad639b34e5 in_seq=0 out_seq=0 2024-06-10T00:35:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.283+0000 7f1ba6ffd700 10 client.16041 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:35:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1ba6ffd700 1 -- 172.21.15.62:0/4117189050 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55791046ed50 con 0x55791052a060 2024-06-10T00:35:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1ba6ffd700 10 client.16041 handle_command_reply: tid=0 2024-06-10T00:35:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1bd817ab80 1 client.16041 shutdown 2024-06-10T00:35:01.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1bd817ab80 1 -- 172.21.15.62:0/4117189050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 msgr2=0x5579105198c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/4117189050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x5579105198c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000bb70 tx=0x7f1bb000be80 comp rx=0 tx=0).stop 2024-06-10T00:35:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1bd817ab80 1 -- 172.21.15.62:0/4117189050 shutdown_connections 2024-06-10T00:35:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/4117189050 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55791052a060 0x55791052c450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/4117189050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 0x557910519e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/4117189050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579105107a0 0x55791051c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/4117189050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x5579105198c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1bd817ab80 1 -- 172.21.15.62:0/4117189050 >> 172.21.15.62:0/4117189050 conn(0x55791043d080 msgr2=0x55791050a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1ba6ffd700 0 client.16041 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:35:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1bd817ab80 1 -- 172.21.15.62:0/4117189050 shutdown_connections 2024-06-10T00:35:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1bd817ab80 1 -- 172.21.15.62:0/4117189050 wait complete. 2024-06-10T00:35:01.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.284+0000 7f1bd817ab80 20 client.16041 trim_cache size 0 max 16384 2024-06-10T00:35:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bd817ab80 1 Processor -- start 2024-06-10T00:35:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bd817ab80 1 -- start start 2024-06-10T00:35:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bd817ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579105107a0 0x55791051c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bd817ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x55791050ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bd817ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557910516880 0x55791050b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bd817ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc81ac370 con 0x557910515f60 2024-06-10T00:35:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bd817ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc81adeb0 con 0x557910516880 2024-06-10T00:35:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bd817ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc81aca40 con 0x5579105107a0 2024-06-10T00:35:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bc67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x55791050ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bc67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x55791050ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60456/0 (socket says 172.21.15.62:60456) 2024-06-10T00:35:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bc67fc700 1 -- 172.21.15.62:0/3886935853 learned_addr learned my addr 172.21.15.62:0/3886935853 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/3886935853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579105107a0 0x55791051c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bc67fc700 1 -- 172.21.15.62:0/3886935853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579105107a0 msgr2=0x55791051c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bc67fc700 1 --2- 172.21.15.62:0/3886935853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579105107a0 0x55791051c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bc67fc700 1 -- 172.21.15.62:0/3886935853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557910516880 msgr2=0x55791050b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bc77fe700 1 --2- 172.21.15.62:0/3886935853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557910516880 0x55791050b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bc67fc700 1 --2- 172.21.15.62:0/3886935853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557910516880 0x55791050b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bc67fc700 1 -- 172.21.15.62:0/3886935853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557910517e50 con 0x557910515f60 2024-06-10T00:35:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/3886935853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579105107a0 0x55791051c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:35:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.300+0000 7f1bc67fc700 1 --2- 172.21.15.62:0/3886935853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x55791050ae80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00059d0 tx=0x7f1bb0004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57eaef38a254b242 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.301+0000 7f1bbd7fa700 1 -- 172.21.15.62:0/3886935853 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0003ca0 con 0x557910515f60 2024-06-10T00:35:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.301+0000 7f1bbd7fa700 1 -- 172.21.15.62:0/3886935853 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bb0003e40 con 0x557910515f60 2024-06-10T00:35:01.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.301+0000 7f1bbd7fa700 1 -- 172.21.15.62:0/3886935853 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0011d30 con 0x557910515f60 2024-06-10T00:35:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.301+0000 7f1bd817ab80 1 -- 172.21.15.62:0/3886935853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 msgr2=0x55791050ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.301+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/3886935853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x55791050ae80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00059d0 tx=0x7f1bb0004550 comp rx=0 tx=0).stop 2024-06-10T00:35:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.301+0000 7f1bd817ab80 1 -- 172.21.15.62:0/3886935853 shutdown_connections 2024-06-10T00:35:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.301+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/3886935853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557910516880 0x55791050b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.301+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/3886935853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579105107a0 0x55791051c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.301+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/3886935853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557910515f60 0x55791050ae80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.301+0000 7f1bd817ab80 1 -- 172.21.15.62:0/3886935853 >> 172.21.15.62:0/3886935853 conn(0x55791043d080 msgr2=0x5579105181d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.301+0000 7f1bd817ab80 1 -- 172.21.15.62:0/3886935853 shutdown_connections 2024-06-10T00:35:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.301+0000 7f1bd817ab80 1 -- 172.21.15.62:0/3886935853 wait complete. 2024-06-10T00:35:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.302+0000 7f1bd817ab80 1 -- start start 2024-06-10T00:35:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.302+0000 7f1bd817ab80 1 Processor -- start 2024-06-10T00:35:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.302+0000 7f1bd817ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579105107a0 0x557910522700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.302+0000 7f1bd817ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557910515f60 0x557910522c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.302+0000 7f1bd817ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 0x557910531110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.302+0000 7f1bd817ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc81ad7e0 con 0x5579105107a0 2024-06-10T00:35:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.302+0000 7f1bd817ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc81ad110 con 0x557910515f60 2024-06-10T00:35:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.302+0000 7f1bd817ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc81aa310 con 0x557910516880 2024-06-10T00:35:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.302+0000 7f1bd817ab80 10 client.0 _ll_register_callbacks cb 0x5579103f75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.302+0000 7f1bc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579105107a0 0x557910522700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.302+0000 7f1bc6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579105107a0 0x557910522700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60468/0 (socket says 172.21.15.62:60468) 2024-06-10T00:35:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.302+0000 7f1bc6ffd700 1 -- 172.21.15.62:0/624220550 learned_addr learned my addr 172.21.15.62:0/624220550 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1bc67fc700 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557910515f60 0x557910522c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1bc77fe700 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 0x557910531110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1bc6ffd700 1 -- 172.21.15.62:0/624220550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 msgr2=0x557910531110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 0x557910531110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1bc6ffd700 1 -- 172.21.15.62:0/624220550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557910515f60 msgr2=0x557910522c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557910515f60 0x557910522c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1bc6ffd700 1 -- 172.21.15.62:0/624220550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557910531810 con 0x5579105107a0 2024-06-10T00:35:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1bc77fe700 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 0x557910531110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:35:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1bc67fc700 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557910515f60 0x557910522c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:35:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1bc6ffd700 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579105107a0 0x557910522700 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8010760 tx=0x7f1bb8010810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364817ab94f4550d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:01.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1ba6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1ba6ffd700 1 -- 172.21.15.62:0/624220550 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb802ac10 con 0x5579105107a0 2024-06-10T00:35:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1bd817ab80 1 -- 172.21.15.62:0/624220550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bb0003e40 con 0x5579105107a0 2024-06-10T00:35:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1ba6ffd700 1 -- 172.21.15.62:0/624220550 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f1bb802adb0 con 0x5579105107a0 2024-06-10T00:35:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1ba6ffd700 1 -- 172.21.15.62:0/624220550 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1bb8047050 con 0x5579105107a0 2024-06-10T00:35:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1bd817ab80 10 client.16047 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.303+0000 7f1bd817ab80 1 -- 172.21.15.62:0/624220550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557910594680 con 0x5579105107a0 2024-06-10T00:35:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.304+0000 7f1ba6ffd700 1 -- 172.21.15.62:0/624220550 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bb0003e40 con 0x5579105107a0 2024-06-10T00:35:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.304+0000 7f1ba6ffd700 1 -- 172.21.15.62:0/624220550 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1bb801e7e0 con 0x5579105107a0 2024-06-10T00:35:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.304+0000 7f1bd817ab80 10 client.16047 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.304+0000 7f1bd817ab80 10 client.16047 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:35:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.304+0000 7f1bd817ab80 20 client.16047 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.304+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x557910594ae0 0x557910596f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:01.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.304+0000 7f1bd817ab80 4 client.16047 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:35:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.304+0000 7f1bd817ab80 1 -- 172.21.15.62:0/624220550 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579104692d0 con 0x557910594ae0 2024-06-10T00:35:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.304+0000 7f1bc67fc700 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x557910594ae0 0x557910596f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1bc67fc700 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x557910594ae0 0x557910596f00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bb729e02d6000355 server_cookie=60e892b1864ce775 in_seq=0 out_seq=0 2024-06-10T00:35:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1ba6ffd700 10 client.16047 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:35:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1ba6ffd700 1 -- 172.21.15.62:0/624220550 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579104692d0 con 0x557910594ae0 2024-06-10T00:35:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1ba6ffd700 10 client.16047 handle_command_reply: tid=0 2024-06-10T00:35:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1bd817ab80 1 client.16047 shutdown 2024-06-10T00:35:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1bd817ab80 1 -- 172.21.15.62:0/624220550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579105107a0 msgr2=0x557910522700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579105107a0 0x557910522700 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8010760 tx=0x7f1bb8010810 comp rx=0 tx=0).stop 2024-06-10T00:35:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1bd817ab80 1 -- 172.21.15.62:0/624220550 shutdown_connections 2024-06-10T00:35:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x557910594ae0 0x557910596f00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1ba6ffd700 0 client.16047 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:35:01.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5579105107a0 0x557910522700 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557910516880 0x557910531110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1bd817ab80 1 --2- 172.21.15.62:0/624220550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557910515f60 0x557910522c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1bd817ab80 1 -- 172.21.15.62:0/624220550 >> 172.21.15.62:0/624220550 conn(0x55791043d080 msgr2=0x557910519530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:01.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1bd817ab80 1 -- 172.21.15.62:0/624220550 shutdown_connections 2024-06-10T00:35:01.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1bd817ab80 1 -- 172.21.15.62:0/624220550 wait complete. 2024-06-10T00:35:01.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.305+0000 7f1bd817ab80 20 client.16047 trim_cache size 0 max 16384 2024-06-10T00:35:01.311 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:35:01.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.306+0000 7f1bcd8f7700 1 -- 172.21.15.62:0/2732062490 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1bb40a2d90 msgr2=0x7f1bb40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:01.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.306+0000 7f1bcd8f7700 1 --2- 172.21.15.62:0/2732062490 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1bb40a2d90 0x7f1bb40a5250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8005950 tx=0x7f1bb800b410 comp rx=0 tx=0).stop 2024-06-10T00:35:01.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.306+0000 7f1bcd8f7700 1 -- 172.21.15.62:0/2732062490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc8107660 msgr2=0x7f1bc81a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:01.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.306+0000 7f1bcd8f7700 1 --2- 172.21.15.62:0/2732062490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc8107660 0x7f1bc81a4bb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000d010 tx=0x7f1bc0004ce0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.306+0000 7f1bcd8f7700 1 -- 172.21.15.62:0/2732062490 shutdown_connections 2024-06-10T00:35:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.306+0000 7f1bcd8f7700 1 --2- 172.21.15.62:0/2732062490 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1bb40a2d90 0x7f1bb40a5250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.306+0000 7f1bcd8f7700 1 --2- 172.21.15.62:0/2732062490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bc8107660 0x7f1bc81a4bb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.306+0000 7f1bcd8f7700 1 --2- 172.21.15.62:0/2732062490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc8101f40 0x7f1bc8199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.306+0000 7f1bcd8f7700 1 --2- 172.21.15.62:0/2732062490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bc8102900 0x7f1bc819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.306+0000 7f1bcd8f7700 1 -- 172.21.15.62:0/2732062490 >> 172.21.15.62:0/2732062490 conn(0x7f1bc80766b0 msgr2=0x7f1bc80fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.306+0000 7f1bcd8f7700 1 -- 172.21.15.62:0/2732062490 shutdown_connections 2024-06-10T00:35:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:01.306+0000 7f1bcd8f7700 1 -- 172.21.15.62:0/2732062490 wait complete. 2024-06-10T00:35:01.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:01 smithi062 ceph-mon[32390]: pgmap v68: 129 pgs: 129 active+clean; 7.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 200 KiB/s wr, 35 op/s 2024-06-10T00:35:01.676 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:01 smithi138 ceph-mon[36549]: pgmap v68: 129 pgs: 129 active+clean; 7.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 200 KiB/s wr, 35 op/s 2024-06-10T00:35:01.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:01 smithi105 ceph-mon[36321]: pgmap v68: 129 pgs: 129 active+clean; 7.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 200 KiB/s wr, 35 op/s 2024-06-10T00:35:02.099 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T00:35:02.349 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:35:03.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:03 smithi105 ceph-mon[36321]: pgmap v69: 129 pgs: 129 active+clean; 8.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 332 KiB/s wr, 81 op/s 2024-06-10T00:35:03.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:03 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:35:03.798 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:03 smithi062 ceph-mon[32390]: pgmap v69: 129 pgs: 129 active+clean; 8.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 332 KiB/s wr, 81 op/s 2024-06-10T00:35:03.799 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:03 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:35:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:03 smithi138 ceph-mon[36549]: pgmap v69: 129 pgs: 129 active+clean; 8.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 332 KiB/s wr, 81 op/s 2024-06-10T00:35:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:03 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:35:04.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.223+0000 7f6b1f9db700 1 -- 172.21.15.62:0/1447816935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1806b3c0 msgr2=0x7f6b1810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:04.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.223+0000 7f6b1f9db700 1 --2- 172.21.15.62:0/1447816935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1806b3c0 0x7f6b1810f310 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009990 tx=0x7f6b10009ca0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.223+0000 7f6b1f9db700 1 -- 172.21.15.62:0/1447816935 shutdown_connections 2024-06-10T00:35:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.223+0000 7f6b1f9db700 1 --2- 172.21.15.62:0/1447816935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1806b3c0 0x7f6b1810f310 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.223+0000 7f6b1f9db700 1 --2- 172.21.15.62:0/1447816935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b181084f0 0x7f6b181088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.223+0000 7f6b1f9db700 1 --2- 172.21.15.62:0/1447816935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b1806aa00 0x7f6b1806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.223+0000 7f6b1f9db700 1 -- 172.21.15.62:0/1447816935 >> 172.21.15.62:0/1447816935 conn(0x7f6b18078300 msgr2=0x7f6b18078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.223+0000 7f6b1f9db700 1 -- 172.21.15.62:0/1447816935 shutdown_connections 2024-06-10T00:35:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.223+0000 7f6b1f9db700 1 -- 172.21.15.62:0/1447816935 wait complete. 2024-06-10T00:35:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.224+0000 7f6b1f9db700 1 Processor -- start 2024-06-10T00:35:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.224+0000 7f6b1f9db700 1 -- start start 2024-06-10T00:35:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.224+0000 7f6b1f9db700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b1806aa00 0x7f6b1819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.224+0000 7f6b1f9db700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1806b3c0 0x7f6b1819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.224+0000 7f6b1f9db700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b181084f0 0x7f6b181989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.224+0000 7f6b1f9db700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18112750 con 0x7f6b1806b3c0 2024-06-10T00:35:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.224+0000 7f6b1f9db700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181125d0 con 0x7f6b181084f0 2024-06-10T00:35:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.224+0000 7f6b1f9db700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181128d0 con 0x7f6b1806aa00 2024-06-10T00:35:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b1cf76700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1806b3c0 0x7f6b1819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b1d777700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b1806aa00 0x7f6b1819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b1cf76700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1806b3c0 0x7f6b1819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60488/0 (socket says 172.21.15.62:60488) 2024-06-10T00:35:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b1cf76700 1 -- 172.21.15.62:0/298263662 learned_addr learned my addr 172.21.15.62:0/298263662 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b1df78700 1 --2- 172.21.15.62:0/298263662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b181084f0 0x7f6b181989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b1d777700 1 -- 172.21.15.62:0/298263662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b181084f0 msgr2=0x7f6b181989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b1d777700 1 --2- 172.21.15.62:0/298263662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b181084f0 0x7f6b181989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b1d777700 1 -- 172.21.15.62:0/298263662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1806b3c0 msgr2=0x7f6b1819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b1d777700 1 --2- 172.21.15.62:0/298263662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1806b3c0 0x7f6b1819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b1d777700 1 -- 172.21.15.62:0/298263662 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b18199130 con 0x7f6b1806aa00 2024-06-10T00:35:04.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/298263662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1806b3c0 0x7f6b1819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:35:04.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b1d777700 1 --2- 172.21.15.62:0/298263662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b1806aa00 0x7f6b1819e810 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800ef10 tx=0x7f6b0800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:04.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b167fc700 1 -- 172.21.15.62:0/298263662 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800c9c0 con 0x7f6b1806aa00 2024-06-10T00:35:04.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b1f9db700 1 -- 172.21.15.62:0/298263662 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b181993c0 con 0x7f6b1806aa00 2024-06-10T00:35:04.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b167fc700 1 -- 172.21.15.62:0/298263662 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b08019070 con 0x7f6b1806aa00 2024-06-10T00:35:04.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b1f9db700 1 -- 172.21.15.62:0/298263662 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b181a56e0 con 0x7f6b1806aa00 2024-06-10T00:35:04.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.225+0000 7f6b167fc700 1 -- 172.21.15.62:0/298263662 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b080146e0 con 0x7f6b1806aa00 2024-06-10T00:35:04.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.228+0000 7f6b167fc700 1 -- 172.21.15.62:0/298263662 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6b08012050 con 0x7f6b1806aa00 2024-06-10T00:35:04.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.229+0000 7f6b167fc700 1 --2- 172.21.15.62:0/298263662 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6b000a2d90 0x7f6b000a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.229+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/298263662 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6b000a2d90 0x7f6b000a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.229+0000 7f6b167fc700 1 -- 172.21.15.62:0/298263662 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f6b080cf780 con 0x7f6b1806aa00 2024-06-10T00:35:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.229+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/298263662 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6b000a2d90 0x7f6b000a5250 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6b18078900 tx=0x7f6b0c009260 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.229+0000 7f6b2a25eb80 1 Processor -- start 2024-06-10T00:35:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.229+0000 7f6b2a25eb80 1 -- start start 2024-06-10T00:35:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b2a25eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 0x55ccea4a8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b2a25eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea4a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b2a25eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a3330 0x55ccea4a3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b2a25eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ccea3f0810 con 0x55ccea4a9390 2024-06-10T00:35:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b2a25eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ccea398720 con 0x55ccea4a8a70 2024-06-10T00:35:04.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b2a25eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ccea3ec7d0 con 0x55ccea4a3330 2024-06-10T00:35:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b1cf76700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea4a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b1cf76700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea4a2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60492/0 (socket says 172.21.15.62:60492) 2024-06-10T00:35:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b1cf76700 1 -- 172.21.15.62:0/2042124190 learned_addr learned my addr 172.21.15.62:0/2042124190 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b1d777700 1 --2- 172.21.15.62:0/2042124190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 0x55ccea4a8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b1df78700 1 --2- 172.21.15.62:0/2042124190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a3330 0x55ccea4a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b1cf76700 1 -- 172.21.15.62:0/2042124190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a3330 msgr2=0x55ccea4a3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/2042124190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a3330 0x55ccea4a3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b1cf76700 1 -- 172.21.15.62:0/2042124190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 msgr2=0x55ccea4a8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/2042124190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 0x55ccea4a8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b1cf76700 1 -- 172.21.15.62:0/2042124190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccea38ef50 con 0x55ccea4a9390 2024-06-10T00:35:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b1d777700 1 --2- 172.21.15.62:0/2042124190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 0x55ccea4a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:35:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.230+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/2042124190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea4a2df0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009880 tx=0x7f6b0c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d51f91b2c4995e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:04.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.231+0000 7f6aff7fe700 1 -- 172.21.15.62:0/2042124190 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c026630 con 0x55ccea4a9390 2024-06-10T00:35:04.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.231+0000 7f6aff7fe700 1 -- 172.21.15.62:0/2042124190 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b0c00ee40 con 0x55ccea4a9390 2024-06-10T00:35:04.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.231+0000 7f6aff7fe700 1 -- 172.21.15.62:0/2042124190 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c01d910 con 0x55ccea4a9390 2024-06-10T00:35:04.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.231+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/2042124190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 msgr2=0x55ccea4a2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:04.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.231+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/2042124190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea4a2df0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009880 tx=0x7f6b0c014000 comp rx=0 tx=0).stop 2024-06-10T00:35:04.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.231+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/2042124190 shutdown_connections 2024-06-10T00:35:04.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.231+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/2042124190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a3330 0x55ccea4a3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.231+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/2042124190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 0x55ccea4a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.231+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/2042124190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea4a2df0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.231+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/2042124190 >> 172.21.15.62:0/2042124190 conn(0x55ccea3c9cc0 msgr2=0x55ccea499850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.231+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/2042124190 shutdown_connections 2024-06-10T00:35:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.232+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/2042124190 wait complete. 2024-06-10T00:35:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.232+0000 7f6b2a25eb80 1 -- start start 2024-06-10T00:35:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.232+0000 7f6b2a25eb80 1 Processor -- start 2024-06-10T00:35:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.232+0000 7f6b2a25eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a3330 0x55ccea4af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.232+0000 7f6b2a25eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 0x55ccea4ac510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.232+0000 7f6b2a25eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a9390 0x55ccea4aca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.232+0000 7f6b2a25eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ccea4afdd0 con 0x55ccea4a3330 2024-06-10T00:35:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.232+0000 7f6b2a25eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ccea390bf0 con 0x55ccea4a8a70 2024-06-10T00:35:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.232+0000 7f6b2a25eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ccea4aff50 con 0x55ccea4a9390 2024-06-10T00:35:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.232+0000 7f6b2a25eb80 10 client.0 _ll_register_callbacks cb 0x55ccea38a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.232+0000 7f6b1d777700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a3330 0x55ccea4af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6b1d777700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a3330 0x55ccea4af1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60500/0 (socket says 172.21.15.62:60500) 2024-06-10T00:35:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6b1d777700 1 -- 172.21.15.62:0/2176666500 learned_addr learned my addr 172.21.15.62:0/2176666500 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6b1df78700 1 --2- 172.21.15.62:0/2176666500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a9390 0x55ccea4aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/2176666500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 0x55ccea4ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6b1d777700 1 -- 172.21.15.62:0/2176666500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a9390 msgr2=0x55ccea4aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6b1d777700 1 --2- 172.21.15.62:0/2176666500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a9390 0x55ccea4aca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6b1d777700 1 -- 172.21.15.62:0/2176666500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 msgr2=0x55ccea4ac510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6b1d777700 1 --2- 172.21.15.62:0/2176666500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 0x55ccea4ac510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6b1d777700 1 -- 172.21.15.62:0/2176666500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccea33f8d0 con 0x55ccea4a3330 2024-06-10T00:35:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/2176666500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 0x55ccea4ac510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:35:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6b1d777700 1 --2- 172.21.15.62:0/2176666500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a3330 0x55ccea4af1c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800c6f0 tx=0x7f6b0809c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f2cbcbb87bb5d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6afcff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6afcff9700 1 -- 172.21.15.62:0/2176666500 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b080b4060 con 0x55ccea4a3330 2024-06-10T00:35:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6afcff9700 1 -- 172.21.15.62:0/2176666500 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f6b080860c0 con 0x55ccea4a3330 2024-06-10T00:35:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/2176666500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ccea3fdd10 con 0x55ccea4a3330 2024-06-10T00:35:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6afcff9700 1 -- 172.21.15.62:0/2176666500 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6b080a4500 con 0x55ccea4a3330 2024-06-10T00:35:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6b2a25eb80 10 client.16059 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.233+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/2176666500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ccea4ba800 con 0x55ccea4a3330 2024-06-10T00:35:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.234+0000 7f6afcff9700 1 -- 172.21.15.62:0/2176666500 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ccea3fdd10 con 0x55ccea4a3330 2024-06-10T00:35:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.234+0000 7f6afcff9700 1 -- 172.21.15.62:0/2176666500 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6b080cf2f0 con 0x55ccea4a3330 2024-06-10T00:35:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.234+0000 7f6b2a25eb80 10 client.16059 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.234+0000 7f6b2a25eb80 10 client.16059 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:35:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.234+0000 7f6b2a25eb80 20 client.16059 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.234+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/2176666500 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55ccea4bce40 0x55ccea4bf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.234+0000 7f6b2a25eb80 4 client.16059 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:35:04.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.234+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/2176666500 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccea3f8290 con 0x55ccea4bce40 2024-06-10T00:35:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.234+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/2176666500 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55ccea4bce40 0x55ccea4bf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.234+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/2176666500 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55ccea4bce40 0x55ccea4bf230 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6292d71222820e8f server_cookie=dc1152167a06b03a in_seq=0 out_seq=0 2024-06-10T00:35:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.234+0000 7f6afcff9700 10 client.16059 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:35:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.235+0000 7f6afcff9700 1 -- 172.21.15.62:0/2176666500 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ccea3f8290 con 0x55ccea4bce40 2024-06-10T00:35:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.235+0000 7f6afcff9700 10 client.16059 handle_command_reply: tid=0 2024-06-10T00:35:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.235+0000 7f6b2a25eb80 1 client.16059 shutdown 2024-06-10T00:35:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.235+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/2176666500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a3330 msgr2=0x55ccea4af1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.235+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/2176666500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a3330 0x55ccea4af1c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800c6f0 tx=0x7f6b0809c010 comp rx=0 tx=0).stop 2024-06-10T00:35:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.236+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/2176666500 shutdown_connections 2024-06-10T00:35:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.236+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/2176666500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a3330 0x55ccea4af1c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.236+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/2176666500 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55ccea4bce40 0x55ccea4bf230 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.236+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/2176666500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a9390 0x55ccea4aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.236+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/2176666500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 0x55ccea4ac510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.236+0000 7f6afcff9700 0 client.16059 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:35:04.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.236+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/2176666500 >> 172.21.15.62:0/2176666500 conn(0x55ccea3c9cc0 msgr2=0x55ccea49d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:04.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.236+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/2176666500 shutdown_connections 2024-06-10T00:35:04.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.236+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/2176666500 wait complete. 2024-06-10T00:35:04.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.236+0000 7f6b2a25eb80 20 client.16059 trim_cache size 0 max 16384 2024-06-10T00:35:04.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.251+0000 7f6b2a25eb80 1 Processor -- start 2024-06-10T00:35:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.251+0000 7f6b2a25eb80 1 -- start start 2024-06-10T00:35:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.251+0000 7f6b2a25eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a3330 0x55ccea4a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.251+0000 7f6b2a25eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 0x55ccea4b9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.251+0000 7f6b2a25eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea493510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.251+0000 7f6b2a25eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181ac4c0 con 0x55ccea4a9390 2024-06-10T00:35:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.251+0000 7f6b2a25eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181ae000 con 0x55ccea4a8a70 2024-06-10T00:35:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.251+0000 7f6b2a25eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181acb90 con 0x55ccea4a3330 2024-06-10T00:35:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.252+0000 7f6b1df78700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea493510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.252+0000 7f6b1df78700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea493510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60502/0 (socket says 172.21.15.62:60502) 2024-06-10T00:35:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.252+0000 7f6b1df78700 1 -- 172.21.15.62:0/3671242404 learned_addr learned my addr 172.21.15.62:0/3671242404 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.252+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/3671242404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 0x55ccea4b9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.252+0000 7f6b1d777700 1 --2- 172.21.15.62:0/3671242404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a3330 0x55ccea4a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.252+0000 7f6b1df78700 1 -- 172.21.15.62:0/3671242404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a3330 msgr2=0x55ccea4a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:04.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.252+0000 7f6b1df78700 1 --2- 172.21.15.62:0/3671242404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a3330 0x55ccea4a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.252+0000 7f6b1df78700 1 -- 172.21.15.62:0/3671242404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 msgr2=0x55ccea4b9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:04.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.252+0000 7f6b1df78700 1 --2- 172.21.15.62:0/3671242404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 0x55ccea4b9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.252+0000 7f6b1df78700 1 -- 172.21.15.62:0/3671242404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccea4bf8b0 con 0x55ccea4a9390 2024-06-10T00:35:04.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.252+0000 7f6b1df78700 1 --2- 172.21.15.62:0/3671242404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea493510 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6b10015000 tx=0x7f6b10005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4daf6b26388ac144 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:04.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.252+0000 7f6afeffd700 1 -- 172.21.15.62:0/3671242404 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10029070 con 0x55ccea4a9390 2024-06-10T00:35:04.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.252+0000 7f6afeffd700 1 -- 172.21.15.62:0/3671242404 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b1000be90 con 0x55ccea4a9390 2024-06-10T00:35:04.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.252+0000 7f6afeffd700 1 -- 172.21.15.62:0/3671242404 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10003e50 con 0x55ccea4a9390 2024-06-10T00:35:04.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.253+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/3671242404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 msgr2=0x55ccea493510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:04.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.253+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/3671242404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea493510 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6b10015000 tx=0x7f6b10005960 comp rx=0 tx=0).stop 2024-06-10T00:35:04.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.253+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/3671242404 shutdown_connections 2024-06-10T00:35:04.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.253+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/3671242404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea493510 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.253+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/3671242404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a3330 0x55ccea4a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.253+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/3671242404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a8a70 0x55ccea4b9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.253+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/3671242404 >> 172.21.15.62:0/3671242404 conn(0x55ccea3c9cc0 msgr2=0x55ccea492e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:04.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.253+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/3671242404 shutdown_connections 2024-06-10T00:35:04.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.253+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/3671242404 wait complete. 2024-06-10T00:35:04.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.253+0000 7f6b2a25eb80 1 -- start start 2024-06-10T00:35:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.253+0000 7f6b2a25eb80 1 Processor -- start 2024-06-10T00:35:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b2a25eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a3330 0x55ccea4b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b2a25eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a8a70 0x55ccea4c4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b2a25eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea4c4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b2a25eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181ad930 con 0x55ccea4a9390 2024-06-10T00:35:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b2a25eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181ad260 con 0x55ccea4a3330 2024-06-10T00:35:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b2a25eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181aa480 con 0x55ccea4a8a70 2024-06-10T00:35:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b2a25eb80 10 client.0 _ll_register_callbacks cb 0x55ccea38a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b1df78700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea4c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b1df78700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea4c4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60512/0 (socket says 172.21.15.62:60512) 2024-06-10T00:35:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b1df78700 1 -- 172.21.15.62:0/994766988 learned_addr learned my addr 172.21.15.62:0/994766988 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a8a70 0x55ccea4c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b1d777700 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a3330 0x55ccea4b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b1cf76700 1 -- 172.21.15.62:0/994766988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a3330 msgr2=0x55ccea4b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:04.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a3330 0x55ccea4b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b1cf76700 1 -- 172.21.15.62:0/994766988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 msgr2=0x55ccea4c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:04.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea4c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b1cf76700 1 -- 172.21.15.62:0/994766988 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccea4c5180 con 0x55ccea4a8a70 2024-06-10T00:35:04.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b1d777700 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a3330 0x55ccea4b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:35:04.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.254+0000 7f6b1df78700 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea4c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:35:04.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6b1cf76700 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a8a70 0x55ccea4c4450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c01d450 tx=0x7f6b0c009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10567d88fb3db703 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:04.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6afcff9700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:35:04.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6afcff9700 1 -- 172.21.15.62:0/994766988 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c035040 con 0x55ccea4a8a70 2024-06-10T00:35:04.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/994766988 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b1000be90 con 0x55ccea4a8a70 2024-06-10T00:35:04.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6afcff9700 1 -- 172.21.15.62:0/994766988 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f6b0c005480 con 0x55ccea4a8a70 2024-06-10T00:35:04.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6afcff9700 1 -- 172.21.15.62:0/994766988 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6b0c042050 con 0x55ccea4a8a70 2024-06-10T00:35:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6b2a25eb80 10 client.25612 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/994766988 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ccea527b00 con 0x55ccea4a8a70 2024-06-10T00:35:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6afcff9700 1 -- 172.21.15.62:0/994766988 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b1000be90 con 0x55ccea4a8a70 2024-06-10T00:35:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6afcff9700 1 -- 172.21.15.62:0/994766988 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6b0c01d750 con 0x55ccea4a8a70 2024-06-10T00:35:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6b2a25eb80 10 client.25612 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6b2a25eb80 10 client.25612 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:35:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6b2a25eb80 20 client.25612 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55ccea527f00 0x55ccea52a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6b2a25eb80 4 client.25612 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:35:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.255+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/994766988 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccea272250 con 0x55ccea527f00 2024-06-10T00:35:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.256+0000 7f6b1d777700 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55ccea527f00 0x55ccea52a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.256+0000 7f6afcff9700 10 client.25612 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:35:04.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.256+0000 7f6b1d777700 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55ccea527f00 0x55ccea52a320 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a28afcfd1ba54c0d server_cookie=55ca5ad258f956f2 in_seq=0 out_seq=0 2024-06-10T00:35:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.256+0000 7f6afcff9700 1 -- 172.21.15.62:0/994766988 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccea272250 con 0x55ccea527f00 2024-06-10T00:35:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.256+0000 7f6afcff9700 10 client.25612 handle_command_reply: tid=0 2024-06-10T00:35:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.256+0000 7f6b2a25eb80 1 client.25612 shutdown 2024-06-10T00:35:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.256+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/994766988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a8a70 msgr2=0x55ccea4c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.256+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a8a70 0x55ccea4c4450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c01d450 tx=0x7f6b0c009f20 comp rx=0 tx=0).stop 2024-06-10T00:35:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.256+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/994766988 shutdown_connections 2024-06-10T00:35:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.256+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccea4a9390 0x55ccea4c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.256+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55ccea527f00 0x55ccea52a320 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.256+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ccea4a3330 0x55ccea4b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.256+0000 7f6b2a25eb80 1 --2- 172.21.15.62:0/994766988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccea4a8a70 0x55ccea4c4450 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.256+0000 7f6afcff9700 0 client.25612 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:35:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.257+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/994766988 >> 172.21.15.62:0/994766988 conn(0x55ccea3c9cc0 msgr2=0x55ccea49d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:04.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.257+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/994766988 shutdown_connections 2024-06-10T00:35:04.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.257+0000 7f6b2a25eb80 1 -- 172.21.15.62:0/994766988 wait complete. 2024-06-10T00:35:04.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.257+0000 7f6b2a25eb80 20 client.25612 trim_cache size 0 max 16384 2024-06-10T00:35:04.263 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:35:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.257+0000 7f6b1f9db700 1 -- 172.21.15.62:0/298263662 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6b000a2d90 msgr2=0x7f6b000a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.257+0000 7f6b1f9db700 1 --2- 172.21.15.62:0/298263662 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6b000a2d90 0x7f6b000a5250 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6b18078900 tx=0x7f6b0c009260 comp rx=0 tx=0).stop 2024-06-10T00:35:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.257+0000 7f6b1f9db700 1 -- 172.21.15.62:0/298263662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b1806aa00 msgr2=0x7f6b1819e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.257+0000 7f6b1f9db700 1 --2- 172.21.15.62:0/298263662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b1806aa00 0x7f6b1819e810 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800ef10 tx=0x7f6b0800c3a0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.257+0000 7f6b1f9db700 1 -- 172.21.15.62:0/298263662 shutdown_connections 2024-06-10T00:35:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.257+0000 7f6b1f9db700 1 --2- 172.21.15.62:0/298263662 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6b000a2d90 0x7f6b000a5250 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.257+0000 7f6b1f9db700 1 --2- 172.21.15.62:0/298263662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b181084f0 0x7f6b181989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.257+0000 7f6b1f9db700 1 --2- 172.21.15.62:0/298263662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b1806aa00 0x7f6b1819e810 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.257+0000 7f6b1f9db700 1 --2- 172.21.15.62:0/298263662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b1806b3c0 0x7f6b1819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.257+0000 7f6b1f9db700 1 -- 172.21.15.62:0/298263662 >> 172.21.15.62:0/298263662 conn(0x7f6b18078300 msgr2=0x7f6b180fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.258+0000 7f6b1f9db700 1 -- 172.21.15.62:0/298263662 shutdown_connections 2024-06-10T00:35:04.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:04.258+0000 7f6b1f9db700 1 -- 172.21.15.62:0/298263662 wait complete. 2024-06-10T00:35:04.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:04 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:35:04.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:04 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:35:04.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:04 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:35:04.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:04 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:35:04.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:04 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:35:04.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:04 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:35:04.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:04 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:35:04.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:04 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:35:04.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:04 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:35:05.028 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T00:35:05.277 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:35:05.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:05 smithi062 ceph-mon[32390]: pgmap v70: 129 pgs: 129 active+clean; 8.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 200 KiB/s wr, 66 op/s 2024-06-10T00:35:05.683 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:05 smithi105 ceph-mon[36321]: pgmap v70: 129 pgs: 129 active+clean; 8.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 200 KiB/s wr, 66 op/s 2024-06-10T00:35:05.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:05 smithi138 ceph-mon[36549]: pgmap v70: 129 pgs: 129 active+clean; 8.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 200 KiB/s wr, 66 op/s 2024-06-10T00:35:07.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.014+0000 7f8d5e4ad700 1 -- 172.21.15.62:0/2572548054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d580746c0 msgr2=0x7f8d58074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:07.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.014+0000 7f8d5e4ad700 1 --2- 172.21.15.62:0/2572548054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d580746c0 0x7f8d58074aa0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009990 tx=0x7f8d48009ca0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.014+0000 7f8d5e4ad700 1 -- 172.21.15.62:0/2572548054 shutdown_connections 2024-06-10T00:35:07.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.014+0000 7f8d5e4ad700 1 --2- 172.21.15.62:0/2572548054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d580746c0 0x7f8d58074aa0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.014+0000 7f8d5e4ad700 1 --2- 172.21.15.62:0/2572548054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d5810ee90 0x7f8d58111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.014+0000 7f8d5e4ad700 1 --2- 172.21.15.62:0/2572548054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d58074fe0 0x7f8d5810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.014+0000 7f8d5e4ad700 1 -- 172.21.15.62:0/2572548054 >> 172.21.15.62:0/2572548054 conn(0x7f8d580795f0 msgr2=0x7f8d58079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:07.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.014+0000 7f8d5e4ad700 1 -- 172.21.15.62:0/2572548054 shutdown_connections 2024-06-10T00:35:07.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.014+0000 7f8d5e4ad700 1 -- 172.21.15.62:0/2572548054 wait complete. 2024-06-10T00:35:07.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.014+0000 7f8d5e4ad700 1 Processor -- start 2024-06-10T00:35:07.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.014+0000 7f8d5e4ad700 1 -- start start 2024-06-10T00:35:07.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d5e4ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d580746c0 0x7f8d5819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d5e4ad700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d58074fe0 0x7f8d5819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d5e4ad700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d5810ee90 0x7f8d5819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d5e4ad700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d58114960 con 0x7f8d58074fe0 2024-06-10T00:35:07.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d5e4ad700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d581147e0 con 0x7f8d5810ee90 2024-06-10T00:35:07.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d5e4ad700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d58114ae0 con 0x7f8d580746c0 2024-06-10T00:35:07.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d58074fe0 0x7f8d5819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d58074fe0 0x7f8d5819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60522/0 (socket says 172.21.15.62:60522) 2024-06-10T00:35:07.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d57fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d580746c0 0x7f8d5819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d577fe700 1 -- 172.21.15.62:0/2014468023 learned_addr learned my addr 172.21.15.62:0/2014468023 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:07.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d5ca4a700 1 --2- 172.21.15.62:0/2014468023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d5810ee90 0x7f8d5819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d577fe700 1 -- 172.21.15.62:0/2014468023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d580746c0 msgr2=0x7f8d5819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:07.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d577fe700 1 --2- 172.21.15.62:0/2014468023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d580746c0 0x7f8d5819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d577fe700 1 -- 172.21.15.62:0/2014468023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d5810ee90 msgr2=0x7f8d5819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:07.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d577fe700 1 --2- 172.21.15.62:0/2014468023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d5810ee90 0x7f8d5819f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d577fe700 1 -- 172.21.15.62:0/2014468023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5819fe20 con 0x7f8d58074fe0 2024-06-10T00:35:07.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d5ca4a700 1 --2- 172.21.15.62:0/2014468023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d5810ee90 0x7f8d5819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:35:07.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.015+0000 7f8d577fe700 1 --2- 172.21.15.62:0/2014468023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d58074fe0 0x7f8d5819b350 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00c920 tx=0x7f8d4c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:07.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.016+0000 7f8d557fa700 1 -- 172.21.15.62:0/2014468023 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c007b00 con 0x7f8d58074fe0 2024-06-10T00:35:07.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.016+0000 7f8d5e4ad700 1 -- 172.21.15.62:0/2014468023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d581a0110 con 0x7f8d58074fe0 2024-06-10T00:35:07.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.016+0000 7f8d557fa700 1 -- 172.21.15.62:0/2014468023 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d4c004d10 con 0x7f8d58074fe0 2024-06-10T00:35:07.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.016+0000 7f8d557fa700 1 -- 172.21.15.62:0/2014468023 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c005630 con 0x7f8d58074fe0 2024-06-10T00:35:07.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.016+0000 7f8d5e4ad700 1 -- 172.21.15.62:0/2014468023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d581a7990 con 0x7f8d58074fe0 2024-06-10T00:35:07.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.018+0000 7f8d557fa700 1 -- 172.21.15.62:0/2014468023 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8d4c004750 con 0x7f8d58074fe0 2024-06-10T00:35:07.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.019+0000 7f8d557fa700 1 --2- 172.21.15.62:0/2014468023 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8d400a2d90 0x7f8d400a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.019+0000 7f8d57fff700 1 --2- 172.21.15.62:0/2014468023 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8d400a2d90 0x7f8d400a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.019+0000 7f8d557fa700 1 -- 172.21.15.62:0/2014468023 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f8d4c0cf390 con 0x7f8d58074fe0 2024-06-10T00:35:07.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.019+0000 7f8d57fff700 1 --2- 172.21.15.62:0/2014468023 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8d400a2d90 0x7f8d400a5250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800b5c0 tx=0x7f8d48005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:07.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.020+0000 7f8d68d30b80 1 Processor -- start 2024-06-10T00:35:07.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.020+0000 7f8d68d30b80 1 -- start start 2024-06-10T00:35:07.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.020+0000 7f8d68d30b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4640 0x5568175e4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.020+0000 7f8d68d30b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4f60 0x5568175de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.020+0000 7f8d68d30b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175def20 0x5568175df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.020+0000 7f8d68d30b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556817530810 con 0x5568175e4640 2024-06-10T00:35:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.020+0000 7f8d68d30b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5568174d8720 con 0x5568175e4f60 2024-06-10T00:35:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.020+0000 7f8d68d30b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55681752c7d0 con 0x5568175def20 2024-06-10T00:35:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.020+0000 7f8d57fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4640 0x5568175e4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.020+0000 7f8d57fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4640 0x5568175e4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60538/0 (socket says 172.21.15.62:60538) 2024-06-10T00:35:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.020+0000 7f8d57fff700 1 -- 172.21.15.62:0/1868382388 learned_addr learned my addr 172.21.15.62:0/1868382388 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.021+0000 7f8d5ca4a700 1 --2- 172.21.15.62:0/1868382388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175def20 0x5568175df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.021+0000 7f8d57fff700 1 -- 172.21.15.62:0/1868382388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175def20 msgr2=0x5568175df340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.021+0000 7f8d57fff700 1 --2- 172.21.15.62:0/1868382388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175def20 0x5568175df340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.021+0000 7f8d57fff700 1 -- 172.21.15.62:0/1868382388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4f60 msgr2=0x5568175de9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.021+0000 7f8d57fff700 1 --2- 172.21.15.62:0/1868382388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4f60 0x5568175de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.021+0000 7f8d57fff700 1 -- 172.21.15.62:0/1868382388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568174cef50 con 0x5568175e4640 2024-06-10T00:35:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.021+0000 7f8d57fff700 1 --2- 172.21.15.62:0/1868382388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4640 0x5568175e4a20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8d48015000 tx=0x7f8d48020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1771b4bbfa551e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:07.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.021+0000 7f8d3e7fc700 1 -- 172.21.15.62:0/1868382388 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4800be50 con 0x5568175e4640 2024-06-10T00:35:07.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.021+0000 7f8d3e7fc700 1 -- 172.21.15.62:0/1868382388 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d4800fd50 con 0x5568175e4640 2024-06-10T00:35:07.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.021+0000 7f8d3e7fc700 1 -- 172.21.15.62:0/1868382388 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4802f400 con 0x5568175e4640 2024-06-10T00:35:07.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.022+0000 7f8d68d30b80 1 -- 172.21.15.62:0/1868382388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4640 msgr2=0x5568175e4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:07.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.022+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/1868382388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4640 0x5568175e4a20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8d48015000 tx=0x7f8d48020000 comp rx=0 tx=0).stop 2024-06-10T00:35:07.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.022+0000 7f8d68d30b80 1 -- 172.21.15.62:0/1868382388 shutdown_connections 2024-06-10T00:35:07.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.022+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/1868382388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4640 0x5568175e4a20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.022+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/1868382388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175def20 0x5568175df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.022+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/1868382388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4f60 0x5568175de9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.022+0000 7f8d68d30b80 1 -- 172.21.15.62:0/1868382388 >> 172.21.15.62:0/1868382388 conn(0x556817509cc0 msgr2=0x5568175d9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:07.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.022+0000 7f8d68d30b80 1 -- 172.21.15.62:0/1868382388 shutdown_connections 2024-06-10T00:35:07.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.022+0000 7f8d68d30b80 1 -- 172.21.15.62:0/1868382388 wait complete. 2024-06-10T00:35:07.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.022+0000 7f8d68d30b80 1 -- start start 2024-06-10T00:35:07.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d68d30b80 1 Processor -- start 2024-06-10T00:35:07.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d68d30b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175def20 0x5568175ead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d68d30b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175e4640 0x5568175e8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d68d30b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4f60 0x5568175e8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d68d30b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5568175eb910 con 0x5568175def20 2024-06-10T00:35:07.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d68d30b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5568174d0bf0 con 0x5568175e4f60 2024-06-10T00:35:07.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d68d30b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5568175eba90 con 0x5568175e4640 2024-06-10T00:35:07.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d68d30b80 10 client.0 _ll_register_callbacks cb 0x5568174ca920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:07.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d57fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175def20 0x5568175ead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d57fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175def20 0x5568175ead20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60546/0 (socket says 172.21.15.62:60546) 2024-06-10T00:35:07.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d57fff700 1 -- 172.21.15.62:0/3725097411 learned_addr learned my addr 172.21.15.62:0/3725097411 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:07.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d577fe700 1 --2- 172.21.15.62:0/3725097411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175e4640 0x5568175e8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d5ca4a700 1 --2- 172.21.15.62:0/3725097411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4f60 0x5568175e8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d57fff700 1 -- 172.21.15.62:0/3725097411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175e4640 msgr2=0x5568175e8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:07.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d57fff700 1 --2- 172.21.15.62:0/3725097411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175e4640 0x5568175e8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d57fff700 1 -- 172.21.15.62:0/3725097411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4f60 msgr2=0x5568175e8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:07.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d57fff700 1 --2- 172.21.15.62:0/3725097411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4f60 0x5568175e8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.023+0000 7f8d57fff700 1 -- 172.21.15.62:0/3725097411 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55681747f8d0 con 0x5568175def20 2024-06-10T00:35:07.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.024+0000 7f8d57fff700 1 --2- 172.21.15.62:0/3725097411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175def20 0x5568175ead20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009350 tx=0x7f8d4800fb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896374ac9b7657d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:07.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.024+0000 7f8d377fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:07.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.024+0000 7f8d377fe700 1 -- 172.21.15.62:0/3725097411 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48024620 con 0x5568175def20 2024-06-10T00:35:07.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.024+0000 7f8d377fe700 1 -- 172.21.15.62:0/3725097411 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f8d4802f400 con 0x5568175def20 2024-06-10T00:35:07.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.024+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3725097411 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d4800fd50 con 0x5568175def20 2024-06-10T00:35:07.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.024+0000 7f8d377fe700 1 -- 172.21.15.62:0/3725097411 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d4802f600 con 0x5568175def20 2024-06-10T00:35:07.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.024+0000 7f8d68d30b80 10 client.16077 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:07.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.024+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3725097411 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568175f62d0 con 0x5568175def20 2024-06-10T00:35:07.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.024+0000 7f8d377fe700 1 -- 172.21.15.62:0/3725097411 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d4800fd50 con 0x5568175def20 2024-06-10T00:35:07.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.025+0000 7f8d377fe700 1 -- 172.21.15.62:0/3725097411 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8d480376a0 con 0x5568175def20 2024-06-10T00:35:07.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.025+0000 7f8d68d30b80 10 client.16077 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:07.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.025+0000 7f8d68d30b80 10 client.16077 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:35:07.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.025+0000 7f8d68d30b80 20 client.16077 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:07.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.025+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/3725097411 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5568175f87a0 0x5568175fab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.025+0000 7f8d68d30b80 4 client.16077 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:35:07.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.025+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3725097411 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55681753dd10 con 0x5568175f87a0 2024-06-10T00:35:07.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.025+0000 7f8d577fe700 1 --2- 172.21.15.62:0/3725097411 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5568175f87a0 0x5568175fab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.026+0000 7f8d377fe700 10 client.16077 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:35:07.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.026+0000 7f8d577fe700 1 --2- 172.21.15.62:0/3725097411 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5568175f87a0 0x5568175fab90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb1de723c3efbe11 server_cookie=deef4714e28dc03c in_seq=0 out_seq=0 2024-06-10T00:35:07.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.026+0000 7f8d377fe700 1 -- 172.21.15.62:0/3725097411 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55681753dd10 con 0x5568175f87a0 2024-06-10T00:35:07.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.026+0000 7f8d377fe700 10 client.16077 handle_command_reply: tid=0 2024-06-10T00:35:07.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.026+0000 7f8d68d30b80 1 client.16077 shutdown 2024-06-10T00:35:07.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.027+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3725097411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175def20 msgr2=0x5568175ead20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:07.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.027+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/3725097411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175def20 0x5568175ead20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009350 tx=0x7f8d4800fb50 comp rx=0 tx=0).stop 2024-06-10T00:35:07.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.027+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3725097411 shutdown_connections 2024-06-10T00:35:07.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.027+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/3725097411 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5568175f87a0 0x5568175fab90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.027+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/3725097411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175def20 0x5568175ead20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.027+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/3725097411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4f60 0x5568175e8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.027+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/3725097411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175e4640 0x5568175e8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.027+0000 7f8d377fe700 0 client.16077 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:35:07.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.027+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3725097411 >> 172.21.15.62:0/3725097411 conn(0x556817509cc0 msgr2=0x5568175dd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:07.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.027+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3725097411 shutdown_connections 2024-06-10T00:35:07.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.027+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3725097411 wait complete. 2024-06-10T00:35:07.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.027+0000 7f8d68d30b80 20 client.16077 trim_cache size 0 max 16384 2024-06-10T00:35:07.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.042+0000 7f8d68d30b80 1 Processor -- start 2024-06-10T00:35:07.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.042+0000 7f8d68d30b80 1 -- start start 2024-06-10T00:35:07.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.042+0000 7f8d68d30b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175def20 0x5568175de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.042+0000 7f8d68d30b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4640 0x5568175e5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.042+0000 7f8d68d30b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175e4f60 0x5568175e3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.042+0000 7f8d68d30b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d581b06c0 con 0x5568175e4640 2024-06-10T00:35:07.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.042+0000 7f8d68d30b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d581b2200 con 0x5568175def20 2024-06-10T00:35:07.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.042+0000 7f8d68d30b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d581b0d90 con 0x5568175e4f60 2024-06-10T00:35:07.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.042+0000 7f8d577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4640 0x5568175e5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.042+0000 7f8d577fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4640 0x5568175e5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60556/0 (socket says 172.21.15.62:60556) 2024-06-10T00:35:07.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.043+0000 7f8d5ca4a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175e4f60 0x5568175e3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.043+0000 7f8d577fe700 1 -- 172.21.15.62:0/241504468 learned_addr learned my addr 172.21.15.62:0/241504468 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:07.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.043+0000 7f8d57fff700 1 --2- 172.21.15.62:0/241504468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175def20 0x5568175de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.043+0000 7f8d577fe700 1 -- 172.21.15.62:0/241504468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175e4f60 msgr2=0x5568175e3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:07.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.043+0000 7f8d577fe700 1 --2- 172.21.15.62:0/241504468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175e4f60 0x5568175e3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.043+0000 7f8d577fe700 1 -- 172.21.15.62:0/241504468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175def20 msgr2=0x5568175de9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:07.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.043+0000 7f8d577fe700 1 --2- 172.21.15.62:0/241504468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175def20 0x5568175de9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.043+0000 7f8d577fe700 1 -- 172.21.15.62:0/241504468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568175db2c0 con 0x5568175e4640 2024-06-10T00:35:07.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.043+0000 7f8d577fe700 1 --2- 172.21.15.62:0/241504468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4640 0x5568175e5a50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00cfb0 tx=0x7f8d4c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a6a3da76063a85 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:07.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.043+0000 7f8d3dffb700 1 -- 172.21.15.62:0/241504468 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c0b4070 con 0x5568175e4640 2024-06-10T00:35:07.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.043+0000 7f8d3dffb700 1 -- 172.21.15.62:0/241504468 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d4c09b500 con 0x5568175e4640 2024-06-10T00:35:07.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.043+0000 7f8d3dffb700 1 -- 172.21.15.62:0/241504468 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c01d040 con 0x5568175e4640 2024-06-10T00:35:07.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.044+0000 7f8d68d30b80 1 -- 172.21.15.62:0/241504468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4640 msgr2=0x5568175e5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:07.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.044+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/241504468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4640 0x5568175e5a50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00cfb0 tx=0x7f8d4c000bc0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.044+0000 7f8d68d30b80 1 -- 172.21.15.62:0/241504468 shutdown_connections 2024-06-10T00:35:07.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.044+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/241504468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175e4f60 0x5568175e3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.044+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/241504468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175def20 0x5568175de9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.044+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/241504468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4640 0x5568175e5a50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.044+0000 7f8d68d30b80 1 -- 172.21.15.62:0/241504468 >> 172.21.15.62:0/241504468 conn(0x556817509cc0 msgr2=0x5568175e5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:07.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.044+0000 7f8d68d30b80 1 -- 172.21.15.62:0/241504468 shutdown_connections 2024-06-10T00:35:07.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.044+0000 7f8d68d30b80 1 -- 172.21.15.62:0/241504468 wait complete. 2024-06-10T00:35:07.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.044+0000 7f8d68d30b80 1 -- start start 2024-06-10T00:35:07.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.044+0000 7f8d68d30b80 1 Processor -- start 2024-06-10T00:35:07.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d68d30b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175def20 0x5568175fe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d68d30b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4640 0x5568175fb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d68d30b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4f60 0x5568175fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d68d30b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d581b1b30 con 0x5568175e4f60 2024-06-10T00:35:07.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d68d30b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d581b1460 con 0x5568175e4640 2024-06-10T00:35:07.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d68d30b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d581ae6b0 con 0x5568175def20 2024-06-10T00:35:07.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d68d30b80 10 client.0 _ll_register_callbacks cb 0x5568174ca920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:07.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d5ca4a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4f60 0x5568175fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d5ca4a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4f60 0x5568175fbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60568/0 (socket says 172.21.15.62:60568) 2024-06-10T00:35:07.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d5ca4a700 1 -- 172.21.15.62:0/3910671148 learned_addr learned my addr 172.21.15.62:0/3910671148 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:07.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d57fff700 1 --2- 172.21.15.62:0/3910671148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175def20 0x5568175fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d577fe700 1 --2- 172.21.15.62:0/3910671148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4640 0x5568175fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d5ca4a700 1 -- 172.21.15.62:0/3910671148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175def20 msgr2=0x5568175fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:07.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d5ca4a700 1 --2- 172.21.15.62:0/3910671148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175def20 0x5568175fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d5ca4a700 1 -- 172.21.15.62:0/3910671148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4640 msgr2=0x5568175fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:07.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d5ca4a700 1 --2- 172.21.15.62:0/3910671148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4640 0x5568175fb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d5ca4a700 1 -- 172.21.15.62:0/3910671148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568175fc530 con 0x5568175e4f60 2024-06-10T00:35:07.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d577fe700 1 --2- 172.21.15.62:0/3910671148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4640 0x5568175fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:35:07.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d5ca4a700 1 --2- 172.21.15.62:0/3910671148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4f60 0x5568175fbca0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8d500146d0 tx=0x7f8d500149e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9ffcf750b59942 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:07.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.045+0000 7f8d377fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:07.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.046+0000 7f8d377fe700 1 -- 172.21.15.62:0/3910671148 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5001b070 con 0x5568175e4f60 2024-06-10T00:35:07.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.046+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3910671148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d4c09b500 con 0x5568175e4f60 2024-06-10T00:35:07.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.046+0000 7f8d377fe700 1 -- 172.21.15.62:0/3910671148 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f8d50004050 con 0x5568175e4f60 2024-06-10T00:35:07.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.046+0000 7f8d377fe700 1 -- 172.21.15.62:0/3910671148 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d50027050 con 0x5568175e4f60 2024-06-10T00:35:07.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.046+0000 7f8d68d30b80 10 client.16083 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:07.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.046+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3910671148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568176631e0 con 0x5568175e4f60 2024-06-10T00:35:07.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.046+0000 7f8d377fe700 1 -- 172.21.15.62:0/3910671148 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d4c09b500 con 0x5568175e4f60 2024-06-10T00:35:07.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.046+0000 7f8d377fe700 1 -- 172.21.15.62:0/3910671148 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8d500169a0 con 0x5568175e4f60 2024-06-10T00:35:07.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.046+0000 7f8d68d30b80 10 client.16083 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:07.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.046+0000 7f8d68d30b80 10 client.16083 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:35:07.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.046+0000 7f8d68d30b80 20 client.16083 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:07.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.046+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/3910671148 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x556817663920 0x556817665d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:07.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.046+0000 7f8d68d30b80 4 client.16083 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:35:07.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.046+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3910671148 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568175d3a20 con 0x556817663920 2024-06-10T00:35:07.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.047+0000 7f8d57fff700 1 --2- 172.21.15.62:0/3910671148 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x556817663920 0x556817665d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:07.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.047+0000 7f8d57fff700 1 --2- 172.21.15.62:0/3910671148 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x556817663920 0x556817665d40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc261c06835ecf46 server_cookie=38a2198051e8de09 in_seq=0 out_seq=0 2024-06-10T00:35:07.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.047+0000 7f8d377fe700 10 client.16083 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:35:07.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.047+0000 7f8d377fe700 1 -- 172.21.15.62:0/3910671148 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568175d3a20 con 0x556817663920 2024-06-10T00:35:07.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.047+0000 7f8d377fe700 10 client.16083 handle_command_reply: tid=0 2024-06-10T00:35:07.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.047+0000 7f8d68d30b80 1 client.16083 shutdown 2024-06-10T00:35:07.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3910671148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4f60 msgr2=0x5568175fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:07.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/3910671148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4f60 0x5568175fbca0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8d500146d0 tx=0x7f8d500149e0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3910671148 shutdown_connections 2024-06-10T00:35:07.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/3910671148 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x556817663920 0x556817665d40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/3910671148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5568175e4f60 0x5568175fbca0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/3910671148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568175def20 0x5568175fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d377fe700 0 client.16083 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:35:07.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d68d30b80 1 --2- 172.21.15.62:0/3910671148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568175e4640 0x5568175fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3910671148 >> 172.21.15.62:0/3910671148 conn(0x556817509cc0 msgr2=0x5568175e67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:07.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3910671148 shutdown_connections 2024-06-10T00:35:07.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d68d30b80 1 -- 172.21.15.62:0/3910671148 wait complete. 2024-06-10T00:35:07.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d68d30b80 20 client.16083 trim_cache size 0 max 16384 2024-06-10T00:35:07.054 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:35:07.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d5e4ad700 1 -- 172.21.15.62:0/2014468023 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8d400a2d90 msgr2=0x7f8d400a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:07.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d5e4ad700 1 --2- 172.21.15.62:0/2014468023 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8d400a2d90 0x7f8d400a5250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800b5c0 tx=0x7f8d48005960 comp rx=0 tx=0).stop 2024-06-10T00:35:07.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d5e4ad700 1 -- 172.21.15.62:0/2014468023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d58074fe0 msgr2=0x7f8d5819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:07.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.048+0000 7f8d5e4ad700 1 --2- 172.21.15.62:0/2014468023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d58074fe0 0x7f8d5819b350 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00c920 tx=0x7f8d4c00cc30 comp rx=0 tx=0).stop 2024-06-10T00:35:07.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.049+0000 7f8d5e4ad700 1 -- 172.21.15.62:0/2014468023 shutdown_connections 2024-06-10T00:35:07.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.049+0000 7f8d5e4ad700 1 --2- 172.21.15.62:0/2014468023 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8d400a2d90 0x7f8d400a5250 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.049+0000 7f8d5e4ad700 1 --2- 172.21.15.62:0/2014468023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d5810ee90 0x7f8d5819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.049+0000 7f8d5e4ad700 1 --2- 172.21.15.62:0/2014468023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d580746c0 0x7f8d5819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.049+0000 7f8d5e4ad700 1 --2- 172.21.15.62:0/2014468023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d58074fe0 0x7f8d5819b350 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:07.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.049+0000 7f8d5e4ad700 1 -- 172.21.15.62:0/2014468023 >> 172.21.15.62:0/2014468023 conn(0x7f8d580795f0 msgr2=0x7f8d58100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:07.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.049+0000 7f8d5e4ad700 1 -- 172.21.15.62:0/2014468023 shutdown_connections 2024-06-10T00:35:07.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:07.049+0000 7f8d5e4ad700 1 -- 172.21.15.62:0/2014468023 wait complete. 2024-06-10T00:35:07.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:07 smithi105 ceph-mon[36321]: pgmap v71: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 209 KiB/s wr, 72 op/s 2024-06-10T00:35:07.837 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:07 smithi062 ceph-mon[32390]: pgmap v71: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 209 KiB/s wr, 72 op/s 2024-06-10T00:35:07.869 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T00:35:07.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:07 smithi138 ceph-mon[36549]: pgmap v71: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 209 KiB/s wr, 72 op/s 2024-06-10T00:35:08.123 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:35:09.602 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:09 smithi062 ceph-mon[32390]: pgmap v72: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 141 KiB/s wr, 51 op/s 2024-06-10T00:35:09.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:09 smithi105 ceph-mon[36321]: pgmap v72: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 141 KiB/s wr, 51 op/s 2024-06-10T00:35:09.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.842+0000 7f47ca7a9700 1 -- 172.21.15.62:0/2096168310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c4107550 msgr2=0x7f47c4107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:09.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.842+0000 7f47ca7a9700 1 --2- 172.21.15.62:0/2096168310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c4107550 0x7f47c4107930 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009990 tx=0x7f47b4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.842+0000 7f47ca7a9700 1 -- 172.21.15.62:0/2096168310 shutdown_connections 2024-06-10T00:35:09.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.842+0000 7f47ca7a9700 1 --2- 172.21.15.62:0/2096168310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c4107550 0x7f47c4107930 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.842+0000 7f47ca7a9700 1 --2- 172.21.15.62:0/2096168310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47c41017f0 0x7f47c410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.842+0000 7f47ca7a9700 1 --2- 172.21.15.62:0/2096168310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c4100e30 0x7f47c41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.842+0000 7f47ca7a9700 1 -- 172.21.15.62:0/2096168310 >> 172.21.15.62:0/2096168310 conn(0x7f47c40766b0 msgr2=0x7f47c4076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:09.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.842+0000 7f47ca7a9700 1 -- 172.21.15.62:0/2096168310 shutdown_connections 2024-06-10T00:35:09.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.842+0000 7f47ca7a9700 1 -- 172.21.15.62:0/2096168310 wait complete. 2024-06-10T00:35:09.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.842+0000 7f47ca7a9700 1 Processor -- start 2024-06-10T00:35:09.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.842+0000 7f47ca7a9700 1 -- start start 2024-06-10T00:35:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47ca7a9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c4100e30 0x7f47c4198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47ca7a9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47c41017f0 0x7f47c4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47ca7a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c4107550 0x7f47c419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47ca7a9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4112810 con 0x7f47c4107550 2024-06-10T00:35:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47ca7a9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4112690 con 0x7f47c41017f0 2024-06-10T00:35:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47ca7a9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4112990 con 0x7f47c4100e30 2024-06-10T00:35:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47c8d46700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c4107550 0x7f47c419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47c8d46700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c4107550 0x7f47c419d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44242/0 (socket says 172.21.15.62:44242) 2024-06-10T00:35:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47c8d46700 1 -- 172.21.15.62:0/3739537075 learned_addr learned my addr 172.21.15.62:0/3739537075 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47c3fff700 1 --2- 172.21.15.62:0/3739537075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c4100e30 0x7f47c4198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47c37fe700 1 --2- 172.21.15.62:0/3739537075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47c41017f0 0x7f47c4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47c8d46700 1 -- 172.21.15.62:0/3739537075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c4100e30 msgr2=0x7f47c4198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:09.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47c8d46700 1 --2- 172.21.15.62:0/3739537075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c4100e30 0x7f47c4198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47c8d46700 1 -- 172.21.15.62:0/3739537075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47c41017f0 msgr2=0x7f47c4199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:09.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47c8d46700 1 --2- 172.21.15.62:0/3739537075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47c41017f0 0x7f47c4199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.843+0000 7f47c8d46700 1 -- 172.21.15.62:0/3739537075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c419dd50 con 0x7f47c4107550 2024-06-10T00:35:09.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.844+0000 7f47c8d46700 1 --2- 172.21.15.62:0/3739537075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c4107550 0x7f47c419d610 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00ada0 tx=0x7f47bc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:09.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.844+0000 7f47c17fa700 1 -- 172.21.15.62:0/3739537075 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc017020 con 0x7f47c4107550 2024-06-10T00:35:09.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.844+0000 7f47ca7a9700 1 -- 172.21.15.62:0/3739537075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c419e040 con 0x7f47c4107550 2024-06-10T00:35:09.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.844+0000 7f47c17fa700 1 -- 172.21.15.62:0/3739537075 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47bc00a0d0 con 0x7f47c4107550 2024-06-10T00:35:09.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.844+0000 7f47c17fa700 1 -- 172.21.15.62:0/3739537075 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc005480 con 0x7f47c4107550 2024-06-10T00:35:09.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.844+0000 7f47ca7a9700 1 -- 172.21.15.62:0/3739537075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c41a58c0 con 0x7f47c4107550 2024-06-10T00:35:09.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.847+0000 7f47c17fa700 1 -- 172.21.15.62:0/3739537075 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f47bc005620 con 0x7f47c4107550 2024-06-10T00:35:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.848+0000 7f47c17fa700 1 --2- 172.21.15.62:0/3739537075 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f47ac0a2d90 0x7f47ac0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.848+0000 7f47c3fff700 1 --2- 172.21.15.62:0/3739537075 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f47ac0a2d90 0x7f47ac0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.848+0000 7f47c17fa700 1 -- 172.21.15.62:0/3739537075 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f47bc0cef90 con 0x7f47c4107550 2024-06-10T00:35:09.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.848+0000 7f47c3fff700 1 --2- 172.21.15.62:0/3739537075 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f47ac0a2d90 0x7f47ac0a5250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47b400b5c0 tx=0x7f47b4005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:09.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.848+0000 7f47d502cb80 1 Processor -- start 2024-06-10T00:35:09.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.848+0000 7f47d502cb80 1 -- start start 2024-06-10T00:35:09.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47d502cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc35f60 0x55edbcc36340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47d502cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc36880 0x55edbcc30260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47d502cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc307a0 0x55edbcc30bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47d502cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55edbcb22f60 con 0x55edbcc35f60 2024-06-10T00:35:09.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47d502cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55edbcb81850 con 0x55edbcc36880 2024-06-10T00:35:09.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47d502cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55edbcae9460 con 0x55edbcc307a0 2024-06-10T00:35:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47c3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc35f60 0x55edbcc36340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47c3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc35f60 0x55edbcc36340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44256/0 (socket says 172.21.15.62:44256) 2024-06-10T00:35:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47c3fff700 1 -- 172.21.15.62:0/2782025461 learned_addr learned my addr 172.21.15.62:0/2782025461 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47c37fe700 1 --2- 172.21.15.62:0/2782025461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc36880 0x55edbcc30260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47c8d46700 1 --2- 172.21.15.62:0/2782025461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc307a0 0x55edbcc30bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47c3fff700 1 -- 172.21.15.62:0/2782025461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc307a0 msgr2=0x55edbcc30bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47c3fff700 1 --2- 172.21.15.62:0/2782025461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc307a0 0x55edbcc30bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47c3fff700 1 -- 172.21.15.62:0/2782025461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc36880 msgr2=0x55edbcc30260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47c3fff700 1 --2- 172.21.15.62:0/2782025461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc36880 0x55edbcc30260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47c3fff700 1 -- 172.21.15.62:0/2782025461 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edbcad08d0 con 0x55edbcc35f60 2024-06-10T00:35:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47c37fe700 1 --2- 172.21.15.62:0/2782025461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc36880 0x55edbcc30260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:35:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.849+0000 7f47c3fff700 1 --2- 172.21.15.62:0/2782025461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc35f60 0x55edbcc36340 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f47c4077c30 tx=0x7f47b4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ded773f0acf75e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.850+0000 7f47aa7fc700 1 -- 172.21.15.62:0/2782025461 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b400f620 con 0x55edbcc35f60 2024-06-10T00:35:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.850+0000 7f47aa7fc700 1 -- 172.21.15.62:0/2782025461 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b400fca0 con 0x55edbcc35f60 2024-06-10T00:35:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.850+0000 7f47aa7fc700 1 -- 172.21.15.62:0/2782025461 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b402f400 con 0x55edbcc35f60 2024-06-10T00:35:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.850+0000 7f47d502cb80 1 -- 172.21.15.62:0/2782025461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc35f60 msgr2=0x55edbcc36340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.850+0000 7f47d502cb80 1 --2- 172.21.15.62:0/2782025461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc35f60 0x55edbcc36340 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f47c4077c30 tx=0x7f47b4020000 comp rx=0 tx=0).stop 2024-06-10T00:35:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.850+0000 7f47d502cb80 1 -- 172.21.15.62:0/2782025461 shutdown_connections 2024-06-10T00:35:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.850+0000 7f47d502cb80 1 --2- 172.21.15.62:0/2782025461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc35f60 0x55edbcc36340 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.850+0000 7f47d502cb80 1 --2- 172.21.15.62:0/2782025461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc307a0 0x55edbcc30bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.850+0000 7f47d502cb80 1 --2- 172.21.15.62:0/2782025461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc36880 0x55edbcc30260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.850+0000 7f47d502cb80 1 -- 172.21.15.62:0/2782025461 >> 172.21.15.62:0/2782025461 conn(0x55edbcb5d080 msgr2=0x55edbcc2b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.850+0000 7f47d502cb80 1 -- 172.21.15.62:0/2782025461 shutdown_connections 2024-06-10T00:35:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.850+0000 7f47d502cb80 1 -- 172.21.15.62:0/2782025461 wait complete. 2024-06-10T00:35:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.851+0000 7f47d502cb80 1 -- start start 2024-06-10T00:35:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.851+0000 7f47d502cb80 1 Processor -- start 2024-06-10T00:35:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.851+0000 7f47d502cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc307a0 0x55edbcc3c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.851+0000 7f47d502cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc35f60 0x55edbcc39880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.851+0000 7f47d502cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc36880 0x55edbcc39dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.851+0000 7f47d502cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55edbcb21fc0 con 0x55edbcc307a0 2024-06-10T00:35:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.851+0000 7f47d502cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55edbcb7d810 con 0x55edbcc36880 2024-06-10T00:35:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.851+0000 7f47d502cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55edbcc3d1a0 con 0x55edbcc35f60 2024-06-10T00:35:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.851+0000 7f47d502cb80 10 client.0 _ll_register_callbacks cb 0x55edbcb175b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.851+0000 7f47c3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc307a0 0x55edbcc3c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47c3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc307a0 0x55edbcc3c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44262/0 (socket says 172.21.15.62:44262) 2024-06-10T00:35:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47c3fff700 1 -- 172.21.15.62:0/151790807 learned_addr learned my addr 172.21.15.62:0/151790807 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47c37fe700 1 --2- 172.21.15.62:0/151790807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc35f60 0x55edbcc39880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47c8d46700 1 --2- 172.21.15.62:0/151790807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc36880 0x55edbcc39dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47c3fff700 1 -- 172.21.15.62:0/151790807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc35f60 msgr2=0x55edbcc39880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47c3fff700 1 --2- 172.21.15.62:0/151790807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc35f60 0x55edbcc39880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47c3fff700 1 -- 172.21.15.62:0/151790807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc36880 msgr2=0x55edbcc39dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47c3fff700 1 --2- 172.21.15.62:0/151790807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc36880 0x55edbcc39dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47c3fff700 1 -- 172.21.15.62:0/151790807 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edbcb47750 con 0x55edbcc307a0 2024-06-10T00:35:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47c8d46700 1 --2- 172.21.15.62:0/151790807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc36880 0x55edbcc39dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:35:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47c3fff700 1 --2- 172.21.15.62:0/151790807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc307a0 0x55edbcc3c580 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f47b4024980 tx=0x7f47b40038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c5f5e1cc0c23c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47a37fe700 1 -- 172.21.15.62:0/151790807 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b400f830 con 0x55edbcc307a0 2024-06-10T00:35:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47a37fe700 1 -- 172.21.15.62:0/151790807 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f47b400f9d0 con 0x55edbcc307a0 2024-06-10T00:35:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47d502cb80 1 -- 172.21.15.62:0/151790807 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47b400fca0 con 0x55edbcc307a0 2024-06-10T00:35:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.852+0000 7f47a37fe700 1 -- 172.21.15.62:0/151790807 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47b4037400 con 0x55edbcc307a0 2024-06-10T00:35:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.853+0000 7f47d502cb80 10 client.16095 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.853+0000 7f47d502cb80 1 -- 172.21.15.62:0/151790807 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55edbcc478b0 con 0x55edbcc307a0 2024-06-10T00:35:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.853+0000 7f47a37fe700 1 -- 172.21.15.62:0/151790807 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b400fca0 con 0x55edbcc307a0 2024-06-10T00:35:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.853+0000 7f47a37fe700 1 -- 172.21.15.62:0/151790807 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f47b402f660 con 0x55edbcc307a0 2024-06-10T00:35:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.853+0000 7f47d502cb80 10 client.16095 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.853+0000 7f47d502cb80 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:35:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.853+0000 7f47d502cb80 20 client.16095 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.853+0000 7f47d502cb80 1 --2- 172.21.15.62:0/151790807 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55edbcc49d80 0x55edbcc4c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.853+0000 7f47d502cb80 4 client.16095 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:35:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.853+0000 7f47d502cb80 1 -- 172.21.15.62:0/151790807 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edbcb8ed50 con 0x55edbcc49d80 2024-06-10T00:35:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.854+0000 7f47c37fe700 1 --2- 172.21.15.62:0/151790807 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55edbcc49d80 0x55edbcc4c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.854+0000 7f47c37fe700 1 --2- 172.21.15.62:0/151790807 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55edbcc49d80 0x55edbcc4c170 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ad03c7c2f53655 server_cookie=4d474c88d2c79078 in_seq=0 out_seq=0 2024-06-10T00:35:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.854+0000 7f47a37fe700 10 client.16095 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47a37fe700 1 -- 172.21.15.62:0/151790807 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55edbcb8ed50 con 0x55edbcc49d80 2024-06-10T00:35:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47a37fe700 10 client.16095 handle_command_reply: tid=0 2024-06-10T00:35:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47d502cb80 1 client.16095 shutdown 2024-06-10T00:35:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47d502cb80 1 -- 172.21.15.62:0/151790807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc307a0 msgr2=0x55edbcc3c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47d502cb80 1 --2- 172.21.15.62:0/151790807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc307a0 0x55edbcc3c580 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f47b4024980 tx=0x7f47b40038b0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47d502cb80 1 -- 172.21.15.62:0/151790807 shutdown_connections 2024-06-10T00:35:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47d502cb80 1 --2- 172.21.15.62:0/151790807 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55edbcc49d80 0x55edbcc4c170 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47d502cb80 1 --2- 172.21.15.62:0/151790807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc307a0 0x55edbcc3c580 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47d502cb80 1 --2- 172.21.15.62:0/151790807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc36880 0x55edbcc39dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47d502cb80 1 --2- 172.21.15.62:0/151790807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc35f60 0x55edbcc39880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47d502cb80 1 -- 172.21.15.62:0/151790807 >> 172.21.15.62:0/151790807 conn(0x55edbcb5d080 msgr2=0x55edbcc2a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47a37fe700 0 client.16095 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47d502cb80 1 -- 172.21.15.62:0/151790807 shutdown_connections 2024-06-10T00:35:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47d502cb80 1 -- 172.21.15.62:0/151790807 wait complete. 2024-06-10T00:35:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.855+0000 7f47d502cb80 20 client.16095 trim_cache size 0 max 16384 2024-06-10T00:35:09.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47d502cb80 1 Processor -- start 2024-06-10T00:35:09.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47d502cb80 1 -- start start 2024-06-10T00:35:09.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47d502cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc307a0 0x55edbcc374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47d502cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc35f60 0x55edbcc3b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47d502cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc36880 0x55edbcc2ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47d502cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f47c41ac3f0 con 0x55edbcc36880 2024-06-10T00:35:09.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47d502cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f47c41adf30 con 0x55edbcc307a0 2024-06-10T00:35:09.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47d502cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47c41acac0 con 0x55edbcc35f60 2024-06-10T00:35:09.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47c8d46700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc36880 0x55edbcc2ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47c8d46700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc36880 0x55edbcc2ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44268/0 (socket says 172.21.15.62:44268) 2024-06-10T00:35:09.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47c8d46700 1 -- 172.21.15.62:0/1780330962 learned_addr learned my addr 172.21.15.62:0/1780330962 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:09.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47c37fe700 1 --2- 172.21.15.62:0/1780330962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc35f60 0x55edbcc3b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47c3fff700 1 --2- 172.21.15.62:0/1780330962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc307a0 0x55edbcc374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47c8d46700 1 -- 172.21.15.62:0/1780330962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc35f60 msgr2=0x55edbcc3b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:09.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47c8d46700 1 --2- 172.21.15.62:0/1780330962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc35f60 0x55edbcc3b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47c8d46700 1 -- 172.21.15.62:0/1780330962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc307a0 msgr2=0x55edbcc374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:09.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47c8d46700 1 --2- 172.21.15.62:0/1780330962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc307a0 0x55edbcc374f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.871+0000 7f47c8d46700 1 -- 172.21.15.62:0/1780330962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edbcc3c8f0 con 0x55edbcc36880 2024-06-10T00:35:09.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.872+0000 7f47c8d46700 1 --2- 172.21.15.62:0/1780330962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc36880 0x55edbcc2ae80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f47bc09b120 tx=0x7f47bc09b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c824aaec9e13cba6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:09.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.872+0000 7f47a9ffb700 1 -- 172.21.15.62:0/1780330962 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0a3d70 con 0x55edbcc36880 2024-06-10T00:35:09.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.872+0000 7f47a9ffb700 1 -- 172.21.15.62:0/1780330962 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47bc099490 con 0x55edbcc36880 2024-06-10T00:35:09.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.872+0000 7f47a9ffb700 1 -- 172.21.15.62:0/1780330962 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc09f170 con 0x55edbcc36880 2024-06-10T00:35:09.878 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:09 smithi138 ceph-mon[36549]: pgmap v72: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 141 KiB/s wr, 51 op/s 2024-06-10T00:35:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.872+0000 7f47d502cb80 1 -- 172.21.15.62:0/1780330962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc36880 msgr2=0x55edbcc2ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.872+0000 7f47d502cb80 1 --2- 172.21.15.62:0/1780330962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc36880 0x55edbcc2ae80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f47bc09b120 tx=0x7f47bc09b910 comp rx=0 tx=0).stop 2024-06-10T00:35:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.872+0000 7f47d502cb80 1 -- 172.21.15.62:0/1780330962 shutdown_connections 2024-06-10T00:35:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.872+0000 7f47d502cb80 1 --2- 172.21.15.62:0/1780330962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc36880 0x55edbcc2ae80 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.872+0000 7f47d502cb80 1 --2- 172.21.15.62:0/1780330962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc307a0 0x55edbcc374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.872+0000 7f47d502cb80 1 --2- 172.21.15.62:0/1780330962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc35f60 0x55edbcc3b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.872+0000 7f47d502cb80 1 -- 172.21.15.62:0/1780330962 >> 172.21.15.62:0/1780330962 conn(0x55edbcb5d080 msgr2=0x55edbcc3b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.873+0000 7f47d502cb80 1 -- 172.21.15.62:0/1780330962 shutdown_connections 2024-06-10T00:35:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.873+0000 7f47d502cb80 1 -- 172.21.15.62:0/1780330962 wait complete. 2024-06-10T00:35:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.873+0000 7f47d502cb80 1 -- start start 2024-06-10T00:35:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.873+0000 7f47d502cb80 1 Processor -- start 2024-06-10T00:35:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.873+0000 7f47d502cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc307a0 0x55edbcc4bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.873+0000 7f47d502cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc35f60 0x55edbcc4c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.873+0000 7f47d502cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc36880 0x55edbccb26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.873+0000 7f47d502cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f47c41ad860 con 0x55edbcc307a0 2024-06-10T00:35:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.873+0000 7f47d502cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f47c41ad190 con 0x55edbcc35f60 2024-06-10T00:35:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.873+0000 7f47d502cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47c41aa390 con 0x55edbcc36880 2024-06-10T00:35:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.873+0000 7f47d502cb80 10 client.0 _ll_register_callbacks cb 0x55edbcb175b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47c3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc307a0 0x55edbcc4bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47c8d46700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc36880 0x55edbccb26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47c3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc307a0 0x55edbcc4bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44272/0 (socket says 172.21.15.62:44272) 2024-06-10T00:35:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47c3fff700 1 -- 172.21.15.62:0/3483586828 learned_addr learned my addr 172.21.15.62:0/3483586828 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47c37fe700 1 --2- 172.21.15.62:0/3483586828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc35f60 0x55edbcc4c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47c3fff700 1 -- 172.21.15.62:0/3483586828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc36880 msgr2=0x55edbccb26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47c3fff700 1 --2- 172.21.15.62:0/3483586828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc36880 0x55edbccb26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47c3fff700 1 -- 172.21.15.62:0/3483586828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc35f60 msgr2=0x55edbcc4c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47c3fff700 1 --2- 172.21.15.62:0/3483586828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc35f60 0x55edbcc4c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47c3fff700 1 -- 172.21.15.62:0/3483586828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edbccb2bf0 con 0x55edbcc307a0 2024-06-10T00:35:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47c37fe700 1 --2- 172.21.15.62:0/3483586828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc35f60 0x55edbcc4c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:35:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47c8d46700 1 --2- 172.21.15.62:0/3483586828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc36880 0x55edbccb26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:35:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47c3fff700 1 --2- 172.21.15.62:0/3483586828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc307a0 0x55edbcc4bfb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f47b4006010 tx=0x7f47b4005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380f68ff172f9383 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47a37fe700 1 -- 172.21.15.62:0/3483586828 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b402f610 con 0x55edbcc307a0 2024-06-10T00:35:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47a37fe700 1 -- 172.21.15.62:0/3483586828 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f47b402f7b0 con 0x55edbcc307a0 2024-06-10T00:35:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.874+0000 7f47d502cb80 1 -- 172.21.15.62:0/3483586828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47bc099490 con 0x55edbcc307a0 2024-06-10T00:35:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.875+0000 7f47a37fe700 1 -- 172.21.15.62:0/3483586828 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47b400f830 con 0x55edbcc307a0 2024-06-10T00:35:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.875+0000 7f47d502cb80 10 client.16101 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:09.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.875+0000 7f47d502cb80 1 -- 172.21.15.62:0/3483586828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55edbccb3e00 con 0x55edbcc307a0 2024-06-10T00:35:09.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.875+0000 7f47a37fe700 1 -- 172.21.15.62:0/3483586828 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47bc099490 con 0x55edbcc307a0 2024-06-10T00:35:09.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.875+0000 7f47a37fe700 1 -- 172.21.15.62:0/3483586828 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f47b4048420 con 0x55edbcc307a0 2024-06-10T00:35:09.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.875+0000 7f47d502cb80 10 client.16101 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:09.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.875+0000 7f47d502cb80 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:35:09.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.875+0000 7f47d502cb80 20 client.16101 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:09.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.875+0000 7f47d502cb80 1 --2- 172.21.15.62:0/3483586828 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55edbccb43f0 0x55edbccb6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:09.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.875+0000 7f47d502cb80 4 client.16101 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:35:09.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.875+0000 7f47d502cb80 1 -- 172.21.15.62:0/3483586828 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55edbcb892d0 con 0x55edbccb43f0 2024-06-10T00:35:09.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.875+0000 7f47c37fe700 1 --2- 172.21.15.62:0/3483586828 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55edbccb43f0 0x55edbccb6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:09.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.876+0000 7f47c37fe700 1 --2- 172.21.15.62:0/3483586828 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55edbccb43f0 0x55edbccb6810 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40ef7a9750f5fdda server_cookie=ba9ea3801f88d143 in_seq=0 out_seq=0 2024-06-10T00:35:09.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.876+0000 7f47a37fe700 10 client.16101 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:09.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.876+0000 7f47a37fe700 1 -- 172.21.15.62:0/3483586828 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55edbcb892d0 con 0x55edbccb43f0 2024-06-10T00:35:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.876+0000 7f47a37fe700 10 client.16101 handle_command_reply: tid=0 2024-06-10T00:35:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.876+0000 7f47d502cb80 1 client.16101 shutdown 2024-06-10T00:35:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.876+0000 7f47d502cb80 1 -- 172.21.15.62:0/3483586828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc307a0 msgr2=0x55edbcc4bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.876+0000 7f47d502cb80 1 --2- 172.21.15.62:0/3483586828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc307a0 0x55edbcc4bfb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f47b4006010 tx=0x7f47b4005ce0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.876+0000 7f47d502cb80 1 -- 172.21.15.62:0/3483586828 shutdown_connections 2024-06-10T00:35:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.876+0000 7f47d502cb80 1 --2- 172.21.15.62:0/3483586828 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55edbccb43f0 0x55edbccb6810 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.876+0000 7f47d502cb80 1 --2- 172.21.15.62:0/3483586828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55edbcc307a0 0x55edbcc4bfb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.876+0000 7f47a37fe700 0 client.16101 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.876+0000 7f47d502cb80 1 --2- 172.21.15.62:0/3483586828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edbcc36880 0x55edbccb26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.876+0000 7f47d502cb80 1 --2- 172.21.15.62:0/3483586828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edbcc35f60 0x55edbcc4c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.876+0000 7f47d502cb80 1 -- 172.21.15.62:0/3483586828 >> 172.21.15.62:0/3483586828 conn(0x55edbcb5d080 msgr2=0x55edbcc399c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.877+0000 7f47d502cb80 1 -- 172.21.15.62:0/3483586828 shutdown_connections 2024-06-10T00:35:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.877+0000 7f47d502cb80 1 -- 172.21.15.62:0/3483586828 wait complete. 2024-06-10T00:35:09.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.877+0000 7f47d502cb80 20 client.16101 trim_cache size 0 max 16384 2024-06-10T00:35:09.888 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:35:09.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.877+0000 7f47ca7a9700 1 -- 172.21.15.62:0/3739537075 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f47ac0a2d90 msgr2=0x7f47ac0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:09.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.877+0000 7f47ca7a9700 1 --2- 172.21.15.62:0/3739537075 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f47ac0a2d90 0x7f47ac0a5250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47b400b5c0 tx=0x7f47b4005960 comp rx=0 tx=0).stop 2024-06-10T00:35:09.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.877+0000 7f47ca7a9700 1 -- 172.21.15.62:0/3739537075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c4107550 msgr2=0x7f47c419d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:09.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.877+0000 7f47ca7a9700 1 --2- 172.21.15.62:0/3739537075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c4107550 0x7f47c419d610 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00ada0 tx=0x7f47bc00c330 comp rx=0 tx=0).stop 2024-06-10T00:35:09.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.877+0000 7f47ca7a9700 1 -- 172.21.15.62:0/3739537075 shutdown_connections 2024-06-10T00:35:09.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.877+0000 7f47ca7a9700 1 --2- 172.21.15.62:0/3739537075 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f47ac0a2d90 0x7f47ac0a5250 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.877+0000 7f47ca7a9700 1 --2- 172.21.15.62:0/3739537075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c4107550 0x7f47c419d610 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.878+0000 7f47ca7a9700 1 --2- 172.21.15.62:0/3739537075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c4100e30 0x7f47c4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.878+0000 7f47ca7a9700 1 --2- 172.21.15.62:0/3739537075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47c41017f0 0x7f47c4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:09.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.878+0000 7f47ca7a9700 1 -- 172.21.15.62:0/3739537075 >> 172.21.15.62:0/3739537075 conn(0x7f47c40766b0 msgr2=0x7f47c40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:09.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.878+0000 7f47ca7a9700 1 -- 172.21.15.62:0/3739537075 shutdown_connections 2024-06-10T00:35:09.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:09.878+0000 7f47ca7a9700 1 -- 172.21.15.62:0/3739537075 wait complete. 2024-06-10T00:35:10.514 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:35:10.767 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:35:11.591 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:11 smithi062 ceph-mon[32390]: pgmap v73: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 141 KiB/s wr, 51 op/s 2024-06-10T00:35:11.645 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T00:35:11.645 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T00:35:11.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:11 smithi105 ceph-mon[36321]: pgmap v73: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 141 KiB/s wr, 51 op/s 2024-06-10T00:35:11.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:11 smithi138 ceph-mon[36549]: pgmap v73: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 141 KiB/s wr, 51 op/s 2024-06-10T00:35:12.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.594+0000 7f419932f700 1 -- 172.21.15.62:0/3102940522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41941066b0 msgr2=0x7f419410e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:12.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.594+0000 7f419932f700 1 --2- 172.21.15.62:0/3102940522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41941066b0 0x7f419410e280 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f418c00b0d0 tx=0x7f418c00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:35:12.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.594+0000 7f419932f700 1 -- 172.21.15.62:0/3102940522 shutdown_connections 2024-06-10T00:35:12.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.594+0000 7f419932f700 1 --2- 172.21.15.62:0/3102940522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41941066b0 0x7f419410e280 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:12.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.594+0000 7f419932f700 1 --2- 172.21.15.62:0/3102940522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41941015d0 0x7f41941019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:12.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.594+0000 7f419932f700 1 --2- 172.21.15.62:0/3102940522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4194101f80 0x7f4194105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.594+0000 7f419932f700 1 -- 172.21.15.62:0/3102940522 >> 172.21.15.62:0/3102940522 conn(0x7f4194078400 msgr2=0x7f4194078810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.595+0000 7f419932f700 1 -- 172.21.15.62:0/3102940522 shutdown_connections 2024-06-10T00:35:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.595+0000 7f419932f700 1 -- 172.21.15.62:0/3102940522 wait complete. 2024-06-10T00:35:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.595+0000 7f419932f700 1 Processor -- start 2024-06-10T00:35:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.596+0000 7f419932f700 1 -- start start 2024-06-10T00:35:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.596+0000 7f419932f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41941015d0 0x7f419419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.596+0000 7f419932f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4194101f80 0x7f419419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.596+0000 7f419932f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41941066b0 0x7f4194198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.596+0000 7f419932f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4194111710 con 0x7f41941066b0 2024-06-10T00:35:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.596+0000 7f419932f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4194111590 con 0x7f4194101f80 2024-06-10T00:35:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.596+0000 7f419932f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4194111890 con 0x7f41941015d0 2024-06-10T00:35:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.596+0000 7f41937fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41941066b0 0x7f4194198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.596+0000 7f41937fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41941066b0 0x7f4194198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44286/0 (socket says 172.21.15.62:44286) 2024-06-10T00:35:12.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.596+0000 7f41937fe700 1 -- 172.21.15.62:0/3411486796 learned_addr learned my addr 172.21.15.62:0/3411486796 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:12.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.596+0000 7f4192ffd700 1 --2- 172.21.15.62:0/3411486796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41941015d0 0x7f419419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:12.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.596+0000 7f418a7fc700 1 --2- 172.21.15.62:0/3411486796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4194101f80 0x7f419419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:12.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.596+0000 7f41937fe700 1 -- 172.21.15.62:0/3411486796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41941015d0 msgr2=0x7f419419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:12.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.597+0000 7f41937fe700 1 --2- 172.21.15.62:0/3411486796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41941015d0 0x7f419419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:12.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.597+0000 7f41937fe700 1 -- 172.21.15.62:0/3411486796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4194101f80 msgr2=0x7f419419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:12.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.597+0000 7f41937fe700 1 --2- 172.21.15.62:0/3411486796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4194101f80 0x7f419419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:12.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.597+0000 7f41937fe700 1 -- 172.21.15.62:0/3411486796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41941991c0 con 0x7f41941066b0 2024-06-10T00:35:12.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.597+0000 7f41937fe700 1 --2- 172.21.15.62:0/3411486796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41941066b0 0x7f4194198a60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f418c006070 tx=0x7f418c004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:12.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.597+0000 7f4190ff9700 1 -- 172.21.15.62:0/3411486796 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c021070 con 0x7f41941066b0 2024-06-10T00:35:12.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.597+0000 7f419932f700 1 -- 172.21.15.62:0/3411486796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4194199450 con 0x7f41941066b0 2024-06-10T00:35:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.597+0000 7f419932f700 1 -- 172.21.15.62:0/3411486796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41941a5770 con 0x7f41941066b0 2024-06-10T00:35:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.597+0000 7f4190ff9700 1 -- 172.21.15.62:0/3411486796 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f418c007c00 con 0x7f41941066b0 2024-06-10T00:35:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.597+0000 7f4190ff9700 1 -- 172.21.15.62:0/3411486796 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c0052a0 con 0x7f41941066b0 2024-06-10T00:35:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.598+0000 7f419932f700 1 -- 172.21.15.62:0/3411486796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4194069180 con 0x7f41941066b0 2024-06-10T00:35:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.599+0000 7f4190ff9700 1 -- 172.21.15.62:0/3411486796 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f418c024070 con 0x7f41941066b0 2024-06-10T00:35:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.600+0000 7f4190ff9700 1 --2- 172.21.15.62:0/3411486796 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f41800a2e10 0x7f41800a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:12.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.600+0000 7f4190ff9700 1 -- 172.21.15.62:0/3411486796 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f418c0d0900 con 0x7f41941066b0 2024-06-10T00:35:12.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.600+0000 7f4192ffd700 1 --2- 172.21.15.62:0/3411486796 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f41800a2e10 0x7f41800a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:12.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.601+0000 7f4192ffd700 1 --2- 172.21.15.62:0/3411486796 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f41800a2e10 0x7f41800a52d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4184005950 tx=0x7f418400a300 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:12.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.605+0000 7f4190ff9700 1 -- 172.21.15.62:0/3411486796 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f418c007db0 con 0x7f41941066b0 2024-06-10T00:35:12.610 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:35:12.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.864+0000 7f419932f700 1 -- 172.21.15.62:0/3411486796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4194199f90 con 0x7f41941066b0 2024-06-10T00:35:12.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.865+0000 7f4190ff9700 1 -- 172.21.15.62:0/3411486796 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f418c09d200 con 0x7f41941066b0 2024-06-10T00:35:12.866 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:35:12.866 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:32:47.783154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:35:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.867+0000 7f419932f700 1 -- 172.21.15.62:0/3411486796 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f41800a2e10 msgr2=0x7f41800a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.867+0000 7f419932f700 1 --2- 172.21.15.62:0/3411486796 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f41800a2e10 0x7f41800a52d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4184005950 tx=0x7f418400a300 comp rx=0 tx=0).stop 2024-06-10T00:35:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.867+0000 7f419932f700 1 -- 172.21.15.62:0/3411486796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41941066b0 msgr2=0x7f4194198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.867+0000 7f419932f700 1 --2- 172.21.15.62:0/3411486796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41941066b0 0x7f4194198a60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f418c006070 tx=0x7f418c004e40 comp rx=0 tx=0).stop 2024-06-10T00:35:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.867+0000 7f419932f700 1 -- 172.21.15.62:0/3411486796 shutdown_connections 2024-06-10T00:35:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.867+0000 7f419932f700 1 --2- 172.21.15.62:0/3411486796 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f41800a2e10 0x7f41800a52d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.867+0000 7f419932f700 1 --2- 172.21.15.62:0/3411486796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41941066b0 0x7f4194198a60 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.867+0000 7f419932f700 1 --2- 172.21.15.62:0/3411486796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41941015d0 0x7f419419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.867+0000 7f419932f700 1 --2- 172.21.15.62:0/3411486796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4194101f80 0x7f419419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.867+0000 7f419932f700 1 -- 172.21.15.62:0/3411486796 >> 172.21.15.62:0/3411486796 conn(0x7f4194078400 msgr2=0x7f419406ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.867+0000 7f419932f700 1 -- 172.21.15.62:0/3411486796 shutdown_connections 2024-06-10T00:35:12.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:12.867+0000 7f419932f700 1 -- 172.21.15.62:0/3411486796 wait complete. 2024-06-10T00:35:12.869 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-06-10T00:35:13.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:13 smithi062 ceph-mon[32390]: pgmap v74: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 144 KiB/s wr, 56 op/s 2024-06-10T00:35:13.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:13 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3411486796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:35:13.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:13 smithi105 ceph-mon[36321]: pgmap v74: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 144 KiB/s wr, 56 op/s 2024-06-10T00:35:13.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:13 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3411486796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:35:13.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:13 smithi138 ceph-mon[36549]: pgmap v74: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 144 KiB/s wr, 56 op/s 2024-06-10T00:35:13.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:13 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3411486796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:35:14.093 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db --set-id 547e97b4 --release --await' 2024-06-10T00:35:14.093 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24464 quiesce db --set-id 547e97b4 --release --await 2024-06-10T00:35:15.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.212+0000 7f537d751700 1 -- 172.21.15.62:0/1239236613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5378101390 msgr2=0x7f5378101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:15.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.212+0000 7f537d751700 1 --2- 172.21.15.62:0/1239236613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5378101390 0x7f5378101810 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f536c009990 tx=0x7f536c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.212+0000 7f537d751700 1 -- 172.21.15.62:0/1239236613 shutdown_connections 2024-06-10T00:35:15.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.212+0000 7f537d751700 1 --2- 172.21.15.62:0/1239236613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5378101d50 0x7f537810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.212+0000 7f537d751700 1 --2- 172.21.15.62:0/1239236613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5378107400 0x7f53781077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.212+0000 7f537d751700 1 --2- 172.21.15.62:0/1239236613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5378101390 0x7f5378101810 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.212+0000 7f537d751700 1 -- 172.21.15.62:0/1239236613 >> 172.21.15.62:0/1239236613 conn(0x7f5378078340 msgr2=0x7f5378078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:15.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.212+0000 7f537d751700 1 -- 172.21.15.62:0/1239236613 shutdown_connections 2024-06-10T00:35:15.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.212+0000 7f537d751700 1 -- 172.21.15.62:0/1239236613 wait complete. 2024-06-10T00:35:15.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.213+0000 7f537d751700 1 Processor -- start 2024-06-10T00:35:15.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.213+0000 7f537d751700 1 -- start start 2024-06-10T00:35:15.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.213+0000 7f537d751700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5378101390 0x7f537819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.213+0000 7f537d751700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5378101d50 0x7f537819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f537d751700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5378107400 0x7f5378198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f537d751700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5378111690 con 0x7f5378101390 2024-06-10T00:35:15.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f537d751700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5378111510 con 0x7f5378107400 2024-06-10T00:35:15.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f537d751700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5378111810 con 0x7f5378101d50 2024-06-10T00:35:15.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5378101390 0x7f537819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5378101390 0x7f537819e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44308/0 (socket says 172.21.15.62:44308) 2024-06-10T00:35:15.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f5376ffd700 1 -- 172.21.15.62:0/4144832771 learned_addr learned my addr 172.21.15.62:0/4144832771 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:15.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f53777fe700 1 --2- 172.21.15.62:0/4144832771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5378107400 0x7f5378198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f53767fc700 1 --2- 172.21.15.62:0/4144832771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5378101d50 0x7f537819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f5376ffd700 1 -- 172.21.15.62:0/4144832771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5378101d50 msgr2=0x7f537819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:15.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f5376ffd700 1 --2- 172.21.15.62:0/4144832771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5378101d50 0x7f537819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f5376ffd700 1 -- 172.21.15.62:0/4144832771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5378107400 msgr2=0x7f5378198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:15.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f5376ffd700 1 --2- 172.21.15.62:0/4144832771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5378107400 0x7f5378198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f5376ffd700 1 -- 172.21.15.62:0/4144832771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53781991f0 con 0x7f5378101390 2024-06-10T00:35:15.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f5376ffd700 1 --2- 172.21.15.62:0/4144832771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5378101390 0x7f537819e880 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f536800d980 tx=0x7f536800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:15.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f535ffff700 1 -- 172.21.15.62:0/4144832771 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368017070 con 0x7f5378101390 2024-06-10T00:35:15.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f535ffff700 1 -- 172.21.15.62:0/4144832771 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5368005020 con 0x7f5378101390 2024-06-10T00:35:15.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.214+0000 7f537d751700 1 -- 172.21.15.62:0/4144832771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53781994e0 con 0x7f5378101390 2024-06-10T00:35:15.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.215+0000 7f535ffff700 1 -- 172.21.15.62:0/4144832771 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536800fbf0 con 0x7f5378101390 2024-06-10T00:35:15.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.215+0000 7f537d751700 1 -- 172.21.15.62:0/4144832771 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53781a5750 con 0x7f5378101390 2024-06-10T00:35:15.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.217+0000 7f535ffff700 1 -- 172.21.15.62:0/4144832771 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f53680051d0 con 0x7f5378101390 2024-06-10T00:35:15.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.218+0000 7f535ffff700 1 --2- 172.21.15.62:0/4144832771 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f53600a2e10 0x7f53600a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.218+0000 7f53767fc700 1 --2- 172.21.15.62:0/4144832771 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f53600a2e10 0x7f53600a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.218+0000 7f535ffff700 1 -- 172.21.15.62:0/4144832771 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f53680cf4f0 con 0x7f5378101390 2024-06-10T00:35:15.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f5387fd4b80 1 Processor -- start 2024-06-10T00:35:15.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f53767fc700 1 --2- 172.21.15.62:0/4144832771 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f53600a2e10 0x7f53600a52d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f536c00b5c0 tx=0x7f536c005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:15.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f5387fd4b80 1 -- start start 2024-06-10T00:35:15.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f5387fd4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0e8d0 0x55c05ba0ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f5387fd4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0f1f0 0x55c05ba13090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f5387fd4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 0x55c05ba13a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f5387fd4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c05b8fcfe0 con 0x55c05ba0f1f0 2024-06-10T00:35:15.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f5387fd4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c05b95b8d0 con 0x55c05ba0e8d0 2024-06-10T00:35:15.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f5387fd4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c05b8c34e0 con 0x55c05ba135d0 2024-06-10T00:35:15.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f53767fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0f1f0 0x55c05ba13090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f53777fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 0x55c05ba13a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f53767fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0f1f0 0x55c05ba13090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44316/0 (socket says 172.21.15.62:44316) 2024-06-10T00:35:15.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f53767fc700 1 -- 172.21.15.62:0/3291642950 learned_addr learned my addr 172.21.15.62:0/3291642950 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:15.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f5376ffd700 1 --2- 172.21.15.62:0/3291642950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0e8d0 0x55c05ba0ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f53767fc700 1 -- 172.21.15.62:0/3291642950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 msgr2=0x55c05ba13a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:15.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f53767fc700 1 --2- 172.21.15.62:0/3291642950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 0x55c05ba13a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.219+0000 7f53767fc700 1 -- 172.21.15.62:0/3291642950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0e8d0 msgr2=0x55c05ba0ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:15.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.220+0000 7f53767fc700 1 --2- 172.21.15.62:0/3291642950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0e8d0 0x55c05ba0ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.220+0000 7f53767fc700 1 -- 172.21.15.62:0/3291642950 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c05b8aa950 con 0x55c05ba0f1f0 2024-06-10T00:35:15.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.220+0000 7f53777fe700 1 --2- 172.21.15.62:0/3291642950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 0x55c05ba13a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:35:15.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.220+0000 7f53767fc700 1 --2- 172.21.15.62:0/3291642950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0f1f0 0x55c05ba13090 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f536c009990 tx=0x7f536c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9f2738e31754b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:15.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.220+0000 7f535d7fa700 1 -- 172.21.15.62:0/3291642950 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c00f520 con 0x55c05ba0f1f0 2024-06-10T00:35:15.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.220+0000 7f535d7fa700 1 -- 172.21.15.62:0/3291642950 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f536c00fba0 con 0x55c05ba0f1f0 2024-06-10T00:35:15.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.220+0000 7f535d7fa700 1 -- 172.21.15.62:0/3291642950 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c024d50 con 0x55c05ba0f1f0 2024-06-10T00:35:15.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.221+0000 7f5387fd4b80 1 -- 172.21.15.62:0/3291642950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0f1f0 msgr2=0x55c05ba13090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:15.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.221+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/3291642950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0f1f0 0x55c05ba13090 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f536c009990 tx=0x7f536c020000 comp rx=0 tx=0).stop 2024-06-10T00:35:15.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.221+0000 7f5387fd4b80 1 -- 172.21.15.62:0/3291642950 shutdown_connections 2024-06-10T00:35:15.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.221+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/3291642950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 0x55c05ba13a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.221+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/3291642950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0e8d0 0x55c05ba0ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.221+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/3291642950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0f1f0 0x55c05ba13090 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.221+0000 7f5387fd4b80 1 -- 172.21.15.62:0/3291642950 >> 172.21.15.62:0/3291642950 conn(0x55c05b937100 msgr2=0x55c05ba052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:15.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.221+0000 7f5387fd4b80 1 -- 172.21.15.62:0/3291642950 shutdown_connections 2024-06-10T00:35:15.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.221+0000 7f5387fd4b80 1 -- 172.21.15.62:0/3291642950 wait complete. 2024-06-10T00:35:15.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.221+0000 7f5387fd4b80 1 -- start start 2024-06-10T00:35:15.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.221+0000 7f5387fd4b80 1 Processor -- start 2024-06-10T00:35:15.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5387fd4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba1aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5387fd4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0f1f0 0x55c05ba17e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5387fd4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 0x55c05ba18360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5387fd4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c05b8fc040 con 0x55c05ba0e8d0 2024-06-10T00:35:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5387fd4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c05b957890 con 0x55c05ba0f1f0 2024-06-10T00:35:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5387fd4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c05ba1b640 con 0x55c05ba135d0 2024-06-10T00:35:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5387fd4b80 10 client.0 _ll_register_callbacks cb 0x55c05b8f1630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba1aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba1aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44322/0 (socket says 172.21.15.62:44322) 2024-06-10T00:35:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5376ffd700 1 -- 172.21.15.62:0/1202594583 learned_addr learned my addr 172.21.15.62:0/1202594583 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f53767fc700 1 --2- 172.21.15.62:0/1202594583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0f1f0 0x55c05ba17e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f53777fe700 1 --2- 172.21.15.62:0/1202594583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 0x55c05ba18360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5376ffd700 1 -- 172.21.15.62:0/1202594583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 msgr2=0x55c05ba18360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:15.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5376ffd700 1 --2- 172.21.15.62:0/1202594583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 0x55c05ba18360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5376ffd700 1 -- 172.21.15.62:0/1202594583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0f1f0 msgr2=0x55c05ba17e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5376ffd700 1 --2- 172.21.15.62:0/1202594583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0f1f0 0x55c05ba17e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.222+0000 7f5376ffd700 1 -- 172.21.15.62:0/1202594583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c05b9217d0 con 0x55c05ba0e8d0 2024-06-10T00:35:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.223+0000 7f5376ffd700 1 --2- 172.21.15.62:0/1202594583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba1aab0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f536809b380 tx=0x7f53680861a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d39332eb7bf9ae server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.223+0000 7f535aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.223+0000 7f535aff5700 1 -- 172.21.15.62:0/1202594583 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536809fa30 con 0x55c05ba0e8d0 2024-06-10T00:35:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.223+0000 7f535aff5700 1 -- 172.21.15.62:0/1202594583 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f53680cf0c0 con 0x55c05ba0e8d0 2024-06-10T00:35:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.223+0000 7f5387fd4b80 1 -- 172.21.15.62:0/1202594583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f536c00fba0 con 0x55c05ba0e8d0 2024-06-10T00:35:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.223+0000 7f535aff5700 1 -- 172.21.15.62:0/1202594583 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f53680a42f0 con 0x55c05ba0e8d0 2024-06-10T00:35:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.223+0000 7f5387fd4b80 10 client.16119 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.223+0000 7f5387fd4b80 1 -- 172.21.15.62:0/1202594583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c05ba25e00 con 0x55c05ba0e8d0 2024-06-10T00:35:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.223+0000 7f535aff5700 1 -- 172.21.15.62:0/1202594583 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53680cf2c0 con 0x55c05ba0e8d0 2024-06-10T00:35:15.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.224+0000 7f535aff5700 1 -- 172.21.15.62:0/1202594583 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f536809b970 con 0x55c05ba0e8d0 2024-06-10T00:35:15.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.224+0000 7f5387fd4b80 10 client.16119 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:15.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.224+0000 7f5387fd4b80 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:35:15.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.224+0000 7f5387fd4b80 20 client.16119 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:15.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.224+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/1202594583 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c05ba28440 0x55c05ba2a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.224+0000 7f5387fd4b80 4 client.16119 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:35:15.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.224+0000 7f5387fd4b80 1 -- 172.21.15.62:0/1202594583 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f536c00fba0 con 0x55c05ba28440 2024-06-10T00:35:15.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.224+0000 7f53767fc700 1 --2- 172.21.15.62:0/1202594583 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c05ba28440 0x55c05ba2a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.225+0000 7f535aff5700 10 client.16119 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.225+0000 7f53767fc700 1 --2- 172.21.15.62:0/1202594583 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c05ba28440 0x55c05ba2a830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc191e37d9bcc608 server_cookie=e70fe6bcedaf28c8 in_seq=0 out_seq=0 2024-06-10T00:35:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.225+0000 7f535aff5700 1 -- 172.21.15.62:0/1202594583 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f536c00fba0 con 0x55c05ba28440 2024-06-10T00:35:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.225+0000 7f535aff5700 10 client.16119 handle_command_reply: tid=0 2024-06-10T00:35:15.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.225+0000 7f5387fd4b80 1 client.16119 shutdown 2024-06-10T00:35:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.226+0000 7f5387fd4b80 1 -- 172.21.15.62:0/1202594583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 msgr2=0x55c05ba1aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.226+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/1202594583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba1aab0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f536809b380 tx=0x7f53680861a0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.226+0000 7f5387fd4b80 1 -- 172.21.15.62:0/1202594583 shutdown_connections 2024-06-10T00:35:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.226+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/1202594583 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c05ba28440 0x55c05ba2a830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.226+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/1202594583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba1aab0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.226+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/1202594583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 0x55c05ba18360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.226+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/1202594583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0f1f0 0x55c05ba17e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.226+0000 7f535aff5700 0 client.16119 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.226+0000 7f5387fd4b80 1 -- 172.21.15.62:0/1202594583 >> 172.21.15.62:0/1202594583 conn(0x55c05b937100 msgr2=0x55c05ba14aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.226+0000 7f5387fd4b80 1 -- 172.21.15.62:0/1202594583 shutdown_connections 2024-06-10T00:35:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.226+0000 7f5387fd4b80 1 -- 172.21.15.62:0/1202594583 wait complete. 2024-06-10T00:35:15.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.226+0000 7f5387fd4b80 20 client.16119 trim_cache size 0 max 16384 2024-06-10T00:35:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.262+0000 7f5387fd4b80 1 Processor -- start 2024-06-10T00:35:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.262+0000 7f5387fd4b80 1 -- start start 2024-06-10T00:35:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.262+0000 7f5387fd4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba17720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.262+0000 7f5387fd4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba0f1f0 0x55c05ba149c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.262+0000 7f5387fd4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba135d0 0x55c05ba19d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.262+0000 7f5387fd4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f53781ac3b0 con 0x55c05ba0e8d0 2024-06-10T00:35:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.262+0000 7f5387fd4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f53781adef0 con 0x55c05ba135d0 2024-06-10T00:35:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.262+0000 7f5387fd4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53781aca80 con 0x55c05ba0f1f0 2024-06-10T00:35:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba17720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba17720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44332/0 (socket says 172.21.15.62:44332) 2024-06-10T00:35:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f5376ffd700 1 -- 172.21.15.62:0/1673409716 learned_addr learned my addr 172.21.15.62:0/1673409716 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f53777fe700 1 --2- 172.21.15.62:0/1673409716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba135d0 0x55c05ba19d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f53767fc700 1 --2- 172.21.15.62:0/1673409716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba0f1f0 0x55c05ba149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f5376ffd700 1 -- 172.21.15.62:0/1673409716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba0f1f0 msgr2=0x55c05ba149c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f5376ffd700 1 --2- 172.21.15.62:0/1673409716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba0f1f0 0x55c05ba149c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f5376ffd700 1 -- 172.21.15.62:0/1673409716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba135d0 msgr2=0x55c05ba19d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f5376ffd700 1 --2- 172.21.15.62:0/1673409716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba135d0 0x55c05ba19d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f5376ffd700 1 -- 172.21.15.62:0/1673409716 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c05ba1ae20 con 0x55c05ba0e8d0 2024-06-10T00:35:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f53767fc700 1 --2- 172.21.15.62:0/1673409716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba0f1f0 0x55c05ba149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:35:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f53777fe700 1 --2- 172.21.15.62:0/1673409716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba135d0 0x55c05ba19d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:35:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f5376ffd700 1 --2- 172.21.15.62:0/1673409716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba17720 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5368084be0 tx=0x7f536809b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79179f021733c108 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:15.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f535cff9700 1 -- 172.21.15.62:0/1673409716 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368097cf0 con 0x55c05ba0e8d0 2024-06-10T00:35:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f535cff9700 1 -- 172.21.15.62:0/1673409716 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53680cf130 con 0x55c05ba0e8d0 2024-06-10T00:35:15.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.263+0000 7f535cff9700 1 -- 172.21.15.62:0/1673409716 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536809b8e0 con 0x55c05ba0e8d0 2024-06-10T00:35:15.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.264+0000 7f5387fd4b80 1 -- 172.21.15.62:0/1673409716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 msgr2=0x55c05ba17720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:15.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.264+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/1673409716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba17720 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5368084be0 tx=0x7f536809b800 comp rx=0 tx=0).stop 2024-06-10T00:35:15.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.264+0000 7f5387fd4b80 1 -- 172.21.15.62:0/1673409716 shutdown_connections 2024-06-10T00:35:15.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.264+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/1673409716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba17720 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.264+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/1673409716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba135d0 0x55c05ba19d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.264+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/1673409716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba0f1f0 0x55c05ba149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.264+0000 7f5387fd4b80 1 -- 172.21.15.62:0/1673409716 >> 172.21.15.62:0/1673409716 conn(0x55c05b937100 msgr2=0x55c05ba19650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:15.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.264+0000 7f5387fd4b80 1 -- 172.21.15.62:0/1673409716 shutdown_connections 2024-06-10T00:35:15.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.264+0000 7f5387fd4b80 1 -- 172.21.15.62:0/1673409716 wait complete. 2024-06-10T00:35:15.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.265+0000 7f5387fd4b80 1 -- start start 2024-06-10T00:35:15.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.265+0000 7f5387fd4b80 1 Processor -- start 2024-06-10T00:35:15.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.265+0000 7f5387fd4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba1d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.265+0000 7f5387fd4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0f1f0 0x55c05ba1dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.265+0000 7f5387fd4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 0x55c05ba1e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.265+0000 7f5387fd4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f53781ad820 con 0x55c05ba0e8d0 2024-06-10T00:35:15.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.265+0000 7f5387fd4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f53781ad150 con 0x55c05ba0f1f0 2024-06-10T00:35:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.265+0000 7f5387fd4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53781aa370 con 0x55c05ba135d0 2024-06-10T00:35:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.265+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba1d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.265+0000 7f5387fd4b80 10 client.0 _ll_register_callbacks cb 0x55c05b8f1630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.265+0000 7f5376ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba1d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44338/0 (socket says 172.21.15.62:44338) 2024-06-10T00:35:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.265+0000 7f5376ffd700 1 -- 172.21.15.62:0/2814965607 learned_addr learned my addr 172.21.15.62:0/2814965607 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f53767fc700 1 --2- 172.21.15.62:0/2814965607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0f1f0 0x55c05ba1dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f53777fe700 1 --2- 172.21.15.62:0/2814965607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 0x55c05ba1e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f5376ffd700 1 -- 172.21.15.62:0/2814965607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 msgr2=0x55c05ba1e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f5376ffd700 1 --2- 172.21.15.62:0/2814965607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 0x55c05ba1e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f5376ffd700 1 -- 172.21.15.62:0/2814965607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0f1f0 msgr2=0x55c05ba1dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f5376ffd700 1 --2- 172.21.15.62:0/2814965607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0f1f0 0x55c05ba1dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f5376ffd700 1 -- 172.21.15.62:0/2814965607 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c05ba931f0 con 0x55c05ba0e8d0 2024-06-10T00:35:15.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f53767fc700 1 --2- 172.21.15.62:0/2814965607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0f1f0 0x55c05ba1dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:35:15.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f5376ffd700 1 --2- 172.21.15.62:0/2814965607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba1d8a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5368084c30 tx=0x7f53680ba1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7cb76159e43318 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:15.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f535aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:15.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f535aff5700 1 -- 172.21.15.62:0/2814965607 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53680cf2e0 con 0x55c05ba0e8d0 2024-06-10T00:35:15.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f535aff5700 1 -- 172.21.15.62:0/2814965607 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f536809baf0 con 0x55c05ba0e8d0 2024-06-10T00:35:15.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f5387fd4b80 1 -- 172.21.15.62:0/2814965607 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53680cf130 con 0x55c05ba0e8d0 2024-06-10T00:35:15.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f535aff5700 1 -- 172.21.15.62:0/2814965607 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f536809fa30 con 0x55c05ba0e8d0 2024-06-10T00:35:15.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f5387fd4b80 10 client.16125 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:15.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f5387fd4b80 1 -- 172.21.15.62:0/2814965607 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c05ba94480 con 0x55c05ba0e8d0 2024-06-10T00:35:15.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.266+0000 7f535aff5700 1 -- 172.21.15.62:0/2814965607 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53680cf130 con 0x55c05ba0e8d0 2024-06-10T00:35:15.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.267+0000 7f535aff5700 1 -- 172.21.15.62:0/2814965607 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5368004a30 con 0x55c05ba0e8d0 2024-06-10T00:35:15.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.267+0000 7f5387fd4b80 10 client.16125 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:15.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.267+0000 7f5387fd4b80 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:35:15.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.267+0000 7f5387fd4b80 20 client.16125 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:15.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.267+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/2814965607 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c05ba948e0 0x55c05ba96cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:15.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.267+0000 7f5387fd4b80 4 client.16125 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T00:35:15.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.267+0000 7f5387fd4b80 1 -- 172.21.15.62:0/2814965607 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c05b968dd0 con 0x55c05ba948e0 2024-06-10T00:35:15.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.267+0000 7f53767fc700 1 --2- 172.21.15.62:0/2814965607 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c05ba948e0 0x55c05ba96cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:15.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.267+0000 7f53767fc700 1 --2- 172.21.15.62:0/2814965607 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c05ba948e0 0x55c05ba96cd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=396e53e78ed475a0 server_cookie=5f2646af973de713 in_seq=0 out_seq=0 2024-06-10T00:35:15.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.267+0000 7f535aff5700 10 client.16125 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:15.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.268+0000 7f535aff5700 1 -- 172.21.15.62:0/2814965607 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c05b968dd0 con 0x55c05ba948e0 2024-06-10T00:35:15.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.268+0000 7f535aff5700 10 client.16125 handle_command_reply: tid=0 2024-06-10T00:35:15.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.269+0000 7f5387fd4b80 1 client.16125 shutdown 2024-06-10T00:35:15.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.269+0000 7f5387fd4b80 1 -- 172.21.15.62:0/2814965607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 msgr2=0x55c05ba1d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:15.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.269+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/2814965607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba1d8a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5368084c30 tx=0x7f53680ba1a0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.269+0000 7f5387fd4b80 1 -- 172.21.15.62:0/2814965607 shutdown_connections 2024-06-10T00:35:15.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.269+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/2814965607 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c05ba948e0 0x55c05ba96cd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.269+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/2814965607 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c05ba0e8d0 0x55c05ba1d8a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.269+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/2814965607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c05ba135d0 0x55c05ba1e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.269+0000 7f5387fd4b80 1 --2- 172.21.15.62:0/2814965607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c05ba0f1f0 0x55c05ba1dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.269+0000 7f5387fd4b80 1 -- 172.21.15.62:0/2814965607 >> 172.21.15.62:0/2814965607 conn(0x55c05b937100 msgr2=0x55c05ba19ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:15.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.269+0000 7f535aff5700 0 client.16125 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:15.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.269+0000 7f5387fd4b80 1 -- 172.21.15.62:0/2814965607 shutdown_connections 2024-06-10T00:35:15.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.269+0000 7f5387fd4b80 1 -- 172.21.15.62:0/2814965607 wait complete. 2024-06-10T00:35:15.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.269+0000 7f5387fd4b80 20 client.16125 trim_cache size 0 max 16384 2024-06-10T00:35:15.275 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:35:15.275 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-06-10T00:35:15.275 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "393b1fee-456e-45a1-bbf3-93ad586976f5", 2024-06-10T00:35:15.275 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-06-10T00:35:15.275 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:35:15.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.270+0000 7f537d751700 1 -- 172.21.15.62:0/4144832771 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f53600a2e10 msgr2=0x7f53600a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:15.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.270+0000 7f537d751700 1 --2- 172.21.15.62:0/4144832771 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f53600a2e10 0x7f53600a52d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f536c00b5c0 tx=0x7f536c005960 comp rx=0 tx=0).stop 2024-06-10T00:35:15.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.270+0000 7f537d751700 1 -- 172.21.15.62:0/4144832771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5378101390 msgr2=0x7f537819e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:15.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.270+0000 7f537d751700 1 --2- 172.21.15.62:0/4144832771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5378101390 0x7f537819e880 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f536800d980 tx=0x7f536800dc90 comp rx=0 tx=0).stop 2024-06-10T00:35:15.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.270+0000 7f537d751700 1 -- 172.21.15.62:0/4144832771 shutdown_connections 2024-06-10T00:35:15.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.270+0000 7f537d751700 1 --2- 172.21.15.62:0/4144832771 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f53600a2e10 0x7f53600a52d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.270+0000 7f537d751700 1 --2- 172.21.15.62:0/4144832771 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5378101390 0x7f537819e880 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.270+0000 7f537d751700 1 --2- 172.21.15.62:0/4144832771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5378107400 0x7f5378198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.270+0000 7f537d751700 1 --2- 172.21.15.62:0/4144832771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5378101d50 0x7f537819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:15.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.270+0000 7f537d751700 1 -- 172.21.15.62:0/4144832771 >> 172.21.15.62:0/4144832771 conn(0x7f5378078340 msgr2=0x7f5378100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:15.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.270+0000 7f537d751700 1 -- 172.21.15.62:0/4144832771 shutdown_connections 2024-06-10T00:35:15.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:15.270+0000 7f537d751700 1 -- 172.21.15.62:0/4144832771 wait complete. 2024-06-10T00:35:15.597 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:15 smithi062 ceph-mon[32390]: pgmap v75: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-10T00:35:15.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:15 smithi105 ceph-mon[36321]: pgmap v75: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-10T00:35:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:15 smithi138 ceph-mon[36549]: pgmap v75: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-10T00:35:15.964 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:35:16.091 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:35:16.544 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:16 smithi062 ceph-mon[32390]: scrub queued for path: / 2024-06-10T00:35:16.545 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:16 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:35:16.545 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:16 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:35:16.545 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:16 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:35:16.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:16 smithi105 ceph-mon[36321]: scrub queued for path: / 2024-06-10T00:35:16.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:16 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:35:16.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:16 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:35:16.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:16 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:35:16.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:16 smithi138 ceph-mon[36549]: scrub queued for path: / 2024-06-10T00:35:16.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:16 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:35:16.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:16 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:35:16.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:16 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:35:17.373 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:35:17.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:17 smithi062 ceph-mon[32390]: pgmap v76: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-10T00:35:17.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:17 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:35:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:17 smithi105 ceph-mon[36321]: pgmap v76: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-10T00:35:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:17 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:35:17.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:17 smithi138 ceph-mon[36549]: pgmap v76: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-10T00:35:17.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:17 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:35:19.330 INFO:tasks.workunit.client.0.smithi062.stdout:checking for stat64... yes 2024-06-10T00:35:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.352+0000 7f464a666700 1 -- 172.21.15.62:0/684661616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4644101df0 msgr2=0x7f46441021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.352+0000 7f464a666700 1 --2- 172.21.15.62:0/684661616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4644101df0 0x7f46441021d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f46340099c0 tx=0x7f4634009cd0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.352+0000 7f464a666700 1 -- 172.21.15.62:0/684661616 shutdown_connections 2024-06-10T00:35:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.352+0000 7f464a666700 1 --2- 172.21.15.62:0/684661616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4644101df0 0x7f46441021d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.352+0000 7f464a666700 1 --2- 172.21.15.62:0/684661616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4644106750 0x7f464410e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.352+0000 7f464a666700 1 --2- 172.21.15.62:0/684661616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46441027a0 0x7f4644106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.352+0000 7f464a666700 1 -- 172.21.15.62:0/684661616 >> 172.21.15.62:0/684661616 conn(0x7f4644077a00 msgr2=0x7f46440fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.353+0000 7f464a666700 1 -- 172.21.15.62:0/684661616 shutdown_connections 2024-06-10T00:35:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.353+0000 7f464a666700 1 -- 172.21.15.62:0/684661616 wait complete. 2024-06-10T00:35:19.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.353+0000 7f464a666700 1 Processor -- start 2024-06-10T00:35:19.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.353+0000 7f464a666700 1 -- start start 2024-06-10T00:35:19.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f464a666700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4644101df0 0x7f46440fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f464a666700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46441027a0 0x7f46440fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f464a666700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4644106750 0x7f46440fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f464a666700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4644111670 con 0x7f4644106750 2024-06-10T00:35:19.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f464a666700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f46441114f0 con 0x7f4644101df0 2024-06-10T00:35:19.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f464a666700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46441117f0 con 0x7f46441027a0 2024-06-10T00:35:19.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f4648c03700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4644106750 0x7f46440fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f4648c03700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4644106750 0x7f46440fd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56774/0 (socket says 172.21.15.62:56774) 2024-06-10T00:35:19.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f4648c03700 1 -- 172.21.15.62:0/2521630085 learned_addr learned my addr 172.21.15.62:0/2521630085 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:19.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f46437fe700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46441027a0 0x7f46440fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f4643fff700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4644101df0 0x7f46440fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f46437fe700 1 -- 172.21.15.62:0/2521630085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4644101df0 msgr2=0x7f46440fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f46437fe700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4644101df0 0x7f46440fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f46437fe700 1 -- 172.21.15.62:0/2521630085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4644106750 msgr2=0x7f46440fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f46437fe700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4644106750 0x7f46440fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f46437fe700 1 -- 172.21.15.62:0/2521630085 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46441a70a0 con 0x7f46441027a0 2024-06-10T00:35:19.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.354+0000 7f4648c03700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4644106750 0x7f46440fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:35:19.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.355+0000 7f46437fe700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46441027a0 0x7f46440fd3b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f463800e9f0 tx=0x7f463800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:19.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.355+0000 7f46417fa700 1 -- 172.21.15.62:0/2521630085 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463800c9c0 con 0x7f46441027a0 2024-06-10T00:35:19.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.355+0000 7f46417fa700 1 -- 172.21.15.62:0/2521630085 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f463800cb60 con 0x7f46441027a0 2024-06-10T00:35:19.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.355+0000 7f464a666700 1 -- 172.21.15.62:0/2521630085 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46441a7390 con 0x7f46441027a0 2024-06-10T00:35:19.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.355+0000 7f46417fa700 1 -- 172.21.15.62:0/2521630085 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638018680 con 0x7f46441027a0 2024-06-10T00:35:19.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.355+0000 7f464a666700 1 -- 172.21.15.62:0/2521630085 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46441a7880 con 0x7f46441027a0 2024-06-10T00:35:19.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.358+0000 7f46417fa700 1 -- 172.21.15.62:0/2521630085 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4638010430 con 0x7f46441027a0 2024-06-10T00:35:19.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.359+0000 7f46417fa700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f462c0a2d90 0x7f462c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.359+0000 7f46417fa700 1 -- 172.21.15.62:0/2521630085 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f46380cf9c0 con 0x7f46441027a0 2024-06-10T00:35:19.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.359+0000 7f4643fff700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f462c0a2d90 0x7f462c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.359+0000 7f4654ee9b80 1 Processor -- start 2024-06-10T00:35:19.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.360+0000 7f4654ee9b80 1 -- start start 2024-06-10T00:35:19.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.360+0000 7f4654ee9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1b8b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.360+0000 7f4654ee9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1b85260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.360+0000 7f4654ee9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b85bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.360+0000 7f4654ee9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555de1a77f60 con 0x555de1b8b880 2024-06-10T00:35:19.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.360+0000 7f4654ee9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555de1ad6850 con 0x555de1b857a0 2024-06-10T00:35:19.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.360+0000 7f4654ee9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555de1a3e460 con 0x555de1b8af60 2024-06-10T00:35:19.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.360+0000 7f4643fff700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f462c0a2d90 0x7f462c0a5250 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4634000c00 tx=0x7f4634005fb0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:19.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.360+0000 7f4643fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1b8b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.360+0000 7f4643fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1b8b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:55626/0 (socket says 172.21.15.62:55626) 2024-06-10T00:35:19.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.360+0000 7f4643fff700 1 -- 172.21.15.62:0/1773501190 learned_addr learned my addr 172.21.15.62:0/1773501190 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:19.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.360+0000 7f46437fe700 1 --2- 172.21.15.62:0/1773501190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1b85260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.360+0000 7f4648c03700 1 --2- 172.21.15.62:0/1773501190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b85bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.361+0000 7f4643fff700 1 -- 172.21.15.62:0/1773501190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 msgr2=0x555de1b85bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.361+0000 7f4643fff700 1 --2- 172.21.15.62:0/1773501190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b85bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.361+0000 7f4643fff700 1 -- 172.21.15.62:0/1773501190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 msgr2=0x555de1b85260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.361+0000 7f4643fff700 1 --2- 172.21.15.62:0/1773501190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1b85260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.361+0000 7f4643fff700 1 -- 172.21.15.62:0/1773501190 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555de1a258d0 con 0x555de1b8af60 2024-06-10T00:35:19.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.361+0000 7f4648c03700 1 --2- 172.21.15.62:0/1773501190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b85bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:35:19.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.361+0000 7f4643fff700 1 --2- 172.21.15.62:0/1773501190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1b8b340 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f463400baf0 tx=0x7f463401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c87fb73f4fb3935c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:19.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.361+0000 7f462a7fc700 1 -- 172.21.15.62:0/1773501190 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634034070 con 0x555de1b8af60 2024-06-10T00:35:19.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.361+0000 7f462a7fc700 1 -- 172.21.15.62:0/1773501190 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4634024960 con 0x555de1b8af60 2024-06-10T00:35:19.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.361+0000 7f462a7fc700 1 -- 172.21.15.62:0/1773501190 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463402fb40 con 0x555de1b8af60 2024-06-10T00:35:19.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.362+0000 7f4654ee9b80 1 -- 172.21.15.62:0/1773501190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 msgr2=0x555de1b8b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.362+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/1773501190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1b8b340 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f463400baf0 tx=0x7f463401f000 comp rx=0 tx=0).stop 2024-06-10T00:35:19.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.362+0000 7f4654ee9b80 1 -- 172.21.15.62:0/1773501190 shutdown_connections 2024-06-10T00:35:19.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.362+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/1773501190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b85bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.362+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/1773501190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1b8b340 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.362+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/1773501190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1b85260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.362+0000 7f4654ee9b80 1 -- 172.21.15.62:0/1773501190 >> 172.21.15.62:0/1773501190 conn(0x555de1ab2080 msgr2=0x555de1b80220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:19.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.362+0000 7f4654ee9b80 1 -- 172.21.15.62:0/1773501190 shutdown_connections 2024-06-10T00:35:19.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.362+0000 7f4654ee9b80 1 -- 172.21.15.62:0/1773501190 wait complete. 2024-06-10T00:35:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.363+0000 7f4654ee9b80 1 -- start start 2024-06-10T00:35:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.363+0000 7f4654ee9b80 1 Processor -- start 2024-06-10T00:35:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.364+0000 7f4654ee9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b916c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.364+0000 7f4654ee9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1b8e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.364+0000 7f4654ee9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1b8ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.364+0000 7f4654ee9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555de1a76fc0 con 0x555de1b8b880 2024-06-10T00:35:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.364+0000 7f4654ee9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555de1ad2810 con 0x555de1b857a0 2024-06-10T00:35:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.364+0000 7f4654ee9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555de1b92200 con 0x555de1b8af60 2024-06-10T00:35:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.364+0000 7f4643fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b916c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.364+0000 7f4643fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b916c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:49562/0 (socket says 172.21.15.62:49562) 2024-06-10T00:35:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.364+0000 7f4643fff700 1 -- 172.21.15.62:0/1753224721 learned_addr learned my addr 172.21.15.62:0/1753224721 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.364+0000 7f4648c03700 1 --2- 172.21.15.62:0/1753224721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1b8ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.364+0000 7f46437fe700 1 --2- 172.21.15.62:0/1753224721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1b8e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.365+0000 7f4654ee9b80 10 client.0 _ll_register_callbacks cb 0x555de1a6c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.365+0000 7f4643fff700 1 -- 172.21.15.62:0/1753224721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 msgr2=0x555de1b8e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.365+0000 7f4643fff700 1 --2- 172.21.15.62:0/1753224721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1b8e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.365+0000 7f4643fff700 1 -- 172.21.15.62:0/1753224721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 msgr2=0x555de1b8ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.365+0000 7f4643fff700 1 --2- 172.21.15.62:0/1753224721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1b8ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.365+0000 7f4643fff700 1 -- 172.21.15.62:0/1753224721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555de1a9c750 con 0x555de1b857a0 2024-06-10T00:35:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.365+0000 7f4654ee9b80 1 -- 172.21.15.62:0/1753224721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4634024960 con 0x555de1b857a0 2024-06-10T00:35:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.365+0000 7f4643fff700 1 --2- 172.21.15.62:0/1753224721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b916c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f463402f6c0 tx=0x7f4634024dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ed635b1c79a2d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.365+0000 7f46237fe700 10 client.25670 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:35:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.365+0000 7f46237fe700 1 -- 172.21.15.62:0/1753224721 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634034040 con 0x555de1b857a0 2024-06-10T00:35:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.365+0000 7f46237fe700 1 -- 172.21.15.62:0/1753224721 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f463402f7c0 con 0x555de1b857a0 2024-06-10T00:35:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.365+0000 7f46237fe700 1 -- 172.21.15.62:0/1753224721 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f463402f9c0 con 0x555de1b857a0 2024-06-10T00:35:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.366+0000 7f4654ee9b80 10 client.25670 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.366+0000 7f4654ee9b80 1 -- 172.21.15.62:0/1753224721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=30}) -- 0x555de1b9c8c0 con 0x555de1b857a0 2024-06-10T00:35:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.366+0000 7f46237fe700 1 -- 172.21.15.62:0/1753224721 <== mon.2 v2:172.21.15.105:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4634040030 con 0x555de1b857a0 2024-06-10T00:35:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.366+0000 7f4654ee9b80 10 client.25670 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.366+0000 7f4654ee9b80 10 client.25670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:35:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.366+0000 7f4654ee9b80 20 client.25670 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.366+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/1753224721 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555de1b9ed90 0x555de1ba1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.366+0000 7f4654ee9b80 4 client.25670 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:35:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.366+0000 7f4654ee9b80 1 -- 172.21.15.62:0/1753224721 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555de1ae3d50 con 0x555de1b9ed90 2024-06-10T00:35:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.366+0000 7f46437fe700 1 --2- 172.21.15.62:0/1753224721 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555de1b9ed90 0x555de1ba1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.367+0000 7f46237fe700 10 client.25670 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.367+0000 7f46437fe700 1 --2- 172.21.15.62:0/1753224721 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555de1b9ed90 0x555de1ba1180 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed55621b4c1d14d server_cookie=1597abe15acc6b7c in_seq=0 out_seq=0 2024-06-10T00:35:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f46237fe700 1 -- 172.21.15.62:0/1753224721 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555de1ae3d50 con 0x555de1b9ed90 2024-06-10T00:35:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f46237fe700 10 client.25670 handle_command_reply: tid=0 2024-06-10T00:35:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f4654ee9b80 1 client.25670 shutdown 2024-06-10T00:35:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f4654ee9b80 1 -- 172.21.15.62:0/1753224721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 msgr2=0x555de1b916c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/1753224721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b916c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f463402f6c0 tx=0x7f4634024dc0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f4654ee9b80 1 -- 172.21.15.62:0/1753224721 shutdown_connections 2024-06-10T00:35:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/1753224721 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555de1b9ed90 0x555de1ba1180 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f46237fe700 0 client.25670 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/1753224721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1b8ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/1753224721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b916c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/1753224721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1b8e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f4654ee9b80 1 -- 172.21.15.62:0/1753224721 >> 172.21.15.62:0/1753224721 conn(0x555de1ab2080 msgr2=0x555de1b7f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f4654ee9b80 1 -- 172.21.15.62:0/1753224721 shutdown_connections 2024-06-10T00:35:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f4654ee9b80 1 -- 172.21.15.62:0/1753224721 wait complete. 2024-06-10T00:35:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.368+0000 7f4654ee9b80 20 client.25670 trim_cache size 0 max 16384 2024-06-10T00:35:19.389 INFO:tasks.workunit.client.0.smithi062.stdout:checking for fseeko64... yes 2024-06-10T00:35:19.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.390+0000 7f4654ee9b80 1 Processor -- start 2024-06-10T00:35:19.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.390+0000 7f4654ee9b80 1 -- start start 2024-06-10T00:35:19.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.390+0000 7f4654ee9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b91200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.390+0000 7f4654ee9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1b7fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.390+0000 7f4654ee9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1b803c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.390+0000 7f4654ee9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f46441b0780 con 0x555de1b8b880 2024-06-10T00:35:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.390+0000 7f4654ee9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f46441b22c0 con 0x555de1b857a0 2024-06-10T00:35:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.390+0000 7f4654ee9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46441b0e50 con 0x555de1b8af60 2024-06-10T00:35:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.390+0000 7f4648c03700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1b803c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.390+0000 7f4643fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b91200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.390+0000 7f4648c03700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1b803c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56798/0 (socket says 172.21.15.62:56798) 2024-06-10T00:35:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.390+0000 7f4648c03700 1 -- 172.21.15.62:0/3988784250 learned_addr learned my addr 172.21.15.62:0/3988784250 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.390+0000 7f46437fe700 1 --2- 172.21.15.62:0/3988784250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1b7fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.391+0000 7f4643fff700 1 -- 172.21.15.62:0/3988784250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 msgr2=0x555de1b7fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.391+0000 7f4643fff700 1 --2- 172.21.15.62:0/3988784250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1b7fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.391+0000 7f4643fff700 1 -- 172.21.15.62:0/3988784250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 msgr2=0x555de1b803c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.391+0000 7f4643fff700 1 --2- 172.21.15.62:0/3988784250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1b803c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.391+0000 7f4643fff700 1 -- 172.21.15.62:0/3988784250 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555de1b860a0 con 0x555de1b857a0 2024-06-10T00:35:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.391+0000 7f4643fff700 1 --2- 172.21.15.62:0/3988784250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b91200 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4634024e00 tx=0x7f4634028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a67e90db64b8a846 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.391+0000 7f4629ffb700 1 -- 172.21.15.62:0/3988784250 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634034040 con 0x555de1b857a0 2024-06-10T00:35:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.391+0000 7f4629ffb700 1 -- 172.21.15.62:0/3988784250 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4634024960 con 0x555de1b857a0 2024-06-10T00:35:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.391+0000 7f4629ffb700 1 -- 172.21.15.62:0/3988784250 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463400fba0 con 0x555de1b857a0 2024-06-10T00:35:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.392+0000 7f4654ee9b80 1 -- 172.21.15.62:0/3988784250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 msgr2=0x555de1b91200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.392+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/3988784250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b91200 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4634024e00 tx=0x7f4634028000 comp rx=0 tx=0).stop 2024-06-10T00:35:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.392+0000 7f4654ee9b80 1 -- 172.21.15.62:0/3988784250 shutdown_connections 2024-06-10T00:35:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.392+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/3988784250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1b803c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.392+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/3988784250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1b91200 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.392+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/3988784250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1b7fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.392+0000 7f4654ee9b80 1 -- 172.21.15.62:0/3988784250 >> 172.21.15.62:0/3988784250 conn(0x555de1ab2080 msgr2=0x555de1b8e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.392+0000 7f4654ee9b80 1 -- 172.21.15.62:0/3988784250 shutdown_connections 2024-06-10T00:35:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.392+0000 7f4654ee9b80 1 -- 172.21.15.62:0/3988784250 wait complete. 2024-06-10T00:35:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.392+0000 7f4654ee9b80 1 -- start start 2024-06-10T00:35:19.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.393+0000 7f4654ee9b80 1 Processor -- start 2024-06-10T00:35:19.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.393+0000 7f4654ee9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1ba2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.393+0000 7f4654ee9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1ba2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.393+0000 7f4654ee9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1c086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.393+0000 7f4654ee9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f46441b1bf0 con 0x555de1b8b880 2024-06-10T00:35:19.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.393+0000 7f4654ee9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f46441b1520 con 0x555de1b857a0 2024-06-10T00:35:19.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.393+0000 7f4654ee9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46441ae720 con 0x555de1b8af60 2024-06-10T00:35:19.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.393+0000 7f4654ee9b80 10 client.0 _ll_register_callbacks cb 0x555de1a6c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:19.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.393+0000 7f4643fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1ba2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.393+0000 7f4643fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1ba2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:49584/0 (socket says 172.21.15.62:49584) 2024-06-10T00:35:19.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.393+0000 7f4643fff700 1 -- 172.21.15.62:0/3533108260 learned_addr learned my addr 172.21.15.62:0/3533108260 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:19.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.393+0000 7f4648c03700 1 --2- 172.21.15.62:0/3533108260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1c086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.393+0000 7f46437fe700 1 --2- 172.21.15.62:0/3533108260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1ba2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.394+0000 7f4643fff700 1 -- 172.21.15.62:0/3533108260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 msgr2=0x555de1ba2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.394+0000 7f4643fff700 1 --2- 172.21.15.62:0/3533108260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1ba2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.394+0000 7f4643fff700 1 -- 172.21.15.62:0/3533108260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 msgr2=0x555de1c086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.394+0000 7f4643fff700 1 --2- 172.21.15.62:0/3533108260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1c086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.394+0000 7f4643fff700 1 -- 172.21.15.62:0/3533108260 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555de1c08cc0 con 0x555de1b857a0 2024-06-10T00:35:19.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.394+0000 7f4643fff700 1 --2- 172.21.15.62:0/3533108260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1ba2780 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4634009350 tx=0x7f463400fa60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47edae68eaf575bb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:19.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.394+0000 7f46237fe700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:35:19.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.394+0000 7f46237fe700 1 -- 172.21.15.62:0/3533108260 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634034040 con 0x555de1b857a0 2024-06-10T00:35:19.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.394+0000 7f4654ee9b80 1 -- 172.21.15.62:0/3533108260 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4634024960 con 0x555de1b857a0 2024-06-10T00:35:19.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.394+0000 7f46237fe700 1 -- 172.21.15.62:0/3533108260 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f463400fc80 con 0x555de1b857a0 2024-06-10T00:35:19.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.394+0000 7f46237fe700 1 -- 172.21.15.62:0/3533108260 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4634057050 con 0x555de1b857a0 2024-06-10T00:35:19.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.394+0000 7f4654ee9b80 10 client.25676 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:19.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.395+0000 7f4654ee9b80 1 -- 172.21.15.62:0/3533108260 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555de1c08cc0 con 0x555de1b857a0 2024-06-10T00:35:19.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.395+0000 7f46237fe700 1 -- 172.21.15.62:0/3533108260 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4634024960 con 0x555de1b857a0 2024-06-10T00:35:19.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.395+0000 7f46237fe700 1 -- 172.21.15.62:0/3533108260 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4634052030 con 0x555de1b857a0 2024-06-10T00:35:19.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.395+0000 7f4654ee9b80 10 client.25676 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:19.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.395+0000 7f4654ee9b80 10 client.25676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:35:19.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.395+0000 7f4654ee9b80 20 client.25676 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:19.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.395+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/3533108260 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555de1c0a1e0 0x555de1c0c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.395+0000 7f4654ee9b80 4 client.25676 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:35:19.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.395+0000 7f4654ee9b80 1 -- 172.21.15.62:0/3533108260 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555de1ade2d0 con 0x555de1c0a1e0 2024-06-10T00:35:19.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.395+0000 7f46437fe700 1 --2- 172.21.15.62:0/3533108260 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555de1c0a1e0 0x555de1c0c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.396+0000 7f46437fe700 1 --2- 172.21.15.62:0/3533108260 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555de1c0a1e0 0x555de1c0c5d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe8cc47f9d49941 server_cookie=23eb477ac797010d in_seq=0 out_seq=0 2024-06-10T00:35:19.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.396+0000 7f46237fe700 10 client.25676 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:19.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.397+0000 7f46237fe700 1 -- 172.21.15.62:0/3533108260 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555de1ade2d0 con 0x555de1c0a1e0 2024-06-10T00:35:19.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.397+0000 7f46237fe700 10 client.25676 handle_command_reply: tid=0 2024-06-10T00:35:19.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.397+0000 7f4654ee9b80 1 client.25676 shutdown 2024-06-10T00:35:19.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.397+0000 7f4654ee9b80 1 -- 172.21.15.62:0/3533108260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 msgr2=0x555de1ba2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.397+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/3533108260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1ba2780 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4634009350 tx=0x7f463400fa60 comp rx=0 tx=0).stop 2024-06-10T00:35:19.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.397+0000 7f4654ee9b80 1 -- 172.21.15.62:0/3533108260 shutdown_connections 2024-06-10T00:35:19.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.397+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/3533108260 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555de1c0a1e0 0x555de1c0c5d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.397+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/3533108260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555de1b8b880 0x555de1c086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.397+0000 7f46237fe700 0 client.25676 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:19.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.397+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/3533108260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555de1b857a0 0x555de1ba2780 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.397+0000 7f4654ee9b80 1 --2- 172.21.15.62:0/3533108260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555de1b8af60 0x555de1ba2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.397+0000 7f4654ee9b80 1 -- 172.21.15.62:0/3533108260 >> 172.21.15.62:0/3533108260 conn(0x555de1ab2080 msgr2=0x555de1b8ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:19.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.397+0000 7f4654ee9b80 1 -- 172.21.15.62:0/3533108260 shutdown_connections 2024-06-10T00:35:19.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.397+0000 7f4654ee9b80 1 -- 172.21.15.62:0/3533108260 wait complete. 2024-06-10T00:35:19.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.398+0000 7f4654ee9b80 20 client.25676 trim_cache size 0 max 16384 2024-06-10T00:35:19.401 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:35:19.401 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-06-10T00:35:19.401 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-06-10T00:35:19.401 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:35:19.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.398+0000 7f464a666700 1 -- 172.21.15.62:0/2521630085 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f462c0a2d90 msgr2=0x7f462c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.398+0000 7f464a666700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f462c0a2d90 0x7f462c0a5250 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4634000c00 tx=0x7f4634005fb0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.398+0000 7f464a666700 1 -- 172.21.15.62:0/2521630085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46441027a0 msgr2=0x7f46440fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.398+0000 7f464a666700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46441027a0 0x7f46440fd3b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f463800e9f0 tx=0x7f463800ed00 comp rx=0 tx=0).stop 2024-06-10T00:35:19.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.398+0000 7f464a666700 1 -- 172.21.15.62:0/2521630085 shutdown_connections 2024-06-10T00:35:19.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.398+0000 7f464a666700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f462c0a2d90 0x7f462c0a5250 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.398+0000 7f464a666700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4644106750 0x7f46440fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.398+0000 7f464a666700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4644101df0 0x7f46440fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.398+0000 7f464a666700 1 --2- 172.21.15.62:0/2521630085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46441027a0 0x7f46440fd3b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.398+0000 7f464a666700 1 -- 172.21.15.62:0/2521630085 >> 172.21.15.62:0/2521630085 conn(0x7f4644077a00 msgr2=0x7f4644100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:19.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.398+0000 7f464a666700 1 -- 172.21.15.62:0/2521630085 shutdown_connections 2024-06-10T00:35:19.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.398+0000 7f464a666700 1 -- 172.21.15.62:0/2521630085 wait complete. 2024-06-10T00:35:19.449 INFO:tasks.workunit.client.0.smithi062.stdout:checking for lrand48_r... yes 2024-06-10T00:35:19.514 INFO:tasks.workunit.client.0.smithi062.stdout:checking for srand48_r... yes 2024-06-10T00:35:19.537 INFO:tasks.workunit.client.0.smithi062.stdout:configure: creating ./config.status 2024-06-10T00:35:19.601 INFO:tasks.workunit.client.0.smithi062.stdout: /bin/sh ./config.status 2024-06-10T00:35:19.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:19 smithi062 ceph-mon[32390]: pgmap v77: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-06-10T00:35:19.652 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: creating Makefile 2024-06-10T00:35:19.666 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: creating config.h 2024-06-10T00:35:19.692 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: executing depfiles commands 2024-06-10T00:35:19.718 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: executing default commands 2024-06-10T00:35:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.221+0000 7f84c785e700 1 -- 172.21.15.62:0/1766408472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c0102740 msgr2=0x7f84c0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.221+0000 7f84c785e700 1 --2- 172.21.15.62:0/1766408472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c0102740 0x7f84c0102b20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f84b000b0d0 tx=0x7f84b000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.221+0000 7f84c785e700 1 -- 172.21.15.62:0/1766408472 shutdown_connections 2024-06-10T00:35:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.221+0000 7f84c785e700 1 --2- 172.21.15.62:0/1766408472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c0102740 0x7f84c0102b20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.221+0000 7f84c785e700 1 --2- 172.21.15.62:0/1766408472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c0107840 0x7f84c010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.221+0000 7f84c785e700 1 --2- 172.21.15.62:0/1766408472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c01030f0 0x7f84c0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.221+0000 7f84c785e700 1 -- 172.21.15.62:0/1766408472 >> 172.21.15.62:0/1766408472 conn(0x7f84c00795f0 msgr2=0x7f84c0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.221+0000 7f84c785e700 1 -- 172.21.15.62:0/1766408472 shutdown_connections 2024-06-10T00:35:19.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.221+0000 7f84c785e700 1 -- 172.21.15.62:0/1766408472 wait complete. 2024-06-10T00:35:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.222+0000 7f84c785e700 1 Processor -- start 2024-06-10T00:35:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.222+0000 7f84c785e700 1 -- start start 2024-06-10T00:35:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.222+0000 7f84c785e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c0102740 0x7f84c019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.222+0000 7f84c785e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c01030f0 0x7f84c019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.222+0000 7f84c785e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c0107840 0x7f84c019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.222+0000 7f84c785e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0112820 con 0x7f84c01030f0 2024-06-10T00:35:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.222+0000 7f84c785e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01126a0 con 0x7f84c0102740 2024-06-10T00:35:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.222+0000 7f84c785e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01129a0 con 0x7f84c0107840 2024-06-10T00:35:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.222+0000 7f84c4df9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c01030f0 0x7f84c019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.222+0000 7f84c4df9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c01030f0 0x7f84c019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56722/0 (socket says 172.21.15.62:56722) 2024-06-10T00:35:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.222+0000 7f84c4df9700 1 -- 172.21.15.62:0/648406466 learned_addr learned my addr 172.21.15.62:0/648406466 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:19.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.222+0000 7f84c55fa700 1 --2- 172.21.15.62:0/648406466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c0102740 0x7f84c019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.222+0000 7f84c5dfb700 1 --2- 172.21.15.62:0/648406466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c0107840 0x7f84c019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.223+0000 7f84c4df9700 1 -- 172.21.15.62:0/648406466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c0107840 msgr2=0x7f84c019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.223+0000 7f84c4df9700 1 --2- 172.21.15.62:0/648406466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c0107840 0x7f84c019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.223+0000 7f84c4df9700 1 -- 172.21.15.62:0/648406466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c0102740 msgr2=0x7f84c019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.223+0000 7f84c4df9700 1 --2- 172.21.15.62:0/648406466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c0102740 0x7f84c019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.223+0000 7f84c4df9700 1 -- 172.21.15.62:0/648406466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c019fe30 con 0x7f84c01030f0 2024-06-10T00:35:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.223+0000 7f84c4df9700 1 --2- 172.21.15.62:0/648406466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c01030f0 0x7f84c019b360 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f84b400b530 tx=0x7f84b400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.223+0000 7f84be7fc700 1 -- 172.21.15.62:0/648406466 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4015020 con 0x7f84c01030f0 2024-06-10T00:35:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.223+0000 7f84be7fc700 1 -- 172.21.15.62:0/648406466 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b4011d10 con 0x7f84c01030f0 2024-06-10T00:35:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.223+0000 7f84be7fc700 1 -- 172.21.15.62:0/648406466 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b400f2e0 con 0x7f84c01030f0 2024-06-10T00:35:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.223+0000 7f84c785e700 1 -- 172.21.15.62:0/648406466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c01a0120 con 0x7f84c01030f0 2024-06-10T00:35:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.223+0000 7f84c785e700 1 -- 172.21.15.62:0/648406466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c01a79a0 con 0x7f84c01030f0 2024-06-10T00:35:19.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.225+0000 7f84be7fc700 1 -- 172.21.15.62:0/648406466 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f84b4018020 con 0x7f84c01030f0 2024-06-10T00:35:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.226+0000 7f84be7fc700 1 --2- 172.21.15.62:0/648406466 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f84a80a2d90 0x7f84a80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.226+0000 7f84c55fa700 1 --2- 172.21.15.62:0/648406466 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f84a80a2d90 0x7f84a80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.226+0000 7f84be7fc700 1 -- 172.21.15.62:0/648406466 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f84b40cef50 con 0x7f84c01030f0 2024-06-10T00:35:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84c55fa700 1 --2- 172.21.15.62:0/648406466 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f84a80a2d90 0x7f84a80a5250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009fd0 tx=0x7f84b000b040 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84d20e1b80 1 Processor -- start 2024-06-10T00:35:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84d20e1b80 1 -- start start 2024-06-10T00:35:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84d20e1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658da020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84d20e1b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 0x5591658d3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84d20e1b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 0x5591658d4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84d20e1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559165825910 con 0x5591658d9c40 2024-06-10T00:35:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84d20e1b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5591657cd820 con 0x5591658d43d0 2024-06-10T00:35:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84d20e1b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5591658218d0 con 0x5591658da560 2024-06-10T00:35:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84c55fa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658da020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84c55fa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658da020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56724/0 (socket says 172.21.15.62:56724) 2024-06-10T00:35:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84c55fa700 1 -- 172.21.15.62:0/969180501 learned_addr learned my addr 172.21.15.62:0/969180501 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84c5dfb700 1 --2- 172.21.15.62:0/969180501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 0x5591658d4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84c4df9700 1 --2- 172.21.15.62:0/969180501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 0x5591658d3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.227+0000 7f84c55fa700 1 -- 172.21.15.62:0/969180501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 msgr2=0x5591658d3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.228+0000 7f84c55fa700 1 --2- 172.21.15.62:0/969180501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 0x5591658d3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.228+0000 7f84c55fa700 1 -- 172.21.15.62:0/969180501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 msgr2=0x5591658d4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.228+0000 7f84c55fa700 1 --2- 172.21.15.62:0/969180501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 0x5591658d4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.228+0000 7f84c55fa700 1 -- 172.21.15.62:0/969180501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591657c4050 con 0x5591658d9c40 2024-06-10T00:35:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.228+0000 7f84c5dfb700 1 --2- 172.21.15.62:0/969180501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 0x5591658d4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:35:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.228+0000 7f84c55fa700 1 --2- 172.21.15.62:0/969180501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658da020 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f84b000b010 tx=0x7f84b001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca72e0df4e52b26 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.228+0000 7f84a77fe700 1 -- 172.21.15.62:0/969180501 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b002c070 con 0x5591658d9c40 2024-06-10T00:35:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.228+0000 7f84a77fe700 1 -- 172.21.15.62:0/969180501 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b0007bc0 con 0x5591658d9c40 2024-06-10T00:35:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.228+0000 7f84a77fe700 1 -- 172.21.15.62:0/969180501 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0028cf0 con 0x5591658d9c40 2024-06-10T00:35:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.229+0000 7f84d20e1b80 1 -- 172.21.15.62:0/969180501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 msgr2=0x5591658da020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.229+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/969180501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658da020 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f84b000b010 tx=0x7f84b001d000 comp rx=0 tx=0).stop 2024-06-10T00:35:19.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.229+0000 7f84d20e1b80 1 -- 172.21.15.62:0/969180501 shutdown_connections 2024-06-10T00:35:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.229+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/969180501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658da020 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.229+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/969180501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 0x5591658d4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.229+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/969180501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 0x5591658d3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.229+0000 7f84d20e1b80 1 -- 172.21.15.62:0/969180501 >> 172.21.15.62:0/969180501 conn(0x5591657fedc0 msgr2=0x5591658cf120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.229+0000 7f84d20e1b80 1 -- 172.21.15.62:0/969180501 shutdown_connections 2024-06-10T00:35:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.229+0000 7f84d20e1b80 1 -- 172.21.15.62:0/969180501 wait complete. 2024-06-10T00:35:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.229+0000 7f84d20e1b80 1 -- start start 2024-06-10T00:35:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.229+0000 7f84d20e1b80 1 Processor -- start 2024-06-10T00:35:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84d20e1b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 0x5591658e02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84d20e1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658dd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84d20e1b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 0x5591658ddae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84d20e1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5591658e0f70 con 0x5591658d9c40 2024-06-10T00:35:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84d20e1b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5591657c5cf0 con 0x5591658d43d0 2024-06-10T00:35:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84d20e1b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5591658e10f0 con 0x5591658da560 2024-06-10T00:35:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84d20e1b80 10 client.0 _ll_register_callbacks cb 0x5591657bfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84c4df9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658dd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84c4df9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658dd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56738/0 (socket says 172.21.15.62:56738) 2024-06-10T00:35:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84c4df9700 1 -- 172.21.15.62:0/455348312 learned_addr learned my addr 172.21.15.62:0/455348312 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84c55fa700 1 --2- 172.21.15.62:0/455348312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 0x5591658e02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84c5dfb700 1 --2- 172.21.15.62:0/455348312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 0x5591658ddae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84c4df9700 1 -- 172.21.15.62:0/455348312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 msgr2=0x5591658ddae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84c4df9700 1 --2- 172.21.15.62:0/455348312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 0x5591658ddae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84c4df9700 1 -- 172.21.15.62:0/455348312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 msgr2=0x5591658e02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84c4df9700 1 --2- 172.21.15.62:0/455348312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 0x5591658e02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84c4df9700 1 -- 172.21.15.62:0/455348312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591657749d0 con 0x5591658d9c40 2024-06-10T00:35:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84c4df9700 1 --2- 172.21.15.62:0/455348312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658dd5a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f84b40cc4b0 tx=0x7f84b409b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ec6221a28f3f23 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84a4ff9700 1 -- 172.21.15.62:0/455348312 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b40a3e40 con 0x5591658d9c40 2024-06-10T00:35:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84a4ff9700 1 -- 172.21.15.62:0/455348312 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f84b40bb0c0 con 0x5591658d9c40 2024-06-10T00:35:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.230+0000 7f84d20e1b80 1 -- 172.21.15.62:0/455348312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b0007bc0 con 0x5591658d9c40 2024-06-10T00:35:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.231+0000 7f84a4ff9700 1 -- 172.21.15.62:0/455348312 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f84b40bb2c0 con 0x5591658d9c40 2024-06-10T00:35:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.231+0000 7f84d20e1b80 10 client.16137 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.231+0000 7f84d20e1b80 1 -- 172.21.15.62:0/455348312 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591658eb850 con 0x5591658d9c40 2024-06-10T00:35:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.231+0000 7f84a4ff9700 1 -- 172.21.15.62:0/455348312 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b0007bc0 con 0x5591658d9c40 2024-06-10T00:35:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.231+0000 7f84a4ff9700 1 -- 172.21.15.62:0/455348312 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f84b409f1b0 con 0x5591658d9c40 2024-06-10T00:35:19.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.231+0000 7f84d20e1b80 10 client.16137 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.231+0000 7f84d20e1b80 10 client.16137 resolve_mds: validated gid 24464 aka daemon mds.b 2024-06-10T00:35:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.231+0000 7f84d20e1b80 20 client.16137 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.231+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/455348312 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5591658edde0 0x5591658f01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.231+0000 7f84d20e1b80 4 client.16137 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:35:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.231+0000 7f84d20e1b80 1 -- 172.21.15.62:0/455348312 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559165832e10 con 0x5591658edde0 2024-06-10T00:35:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.231+0000 7f84c55fa700 1 --2- 172.21.15.62:0/455348312 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5591658edde0 0x5591658f01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.232+0000 7f84c55fa700 1 --2- 172.21.15.62:0/455348312 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5591658edde0 0x5591658f01d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41f5b7f1ce290a90 server_cookie=3014c31c85d8b05b in_seq=0 out_seq=0 2024-06-10T00:35:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.232+0000 7f84a4ff9700 10 client.16137 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84a4ff9700 1 -- 172.21.15.62:0/455348312 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559165832e10 con 0x5591658edde0 2024-06-10T00:35:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84a4ff9700 10 client.16137 handle_command_reply: tid=0 2024-06-10T00:35:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84d20e1b80 1 client.16137 shutdown 2024-06-10T00:35:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84d20e1b80 1 -- 172.21.15.62:0/455348312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 msgr2=0x5591658dd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/455348312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658dd5a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f84b40cc4b0 tx=0x7f84b409b940 comp rx=0 tx=0).stop 2024-06-10T00:35:19.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84d20e1b80 1 -- 172.21.15.62:0/455348312 shutdown_connections 2024-06-10T00:35:19.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/455348312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 0x5591658ddae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/455348312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 0x5591658e02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/455348312 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5591658edde0 0x5591658f01d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/455348312 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658dd5a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84a4ff9700 0 client.16137 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:19.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84d20e1b80 1 -- 172.21.15.62:0/455348312 >> 172.21.15.62:0/455348312 conn(0x5591657fedc0 msgr2=0x5591658ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:19.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84d20e1b80 1 -- 172.21.15.62:0/455348312 shutdown_connections 2024-06-10T00:35:19.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84d20e1b80 1 -- 172.21.15.62:0/455348312 wait complete. 2024-06-10T00:35:19.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.233+0000 7f84d20e1b80 20 client.16137 trim_cache size 0 max 16384 2024-06-10T00:35:19.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84d20e1b80 1 Processor -- start 2024-06-10T00:35:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84d20e1b80 1 -- start start 2024-06-10T00:35:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84d20e1b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658d43d0 0x5591658d3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84d20e1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84d20e1b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658da560 0x5591658ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84d20e1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b0880 con 0x5591658d9c40 2024-06-10T00:35:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84d20e1b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b23c0 con 0x5591658da560 2024-06-10T00:35:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84d20e1b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b0f50 con 0x5591658d43d0 2024-06-10T00:35:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84c4df9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84c4df9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56744/0 (socket says 172.21.15.62:56744) 2024-06-10T00:35:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84c4df9700 1 -- 172.21.15.62:0/1000926208 learned_addr learned my addr 172.21.15.62:0/1000926208 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84c55fa700 1 --2- 172.21.15.62:0/1000926208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658d43d0 0x5591658d3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84c5dfb700 1 --2- 172.21.15.62:0/1000926208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658da560 0x5591658ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84c4df9700 1 -- 172.21.15.62:0/1000926208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658d43d0 msgr2=0x5591658d3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84c4df9700 1 --2- 172.21.15.62:0/1000926208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658d43d0 0x5591658d3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84c4df9700 1 -- 172.21.15.62:0/1000926208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658da560 msgr2=0x5591658ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84c4df9700 1 --2- 172.21.15.62:0/1000926208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658da560 0x5591658ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.254+0000 7f84c4df9700 1 -- 172.21.15.62:0/1000926208 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591658f07b0 con 0x5591658d9c40 2024-06-10T00:35:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.255+0000 7f84c4df9700 1 --2- 172.21.15.62:0/1000926208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658ce830 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f84b400bc40 tx=0x7f84b4004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74bdc48ada97f40 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.255+0000 7f84a6ffd700 1 -- 172.21.15.62:0/1000926208 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b40a3a30 con 0x5591658d9c40 2024-06-10T00:35:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.255+0000 7f84a6ffd700 1 -- 172.21.15.62:0/1000926208 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b40a3bd0 con 0x5591658d9c40 2024-06-10T00:35:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.255+0000 7f84a6ffd700 1 -- 172.21.15.62:0/1000926208 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b409f120 con 0x5591658d9c40 2024-06-10T00:35:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.256+0000 7f84d20e1b80 1 -- 172.21.15.62:0/1000926208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 msgr2=0x5591658ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.256+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/1000926208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658ce830 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f84b400bc40 tx=0x7f84b4004860 comp rx=0 tx=0).stop 2024-06-10T00:35:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.256+0000 7f84d20e1b80 1 -- 172.21.15.62:0/1000926208 shutdown_connections 2024-06-10T00:35:19.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.256+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/1000926208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658da560 0x5591658ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.256+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/1000926208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658d43d0 0x5591658d3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.256+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/1000926208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658ce830 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.256+0000 7f84d20e1b80 1 -- 172.21.15.62:0/1000926208 >> 172.21.15.62:0/1000926208 conn(0x5591657fedc0 msgr2=0x5591658daa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:19.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.256+0000 7f84d20e1b80 1 -- 172.21.15.62:0/1000926208 shutdown_connections 2024-06-10T00:35:19.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.256+0000 7f84d20e1b80 1 -- 172.21.15.62:0/1000926208 wait complete. 2024-06-10T00:35:19.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.256+0000 7f84d20e1b80 1 -- start start 2024-06-10T00:35:19.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.256+0000 7f84d20e1b80 1 Processor -- start 2024-06-10T00:35:19.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84d20e1b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 0x5591658e3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84d20e1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658e3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84d20e1b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 0x5591659589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84d20e1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b1cf0 con 0x5591658d9c40 2024-06-10T00:35:19.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84d20e1b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01b1620 con 0x5591658d43d0 2024-06-10T00:35:19.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84d20e1b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01ae820 con 0x5591658da560 2024-06-10T00:35:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84d20e1b80 10 client.0 _ll_register_callbacks cb 0x5591657bfa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84c4df9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658e3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84c4df9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658e3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56750/0 (socket says 172.21.15.62:56750) 2024-06-10T00:35:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84c4df9700 1 -- 172.21.15.62:0/3387943871 learned_addr learned my addr 172.21.15.62:0/3387943871 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84c55fa700 1 --2- 172.21.15.62:0/3387943871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 0x5591658e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84c5dfb700 1 --2- 172.21.15.62:0/3387943871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 0x5591659589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84c4df9700 1 -- 172.21.15.62:0/3387943871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 msgr2=0x5591659589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84c4df9700 1 --2- 172.21.15.62:0/3387943871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 0x5591659589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84c4df9700 1 -- 172.21.15.62:0/3387943871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 msgr2=0x5591658e3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84c4df9700 1 --2- 172.21.15.62:0/3387943871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 0x5591658e3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84c4df9700 1 -- 172.21.15.62:0/3387943871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559165958f20 con 0x5591658d9c40 2024-06-10T00:35:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84c5dfb700 1 --2- 172.21.15.62:0/3387943871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 0x5591659589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:35:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84c4df9700 1 --2- 172.21.15.62:0/3387943871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658e3ea0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f84b409aed0 tx=0x7f84b4010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d04172dfb0da9e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84a4ff9700 1 -- 172.21.15.62:0/3387943871 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b409b680 con 0x5591658d9c40 2024-06-10T00:35:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84a4ff9700 1 -- 172.21.15.62:0/3387943871 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f84b409f120 con 0x5591658d9c40 2024-06-10T00:35:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.257+0000 7f84d20e1b80 1 -- 172.21.15.62:0/3387943871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b40a3bd0 con 0x5591658d9c40 2024-06-10T00:35:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.258+0000 7f84a4ff9700 1 -- 172.21.15.62:0/3387943871 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f84b40174b0 con 0x5591658d9c40 2024-06-10T00:35:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.258+0000 7f84d20e1b80 10 client.16143 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.258+0000 7f84d20e1b80 1 -- 172.21.15.62:0/3387943871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55916595a230 con 0x5591658d9c40 2024-06-10T00:35:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.258+0000 7f84a4ff9700 1 -- 172.21.15.62:0/3387943871 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b40a3bd0 con 0x5591658d9c40 2024-06-10T00:35:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.258+0000 7f84a4ff9700 1 -- 172.21.15.62:0/3387943871 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f84b4096f20 con 0x5591658d9c40 2024-06-10T00:35:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.258+0000 7f84d20e1b80 10 client.16143 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.258+0000 7f84d20e1b80 10 client.16143 resolve_mds: validated gid 24464 aka daemon mds.b 2024-06-10T00:35:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.258+0000 7f84d20e1b80 20 client.16143 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.258+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/3387943871 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55916595a650 0x55916595ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.258+0000 7f84d20e1b80 4 client.16143 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "547e97b4", "release": true, "await": true}] 2024-06-10T00:35:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.258+0000 7f84d20e1b80 1 -- 172.21.15.62:0/3387943871 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "quiesce db", "set_id": "547e97b4", "release": true, "await": true}) -- 0x5591658d4e40 con 0x55916595a650 2024-06-10T00:35:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.258+0000 7f84c55fa700 1 --2- 172.21.15.62:0/3387943871 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55916595a650 0x55916595ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.259+0000 7f84c55fa700 1 --2- 172.21.15.62:0/3387943871 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55916595a650 0x55916595ca70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6c359b4eb62e9b1 server_cookie=e31c46ed1e7b829a in_seq=0 out_seq=0 2024-06-10T00:35:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.259+0000 7f84a4ff9700 10 client.16143 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.291+0000 7f84a4ff9700 1 -- 172.21.15.62:0/3387943871 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5591658d4e40 con 0x55916595a650 2024-06-10T00:35:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.291+0000 7f84a4ff9700 10 client.16143 handle_command_reply: tid=0 2024-06-10T00:35:19.765 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:35:19.766 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 30, 2024-06-10T00:35:19.766 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24464, 2024-06-10T00:35:19.766 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 8, 2024-06-10T00:35:19.766 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-06-10T00:35:19.766 INFO:teuthology.orchestra.run.smithi062.stdout: "547e97b4": { 2024-06-10T00:35:19.766 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 8, 2024-06-10T00:35:19.766 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-06-10T00:35:19.766 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-06-10T00:35:19.766 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-06-10T00:35:19.766 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-06-10T00:35:19.766 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:35:19.766 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-06-10T00:35:19.767 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 150.0, 2024-06-10T00:35:19.767 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-06-10T00:35:19.767 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-06-10T00:35:19.767 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-06-10T00:35:19.767 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-06-10T00:35:19.767 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-06-10T00:35:19.767 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-06-10T00:35:19.767 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:35:19.767 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:35:19.767 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:35:19.767 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:35:19.767 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:35:19.767 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:35:19.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.291+0000 7f84d20e1b80 1 client.16143 shutdown 2024-06-10T00:35:19.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.291+0000 7f84d20e1b80 1 -- 172.21.15.62:0/3387943871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 msgr2=0x5591658e3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.291+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/3387943871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658e3ea0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f84b409aed0 tx=0x7f84b4010a10 comp rx=0 tx=0).stop 2024-06-10T00:35:19.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.291+0000 7f84d20e1b80 1 -- 172.21.15.62:0/3387943871 shutdown_connections 2024-06-10T00:35:19.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.291+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/3387943871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591658da560 0x5591659589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.291+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/3387943871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591658d43d0 0x5591658e3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.291+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/3387943871 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55916595a650 0x55916595ca70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84d20e1b80 1 --2- 172.21.15.62:0/3387943871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5591658d9c40 0x5591658e3ea0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84a4ff9700 0 client.16143 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:35:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84d20e1b80 1 -- 172.21.15.62:0/3387943871 >> 172.21.15.62:0/3387943871 conn(0x5591657fedc0 msgr2=0x5591658ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84d20e1b80 1 -- 172.21.15.62:0/3387943871 shutdown_connections 2024-06-10T00:35:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84d20e1b80 1 -- 172.21.15.62:0/3387943871 wait complete. 2024-06-10T00:35:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84d20e1b80 20 client.16143 trim_cache size 0 max 16384 2024-06-10T00:35:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84c785e700 1 -- 172.21.15.62:0/648406466 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f84a80a2d90 msgr2=0x7f84a80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84c785e700 1 --2- 172.21.15.62:0/648406466 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f84a80a2d90 0x7f84a80a5250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009fd0 tx=0x7f84b000b040 comp rx=0 tx=0).stop 2024-06-10T00:35:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84c785e700 1 -- 172.21.15.62:0/648406466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c01030f0 msgr2=0x7f84c019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84c785e700 1 --2- 172.21.15.62:0/648406466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c01030f0 0x7f84c019b360 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f84b400b530 tx=0x7f84b400b840 comp rx=0 tx=0).stop 2024-06-10T00:35:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84c785e700 1 -- 172.21.15.62:0/648406466 shutdown_connections 2024-06-10T00:35:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84c785e700 1 --2- 172.21.15.62:0/648406466 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f84a80a2d90 0x7f84a80a5250 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84c785e700 1 --2- 172.21.15.62:0/648406466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c0107840 0x7f84c019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84c785e700 1 --2- 172.21.15.62:0/648406466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c0102740 0x7f84c019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84c785e700 1 --2- 172.21.15.62:0/648406466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c01030f0 0x7f84c019b360 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.292+0000 7f84c785e700 1 -- 172.21.15.62:0/648406466 >> 172.21.15.62:0/648406466 conn(0x7f84c00795f0 msgr2=0x7f84c01006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:19.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.293+0000 7f84c785e700 1 -- 172.21.15.62:0/648406466 shutdown_connections 2024-06-10T00:35:19.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:19.293+0000 7f84c785e700 1 -- 172.21.15.62:0/648406466 wait complete. 2024-06-10T00:35:19.770 INFO:tasks.workunit.client.0.smithi062.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-06-10T00:35:19.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:19 smithi105 ceph-mon[36321]: pgmap v77: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-06-10T00:35:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:19 smithi138 ceph-mon[36549]: pgmap v77: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-06-10T00:35:19.935 INFO:tasks.workunit.client.0.smithi062.stdout:rm -f stamp-h1 2024-06-10T00:35:19.938 INFO:tasks.workunit.client.0.smithi062.stdout:touch config.h.in 2024-06-10T00:35:19.939 INFO:tasks.workunit.client.0.smithi062.stdout:cd . && /bin/sh ./config.status config.h 2024-06-10T00:35:19.982 INFO:tasks.workunit.client.0.smithi062.stdout:config.status: creating config.h 2024-06-10T00:35:20.012 INFO:tasks.workunit.client.0.smithi062.stdout:make all-am 2024-06-10T00:35:20.018 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-06-10T00:35:20.018 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-06-10T00:35:20.270 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-06-10T00:35:20.274 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-06-10T00:35:20.416 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-06-10T00:35:20.419 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-06-10T00:35:20.521 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-06-10T00:35:20.524 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-06-10T00:35:20.564 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '547e97b4', total seconds elapsed: 40.0 2024-06-10T00:35:20.616 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-06-10T00:35:20.619 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-06-10T00:35:20.791 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-06-10T00:35:20.794 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-06-10T00:35:20.844 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-06-10T00:35:20.848 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-06-10T00:35:20.891 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-06-10T00:35:20.894 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-06-10T00:35:20.941 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-06-10T00:35:20.945 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-06-10T00:35:21.047 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T00:35:21.047 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:35:21.174 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-06-10T00:35:21.177 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-06-10T00:35:21.304 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:35:21.317 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-06-10T00:35:21.320 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-06-10T00:35:21.487 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-06-10T00:35:21.490 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-06-10T00:35:21.559 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-06-10T00:35:21.562 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-06-10T00:35:21.634 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-06-10T00:35:21.637 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-06-10T00:35:21.678 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:21 smithi062 ceph-mon[32390]: pgmap v78: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-06-10T00:35:21.744 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-06-10T00:35:21.748 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-06-10T00:35:21.776 INFO:tasks.workunit.client.0.smithi062.stderr:In file included from parser.h:22, 2024-06-10T00:35:21.777 INFO:tasks.workunit.client.0.smithi062.stderr: from parser.c:25: 2024-06-10T00:35:21.777 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c: In function ‘init_filesys’: 2024-06-10T00:35:21.777 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-06-10T00:35:21.777 INFO:tasks.workunit.client.0.smithi062.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-06-10T00:35:21.777 INFO:tasks.workunit.client.0.smithi062.stderr: | ^ 2024-06-10T00:35:21.777 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-06-10T00:35:21.777 INFO:tasks.workunit.client.0.smithi062.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-06-10T00:35:21.777 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~ 2024-06-10T00:35:21.777 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-06-10T00:35:21.777 INFO:tasks.workunit.client.0.smithi062.stderr: 36 | container_of(ptr, type, member) 2024-06-10T00:35:21.777 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~ 2024-06-10T00:35:21.778 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-06-10T00:35:21.778 INFO:tasks.workunit.client.0.smithi062.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-06-10T00:35:21.778 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~ 2024-06-10T00:35:21.778 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-06-10T00:35:21.778 INFO:tasks.workunit.client.0.smithi062.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-06-10T00:35:21.778 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-06-10T00:35:21.778 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-06-10T00:35:21.778 INFO:tasks.workunit.client.0.smithi062.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-06-10T00:35:21.778 INFO:tasks.workunit.client.0.smithi062.stderr: | ^ 2024-06-10T00:35:21.778 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-06-10T00:35:21.778 INFO:tasks.workunit.client.0.smithi062.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-06-10T00:35:21.778 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~ 2024-06-10T00:35:21.778 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-06-10T00:35:21.779 INFO:tasks.workunit.client.0.smithi062.stderr: 36 | container_of(ptr, type, member) 2024-06-10T00:35:21.779 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~ 2024-06-10T00:35:21.779 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-06-10T00:35:21.779 INFO:tasks.workunit.client.0.smithi062.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-06-10T00:35:21.779 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~ 2024-06-10T00:35:21.779 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-06-10T00:35:21.779 INFO:tasks.workunit.client.0.smithi062.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-06-10T00:35:21.779 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-06-10T00:35:21.779 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-06-10T00:35:21.779 INFO:tasks.workunit.client.0.smithi062.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-06-10T00:35:21.779 INFO:tasks.workunit.client.0.smithi062.stderr: | ^ 2024-06-10T00:35:21.779 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-06-10T00:35:21.780 INFO:tasks.workunit.client.0.smithi062.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-06-10T00:35:21.780 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~ 2024-06-10T00:35:21.780 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-06-10T00:35:21.780 INFO:tasks.workunit.client.0.smithi062.stderr: 36 | container_of(ptr, type, member) 2024-06-10T00:35:21.780 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~ 2024-06-10T00:35:21.780 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-06-10T00:35:21.780 INFO:tasks.workunit.client.0.smithi062.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-06-10T00:35:21.780 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~ 2024-06-10T00:35:21.780 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-06-10T00:35:21.780 INFO:tasks.workunit.client.0.smithi062.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-06-10T00:35:21.780 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-06-10T00:35:21.780 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-06-10T00:35:21.780 INFO:tasks.workunit.client.0.smithi062.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-06-10T00:35:21.780 INFO:tasks.workunit.client.0.smithi062.stderr: | ^ 2024-06-10T00:35:21.781 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-06-10T00:35:21.781 INFO:tasks.workunit.client.0.smithi062.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-06-10T00:35:21.781 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~ 2024-06-10T00:35:21.781 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-06-10T00:35:21.781 INFO:tasks.workunit.client.0.smithi062.stderr: 36 | container_of(ptr, type, member) 2024-06-10T00:35:21.781 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~ 2024-06-10T00:35:21.781 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-06-10T00:35:21.781 INFO:tasks.workunit.client.0.smithi062.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-06-10T00:35:21.781 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~ 2024-06-10T00:35:21.781 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-06-10T00:35:21.781 INFO:tasks.workunit.client.0.smithi062.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-06-10T00:35:21.781 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-06-10T00:35:21.781 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c: In function ‘init_tg_stats’: 2024-06-10T00:35:21.781 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-06-10T00:35:21.782 INFO:tasks.workunit.client.0.smithi062.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-06-10T00:35:21.782 INFO:tasks.workunit.client.0.smithi062.stderr: | ^ 2024-06-10T00:35:21.782 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-06-10T00:35:21.782 INFO:tasks.workunit.client.0.smithi062.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-06-10T00:35:21.782 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~ 2024-06-10T00:35:21.782 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-06-10T00:35:21.782 INFO:tasks.workunit.client.0.smithi062.stderr: 36 | container_of(ptr, type, member) 2024-06-10T00:35:21.782 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~ 2024-06-10T00:35:21.782 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-06-10T00:35:21.782 INFO:tasks.workunit.client.0.smithi062.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-06-10T00:35:21.782 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~ 2024-06-10T00:35:21.782 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-06-10T00:35:21.782 INFO:tasks.workunit.client.0.smithi062.stderr: 971 | list_for_each_entry(tmp_list, 2024-06-10T00:35:21.783 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-06-10T00:35:21.783 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-06-10T00:35:21.783 INFO:tasks.workunit.client.0.smithi062.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-06-10T00:35:21.783 INFO:tasks.workunit.client.0.smithi062.stderr: | ^ 2024-06-10T00:35:21.783 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-06-10T00:35:21.783 INFO:tasks.workunit.client.0.smithi062.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-06-10T00:35:21.783 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~ 2024-06-10T00:35:21.783 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-06-10T00:35:21.783 INFO:tasks.workunit.client.0.smithi062.stderr: 36 | container_of(ptr, type, member) 2024-06-10T00:35:21.783 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~ 2024-06-10T00:35:21.783 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-06-10T00:35:21.783 INFO:tasks.workunit.client.0.smithi062.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-06-10T00:35:21.783 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~ 2024-06-10T00:35:21.784 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-06-10T00:35:21.784 INFO:tasks.workunit.client.0.smithi062.stderr: 971 | list_for_each_entry(tmp_list, 2024-06-10T00:35:21.784 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-06-10T00:35:21.784 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-06-10T00:35:21.784 INFO:tasks.workunit.client.0.smithi062.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-06-10T00:35:21.784 INFO:tasks.workunit.client.0.smithi062.stderr: | ^ 2024-06-10T00:35:21.784 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-06-10T00:35:21.784 INFO:tasks.workunit.client.0.smithi062.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-06-10T00:35:21.784 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~ 2024-06-10T00:35:21.784 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-06-10T00:35:21.784 INFO:tasks.workunit.client.0.smithi062.stderr: 36 | container_of(ptr, type, member) 2024-06-10T00:35:21.784 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~ 2024-06-10T00:35:21.784 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-06-10T00:35:21.784 INFO:tasks.workunit.client.0.smithi062.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-06-10T00:35:21.785 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~ 2024-06-10T00:35:21.785 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-06-10T00:35:21.785 INFO:tasks.workunit.client.0.smithi062.stderr: 980 | list_for_each_entry(tmp_list, 2024-06-10T00:35:21.785 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-06-10T00:35:21.785 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-06-10T00:35:21.785 INFO:tasks.workunit.client.0.smithi062.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-06-10T00:35:21.785 INFO:tasks.workunit.client.0.smithi062.stderr: | ^ 2024-06-10T00:35:21.785 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-06-10T00:35:21.785 INFO:tasks.workunit.client.0.smithi062.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-06-10T00:35:21.785 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~ 2024-06-10T00:35:21.785 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-06-10T00:35:21.785 INFO:tasks.workunit.client.0.smithi062.stderr: 36 | container_of(ptr, type, member) 2024-06-10T00:35:21.785 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~ 2024-06-10T00:35:21.786 INFO:tasks.workunit.client.0.smithi062.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-06-10T00:35:21.786 INFO:tasks.workunit.client.0.smithi062.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-06-10T00:35:21.786 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~ 2024-06-10T00:35:21.786 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-06-10T00:35:21.786 INFO:tasks.workunit.client.0.smithi062.stderr: 980 | list_for_each_entry(tmp_list, 2024-06-10T00:35:21.786 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-06-10T00:35:21.786 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:21 smithi105 ceph-mon[36321]: pgmap v78: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-06-10T00:35:21.842 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c: In function ‘get_optstr’: 2024-06-10T00:35:21.842 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-06-10T00:35:21.842 INFO:tasks.workunit.client.0.smithi062.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-06-10T00:35:21.842 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-06-10T00:35:21.842 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:200:14: note: source object declared here 2024-06-10T00:35:21.842 INFO:tasks.workunit.client.0.smithi062.stderr: 200 | char temp[BUFSIZE]; 2024-06-10T00:35:21.842 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~ 2024-06-10T00:35:21.849 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c: In function ‘size64_convert’: 2024-06-10T00:35:21.849 INFO:tasks.workunit.client.0.smithi062.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-06-10T00:35:21.849 INFO:tasks.workunit.client.0.smithi062.stderr: 119 | memcpy(unit, "\0", 3); 2024-06-10T00:35:21.850 INFO:tasks.workunit.client.0.smithi062.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-06-10T00:35:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:21 smithi138 ceph-mon[36549]: pgmap v78: 129 pgs: 129 active+clean; 8.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-06-10T00:35:22.159 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-06-10T00:35:22.163 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-06-10T00:35:22.220 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-06-10T00:35:22.224 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-06-10T00:35:22.314 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-06-10T00:35:22.318 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-06-10T00:35:22.349 INFO:tasks.workunit.client.0.smithi062.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-06-10T00:35:22.352 INFO:tasks.workunit.client.0.smithi062.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-06-10T00:35:22.384 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-06-10T00:35:22.385 INFO:tasks.workunit.client.0.smithi062.stderr:+ cd .. 2024-06-10T00:35:22.385 INFO:tasks.workunit.client.0.smithi062.stderr:+ mkdir tmp 2024-06-10T00:35:22.390 INFO:tasks.workunit.client.0.smithi062.stderr:+ cd tmp 2024-06-10T00:35:22.391 INFO:tasks.workunit.client.0.smithi062.stderr:+ for f in $mydir/*.ffsb 2024-06-10T00:35:22.391 INFO:tasks.workunit.client.0.smithi062.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-06-10T00:35:22.392 INFO:tasks.workunit.client.0.smithi062.stdout:FFSB version 6.0-RC2 started 2024-06-10T00:35:22.392 INFO:tasks.workunit.client.0.smithi062.stdout: 2024-06-10T00:35:22.392 INFO:tasks.workunit.client.0.smithi062.stdout:benchmark time = 300 2024-06-10T00:35:22.392 INFO:tasks.workunit.client.0.smithi062.stdout:ThreadGroup 0 2024-06-10T00:35:22.392 INFO:tasks.workunit.client.0.smithi062.stdout:================ 2024-06-10T00:35:22.392 INFO:tasks.workunit.client.0.smithi062.stdout: num_threads = 32 2024-06-10T00:35:22.392 INFO:tasks.workunit.client.0.smithi062.stdout: 2024-06-10T00:35:22.392 INFO:tasks.workunit.client.0.smithi062.stdout: read_random = off 2024-06-10T00:35:22.392 INFO:tasks.workunit.client.0.smithi062.stdout: read_size = 0 (0B) 2024-06-10T00:35:22.392 INFO:tasks.workunit.client.0.smithi062.stdout: read_blocksize = 0 (0B) 2024-06-10T00:35:22.392 INFO:tasks.workunit.client.0.smithi062.stdout: read_skip = off 2024-06-10T00:35:22.393 INFO:tasks.workunit.client.0.smithi062.stdout: read_skipsize = 0 (0B) 2024-06-10T00:35:22.393 INFO:tasks.workunit.client.0.smithi062.stdout: 2024-06-10T00:35:22.393 INFO:tasks.workunit.client.0.smithi062.stdout: write_random = on 2024-06-10T00:35:22.393 INFO:tasks.workunit.client.0.smithi062.stdout: write_size = 5242880 (5MB) 2024-06-10T00:35:22.393 INFO:tasks.workunit.client.0.smithi062.stdout: fsync_file = 0 2024-06-10T00:35:22.393 INFO:tasks.workunit.client.0.smithi062.stdout: write_blocksize = 4096 (4KB) 2024-06-10T00:35:22.393 INFO:tasks.workunit.client.0.smithi062.stdout: wait time = 0 2024-06-10T00:35:22.393 INFO:tasks.workunit.client.0.smithi062.stdout: 2024-06-10T00:35:22.393 INFO:tasks.workunit.client.0.smithi062.stdout: op weights 2024-06-10T00:35:22.393 INFO:tasks.workunit.client.0.smithi062.stdout: read = 0 (0.00%) 2024-06-10T00:35:22.393 INFO:tasks.workunit.client.0.smithi062.stdout: readall = 0 (0.00%) 2024-06-10T00:35:22.393 INFO:tasks.workunit.client.0.smithi062.stdout: write = 1 (100.00%) 2024-06-10T00:35:22.393 INFO:tasks.workunit.client.0.smithi062.stdout: create = 0 (0.00%) 2024-06-10T00:35:22.394 INFO:tasks.workunit.client.0.smithi062.stdout: append = 0 (0.00%) 2024-06-10T00:35:22.394 INFO:tasks.workunit.client.0.smithi062.stdout: delete = 0 (0.00%) 2024-06-10T00:35:22.394 INFO:tasks.workunit.client.0.smithi062.stdout: metaop = 0 (0.00%) 2024-06-10T00:35:22.394 INFO:tasks.workunit.client.0.smithi062.stdout: createdir = 0 (0.00%) 2024-06-10T00:35:22.394 INFO:tasks.workunit.client.0.smithi062.stdout: stat = 0 (0.00%) 2024-06-10T00:35:22.394 INFO:tasks.workunit.client.0.smithi062.stdout: writeall = 0 (0.00%) 2024-06-10T00:35:22.394 INFO:tasks.workunit.client.0.smithi062.stdout: writeall_fsync = 0 (0.00%) 2024-06-10T00:35:22.394 INFO:tasks.workunit.client.0.smithi062.stdout: open_close = 0 (0.00%) 2024-06-10T00:35:22.394 INFO:tasks.workunit.client.0.smithi062.stdout: write_fsync = 0 (0.00%) 2024-06-10T00:35:22.394 INFO:tasks.workunit.client.0.smithi062.stdout: create_fsync = 0 (0.00%) 2024-06-10T00:35:22.394 INFO:tasks.workunit.client.0.smithi062.stdout: append_fsync = 0 (0.00%) 2024-06-10T00:35:22.394 INFO:tasks.workunit.client.0.smithi062.stdout: 2024-06-10T00:35:22.394 INFO:tasks.workunit.client.0.smithi062.stdout:FileSystem . 2024-06-10T00:35:22.395 INFO:tasks.workunit.client.0.smithi062.stdout:========== 2024-06-10T00:35:22.395 INFO:tasks.workunit.client.0.smithi062.stdout: num_dirs = 0 2024-06-10T00:35:22.395 INFO:tasks.workunit.client.0.smithi062.stdout: starting files = 128 2024-06-10T00:35:22.395 INFO:tasks.workunit.client.0.smithi062.stdout: 2024-06-10T00:35:22.395 INFO:tasks.workunit.client.0.smithi062.stdout: min file size = 104857600 (100MB) 2024-06-10T00:35:22.395 INFO:tasks.workunit.client.0.smithi062.stdout: max file size = 104857600 (100MB) 2024-06-10T00:35:22.395 INFO:tasks.workunit.client.0.smithi062.stdout: directio = off 2024-06-10T00:35:22.395 INFO:tasks.workunit.client.0.smithi062.stdout: alignedio = on 2024-06-10T00:35:22.395 INFO:tasks.workunit.client.0.smithi062.stdout: bufferedio = off 2024-06-10T00:35:22.395 INFO:tasks.workunit.client.0.smithi062.stdout: 2024-06-10T00:35:22.395 INFO:tasks.workunit.client.0.smithi062.stdout: aging is off 2024-06-10T00:35:22.395 INFO:tasks.workunit.client.0.smithi062.stdout: current utilization = 0.04% 2024-06-10T00:35:22.395 INFO:tasks.workunit.client.0.smithi062.stdout: 2024-06-10T00:35:22.396 INFO:tasks.workunit.client.0.smithi062.stdout:checking existing fs: . 2024-06-10T00:35:22.396 INFO:tasks.workunit.client.0.smithi062.stdout:recreating new fileset 2024-06-10T00:35:22.396 INFO:tasks.workunit.client.0.smithi062.stderr:opendir: No such file or directory 2024-06-10T00:35:23.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.137+0000 7ff1664e7700 1 -- 172.21.15.62:0/2011954290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1601073f0 msgr2=0x7ff1601077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:23.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.137+0000 7ff1664e7700 1 --2- 172.21.15.62:0/2011954290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1601073f0 0x7ff1601077d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff150009990 tx=0x7ff150009ca0 comp rx=0 tx=0).stop 2024-06-10T00:35:23.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.137+0000 7ff1664e7700 1 -- 172.21.15.62:0/2011954290 shutdown_connections 2024-06-10T00:35:23.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.137+0000 7ff1664e7700 1 --2- 172.21.15.62:0/2011954290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1601073f0 0x7ff1601077d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:23.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.137+0000 7ff1664e7700 1 --2- 172.21.15.62:0/2011954290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff160101d40 0x7ff16010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:23.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.137+0000 7ff1664e7700 1 --2- 172.21.15.62:0/2011954290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff160101380 0x7ff160101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:23.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.137+0000 7ff1664e7700 1 -- 172.21.15.62:0/2011954290 >> 172.21.15.62:0/2011954290 conn(0x7ff160078300 msgr2=0x7ff160078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:23.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.138+0000 7ff1664e7700 1 -- 172.21.15.62:0/2011954290 shutdown_connections 2024-06-10T00:35:23.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.138+0000 7ff1664e7700 1 -- 172.21.15.62:0/2011954290 wait complete. 2024-06-10T00:35:23.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.139+0000 7ff1664e7700 1 Processor -- start 2024-06-10T00:35:23.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.139+0000 7ff1664e7700 1 -- start start 2024-06-10T00:35:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.139+0000 7ff1664e7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff160101380 0x7ff16019d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.139+0000 7ff1664e7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff160101d40 0x7ff16019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.139+0000 7ff1664e7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1601073f0 0x7ff160197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.139+0000 7ff1664e7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff160111660 con 0x7ff160101380 2024-06-10T00:35:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.139+0000 7ff1664e7700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff1601114e0 con 0x7ff1601073f0 2024-06-10T00:35:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.139+0000 7ff1664e7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1601117e0 con 0x7ff160101d40 2024-06-10T00:35:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.139+0000 7ff15ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff160101380 0x7ff16019d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.139+0000 7ff15ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff160101380 0x7ff16019d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56826/0 (socket says 172.21.15.62:56826) 2024-06-10T00:35:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.139+0000 7ff15ffff700 1 -- 172.21.15.62:0/4254328532 learned_addr learned my addr 172.21.15.62:0/4254328532 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.139+0000 7ff15f7fe700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff160101d40 0x7ff16019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.139+0000 7ff164a84700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1601073f0 0x7ff160197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.140+0000 7ff15ffff700 1 -- 172.21.15.62:0/4254328532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff160101d40 msgr2=0x7ff16019dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.140+0000 7ff15ffff700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff160101d40 0x7ff16019dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:23.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.140+0000 7ff15ffff700 1 -- 172.21.15.62:0/4254328532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1601073f0 msgr2=0x7ff160197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:23.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.140+0000 7ff15ffff700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1601073f0 0x7ff160197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:23.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.140+0000 7ff15ffff700 1 -- 172.21.15.62:0/4254328532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff160198080 con 0x7ff160101380 2024-06-10T00:35:23.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.140+0000 7ff15f7fe700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff160101d40 0x7ff16019dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:35:23.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.140+0000 7ff164a84700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1601073f0 0x7ff160197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:35:23.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.140+0000 7ff15ffff700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff160101380 0x7ff16019d760 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff15000bae0 tx=0x7ff15000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:23.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.140+0000 7ff15d7fa700 1 -- 172.21.15.62:0/4254328532 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150028070 con 0x7ff160101380 2024-06-10T00:35:23.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.140+0000 7ff1664e7700 1 -- 172.21.15.62:0/4254328532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff160198310 con 0x7ff160101380 2024-06-10T00:35:23.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.140+0000 7ff15d7fa700 1 -- 172.21.15.62:0/4254328532 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff150003d70 con 0x7ff160101380 2024-06-10T00:35:23.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.140+0000 7ff15d7fa700 1 -- 172.21.15.62:0/4254328532 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15000f6c0 con 0x7ff160101380 2024-06-10T00:35:23.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.140+0000 7ff1664e7700 1 -- 172.21.15.62:0/4254328532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16006aa00 con 0x7ff160101380 2024-06-10T00:35:23.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.141+0000 7ff1664e7700 1 -- 172.21.15.62:0/4254328532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff160069180 con 0x7ff160101380 2024-06-10T00:35:23.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.142+0000 7ff15d7fa700 1 -- 172.21.15.62:0/4254328532 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff15002b070 con 0x7ff160101380 2024-06-10T00:35:23.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.143+0000 7ff15d7fa700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff1480a71a0 0x7ff1480a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:23.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.143+0000 7ff15d7fa700 1 -- 172.21.15.62:0/4254328532 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7ff1500d02d0 con 0x7ff160101380 2024-06-10T00:35:23.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.143+0000 7ff15f7fe700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff1480a71a0 0x7ff1480a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:23.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.144+0000 7ff15f7fe700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff1480a71a0 0x7ff1480a9660 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff1600789e0 tx=0x7ff154009260 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:23.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.148+0000 7ff15d7fa700 1 -- 172.21.15.62:0/4254328532 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff150013070 con 0x7ff160101380 2024-06-10T00:35:23.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.407+0000 7ff1664e7700 1 -- 172.21.15.62:0/4254328532 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff16006b430 con 0x7ff160101380 2024-06-10T00:35:23.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.407+0000 7ff15d7fa700 1 -- 172.21.15.62:0/4254328532 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7ff15009cbd0 con 0x7ff160101380 2024-06-10T00:35:23.408 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:35:23.408 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:32:47.783154+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:35:23.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.410+0000 7ff1664e7700 1 -- 172.21.15.62:0/4254328532 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff1480a71a0 msgr2=0x7ff1480a9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:23.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.410+0000 7ff1664e7700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff1480a71a0 0x7ff1480a9660 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff1600789e0 tx=0x7ff154009260 comp rx=0 tx=0).stop 2024-06-10T00:35:23.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.410+0000 7ff1664e7700 1 -- 172.21.15.62:0/4254328532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff160101380 msgr2=0x7ff16019d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:23.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.410+0000 7ff1664e7700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff160101380 0x7ff16019d760 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff15000bae0 tx=0x7ff15000bb10 comp rx=0 tx=0).stop 2024-06-10T00:35:23.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.410+0000 7ff1664e7700 1 -- 172.21.15.62:0/4254328532 shutdown_connections 2024-06-10T00:35:23.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.410+0000 7ff1664e7700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff1480a71a0 0x7ff1480a9660 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:23.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.410+0000 7ff1664e7700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff160101380 0x7ff16019d760 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:23.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.410+0000 7ff1664e7700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1601073f0 0x7ff160197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:23.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.410+0000 7ff1664e7700 1 --2- 172.21.15.62:0/4254328532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff160101d40 0x7ff16019dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:23.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.410+0000 7ff1664e7700 1 -- 172.21.15.62:0/4254328532 >> 172.21.15.62:0/4254328532 conn(0x7ff160078300 msgr2=0x7ff160100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:23.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.410+0000 7ff1664e7700 1 -- 172.21.15.62:0/4254328532 shutdown_connections 2024-06-10T00:35:23.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:23.410+0000 7ff1664e7700 1 -- 172.21.15.62:0/4254328532 wait complete. 2024-06-10T00:35:23.415 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-06-10T00:35:23.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:23 smithi105 ceph-mon[36321]: pgmap v79: 129 pgs: 129 active+clean; 8.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-06-10T00:35:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:23 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/4254328532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:35:23.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:23 smithi062 ceph-mon[32390]: pgmap v79: 129 pgs: 129 active+clean; 8.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-06-10T00:35:23.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:23 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/4254328532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:35:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:23 smithi138 ceph-mon[36549]: pgmap v79: 129 pgs: 129 active+clean; 8.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-06-10T00:35:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:23 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/4254328532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:35:24.489 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T00:35:24.877 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:35:25.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:25 smithi062 ceph-mon[32390]: pgmap v80: 129 pgs: 129 active+clean; 8.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-06-10T00:35:25.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:25 smithi105 ceph-mon[36321]: pgmap v80: 129 pgs: 129 active+clean; 8.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-06-10T00:35:25.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:25 smithi138 ceph-mon[36549]: pgmap v80: 129 pgs: 129 active+clean; 8.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-06-10T00:35:27.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.359+0000 7f521e80a700 1 -- 172.21.15.62:0/1103073789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5218101380 msgr2=0x7f5218101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:27.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.359+0000 7f521e80a700 1 --2- 172.21.15.62:0/1103073789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5218101380 0x7f5218101800 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f520c009990 tx=0x7f520c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.359+0000 7f521e80a700 1 -- 172.21.15.62:0/1103073789 shutdown_connections 2024-06-10T00:35:27.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.359+0000 7f521e80a700 1 --2- 172.21.15.62:0/1103073789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5218101d40 0x7f521810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.359+0000 7f521e80a700 1 --2- 172.21.15.62:0/1103073789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52181073f0 0x7f52181077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.359+0000 7f521e80a700 1 --2- 172.21.15.62:0/1103073789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5218101380 0x7f5218101800 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.359+0000 7f521e80a700 1 -- 172.21.15.62:0/1103073789 >> 172.21.15.62:0/1103073789 conn(0x7f5218078300 msgr2=0x7f5218078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:27.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.359+0000 7f521e80a700 1 -- 172.21.15.62:0/1103073789 shutdown_connections 2024-06-10T00:35:27.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.359+0000 7f521e80a700 1 -- 172.21.15.62:0/1103073789 wait complete. 2024-06-10T00:35:27.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.360+0000 7f521e80a700 1 Processor -- start 2024-06-10T00:35:27.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.360+0000 7f521e80a700 1 -- start start 2024-06-10T00:35:27.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.360+0000 7f521e80a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5218101380 0x7f521819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.360+0000 7f521e80a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218101d40 0x7f521819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.360+0000 7f521e80a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52181073f0 0x7f52181989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.360+0000 7f521e80a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5218111680 con 0x7f5218101380 2024-06-10T00:35:27.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.360+0000 7f521e80a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5218111500 con 0x7f52181073f0 2024-06-10T00:35:27.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.360+0000 7f521e80a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5218111800 con 0x7f5218101d40 2024-06-10T00:35:27.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.360+0000 7f5217fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5218101380 0x7f521819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.360+0000 7f52177fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218101d40 0x7f521819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.360+0000 7f5217fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5218101380 0x7f521819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56838/0 (socket says 172.21.15.62:56838) 2024-06-10T00:35:27.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.360+0000 7f52177fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218101d40 0x7f521819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:55696/0 (socket says 172.21.15.62:55696) 2024-06-10T00:35:27.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.360+0000 7f5217fff700 1 -- 172.21.15.62:0/4266267854 learned_addr learned my addr 172.21.15.62:0/4266267854 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:27.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.360+0000 7f521cda7700 1 --2- 172.21.15.62:0/4266267854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52181073f0 0x7f52181989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.361+0000 7f5217fff700 1 -- 172.21.15.62:0/4266267854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218101d40 msgr2=0x7f521819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:27.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.361+0000 7f5217fff700 1 --2- 172.21.15.62:0/4266267854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218101d40 0x7f521819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.361+0000 7f5217fff700 1 -- 172.21.15.62:0/4266267854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52181073f0 msgr2=0x7f52181989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:27.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.361+0000 7f5217fff700 1 --2- 172.21.15.62:0/4266267854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52181073f0 0x7f52181989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.361+0000 7f5217fff700 1 -- 172.21.15.62:0/4266267854 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5218199150 con 0x7f5218101380 2024-06-10T00:35:27.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.361+0000 7f521cda7700 1 --2- 172.21.15.62:0/4266267854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52181073f0 0x7f52181989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:35:27.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.361+0000 7f5217fff700 1 --2- 172.21.15.62:0/4266267854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5218101380 0x7f521819e830 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f520800e9c0 tx=0x7f520800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:27.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.361+0000 7f52157fa700 1 -- 172.21.15.62:0/4266267854 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520800cbd0 con 0x7f5218101380 2024-06-10T00:35:27.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.361+0000 7f52157fa700 1 -- 172.21.15.62:0/4266267854 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f520800ee60 con 0x7f5218101380 2024-06-10T00:35:27.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.361+0000 7f52157fa700 1 -- 172.21.15.62:0/4266267854 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208005630 con 0x7f5218101380 2024-06-10T00:35:27.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.361+0000 7f521e80a700 1 -- 172.21.15.62:0/4266267854 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5218199440 con 0x7f5218101380 2024-06-10T00:35:27.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.361+0000 7f521e80a700 1 -- 172.21.15.62:0/4266267854 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52181a5700 con 0x7f5218101380 2024-06-10T00:35:27.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.364+0000 7f52157fa700 1 -- 172.21.15.62:0/4266267854 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5208004750 con 0x7f5218101380 2024-06-10T00:35:27.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.365+0000 7f52157fa700 1 --2- 172.21.15.62:0/4266267854 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f52000a2d90 0x7f52000a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.365+0000 7f52177fe700 1 --2- 172.21.15.62:0/4266267854 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f52000a2d90 0x7f52000a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.365+0000 7f52157fa700 1 -- 172.21.15.62:0/4266267854 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f52080cfa50 con 0x7f5218101380 2024-06-10T00:35:27.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.365+0000 7f52177fe700 1 --2- 172.21.15.62:0/4266267854 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f52000a2d90 0x7f52000a5250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f520c00b2e0 tx=0x7f520c005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:27.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.365+0000 7f522908db80 1 Processor -- start 2024-06-10T00:35:27.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f522908db80 1 -- start start 2024-06-10T00:35:27.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f522908db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a316340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f522908db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a316880 0x55f63a310260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f522908db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a3107a0 0x55f63a310bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f522908db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f63a202f60 con 0x55f63a316880 2024-06-10T00:35:27.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f522908db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f63a261850 con 0x55f63a315f60 2024-06-10T00:35:27.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f522908db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f63a1c9460 con 0x55f63a3107a0 2024-06-10T00:35:27.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f5217fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a316340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f52177fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a316880 0x55f63a310260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f5217fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a316340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:49620/0 (socket says 172.21.15.62:49620) 2024-06-10T00:35:27.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f5217fff700 1 -- 172.21.15.62:0/3507840057 learned_addr learned my addr 172.21.15.62:0/3507840057 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:27.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f521cda7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a3107a0 0x55f63a310bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f5217fff700 1 -- 172.21.15.62:0/3507840057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a3107a0 msgr2=0x55f63a310bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:27.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f5217fff700 1 --2- 172.21.15.62:0/3507840057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a3107a0 0x55f63a310bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f5217fff700 1 -- 172.21.15.62:0/3507840057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a316880 msgr2=0x55f63a310260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:27.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f5217fff700 1 --2- 172.21.15.62:0/3507840057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a316880 0x55f63a310260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.366+0000 7f5217fff700 1 -- 172.21.15.62:0/3507840057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f63a1b08d0 con 0x55f63a315f60 2024-06-10T00:35:27.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.367+0000 7f5217fff700 1 --2- 172.21.15.62:0/3507840057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a316340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5208098530 tx=0x7f520809b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2953e275df4a03b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:27.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.367+0000 7f51fe7fc700 1 -- 172.21.15.62:0/3507840057 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52080a47d0 con 0x55f63a315f60 2024-06-10T00:35:27.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.367+0000 7f51fe7fc700 1 -- 172.21.15.62:0/3507840057 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52080b3020 con 0x55f63a315f60 2024-06-10T00:35:27.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.367+0000 7f51fe7fc700 1 -- 172.21.15.62:0/3507840057 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520809c050 con 0x55f63a315f60 2024-06-10T00:35:27.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.368+0000 7f522908db80 1 -- 172.21.15.62:0/3507840057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 msgr2=0x55f63a316340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:27.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.368+0000 7f522908db80 1 --2- 172.21.15.62:0/3507840057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a316340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5208098530 tx=0x7f520809b7b0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.368+0000 7f522908db80 1 -- 172.21.15.62:0/3507840057 shutdown_connections 2024-06-10T00:35:27.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.368+0000 7f522908db80 1 --2- 172.21.15.62:0/3507840057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a3107a0 0x55f63a310bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.368+0000 7f522908db80 1 --2- 172.21.15.62:0/3507840057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a316340 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.368+0000 7f522908db80 1 --2- 172.21.15.62:0/3507840057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a316880 0x55f63a310260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.368+0000 7f522908db80 1 -- 172.21.15.62:0/3507840057 >> 172.21.15.62:0/3507840057 conn(0x55f63a23d080 msgr2=0x55f63a30b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:27.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.368+0000 7f522908db80 1 -- 172.21.15.62:0/3507840057 shutdown_connections 2024-06-10T00:35:27.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.368+0000 7f522908db80 1 -- 172.21.15.62:0/3507840057 wait complete. 2024-06-10T00:35:27.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.368+0000 7f522908db80 1 -- start start 2024-06-10T00:35:27.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.369+0000 7f522908db80 1 Processor -- start 2024-06-10T00:35:27.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.369+0000 7f522908db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a3107a0 0x55f63a31c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.369+0000 7f522908db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a315f60 0x55f63a319ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.369+0000 7f522908db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a316880 0x55f63a31a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.369+0000 7f522908db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f63a201fc0 con 0x55f63a3107a0 2024-06-10T00:35:27.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.369+0000 7f522908db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f63a25d810 con 0x55f63a316880 2024-06-10T00:35:27.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.369+0000 7f522908db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f63a321480 con 0x55f63a315f60 2024-06-10T00:35:27.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.369+0000 7f5217fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a3107a0 0x55f63a31c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.369+0000 7f5217fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a3107a0 0x55f63a31c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56858/0 (socket says 172.21.15.62:56858) 2024-06-10T00:35:27.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.369+0000 7f5217fff700 1 -- 172.21.15.62:0/2553124990 learned_addr learned my addr 172.21.15.62:0/2553124990 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:27.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.369+0000 7f522908db80 10 client.0 _ll_register_callbacks cb 0x55f63a1f75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:27.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.369+0000 7f52177fe700 1 --2- 172.21.15.62:0/2553124990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a315f60 0x55f63a319ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.370+0000 7f521cda7700 1 --2- 172.21.15.62:0/2553124990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a316880 0x55f63a31a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.370+0000 7f5217fff700 1 -- 172.21.15.62:0/2553124990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a315f60 msgr2=0x55f63a319ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:27.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.370+0000 7f5217fff700 1 --2- 172.21.15.62:0/2553124990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a315f60 0x55f63a319ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.370+0000 7f5217fff700 1 -- 172.21.15.62:0/2553124990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a316880 msgr2=0x55f63a31a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:27.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.370+0000 7f5217fff700 1 --2- 172.21.15.62:0/2553124990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a316880 0x55f63a31a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.370+0000 7f5217fff700 1 -- 172.21.15.62:0/2553124990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f63a227750 con 0x55f63a3107a0 2024-06-10T00:35:27.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.370+0000 7f521cda7700 1 --2- 172.21.15.62:0/2553124990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a316880 0x55f63a31a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:35:27.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.370+0000 7f5217fff700 1 --2- 172.21.15.62:0/2553124990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a3107a0 0x55f63a31c7c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f52080a42a0 tx=0x7f52080097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8248a0dcd3890a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:27.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.370+0000 7f51f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:27.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.370+0000 7f51f77fe700 1 -- 172.21.15.62:0/2553124990 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208065020 con 0x55f63a3107a0 2024-06-10T00:35:27.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.370+0000 7f51f77fe700 1 -- 172.21.15.62:0/2553124990 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f520800cd70 con 0x55f63a3107a0 2024-06-10T00:35:27.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.370+0000 7f522908db80 1 -- 172.21.15.62:0/2553124990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52080b3020 con 0x55f63a3107a0 2024-06-10T00:35:27.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.371+0000 7f51f77fe700 1 -- 172.21.15.62:0/2553124990 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f52080ba050 con 0x55f63a3107a0 2024-06-10T00:35:27.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.371+0000 7f522908db80 10 client.16179 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:27.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.371+0000 7f522908db80 1 -- 172.21.15.62:0/2553124990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f63a32bdf0 con 0x55f63a3107a0 2024-06-10T00:35:27.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.371+0000 7f51f77fe700 1 -- 172.21.15.62:0/2553124990 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52080b3020 con 0x55f63a3107a0 2024-06-10T00:35:27.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.371+0000 7f51f77fe700 1 -- 172.21.15.62:0/2553124990 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f52080978b0 con 0x55f63a3107a0 2024-06-10T00:35:27.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.371+0000 7f522908db80 10 client.16179 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:27.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.371+0000 7f522908db80 10 client.16179 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:35:27.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.371+0000 7f522908db80 20 client.16179 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:27.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.371+0000 7f522908db80 1 --2- 172.21.15.62:0/2553124990 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f63a32e2c0 0x55f63a3306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.371+0000 7f522908db80 4 client.16179 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:35:27.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.371+0000 7f522908db80 1 -- 172.21.15.62:0/2553124990 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f63a26ed50 con 0x55f63a32e2c0 2024-06-10T00:35:27.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.371+0000 7f52177fe700 1 --2- 172.21.15.62:0/2553124990 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f63a32e2c0 0x55f63a3306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.372+0000 7f51f77fe700 10 client.16179 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:35:27.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.372+0000 7f52177fe700 1 --2- 172.21.15.62:0/2553124990 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f63a32e2c0 0x55f63a3306b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d555c0368c42c51 server_cookie=694b8c5818ac1fe in_seq=0 out_seq=0 2024-06-10T00:35:27.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.372+0000 7f51f77fe700 1 -- 172.21.15.62:0/2553124990 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f63a26ed50 con 0x55f63a32e2c0 2024-06-10T00:35:27.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.372+0000 7f51f77fe700 10 client.16179 handle_command_reply: tid=0 2024-06-10T00:35:27.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.372+0000 7f522908db80 1 client.16179 shutdown 2024-06-10T00:35:27.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.373+0000 7f522908db80 1 -- 172.21.15.62:0/2553124990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a3107a0 msgr2=0x55f63a31c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:27.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.373+0000 7f522908db80 1 --2- 172.21.15.62:0/2553124990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a3107a0 0x55f63a31c7c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f52080a42a0 tx=0x7f52080097b0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.373+0000 7f522908db80 1 -- 172.21.15.62:0/2553124990 shutdown_connections 2024-06-10T00:35:27.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.373+0000 7f522908db80 1 --2- 172.21.15.62:0/2553124990 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f63a32e2c0 0x55f63a3306b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.373+0000 7f522908db80 1 --2- 172.21.15.62:0/2553124990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a3107a0 0x55f63a31c7c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.373+0000 7f522908db80 1 --2- 172.21.15.62:0/2553124990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a316880 0x55f63a31a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.373+0000 7f51f77fe700 0 client.16179 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:35:27.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.373+0000 7f522908db80 1 --2- 172.21.15.62:0/2553124990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a315f60 0x55f63a319ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.373+0000 7f522908db80 1 -- 172.21.15.62:0/2553124990 >> 172.21.15.62:0/2553124990 conn(0x55f63a23d080 msgr2=0x55f63a30a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:27.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.373+0000 7f522908db80 1 -- 172.21.15.62:0/2553124990 shutdown_connections 2024-06-10T00:35:27.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.373+0000 7f522908db80 1 -- 172.21.15.62:0/2553124990 wait complete. 2024-06-10T00:35:27.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.373+0000 7f522908db80 20 client.16179 trim_cache size 0 max 16384 2024-06-10T00:35:27.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.404+0000 7f522908db80 1 Processor -- start 2024-06-10T00:35:27.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.405+0000 7f522908db80 1 -- start start 2024-06-10T00:35:27.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.405+0000 7f522908db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a310260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.405+0000 7f522908db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a316880 0x55f63a3154b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.405+0000 7f522908db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a32a7f0 0x55f63a30ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.405+0000 7f522908db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f52181ac330 con 0x55f63a32a7f0 2024-06-10T00:35:27.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.405+0000 7f522908db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f52181ade70 con 0x55f63a315f60 2024-06-10T00:35:27.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.405+0000 7f522908db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f52181aca00 con 0x55f63a316880 2024-06-10T00:35:27.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.405+0000 7f52177fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a316880 0x55f63a3154b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.405+0000 7f52177fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a316880 0x55f63a3154b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:55718/0 (socket says 172.21.15.62:55718) 2024-06-10T00:35:27.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.405+0000 7f52177fe700 1 -- 172.21.15.62:0/251444061 learned_addr learned my addr 172.21.15.62:0/251444061 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:27.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.406+0000 7f52177fe700 1 -- 172.21.15.62:0/251444061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 msgr2=0x55f63a310260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:27.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.406+0000 7f52177fe700 1 --2- 172.21.15.62:0/251444061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a310260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.406+0000 7f52177fe700 1 -- 172.21.15.62:0/251444061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a32a7f0 msgr2=0x55f63a30ae80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T00:35:27.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.406+0000 7f52177fe700 1 --2- 172.21.15.62:0/251444061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a32a7f0 0x55f63a30ae80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.406+0000 7f52177fe700 1 -- 172.21.15.62:0/251444061 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f63a318030 con 0x55f63a316880 2024-06-10T00:35:27.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.406+0000 7f52177fe700 1 --2- 172.21.15.62:0/251444061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a316880 0x55f63a3154b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f520c009990 tx=0x7f520c021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86a26306839f4906 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:27.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.407+0000 7f51fdffb700 1 -- 172.21.15.62:0/251444061 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c003a40 con 0x55f63a316880 2024-06-10T00:35:27.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.407+0000 7f51fdffb700 1 -- 172.21.15.62:0/251444061 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f520c003be0 con 0x55f63a316880 2024-06-10T00:35:27.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.408+0000 7f51fdffb700 1 -- 172.21.15.62:0/251444061 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c022670 con 0x55f63a316880 2024-06-10T00:35:27.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.408+0000 7f522908db80 1 -- 172.21.15.62:0/251444061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a316880 msgr2=0x55f63a3154b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:27.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.408+0000 7f522908db80 1 --2- 172.21.15.62:0/251444061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a316880 0x55f63a3154b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f520c009990 tx=0x7f520c021000 comp rx=0 tx=0).stop 2024-06-10T00:35:27.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.408+0000 7f522908db80 1 -- 172.21.15.62:0/251444061 shutdown_connections 2024-06-10T00:35:27.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.408+0000 7f522908db80 1 --2- 172.21.15.62:0/251444061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a32a7f0 0x55f63a30ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.408+0000 7f522908db80 1 --2- 172.21.15.62:0/251444061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a310260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.408+0000 7f522908db80 1 --2- 172.21.15.62:0/251444061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a316880 0x55f63a3154b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.408+0000 7f522908db80 1 -- 172.21.15.62:0/251444061 >> 172.21.15.62:0/251444061 conn(0x55f63a23d080 msgr2=0x55f63a314050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:27.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.412+0000 7f522908db80 1 -- 172.21.15.62:0/251444061 shutdown_connections 2024-06-10T00:35:27.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.412+0000 7f522908db80 1 -- 172.21.15.62:0/251444061 wait complete. 2024-06-10T00:35:27.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.412+0000 7f522908db80 1 -- start start 2024-06-10T00:35:27.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.413+0000 7f522908db80 1 Processor -- start 2024-06-10T00:35:27.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.413+0000 7f522908db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a333ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.413+0000 7f522908db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a316880 0x55f63a334010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.413+0000 7f522908db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a32a7f0 0x55f63a320060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.413+0000 7f522908db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f52181ad7a0 con 0x55f63a32a7f0 2024-06-10T00:35:27.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.413+0000 7f522908db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f52181ad0d0 con 0x55f63a315f60 2024-06-10T00:35:27.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.413+0000 7f522908db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f52181aa2d0 con 0x55f63a316880 2024-06-10T00:35:27.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.413+0000 7f522908db80 10 client.0 _ll_register_callbacks cb 0x55f63a1f75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:27.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.413+0000 7f5217fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a333ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.413+0000 7f5217fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a333ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:49628/0 (socket says 172.21.15.62:49628) 2024-06-10T00:35:27.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.413+0000 7f5217fff700 1 -- 172.21.15.62:0/461111869 learned_addr learned my addr 172.21.15.62:0/461111869 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:27.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.414+0000 7f52177fe700 1 --2- 172.21.15.62:0/461111869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a316880 0x55f63a334010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.414+0000 7f5217fff700 1 -- 172.21.15.62:0/461111869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a316880 msgr2=0x55f63a334010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:27.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.414+0000 7f5217fff700 1 --2- 172.21.15.62:0/461111869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a316880 0x55f63a334010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.414+0000 7f5217fff700 1 -- 172.21.15.62:0/461111869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a32a7f0 msgr2=0x55f63a320060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:27.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.414+0000 7f5217fff700 1 --2- 172.21.15.62:0/461111869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a32a7f0 0x55f63a320060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.414+0000 7f5217fff700 1 -- 172.21.15.62:0/461111869 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f63a3208a0 con 0x55f63a315f60 2024-06-10T00:35:27.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.414+0000 7f52177fe700 1 --2- 172.21.15.62:0/461111869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a316880 0x55f63a334010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:35:27.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.414+0000 7f5217fff700 1 --2- 172.21.15.62:0/461111869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a333ad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5208097fc0 tx=0x7f52080097b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb29a1b0e7e0736 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:27.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.415+0000 7f51f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:35:27.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.416+0000 7f51f77fe700 1 -- 172.21.15.62:0/461111869 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208065070 con 0x55f63a315f60 2024-06-10T00:35:27.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.416+0000 7f51f77fe700 1 -- 172.21.15.62:0/461111869 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f52080ba1f0 con 0x55f63a315f60 2024-06-10T00:35:27.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.416+0000 7f522908db80 1 -- 172.21.15.62:0/461111869 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f520c003be0 con 0x55f63a315f60 2024-06-10T00:35:27.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.416+0000 7f51f77fe700 1 -- 172.21.15.62:0/461111869 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f52080a9050 con 0x55f63a315f60 2024-06-10T00:35:27.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.416+0000 7f522908db80 10 client.25697 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:27.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.416+0000 7f522908db80 1 -- 172.21.15.62:0/461111869 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f63a3208a0 con 0x55f63a315f60 2024-06-10T00:35:27.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.417+0000 7f51f77fe700 1 -- 172.21.15.62:0/461111869 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f520c003be0 con 0x55f63a315f60 2024-06-10T00:35:27.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.417+0000 7f51f77fe700 1 -- 172.21.15.62:0/461111869 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f520809aa20 con 0x55f63a315f60 2024-06-10T00:35:27.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.417+0000 7f522908db80 10 client.25697 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:27.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.417+0000 7f522908db80 10 client.25697 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:35:27.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.417+0000 7f522908db80 20 client.25697 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:27.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.417+0000 7f522908db80 1 --2- 172.21.15.62:0/461111869 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f63a39d3e0 0x55f63a39f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:27.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.417+0000 7f522908db80 4 client.25697 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:35:27.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.417+0000 7f522908db80 1 -- 172.21.15.62:0/461111869 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f63a2692d0 con 0x55f63a39d3e0 2024-06-10T00:35:27.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.418+0000 7f52177fe700 1 --2- 172.21.15.62:0/461111869 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f63a39d3e0 0x55f63a39f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:27.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.420+0000 7f52177fe700 1 --2- 172.21.15.62:0/461111869 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f63a39d3e0 0x55f63a39f7d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2c11a7b945eed6fa server_cookie=6102ef100b2ff661 in_seq=0 out_seq=0 2024-06-10T00:35:27.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.420+0000 7f51f77fe700 10 client.25697 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:35:27.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.420+0000 7f51f77fe700 1 -- 172.21.15.62:0/461111869 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f63a2692d0 con 0x55f63a39d3e0 2024-06-10T00:35:27.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.420+0000 7f51f77fe700 10 client.25697 handle_command_reply: tid=0 2024-06-10T00:35:27.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.420+0000 7f522908db80 1 client.25697 shutdown 2024-06-10T00:35:27.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.420+0000 7f522908db80 1 -- 172.21.15.62:0/461111869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 msgr2=0x55f63a333ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:27.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.420+0000 7f522908db80 1 --2- 172.21.15.62:0/461111869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a333ad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5208097fc0 tx=0x7f52080097b0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.421+0000 7f522908db80 1 -- 172.21.15.62:0/461111869 shutdown_connections 2024-06-10T00:35:27.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.421+0000 7f522908db80 1 --2- 172.21.15.62:0/461111869 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f63a39d3e0 0x55f63a39f7d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.421+0000 7f522908db80 1 --2- 172.21.15.62:0/461111869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63a32a7f0 0x55f63a320060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.421+0000 7f522908db80 1 --2- 172.21.15.62:0/461111869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63a315f60 0x55f63a333ad0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.421+0000 7f522908db80 1 --2- 172.21.15.62:0/461111869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63a316880 0x55f63a334010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.421+0000 7f522908db80 1 -- 172.21.15.62:0/461111869 >> 172.21.15.62:0/461111869 conn(0x55f63a23d080 msgr2=0x55f63a318280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:27.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.421+0000 7f51f77fe700 0 client.25697 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:35:27.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.421+0000 7f522908db80 1 -- 172.21.15.62:0/461111869 shutdown_connections 2024-06-10T00:35:27.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.421+0000 7f522908db80 1 -- 172.21.15.62:0/461111869 wait complete. 2024-06-10T00:35:27.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.422+0000 7f522908db80 20 client.25697 trim_cache size 0 max 16384 2024-06-10T00:35:27.424 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:35:27.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.424+0000 7f521e80a700 1 -- 172.21.15.62:0/4266267854 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f52000a2d90 msgr2=0x7f52000a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:27.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.424+0000 7f521e80a700 1 --2- 172.21.15.62:0/4266267854 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f52000a2d90 0x7f52000a5250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f520c00b2e0 tx=0x7f520c005960 comp rx=0 tx=0).stop 2024-06-10T00:35:27.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.424+0000 7f521e80a700 1 -- 172.21.15.62:0/4266267854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5218101380 msgr2=0x7f521819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:27.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.424+0000 7f521e80a700 1 --2- 172.21.15.62:0/4266267854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5218101380 0x7f521819e830 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f520800e9c0 tx=0x7f520800ecd0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.425+0000 7f521e80a700 1 -- 172.21.15.62:0/4266267854 shutdown_connections 2024-06-10T00:35:27.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.425+0000 7f521e80a700 1 --2- 172.21.15.62:0/4266267854 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f52000a2d90 0x7f52000a5250 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.425+0000 7f521e80a700 1 --2- 172.21.15.62:0/4266267854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5218101380 0x7f521819e830 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.425+0000 7f521e80a700 1 --2- 172.21.15.62:0/4266267854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52181073f0 0x7f52181989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.425+0000 7f521e80a700 1 --2- 172.21.15.62:0/4266267854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218101d40 0x7f521819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:27.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.425+0000 7f521e80a700 1 -- 172.21.15.62:0/4266267854 >> 172.21.15.62:0/4266267854 conn(0x7f5218078300 msgr2=0x7f5218100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:27.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.425+0000 7f521e80a700 1 -- 172.21.15.62:0/4266267854 shutdown_connections 2024-06-10T00:35:27.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:27.427+0000 7f521e80a700 1 -- 172.21.15.62:0/4266267854 wait complete. 2024-06-10T00:35:27.759 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:27 smithi062 ceph-mon[32390]: pgmap v81: 129 pgs: 129 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 110 MiB/s wr, 87 op/s 2024-06-10T00:35:27.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:27 smithi105 ceph-mon[36321]: pgmap v81: 129 pgs: 129 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 110 MiB/s wr, 87 op/s 2024-06-10T00:35:27.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:27 smithi138 ceph-mon[36549]: pgmap v81: 129 pgs: 129 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 110 MiB/s wr, 87 op/s 2024-06-10T00:35:28.341 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T00:35:28.853 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:35:29.611 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:29 smithi062 ceph-mon[32390]: pgmap v82: 129 pgs: 129 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 110 MiB/s wr, 83 op/s 2024-06-10T00:35:29.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:29 smithi105 ceph-mon[36321]: pgmap v82: 129 pgs: 129 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 110 MiB/s wr, 83 op/s 2024-06-10T00:35:29.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:29 smithi138 ceph-mon[36549]: pgmap v82: 129 pgs: 129 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 110 MiB/s wr, 83 op/s 2024-06-10T00:35:30.844 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:30 smithi062 ceph-mon[32390]: pgmap v83: 129 pgs: 129 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 110 MiB/s wr, 83 op/s 2024-06-10T00:35:30.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:30 smithi138 ceph-mon[36549]: pgmap v83: 129 pgs: 129 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 110 MiB/s wr, 83 op/s 2024-06-10T00:35:31.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:30 smithi105 ceph-mon[36321]: pgmap v83: 129 pgs: 129 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 110 MiB/s wr, 83 op/s 2024-06-10T00:35:31.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.577+0000 7fa3e4b8b700 1 -- 172.21.15.62:0/1898445743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e0101390 msgr2=0x7fa3e0101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:31.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.578+0000 7fa3e4b8b700 1 --2- 172.21.15.62:0/1898445743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e0101390 0x7fa3e0101810 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009990 tx=0x7fa3d4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.579+0000 7fa3e4b8b700 1 -- 172.21.15.62:0/1898445743 shutdown_connections 2024-06-10T00:35:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.579+0000 7fa3e4b8b700 1 --2- 172.21.15.62:0/1898445743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e0101d50 0x7fa3e010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.579+0000 7fa3e4b8b700 1 --2- 172.21.15.62:0/1898445743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3e0107400 0x7fa3e01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.579+0000 7fa3e4b8b700 1 --2- 172.21.15.62:0/1898445743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e0101390 0x7fa3e0101810 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.579+0000 7fa3e4b8b700 1 -- 172.21.15.62:0/1898445743 >> 172.21.15.62:0/1898445743 conn(0x7fa3e0078340 msgr2=0x7fa3e0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:31.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.581+0000 7fa3e4b8b700 1 -- 172.21.15.62:0/1898445743 shutdown_connections 2024-06-10T00:35:31.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.581+0000 7fa3e4b8b700 1 -- 172.21.15.62:0/1898445743 wait complete. 2024-06-10T00:35:31.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.582+0000 7fa3e4b8b700 1 Processor -- start 2024-06-10T00:35:31.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.583+0000 7fa3e4b8b700 1 -- start start 2024-06-10T00:35:31.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.584+0000 7fa3e4b8b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e0101390 0x7fa3e019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.584+0000 7fa3e4b8b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3e0101d50 0x7fa3e019dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.584+0000 7fa3e4b8b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e0107400 0x7fa3e01979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.584+0000 7fa3e4b8b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0111710 con 0x7fa3e0107400 2024-06-10T00:35:31.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.584+0000 7fa3e4b8b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0111590 con 0x7fa3e0101d50 2024-06-10T00:35:31.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.584+0000 7fa3e4b8b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0111890 con 0x7fa3e0101390 2024-06-10T00:35:31.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.584+0000 7fa3ded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e0107400 0x7fa3e01979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:31.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.584+0000 7fa3ded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e0107400 0x7fa3e01979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43012/0 (socket says 172.21.15.62:43012) 2024-06-10T00:35:31.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.585+0000 7fa3ded9d700 1 -- 172.21.15.62:0/3861376612 learned_addr learned my addr 172.21.15.62:0/3861376612 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:31.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.585+0000 7fa3ded9d700 1 -- 172.21.15.62:0/3861376612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e0101390 msgr2=0x7fa3e019d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:31.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.585+0000 7fa3ded9d700 1 --2- 172.21.15.62:0/3861376612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e0101390 0x7fa3e019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.585+0000 7fa3ddd9b700 1 --2- 172.21.15.62:0/3861376612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3e0101d50 0x7fa3e019dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:31.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.585+0000 7fa3ded9d700 1 -- 172.21.15.62:0/3861376612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3e0101d50 msgr2=0x7fa3e019dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:31.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.585+0000 7fa3ded9d700 1 --2- 172.21.15.62:0/3861376612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3e0101d50 0x7fa3e019dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.585+0000 7fa3ded9d700 1 -- 172.21.15.62:0/3861376612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e0198110 con 0x7fa3e0107400 2024-06-10T00:35:31.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.585+0000 7fa3ddd9b700 1 --2- 172.21.15.62:0/3861376612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3e0101d50 0x7fa3e019dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:35:31.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.586+0000 7fa3ded9d700 1 --2- 172.21.15.62:0/3861376612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e0107400 0x7fa3e01979b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800ef30 tx=0x7fa3d800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:31.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.586+0000 7fa3cf7fe700 1 -- 172.21.15.62:0/3861376612 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d80092e0 con 0x7fa3e0107400 2024-06-10T00:35:31.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.586+0000 7fa3cf7fe700 1 -- 172.21.15.62:0/3861376612 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d8004bb0 con 0x7fa3e0107400 2024-06-10T00:35:31.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.586+0000 7fa3cf7fe700 1 -- 172.21.15.62:0/3861376612 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d80092e0 con 0x7fa3e0107400 2024-06-10T00:35:31.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.586+0000 7fa3e4b8b700 1 -- 172.21.15.62:0/3861376612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e01983a0 con 0x7fa3e0107400 2024-06-10T00:35:31.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.587+0000 7fa3e4b8b700 1 -- 172.21.15.62:0/3861376612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e01a46c0 con 0x7fa3e0107400 2024-06-10T00:35:31.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.591+0000 7fa3cf7fe700 1 -- 172.21.15.62:0/3861376612 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa3d801b030 con 0x7fa3e0107400 2024-06-10T00:35:31.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.592+0000 7fa3cf7fe700 1 --2- 172.21.15.62:0/3861376612 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa3c80a31d0 0x7fa3c80a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.592+0000 7fa3cf7fe700 1 -- 172.21.15.62:0/3861376612 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fa3d8019070 con 0x7fa3e0107400 2024-06-10T00:35:31.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.592+0000 7fa3de59c700 1 --2- 172.21.15.62:0/3861376612 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa3c80a31d0 0x7fa3c80a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:31.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.593+0000 7fa3de59c700 1 --2- 172.21.15.62:0/3861376612 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa3c80a31d0 0x7fa3c80a5690 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0198af0 tx=0x7fa3d0005c90 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:31.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.593+0000 7fa3ef40eb80 1 Processor -- start 2024-06-10T00:35:31.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.593+0000 7fa3ef40eb80 1 -- start start 2024-06-10T00:35:31.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.593+0000 7fa3ef40eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa0d000 0x55830fa0b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.593+0000 7fa3ef40eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa0b7a0 0x55830fa0bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3ef40eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa198e0 0x55830fa1bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3ef40eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55830f8fdf60 con 0x55830fa0d000 2024-06-10T00:35:31.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3ef40eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55830f95c850 con 0x55830fa198e0 2024-06-10T00:35:31.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3ef40eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55830f8c4460 con 0x55830fa0b7a0 2024-06-10T00:35:31.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3de59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa0d000 0x55830fa0b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:31.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3de59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa0d000 0x55830fa0b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43024/0 (socket says 172.21.15.62:43024) 2024-06-10T00:35:31.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3de59c700 1 -- 172.21.15.62:0/2595509891 learned_addr learned my addr 172.21.15.62:0/2595509891 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:31.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3ddd9b700 1 --2- 172.21.15.62:0/2595509891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa0b7a0 0x55830fa0bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:31.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3de59c700 1 -- 172.21.15.62:0/2595509891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa0b7a0 msgr2=0x55830fa0bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:31.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3de59c700 1 --2- 172.21.15.62:0/2595509891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa0b7a0 0x55830fa0bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3de59c700 1 -- 172.21.15.62:0/2595509891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa198e0 msgr2=0x55830fa1bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:31.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3ded9d700 1 --2- 172.21.15.62:0/2595509891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa198e0 0x55830fa1bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:31.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3de59c700 1 --2- 172.21.15.62:0/2595509891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa198e0 0x55830fa1bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3de59c700 1 -- 172.21.15.62:0/2595509891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55830f8ab8d0 con 0x55830fa0d000 2024-06-10T00:35:31.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.594+0000 7fa3ded9d700 1 --2- 172.21.15.62:0/2595509891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa198e0 0x55830fa1bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:35:31.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.595+0000 7fa3de59c700 1 --2- 172.21.15.62:0/2595509891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa0d000 0x55830fa0b260 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000cba0 tx=0x7fa3d000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789ebfcc1a0b5df server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:31.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.595+0000 7fa3ccff9700 1 -- 172.21.15.62:0/2595509891 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d000fae0 con 0x55830fa0d000 2024-06-10T00:35:31.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.595+0000 7fa3ccff9700 1 -- 172.21.15.62:0/2595509891 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d0021070 con 0x55830fa0d000 2024-06-10T00:35:31.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.595+0000 7fa3ccff9700 1 -- 172.21.15.62:0/2595509891 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d000e610 con 0x55830fa0d000 2024-06-10T00:35:31.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.597+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/2595509891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa0d000 msgr2=0x55830fa0b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:31.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.598+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/2595509891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa0d000 0x55830fa0b260 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000cba0 tx=0x7fa3d000f000 comp rx=0 tx=0).stop 2024-06-10T00:35:31.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.599+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/2595509891 shutdown_connections 2024-06-10T00:35:31.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.599+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/2595509891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa0d000 0x55830fa0b260 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.599+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/2595509891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa198e0 0x55830fa1bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.599+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/2595509891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa0b7a0 0x55830fa0bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.599+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/2595509891 >> 172.21.15.62:0/2595509891 conn(0x55830f938080 msgr2=0x55830fa06220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:31.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.600+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/2595509891 shutdown_connections 2024-06-10T00:35:31.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.600+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/2595509891 wait complete. 2024-06-10T00:35:31.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.601+0000 7fa3ef40eb80 1 -- start start 2024-06-10T00:35:31.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.601+0000 7fa3ef40eb80 1 Processor -- start 2024-06-10T00:35:31.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.602+0000 7fa3ef40eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa0b7a0 0x55830fa19110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.602+0000 7fa3ef40eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa0d000 0x55830fa16410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.602+0000 7fa3ef40eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa198e0 0x55830fa16950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.602+0000 7fa3ef40eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55830f8fcfc0 con 0x55830fa198e0 2024-06-10T00:35:31.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.602+0000 7fa3ef40eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55830f958810 con 0x55830fa0b7a0 2024-06-10T00:35:31.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.602+0000 7fa3ef40eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55830fa1c620 con 0x55830fa0d000 2024-06-10T00:35:31.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.602+0000 7fa3ef40eb80 10 client.0 _ll_register_callbacks cb 0x55830f8f25b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:31.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.603+0000 7fa3ded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa198e0 0x55830fa16950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:31.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.603+0000 7fa3ded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa198e0 0x55830fa16950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43040/0 (socket says 172.21.15.62:43040) 2024-06-10T00:35:31.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.603+0000 7fa3ded9d700 1 -- 172.21.15.62:0/392204842 learned_addr learned my addr 172.21.15.62:0/392204842 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:31.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.603+0000 7fa3de59c700 1 --2- 172.21.15.62:0/392204842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa0b7a0 0x55830fa19110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:31.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.604+0000 7fa3de59c700 1 -- 172.21.15.62:0/392204842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa0d000 msgr2=0x55830fa16410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:35:31.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.604+0000 7fa3de59c700 1 --2- 172.21.15.62:0/392204842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa0d000 0x55830fa16410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.604+0000 7fa3de59c700 1 -- 172.21.15.62:0/392204842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa198e0 msgr2=0x55830fa16950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:31.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.604+0000 7fa3de59c700 1 --2- 172.21.15.62:0/392204842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa198e0 0x55830fa16950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.604+0000 7fa3de59c700 1 -- 172.21.15.62:0/392204842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55830f922750 con 0x55830fa0b7a0 2024-06-10T00:35:31.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.604+0000 7fa3de59c700 1 --2- 172.21.15.62:0/392204842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa0b7a0 0x55830fa19110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000afd0 tx=0x7fa3d000a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c4a629e9df8f22b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:31.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.604+0000 7fa3c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:35:31.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.605+0000 7fa3c5ffb700 1 -- 172.21.15.62:0/392204842 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0026610 con 0x55830fa0b7a0 2024-06-10T00:35:31.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.605+0000 7fa3c5ffb700 1 -- 172.21.15.62:0/392204842 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fa3d001c800 con 0x55830fa0b7a0 2024-06-10T00:35:31.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.605+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/392204842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d0021070 con 0x55830fa0b7a0 2024-06-10T00:35:31.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.605+0000 7fa3c5ffb700 1 -- 172.21.15.62:0/392204842 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa3d0039050 con 0x55830fa0b7a0 2024-06-10T00:35:31.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.605+0000 7fa3ef40eb80 10 client.25700 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:31.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.605+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/392204842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55830fa26b80 con 0x55830fa0b7a0 2024-06-10T00:35:31.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.606+0000 7fa3c5ffb700 1 -- 172.21.15.62:0/392204842 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d0021070 con 0x55830fa0b7a0 2024-06-10T00:35:31.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.606+0000 7fa3c5ffb700 1 -- 172.21.15.62:0/392204842 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa3d0030940 con 0x55830fa0b7a0 2024-06-10T00:35:31.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.606+0000 7fa3ef40eb80 10 client.25700 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:31.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.606+0000 7fa3ef40eb80 10 client.25700 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:35:31.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.606+0000 7fa3ef40eb80 20 client.25700 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:31.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.606+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/392204842 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55830fa29160 0x55830fa2b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.606+0000 7fa3ef40eb80 4 client.25700 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:35:31.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.606+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/392204842 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55830f969d50 con 0x55830fa29160 2024-06-10T00:35:31.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.607+0000 7fa3ddd9b700 1 --2- 172.21.15.62:0/392204842 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55830fa29160 0x55830fa2b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:31.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.608+0000 7fa3ddd9b700 1 --2- 172.21.15.62:0/392204842 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55830fa29160 0x55830fa2b550 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=550bda6ca63406b2 server_cookie=8f9ac7f5b712cc4f in_seq=0 out_seq=0 2024-06-10T00:35:31.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.608+0000 7fa3c5ffb700 10 client.25700 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:35:31.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.609+0000 7fa3c5ffb700 1 -- 172.21.15.62:0/392204842 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55830f969d50 con 0x55830fa29160 2024-06-10T00:35:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.609+0000 7fa3c5ffb700 10 client.25700 handle_command_reply: tid=0 2024-06-10T00:35:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.609+0000 7fa3ef40eb80 1 client.25700 shutdown 2024-06-10T00:35:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.609+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/392204842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa0b7a0 msgr2=0x55830fa19110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.609+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/392204842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa0b7a0 0x55830fa19110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000afd0 tx=0x7fa3d000a7a0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.609+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/392204842 shutdown_connections 2024-06-10T00:35:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.609+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/392204842 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55830fa29160 0x55830fa2b550 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.610+0000 7fa3c5ffb700 0 client.25700 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:35:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.610+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/392204842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa198e0 0x55830fa16950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.610+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/392204842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa0b7a0 0x55830fa19110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.610+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/392204842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa0d000 0x55830fa16410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.610+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/392204842 >> 172.21.15.62:0/392204842 conn(0x55830f938080 msgr2=0x55830fa05700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.610+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/392204842 shutdown_connections 2024-06-10T00:35:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.610+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/392204842 wait complete. 2024-06-10T00:35:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.610+0000 7fa3ef40eb80 20 client.25700 trim_cache size 0 max 16384 2024-06-10T00:35:31.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.639+0000 7fa3ef40eb80 1 Processor -- start 2024-06-10T00:35:31.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.639+0000 7fa3ef40eb80 1 -- start start 2024-06-10T00:35:31.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.639+0000 7fa3ef40eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa0b7a0 0x55830fa05e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.639+0000 7fa3ef40eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa0d000 0x55830fa063c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.639+0000 7fa3ef40eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa198e0 0x55830fa163d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.639+0000 7fa3ef40eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01ac510 con 0x55830fa0b7a0 2024-06-10T00:35:31.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.640+0000 7fa3de59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa0b7a0 0x55830fa05e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:31.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.640+0000 7fa3ef40eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01ae050 con 0x55830fa0d000 2024-06-10T00:35:31.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.640+0000 7fa3ef40eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01acbe0 con 0x55830fa198e0 2024-06-10T00:35:31.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.640+0000 7fa3de59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa0b7a0 0x55830fa05e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43050/0 (socket says 172.21.15.62:43050) 2024-06-10T00:35:31.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.640+0000 7fa3de59c700 1 -- 172.21.15.62:0/3632389315 learned_addr learned my addr 172.21.15.62:0/3632389315 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:31.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.640+0000 7fa3de59c700 1 -- 172.21.15.62:0/3632389315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa198e0 msgr2=0x55830fa163d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:31.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.640+0000 7fa3de59c700 1 --2- 172.21.15.62:0/3632389315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa198e0 0x55830fa163d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.640+0000 7fa3de59c700 1 -- 172.21.15.62:0/3632389315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa0d000 msgr2=0x55830fa063c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:35:31.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.640+0000 7fa3de59c700 1 --2- 172.21.15.62:0/3632389315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa0d000 0x55830fa063c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.640+0000 7fa3de59c700 1 -- 172.21.15.62:0/3632389315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55830fa02810 con 0x55830fa0b7a0 2024-06-10T00:35:31.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.640+0000 7fa3de59c700 1 --2- 172.21.15.62:0/3632389315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa0b7a0 0x55830fa05e80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000ffd0 tx=0x7fa3d0005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609115d6a81ebecd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:31.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.641+0000 7fa3c7fff700 1 -- 172.21.15.62:0/3632389315 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d001c800 con 0x55830fa0b7a0 2024-06-10T00:35:31.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.641+0000 7fa3c7fff700 1 -- 172.21.15.62:0/3632389315 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d0021040 con 0x55830fa0b7a0 2024-06-10T00:35:31.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.641+0000 7fa3c7fff700 1 -- 172.21.15.62:0/3632389315 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d000e950 con 0x55830fa0b7a0 2024-06-10T00:35:31.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.642+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/3632389315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa0b7a0 msgr2=0x55830fa05e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:31.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.642+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/3632389315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa0b7a0 0x55830fa05e80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000ffd0 tx=0x7fa3d0005d20 comp rx=0 tx=0).stop 2024-06-10T00:35:31.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.642+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/3632389315 shutdown_connections 2024-06-10T00:35:31.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.642+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/3632389315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa0b7a0 0x55830fa05e80 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.642+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/3632389315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa198e0 0x55830fa163d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.642+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/3632389315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa0d000 0x55830fa063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.642+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/3632389315 >> 172.21.15.62:0/3632389315 conn(0x55830f938080 msgr2=0x55830fa15cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:31.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.642+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/3632389315 shutdown_connections 2024-06-10T00:35:31.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.642+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/3632389315 wait complete. 2024-06-10T00:35:31.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.642+0000 7fa3ef40eb80 1 -- start start 2024-06-10T00:35:31.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.642+0000 7fa3ef40eb80 1 Processor -- start 2024-06-10T00:35:31.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.643+0000 7fa3ef40eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa0b7a0 0x55830fa11280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.643+0000 7fa3ef40eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa0d000 0x55830fa0f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.643+0000 7fa3ef40eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa198e0 0x55830fa0fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.643+0000 7fa3ef40eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01ad980 con 0x55830fa198e0 2024-06-10T00:35:31.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.643+0000 7fa3ef40eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01ad2b0 con 0x55830fa0b7a0 2024-06-10T00:35:31.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.643+0000 7fa3ef40eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01aa490 con 0x55830fa0d000 2024-06-10T00:35:31.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.643+0000 7fa3ef40eb80 10 client.0 _ll_register_callbacks cb 0x55830f8f25b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:31.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.643+0000 7fa3ded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa198e0 0x55830fa0fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:31.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.643+0000 7fa3ded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa198e0 0x55830fa0fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43054/0 (socket says 172.21.15.62:43054) 2024-06-10T00:35:31.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.643+0000 7fa3ded9d700 1 -- 172.21.15.62:0/830458777 learned_addr learned my addr 172.21.15.62:0/830458777 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:31.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.644+0000 7fa3ded9d700 1 -- 172.21.15.62:0/830458777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa0d000 msgr2=0x55830fa0f660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:35:31.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.644+0000 7fa3ded9d700 1 --2- 172.21.15.62:0/830458777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa0d000 0x55830fa0f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.644+0000 7fa3ded9d700 1 -- 172.21.15.62:0/830458777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa0b7a0 msgr2=0x55830fa11280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:31.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.644+0000 7fa3ded9d700 1 --2- 172.21.15.62:0/830458777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa0b7a0 0x55830fa11280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.644+0000 7fa3ded9d700 1 -- 172.21.15.62:0/830458777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55830fa2d580 con 0x55830fa198e0 2024-06-10T00:35:31.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.644+0000 7fa3ded9d700 1 --2- 172.21.15.62:0/830458777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa198e0 0x55830fa0fce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa3d809af80 tx=0x7fa3d809b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129511e92741bd10 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:31.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.644+0000 7fa3c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:31.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.644+0000 7fa3c5ffb700 1 -- 172.21.15.62:0/830458777 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d809f3a0 con 0x55830fa198e0 2024-06-10T00:35:31.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.644+0000 7fa3c5ffb700 1 -- 172.21.15.62:0/830458777 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fa3d80a3740 con 0x55830fa198e0 2024-06-10T00:35:31.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.645+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/830458777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d0021040 con 0x55830fa198e0 2024-06-10T00:35:31.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.645+0000 7fa3c5ffb700 1 -- 172.21.15.62:0/830458777 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa3d80a3bd0 con 0x55830fa198e0 2024-06-10T00:35:31.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.645+0000 7fa3ef40eb80 10 client.16197 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:31.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.645+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/830458777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55830fa2e8e0 con 0x55830fa198e0 2024-06-10T00:35:31.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.645+0000 7fa3c5ffb700 1 -- 172.21.15.62:0/830458777 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d0021040 con 0x55830fa198e0 2024-06-10T00:35:31.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.646+0000 7fa3c5ffb700 1 -- 172.21.15.62:0/830458777 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa3d8095c80 con 0x55830fa198e0 2024-06-10T00:35:31.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.646+0000 7fa3ef40eb80 10 client.16197 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:31.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.646+0000 7fa3ef40eb80 10 client.16197 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:35:31.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.646+0000 7fa3ef40eb80 20 client.16197 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:31.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.646+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/830458777 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55830fa29160 0x55830faab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:31.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.646+0000 7fa3ef40eb80 4 client.16197 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:35:31.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.646+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/830458777 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55830f9642d0 con 0x55830fa29160 2024-06-10T00:35:31.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.650+0000 7fa3de59c700 1 --2- 172.21.15.62:0/830458777 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55830fa29160 0x55830faab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:31.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.653+0000 7fa3c5ffb700 10 client.16197 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:35:31.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.653+0000 7fa3de59c700 1 --2- 172.21.15.62:0/830458777 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55830fa29160 0x55830faab2a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dcbfb32168175f1b server_cookie=780ff53e5f1da2f7 in_seq=0 out_seq=0 2024-06-10T00:35:31.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.653+0000 7fa3c5ffb700 1 -- 172.21.15.62:0/830458777 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55830f9642d0 con 0x55830fa29160 2024-06-10T00:35:31.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.653+0000 7fa3c5ffb700 10 client.16197 handle_command_reply: tid=0 2024-06-10T00:35:31.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.653+0000 7fa3ef40eb80 1 client.16197 shutdown 2024-06-10T00:35:31.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.653+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/830458777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa198e0 msgr2=0x55830fa0fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:31.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.653+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/830458777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa198e0 0x55830fa0fce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa3d809af80 tx=0x7fa3d809b6f0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.654+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/830458777 shutdown_connections 2024-06-10T00:35:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.654+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/830458777 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55830fa29160 0x55830faab2a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.654+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/830458777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55830fa198e0 0x55830fa0fce0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.654+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/830458777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55830fa0b7a0 0x55830fa11280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.654+0000 7fa3ef40eb80 1 --2- 172.21.15.62:0/830458777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55830fa0d000 0x55830fa0f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.654+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/830458777 >> 172.21.15.62:0/830458777 conn(0x55830f938080 msgr2=0x55830fa25140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.654+0000 7fa3c5ffb700 0 client.16197 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:35:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.654+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/830458777 shutdown_connections 2024-06-10T00:35:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.654+0000 7fa3ef40eb80 1 -- 172.21.15.62:0/830458777 wait complete. 2024-06-10T00:35:31.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.654+0000 7fa3ef40eb80 20 client.16197 trim_cache size 0 max 16384 2024-06-10T00:35:31.656 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:35:31.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.655+0000 7fa3e4b8b700 1 -- 172.21.15.62:0/3861376612 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa3c80a31d0 msgr2=0x7fa3c80a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:31.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.655+0000 7fa3e4b8b700 1 --2- 172.21.15.62:0/3861376612 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa3c80a31d0 0x7fa3c80a5690 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0198af0 tx=0x7fa3d0005c90 comp rx=0 tx=0).stop 2024-06-10T00:35:31.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.655+0000 7fa3e4b8b700 1 -- 172.21.15.62:0/3861376612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e0107400 msgr2=0x7fa3e01979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:31.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.655+0000 7fa3e4b8b700 1 --2- 172.21.15.62:0/3861376612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e0107400 0x7fa3e01979b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800ef30 tx=0x7fa3d800c450 comp rx=0 tx=0).stop 2024-06-10T00:35:31.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.655+0000 7fa3e4b8b700 1 -- 172.21.15.62:0/3861376612 shutdown_connections 2024-06-10T00:35:31.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.655+0000 7fa3e4b8b700 1 --2- 172.21.15.62:0/3861376612 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa3c80a31d0 0x7fa3c80a5690 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.655+0000 7fa3e4b8b700 1 --2- 172.21.15.62:0/3861376612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3e0107400 0x7fa3e01979b0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.655+0000 7fa3e4b8b700 1 --2- 172.21.15.62:0/3861376612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e0101390 0x7fa3e019d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.655+0000 7fa3e4b8b700 1 --2- 172.21.15.62:0/3861376612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3e0101d50 0x7fa3e019dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:31.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.655+0000 7fa3e4b8b700 1 -- 172.21.15.62:0/3861376612 >> 172.21.15.62:0/3861376612 conn(0x7fa3e0078340 msgr2=0x7fa3e00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:31.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.655+0000 7fa3e4b8b700 1 -- 172.21.15.62:0/3861376612 shutdown_connections 2024-06-10T00:35:31.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:31.655+0000 7fa3e4b8b700 1 -- 172.21.15.62:0/3861376612 wait complete. 2024-06-10T00:35:32.762 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T00:35:33.308 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:35:33.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:33 smithi062 ceph-mon[32390]: pgmap v84: 129 pgs: 129 active+clean; 5.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 455 MiB/s wr, 210 op/s 2024-06-10T00:35:33.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:33 smithi138 ceph-mon[36549]: pgmap v84: 129 pgs: 129 active+clean; 5.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 455 MiB/s wr, 210 op/s 2024-06-10T00:35:33.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:33 smithi105 ceph-mon[36321]: pgmap v84: 129 pgs: 129 active+clean; 5.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 455 MiB/s wr, 210 op/s 2024-06-10T00:35:35.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:35 smithi062 ceph-mon[32390]: pgmap v85: 129 pgs: 129 active+clean; 5.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 455 MiB/s wr, 202 op/s 2024-06-10T00:35:35.376 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:35 smithi138 ceph-mon[36549]: pgmap v85: 129 pgs: 129 active+clean; 5.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 455 MiB/s wr, 202 op/s 2024-06-10T00:35:35.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:35 smithi105 ceph-mon[36321]: pgmap v85: 129 pgs: 129 active+clean; 5.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 455 MiB/s wr, 202 op/s 2024-06-10T00:35:35.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.625+0000 7f5db9e4c700 1 -- 172.21.15.62:0/3008326733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db4109490 msgr2=0x7f5db4109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:35.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.627+0000 7f5db27fc700 1 -- 172.21.15.62:0/3008326733 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac00bd70 con 0x7f5db4109490 2024-06-10T00:35:35.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.627+0000 7f5db9e4c700 1 --2- 172.21.15.62:0/3008326733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db4109490 0x7f5db4109870 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00b0a0 tx=0x7f5dac00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.628+0000 7f5db9e4c700 1 -- 172.21.15.62:0/3008326733 shutdown_connections 2024-06-10T00:35:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.628+0000 7f5db9e4c700 1 --2- 172.21.15.62:0/3008326733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db4109490 0x7f5db4109870 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.628+0000 7f5db9e4c700 1 --2- 172.21.15.62:0/3008326733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5db41155e0 0x7f5db41179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.628+0000 7f5db9e4c700 1 --2- 172.21.15.62:0/3008326733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5db4109db0 0x7f5db41150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.628+0000 7f5db9e4c700 1 -- 172.21.15.62:0/3008326733 >> 172.21.15.62:0/3008326733 conn(0x7f5db406d600 msgr2=0x7f5db406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:35.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.628+0000 7f5db9e4c700 1 -- 172.21.15.62:0/3008326733 shutdown_connections 2024-06-10T00:35:35.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.628+0000 7f5db9e4c700 1 -- 172.21.15.62:0/3008326733 wait complete. 2024-06-10T00:35:35.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.630+0000 7f5db9e4c700 1 Processor -- start 2024-06-10T00:35:35.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.630+0000 7f5db9e4c700 1 -- start start 2024-06-10T00:35:35.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.630+0000 7f5db9e4c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5db4109490 0x7f5db41ad810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.630+0000 7f5db9e4c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5db4109db0 0x7f5db41add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.630+0000 7f5db9e4c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db41155e0 0x7f5db41140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.630+0000 7f5db9e4c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5db411b120 con 0x7f5db41155e0 2024-06-10T00:35:35.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.630+0000 7f5db9e4c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5db411afa0 con 0x7f5db4109db0 2024-06-10T00:35:35.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.630+0000 7f5db9e4c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5db411b2a0 con 0x7f5db4109490 2024-06-10T00:35:35.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.630+0000 7f5db37fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5db4109490 0x7f5db41ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.630+0000 7f5db37fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5db4109490 0x7f5db41ad810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:44780/0 (socket says 172.21.15.62:44780) 2024-06-10T00:35:35.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.630+0000 7f5db37fe700 1 -- 172.21.15.62:0/4108451466 learned_addr learned my addr 172.21.15.62:0/4108451466 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.631+0000 7f5db2ffd700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5db4109db0 0x7f5db41add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.631+0000 7f5db3fff700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db41155e0 0x7f5db41140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.631+0000 7f5db2ffd700 1 -- 172.21.15.62:0/4108451466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5db4109490 msgr2=0x7f5db41ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.631+0000 7f5db2ffd700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5db4109490 0x7f5db41ad810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.631+0000 7f5db2ffd700 1 -- 172.21.15.62:0/4108451466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db41155e0 msgr2=0x7f5db41140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.631+0000 7f5db2ffd700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db41155e0 0x7f5db41140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.631+0000 7f5db2ffd700 1 -- 172.21.15.62:0/4108451466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db4114990 con 0x7f5db4109db0 2024-06-10T00:35:35.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.631+0000 7f5db3fff700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db41155e0 0x7f5db41140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:35:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.631+0000 7f5db37fe700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5db4109490 0x7f5db41ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:35:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.631+0000 7f5db2ffd700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5db4109db0 0x7f5db41add50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5da400b450 tx=0x7f5da400b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.633+0000 7f5db0ff9700 1 -- 172.21.15.62:0/4108451466 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4015020 con 0x7f5db4109db0 2024-06-10T00:35:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.633+0000 7f5db0ff9700 1 -- 172.21.15.62:0/4108451466 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da4011d10 con 0x7f5db4109db0 2024-06-10T00:35:35.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.633+0000 7f5db0ff9700 1 -- 172.21.15.62:0/4108451466 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4010e20 con 0x7f5db4109db0 2024-06-10T00:35:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.633+0000 7f5db9e4c700 1 -- 172.21.15.62:0/4108451466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db4114c80 con 0x7f5db4109db0 2024-06-10T00:35:35.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.633+0000 7f5db9e4c700 1 -- 172.21.15.62:0/4108451466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db41b4790 con 0x7f5db4109db0 2024-06-10T00:35:35.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.639+0000 7f5db0ff9700 1 -- 172.21.15.62:0/4108451466 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5da4018020 con 0x7f5db4109db0 2024-06-10T00:35:35.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.640+0000 7f5db0ff9700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5d9c0a2f90 0x7f5d9c0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.640+0000 7f5db0ff9700 1 -- 172.21.15.62:0/4108451466 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f5da40cedb0 con 0x7f5db4109db0 2024-06-10T00:35:35.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.641+0000 7f5db37fe700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5d9c0a2f90 0x7f5d9c0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.642+0000 7f5db37fe700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5d9c0a2f90 0x7f5d9c0a5450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009e60 tx=0x7f5dac006020 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:35.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.645+0000 7f5dc46cfb80 1 Processor -- start 2024-06-10T00:35:35.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.646+0000 7f5dc46cfb80 1 -- start start 2024-06-10T00:35:35.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.646+0000 7f5dc46cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf088ff60 0x564cf0890340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.647+0000 7f5dc46cfb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf0890880 0x564cf088a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.647+0000 7f5dc46cfb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf088a7a0 0x564cf088abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.647+0000 7f5db37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf088ff60 0x564cf0890340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.647+0000 7f5db37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf088ff60 0x564cf0890340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43082/0 (socket says 172.21.15.62:43082) 2024-06-10T00:35:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.647+0000 7f5db37fe700 1 -- 172.21.15.62:0/2107591472 learned_addr learned my addr 172.21.15.62:0/2107591472 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.648+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2107591472 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564cf077cf60 con 0x564cf088ff60 2024-06-10T00:35:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.648+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2107591472 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564cf07db850 con 0x564cf0890880 2024-06-10T00:35:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.648+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2107591472 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564cf0743460 con 0x564cf088a7a0 2024-06-10T00:35:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.648+0000 7f5db2ffd700 1 --2- 172.21.15.62:0/2107591472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf0890880 0x564cf088a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.649+0000 7f5db37fe700 1 -- 172.21.15.62:0/2107591472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf088a7a0 msgr2=0x564cf088abc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:35:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.649+0000 7f5db37fe700 1 --2- 172.21.15.62:0/2107591472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf088a7a0 0x564cf088abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.649+0000 7f5db37fe700 1 -- 172.21.15.62:0/2107591472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf0890880 msgr2=0x564cf088a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.649+0000 7f5db37fe700 1 --2- 172.21.15.62:0/2107591472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf0890880 0x564cf088a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.649+0000 7f5db37fe700 1 -- 172.21.15.62:0/2107591472 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cf072a8d0 con 0x564cf088ff60 2024-06-10T00:35:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.650+0000 7f5db37fe700 1 --2- 172.21.15.62:0/2107591472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf088ff60 0x564cf0890340 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00b070 tx=0x7f5dac01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3b15dfe347ad17 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.651+0000 7f5d99ffb700 1 -- 172.21.15.62:0/2107591472 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac02c070 con 0x564cf088ff60 2024-06-10T00:35:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.651+0000 7f5d99ffb700 1 -- 172.21.15.62:0/2107591472 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dac007b70 con 0x564cf088ff60 2024-06-10T00:35:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.651+0000 7f5d99ffb700 1 -- 172.21.15.62:0/2107591472 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac028680 con 0x564cf088ff60 2024-06-10T00:35:35.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.653+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2107591472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf088ff60 msgr2=0x564cf0890340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:35.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.653+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/2107591472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf088ff60 0x564cf0890340 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00b070 tx=0x7f5dac01d000 comp rx=0 tx=0).stop 2024-06-10T00:35:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.660+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2107591472 shutdown_connections 2024-06-10T00:35:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.660+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/2107591472 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf088ff60 0x564cf0890340 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.660+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/2107591472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf088a7a0 0x564cf088abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.660+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/2107591472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf0890880 0x564cf088a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.660+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2107591472 >> 172.21.15.62:0/2107591472 conn(0x564cf07b7080 msgr2=0x564cf0885220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:35.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.666+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2107591472 shutdown_connections 2024-06-10T00:35:35.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.669+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2107591472 wait complete. 2024-06-10T00:35:35.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.670+0000 7f5dc46cfb80 1 -- start start 2024-06-10T00:35:35.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.671+0000 7f5dc46cfb80 1 Processor -- start 2024-06-10T00:35:35.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.672+0000 7f5dc46cfb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088a7a0 0x564cf08965a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.672+0000 7f5dc46cfb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf088ff60 0x564cf08938a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.672+0000 7f5dc46cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf0890880 0x564cf0893de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.672+0000 7f5db37fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088a7a0 0x564cf08965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.672+0000 7f5db37fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088a7a0 0x564cf08965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:40934/0 (socket says 172.21.15.62:40934) 2024-06-10T00:35:35.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.673+0000 7f5db37fe700 1 -- 172.21.15.62:0/2028096027 learned_addr learned my addr 172.21.15.62:0/2028096027 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:35.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.673+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2028096027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564cf077bfc0 con 0x564cf0890880 2024-06-10T00:35:35.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.673+0000 7f5db2ffd700 1 --2- 172.21.15.62:0/2028096027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf088ff60 0x564cf08938a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.673+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2028096027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564cf07d7810 con 0x564cf088a7a0 2024-06-10T00:35:35.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.673+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2028096027 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564cf0897250 con 0x564cf088ff60 2024-06-10T00:35:35.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.673+0000 7f5dc46cfb80 10 client.0 _ll_register_callbacks cb 0x564cf07715b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:35.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.675+0000 7f5db37fe700 1 -- 172.21.15.62:0/2028096027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf088ff60 msgr2=0x564cf08938a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:35.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.675+0000 7f5db37fe700 1 --2- 172.21.15.62:0/2028096027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf088ff60 0x564cf08938a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.675+0000 7f5db37fe700 1 -- 172.21.15.62:0/2028096027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf0890880 msgr2=0x564cf0893de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:35.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.675+0000 7f5db37fe700 1 --2- 172.21.15.62:0/2028096027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf0890880 0x564cf0893de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.675+0000 7f5db37fe700 1 -- 172.21.15.62:0/2028096027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cf07a1750 con 0x564cf088a7a0 2024-06-10T00:35:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.676+0000 7f5db37fe700 1 --2- 172.21.15.62:0/2028096027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088a7a0 0x564cf08965a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0095f0 tx=0x7f5dac005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5875db9e6174f8a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.676+0000 7f5d92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:35:35.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.676+0000 7f5d92ffd700 1 -- 172.21.15.62:0/2028096027 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac02c040 con 0x564cf088a7a0 2024-06-10T00:35:35.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.677+0000 7f5d92ffd700 1 -- 172.21.15.62:0/2028096027 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f5dac005910 con 0x564cf088a7a0 2024-06-10T00:35:35.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.678+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2028096027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dac007b70 con 0x564cf088a7a0 2024-06-10T00:35:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.681+0000 7f5d92ffd700 1 -- 172.21.15.62:0/2028096027 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5dac03b470 con 0x564cf088a7a0 2024-06-10T00:35:35.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.681+0000 7f5dc46cfb80 10 client.25709 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:35.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.682+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2028096027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564cf08a17d0 con 0x564cf088a7a0 2024-06-10T00:35:35.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.683+0000 7f5d92ffd700 1 -- 172.21.15.62:0/2028096027 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dac03baf0 con 0x564cf088a7a0 2024-06-10T00:35:35.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.683+0000 7f5d92ffd700 1 -- 172.21.15.62:0/2028096027 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5dac011070 con 0x564cf088a7a0 2024-06-10T00:35:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.684+0000 7f5dc46cfb80 10 client.25709 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.684+0000 7f5dc46cfb80 10 client.25709 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:35:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.684+0000 7f5dc46cfb80 20 client.25709 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.684+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/2028096027 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x564cf08a3ca0 0x564cf08a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.685+0000 7f5dc46cfb80 4 client.25709 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:35:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.685+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2028096027 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5dac007b70 con 0x564cf08a3ca0 2024-06-10T00:35:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.685+0000 7f5db2ffd700 1 --2- 172.21.15.62:0/2028096027 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x564cf08a3ca0 0x564cf08a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.686+0000 7f5db2ffd700 1 --2- 172.21.15.62:0/2028096027 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x564cf08a3ca0 0x564cf08a6090 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7ff2e860498035 server_cookie=8c482362f6f83f0e in_seq=0 out_seq=0 2024-06-10T00:35:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.686+0000 7f5d92ffd700 10 client.25709 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:35:35.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.688+0000 7f5d92ffd700 1 -- 172.21.15.62:0/2028096027 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f5dac007b70 con 0x564cf08a3ca0 2024-06-10T00:35:35.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.688+0000 7f5d92ffd700 10 client.25709 handle_command_reply: tid=0 2024-06-10T00:35:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.688+0000 7f5dc46cfb80 1 client.25709 shutdown 2024-06-10T00:35:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.689+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2028096027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088a7a0 msgr2=0x564cf08965a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.689+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/2028096027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088a7a0 0x564cf08965a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0095f0 tx=0x7f5dac005590 comp rx=0 tx=0).stop 2024-06-10T00:35:35.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.694+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2028096027 shutdown_connections 2024-06-10T00:35:35.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.695+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/2028096027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf0890880 0x564cf0893de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.695+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/2028096027 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x564cf08a3ca0 0x564cf08a6090 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.695+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/2028096027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088a7a0 0x564cf08965a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.695+0000 7f5d92ffd700 0 client.25709 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:35:35.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.695+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/2028096027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf088ff60 0x564cf08938a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.695+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2028096027 >> 172.21.15.62:0/2028096027 conn(0x564cf07b7080 msgr2=0x564cf0884700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.695+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2028096027 shutdown_connections 2024-06-10T00:35:35.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.697+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/2028096027 wait complete. 2024-06-10T00:35:35.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.697+0000 7f5dc46cfb80 20 client.25709 trim_cache size 0 max 16384 2024-06-10T00:35:35.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.731+0000 7f5dc46cfb80 1 Processor -- start 2024-06-10T00:35:35.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.731+0000 7f5dc46cfb80 1 -- start start 2024-06-10T00:35:35.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.732+0000 7f5dc46cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf088a7a0 0x564cf088a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.732+0000 7f5dc46cfb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088ff60 0x564cf0891370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.732+0000 7f5dc46cfb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf0890880 0x564cf08960e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.733+0000 7f5dc46cfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41bd5a0 con 0x564cf088a7a0 2024-06-10T00:35:35.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.733+0000 7f5dc46cfb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41bf0e0 con 0x564cf088ff60 2024-06-10T00:35:35.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.733+0000 7f5dc46cfb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41bdc70 con 0x564cf0890880 2024-06-10T00:35:35.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.734+0000 7f5db3fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf0890880 0x564cf08960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.734+0000 7f5db3fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf0890880 0x564cf08960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:44806/0 (socket says 172.21.15.62:44806) 2024-06-10T00:35:35.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.734+0000 7f5db3fff700 1 -- 172.21.15.62:0/886032777 learned_addr learned my addr 172.21.15.62:0/886032777 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:35.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.734+0000 7f5db37fe700 1 --2- 172.21.15.62:0/886032777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf088a7a0 0x564cf088a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.734+0000 7f5db37fe700 1 -- 172.21.15.62:0/886032777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf0890880 msgr2=0x564cf08960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:35.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.734+0000 7f5db37fe700 1 --2- 172.21.15.62:0/886032777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf0890880 0x564cf08960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.734+0000 7f5db37fe700 1 -- 172.21.15.62:0/886032777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088ff60 msgr2=0x564cf0891370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:35:35.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.734+0000 7f5db37fe700 1 --2- 172.21.15.62:0/886032777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088ff60 0x564cf0891370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.734+0000 7f5db37fe700 1 -- 172.21.15.62:0/886032777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cf0891e10 con 0x564cf088a7a0 2024-06-10T00:35:35.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.735+0000 7f5db37fe700 1 --2- 172.21.15.62:0/886032777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf088a7a0 0x564cf088a260 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5dac028420 tx=0x7f5dac003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc72828314b74 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:35.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.736+0000 7f5d997fa700 1 -- 172.21.15.62:0/886032777 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac020070 con 0x564cf088a7a0 2024-06-10T00:35:35.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.736+0000 7f5d997fa700 1 -- 172.21.15.62:0/886032777 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dac007b70 con 0x564cf088a7a0 2024-06-10T00:35:35.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.736+0000 7f5d997fa700 1 -- 172.21.15.62:0/886032777 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac02c070 con 0x564cf088a7a0 2024-06-10T00:35:35.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.737+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/886032777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf088a7a0 msgr2=0x564cf088a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:35.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.737+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/886032777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf088a7a0 0x564cf088a260 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5dac028420 tx=0x7f5dac003be0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.738+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/886032777 shutdown_connections 2024-06-10T00:35:35.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.738+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/886032777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf088a7a0 0x564cf088a260 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.738+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/886032777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf0890880 0x564cf08960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.738+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/886032777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088ff60 0x564cf0891370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.738+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/886032777 >> 172.21.15.62:0/886032777 conn(0x564cf07b7080 msgr2=0x564cf0890c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:35.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.739+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/886032777 shutdown_connections 2024-06-10T00:35:35.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.739+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/886032777 wait complete. 2024-06-10T00:35:35.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.739+0000 7f5dc46cfb80 1 -- start start 2024-06-10T00:35:35.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.740+0000 7f5dc46cfb80 1 Processor -- start 2024-06-10T00:35:35.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.740+0000 7f5dc46cfb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088a7a0 0x564cf08a6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.740+0000 7f5dc46cfb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf088ff60 0x564cf08a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.740+0000 7f5dc46cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf0890880 0x564cf090c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.740+0000 7f5dc46cfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41bea10 con 0x564cf0890880 2024-06-10T00:35:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.740+0000 7f5dc46cfb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41be340 con 0x564cf088a7a0 2024-06-10T00:35:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.740+0000 7f5dc46cfb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41bb540 con 0x564cf088ff60 2024-06-10T00:35:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.740+0000 7f5dc46cfb80 10 client.0 _ll_register_callbacks cb 0x564cf07715b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.741+0000 7f5db3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf0890880 0x564cf090c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.741+0000 7f5db3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf0890880 0x564cf090c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43128/0 (socket says 172.21.15.62:43128) 2024-06-10T00:35:35.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.741+0000 7f5db2ffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf088ff60 0x564cf08a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.741+0000 7f5db3fff700 1 -- 172.21.15.62:0/921773824 learned_addr learned my addr 172.21.15.62:0/921773824 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:35.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.741+0000 7f5db37fe700 1 --2- 172.21.15.62:0/921773824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088a7a0 0x564cf08a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.742+0000 7f5db3fff700 1 -- 172.21.15.62:0/921773824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf088ff60 msgr2=0x564cf08a6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.742+0000 7f5db3fff700 1 --2- 172.21.15.62:0/921773824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf088ff60 0x564cf08a6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.742+0000 7f5db3fff700 1 -- 172.21.15.62:0/921773824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088a7a0 msgr2=0x564cf08a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.742+0000 7f5db3fff700 1 --2- 172.21.15.62:0/921773824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088a7a0 0x564cf08a6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.742+0000 7f5db3fff700 1 -- 172.21.15.62:0/921773824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cf090cb60 con 0x564cf0890880 2024-06-10T00:35:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.742+0000 7f5db3fff700 1 --2- 172.21.15.62:0/921773824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf0890880 0x564cf090c5d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5da80104d0 tx=0x7f5da80107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd10138680d024a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.742+0000 7f5d92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.742+0000 7f5d92ffd700 1 -- 172.21.15.62:0/921773824 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8018070 con 0x564cf0890880 2024-06-10T00:35:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.742+0000 7f5d92ffd700 1 -- 172.21.15.62:0/921773824 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f5da8014430 con 0x564cf0890880 2024-06-10T00:35:35.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.742+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/921773824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dac007b70 con 0x564cf0890880 2024-06-10T00:35:35.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.743+0000 7f5d92ffd700 1 -- 172.21.15.62:0/921773824 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5da8022050 con 0x564cf0890880 2024-06-10T00:35:35.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.743+0000 7f5dc46cfb80 10 client.16212 fetch_fsmap learned FSMap version 30 2024-06-10T00:35:35.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.743+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/921773824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564cf090de40 con 0x564cf0890880 2024-06-10T00:35:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.743+0000 7f5d92ffd700 1 -- 172.21.15.62:0/921773824 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dac007b70 con 0x564cf0890880 2024-06-10T00:35:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.744+0000 7f5d92ffd700 1 -- 172.21.15.62:0/921773824 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5da801d730 con 0x564cf0890880 2024-06-10T00:35:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.744+0000 7f5dc46cfb80 10 client.16212 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T00:35:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.744+0000 7f5dc46cfb80 10 client.16212 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:35:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.744+0000 7f5dc46cfb80 20 client.16212 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.744+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/921773824 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x564cf08a3ca0 0x564cf0910640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.744+0000 7f5dc46cfb80 4 client.16212 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:35:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.744+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/921773824 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564cf07e8d50 con 0x564cf08a3ca0 2024-06-10T00:35:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.744+0000 7f5db37fe700 1 --2- 172.21.15.62:0/921773824 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x564cf08a3ca0 0x564cf0910640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.745+0000 7f5d92ffd700 10 client.16212 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:35:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.745+0000 7f5db37fe700 1 --2- 172.21.15.62:0/921773824 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x564cf08a3ca0 0x564cf0910640 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7b31b6e80d2e72f server_cookie=994b8f127a103ba2 in_seq=0 out_seq=0 2024-06-10T00:35:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.745+0000 7f5d92ffd700 1 -- 172.21.15.62:0/921773824 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564cf07e8d50 con 0x564cf08a3ca0 2024-06-10T00:35:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.745+0000 7f5d92ffd700 10 client.16212 handle_command_reply: tid=0 2024-06-10T00:35:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.745+0000 7f5dc46cfb80 1 client.16212 shutdown 2024-06-10T00:35:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.746+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/921773824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf0890880 msgr2=0x564cf090c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.746+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/921773824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf0890880 0x564cf090c5d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5da80104d0 tx=0x7f5da80107e0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.746+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/921773824 shutdown_connections 2024-06-10T00:35:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.746+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/921773824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564cf0890880 0x564cf090c5d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.746+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/921773824 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x564cf08a3ca0 0x564cf0910640 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.746+0000 7f5d92ffd700 0 client.16212 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:35:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.746+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/921773824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564cf088a7a0 0x564cf08a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.746+0000 7f5dc46cfb80 1 --2- 172.21.15.62:0/921773824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564cf088ff60 0x564cf08a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.746+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/921773824 >> 172.21.15.62:0/921773824 conn(0x564cf07b7080 msgr2=0x564cf088e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.746+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/921773824 shutdown_connections 2024-06-10T00:35:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.747+0000 7f5dc46cfb80 1 -- 172.21.15.62:0/921773824 wait complete. 2024-06-10T00:35:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.747+0000 7f5dc46cfb80 20 client.16212 trim_cache size 0 max 16384 2024-06-10T00:35:35.748 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:35:35.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.748+0000 7f5db9e4c700 1 -- 172.21.15.62:0/4108451466 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5d9c0a2f90 msgr2=0x7f5d9c0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:35.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.748+0000 7f5db9e4c700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5d9c0a2f90 0x7f5d9c0a5450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009e60 tx=0x7f5dac006020 comp rx=0 tx=0).stop 2024-06-10T00:35:35.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.748+0000 7f5db9e4c700 1 -- 172.21.15.62:0/4108451466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5db4109db0 msgr2=0x7f5db41add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:35.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.748+0000 7f5db9e4c700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5db4109db0 0x7f5db41add50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5da400b450 tx=0x7f5da400b760 comp rx=0 tx=0).stop 2024-06-10T00:35:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.749+0000 7f5db9e4c700 1 -- 172.21.15.62:0/4108451466 shutdown_connections 2024-06-10T00:35:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.749+0000 7f5db9e4c700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5d9c0a2f90 0x7f5d9c0a5450 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.749+0000 7f5db9e4c700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db41155e0 0x7f5db41140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.749+0000 7f5db9e4c700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5db4109490 0x7f5db41ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.749+0000 7f5db9e4c700 1 --2- 172.21.15.62:0/4108451466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5db4109db0 0x7f5db41add50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.749+0000 7f5db9e4c700 1 -- 172.21.15.62:0/4108451466 >> 172.21.15.62:0/4108451466 conn(0x7f5db406d600 msgr2=0x7f5db4071090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.749+0000 7f5db9e4c700 1 -- 172.21.15.62:0/4108451466 shutdown_connections 2024-06-10T00:35:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:35.749+0000 7f5db9e4c700 1 -- 172.21.15.62:0/4108451466 wait complete. 2024-06-10T00:35:36.267 INFO:tasks.workunit.client.0.smithi062.stdout:fs setup took 13 secs 2024-06-10T00:35:37.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:37 smithi062 ceph-mon[32390]: pgmap v86: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 814 MiB/s wr, 332 op/s 2024-06-10T00:35:37.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:37 smithi138 ceph-mon[36549]: pgmap v86: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 814 MiB/s wr, 332 op/s 2024-06-10T00:35:37.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:37 smithi105 ceph-mon[36321]: pgmap v86: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 814 MiB/s wr, 332 op/s 2024-06-10T00:35:37.934 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T00:35:38.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:38 smithi138 ceph-mon[36549]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:35:38.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:38 smithi105 ceph-mon[36321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:35:38.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:38 smithi062 ceph-mon[32390]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:35:38.704 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:35:39.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:39 smithi105 ceph-mon[36321]: pgmap v87: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 704 MiB/s wr, 257 op/s 2024-06-10T00:35:39.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:39 smithi062 ceph-mon[32390]: pgmap v87: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 704 MiB/s wr, 257 op/s 2024-06-10T00:35:39.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:39 smithi138 ceph-mon[36549]: pgmap v87: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 704 MiB/s wr, 257 op/s 2024-06-10T00:35:40.381 INFO:tasks.workunit.client.0.smithi062.stdout:Syncing()...4 sec 2024-06-10T00:35:40.382 INFO:tasks.workunit.client.0.smithi062.stdout:Starting Actual Benchmark At: Mon Jun 10 00:35:40 2024 2024-06-10T00:35:40.382 INFO:tasks.workunit.client.0.smithi062.stdout: 2024-06-10T00:35:41.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.026+0000 7f762ecb7700 1 -- 172.21.15.62:0/91908194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76281096f0 msgr2=0x7f7628109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:41.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.026+0000 7f762ecb7700 1 --2- 172.21.15.62:0/91908194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76281096f0 0x7f7628109ad0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76240098f0 tx=0x7f7624009c00 comp rx=0 tx=0).stop 2024-06-10T00:35:41.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.026+0000 7f762ecb7700 1 -- 172.21.15.62:0/91908194 shutdown_connections 2024-06-10T00:35:41.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.026+0000 7f762ecb7700 1 --2- 172.21.15.62:0/91908194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762810dd80 0x7f762807a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.026+0000 7f762ecb7700 1 --2- 172.21.15.62:0/91908194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76281096f0 0x7f7628109ad0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.026+0000 7f762ecb7700 1 --2- 172.21.15.62:0/91908194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f762810a0a0 0x7f762810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.026+0000 7f762ecb7700 1 -- 172.21.15.62:0/91908194 >> 172.21.15.62:0/91908194 conn(0x7f762806d470 msgr2=0x7f762806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:41.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.027+0000 7f762ecb7700 1 -- 172.21.15.62:0/91908194 shutdown_connections 2024-06-10T00:35:41.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.027+0000 7f762ecb7700 1 -- 172.21.15.62:0/91908194 wait complete. 2024-06-10T00:35:41.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.028+0000 7f762ecb7700 1 Processor -- start 2024-06-10T00:35:41.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.028+0000 7f762ecb7700 1 -- start start 2024-06-10T00:35:41.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.028+0000 7f762ecb7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762810a0a0 0x7f7628084e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.028+0000 7f762ecb7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f762810dd80 0x7f762807eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.028+0000 7f762ecb7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f762807f430 0x7f762807f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.028+0000 7f762ecb7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f762807da80 con 0x7f762810a0a0 2024-06-10T00:35:41.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.028+0000 7f762ecb7700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f762807d900 con 0x7f762807f430 2024-06-10T00:35:41.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.028+0000 7f762ecb7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f762807dc00 con 0x7f762810dd80 2024-06-10T00:35:41.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.029+0000 7f762dcb5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762810a0a0 0x7f7628084e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:41.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.029+0000 7f762dcb5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762810a0a0 0x7f7628084e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47362/0 (socket says 172.21.15.62:47362) 2024-06-10T00:35:41.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.029+0000 7f762dcb5700 1 -- 172.21.15.62:0/2208736272 learned_addr learned my addr 172.21.15.62:0/2208736272 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:41.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.029+0000 7f762d4b4700 1 --2- 172.21.15.62:0/2208736272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f762810dd80 0x7f762807eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:41.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.030+0000 7f762dcb5700 1 -- 172.21.15.62:0/2208736272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f762810dd80 msgr2=0x7f762807eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:41.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.030+0000 7f762dcb5700 1 --2- 172.21.15.62:0/2208736272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f762810dd80 0x7f762807eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.030+0000 7f762dcb5700 1 -- 172.21.15.62:0/2208736272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f762807f430 msgr2=0x7f762807f8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T00:35:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.030+0000 7f762dcb5700 1 --2- 172.21.15.62:0/2208736272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f762807f430 0x7f762807f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.030+0000 7f762dcb5700 1 -- 172.21.15.62:0/2208736272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7628131990 con 0x7f762810a0a0 2024-06-10T00:35:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.031+0000 7f762dcb5700 1 --2- 172.21.15.62:0/2208736272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762810a0a0 0x7f7628084e70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7624000c00 tx=0x7f762400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.032+0000 7f761effd700 1 -- 172.21.15.62:0/2208736272 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762400fbb0 con 0x7f762810a0a0 2024-06-10T00:35:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.032+0000 7f762ecb7700 1 -- 172.21.15.62:0/2208736272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7628131c20 con 0x7f762810a0a0 2024-06-10T00:35:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.032+0000 7f762ecb7700 1 -- 172.21.15.62:0/2208736272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76281320b0 con 0x7f762810a0a0 2024-06-10T00:35:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.032+0000 7f761effd700 1 -- 172.21.15.62:0/2208736272 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7624004bb0 con 0x7f762810a0a0 2024-06-10T00:35:41.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.032+0000 7f761effd700 1 -- 172.21.15.62:0/2208736272 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624017e00 con 0x7f762810a0a0 2024-06-10T00:35:41.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.041+0000 7f761effd700 1 -- 172.21.15.62:0/2208736272 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f762400b5c0 con 0x7f762810a0a0 2024-06-10T00:35:41.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.043+0000 7f763a79cb80 1 Processor -- start 2024-06-10T00:35:41.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.043+0000 7f761effd700 1 --2- 172.21.15.62:0/2208736272 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f76140a2f20 0x7f76140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.044+0000 7f763a79cb80 1 -- start start 2024-06-10T00:35:41.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.044+0000 7f762e4b6700 1 --2- 172.21.15.62:0/2208736272 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f76140a2f20 0x7f76140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:41.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.044+0000 7f763a79cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a7558f0 0x561b1a755cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.044+0000 7f763a79cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a756210 0x561b1a74fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.044+0000 7f763a79cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a750080 0x561b1a7504c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.044+0000 7f763a79cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561b1a6a1810 con 0x561b1a750080 2024-06-10T00:35:41.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.044+0000 7f763a79cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561b1a649720 con 0x561b1a756210 2024-06-10T00:35:41.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.044+0000 7f763a79cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561b1a69d7d0 con 0x561b1a7558f0 2024-06-10T00:35:41.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.045+0000 7f762d4b4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a756210 0x561b1a74fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:41.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.045+0000 7f762d4b4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a756210 0x561b1a74fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:59618/0 (socket says 172.21.15.62:59618) 2024-06-10T00:35:41.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.045+0000 7f762d4b4700 1 -- 172.21.15.62:0/1718884045 learned_addr learned my addr 172.21.15.62:0/1718884045 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:41.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.046+0000 7f762e4b6700 1 --2- 172.21.15.62:0/1718884045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a750080 0x561b1a7504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:41.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.046+0000 7f762e4b6700 1 -- 172.21.15.62:0/1718884045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a7558f0 msgr2=0x561b1a755cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:41.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.046+0000 7f762e4b6700 1 --2- 172.21.15.62:0/1718884045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a7558f0 0x561b1a755cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.046+0000 7f762e4b6700 1 -- 172.21.15.62:0/1718884045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a756210 msgr2=0x561b1a74fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:41.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.046+0000 7f762e4b6700 1 --2- 172.21.15.62:0/1718884045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a756210 0x561b1a74fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.046+0000 7f762e4b6700 1 -- 172.21.15.62:0/1718884045 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b1a63ff50 con 0x561b1a750080 2024-06-10T00:35:41.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.047+0000 7f761effd700 1 -- 172.21.15.62:0/2208736272 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f76240bb550 con 0x7f762810a0a0 2024-06-10T00:35:41.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.047+0000 7f762e4b6700 1 --2- 172.21.15.62:0/2208736272 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f76140a2f20 0x7f76140a53e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7618009a70 tx=0x7f7618009260 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:41.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.048+0000 7f762e4b6700 1 --2- 172.21.15.62:0/1718884045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a750080 0x561b1a7504c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f761800e8d0 tx=0x7f76180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2202df044cf42957 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:41.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.048+0000 7f7613fff700 1 -- 172.21.15.62:0/1718884045 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761801f070 con 0x561b1a750080 2024-06-10T00:35:41.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.048+0000 7f7613fff700 1 -- 172.21.15.62:0/1718884045 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7618005020 con 0x561b1a750080 2024-06-10T00:35:41.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.049+0000 7f763a79cb80 1 -- 172.21.15.62:0/1718884045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a750080 msgr2=0x561b1a7504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:41.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.049+0000 7f763a79cb80 1 --2- 172.21.15.62:0/1718884045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a750080 0x561b1a7504c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f761800e8d0 tx=0x7f76180047c0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.050+0000 7f763a79cb80 1 -- 172.21.15.62:0/1718884045 shutdown_connections 2024-06-10T00:35:41.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.050+0000 7f763a79cb80 1 --2- 172.21.15.62:0/1718884045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a750080 0x561b1a7504c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.050+0000 7f763a79cb80 1 --2- 172.21.15.62:0/1718884045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a7558f0 0x561b1a755cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.050+0000 7f763a79cb80 1 --2- 172.21.15.62:0/1718884045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a756210 0x561b1a74fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.050+0000 7f763a79cb80 1 -- 172.21.15.62:0/1718884045 >> 172.21.15.62:0/1718884045 conn(0x561b1a67acc0 msgr2=0x561b1a74aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:41.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.050+0000 7f763a79cb80 1 -- 172.21.15.62:0/1718884045 shutdown_connections 2024-06-10T00:35:41.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.050+0000 7f763a79cb80 1 -- 172.21.15.62:0/1718884045 wait complete. 2024-06-10T00:35:41.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.051+0000 7f763a79cb80 1 -- start start 2024-06-10T00:35:41.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.051+0000 7f763a79cb80 1 Processor -- start 2024-06-10T00:35:41.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.052+0000 7f763a79cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a7558f0 0x561b1a75bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.052+0000 7f763a79cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a756210 0x561b1a75c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.052+0000 7f763a79cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a759170 0x561b1a7595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.052+0000 7f763a79cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561b1a75cc20 con 0x561b1a7558f0 2024-06-10T00:35:41.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.052+0000 7f763a79cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561b1a641bf0 con 0x561b1a759170 2024-06-10T00:35:41.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.052+0000 7f763a79cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561b1a75cda0 con 0x561b1a756210 2024-06-10T00:35:41.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.053+0000 7f763a79cb80 10 client.0 _ll_register_callbacks cb 0x561b1a63b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:41.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.053+0000 7f762dcb5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a7558f0 0x561b1a75bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:41.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.053+0000 7f762dcb5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a7558f0 0x561b1a75bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47378/0 (socket says 172.21.15.62:47378) 2024-06-10T00:35:41.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.053+0000 7f762dcb5700 1 -- 172.21.15.62:0/1048408071 learned_addr learned my addr 172.21.15.62:0/1048408071 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:41.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.054+0000 7f762dcb5700 1 -- 172.21.15.62:0/1048408071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a756210 msgr2=0x561b1a75c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:41.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.054+0000 7f762dcb5700 1 --2- 172.21.15.62:0/1048408071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a756210 0x561b1a75c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.054+0000 7f762dcb5700 1 -- 172.21.15.62:0/1048408071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a759170 msgr2=0x561b1a7595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:41.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.054+0000 7f762dcb5700 1 --2- 172.21.15.62:0/1048408071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a759170 0x561b1a7595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.054+0000 7f762dcb5700 1 -- 172.21.15.62:0/1048408071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b1a5f08d0 con 0x561b1a7558f0 2024-06-10T00:35:41.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.055+0000 7f762dcb5700 1 --2- 172.21.15.62:0/1048408071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a7558f0 0x561b1a75bcc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7624009fd0 tx=0x7f762409c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7096462b718768f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:41.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.055+0000 7f76117fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:35:41.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.055+0000 7f76117fa700 1 -- 172.21.15.62:0/1048408071 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762409c9a0 con 0x561b1a7558f0 2024-06-10T00:35:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.055+0000 7f763a79cb80 1 -- 172.21.15.62:0/1048408071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561b1a6aed10 con 0x561b1a7558f0 2024-06-10T00:35:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.056+0000 7f76117fa700 1 -- 172.21.15.62:0/1048408071 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f7624017620 con 0x561b1a7558f0 2024-06-10T00:35:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.061+0000 7f76117fa700 1 -- 172.21.15.62:0/1048408071 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f76240a0d50 con 0x561b1a7558f0 2024-06-10T00:35:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.061+0000 7f763a79cb80 10 client.16224 fetch_fsmap learned FSMap version 31 2024-06-10T00:35:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.061+0000 7f763a79cb80 1 -- 172.21.15.62:0/1048408071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561b1a767800 con 0x561b1a7558f0 2024-06-10T00:35:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.065+0000 7f76117fa700 1 -- 172.21.15.62:0/1048408071 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561b1a6aed10 con 0x561b1a7558f0 2024-06-10T00:35:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.065+0000 7f76117fa700 1 -- 172.21.15.62:0/1048408071 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f762409cc00 con 0x561b1a7558f0 2024-06-10T00:35:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.066+0000 7f763a79cb80 10 client.16224 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T00:35:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.066+0000 7f763a79cb80 10 client.16224 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:35:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.066+0000 7f763a79cb80 20 client.16224 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.066+0000 7f763a79cb80 1 --2- 172.21.15.62:0/1048408071 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561b1a769cd0 0x561b1a76c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.066+0000 7f763a79cb80 4 client.16224 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:35:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.066+0000 7f763a79cb80 1 -- 172.21.15.62:0/1048408071 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b1a6a9290 con 0x561b1a769cd0 2024-06-10T00:35:41.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.067+0000 7f762d4b4700 1 --2- 172.21.15.62:0/1048408071 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561b1a769cd0 0x561b1a76c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:41.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.078+0000 7f76117fa700 10 client.16224 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:35:41.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.078+0000 7f762d4b4700 1 --2- 172.21.15.62:0/1048408071 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561b1a769cd0 0x561b1a76c140 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f84844fa4033f0b9 server_cookie=8882230a8d8ed81a in_seq=0 out_seq=0 2024-06-10T00:35:41.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.082+0000 7f76117fa700 1 -- 172.21.15.62:0/1048408071 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561b1a6a9290 con 0x561b1a769cd0 2024-06-10T00:35:41.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.082+0000 7f76117fa700 10 client.16224 handle_command_reply: tid=0 2024-06-10T00:35:41.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.082+0000 7f763a79cb80 1 client.16224 shutdown 2024-06-10T00:35:41.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.082+0000 7f763a79cb80 1 -- 172.21.15.62:0/1048408071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a7558f0 msgr2=0x561b1a75bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:41.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.082+0000 7f763a79cb80 1 --2- 172.21.15.62:0/1048408071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a7558f0 0x561b1a75bcc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7624009fd0 tx=0x7f762409c610 comp rx=0 tx=0).stop 2024-06-10T00:35:41.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.083+0000 7f763a79cb80 1 -- 172.21.15.62:0/1048408071 shutdown_connections 2024-06-10T00:35:41.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.083+0000 7f763a79cb80 1 --2- 172.21.15.62:0/1048408071 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561b1a769cd0 0x561b1a76c140 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.083+0000 7f763a79cb80 1 --2- 172.21.15.62:0/1048408071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a7558f0 0x561b1a75bcc0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.083+0000 7f763a79cb80 1 --2- 172.21.15.62:0/1048408071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a759170 0x561b1a7595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.083+0000 7f763a79cb80 1 --2- 172.21.15.62:0/1048408071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a756210 0x561b1a75c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.083+0000 7f763a79cb80 1 -- 172.21.15.62:0/1048408071 >> 172.21.15.62:0/1048408071 conn(0x561b1a67acc0 msgr2=0x561b1a74eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:41.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.084+0000 7f76117fa700 0 client.16224 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:35:41.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.086+0000 7f763a79cb80 1 -- 172.21.15.62:0/1048408071 shutdown_connections 2024-06-10T00:35:41.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.086+0000 7f763a79cb80 1 -- 172.21.15.62:0/1048408071 wait complete. 2024-06-10T00:35:41.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.086+0000 7f763a79cb80 20 client.16224 trim_cache size 0 max 16384 2024-06-10T00:35:41.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.135+0000 7f763a79cb80 1 Processor -- start 2024-06-10T00:35:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.140+0000 7f763a79cb80 1 -- start start 2024-06-10T00:35:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.140+0000 7f763a79cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a7558f0 0x561b1a765fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.141+0000 7f763a79cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a756210 0x561b1a74a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.141+0000 7f763a79cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a759170 0x561b1a74aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.141+0000 7f763a79cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f760c00ef10 con 0x561b1a7558f0 2024-06-10T00:35:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.141+0000 7f763a79cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f760c010a80 con 0x561b1a759170 2024-06-10T00:35:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.141+0000 7f763a79cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f760c00f5e0 con 0x561b1a756210 2024-06-10T00:35:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.141+0000 7f762dcb5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a7558f0 0x561b1a765fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.141+0000 7f762dcb5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a7558f0 0x561b1a765fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47390/0 (socket says 172.21.15.62:47390) 2024-06-10T00:35:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.141+0000 7f762dcb5700 1 -- 172.21.15.62:0/1077610869 learned_addr learned my addr 172.21.15.62:0/1077610869 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.142+0000 7f762dcb5700 1 -- 172.21.15.62:0/1077610869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a756210 msgr2=0x561b1a74a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:35:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.142+0000 7f762dcb5700 1 --2- 172.21.15.62:0/1077610869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a756210 0x561b1a74a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.142+0000 7f762dcb5700 1 -- 172.21.15.62:0/1077610869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a759170 msgr2=0x561b1a74aa20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:35:41.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.142+0000 7f762dcb5700 1 --2- 172.21.15.62:0/1077610869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a759170 0x561b1a74aa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.142+0000 7f762dcb5700 1 -- 172.21.15.62:0/1077610869 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b1a756040 con 0x561b1a7558f0 2024-06-10T00:35:41.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.142+0000 7f762dcb5700 1 --2- 172.21.15.62:0/1077610869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a7558f0 0x561b1a765fa0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7624098e00 tx=0x7f762409c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855401b5072a71a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:41.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.146+0000 7f76137fe700 1 -- 172.21.15.62:0/1077610869 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76240a5580 con 0x561b1a7558f0 2024-06-10T00:35:41.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.146+0000 7f76137fe700 1 -- 172.21.15.62:0/1077610869 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76240a5250 con 0x561b1a7558f0 2024-06-10T00:35:41.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.148+0000 7f763a79cb80 1 -- 172.21.15.62:0/1077610869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a7558f0 msgr2=0x561b1a765fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:41.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.148+0000 7f763a79cb80 1 --2- 172.21.15.62:0/1077610869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a7558f0 0x561b1a765fa0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7624098e00 tx=0x7f762409c430 comp rx=0 tx=0).stop 2024-06-10T00:35:41.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.161+0000 7f763a79cb80 1 -- 172.21.15.62:0/1077610869 shutdown_connections 2024-06-10T00:35:41.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.161+0000 7f763a79cb80 1 --2- 172.21.15.62:0/1077610869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a7558f0 0x561b1a765fa0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.161+0000 7f763a79cb80 1 --2- 172.21.15.62:0/1077610869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a759170 0x561b1a74aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.161+0000 7f763a79cb80 1 --2- 172.21.15.62:0/1077610869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a756210 0x561b1a74a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.161+0000 7f763a79cb80 1 -- 172.21.15.62:0/1077610869 >> 172.21.15.62:0/1077610869 conn(0x561b1a67acc0 msgr2=0x561b1a757900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:41.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.165+0000 7f763a79cb80 1 -- 172.21.15.62:0/1077610869 shutdown_connections 2024-06-10T00:35:41.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.165+0000 7f763a79cb80 1 -- 172.21.15.62:0/1077610869 wait complete. 2024-06-10T00:35:41.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.166+0000 7f763a79cb80 1 -- start start 2024-06-10T00:35:41.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.166+0000 7f763a79cb80 1 Processor -- start 2024-06-10T00:35:41.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.167+0000 7f763a79cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a7558f0 0x561b1a762990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.167+0000 7f763a79cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a756210 0x561b1a75d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.168+0000 7f763a79cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a759170 0x561b1a75d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.168+0000 7f762d4b4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a756210 0x561b1a75d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:41.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.168+0000 7f762d4b4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a756210 0x561b1a75d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:59650/0 (socket says 172.21.15.62:59650) 2024-06-10T00:35:41.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.168+0000 7f762d4b4700 1 -- 172.21.15.62:0/2300313290 learned_addr learned my addr 172.21.15.62:0/2300313290 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:35:41.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.168+0000 7f763a79cb80 1 -- 172.21.15.62:0/2300313290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f760c010380 con 0x561b1a759170 2024-06-10T00:35:41.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.168+0000 7f763a79cb80 1 -- 172.21.15.62:0/2300313290 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f760c00fcb0 con 0x561b1a756210 2024-06-10T00:35:41.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.168+0000 7f763a79cb80 1 -- 172.21.15.62:0/2300313290 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f760c00ceb0 con 0x561b1a7558f0 2024-06-10T00:35:41.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.168+0000 7f763a79cb80 10 client.0 _ll_register_callbacks cb 0x561b1a63b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:35:41.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.171+0000 7f762dcb5700 1 --2- 172.21.15.62:0/2300313290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a7558f0 0x561b1a762990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:41.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.173+0000 7f762dcb5700 1 -- 172.21.15.62:0/2300313290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a756210 msgr2=0x561b1a75d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:41.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.173+0000 7f762dcb5700 1 --2- 172.21.15.62:0/2300313290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a756210 0x561b1a75d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.173+0000 7f762dcb5700 1 -- 172.21.15.62:0/2300313290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a759170 msgr2=0x561b1a75d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:35:41.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.173+0000 7f762dcb5700 1 --2- 172.21.15.62:0/2300313290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a759170 0x561b1a75d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.173+0000 7f762dcb5700 1 -- 172.21.15.62:0/2300313290 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b1a7d2a60 con 0x561b1a7558f0 2024-06-10T00:35:41.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.174+0000 7f762dcb5700 1 --2- 172.21.15.62:0/2300313290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a7558f0 0x561b1a762990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f762409c6a0 tx=0x7f76240a32f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=889cf953d0f8244f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:35:41.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.174+0000 7f76117fa700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:35:41.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.177+0000 7f76117fa700 1 -- 172.21.15.62:0/2300313290 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76240a5580 con 0x561b1a7558f0 2024-06-10T00:35:41.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.177+0000 7f763a79cb80 1 -- 172.21.15.62:0/2300313290 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561b1a523250 con 0x561b1a7558f0 2024-06-10T00:35:41.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.177+0000 7f76117fa700 1 -- 172.21.15.62:0/2300313290 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f76240bb190 con 0x561b1a7558f0 2024-06-10T00:35:41.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.178+0000 7f76117fa700 1 -- 172.21.15.62:0/2300313290 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f76240046d0 con 0x561b1a7558f0 2024-06-10T00:35:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.179+0000 7f763a79cb80 10 client.25735 fetch_fsmap learned FSMap version 31 2024-06-10T00:35:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.179+0000 7f763a79cb80 1 -- 172.21.15.62:0/2300313290 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561b1a7d2a60 con 0x561b1a7558f0 2024-06-10T00:35:41.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.183+0000 7f76117fa700 1 -- 172.21.15.62:0/2300313290 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561b1a523250 con 0x561b1a7558f0 2024-06-10T00:35:41.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.183+0000 7f76117fa700 1 -- 172.21.15.62:0/2300313290 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f762409b860 con 0x561b1a7558f0 2024-06-10T00:35:41.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.183+0000 7f763a79cb80 10 client.25735 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T00:35:41.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.183+0000 7f763a79cb80 10 client.25735 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:35:41.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.183+0000 7f763a79cb80 20 client.25735 populate_metadata read hostname 'smithi062' 2024-06-10T00:35:41.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.183+0000 7f763a79cb80 1 --2- 172.21.15.62:0/2300313290 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561b1a7d3e90 0x561b1a7d62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:35:41.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.183+0000 7f763a79cb80 4 client.25735 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:35:41.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.183+0000 7f763a79cb80 1 -- 172.21.15.62:0/2300313290 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b1a672b50 con 0x561b1a7d3e90 2024-06-10T00:35:41.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.188+0000 7f762d4b4700 1 --2- 172.21.15.62:0/2300313290 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561b1a7d3e90 0x561b1a7d62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:35:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.190+0000 7f762d4b4700 1 --2- 172.21.15.62:0/2300313290 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561b1a7d3e90 0x561b1a7d62b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eae2161096a89556 server_cookie=cf8cf73ba1464f15 in_seq=0 out_seq=0 2024-06-10T00:35:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.192+0000 7f76117fa700 10 client.25735 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:35:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.192+0000 7f76117fa700 1 -- 172.21.15.62:0/2300313290 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b1a672b50 con 0x561b1a7d3e90 2024-06-10T00:35:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.192+0000 7f76117fa700 10 client.25735 handle_command_reply: tid=0 2024-06-10T00:35:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.192+0000 7f763a79cb80 1 client.25735 shutdown 2024-06-10T00:35:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.198+0000 7f763a79cb80 1 -- 172.21.15.62:0/2300313290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a7558f0 msgr2=0x561b1a762990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.198+0000 7f763a79cb80 1 --2- 172.21.15.62:0/2300313290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a7558f0 0x561b1a762990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f762409c6a0 tx=0x7f76240a32f0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.198+0000 7f763a79cb80 1 -- 172.21.15.62:0/2300313290 shutdown_connections 2024-06-10T00:35:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.198+0000 7f763a79cb80 1 --2- 172.21.15.62:0/2300313290 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561b1a7d3e90 0x561b1a7d62b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.198+0000 7f763a79cb80 1 --2- 172.21.15.62:0/2300313290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561b1a759170 0x561b1a75d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.198+0000 7f763a79cb80 1 --2- 172.21.15.62:0/2300313290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b1a7558f0 0x561b1a762990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.198+0000 7f763a79cb80 1 --2- 172.21.15.62:0/2300313290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b1a756210 0x561b1a75d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.198+0000 7f763a79cb80 1 -- 172.21.15.62:0/2300313290 >> 172.21.15.62:0/2300313290 conn(0x561b1a67acc0 msgr2=0x561b1a7532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.198+0000 7f76117fa700 0 client.25735 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:35:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.200+0000 7f763a79cb80 1 -- 172.21.15.62:0/2300313290 shutdown_connections 2024-06-10T00:35:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.200+0000 7f763a79cb80 1 -- 172.21.15.62:0/2300313290 wait complete. 2024-06-10T00:35:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.200+0000 7f763a79cb80 20 client.25735 trim_cache size 0 max 16384 2024-06-10T00:35:41.202 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:35:41.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.202+0000 7f762ecb7700 1 -- 172.21.15.62:0/2208736272 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f76140a2f20 msgr2=0x7f76140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.202+0000 7f762ecb7700 1 --2- 172.21.15.62:0/2208736272 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f76140a2f20 0x7f76140a53e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7618009a70 tx=0x7f7618009260 comp rx=0 tx=0).stop 2024-06-10T00:35:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.202+0000 7f762ecb7700 1 -- 172.21.15.62:0/2208736272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762810a0a0 msgr2=0x7f7628084e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:35:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.202+0000 7f762ecb7700 1 --2- 172.21.15.62:0/2208736272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762810a0a0 0x7f7628084e70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7624000c00 tx=0x7f762400f690 comp rx=0 tx=0).stop 2024-06-10T00:35:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.202+0000 7f762ecb7700 1 -- 172.21.15.62:0/2208736272 shutdown_connections 2024-06-10T00:35:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.202+0000 7f762ecb7700 1 --2- 172.21.15.62:0/2208736272 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f76140a2f20 0x7f76140a53e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.202+0000 7f762ecb7700 1 --2- 172.21.15.62:0/2208736272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762810a0a0 0x7f7628084e70 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.202+0000 7f762ecb7700 1 --2- 172.21.15.62:0/2208736272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f762807f430 0x7f762807f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.203+0000 7f762ecb7700 1 --2- 172.21.15.62:0/2208736272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f762810dd80 0x7f762807eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:35:41.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.203+0000 7f762ecb7700 1 -- 172.21.15.62:0/2208736272 >> 172.21.15.62:0/2208736272 conn(0x7f762806d470 msgr2=0x7f762810aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:35:41.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.203+0000 7f762ecb7700 1 -- 172.21.15.62:0/2208736272 shutdown_connections 2024-06-10T00:35:41.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:35:41.203+0000 7f762ecb7700 1 -- 172.21.15.62:0/2208736272 wait complete. 2024-06-10T00:35:41.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:41 smithi105 ceph-mon[36321]: pgmap v88: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 704 MiB/s wr, 257 op/s 2024-06-10T00:35:41.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:41 smithi062 ceph-mon[32390]: pgmap v88: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 704 MiB/s wr, 257 op/s 2024-06-10T00:35:41.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:41 smithi138 ceph-mon[36549]: pgmap v88: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 704 MiB/s wr, 257 op/s 2024-06-10T00:35:43.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:43 smithi062 ceph-mon[32390]: pgmap v89: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 904 MiB/s wr, 355 op/s 2024-06-10T00:35:43.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:43 smithi062 ceph-mon[32390]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:35:43.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:43 smithi105 ceph-mon[36321]: pgmap v89: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 904 MiB/s wr, 355 op/s 2024-06-10T00:35:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:43 smithi105 ceph-mon[36321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:35:43.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:43 smithi138 ceph-mon[36549]: pgmap v89: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 904 MiB/s wr, 355 op/s 2024-06-10T00:35:43.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:43 smithi138 ceph-mon[36549]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:35:45.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:45 smithi105 ceph-mon[36321]: pgmap v90: 129 pgs: 129 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 613 MiB/s wr, 332 op/s 2024-06-10T00:35:45.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:45 smithi062 ceph-mon[32390]: pgmap v90: 129 pgs: 129 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 613 MiB/s wr, 332 op/s 2024-06-10T00:35:45.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:45 smithi138 ceph-mon[36549]: pgmap v90: 129 pgs: 129 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 613 MiB/s wr, 332 op/s 2024-06-10T00:35:47.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:47 smithi105 ceph-mon[36321]: pgmap v91: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 616 MiB/s wr, 793 op/s 2024-06-10T00:35:47.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:47 smithi062 ceph-mon[32390]: pgmap v91: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 616 MiB/s wr, 793 op/s 2024-06-10T00:35:47.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:47 smithi138 ceph-mon[36549]: pgmap v91: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 616 MiB/s wr, 793 op/s 2024-06-10T00:35:49.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:49 smithi105 ceph-mon[36321]: pgmap v92: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 277 MiB/s wr, 4.41k op/s 2024-06-10T00:35:49.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:49 smithi062 ceph-mon[32390]: pgmap v92: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 277 MiB/s wr, 4.41k op/s 2024-06-10T00:35:49.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:49 smithi138 ceph-mon[36549]: pgmap v92: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 277 MiB/s wr, 4.41k op/s 2024-06-10T00:35:51.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:51 smithi105 ceph-mon[36321]: pgmap v93: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 279 MiB/s wr, 4.67k op/s 2024-06-10T00:35:51.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:51 smithi062 ceph-mon[32390]: pgmap v93: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 279 MiB/s wr, 4.67k op/s 2024-06-10T00:35:51.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:51 smithi138 ceph-mon[36549]: pgmap v93: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 279 MiB/s wr, 4.67k op/s 2024-06-10T00:35:53.296 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T00:35:53.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:53 smithi105 ceph-mon[36321]: pgmap v94: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 302 MiB/s wr, 8.12k op/s 2024-06-10T00:35:53.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:53 smithi062 ceph-mon[32390]: pgmap v94: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 302 MiB/s wr, 8.12k op/s 2024-06-10T00:35:53.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:53 smithi138 ceph-mon[36549]: pgmap v94: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 302 MiB/s wr, 8.12k op/s 2024-06-10T00:35:53.704 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:35:55.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:55 smithi105 ceph-mon[36321]: pgmap v95: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 108 MiB/s wr, 8.88k op/s 2024-06-10T00:35:55.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:55 smithi062 ceph-mon[32390]: pgmap v95: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 108 MiB/s wr, 8.88k op/s 2024-06-10T00:35:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:55 smithi138 ceph-mon[36549]: pgmap v95: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 108 MiB/s wr, 8.88k op/s 2024-06-10T00:35:57.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:57 smithi105 ceph-mon[36321]: pgmap v96: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 57 MiB/s wr, 9.23k op/s 2024-06-10T00:35:57.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:57 smithi062 ceph-mon[32390]: pgmap v96: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 57 MiB/s wr, 9.23k op/s 2024-06-10T00:35:57.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:57 smithi138 ceph-mon[36549]: pgmap v96: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 57 MiB/s wr, 9.23k op/s 2024-06-10T00:35:59.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:35:59 smithi105 ceph-mon[36321]: pgmap v97: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 72 MiB/s wr, 11.59k op/s 2024-06-10T00:35:59.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:35:59 smithi062 ceph-mon[32390]: pgmap v97: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 72 MiB/s wr, 11.59k op/s 2024-06-10T00:35:59.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:35:59 smithi138 ceph-mon[36549]: pgmap v97: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 72 MiB/s wr, 11.59k op/s 2024-06-10T00:36:01.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:01 smithi105 ceph-mon[36321]: pgmap v98: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 57 MiB/s wr, 8.77k op/s 2024-06-10T00:36:01.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:01 smithi062 ceph-mon[32390]: pgmap v98: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 57 MiB/s wr, 8.77k op/s 2024-06-10T00:36:01.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:01 smithi138 ceph-mon[36549]: pgmap v98: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 57 MiB/s wr, 8.77k op/s 2024-06-10T00:36:03.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:03 smithi105 ceph-mon[36321]: pgmap v99: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 71 MiB/s wr, 10.95k op/s 2024-06-10T00:36:03.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:03 smithi062 ceph-mon[32390]: pgmap v99: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 71 MiB/s wr, 10.95k op/s 2024-06-10T00:36:03.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:03 smithi138 ceph-mon[36549]: pgmap v99: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 71 MiB/s wr, 10.95k op/s 2024-06-10T00:36:04.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:04 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:36:04.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:04 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:36:04.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:04 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:36:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:05 smithi105 ceph-mon[36321]: pgmap v100: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 52 MiB/s wr, 8.16k op/s 2024-06-10T00:36:05.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:05 smithi062 ceph-mon[32390]: pgmap v100: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 52 MiB/s wr, 8.16k op/s 2024-06-10T00:36:05.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:05 smithi138 ceph-mon[36549]: pgmap v100: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 52 MiB/s wr, 8.16k op/s 2024-06-10T00:36:07.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:07 smithi105 ceph-mon[36321]: pgmap v101: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 52 MiB/s wr, 8.19k op/s 2024-06-10T00:36:07.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:07 smithi062 ceph-mon[32390]: pgmap v101: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 52 MiB/s wr, 8.19k op/s 2024-06-10T00:36:07.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:07 smithi138 ceph-mon[36549]: pgmap v101: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 52 MiB/s wr, 8.19k op/s 2024-06-10T00:36:09.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:09 smithi105 ceph-mon[36321]: pgmap v102: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 71 MiB/s wr, 10.78k op/s 2024-06-10T00:36:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:09 smithi062 ceph-mon[32390]: pgmap v102: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 71 MiB/s wr, 10.78k op/s 2024-06-10T00:36:09.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:09 smithi138 ceph-mon[36549]: pgmap v102: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 71 MiB/s wr, 10.78k op/s 2024-06-10T00:36:10.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:10 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:36:10.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:10 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:36:10.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:10 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:36:10.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:10 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:36:10.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:10 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:36:10.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:10 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:36:10.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:10 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:36:10.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:10 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:36:10.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:10 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:36:11.413 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:11 smithi062 ceph-mon[32390]: pgmap v103: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 MiB/s wr, 8.34k op/s 2024-06-10T00:36:11.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:11 smithi105 ceph-mon[36321]: pgmap v103: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 MiB/s wr, 8.34k op/s 2024-06-10T00:36:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:11 smithi138 ceph-mon[36549]: pgmap v103: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 56 MiB/s wr, 8.34k op/s 2024-06-10T00:36:13.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:13 smithi062 ceph-mon[32390]: pgmap v104: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 MiB/s wr, 10.14k op/s 2024-06-10T00:36:13.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:13 smithi105 ceph-mon[36321]: pgmap v104: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 MiB/s wr, 10.14k op/s 2024-06-10T00:36:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:13 smithi138 ceph-mon[36549]: pgmap v104: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 67 MiB/s wr, 10.14k op/s 2024-06-10T00:36:15.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:15 smithi062 ceph-mon[32390]: pgmap v105: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 59 MiB/s wr, 8.90k op/s 2024-06-10T00:36:15.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:15 smithi105 ceph-mon[36321]: pgmap v105: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 59 MiB/s wr, 8.90k op/s 2024-06-10T00:36:15.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:15 smithi138 ceph-mon[36549]: pgmap v105: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 59 MiB/s wr, 8.90k op/s 2024-06-10T00:36:17.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:17 smithi105 ceph-mon[36321]: pgmap v106: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 58 MiB/s wr, 8.72k op/s 2024-06-10T00:36:17.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:17 smithi062 ceph-mon[32390]: pgmap v106: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 58 MiB/s wr, 8.72k op/s 2024-06-10T00:36:17.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:17 smithi138 ceph-mon[36549]: pgmap v106: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 58 MiB/s wr, 8.72k op/s 2024-06-10T00:36:18.310 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:36:18.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.779+0000 7ffba1d2e700 1 -- 172.21.15.62:0/4275051936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb9c10ae30 msgr2=0x7ffb9c10b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:36:18.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.779+0000 7ffba1d2e700 1 --2- 172.21.15.62:0/4275051936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb9c10ae30 0x7ffb9c10b210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0098f0 tx=0x7ffb8c009c00 comp rx=0 tx=0).stop 2024-06-10T00:36:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.779+0000 7ffba1d2e700 1 -- 172.21.15.62:0/4275051936 shutdown_connections 2024-06-10T00:36:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.779+0000 7ffba1d2e700 1 --2- 172.21.15.62:0/4275051936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb9c10a0f0 0x7ffb9c10a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.779+0000 7ffba1d2e700 1 --2- 172.21.15.62:0/4275051936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb9c10ae30 0x7ffb9c10b210 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0098f0 tx=0x7ffb8c009c00 comp rx=0 tx=0).stop 2024-06-10T00:36:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.779+0000 7ffba1d2e700 1 --2- 172.21.15.62:0/4275051936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c109c70 0x7ffb9c10b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.779+0000 7ffba1d2e700 1 -- 172.21.15.62:0/4275051936 >> 172.21.15.62:0/4275051936 conn(0x7ffb9c06d600 msgr2=0x7ffb9c06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:36:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.782+0000 7ffba1d2e700 1 -- 172.21.15.62:0/4275051936 shutdown_connections 2024-06-10T00:36:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.784+0000 7ffba1d2e700 1 -- 172.21.15.62:0/4275051936 wait complete. 2024-06-10T00:36:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.786+0000 7ffba1d2e700 1 Processor -- start 2024-06-10T00:36:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.786+0000 7ffba1d2e700 1 -- start start 2024-06-10T00:36:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.787+0000 7ffba1d2e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb9c109c70 0x7ffb9c1a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.787+0000 7ffba1d2e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb9c10a0f0 0x7ffb9c1a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.787+0000 7ffb9b7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb9c109c70 0x7ffb9c1a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.787+0000 7ffb9b7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb9c109c70 0x7ffb9c1a9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:42252/0 (socket says 172.21.15.62:42252) 2024-06-10T00:36:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.788+0000 7ffb9b7fe700 1 -- 172.21.15.62:0/2784864510 learned_addr learned my addr 172.21.15.62:0/2784864510 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:36:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.788+0000 7ffba1d2e700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c1a37e0 0x7ffb9c1a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.788+0000 7ffba1d2e700 1 -- 172.21.15.62:0/2784864510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c11d330 con 0x7ffb9c1a37e0 2024-06-10T00:36:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.788+0000 7ffba1d2e700 1 -- 172.21.15.62:0/2784864510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c11d1b0 con 0x7ffb9c10a0f0 2024-06-10T00:36:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.788+0000 7ffba1d2e700 1 -- 172.21.15.62:0/2784864510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c11d4b0 con 0x7ffb9c109c70 2024-06-10T00:36:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.788+0000 7ffb9affd700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb9c10a0f0 0x7ffb9c1a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.789+0000 7ffb9bfff700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c1a37e0 0x7ffb9c1a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.790+0000 7ffb9b7fe700 1 -- 172.21.15.62:0/2784864510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb9c10a0f0 msgr2=0x7ffb9c1a9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:36:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.790+0000 7ffb9b7fe700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb9c10a0f0 0x7ffb9c1a9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.790+0000 7ffb9b7fe700 1 -- 172.21.15.62:0/2784864510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c1a37e0 msgr2=0x7ffb9c1a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:36:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.790+0000 7ffb9b7fe700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c1a37e0 0x7ffb9c1a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.790+0000 7ffb9b7fe700 1 -- 172.21.15.62:0/2784864510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9c1a43a0 con 0x7ffb9c109c70 2024-06-10T00:36:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.790+0000 7ffb9affd700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb9c10a0f0 0x7ffb9c1a9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:36:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.790+0000 7ffb9b7fe700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb9c109c70 0x7ffb9c1a9660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009230 tx=0x7ffb8c00fa50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:36:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.792+0000 7ffb98ff9700 1 -- 172.21.15.62:0/2784864510 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c0050c0 con 0x7ffb9c109c70 2024-06-10T00:36:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.792+0000 7ffb98ff9700 1 -- 172.21.15.62:0/2784864510 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb8c00b560 con 0x7ffb9c109c70 2024-06-10T00:36:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.792+0000 7ffb98ff9700 1 -- 172.21.15.62:0/2784864510 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c0050c0 con 0x7ffb9c109c70 2024-06-10T00:36:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.793+0000 7ffba1d2e700 1 -- 172.21.15.62:0/2784864510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb9c1a4630 con 0x7ffb9c109c70 2024-06-10T00:36:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.793+0000 7ffba1d2e700 1 -- 172.21.15.62:0/2784864510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb9c1b0750 con 0x7ffb9c109c70 2024-06-10T00:36:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.798+0000 7ffb98ff9700 1 -- 172.21.15.62:0/2784864510 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ffb8c005260 con 0x7ffb9c109c70 2024-06-10T00:36:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.810+0000 7ffb98ff9700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ffb840a3250 0x7ffb840a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.811+0000 7ffb98ff9700 1 -- 172.21.15.62:0/2784864510 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7ffb8c01d070 con 0x7ffb9c109c70 2024-06-10T00:36:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.811+0000 7ffb9affd700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ffb840a3250 0x7ffb840a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.812+0000 7ffb9affd700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ffb840a3250 0x7ffb840a5710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c072370 tx=0x7ffb90009260 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:36:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.812+0000 7ffbac5b1b80 1 Processor -- start 2024-06-10T00:36:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.814+0000 7ffbac5b1b80 1 -- start start 2024-06-10T00:36:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.814+0000 7ffbac5b1b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b7da0 0x5647839b8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.814+0000 7ffbac5b1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b86c0 0x5647839b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.814+0000 7ffbac5b1b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b3730 0x5647839b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.814+0000 7ffbac5b1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5647838fc810 con 0x5647839b86c0 2024-06-10T00:36:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.814+0000 7ffbac5b1b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5647838a4720 con 0x5647839b3730 2024-06-10T00:36:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.814+0000 7ffbac5b1b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5647838f87d0 con 0x5647839b7da0 2024-06-10T00:36:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.815+0000 7ffb9bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b3730 0x5647839b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.815+0000 7ffb9bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b3730 0x5647839b3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:38374/0 (socket says 172.21.15.62:38374) 2024-06-10T00:36:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.815+0000 7ffb9bfff700 1 -- 172.21.15.62:0/3723521769 learned_addr learned my addr 172.21.15.62:0/3723521769 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:36:18.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.815+0000 7ffb9affd700 1 --2- 172.21.15.62:0/3723521769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b86c0 0x5647839b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.817+0000 7ffb9affd700 1 -- 172.21.15.62:0/3723521769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b7da0 msgr2=0x5647839b8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:36:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.817+0000 7ffb9affd700 1 --2- 172.21.15.62:0/3723521769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b7da0 0x5647839b8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.817+0000 7ffb9affd700 1 -- 172.21.15.62:0/3723521769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b3730 msgr2=0x5647839b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:36:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.817+0000 7ffb9affd700 1 --2- 172.21.15.62:0/3723521769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b3730 0x5647839b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.817+0000 7ffb9affd700 1 -- 172.21.15.62:0/3723521769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56478389af50 con 0x5647839b86c0 2024-06-10T00:36:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.817+0000 7ffb9bfff700 1 --2- 172.21.15.62:0/3723521769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b3730 0x5647839b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:36:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.817+0000 7ffb9affd700 1 --2- 172.21.15.62:0/3723521769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b86c0 0x5647839b31f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ffb900042f0 tx=0x7ffb90014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34106af3f37a7f3c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:36:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.820+0000 7ffb81ffb700 1 -- 172.21.15.62:0/3723521769 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90003d10 con 0x5647839b86c0 2024-06-10T00:36:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.820+0000 7ffb81ffb700 1 -- 172.21.15.62:0/3723521769 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb900268e0 con 0x5647839b86c0 2024-06-10T00:36:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.821+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/3723521769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b86c0 msgr2=0x5647839b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:36:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.821+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/3723521769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b86c0 0x5647839b31f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ffb900042f0 tx=0x7ffb90014000 comp rx=0 tx=0).stop 2024-06-10T00:36:18.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.825+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/3723521769 shutdown_connections 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.825+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/3723521769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b3730 0x5647839b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.825+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/3723521769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b7da0 0x5647839b8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.825+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/3723521769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b86c0 0x5647839b31f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.825+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/3723521769 >> 172.21.15.62:0/3723521769 conn(0x5647838d5cc0 msgr2=0x5647839a5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.826+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/3723521769 shutdown_connections 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.831+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/3723521769 wait complete. 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.831+0000 7ffbac5b1b80 1 -- start start 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.831+0000 7ffbac5b1b80 1 Processor -- start 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.832+0000 7ffbac5b1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b3730 0x5647839bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.832+0000 7ffbac5b1b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b7da0 0x5647839bc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.832+0000 7ffbac5b1b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b86c0 0x5647839bcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.832+0000 7ffbac5b1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5647839c0250 con 0x5647839b3730 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.832+0000 7ffbac5b1b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56478389cbf0 con 0x5647839b86c0 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.832+0000 7ffbac5b1b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5647839c03d0 con 0x5647839b7da0 2024-06-10T00:36:18.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.832+0000 7ffbac5b1b80 10 client.0 _ll_register_callbacks cb 0x564783896920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:36:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.833+0000 7ffb9b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b3730 0x5647839bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.833+0000 7ffb9affd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b7da0 0x5647839bc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.833+0000 7ffb9b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b3730 0x5647839bf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60978/0 (socket says 172.21.15.62:60978) 2024-06-10T00:36:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.833+0000 7ffb9b7fe700 1 -- 172.21.15.62:0/456410141 learned_addr learned my addr 172.21.15.62:0/456410141 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:36:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.835+0000 7ffb9affd700 1 -- 172.21.15.62:0/456410141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b86c0 msgr2=0x5647839bcd20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:36:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.835+0000 7ffb9affd700 1 --2- 172.21.15.62:0/456410141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b86c0 0x5647839bcd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.835+0000 7ffb9affd700 1 -- 172.21.15.62:0/456410141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b3730 msgr2=0x5647839bf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:36:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.835+0000 7ffb9affd700 1 --2- 172.21.15.62:0/456410141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b3730 0x5647839bf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.835+0000 7ffb9affd700 1 -- 172.21.15.62:0/456410141 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56478384b8d0 con 0x5647839b7da0 2024-06-10T00:36:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.836+0000 7ffb9affd700 1 --2- 172.21.15.62:0/456410141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b7da0 0x5647839bc7e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffb900042f0 tx=0x7ffb90003bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a8ded5dcd802140 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:36:18.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.836+0000 7ffb7affd700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:36:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.836+0000 7ffb7affd700 1 -- 172.21.15.62:0/456410141 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9001eac0 con 0x5647839b7da0 2024-06-10T00:36:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.836+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/456410141 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb900268e0 con 0x5647839b7da0 2024-06-10T00:36:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.836+0000 7ffb7affd700 1 -- 172.21.15.62:0/456410141 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7ffb9001ec60 con 0x5647839b7da0 2024-06-10T00:36:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.841+0000 7ffb7affd700 1 -- 172.21.15.62:0/456410141 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffb9001ee60 con 0x5647839b7da0 2024-06-10T00:36:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.841+0000 7ffbac5b1b80 10 client.25744 fetch_fsmap learned FSMap version 32 2024-06-10T00:36:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.841+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/456410141 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5647839cabe0 con 0x5647839b7da0 2024-06-10T00:36:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.844+0000 7ffb7affd700 1 -- 172.21.15.62:0/456410141 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb900268e0 con 0x5647839b7da0 2024-06-10T00:36:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.844+0000 7ffb7affd700 1 -- 172.21.15.62:0/456410141 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7ffb90004830 con 0x5647839b7da0 2024-06-10T00:36:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.844+0000 7ffbac5b1b80 10 client.25744 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T00:36:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.844+0000 7ffbac5b1b80 10 client.25744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:36:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.844+0000 7ffbac5b1b80 20 client.25744 populate_metadata read hostname 'smithi062' 2024-06-10T00:36:18.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.844+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/456410141 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5647839cd0b0 0x5647839cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.844+0000 7ffbac5b1b80 4 client.25744 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:36:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.844+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/456410141 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564783909d10 con 0x5647839cd0b0 2024-06-10T00:36:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.845+0000 7ffb9b7fe700 1 --2- 172.21.15.62:0/456410141 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5647839cd0b0 0x5647839cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.846+0000 7ffb7affd700 10 client.25744 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:36:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.846+0000 7ffb9b7fe700 1 --2- 172.21.15.62:0/456410141 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5647839cd0b0 0x5647839cf4a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f5c779001e9f381 server_cookie=8ae31228d7f094dc in_seq=0 out_seq=0 2024-06-10T00:36:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.849+0000 7ffb7affd700 1 -- 172.21.15.62:0/456410141 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564783909d10 con 0x5647839cd0b0 2024-06-10T00:36:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.849+0000 7ffb7affd700 10 client.25744 handle_command_reply: tid=0 2024-06-10T00:36:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.850+0000 7ffbac5b1b80 1 client.25744 shutdown 2024-06-10T00:36:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.854+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/456410141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b7da0 msgr2=0x5647839bc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:36:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.854+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/456410141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b7da0 0x5647839bc7e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffb900042f0 tx=0x7ffb90003bc0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.856+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/456410141 shutdown_connections 2024-06-10T00:36:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.856+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/456410141 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5647839cd0b0 0x5647839cf4a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.856+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/456410141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b3730 0x5647839bf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.856+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/456410141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b86c0 0x5647839bcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.856+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/456410141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b7da0 0x5647839bc7e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.856+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/456410141 >> 172.21.15.62:0/456410141 conn(0x5647838d5cc0 msgr2=0x5647839a99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:36:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.856+0000 7ffb7affd700 0 client.25744 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:36:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.858+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/456410141 shutdown_connections 2024-06-10T00:36:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.859+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/456410141 wait complete. 2024-06-10T00:36:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.860+0000 7ffbac5b1b80 20 client.25744 trim_cache size 0 max 16384 2024-06-10T00:36:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.892+0000 7ffbac5b1b80 1 Processor -- start 2024-06-10T00:36:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.892+0000 7ffbac5b1b80 1 -- start start 2024-06-10T00:36:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.892+0000 7ffbac5b1b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b3730 0x5647839bc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.893+0000 7ffbac5b1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b7da0 0x56478399f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.893+0000 7ffbac5b1b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b86c0 0x5647839b5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.893+0000 7ffbac5b1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8800ef10 con 0x5647839b7da0 2024-06-10T00:36:18.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.893+0000 7ffbac5b1b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffb88010b10 con 0x5647839b3730 2024-06-10T00:36:18.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.893+0000 7ffbac5b1b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8800f5e0 con 0x5647839b86c0 2024-06-10T00:36:18.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.894+0000 7ffb9affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b7da0 0x56478399f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.894+0000 7ffb9affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b7da0 0x56478399f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60988/0 (socket says 172.21.15.62:60988) 2024-06-10T00:36:18.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.894+0000 7ffb9affd700 1 -- 172.21.15.62:0/4053977164 learned_addr learned my addr 172.21.15.62:0/4053977164 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:36:18.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.894+0000 7ffb9bfff700 1 --2- 172.21.15.62:0/4053977164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b86c0 0x5647839b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.894+0000 7ffb9b7fe700 1 --2- 172.21.15.62:0/4053977164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b3730 0x5647839bc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.896+0000 7ffb9affd700 1 -- 172.21.15.62:0/4053977164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b86c0 msgr2=0x5647839b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:36:18.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.896+0000 7ffb9affd700 1 --2- 172.21.15.62:0/4053977164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b86c0 0x5647839b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.896+0000 7ffb9affd700 1 -- 172.21.15.62:0/4053977164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b3730 msgr2=0x5647839bc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:36:18.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.896+0000 7ffb9affd700 1 --2- 172.21.15.62:0/4053977164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b3730 0x5647839bc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.896+0000 7ffb9affd700 1 -- 172.21.15.62:0/4053977164 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647839bad50 con 0x5647839b7da0 2024-06-10T00:36:18.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.903+0000 7ffb9affd700 1 --2- 172.21.15.62:0/4053977164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b7da0 0x56478399f510 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ffb900042c0 tx=0x7ffb90009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7726cde84c97412c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:36:18.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.907+0000 7ffb817fa700 1 -- 172.21.15.62:0/4053977164 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9001ea00 con 0x5647839b7da0 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.907+0000 7ffb817fa700 1 -- 172.21.15.62:0/4053977164 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9001eba0 con 0x5647839b7da0 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.908+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/4053977164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b7da0 msgr2=0x56478399f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.908+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/4053977164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b7da0 0x56478399f510 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ffb900042c0 tx=0x7ffb90009e90 comp rx=0 tx=0).stop 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.914+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/4053977164 shutdown_connections 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.914+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/4053977164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b86c0 0x5647839b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.914+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/4053977164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b3730 0x5647839bc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.914+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/4053977164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b7da0 0x56478399f510 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.914+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/4053977164 >> 172.21.15.62:0/4053977164 conn(0x5647838d5cc0 msgr2=0x56478399ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.916+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/4053977164 shutdown_connections 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.916+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/4053977164 wait complete. 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.916+0000 7ffbac5b1b80 1 -- start start 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.916+0000 7ffbac5b1b80 1 Processor -- start 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.919+0000 7ffbac5b1b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b3730 0x5647839d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.919+0000 7ffbac5b1b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b7da0 0x5647839d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.919+0000 7ffbac5b1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b86c0 0x5647839b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.919+0000 7ffbac5b1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffb88010380 con 0x5647839b86c0 2024-06-10T00:36:18.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.919+0000 7ffbac5b1b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8800fcb0 con 0x5647839b7da0 2024-06-10T00:36:18.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.919+0000 7ffbac5b1b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8800ceb0 con 0x5647839b3730 2024-06-10T00:36:18.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.919+0000 7ffbac5b1b80 10 client.0 _ll_register_callbacks cb 0x564783896920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:36:18.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.920+0000 7ffb9affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b7da0 0x5647839d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.921+0000 7ffb9affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b7da0 0x5647839d2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:38410/0 (socket says 172.21.15.62:38410) 2024-06-10T00:36:18.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.921+0000 7ffb9affd700 1 -- 172.21.15.62:0/3150207802 learned_addr learned my addr 172.21.15.62:0/3150207802 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:36:18.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.921+0000 7ffb9b7fe700 1 --2- 172.21.15.62:0/3150207802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b3730 0x5647839d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.921+0000 7ffb9bfff700 1 --2- 172.21.15.62:0/3150207802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b86c0 0x5647839b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.923+0000 7ffb9affd700 1 -- 172.21.15.62:0/3150207802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b3730 msgr2=0x5647839d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:36:18.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.923+0000 7ffb9affd700 1 --2- 172.21.15.62:0/3150207802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b3730 0x5647839d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.923+0000 7ffb9affd700 1 -- 172.21.15.62:0/3150207802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b86c0 msgr2=0x5647839b8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:36:18.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.923+0000 7ffb9affd700 1 --2- 172.21.15.62:0/3150207802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b86c0 0x5647839b8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.923+0000 7ffb9affd700 1 -- 172.21.15.62:0/3150207802 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647839b9140 con 0x5647839b7da0 2024-06-10T00:36:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.925+0000 7ffb9affd700 1 --2- 172.21.15.62:0/3150207802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b7da0 0x5647839d2630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009410 tx=0x7ffb900092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4343dc1b2b40bc66 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:36:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.925+0000 7ffb7affd700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:36:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.925+0000 7ffb7affd700 1 -- 172.21.15.62:0/3150207802 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9001ea00 con 0x5647839b7da0 2024-06-10T00:36:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.925+0000 7ffb7affd700 1 -- 172.21.15.62:0/3150207802 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7ffb900048d0 con 0x5647839b7da0 2024-06-10T00:36:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.925+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/3150207802 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb9001eba0 con 0x5647839b7da0 2024-06-10T00:36:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.926+0000 7ffb7affd700 1 -- 172.21.15.62:0/3150207802 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffb900051e0 con 0x5647839b7da0 2024-06-10T00:36:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.926+0000 7ffbac5b1b80 10 client.25736 fetch_fsmap learned FSMap version 32 2024-06-10T00:36:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.926+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/3150207802 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564783a37090 con 0x5647839b7da0 2024-06-10T00:36:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.929+0000 7ffb7affd700 1 -- 172.21.15.62:0/3150207802 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9001eba0 con 0x5647839b7da0 2024-06-10T00:36:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.929+0000 7ffb7affd700 1 -- 172.21.15.62:0/3150207802 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7ffb9003c400 con 0x5647839b7da0 2024-06-10T00:36:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.929+0000 7ffbac5b1b80 10 client.25736 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T00:36:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.929+0000 7ffbac5b1b80 10 client.25736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:36:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.929+0000 7ffbac5b1b80 20 client.25736 populate_metadata read hostname 'smithi062' 2024-06-10T00:36:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.929+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/3150207802 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564783a37740 0x564783a39b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:36:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.929+0000 7ffbac5b1b80 4 client.25736 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:36:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.929+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/3150207802 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564783904290 con 0x564783a37740 2024-06-10T00:36:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.932+0000 7ffb9b7fe700 1 --2- 172.21.15.62:0/3150207802 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564783a37740 0x564783a39b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:36:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.936+0000 7ffb9b7fe700 1 --2- 172.21.15.62:0/3150207802 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564783a37740 0x564783a39b60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f6054ee9d5ae6f server_cookie=afd01dc65167f069 in_seq=0 out_seq=0 2024-06-10T00:36:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.936+0000 7ffb7affd700 10 client.25736 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:36:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.937+0000 7ffb7affd700 1 -- 172.21.15.62:0/3150207802 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564783904290 con 0x564783a37740 2024-06-10T00:36:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.937+0000 7ffb7affd700 10 client.25736 handle_command_reply: tid=0 2024-06-10T00:36:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.937+0000 7ffbac5b1b80 1 client.25736 shutdown 2024-06-10T00:36:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.938+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/3150207802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b7da0 msgr2=0x5647839d2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:36:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.938+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/3150207802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b7da0 0x5647839d2630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009410 tx=0x7ffb900092d0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.939+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/3150207802 shutdown_connections 2024-06-10T00:36:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.939+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/3150207802 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564783a37740 0x564783a39b60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.939+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/3150207802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5647839b86c0 0x5647839b8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.939+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/3150207802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647839b3730 0x5647839d20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.939+0000 7ffbac5b1b80 1 --2- 172.21.15.62:0/3150207802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647839b7da0 0x5647839d2630 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.939+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/3150207802 >> 172.21.15.62:0/3150207802 conn(0x5647838d5cc0 msgr2=0x5647839a99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:36:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.939+0000 7ffb7affd700 0 client.25736 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:36:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.940+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/3150207802 shutdown_connections 2024-06-10T00:36:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.940+0000 7ffbac5b1b80 1 -- 172.21.15.62:0/3150207802 wait complete. 2024-06-10T00:36:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.940+0000 7ffbac5b1b80 20 client.25736 trim_cache size 0 max 16384 2024-06-10T00:36:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.941+0000 7ffba1d2e700 1 -- 172.21.15.62:0/2784864510 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ffb840a3250 msgr2=0x7ffb840a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:36:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.941+0000 7ffba1d2e700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ffb840a3250 0x7ffb840a5710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c072370 tx=0x7ffb90009260 comp rx=0 tx=0).stop 2024-06-10T00:36:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.941+0000 7ffba1d2e700 1 -- 172.21.15.62:0/2784864510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb9c109c70 msgr2=0x7ffb9c1a9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:36:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.941+0000 7ffba1d2e700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb9c109c70 0x7ffb9c1a9660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009230 tx=0x7ffb8c00fa50 comp rx=0 tx=0).stop 2024-06-10T00:36:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.941+0000 7ffba1d2e700 1 -- 172.21.15.62:0/2784864510 shutdown_connections 2024-06-10T00:36:18.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.941+0000 7ffba1d2e700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ffb840a3250 0x7ffb840a5710 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.941+0000 7ffba1d2e700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c1a37e0 0x7ffb9c1a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.941+0000 7ffba1d2e700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb9c109c70 0x7ffb9c1a9660 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.941+0000 7ffba1d2e700 1 --2- 172.21.15.62:0/2784864510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb9c10a0f0 0x7ffb9c1a9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:36:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.941+0000 7ffba1d2e700 1 -- 172.21.15.62:0/2784864510 >> 172.21.15.62:0/2784864510 conn(0x7ffb9c06d600 msgr2=0x7ffb9c1160b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:36:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.941+0000 7ffba1d2e700 1 -- 172.21.15.62:0/2784864510 shutdown_connections 2024-06-10T00:36:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:16.942+0000 7ffba1d2e700 1 -- 172.21.15.62:0/2784864510 wait complete. 2024-06-10T00:36:19.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:19 smithi105 ceph-mon[36321]: pgmap v107: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 66 MiB/s wr, 10.35k op/s 2024-06-10T00:36:19.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:19 smithi062 ceph-mon[32390]: pgmap v107: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 66 MiB/s wr, 10.35k op/s 2024-06-10T00:36:19.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:19 smithi138 ceph-mon[36549]: pgmap v107: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 66 MiB/s wr, 10.35k op/s 2024-06-10T00:36:21.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:21 smithi105 ceph-mon[36321]: pgmap v108: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 46 MiB/s wr, 7.49k op/s 2024-06-10T00:36:21.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:21 smithi062 ceph-mon[32390]: pgmap v108: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 46 MiB/s wr, 7.49k op/s 2024-06-10T00:36:21.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:21 smithi138 ceph-mon[36549]: pgmap v108: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 46 MiB/s wr, 7.49k op/s 2024-06-10T00:36:23.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:23 smithi105 ceph-mon[36321]: pgmap v109: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 65 MiB/s wr, 10.71k op/s 2024-06-10T00:36:23.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:23 smithi062 ceph-mon[32390]: pgmap v109: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 65 MiB/s wr, 10.71k op/s 2024-06-10T00:36:23.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:23 smithi138 ceph-mon[36549]: pgmap v109: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 65 MiB/s wr, 10.71k op/s 2024-06-10T00:36:25.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:25 smithi105 ceph-mon[36321]: pgmap v110: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 54 MiB/s wr, 9.01k op/s 2024-06-10T00:36:25.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:25 smithi062 ceph-mon[32390]: pgmap v110: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 54 MiB/s wr, 9.01k op/s 2024-06-10T00:36:25.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:25 smithi138 ceph-mon[36549]: pgmap v110: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 54 MiB/s wr, 9.01k op/s 2024-06-10T00:36:27.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:27 smithi105 ceph-mon[36321]: pgmap v111: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 46 MiB/s wr, 7.83k op/s 2024-06-10T00:36:27.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:27 smithi062 ceph-mon[32390]: pgmap v111: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 46 MiB/s wr, 7.83k op/s 2024-06-10T00:36:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:27 smithi138 ceph-mon[36549]: pgmap v111: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 46 MiB/s wr, 7.83k op/s 2024-06-10T00:36:29.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:29 smithi105 ceph-mon[36321]: pgmap v112: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 54 MiB/s wr, 9.31k op/s 2024-06-10T00:36:29.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:29 smithi062 ceph-mon[32390]: pgmap v112: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 54 MiB/s wr, 9.31k op/s 2024-06-10T00:36:29.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:29 smithi138 ceph-mon[36549]: pgmap v112: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 54 MiB/s wr, 9.31k op/s 2024-06-10T00:36:31.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:31 smithi105 ceph-mon[36321]: pgmap v113: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 44 MiB/s wr, 7.37k op/s 2024-06-10T00:36:31.827 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T00:36:31.828 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T00:36:31.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:31 smithi062 ceph-mon[32390]: pgmap v113: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 44 MiB/s wr, 7.37k op/s 2024-06-10T00:36:31.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:31 smithi138 ceph-mon[36549]: pgmap v113: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 44 MiB/s wr, 7.37k op/s 2024-06-10T00:36:32.135 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:36:33.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:33 smithi105 ceph-mon[36321]: pgmap v114: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 48 MiB/s wr, 8.06k op/s 2024-06-10T00:36:33.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:33 smithi062 ceph-mon[32390]: pgmap v114: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 48 MiB/s wr, 8.06k op/s 2024-06-10T00:36:33.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:33 smithi138 ceph-mon[36549]: pgmap v114: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 48 MiB/s wr, 8.06k op/s 2024-06-10T00:36:35.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:35 smithi105 ceph-mon[36321]: pgmap v115: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 29 MiB/s wr, 4.98k op/s 2024-06-10T00:36:35.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:35 smithi062 ceph-mon[32390]: pgmap v115: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 29 MiB/s wr, 4.98k op/s 2024-06-10T00:36:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:35 smithi138 ceph-mon[36549]: pgmap v115: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 29 MiB/s wr, 4.98k op/s 2024-06-10T00:36:37.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:37 smithi105 ceph-mon[36321]: pgmap v116: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 26 MiB/s wr, 4.33k op/s 2024-06-10T00:36:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:37 smithi062 ceph-mon[32390]: pgmap v116: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 26 MiB/s wr, 4.33k op/s 2024-06-10T00:36:37.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:37 smithi138 ceph-mon[36549]: pgmap v116: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 26 MiB/s wr, 4.33k op/s 2024-06-10T00:36:39.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:39 smithi105 ceph-mon[36321]: pgmap v117: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 28 MiB/s wr, 4.78k op/s 2024-06-10T00:36:39.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:39 smithi062 ceph-mon[32390]: pgmap v117: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 28 MiB/s wr, 4.78k op/s 2024-06-10T00:36:39.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:39 smithi138 ceph-mon[36549]: pgmap v117: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 28 MiB/s wr, 4.78k op/s 2024-06-10T00:36:41.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:41 smithi105 ceph-mon[36321]: pgmap v118: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 19 MiB/s wr, 3.19k op/s 2024-06-10T00:36:41.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:41 smithi062 ceph-mon[32390]: pgmap v118: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 19 MiB/s wr, 3.19k op/s 2024-06-10T00:36:41.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:41 smithi138 ceph-mon[36549]: pgmap v118: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 19 MiB/s wr, 3.19k op/s 2024-06-10T00:36:43.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:43 smithi105 ceph-mon[36321]: pgmap v119: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 22 MiB/s wr, 3.67k op/s 2024-06-10T00:36:43.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:43 smithi062 ceph-mon[32390]: pgmap v119: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 22 MiB/s wr, 3.67k op/s 2024-06-10T00:36:43.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:43 smithi138 ceph-mon[36549]: pgmap v119: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 22 MiB/s wr, 3.67k op/s 2024-06-10T00:36:45.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:45 smithi105 ceph-mon[36321]: pgmap v120: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 23 MiB/s wr, 3.74k op/s 2024-06-10T00:36:45.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:45 smithi062 ceph-mon[32390]: pgmap v120: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 23 MiB/s wr, 3.74k op/s 2024-06-10T00:36:45.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:45 smithi138 ceph-mon[36549]: pgmap v120: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 23 MiB/s wr, 3.74k op/s 2024-06-10T00:36:47.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:47 smithi105 ceph-mon[36321]: pgmap v121: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 21 MiB/s wr, 3.34k op/s 2024-06-10T00:36:47.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:47 smithi062 ceph-mon[32390]: pgmap v121: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 21 MiB/s wr, 3.34k op/s 2024-06-10T00:36:47.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:47 smithi138 ceph-mon[36549]: pgmap v121: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 21 MiB/s wr, 3.34k op/s 2024-06-10T00:36:49.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:49 smithi105 ceph-mon[36321]: pgmap v122: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 26 MiB/s wr, 4.17k op/s 2024-06-10T00:36:49.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:49 smithi062 ceph-mon[32390]: pgmap v122: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 26 MiB/s wr, 4.17k op/s 2024-06-10T00:36:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:49 smithi138 ceph-mon[36549]: pgmap v122: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 26 MiB/s wr, 4.17k op/s 2024-06-10T00:36:51.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:51 smithi105 ceph-mon[36321]: pgmap v123: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 30 MiB/s wr, 4.69k op/s 2024-06-10T00:36:51.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:51 smithi062 ceph-mon[32390]: pgmap v123: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 30 MiB/s wr, 4.69k op/s 2024-06-10T00:36:51.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:51 smithi138 ceph-mon[36549]: pgmap v123: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 30 MiB/s wr, 4.69k op/s 2024-06-10T00:36:53.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:53 smithi105 ceph-mon[36321]: pgmap v124: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 44 MiB/s wr, 6.91k op/s 2024-06-10T00:36:53.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:53 smithi062 ceph-mon[32390]: pgmap v124: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 44 MiB/s wr, 6.91k op/s 2024-06-10T00:36:53.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:53 smithi138 ceph-mon[36549]: pgmap v124: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 44 MiB/s wr, 6.91k op/s 2024-06-10T00:36:55.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:55 smithi105 ceph-mon[36321]: pgmap v125: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 43 MiB/s wr, 6.82k op/s 2024-06-10T00:36:55.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:55 smithi062 ceph-mon[32390]: pgmap v125: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 43 MiB/s wr, 6.82k op/s 2024-06-10T00:36:55.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:55 smithi138 ceph-mon[36549]: pgmap v125: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 43 MiB/s wr, 6.82k op/s 2024-06-10T00:36:57.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:57 smithi105 ceph-mon[36321]: pgmap v126: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 40 MiB/s wr, 6.31k op/s 2024-06-10T00:36:57.833 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:57 smithi062 ceph-mon[32390]: pgmap v126: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 40 MiB/s wr, 6.31k op/s 2024-06-10T00:36:57.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:57 smithi138 ceph-mon[36549]: pgmap v126: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 40 MiB/s wr, 6.31k op/s 2024-06-10T00:36:59.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:36:59 smithi062 ceph-mon[32390]: pgmap v127: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 58 MiB/s wr, 8.83k op/s 2024-06-10T00:36:59.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:36:59 smithi105 ceph-mon[36321]: pgmap v127: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 58 MiB/s wr, 8.83k op/s 2024-06-10T00:36:59.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:36:59 smithi138 ceph-mon[36549]: pgmap v127: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 58 MiB/s wr, 8.83k op/s 2024-06-10T00:37:00.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:00 smithi105 ceph-mon[36321]: scrub queued for path: / 2024-06-10T00:37:00.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:00 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:00.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:00 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:37:00.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:00 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:37:00.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:00 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:37:00.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:00 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:37:00.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:00 smithi062 ceph-mon[32390]: scrub queued for path: / 2024-06-10T00:37:00.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:00 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:00.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:00 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:37:00.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:00 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:37:00.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:00 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:37:00.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:00 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:37:00.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:00 smithi138 ceph-mon[36549]: scrub queued for path: / 2024-06-10T00:37:00.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:00 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:00.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:00 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:37:00.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:00 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:37:00.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:00 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:37:00.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:00 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:37:01.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.681+0000 7f7b6c462700 1 -- 172.21.15.62:0/231261059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b64110c70 msgr2=0x7f7b64109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:37:01.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.681+0000 7f7b6c462700 1 --2- 172.21.15.62:0/231261059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b64110c70 0x7f7b64109490 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7b600098f0 tx=0x7f7b60009c00 comp rx=0 tx=0).stop 2024-06-10T00:37:01.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.681+0000 7f7b6c462700 1 -- 172.21.15.62:0/231261059 shutdown_connections 2024-06-10T00:37:01.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.681+0000 7f7b6c462700 1 --2- 172.21.15.62:0/231261059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b64110c70 0x7f7b64109490 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.681+0000 7f7b6c462700 1 --2- 172.21.15.62:0/231261059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b6411bc70 0x7f7b6411e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.681+0000 7f7b6c462700 1 --2- 172.21.15.62:0/231261059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b641099d0 0x7f7b64109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.681+0000 7f7b6c462700 1 -- 172.21.15.62:0/231261059 >> 172.21.15.62:0/231261059 conn(0x7f7b6406d600 msgr2=0x7f7b6406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:37:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.681+0000 7f7b6c462700 1 -- 172.21.15.62:0/231261059 shutdown_connections 2024-06-10T00:37:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.682+0000 7f7b6c462700 1 -- 172.21.15.62:0/231261059 wait complete. 2024-06-10T00:37:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.683+0000 7f7b6c462700 1 Processor -- start 2024-06-10T00:37:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.683+0000 7f7b6c462700 1 -- start start 2024-06-10T00:37:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.683+0000 7f7b6c462700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b641099d0 0x7f7b641ad810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.683+0000 7f7b6c462700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b64110c70 0x7f7b641add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.683+0000 7f7b6c462700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b6411bc70 0x7f7b641a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.683+0000 7f7b6c462700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7b64121680 con 0x7f7b6411bc70 2024-06-10T00:37:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.683+0000 7f7b6c462700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7b64121500 con 0x7f7b64110c70 2024-06-10T00:37:01.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.683+0000 7f7b6c462700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b64121800 con 0x7f7b641099d0 2024-06-10T00:37:01.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.684+0000 7f7b699fd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b64110c70 0x7f7b641add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.684+0000 7f7b699fd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b64110c70 0x7f7b641add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:36472/0 (socket says 172.21.15.62:36472) 2024-06-10T00:37:01.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.684+0000 7f7b699fd700 1 -- 172.21.15.62:0/946624474 learned_addr learned my addr 172.21.15.62:0/946624474 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:37:01.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.685+0000 7f7b6a9ff700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b6411bc70 0x7f7b641a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.685+0000 7f7b699fd700 1 -- 172.21.15.62:0/946624474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b641099d0 msgr2=0x7f7b641ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:37:01.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.685+0000 7f7b6a1fe700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b641099d0 0x7f7b641ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.685+0000 7f7b699fd700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b641099d0 0x7f7b641ad810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.685+0000 7f7b699fd700 1 -- 172.21.15.62:0/946624474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b6411bc70 msgr2=0x7f7b641a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:37:01.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.685+0000 7f7b699fd700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b6411bc70 0x7f7b641a7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.685+0000 7f7b699fd700 1 -- 172.21.15.62:0/946624474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b641a8180 con 0x7f7b64110c70 2024-06-10T00:37:01.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.685+0000 7f7b6a9ff700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b6411bc70 0x7f7b641a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:37:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.685+0000 7f7b6a1fe700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b641099d0 0x7f7b641ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:37:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.686+0000 7f7b699fd700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b64110c70 0x7f7b641add50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400d980 tx=0x7f7b5400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:37:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.687+0000 7f7b5b7fe700 1 -- 172.21.15.62:0/946624474 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54016070 con 0x7f7b64110c70 2024-06-10T00:37:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.687+0000 7f7b5b7fe700 1 -- 172.21.15.62:0/946624474 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b5400de60 con 0x7f7b64110c70 2024-06-10T00:37:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.687+0000 7f7b5b7fe700 1 -- 172.21.15.62:0/946624474 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5400f620 con 0x7f7b64110c70 2024-06-10T00:37:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.688+0000 7f7b6c462700 1 -- 172.21.15.62:0/946624474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b641a8470 con 0x7f7b64110c70 2024-06-10T00:37:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.688+0000 7f7b6c462700 1 -- 172.21.15.62:0/946624474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b64111450 con 0x7f7b64110c70 2024-06-10T00:37:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.694+0000 7f7b76ce5b80 1 Processor -- start 2024-06-10T00:37:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.696+0000 7f7b76ce5b80 1 -- start start 2024-06-10T00:37:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.696+0000 7f7b76ce5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192eccfe0 0x55a192ecd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.696+0000 7f7b76ce5b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ecd900 0x55a192ec72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.696+0000 7f7b76ce5b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192ec7820 0x55a192ec7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.696+0000 7f7b76ce5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a192db9fe0 con 0x55a192eccfe0 2024-06-10T00:37:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.696+0000 7f7b76ce5b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a192e188d0 con 0x55a192ecd900 2024-06-10T00:37:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.696+0000 7f7b76ce5b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a192d804e0 con 0x55a192ec7820 2024-06-10T00:37:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.697+0000 7f7b6a1fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ecd900 0x55a192ec72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.697+0000 7f7b6a1fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ecd900 0x55a192ec72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:36486/0 (socket says 172.21.15.62:36486) 2024-06-10T00:37:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.697+0000 7f7b6a1fe700 1 -- 172.21.15.62:0/3816619847 learned_addr learned my addr 172.21.15.62:0/3816619847 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:37:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.697+0000 7f7b699fd700 1 --2- 172.21.15.62:0/3816619847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192ec7820 0x55a192ec7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.697+0000 7f7b6a9ff700 1 --2- 172.21.15.62:0/3816619847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192eccfe0 0x55a192ecd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.698+0000 7f7b6a1fe700 1 -- 172.21.15.62:0/3816619847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192ec7820 msgr2=0x55a192ec7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:37:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.698+0000 7f7b6a1fe700 1 --2- 172.21.15.62:0/3816619847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192ec7820 0x55a192ec7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.698+0000 7f7b6a1fe700 1 -- 172.21.15.62:0/3816619847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192eccfe0 msgr2=0x55a192ecd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:37:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.698+0000 7f7b6a1fe700 1 --2- 172.21.15.62:0/3816619847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192eccfe0 0x55a192ecd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.698+0000 7f7b6a1fe700 1 -- 172.21.15.62:0/3816619847 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a192d67950 con 0x55a192ecd900 2024-06-10T00:37:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.699+0000 7f7b5b7fe700 1 -- 172.21.15.62:0/946624474 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7b54019070 con 0x7f7b64110c70 2024-06-10T00:37:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.699+0000 7f7b6a1fe700 1 --2- 172.21.15.62:0/3816619847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ecd900 0x55a192ec72e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009230 tx=0x7f7b60005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2c91e1b388ddd18 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:37:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.700+0000 7f7b5b7fe700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7b500a2f60 0x7f7b500a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.700+0000 7f7b5b7fe700 1 -- 172.21.15.62:0/946624474 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f7b540cf240 con 0x7f7b64110c70 2024-06-10T00:37:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.700+0000 7f7b6a9ff700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7b500a2f60 0x7f7b500a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.701+0000 7f7b58d70700 1 -- 172.21.15.62:0/3816619847 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60025070 con 0x55a192ecd900 2024-06-10T00:37:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.701+0000 7f7b58d70700 1 -- 172.21.15.62:0/3816619847 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6000f460 con 0x55a192ecd900 2024-06-10T00:37:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.702+0000 7f7b58d70700 1 -- 172.21.15.62:0/3816619847 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60004a30 con 0x55a192ecd900 2024-06-10T00:37:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.702+0000 7f7b6a9ff700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7b500a2f60 0x7f7b500a5420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b6406bc00 tx=0x7f7b5c00b040 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:37:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.702+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/3816619847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ecd900 msgr2=0x55a192ec72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:37:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.702+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/3816619847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ecd900 0x55a192ec72e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7b60009230 tx=0x7f7b60005040 comp rx=0 tx=0).stop 2024-06-10T00:37:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.702+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/3816619847 shutdown_connections 2024-06-10T00:37:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.702+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/3816619847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192eccfe0 0x55a192ecd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.702+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/3816619847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192ec7820 0x55a192ec7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.702+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/3816619847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ecd900 0x55a192ec72e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.702+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/3816619847 >> 172.21.15.62:0/3816619847 conn(0x55a192df4100 msgr2=0x55a192ec22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:37:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.703+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/3816619847 shutdown_connections 2024-06-10T00:37:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.704+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/3816619847 wait complete. 2024-06-10T00:37:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.704+0000 7f7b76ce5b80 1 -- start start 2024-06-10T00:37:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.705+0000 7f7b76ce5b80 1 Processor -- start 2024-06-10T00:37:01.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.705+0000 7f7b76ce5b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ec7820 0x55a192ed35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.705+0000 7f7b76ce5b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192eccfe0 0x55a192ed08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.705+0000 7f7b76ce5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ecd900 0x55a192ed0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.705+0000 7f7b76ce5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a192db9040 con 0x55a192ecd900 2024-06-10T00:37:01.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.705+0000 7f7b76ce5b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a192e14890 con 0x55a192ec7820 2024-06-10T00:37:01.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.705+0000 7f7b76ce5b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a192ed4320 con 0x55a192eccfe0 2024-06-10T00:37:01.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.705+0000 7f7b76ce5b80 10 client.0 _ll_register_callbacks cb 0x55a192dae630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:37:01.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.705+0000 7f7b6a9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ecd900 0x55a192ed0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.705+0000 7f7b6a9ff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ecd900 0x55a192ed0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48254/0 (socket says 172.21.15.62:48254) 2024-06-10T00:37:01.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.705+0000 7f7b6a9ff700 1 -- 172.21.15.62:0/4078406814 learned_addr learned my addr 172.21.15.62:0/4078406814 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:37:01.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.705+0000 7f7b6a1fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ec7820 0x55a192ed35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.706+0000 7f7b699fd700 1 --2- 172.21.15.62:0/4078406814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192eccfe0 0x55a192ed08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.706+0000 7f7b6a1fe700 1 -- 172.21.15.62:0/4078406814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192eccfe0 msgr2=0x55a192ed08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:37:01.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.706+0000 7f7b6a1fe700 1 --2- 172.21.15.62:0/4078406814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192eccfe0 0x55a192ed08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.706+0000 7f7b6a1fe700 1 -- 172.21.15.62:0/4078406814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ecd900 msgr2=0x55a192ed0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:37:01.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.706+0000 7f7b6a1fe700 1 --2- 172.21.15.62:0/4078406814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ecd900 0x55a192ed0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.706+0000 7f7b6a1fe700 1 -- 172.21.15.62:0/4078406814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a192dde7d0 con 0x55a192ec7820 2024-06-10T00:37:01.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.706+0000 7f7b6a9ff700 1 --2- 172.21.15.62:0/4078406814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ecd900 0x55a192ed0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:37:01.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.706+0000 7f7b6a1fe700 1 --2- 172.21.15.62:0/4078406814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ec7820 0x55a192ed35c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b60005e90 tx=0x7f7b6000f6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c76af0255dd2ca9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:37:01.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.706+0000 7f7b4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:37:01.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.706+0000 7f7b4dffb700 1 -- 172.21.15.62:0/4078406814 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60025070 con 0x55a192ec7820 2024-06-10T00:37:01.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.707+0000 7f7b4dffb700 1 -- 172.21.15.62:0/4078406814 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f7b600046d0 con 0x55a192ec7820 2024-06-10T00:37:01.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.707+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/4078406814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b6000f460 con 0x55a192ec7820 2024-06-10T00:37:01.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.707+0000 7f7b4dffb700 1 -- 172.21.15.62:0/4078406814 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7b600048d0 con 0x55a192ec7820 2024-06-10T00:37:01.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.707+0000 7f7b76ce5b80 10 client.25748 fetch_fsmap learned FSMap version 32 2024-06-10T00:37:01.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.707+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/4078406814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a192ede810 con 0x55a192ec7820 2024-06-10T00:37:01.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.707+0000 7f7b4dffb700 1 -- 172.21.15.62:0/4078406814 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6000b560 con 0x55a192ec7820 2024-06-10T00:37:01.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.708+0000 7f7b4dffb700 1 -- 172.21.15.62:0/4078406814 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f7b60020410 con 0x55a192ec7820 2024-06-10T00:37:01.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.708+0000 7f7b76ce5b80 10 client.25748 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T00:37:01.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.708+0000 7f7b76ce5b80 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:37:01.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.708+0000 7f7b76ce5b80 20 client.25748 populate_metadata read hostname 'smithi062' 2024-06-10T00:37:01.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.708+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/4078406814 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a192ee0ce0 0x55a192ee30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.708+0000 7f7b76ce5b80 4 client.25748 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:37:01.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.708+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/4078406814 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7b6000f460 con 0x55a192ee0ce0 2024-06-10T00:37:01.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.708+0000 7f7b699fd700 1 --2- 172.21.15.62:0/4078406814 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a192ee0ce0 0x55a192ee30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.709+0000 7f7b699fd700 1 --2- 172.21.15.62:0/4078406814 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a192ee0ce0 0x55a192ee30d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef930a1d7eb45d7 server_cookie=5c072df0c696ffa9 in_seq=0 out_seq=0 2024-06-10T00:37:01.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.709+0000 7f7b4dffb700 10 client.25748 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:37:01.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.710+0000 7f7b4dffb700 1 -- 172.21.15.62:0/4078406814 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f7b6000f460 con 0x55a192ee0ce0 2024-06-10T00:37:01.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.710+0000 7f7b4dffb700 10 client.25748 handle_command_reply: tid=0 2024-06-10T00:37:01.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.710+0000 7f7b76ce5b80 1 client.25748 shutdown 2024-06-10T00:37:01.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.710+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/4078406814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ec7820 msgr2=0x55a192ed35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:37:01.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.710+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/4078406814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ec7820 0x55a192ed35c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b60005e90 tx=0x7f7b6000f6c0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.711+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/4078406814 shutdown_connections 2024-06-10T00:37:01.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.711+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/4078406814 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a192ee0ce0 0x55a192ee30d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.711+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/4078406814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ecd900 0x55a192ed0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.711+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/4078406814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ec7820 0x55a192ed35c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.711+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/4078406814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192eccfe0 0x55a192ed08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.711+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/4078406814 >> 172.21.15.62:0/4078406814 conn(0x55a192df4100 msgr2=0x55a192ec1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:37:01.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.711+0000 7f7b4dffb700 0 client.25748 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:37:01.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.711+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/4078406814 shutdown_connections 2024-06-10T00:37:01.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.711+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/4078406814 wait complete. 2024-06-10T00:37:01.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.711+0000 7f7b76ce5b80 20 client.25748 trim_cache size 0 max 16384 2024-06-10T00:37:01.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.743+0000 7f7b76ce5b80 1 Processor -- start 2024-06-10T00:37:01.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.743+0000 7f7b76ce5b80 1 -- start start 2024-06-10T00:37:01.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.743+0000 7f7b76ce5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ec7820 0x55a192ec72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.743+0000 7f7b76ce5b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192eccfe0 0x55a192ece410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.743+0000 7f7b76ce5b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192ecd900 0x55a192ec1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.743+0000 7f7b76ce5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4800eda0 con 0x55a192ec7820 2024-06-10T00:37:01.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.743+0000 7f7b76ce5b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7b480109a0 con 0x55a192eccfe0 2024-06-10T00:37:01.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.743+0000 7f7b76ce5b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4800f470 con 0x55a192ecd900 2024-06-10T00:37:01.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.743+0000 7f7b6a1fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ec7820 0x55a192ec72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.743+0000 7f7b6a1fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ec7820 0x55a192ec72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48262/0 (socket says 172.21.15.62:48262) 2024-06-10T00:37:01.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.743+0000 7f7b6a1fe700 1 -- 172.21.15.62:0/3629346960 learned_addr learned my addr 172.21.15.62:0/3629346960 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:37:01.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.743+0000 7f7b6a9ff700 1 --2- 172.21.15.62:0/3629346960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192ecd900 0x55a192ec1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.743+0000 7f7b699fd700 1 --2- 172.21.15.62:0/3629346960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192eccfe0 0x55a192ece410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.744+0000 7f7b6a1fe700 1 -- 172.21.15.62:0/3629346960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192ecd900 msgr2=0x55a192ec1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:37:01.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.744+0000 7f7b6a1fe700 1 --2- 172.21.15.62:0/3629346960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192ecd900 0x55a192ec1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.744+0000 7f7b6a1fe700 1 -- 172.21.15.62:0/3629346960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192eccfe0 msgr2=0x55a192ece410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:37:01.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.744+0000 7f7b6a1fe700 1 --2- 172.21.15.62:0/3629346960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192eccfe0 0x55a192ece410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.744+0000 7f7b6a1fe700 1 -- 172.21.15.62:0/3629346960 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a192ed3dc0 con 0x55a192ec7820 2024-06-10T00:37:01.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.744+0000 7f7b6a1fe700 1 --2- 172.21.15.62:0/3629346960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ec7820 0x55a192ec72e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7b60005780 tx=0x7f7b60005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620dcadb66cc6120 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:37:01.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.745+0000 7f7b4ffff700 1 -- 172.21.15.62:0/3629346960 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60025040 con 0x55a192ec7820 2024-06-10T00:37:01.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.745+0000 7f7b4ffff700 1 -- 172.21.15.62:0/3629346960 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b600046d0 con 0x55a192ec7820 2024-06-10T00:37:01.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.745+0000 7f7b4ffff700 1 -- 172.21.15.62:0/3629346960 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000b560 con 0x55a192ec7820 2024-06-10T00:37:01.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.746+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/3629346960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ec7820 msgr2=0x55a192ec72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:37:01.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.746+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/3629346960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ec7820 0x55a192ec72e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7b60005780 tx=0x7f7b60005bc0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.746+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/3629346960 shutdown_connections 2024-06-10T00:37:01.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.746+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/3629346960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ec7820 0x55a192ec72e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.746+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/3629346960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192ecd900 0x55a192ec1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.746+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/3629346960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192eccfe0 0x55a192ece410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.746+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/3629346960 >> 172.21.15.62:0/3629346960 conn(0x55a192df4100 msgr2=0x55a192ed2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:37:01.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.746+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/3629346960 shutdown_connections 2024-06-10T00:37:01.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.746+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/3629346960 wait complete. 2024-06-10T00:37:01.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.746+0000 7f7b76ce5b80 1 -- start start 2024-06-10T00:37:01.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.746+0000 7f7b76ce5b80 1 Processor -- start 2024-06-10T00:37:01.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.747+0000 7f7b76ce5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ec7820 0x55a192ed98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.747+0000 7f7b76ce5b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192eccfe0 0x55a192ed9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.747+0000 7f7b76ce5b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ecd900 0x55a192ed6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.747+0000 7f7b76ce5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48010210 con 0x55a192ec7820 2024-06-10T00:37:01.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.747+0000 7f7b6a1fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ec7820 0x55a192ed98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.747+0000 7f7b6a1fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ec7820 0x55a192ed98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48266/0 (socket says 172.21.15.62:48266) 2024-06-10T00:37:01.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.747+0000 7f7b6a1fe700 1 -- 172.21.15.62:0/1625013394 learned_addr learned my addr 172.21.15.62:0/1625013394 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:37:01.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.747+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/1625013394 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4800fb40 con 0x55a192ecd900 2024-06-10T00:37:01.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.747+0000 7f7b699fd700 1 --2- 172.21.15.62:0/1625013394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192eccfe0 0x55a192ed9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.747+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/1625013394 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4800cd40 con 0x55a192eccfe0 2024-06-10T00:37:01.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.747+0000 7f7b6a9ff700 1 --2- 172.21.15.62:0/1625013394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ecd900 0x55a192ed6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.748+0000 7f7b76ce5b80 10 client.0 _ll_register_callbacks cb 0x55a192dae630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:37:01.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.748+0000 7f7b699fd700 1 -- 172.21.15.62:0/1625013394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ecd900 msgr2=0x55a192ed6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:37:01.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.748+0000 7f7b699fd700 1 --2- 172.21.15.62:0/1625013394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ecd900 0x55a192ed6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.748+0000 7f7b699fd700 1 -- 172.21.15.62:0/1625013394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ec7820 msgr2=0x55a192ed98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:37:01.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.748+0000 7f7b699fd700 1 --2- 172.21.15.62:0/1625013394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ec7820 0x55a192ed98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.748+0000 7f7b699fd700 1 -- 172.21.15.62:0/1625013394 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a192ed6ea0 con 0x55a192eccfe0 2024-06-10T00:37:01.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.749+0000 7f7b699fd700 1 --2- 172.21.15.62:0/1625013394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192eccfe0 0x55a192ed9e20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7b540cc580 tx=0x7f7b540d16d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=807aaf606b372114 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:37:01.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.749+0000 7f7b4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:37:01.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.749+0000 7f7b4dffb700 1 -- 172.21.15.62:0/1625013394 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54020040 con 0x55a192eccfe0 2024-06-10T00:37:01.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.749+0000 7f7b4dffb700 1 -- 172.21.15.62:0/1625013394 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f7b5409b7b0 con 0x55a192eccfe0 2024-06-10T00:37:01.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.749+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/1625013394 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b600046d0 con 0x55a192eccfe0 2024-06-10T00:37:01.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.750+0000 7f7b4dffb700 1 -- 172.21.15.62:0/1625013394 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7b5409acb0 con 0x55a192eccfe0 2024-06-10T00:37:01.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.750+0000 7f7b76ce5b80 10 client.25765 fetch_fsmap learned FSMap version 32 2024-06-10T00:37:01.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.750+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/1625013394 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a192f4c1b0 con 0x55a192eccfe0 2024-06-10T00:37:01.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.750+0000 7f7b4dffb700 1 -- 172.21.15.62:0/1625013394 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b600046d0 con 0x55a192eccfe0 2024-06-10T00:37:01.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.751+0000 7f7b4dffb700 1 -- 172.21.15.62:0/1625013394 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f7b540a3b30 con 0x55a192eccfe0 2024-06-10T00:37:01.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.751+0000 7f7b76ce5b80 10 client.25765 fetch_fsmap finished waiting for FSMap version 32 2024-06-10T00:37:01.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.751+0000 7f7b76ce5b80 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:37:01.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.751+0000 7f7b76ce5b80 20 client.25765 populate_metadata read hostname 'smithi062' 2024-06-10T00:37:01.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.751+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/1625013394 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a192f4c9d0 0x55a192f4edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:01.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.751+0000 7f7b76ce5b80 4 client.25765 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T00:37:01.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.751+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/1625013394 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a192e25dd0 con 0x55a192f4c9d0 2024-06-10T00:37:01.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.752+0000 7f7b6a1fe700 1 --2- 172.21.15.62:0/1625013394 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a192f4c9d0 0x55a192f4edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:01.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.752+0000 7f7b6a1fe700 1 --2- 172.21.15.62:0/1625013394 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a192f4c9d0 0x55a192f4edf0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a3060a973735c12 server_cookie=d333a7f47be4134e in_seq=0 out_seq=0 2024-06-10T00:37:01.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.752+0000 7f7b4dffb700 10 client.25765 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:37:01.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.753+0000 7f7b4dffb700 1 -- 172.21.15.62:0/1625013394 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a192e25dd0 con 0x55a192f4c9d0 2024-06-10T00:37:01.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.753+0000 7f7b4dffb700 10 client.25765 handle_command_reply: tid=0 2024-06-10T00:37:01.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.753+0000 7f7b76ce5b80 1 client.25765 shutdown 2024-06-10T00:37:01.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.754+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/1625013394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192eccfe0 msgr2=0x55a192ed9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:37:01.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.754+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/1625013394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192eccfe0 0x55a192ed9e20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7b540cc580 tx=0x7f7b540d16d0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.754+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/1625013394 shutdown_connections 2024-06-10T00:37:01.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.754+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/1625013394 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a192f4c9d0 0x55a192f4edf0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.754+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/1625013394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a192ec7820 0x55a192ed98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.754+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/1625013394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a192ecd900 0x55a192ed6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.754+0000 7f7b76ce5b80 1 --2- 172.21.15.62:0/1625013394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a192eccfe0 0x55a192ed9e20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.585 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:37:01.585 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-06-10T00:37:01.585 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "a0b1c3b8-03f2-4bfd-83cb-35531ec7daaa", 2024-06-10T00:37:01.585 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-06-10T00:37:01.585 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:37:01.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.754+0000 7f7b4dffb700 0 client.25765 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:37:01.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.754+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/1625013394 >> 172.21.15.62:0/1625013394 conn(0x55a192df4100 msgr2=0x55a192ed0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:37:01.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.754+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/1625013394 shutdown_connections 2024-06-10T00:37:01.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.754+0000 7f7b76ce5b80 1 -- 172.21.15.62:0/1625013394 wait complete. 2024-06-10T00:37:01.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.754+0000 7f7b76ce5b80 20 client.25765 trim_cache size 0 max 16384 2024-06-10T00:37:01.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.755+0000 7f7b6c462700 1 -- 172.21.15.62:0/946624474 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7b500a2f60 msgr2=0x7f7b500a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:37:01.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.755+0000 7f7b6c462700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7b500a2f60 0x7f7b500a5420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b6406bc00 tx=0x7f7b5c00b040 comp rx=0 tx=0).stop 2024-06-10T00:37:01.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.755+0000 7f7b6c462700 1 -- 172.21.15.62:0/946624474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b64110c70 msgr2=0x7f7b641add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:37:01.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.755+0000 7f7b6c462700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b64110c70 0x7f7b641add50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400d980 tx=0x7f7b5400dd40 comp rx=0 tx=0).stop 2024-06-10T00:37:01.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.755+0000 7f7b6c462700 1 -- 172.21.15.62:0/946624474 shutdown_connections 2024-06-10T00:37:01.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.755+0000 7f7b6c462700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7b500a2f60 0x7f7b500a5420 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.755+0000 7f7b6c462700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b6411bc70 0x7f7b641a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.755+0000 7f7b6c462700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b641099d0 0x7f7b641ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.755+0000 7f7b6c462700 1 --2- 172.21.15.62:0/946624474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b64110c70 0x7f7b641add50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:01.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.755+0000 7f7b6c462700 1 -- 172.21.15.62:0/946624474 >> 172.21.15.62:0/946624474 conn(0x7f7b6406d600 msgr2=0x7f7b64119c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:37:01.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.755+0000 7f7b6c462700 1 -- 172.21.15.62:0/946624474 shutdown_connections 2024-06-10T00:37:01.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:36:59.755+0000 7f7b6c462700 1 -- 172.21.15.62:0/946624474 wait complete. 2024-06-10T00:37:01.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:01 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:37:01.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:01 smithi105 ceph-mon[36321]: pgmap v128: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 54 MiB/s wr, 8.04k op/s 2024-06-10T00:37:01.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:01 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:37:01.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:01 smithi062 ceph-mon[32390]: pgmap v128: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 54 MiB/s wr, 8.04k op/s 2024-06-10T00:37:01.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:01 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:37:01.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:01 smithi138 ceph-mon[36549]: pgmap v128: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 54 MiB/s wr, 8.04k op/s 2024-06-10T00:37:03.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:03 smithi105 ceph-mon[36321]: pgmap v129: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 62 MiB/s wr, 9.47k op/s 2024-06-10T00:37:03.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:03 smithi062 ceph-mon[32390]: pgmap v129: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 62 MiB/s wr, 9.47k op/s 2024-06-10T00:37:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:03 smithi138 ceph-mon[36549]: pgmap v129: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 62 MiB/s wr, 9.47k op/s 2024-06-10T00:37:05.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:05 smithi105 ceph-mon[36321]: pgmap v130: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 49 MiB/s wr, 7.42k op/s 2024-06-10T00:37:05.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:05 smithi062 ceph-mon[32390]: pgmap v130: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 49 MiB/s wr, 7.42k op/s 2024-06-10T00:37:05.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:05 smithi138 ceph-mon[36549]: pgmap v130: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 49 MiB/s wr, 7.42k op/s 2024-06-10T00:37:07.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:07 smithi105 ceph-mon[36321]: pgmap v131: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 43 MiB/s wr, 6.49k op/s 2024-06-10T00:37:07.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:07 smithi062 ceph-mon[32390]: pgmap v131: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 43 MiB/s wr, 6.49k op/s 2024-06-10T00:37:07.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:07 smithi138 ceph-mon[36549]: pgmap v131: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 43 MiB/s wr, 6.49k op/s 2024-06-10T00:37:09.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:09 smithi105 ceph-mon[36321]: pgmap v132: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 47 MiB/s wr, 7.42k op/s 2024-06-10T00:37:09.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:09 smithi062 ceph-mon[32390]: pgmap v132: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 47 MiB/s wr, 7.42k op/s 2024-06-10T00:37:09.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:09 smithi138 ceph-mon[36549]: pgmap v132: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 47 MiB/s wr, 7.42k op/s 2024-06-10T00:37:10.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:10 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:37:10.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:10 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:37:10.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:10 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:37:11.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:11 smithi105 ceph-mon[36321]: pgmap v133: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 30 MiB/s wr, 5.19k op/s 2024-06-10T00:37:11.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:11 smithi062 ceph-mon[32390]: pgmap v133: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 30 MiB/s wr, 5.19k op/s 2024-06-10T00:37:11.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:11 smithi138 ceph-mon[36549]: pgmap v133: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 30 MiB/s wr, 5.19k op/s 2024-06-10T00:37:11.993 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.2 2024-06-10T00:37:11.994 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:37:13.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:13 smithi105 ceph-mon[36321]: pgmap v134: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 38 MiB/s wr, 6.71k op/s 2024-06-10T00:37:13.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:13 smithi062 ceph-mon[32390]: pgmap v134: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 38 MiB/s wr, 6.71k op/s 2024-06-10T00:37:13.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:13 smithi138 ceph-mon[36549]: pgmap v134: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 38 MiB/s wr, 6.71k op/s 2024-06-10T00:37:14.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:14 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:37:14.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:14 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:37:14.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:14 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:37:14.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:14 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:37:14.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:14 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:37:14.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:14 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:37:14.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:14 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:37:14.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:14 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:37:14.875 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:14 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:37:15.726 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:37:15.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:15 smithi105 ceph-mon[36321]: pgmap v135: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 MiB/s wr, 4.96k op/s 2024-06-10T00:37:15.792 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:15 smithi062 ceph-mon[32390]: pgmap v135: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 MiB/s wr, 4.96k op/s 2024-06-10T00:37:15.799 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:37:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:15 smithi138 ceph-mon[36549]: pgmap v135: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 MiB/s wr, 4.96k op/s 2024-06-10T00:37:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:17 smithi105 ceph-mon[36321]: pgmap v136: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 4.40k op/s 2024-06-10T00:37:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:17 smithi062 ceph-mon[32390]: pgmap v136: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 4.40k op/s 2024-06-10T00:37:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:17 smithi138 ceph-mon[36549]: pgmap v136: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 4.40k op/s 2024-06-10T00:37:18.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:18 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:18.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:18 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:18.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:18 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:19.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:19 smithi105 ceph-mon[36321]: pgmap v137: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 42 MiB/s wr, 7.54k op/s 2024-06-10T00:37:19.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:19 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:19.828 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:19 smithi062 ceph-mon[32390]: pgmap v137: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 42 MiB/s wr, 7.54k op/s 2024-06-10T00:37:19.828 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:19 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:19 smithi138 ceph-mon[36549]: pgmap v137: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 42 MiB/s wr, 7.54k op/s 2024-06-10T00:37:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:19 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:20.638 INFO:tasks.workunit.client.0.smithi062.stdout:Wrote -1 instead of 4096 bytes. 2024-06-10T00:37:20.638 INFO:tasks.workunit.client.0.smithi062.stdout:Probably out of disk space 2024-06-10T00:37:20.638 INFO:tasks.workunit.client.0.smithi062.stderr:write: Resource temporarily unavailable 2024-06-10T00:37:20.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:37:20.799 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-06-10T00:37:20.799 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-10T00:37:21.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:21 smithi105 ceph-mon[36321]: pgmap v138: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 39 MiB/s wr, 6.64k op/s 2024-06-10T00:37:21.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:21 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:21.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:21 smithi062 ceph-mon[32390]: pgmap v138: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 39 MiB/s wr, 6.64k op/s 2024-06-10T00:37:21.842 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:21 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:21 smithi138 ceph-mon[36549]: pgmap v138: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 39 MiB/s wr, 6.64k op/s 2024-06-10T00:37:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:21 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:22.265 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/ffsb.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2024-06-10T00:37:22.508 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5c9934f8908f4f08b3227e1b77e29b59 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/ffsb.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2024-06-10T00:37:22.512 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-06-10T00:37:22.522 INFO:tasks.quiescer:joining Quiescers 2024-06-10T00:37:22.522 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-06-10T00:37:22.522 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:37:23.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:23 smithi105 ceph-mon[36321]: pgmap v139: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 MiB/s wr, 7.51k op/s 2024-06-10T00:37:23.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:23 smithi062 ceph-mon[32390]: pgmap v139: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 MiB/s wr, 7.51k op/s 2024-06-10T00:37:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:23 smithi138 ceph-mon[36549]: pgmap v139: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 MiB/s wr, 7.51k op/s 2024-06-10T00:37:25.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:25 smithi105 ceph-mon[36321]: pgmap v140: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 MiB/s wr, 6.69k op/s 2024-06-10T00:37:25.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:25 smithi062 ceph-mon[32390]: pgmap v140: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 MiB/s wr, 6.69k op/s 2024-06-10T00:37:25.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:25 smithi138 ceph-mon[36549]: pgmap v140: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 MiB/s wr, 6.69k op/s 2024-06-10T00:37:26.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:26 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:26.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:26 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:27.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:26 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:27.219 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:37:27.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:27 smithi062 ceph-mon[32390]: pgmap v141: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 36 MiB/s wr, 6.03k op/s 2024-06-10T00:37:27.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:27 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:27.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:27 smithi138 ceph-mon[36549]: pgmap v141: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 36 MiB/s wr, 6.03k op/s 2024-06-10T00:37:27.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:27 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:28.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:27 smithi105 ceph-mon[36321]: pgmap v141: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 36 MiB/s wr, 6.03k op/s 2024-06-10T00:37:28.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:27 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:37:28.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:28 smithi062 ceph-mon[32390]: pgmap v142: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 49 MiB/s wr, 7.89k op/s 2024-06-10T00:37:28.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:28 smithi138 ceph-mon[36549]: pgmap v142: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 49 MiB/s wr, 7.89k op/s 2024-06-10T00:37:29.031 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:28 smithi105 ceph-mon[36321]: pgmap v142: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 49 MiB/s wr, 7.89k op/s 2024-06-10T00:37:31.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:31 smithi062 ceph-mon[32390]: pgmap v143: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 33 MiB/s wr, 5.15k op/s 2024-06-10T00:37:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:31 smithi138 ceph-mon[36549]: pgmap v143: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 33 MiB/s wr, 5.15k op/s 2024-06-10T00:37:31.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:31 smithi105 ceph-mon[36321]: pgmap v143: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 33 MiB/s wr, 5.15k op/s 2024-06-10T00:37:32.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:32 smithi062 ceph-mon[32390]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:37:32.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:32 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:37:32.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:32 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:37:32.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:32 smithi138 ceph-mon[36549]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:37:32.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:32 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:37:32.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:32 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:37:32.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:32 smithi105 ceph-mon[36321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:37:32.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:32 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:37:32.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:32 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:37:33.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:33 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:37:33.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:33 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:37:33.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:33 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:37:33.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:33 smithi062 ceph-mon[32390]: pgmap v144: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 38 MiB/s wr, 5.98k op/s 2024-06-10T00:37:33.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:33 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:37:33.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:33 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:37:33.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:33 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:37:33.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:33 smithi138 ceph-mon[36549]: pgmap v144: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 38 MiB/s wr, 5.98k op/s 2024-06-10T00:37:33.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:33 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:37:33.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:33 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:37:33.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:33 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:37:33.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:33 smithi105 ceph-mon[36321]: pgmap v144: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 38 MiB/s wr, 5.98k op/s 2024-06-10T00:37:35.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:35 smithi062 ceph-mon[32390]: pgmap v145: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 35 MiB/s wr, 5.28k op/s 2024-06-10T00:37:35.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:35 smithi138 ceph-mon[36549]: pgmap v145: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 35 MiB/s wr, 5.28k op/s 2024-06-10T00:37:35.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:35 smithi105 ceph-mon[36321]: pgmap v145: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 35 MiB/s wr, 5.28k op/s 2024-06-10T00:37:37.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:37 smithi062 ceph-mon[32390]: pgmap v146: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 31 MiB/s wr, 4.52k op/s 2024-06-10T00:37:37.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:37 smithi062 ceph-mon[32390]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:37:37.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:37 smithi138 ceph-mon[36549]: pgmap v146: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 31 MiB/s wr, 4.52k op/s 2024-06-10T00:37:37.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:37 smithi138 ceph-mon[36549]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:37:37.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:37 smithi105 ceph-mon[36321]: pgmap v146: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 31 MiB/s wr, 4.52k op/s 2024-06-10T00:37:37.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:37 smithi105 ceph-mon[36321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:37:39.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:39 smithi138 ceph-mon[36549]: pgmap v147: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 MiB/s wr, 6.37k op/s 2024-06-10T00:37:39.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:39 smithi105 ceph-mon[36321]: pgmap v147: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 MiB/s wr, 6.37k op/s 2024-06-10T00:37:39.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:39 smithi062 ceph-mon[32390]: pgmap v147: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 MiB/s wr, 6.37k op/s 2024-06-10T00:37:41.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:41 smithi062 ceph-mon[32390]: pgmap v148: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 MiB/s wr, 4.86k op/s 2024-06-10T00:37:41.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:41 smithi138 ceph-mon[36549]: pgmap v148: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 MiB/s wr, 4.86k op/s 2024-06-10T00:37:41.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:41 smithi105 ceph-mon[36321]: pgmap v148: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 MiB/s wr, 4.86k op/s 2024-06-10T00:37:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:43 smithi105 ceph-mon[36321]: pgmap v149: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 MiB/s wr, 6.28k op/s 2024-06-10T00:37:43.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:43 smithi062 ceph-mon[32390]: pgmap v149: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 MiB/s wr, 6.28k op/s 2024-06-10T00:37:43.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:43 smithi138 ceph-mon[36549]: pgmap v149: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 MiB/s wr, 6.28k op/s 2024-06-10T00:37:43.698 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:37:45.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:45 smithi062 ceph-mon[32390]: pgmap v150: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 MiB/s wr, 5.34k op/s 2024-06-10T00:37:45.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:45 smithi105 ceph-mon[36321]: pgmap v150: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 MiB/s wr, 5.34k op/s 2024-06-10T00:37:45.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:45 smithi138 ceph-mon[36549]: pgmap v150: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 MiB/s wr, 5.34k op/s 2024-06-10T00:37:47.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:47 smithi062 ceph-mon[32390]: pgmap v151: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 MiB/s wr, 5.19k op/s 2024-06-10T00:37:47.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:47 smithi105 ceph-mon[36321]: pgmap v151: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 MiB/s wr, 5.19k op/s 2024-06-10T00:37:47.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:47 smithi138 ceph-mon[36549]: pgmap v151: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 MiB/s wr, 5.19k op/s 2024-06-10T00:37:49.393 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:49 smithi062 ceph-mon[32390]: pgmap v152: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 5.96k op/s 2024-06-10T00:37:49.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:49 smithi105 ceph-mon[36321]: pgmap v152: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 5.96k op/s 2024-06-10T00:37:49.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:49 smithi138 ceph-mon[36549]: pgmap v152: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 5.96k op/s 2024-06-10T00:37:51.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:51 smithi105 ceph-mon[36321]: pgmap v153: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 4.45k op/s 2024-06-10T00:37:51.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:51 smithi062 ceph-mon[32390]: pgmap v153: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 4.45k op/s 2024-06-10T00:37:51.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:51 smithi138 ceph-mon[36549]: pgmap v153: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 4.45k op/s 2024-06-10T00:37:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:53 smithi105 ceph-mon[36321]: pgmap v154: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 4.67k op/s 2024-06-10T00:37:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:53 smithi062 ceph-mon[32390]: pgmap v154: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 4.67k op/s 2024-06-10T00:37:53.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:53 smithi138 ceph-mon[36549]: pgmap v154: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 4.67k op/s 2024-06-10T00:37:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:55 smithi105 ceph-mon[36321]: pgmap v155: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.08k op/s 2024-06-10T00:37:55.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:55 smithi062 ceph-mon[32390]: pgmap v155: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.08k op/s 2024-06-10T00:37:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:55 smithi138 ceph-mon[36549]: pgmap v155: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.08k op/s 2024-06-10T00:37:57.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.884+0000 7ffb66582700 1 -- 172.21.15.62:0/1156731415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb60107400 msgr2=0x7ffb601077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:37:57.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.884+0000 7ffb66582700 1 --2- 172.21.15.62:0/1156731415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb60107400 0x7ffb601077e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009990 tx=0x7ffb50009ca0 comp rx=0 tx=0).stop 2024-06-10T00:37:57.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.884+0000 7ffb66582700 1 -- 172.21.15.62:0/1156731415 shutdown_connections 2024-06-10T00:37:57.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.884+0000 7ffb66582700 1 --2- 172.21.15.62:0/1156731415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb60107400 0x7ffb601077e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:57.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.884+0000 7ffb66582700 1 --2- 172.21.15.62:0/1156731415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb60101d50 0x7ffb6010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:57.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.884+0000 7ffb66582700 1 --2- 172.21.15.62:0/1156731415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb60101390 0x7ffb60101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:57.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.884+0000 7ffb66582700 1 -- 172.21.15.62:0/1156731415 >> 172.21.15.62:0/1156731415 conn(0x7ffb60078340 msgr2=0x7ffb60078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:37:57.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.884+0000 7ffb66582700 1 -- 172.21.15.62:0/1156731415 shutdown_connections 2024-06-10T00:37:57.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.884+0000 7ffb66582700 1 -- 172.21.15.62:0/1156731415 wait complete. 2024-06-10T00:37:57.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.885+0000 7ffb66582700 1 Processor -- start 2024-06-10T00:37:57.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.885+0000 7ffb66582700 1 -- start start 2024-06-10T00:37:57.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb66582700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb60101390 0x7ffb60199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:57.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb66582700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb60101d50 0x7ffb60199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:57.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb66582700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb60107400 0x7ffb601a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:57.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb66582700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffb601116a0 con 0x7ffb60101d50 2024-06-10T00:37:57.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb66582700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60111520 con 0x7ffb60101390 2024-06-10T00:37:57.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb66582700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60111820 con 0x7ffb60107400 2024-06-10T00:37:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb5f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb60101d50 0x7ffb60199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb5f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb60101d50 0x7ffb60199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42938/0 (socket says 172.21.15.62:42938) 2024-06-10T00:37:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb5f7fe700 1 -- 172.21.15.62:0/3300639056 learned_addr learned my addr 172.21.15.62:0/3300639056 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:37:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb5ffff700 1 --2- 172.21.15.62:0/3300639056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb60101390 0x7ffb60199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb64b1f700 1 --2- 172.21.15.62:0/3300639056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb60107400 0x7ffb601a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb5f7fe700 1 -- 172.21.15.62:0/3300639056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb60107400 msgr2=0x7ffb601a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:37:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb5f7fe700 1 --2- 172.21.15.62:0/3300639056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb60107400 0x7ffb601a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:57.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb5f7fe700 1 -- 172.21.15.62:0/3300639056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb60101390 msgr2=0x7ffb60199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:37:57.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb5f7fe700 1 --2- 172.21.15.62:0/3300639056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb60101390 0x7ffb60199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:57.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.886+0000 7ffb5f7fe700 1 -- 172.21.15.62:0/3300639056 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb601a5060 con 0x7ffb60101d50 2024-06-10T00:37:57.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.887+0000 7ffb5f7fe700 1 --2- 172.21.15.62:0/3300639056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb60101d50 0x7ffb60199fe0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400c970 tx=0x7ffb5400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:37:57.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.887+0000 7ffb5d7fa700 1 -- 172.21.15.62:0/3300639056 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb540078d0 con 0x7ffb60101d50 2024-06-10T00:37:57.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.887+0000 7ffb66582700 1 -- 172.21.15.62:0/3300639056 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb601a5290 con 0x7ffb60101d50 2024-06-10T00:37:57.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.887+0000 7ffb5d7fa700 1 -- 172.21.15.62:0/3300639056 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb54018480 con 0x7ffb60101d50 2024-06-10T00:37:57.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.887+0000 7ffb5d7fa700 1 -- 172.21.15.62:0/3300639056 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400f6b0 con 0x7ffb60101d50 2024-06-10T00:37:57.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.887+0000 7ffb66582700 1 -- 172.21.15.62:0/3300639056 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb601a5750 con 0x7ffb60101d50 2024-06-10T00:37:57.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.888+0000 7ffb66582700 1 -- 172.21.15.62:0/3300639056 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb60069180 con 0x7ffb60101d50 2024-06-10T00:37:57.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.889+0000 7ffb5d7fa700 1 -- 172.21.15.62:0/3300639056 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ffb54018630 con 0x7ffb60101d50 2024-06-10T00:37:57.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.890+0000 7ffb5d7fa700 1 --2- 172.21.15.62:0/3300639056 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ffb480a2d90 0x7ffb480a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:37:57.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.890+0000 7ffb5ffff700 1 --2- 172.21.15.62:0/3300639056 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ffb480a2d90 0x7ffb480a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:37:57.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.890+0000 7ffb5d7fa700 1 -- 172.21.15.62:0/3300639056 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7ffb540cfbc0 con 0x7ffb60101d50 2024-06-10T00:37:57.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.890+0000 7ffb5ffff700 1 --2- 172.21.15.62:0/3300639056 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ffb480a2d90 0x7ffb480a5250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffb500098e0 tx=0x7ffb50005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:37:57.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:55.893+0000 7ffb5d7fa700 1 -- 172.21.15.62:0/3300639056 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffb5409c550 con 0x7ffb60101d50 2024-06-10T00:37:57.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:56.150+0000 7ffb66582700 1 -- 172.21.15.62:0/3300639056 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb6002de60 con 0x7ffb60101d50 2024-06-10T00:37:57.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:56.151+0000 7ffb5d7fa700 1 -- 172.21.15.62:0/3300639056 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11952 (secure 0 0 0) 0x7ffb5409be50 con 0x7ffb60101d50 2024-06-10T00:37:57.090 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:37:57.091 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:37:35.971947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:37:57.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:56.153+0000 7ffb66582700 1 -- 172.21.15.62:0/3300639056 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ffb480a2d90 msgr2=0x7ffb480a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:37:57.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:56.153+0000 7ffb66582700 1 --2- 172.21.15.62:0/3300639056 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ffb480a2d90 0x7ffb480a5250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffb500098e0 tx=0x7ffb50005960 comp rx=0 tx=0).stop 2024-06-10T00:37:57.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:56.153+0000 7ffb66582700 1 -- 172.21.15.62:0/3300639056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb60101d50 msgr2=0x7ffb60199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:37:57.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:56.153+0000 7ffb66582700 1 --2- 172.21.15.62:0/3300639056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb60101d50 0x7ffb60199fe0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400c970 tx=0x7ffb5400cc80 comp rx=0 tx=0).stop 2024-06-10T00:37:57.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:56.153+0000 7ffb66582700 1 -- 172.21.15.62:0/3300639056 shutdown_connections 2024-06-10T00:37:57.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:56.153+0000 7ffb66582700 1 --2- 172.21.15.62:0/3300639056 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ffb480a2d90 0x7ffb480a5250 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:57.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:56.153+0000 7ffb66582700 1 --2- 172.21.15.62:0/3300639056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb60107400 0x7ffb601a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:57.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:56.153+0000 7ffb66582700 1 --2- 172.21.15.62:0/3300639056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb60101390 0x7ffb60199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:57.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:56.153+0000 7ffb66582700 1 --2- 172.21.15.62:0/3300639056 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb60101d50 0x7ffb60199fe0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:37:57.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:56.153+0000 7ffb66582700 1 -- 172.21.15.62:0/3300639056 >> 172.21.15.62:0/3300639056 conn(0x7ffb60078340 msgr2=0x7ffb60100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:37:57.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:56.153+0000 7ffb66582700 1 -- 172.21.15.62:0/3300639056 shutdown_connections 2024-06-10T00:37:57.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:37:56.153+0000 7ffb66582700 1 -- 172.21.15.62:0/3300639056 wait complete. 2024-06-10T00:37:57.093 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 34 2024-06-10T00:37:57.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:57 smithi062 ceph-mon[32390]: pgmap v156: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 2.73k op/s 2024-06-10T00:37:57.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:57 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3300639056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:37:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:57 smithi105 ceph-mon[36321]: pgmap v156: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 2.73k op/s 2024-06-10T00:37:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:57 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3300639056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:37:57.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:57 smithi138 ceph-mon[36549]: pgmap v156: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 2.73k op/s 2024-06-10T00:37:57.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:57 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3300639056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:37:59.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:37:59 smithi105 ceph-mon[36321]: pgmap v157: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.8 MiB/s wr, 2.31k op/s 2024-06-10T00:37:59.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:37:59 smithi062 ceph-mon[32390]: pgmap v157: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.8 MiB/s wr, 2.31k op/s 2024-06-10T00:37:59.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:37:59 smithi138 ceph-mon[36549]: pgmap v157: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.8 MiB/s wr, 2.31k op/s 2024-06-10T00:38:01.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:01 smithi062 ceph-mon[32390]: pgmap v158: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 1.20k op/s 2024-06-10T00:38:01.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:01 smithi105 ceph-mon[36321]: pgmap v158: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 1.20k op/s 2024-06-10T00:38:01.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:01 smithi138 ceph-mon[36549]: pgmap v158: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 1.20k op/s 2024-06-10T00:38:03.411 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:03 smithi062 ceph-mon[32390]: pgmap v159: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.5 MiB/s wr, 859 op/s 2024-06-10T00:38:03.426 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:03 smithi138 ceph-mon[36549]: pgmap v159: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.5 MiB/s wr, 859 op/s 2024-06-10T00:38:03.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:03 smithi105 ceph-mon[36321]: pgmap v159: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.5 MiB/s wr, 859 op/s 2024-06-10T00:38:04.409 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db / --timeout 90 --expiration 138.2 --await-for 45.0' 2024-06-10T00:38:04.410 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24464 quiesce db / --timeout 90 --expiration 138.2 --await-for 45.0 2024-06-10T00:38:04.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.859+0000 7f1e66af7700 1 -- 172.21.15.62:0/4093265682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e60101390 msgr2=0x7f1e60101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:04.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.859+0000 7f1e66af7700 1 --2- 172.21.15.62:0/4093265682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e60101390 0x7f1e60101810 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009990 tx=0x7f1e4c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.859+0000 7f1e66af7700 1 -- 172.21.15.62:0/4093265682 shutdown_connections 2024-06-10T00:38:04.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.859+0000 7f1e66af7700 1 --2- 172.21.15.62:0/4093265682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e60101d50 0x7f1e6010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.859+0000 7f1e66af7700 1 --2- 172.21.15.62:0/4093265682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e60107400 0x7f1e601077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.859+0000 7f1e66af7700 1 --2- 172.21.15.62:0/4093265682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e60101390 0x7f1e60101810 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.859+0000 7f1e66af7700 1 -- 172.21.15.62:0/4093265682 >> 172.21.15.62:0/4093265682 conn(0x7f1e60078340 msgr2=0x7f1e60078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:04.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.859+0000 7f1e66af7700 1 -- 172.21.15.62:0/4093265682 shutdown_connections 2024-06-10T00:38:04.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.859+0000 7f1e66af7700 1 -- 172.21.15.62:0/4093265682 wait complete. 2024-06-10T00:38:04.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.860+0000 7f1e66af7700 1 Processor -- start 2024-06-10T00:38:04.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.860+0000 7f1e66af7700 1 -- start start 2024-06-10T00:38:04.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.860+0000 7f1e66af7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e60101390 0x7f1e6019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.860+0000 7f1e66af7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e60101d50 0x7f1e6019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.860+0000 7f1e66af7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e60107400 0x7f1e601979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.860+0000 7f1e66af7700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601116f0 con 0x7f1e60107400 2024-06-10T00:38:04.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.860+0000 7f1e66af7700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1e60111570 con 0x7f1e60101d50 2024-06-10T00:38:04.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.860+0000 7f1e66af7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e60111870 con 0x7f1e60101390 2024-06-10T00:38:04.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.860+0000 7f1e65094700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e60107400 0x7f1e601979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.860+0000 7f1e65094700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e60107400 0x7f1e601979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49048/0 (socket says 172.21.15.62:49048) 2024-06-10T00:38:04.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.860+0000 7f1e65094700 1 -- 172.21.15.62:0/4222536491 learned_addr learned my addr 172.21.15.62:0/4222536491 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:04.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.861+0000 7f1e64893700 1 --2- 172.21.15.62:0/4222536491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e60101390 0x7f1e6019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.861+0000 7f1e5ffff700 1 --2- 172.21.15.62:0/4222536491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e60101d50 0x7f1e6019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.861+0000 7f1e65094700 1 -- 172.21.15.62:0/4222536491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e60101390 msgr2=0x7f1e6019d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:04.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.861+0000 7f1e65094700 1 --2- 172.21.15.62:0/4222536491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e60101390 0x7f1e6019d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.861+0000 7f1e65094700 1 -- 172.21.15.62:0/4222536491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e60101d50 msgr2=0x7f1e6019dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:04.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.861+0000 7f1e65094700 1 --2- 172.21.15.62:0/4222536491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e60101d50 0x7f1e6019dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.861+0000 7f1e65094700 1 -- 172.21.15.62:0/4222536491 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e60198130 con 0x7f1e60107400 2024-06-10T00:38:04.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.861+0000 7f1e65094700 1 --2- 172.21.15.62:0/4222536491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e60107400 0x7f1e601979d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1e58007d70 tx=0x7f1e5800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:04.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.861+0000 7f1e5dffb700 1 -- 172.21.15.62:0/4222536491 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58016020 con 0x7f1e60107400 2024-06-10T00:38:04.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.861+0000 7f1e5dffb700 1 -- 172.21.15.62:0/4222536491 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e5800ad40 con 0x7f1e60107400 2024-06-10T00:38:04.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.861+0000 7f1e66af7700 1 -- 172.21.15.62:0/4222536491 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e60198420 con 0x7f1e60107400 2024-06-10T00:38:04.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.861+0000 7f1e5dffb700 1 -- 172.21.15.62:0/4222536491 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e580116f0 con 0x7f1e60107400 2024-06-10T00:38:04.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.862+0000 7f1e66af7700 1 -- 172.21.15.62:0/4222536491 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e601a46e0 con 0x7f1e60107400 2024-06-10T00:38:04.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.864+0000 7f1e5dffb700 1 -- 172.21.15.62:0/4222536491 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1e58019020 con 0x7f1e60107400 2024-06-10T00:38:04.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.865+0000 7f1e5dffb700 1 --2- 172.21.15.62:0/4222536491 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1e500a2e10 0x7f1e500a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.865+0000 7f1e5dffb700 1 -- 172.21.15.62:0/4222536491 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f1e580cf6e0 con 0x7f1e60107400 2024-06-10T00:38:04.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.865+0000 7f1e64893700 1 --2- 172.21.15.62:0/4222536491 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1e500a2e10 0x7f1e500a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.865+0000 7f1e64893700 1 --2- 172.21.15.62:0/4222536491 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1e500a2e10 0x7f1e500a52d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1e54006fd0 tx=0x7f1e54008040 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:04.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.866+0000 7f1e7137ab80 1 Processor -- start 2024-06-10T00:38:04.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.866+0000 7f1e7137ab80 1 -- start start 2024-06-10T00:38:04.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.866+0000 7f1e7137ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5a9f60 0x56111d5aa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.866+0000 7f1e7137ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5aa880 0x56111d5a4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.866+0000 7f1e7137ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a47a0 0x56111d5a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.866+0000 7f1e7137ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56111d496f60 con 0x56111d5a9f60 2024-06-10T00:38:04.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.866+0000 7f1e7137ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56111d4f5850 con 0x56111d5aa880 2024-06-10T00:38:04.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.866+0000 7f1e7137ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56111d45d460 con 0x56111d5a47a0 2024-06-10T00:38:04.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.866+0000 7f1e64893700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5a9f60 0x56111d5aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.866+0000 7f1e64893700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5a9f60 0x56111d5aa340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49056/0 (socket says 172.21.15.62:49056) 2024-06-10T00:38:04.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.866+0000 7f1e64893700 1 -- 172.21.15.62:0/2013882157 learned_addr learned my addr 172.21.15.62:0/2013882157 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:04.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.866+0000 7f1e5ffff700 1 --2- 172.21.15.62:0/2013882157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5aa880 0x56111d5a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.866+0000 7f1e65094700 1 --2- 172.21.15.62:0/2013882157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a47a0 0x56111d5a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.867+0000 7f1e64893700 1 -- 172.21.15.62:0/2013882157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a47a0 msgr2=0x56111d5a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:04.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.867+0000 7f1e64893700 1 --2- 172.21.15.62:0/2013882157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a47a0 0x56111d5a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.867+0000 7f1e64893700 1 -- 172.21.15.62:0/2013882157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5aa880 msgr2=0x56111d5a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:04.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.867+0000 7f1e64893700 1 --2- 172.21.15.62:0/2013882157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5aa880 0x56111d5a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.867+0000 7f1e64893700 1 -- 172.21.15.62:0/2013882157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56111d4448d0 con 0x56111d5a9f60 2024-06-10T00:38:04.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.867+0000 7f1e64893700 1 --2- 172.21.15.62:0/2013882157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5a9f60 0x56111d5aa340 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400d7b0 tx=0x7f1e54010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67d2592ca69dddf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:04.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.867+0000 7f1e4affd700 1 -- 172.21.15.62:0/2013882157 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54022070 con 0x56111d5a9f60 2024-06-10T00:38:04.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.867+0000 7f1e4affd700 1 -- 172.21.15.62:0/2013882157 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e5401eb60 con 0x56111d5a9f60 2024-06-10T00:38:04.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.867+0000 7f1e4affd700 1 -- 172.21.15.62:0/2013882157 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54014c80 con 0x56111d5a9f60 2024-06-10T00:38:04.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.868+0000 7f1e7137ab80 1 -- 172.21.15.62:0/2013882157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5a9f60 msgr2=0x56111d5aa340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:04.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.868+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/2013882157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5a9f60 0x56111d5aa340 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400d7b0 tx=0x7f1e54010000 comp rx=0 tx=0).stop 2024-06-10T00:38:04.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.868+0000 7f1e7137ab80 1 -- 172.21.15.62:0/2013882157 shutdown_connections 2024-06-10T00:38:04.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.868+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/2013882157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5a9f60 0x56111d5aa340 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.868+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/2013882157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a47a0 0x56111d5a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.868+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/2013882157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5aa880 0x56111d5a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.868+0000 7f1e7137ab80 1 -- 172.21.15.62:0/2013882157 >> 172.21.15.62:0/2013882157 conn(0x56111d4d1080 msgr2=0x56111d59f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:04.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.868+0000 7f1e7137ab80 1 -- 172.21.15.62:0/2013882157 shutdown_connections 2024-06-10T00:38:04.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.868+0000 7f1e7137ab80 1 -- 172.21.15.62:0/2013882157 wait complete. 2024-06-10T00:38:04.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.868+0000 7f1e7137ab80 1 -- start start 2024-06-10T00:38:04.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.868+0000 7f1e7137ab80 1 Processor -- start 2024-06-10T00:38:04.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e7137ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5a47a0 0x56111d5b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e7137ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a9f60 0x56111d5ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e7137ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5aa880 0x56111d5ade10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e7137ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56111d495fc0 con 0x56111d5aa880 2024-06-10T00:38:04.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e7137ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56111d4f1810 con 0x56111d5a47a0 2024-06-10T00:38:04.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e7137ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56111d5b1260 con 0x56111d5a9f60 2024-06-10T00:38:04.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e7137ab80 10 client.0 _ll_register_callbacks cb 0x56111d48b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:04.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e65094700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5aa880 0x56111d5ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e65094700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5aa880 0x56111d5ade10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49066/0 (socket says 172.21.15.62:49066) 2024-06-10T00:38:04.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e65094700 1 -- 172.21.15.62:0/1174158744 learned_addr learned my addr 172.21.15.62:0/1174158744 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:04.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e5ffff700 1 --2- 172.21.15.62:0/1174158744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a9f60 0x56111d5ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e64893700 1 --2- 172.21.15.62:0/1174158744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5a47a0 0x56111d5b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e65094700 1 -- 172.21.15.62:0/1174158744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a9f60 msgr2=0x56111d5ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:04.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e65094700 1 --2- 172.21.15.62:0/1174158744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a9f60 0x56111d5ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e65094700 1 -- 172.21.15.62:0/1174158744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5a47a0 msgr2=0x56111d5b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:04.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e65094700 1 --2- 172.21.15.62:0/1174158744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5a47a0 0x56111d5b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e65094700 1 -- 172.21.15.62:0/1174158744 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56111d4bb750 con 0x56111d5aa880 2024-06-10T00:38:04.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.869+0000 7f1e64893700 1 --2- 172.21.15.62:0/1174158744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5a47a0 0x56111d5b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:38:04.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e65094700 1 --2- 172.21.15.62:0/1174158744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5aa880 0x56111d5ade10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1e5809b8f0 tx=0x7f1e5809b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8df1481d7fde286 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:04.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e3ffff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:04.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e3ffff700 1 -- 172.21.15.62:0/1174158744 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5809fd10 con 0x56111d5aa880 2024-06-10T00:38:04.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e3ffff700 1 -- 172.21.15.62:0/1174158744 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f1e580cf2e0 con 0x56111d5aa880 2024-06-10T00:38:04.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e7137ab80 1 -- 172.21.15.62:0/1174158744 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e5401eb60 con 0x56111d5aa880 2024-06-10T00:38:04.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e3ffff700 1 -- 172.21.15.62:0/1174158744 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1e580a45d0 con 0x56111d5aa880 2024-06-10T00:38:04.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e7137ab80 10 client.16278 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:04.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e7137ab80 1 -- 172.21.15.62:0/1174158744 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56111d5bba80 con 0x56111d5aa880 2024-06-10T00:38:04.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e3ffff700 1 -- 172.21.15.62:0/1174158744 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e5401eb60 con 0x56111d5aa880 2024-06-10T00:38:04.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e3ffff700 1 -- 172.21.15.62:0/1174158744 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f1e580a4140 con 0x56111d5aa880 2024-06-10T00:38:04.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e7137ab80 10 client.16278 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:04.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e7137ab80 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:38:04.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e7137ab80 20 client.16278 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:04.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/1174158744 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x56111d5be0b0 0x56111d5c04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e7137ab80 4 client.16278 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:38:04.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.870+0000 7f1e7137ab80 1 -- 172.21.15.62:0/1174158744 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56111d502d50 con 0x56111d5be0b0 2024-06-10T00:38:04.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.871+0000 7f1e64893700 1 --2- 172.21.15.62:0/1174158744 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x56111d5be0b0 0x56111d5c04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.871+0000 7f1e64893700 1 --2- 172.21.15.62:0/1174158744 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x56111d5be0b0 0x56111d5c04a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e2b0a861ed5fa1b server_cookie=be95538200e87a42 in_seq=0 out_seq=0 2024-06-10T00:38:04.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.871+0000 7f1e3ffff700 10 client.16278 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:04.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e3ffff700 1 -- 172.21.15.62:0/1174158744 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56111d502d50 con 0x56111d5be0b0 2024-06-10T00:38:04.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e3ffff700 10 client.16278 handle_command_reply: tid=0 2024-06-10T00:38:04.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e7137ab80 1 client.16278 shutdown 2024-06-10T00:38:04.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e7137ab80 1 -- 172.21.15.62:0/1174158744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5aa880 msgr2=0x56111d5ade10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:04.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/1174158744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5aa880 0x56111d5ade10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1e5809b8f0 tx=0x7f1e5809b3c0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e7137ab80 1 -- 172.21.15.62:0/1174158744 shutdown_connections 2024-06-10T00:38:04.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/1174158744 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x56111d5be0b0 0x56111d5c04a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/1174158744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5aa880 0x56111d5ade10 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/1174158744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5a47a0 0x56111d5b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/1174158744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a9f60 0x56111d5ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e7137ab80 1 -- 172.21.15.62:0/1174158744 >> 172.21.15.62:0/1174158744 conn(0x56111d4d1080 msgr2=0x56111d59e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:04.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e3ffff700 0 client.16278 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:04.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e7137ab80 1 -- 172.21.15.62:0/1174158744 shutdown_connections 2024-06-10T00:38:04.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e7137ab80 1 -- 172.21.15.62:0/1174158744 wait complete. 2024-06-10T00:38:04.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.872+0000 7f1e7137ab80 20 client.16278 trim_cache size 0 max 16384 2024-06-10T00:38:04.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.893+0000 7f1e7137ab80 1 Processor -- start 2024-06-10T00:38:04.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.893+0000 7f1e7137ab80 1 -- start start 2024-06-10T00:38:04.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e7137ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a47a0 0x56111d5ab370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e7137ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5a9f60 0x56111d59ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e7137ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5aa880 0x56111d59f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e7137ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601ac410 con 0x56111d5aa880 2024-06-10T00:38:04.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e7137ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601ae0d0 con 0x56111d5a9f60 2024-06-10T00:38:04.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e7137ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601acae0 con 0x56111d5a47a0 2024-06-10T00:38:04.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e65094700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5aa880 0x56111d59f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e65094700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5aa880 0x56111d59f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49074/0 (socket says 172.21.15.62:49074) 2024-06-10T00:38:04.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e65094700 1 -- 172.21.15.62:0/575069578 learned_addr learned my addr 172.21.15.62:0/575069578 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:04.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e64893700 1 --2- 172.21.15.62:0/575069578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a47a0 0x56111d5ab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e5ffff700 1 --2- 172.21.15.62:0/575069578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5a9f60 0x56111d59ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e65094700 1 -- 172.21.15.62:0/575069578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a47a0 msgr2=0x56111d5ab370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:04.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e65094700 1 --2- 172.21.15.62:0/575069578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a47a0 0x56111d5ab370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e65094700 1 -- 172.21.15.62:0/575069578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5a9f60 msgr2=0x56111d59ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:04.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e65094700 1 --2- 172.21.15.62:0/575069578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5a9f60 0x56111d59ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e65094700 1 -- 172.21.15.62:0/575069578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56111d5ab8b0 con 0x56111d5aa880 2024-06-10T00:38:04.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e65094700 1 --2- 172.21.15.62:0/575069578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5aa880 0x56111d59f3c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1e58007d40 tx=0x7f1e58006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54696916d74082d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:04.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e4a7fc700 1 -- 172.21.15.62:0/575069578 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58086100 con 0x56111d5aa880 2024-06-10T00:38:04.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e4a7fc700 1 -- 172.21.15.62:0/575069578 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e580862f0 con 0x56111d5aa880 2024-06-10T00:38:04.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.894+0000 7f1e4a7fc700 1 -- 172.21.15.62:0/575069578 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58097c00 con 0x56111d5aa880 2024-06-10T00:38:04.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.895+0000 7f1e7137ab80 1 -- 172.21.15.62:0/575069578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5aa880 msgr2=0x56111d59f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:04.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.895+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/575069578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5aa880 0x56111d59f3c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1e58007d40 tx=0x7f1e58006060 comp rx=0 tx=0).stop 2024-06-10T00:38:04.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.895+0000 7f1e7137ab80 1 -- 172.21.15.62:0/575069578 shutdown_connections 2024-06-10T00:38:04.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.895+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/575069578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5aa880 0x56111d59f3c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.895+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/575069578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a47a0 0x56111d5ab370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.895+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/575069578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5a9f60 0x56111d59ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.895+0000 7f1e7137ab80 1 -- 172.21.15.62:0/575069578 >> 172.21.15.62:0/575069578 conn(0x56111d4d1080 msgr2=0x56111d5aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:04.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.895+0000 7f1e7137ab80 1 -- 172.21.15.62:0/575069578 shutdown_connections 2024-06-10T00:38:04.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.895+0000 7f1e7137ab80 1 -- 172.21.15.62:0/575069578 wait complete. 2024-06-10T00:38:04.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.895+0000 7f1e7137ab80 1 -- start start 2024-06-10T00:38:04.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e7137ab80 1 Processor -- start 2024-06-10T00:38:04.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e7137ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5a47a0 0x56111d5b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e7137ab80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a9f60 0x56111d5b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e7137ab80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5aa880 0x56111d5c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e7137ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601ad940 con 0x56111d5a47a0 2024-06-10T00:38:04.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e7137ab80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601ad1b0 con 0x56111d5aa880 2024-06-10T00:38:04.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e7137ab80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601aa3b0 con 0x56111d5a9f60 2024-06-10T00:38:04.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e7137ab80 10 client.0 _ll_register_callbacks cb 0x56111d48b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:04.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e64893700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5a47a0 0x56111d5b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e64893700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5a47a0 0x56111d5b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49090/0 (socket says 172.21.15.62:49090) 2024-06-10T00:38:04.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e64893700 1 -- 172.21.15.62:0/2653473443 learned_addr learned my addr 172.21.15.62:0/2653473443 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:04.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e65094700 1 --2- 172.21.15.62:0/2653473443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5aa880 0x56111d5c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e5ffff700 1 --2- 172.21.15.62:0/2653473443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a9f60 0x56111d5b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e64893700 1 -- 172.21.15.62:0/2653473443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a9f60 msgr2=0x56111d5b6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:04.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e64893700 1 --2- 172.21.15.62:0/2653473443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a9f60 0x56111d5b6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e64893700 1 -- 172.21.15.62:0/2653473443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5aa880 msgr2=0x56111d5c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:04.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.896+0000 7f1e64893700 1 --2- 172.21.15.62:0/2653473443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5aa880 0x56111d5c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e64893700 1 -- 172.21.15.62:0/2653473443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56111d5c5d00 con 0x56111d5a47a0 2024-06-10T00:38:04.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e64893700 1 --2- 172.21.15.62:0/2653473443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5a47a0 0x56111d5b6920 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1e540144e0 tx=0x7f1e54009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4a85c283feeb19 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:04.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e3ffff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:04.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e3ffff700 1 -- 172.21.15.62:0/2653473443 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54022040 con 0x56111d5a47a0 2024-06-10T00:38:04.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e3ffff700 1 -- 172.21.15.62:0/2653473443 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f1e5401ea50 con 0x56111d5a47a0 2024-06-10T00:38:04.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e7137ab80 1 -- 172.21.15.62:0/2653473443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e580862f0 con 0x56111d5a47a0 2024-06-10T00:38:04.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e3ffff700 1 -- 172.21.15.62:0/2653473443 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1e54005a50 con 0x56111d5a47a0 2024-06-10T00:38:04.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e7137ab80 10 client.16284 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:04.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e7137ab80 1 -- 172.21.15.62:0/2653473443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56111d6289d0 con 0x56111d5a47a0 2024-06-10T00:38:04.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e3ffff700 1 -- 172.21.15.62:0/2653473443 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e580862f0 con 0x56111d5a47a0 2024-06-10T00:38:04.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e3ffff700 1 -- 172.21.15.62:0/2653473443 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f1e5401c040 con 0x56111d5a47a0 2024-06-10T00:38:04.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e7137ab80 10 client.16284 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:04.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e7137ab80 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:38:04.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e7137ab80 20 client.16284 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:04.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/2653473443 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x56111d628fc0 0x56111d62b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:04.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e7137ab80 4 client.16284 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:38:04.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.897+0000 7f1e7137ab80 1 -- 172.21.15.62:0/2653473443 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56111d4fd2d0 con 0x56111d628fc0 2024-06-10T00:38:04.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.898+0000 7f1e5ffff700 1 --2- 172.21.15.62:0/2653473443 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x56111d628fc0 0x56111d62b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:04.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.898+0000 7f1e5ffff700 1 --2- 172.21.15.62:0/2653473443 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x56111d628fc0 0x56111d62b3e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0883fcaa00ac87c server_cookie=257d4e13752d35e4 in_seq=0 out_seq=0 2024-06-10T00:38:04.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.898+0000 7f1e3ffff700 10 client.16284 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:04.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.898+0000 7f1e3ffff700 1 -- 172.21.15.62:0/2653473443 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56111d4fd2d0 con 0x56111d628fc0 2024-06-10T00:38:04.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.898+0000 7f1e3ffff700 10 client.16284 handle_command_reply: tid=0 2024-06-10T00:38:04.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.898+0000 7f1e7137ab80 1 client.16284 shutdown 2024-06-10T00:38:04.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e7137ab80 1 -- 172.21.15.62:0/2653473443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5a47a0 msgr2=0x56111d5b6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:04.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/2653473443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5a47a0 0x56111d5b6920 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1e540144e0 tx=0x7f1e54009e20 comp rx=0 tx=0).stop 2024-06-10T00:38:04.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e7137ab80 1 -- 172.21.15.62:0/2653473443 shutdown_connections 2024-06-10T00:38:04.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/2653473443 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x56111d628fc0 0x56111d62b3e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e3ffff700 0 client.16284 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:04.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/2653473443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56111d5a47a0 0x56111d5b6920 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/2653473443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56111d5aa880 0x56111d5c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e7137ab80 1 --2- 172.21.15.62:0/2653473443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56111d5a9f60 0x56111d5b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e7137ab80 1 -- 172.21.15.62:0/2653473443 >> 172.21.15.62:0/2653473443 conn(0x56111d4d1080 msgr2=0x56111d5af100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:04.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e7137ab80 1 -- 172.21.15.62:0/2653473443 shutdown_connections 2024-06-10T00:38:04.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e7137ab80 1 -- 172.21.15.62:0/2653473443 wait complete. 2024-06-10T00:38:04.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e7137ab80 20 client.16284 trim_cache size 0 max 16384 2024-06-10T00:38:04.913 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:38:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-06-10T00:38:04.913 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-06-10T00:38:04.913 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:38:04.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e66af7700 1 -- 172.21.15.62:0/4222536491 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1e500a2e10 msgr2=0x7f1e500a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:04.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e66af7700 1 --2- 172.21.15.62:0/4222536491 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1e500a2e10 0x7f1e500a52d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1e54006fd0 tx=0x7f1e54008040 comp rx=0 tx=0).stop 2024-06-10T00:38:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e66af7700 1 -- 172.21.15.62:0/4222536491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e60107400 msgr2=0x7f1e601979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.899+0000 7f1e66af7700 1 --2- 172.21.15.62:0/4222536491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e60107400 0x7f1e601979d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1e58007d70 tx=0x7f1e5800a340 comp rx=0 tx=0).stop 2024-06-10T00:38:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.900+0000 7f1e66af7700 1 -- 172.21.15.62:0/4222536491 shutdown_connections 2024-06-10T00:38:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.900+0000 7f1e66af7700 1 --2- 172.21.15.62:0/4222536491 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1e500a2e10 0x7f1e500a52d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.900+0000 7f1e66af7700 1 --2- 172.21.15.62:0/4222536491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e60107400 0x7f1e601979d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.900+0000 7f1e66af7700 1 --2- 172.21.15.62:0/4222536491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e60101390 0x7f1e6019d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.900+0000 7f1e66af7700 1 --2- 172.21.15.62:0/4222536491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e60101d50 0x7f1e6019dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:04.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.900+0000 7f1e66af7700 1 -- 172.21.15.62:0/4222536491 >> 172.21.15.62:0/4222536491 conn(0x7f1e60078340 msgr2=0x7f1e600ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:04.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.900+0000 7f1e66af7700 1 -- 172.21.15.62:0/4222536491 shutdown_connections 2024-06-10T00:38:04.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:04.900+0000 7f1e66af7700 1 -- 172.21.15.62:0/4222536491 wait complete. 2024-06-10T00:38:05.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:05 smithi062 ceph-mon[32390]: pgmap v160: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 302 op/s 2024-06-10T00:38:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:05 smithi105 ceph-mon[36321]: pgmap v160: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 302 op/s 2024-06-10T00:38:05.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:05 smithi138 ceph-mon[36549]: pgmap v160: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 302 op/s 2024-06-10T00:38:06.547 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T00:38:06.547 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:38:06.598 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:38:07.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:07 smithi105 ceph-mon[36321]: pgmap v161: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-06-10T00:38:07.533 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:07 smithi062 ceph-mon[32390]: pgmap v161: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-06-10T00:38:07.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:07 smithi138 ceph-mon[36549]: pgmap v161: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-06-10T00:38:09.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:09 smithi105 ceph-mon[36321]: pgmap v162: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-06-10T00:38:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:09 smithi062 ceph-mon[32390]: pgmap v162: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-06-10T00:38:09.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:09 smithi138 ceph-mon[36549]: pgmap v162: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-06-10T00:38:11.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.137+0000 7f18d6716700 1 -- 172.21.15.62:0/3398064602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d0103150 msgr2=0x7f18d0107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:11.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.137+0000 7f18d6716700 1 --2- 172.21.15.62:0/3398064602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d0103150 0x7f18d0107140 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f18c40099c0 tx=0x7f18c4009cd0 comp rx=0 tx=0).stop 2024-06-10T00:38:11.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.137+0000 7f18d6716700 1 -- 172.21.15.62:0/3398064602 shutdown_connections 2024-06-10T00:38:11.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.137+0000 7f18d6716700 1 --2- 172.21.15.62:0/3398064602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d0107870 0x7f18d010f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:11.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.137+0000 7f18d6716700 1 --2- 172.21.15.62:0/3398064602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18d01027a0 0x7f18d0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:11.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.137+0000 7f18d6716700 1 --2- 172.21.15.62:0/3398064602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d0103150 0x7f18d0107140 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:11.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.137+0000 7f18d6716700 1 -- 172.21.15.62:0/3398064602 >> 172.21.15.62:0/3398064602 conn(0x7f18d0079520 msgr2=0x7f18d0079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:11.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.137+0000 7f18d6716700 1 -- 172.21.15.62:0/3398064602 shutdown_connections 2024-06-10T00:38:11.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.137+0000 7f18d6716700 1 -- 172.21.15.62:0/3398064602 wait complete. 2024-06-10T00:38:11.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.138+0000 7f18d6716700 1 Processor -- start 2024-06-10T00:38:11.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.138+0000 7f18d6716700 1 -- start start 2024-06-10T00:38:11.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18d6716700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d01027a0 0x7f18d019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:11.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18d6716700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d0103150 0x7f18d019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:11.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18d6716700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18d0107870 0x7f18d01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:11.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18d6716700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01127a0 con 0x7f18d0103150 2024-06-10T00:38:11.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18d6716700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0112620 con 0x7f18d0107870 2024-06-10T00:38:11.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18d6716700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0112920 con 0x7f18d01027a0 2024-06-10T00:38:11.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18cf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d0103150 0x7f18d019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:11.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18cf7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d0103150 0x7f18d019ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60490/0 (socket says 172.21.15.62:60490) 2024-06-10T00:38:11.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18cf7fe700 1 -- 172.21.15.62:0/2982650289 learned_addr learned my addr 172.21.15.62:0/2982650289 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:11.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18cffff700 1 --2- 172.21.15.62:0/2982650289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d01027a0 0x7f18d019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:11.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18d4cb3700 1 --2- 172.21.15.62:0/2982650289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18d0107870 0x7f18d01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:11.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18cf7fe700 1 -- 172.21.15.62:0/2982650289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d01027a0 msgr2=0x7f18d019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:11.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18cf7fe700 1 --2- 172.21.15.62:0/2982650289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d01027a0 0x7f18d019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:11.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18cf7fe700 1 -- 172.21.15.62:0/2982650289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18d0107870 msgr2=0x7f18d01989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:11.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18cf7fe700 1 --2- 172.21.15.62:0/2982650289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18d0107870 0x7f18d01989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:11.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18cf7fe700 1 -- 172.21.15.62:0/2982650289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d0199140 con 0x7f18d0103150 2024-06-10T00:38:11.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.139+0000 7f18cf7fe700 1 --2- 172.21.15.62:0/2982650289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d0103150 0x7f18d019ed10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f18c4006010 tx=0x7f18c4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:11.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.140+0000 7f18cd7fa700 1 -- 172.21.15.62:0/2982650289 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4027070 con 0x7f18d0103150 2024-06-10T00:38:11.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.140+0000 7f18d6716700 1 -- 172.21.15.62:0/2982650289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d01993d0 con 0x7f18d0103150 2024-06-10T00:38:11.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.140+0000 7f18cd7fa700 1 -- 172.21.15.62:0/2982650289 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c4003780 con 0x7f18d0103150 2024-06-10T00:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:38:11.082 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:37:35.971947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:38:11.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.140+0000 7f18cd7fa700 1 -- 172.21.15.62:0/2982650289 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4022da0 con 0x7f18d0103150 2024-06-10T00:38:11.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.140+0000 7f18d6716700 1 -- 172.21.15.62:0/2982650289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d01a56a0 con 0x7f18d0103150 2024-06-10T00:38:11.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.141+0000 7f18d6716700 1 -- 172.21.15.62:0/2982650289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d0069180 con 0x7f18d0103150 2024-06-10T00:38:11.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.142+0000 7f18cd7fa700 1 -- 172.21.15.62:0/2982650289 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f18c402a070 con 0x7f18d0103150 2024-06-10T00:38:11.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.143+0000 7f18cd7fa700 1 --2- 172.21.15.62:0/2982650289 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f18b80a2d90 0x7f18b80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:11.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.143+0000 7f18cd7fa700 1 -- 172.21.15.62:0/2982650289 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f18c40d0480 con 0x7f18d0103150 2024-06-10T00:38:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.143+0000 7f18cffff700 1 --2- 172.21.15.62:0/2982650289 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f18b80a2d90 0x7f18b80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.143+0000 7f18cffff700 1 --2- 172.21.15.62:0/2982650289 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f18b80a2d90 0x7f18b80a5250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f18d0107400 tx=0x7f18c000b410 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.147+0000 7f18cd7fa700 1 -- 172.21.15.62:0/2982650289 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f18c4013070 con 0x7f18d0103150 2024-06-10T00:38:11.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.402+0000 7f18d6716700 1 -- 172.21.15.62:0/2982650289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18d0199ec0 con 0x7f18d0103150 2024-06-10T00:38:11.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.403+0000 7f18cd7fa700 1 -- 172.21.15.62:0/2982650289 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11952 (secure 0 0 0) 0x7f18c409cd80 con 0x7f18d0103150 2024-06-10T00:38:11.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.405+0000 7f18d6716700 1 -- 172.21.15.62:0/2982650289 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f18b80a2d90 msgr2=0x7f18b80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:11.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.405+0000 7f18d6716700 1 --2- 172.21.15.62:0/2982650289 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f18b80a2d90 0x7f18b80a5250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f18d0107400 tx=0x7f18c000b410 comp rx=0 tx=0).stop 2024-06-10T00:38:11.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.405+0000 7f18d6716700 1 -- 172.21.15.62:0/2982650289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d0103150 msgr2=0x7f18d019ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:11.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.405+0000 7f18d6716700 1 --2- 172.21.15.62:0/2982650289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d0103150 0x7f18d019ed10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f18c4006010 tx=0x7f18c4009f80 comp rx=0 tx=0).stop 2024-06-10T00:38:11.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.406+0000 7f18d6716700 1 -- 172.21.15.62:0/2982650289 shutdown_connections 2024-06-10T00:38:11.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.406+0000 7f18d6716700 1 --2- 172.21.15.62:0/2982650289 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f18b80a2d90 0x7f18b80a5250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:11.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.406+0000 7f18d6716700 1 --2- 172.21.15.62:0/2982650289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18d0107870 0x7f18d01989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:11.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.406+0000 7f18d6716700 1 --2- 172.21.15.62:0/2982650289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d01027a0 0x7f18d019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:11.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.406+0000 7f18d6716700 1 --2- 172.21.15.62:0/2982650289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18d0103150 0x7f18d019ed10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:11.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.406+0000 7f18d6716700 1 -- 172.21.15.62:0/2982650289 >> 172.21.15.62:0/2982650289 conn(0x7f18d0079520 msgr2=0x7f18d0100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:11.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.406+0000 7f18d6716700 1 -- 172.21.15.62:0/2982650289 shutdown_connections 2024-06-10T00:38:11.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:10.406+0000 7f18d6716700 1 -- 172.21.15.62:0/2982650289 wait complete. 2024-06-10T00:38:11.089 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 34 2024-06-10T00:38:11.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:11 smithi062 ceph-mon[32390]: pgmap v163: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-06-10T00:38:11.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:11 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2982650289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:38:11.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:11 smithi105 ceph-mon[36321]: pgmap v163: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-06-10T00:38:11.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:11 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2982650289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:38:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:11 smithi138 ceph-mon[36549]: pgmap v163: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-06-10T00:38:11.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:11 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2982650289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:38:13.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:13 smithi105 ceph-mon[36321]: pgmap v164: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-06-10T00:38:13.611 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:13 smithi062 ceph-mon[32390]: pgmap v164: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-06-10T00:38:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:13 smithi138 ceph-mon[36549]: pgmap v164: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-06-10T00:38:14.447 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:14 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:38:14.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:14 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:38:14.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:14 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:38:15.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:15 smithi105 ceph-mon[36321]: pgmap v165: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 22 op/s 2024-06-10T00:38:15.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:15 smithi062 ceph-mon[32390]: pgmap v165: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 22 op/s 2024-06-10T00:38:15.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:15 smithi138 ceph-mon[36549]: pgmap v165: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 22 op/s 2024-06-10T00:38:17.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:17 smithi105 ceph-mon[36321]: pgmap v166: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-10T00:38:17.574 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:17 smithi062 ceph-mon[32390]: pgmap v166: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-10T00:38:17.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:17 smithi138 ceph-mon[36549]: pgmap v166: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-10T00:38:19.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:19 smithi105 ceph-mon[36321]: pgmap v167: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-10T00:38:19.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:19 smithi062 ceph-mon[32390]: pgmap v167: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-10T00:38:19.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:19 smithi138 ceph-mon[36549]: pgmap v167: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-10T00:38:21.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:21 smithi105 ceph-mon[36321]: pgmap v168: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-10T00:38:21.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:21 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:38:21.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:21 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:38:21.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:21 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:38:21.599 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:21 smithi062 ceph-mon[32390]: pgmap v168: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-10T00:38:21.600 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:21 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:38:21.600 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:21 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:38:21.600 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:21 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:38:21.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:21 smithi138 ceph-mon[36549]: pgmap v168: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-10T00:38:21.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:21 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:38:21.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:21 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:38:21.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:21 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:38:23.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:23 smithi105 ceph-mon[36321]: pgmap v169: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T00:38:23.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:23 smithi062 ceph-mon[32390]: pgmap v169: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T00:38:23.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:23 smithi138 ceph-mon[36549]: pgmap v169: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T00:38:24.219 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:38:24.238 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db --cancel --all' 2024-06-10T00:38:24.239 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24464 quiesce db --cancel --all 2024-06-10T00:38:25.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:25 smithi062 ceph-mon[32390]: pgmap v170: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:38:25.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:25 smithi105 ceph-mon[36321]: pgmap v170: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:38:25.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:25 smithi138 ceph-mon[36549]: pgmap v170: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:38:25.996 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:38:26.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.987+0000 7fa26449a700 1 -- 172.21.15.62:0/3471522622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06a170 msgr2=0x7fa25c06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:26.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.987+0000 7fa26449a700 1 --2- 172.21.15.62:0/3471522622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06a170 0x7fa25c06a5f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009990 tx=0x7fa24c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:38:26.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.987+0000 7fa26449a700 1 -- 172.21.15.62:0/3471522622 shutdown_connections 2024-06-10T00:38:26.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.987+0000 7fa26449a700 1 --2- 172.21.15.62:0/3471522622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa25c06ab30 0x7fa25c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:26.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.988+0000 7fa26449a700 1 --2- 172.21.15.62:0/3471522622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa25c108500 0x7fa25c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:26.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.988+0000 7fa26449a700 1 --2- 172.21.15.62:0/3471522622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06a170 0x7fa25c06a5f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:26.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.988+0000 7fa26449a700 1 -- 172.21.15.62:0/3471522622 >> 172.21.15.62:0/3471522622 conn(0x7fa25c078340 msgr2=0x7fa25c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:26.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.988+0000 7fa26449a700 1 -- 172.21.15.62:0/3471522622 shutdown_connections 2024-06-10T00:38:26.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.988+0000 7fa26449a700 1 -- 172.21.15.62:0/3471522622 wait complete. 2024-06-10T00:38:26.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.988+0000 7fa26449a700 1 Processor -- start 2024-06-10T00:38:26.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.988+0000 7fa26449a700 1 -- start start 2024-06-10T00:38:26.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.988+0000 7fa26449a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa25c06a170 0x7fa25c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:26.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.988+0000 7fa26449a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06ab30 0x7fa25c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:26.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.988+0000 7fa26449a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa25c108500 0x7fa25c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:26.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.988+0000 7fa26449a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c112780 con 0x7fa25c06ab30 2024-06-10T00:38:26.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.988+0000 7fa26449a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c112600 con 0x7fa25c108500 2024-06-10T00:38:26.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.988+0000 7fa26449a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c112900 con 0x7fa25c06a170 2024-06-10T00:38:26.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa261a35700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06ab30 0x7fa25c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:26.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa261a35700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06ab30 0x7fa25c19ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38896/0 (socket says 172.21.15.62:38896) 2024-06-10T00:38:26.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa261a35700 1 -- 172.21.15.62:0/2029976840 learned_addr learned my addr 172.21.15.62:0/2029976840 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:26.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa262236700 1 --2- 172.21.15.62:0/2029976840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa25c06a170 0x7fa25c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:26.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa262a37700 1 --2- 172.21.15.62:0/2029976840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa25c108500 0x7fa25c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:26.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa261a35700 1 -- 172.21.15.62:0/2029976840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa25c06a170 msgr2=0x7fa25c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:26.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa261a35700 1 --2- 172.21.15.62:0/2029976840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa25c06a170 0x7fa25c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:26.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa261a35700 1 -- 172.21.15.62:0/2029976840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa25c108500 msgr2=0x7fa25c198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:26.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa261a35700 1 --2- 172.21.15.62:0/2029976840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa25c108500 0x7fa25c198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:26.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa261a35700 1 -- 172.21.15.62:0/2029976840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25c199200 con 0x7fa25c06ab30 2024-06-10T00:38:26.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa261a35700 1 --2- 172.21.15.62:0/2029976840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06ab30 0x7fa25c19ee20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0038b0 tx=0x7fa24c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:26.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa2537fe700 1 -- 172.21.15.62:0/2029976840 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c027070 con 0x7fa25c06ab30 2024-06-10T00:38:26.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa26449a700 1 -- 172.21.15.62:0/2029976840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25c199490 con 0x7fa25c06ab30 2024-06-10T00:38:26.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa2537fe700 1 -- 172.21.15.62:0/2029976840 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24c004050 con 0x7fa25c06ab30 2024-06-10T00:38:26.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.989+0000 7fa2537fe700 1 -- 172.21.15.62:0/2029976840 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c022c00 con 0x7fa25c06ab30 2024-06-10T00:38:26.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.990+0000 7fa26449a700 1 -- 172.21.15.62:0/2029976840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa25c1a57b0 con 0x7fa25c06ab30 2024-06-10T00:38:26.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.991+0000 7fa26449a700 1 -- 172.21.15.62:0/2029976840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa25c069180 con 0x7fa25c06ab30 2024-06-10T00:38:26.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.992+0000 7fa2537fe700 1 -- 172.21.15.62:0/2029976840 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa24c02a070 con 0x7fa25c06ab30 2024-06-10T00:38:26.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.992+0000 7fa2537fe700 1 --2- 172.21.15.62:0/2029976840 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa2480a2d90 0x7fa2480a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.992+0000 7fa262236700 1 --2- 172.21.15.62:0/2029976840 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa2480a2d90 0x7fa2480a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.992+0000 7fa2537fe700 1 -- 172.21.15.62:0/2029976840 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fa24c0d09d0 con 0x7fa25c06ab30 2024-06-10T00:38:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.993+0000 7fa262236700 1 --2- 172.21.15.62:0/2029976840 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa2480a2d90 0x7fa2480a5250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa25c199be0 tx=0x7fa25800b410 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:26.997+0000 7fa2537fe700 1 -- 172.21.15.62:0/2029976840 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa24c013070 con 0x7fa25c06ab30 2024-06-10T00:38:27.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.225+0000 7f8a0440e700 1 -- 172.21.15.62:0/165113848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc108520 msgr2=0x7f89fc108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:27.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.225+0000 7f8a0440e700 1 --2- 172.21.15.62:0/165113848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc108520 0x7f89fc108900 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009990 tx=0x7f89f8009ca0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.225+0000 7f8a0440e700 1 -- 172.21.15.62:0/165113848 shutdown_connections 2024-06-10T00:38:27.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.225+0000 7f8a0440e700 1 --2- 172.21.15.62:0/165113848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc108520 0x7f89fc108900 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.225+0000 7f8a0440e700 1 --2- 172.21.15.62:0/165113848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89fc06b420 0x7f89fc10f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.225+0000 7f8a0440e700 1 --2- 172.21.15.62:0/165113848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89fc06aa60 0x7f89fc06aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.225+0000 7f8a0440e700 1 -- 172.21.15.62:0/165113848 >> 172.21.15.62:0/165113848 conn(0x7f89fc0783d0 msgr2=0x7f89fc0787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:27.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.225+0000 7f8a0440e700 1 -- 172.21.15.62:0/165113848 shutdown_connections 2024-06-10T00:38:27.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.225+0000 7f8a0440e700 1 -- 172.21.15.62:0/165113848 wait complete. 2024-06-10T00:38:27.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a0440e700 1 Processor -- start 2024-06-10T00:38:27.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a0440e700 1 -- start start 2024-06-10T00:38:27.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a0440e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89fc06aa60 0x7f89fc19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a0440e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc06b420 0x7f89fc19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a0440e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89fc108520 0x7f89fc19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a0440e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1127a0 con 0x7f89fc06b420 2024-06-10T00:38:27.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a0440e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc112620 con 0x7f89fc108520 2024-06-10T00:38:27.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a0440e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc112920 con 0x7f89fc06aa60 2024-06-10T00:38:27.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a019a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc06b420 0x7f89fc19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a019a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc06b420 0x7f89fc19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38912/0 (socket says 172.21.15.62:38912) 2024-06-10T00:38:27.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a019a9700 1 -- 172.21.15.62:0/1633909770 learned_addr learned my addr 172.21.15.62:0/1633909770 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a021aa700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89fc06aa60 0x7f89fc19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a029ab700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89fc108520 0x7f89fc19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a019a9700 1 -- 172.21.15.62:0/1633909770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89fc06aa60 msgr2=0x7f89fc19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a019a9700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89fc06aa60 0x7f89fc19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a019a9700 1 -- 172.21.15.62:0/1633909770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89fc108520 msgr2=0x7f89fc19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a019a9700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89fc108520 0x7f89fc19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a019a9700 1 -- 172.21.15.62:0/1633909770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89fc19fd60 con 0x7f89fc06b420 2024-06-10T00:38:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a029ab700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89fc108520 0x7f89fc19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:38:27.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.226+0000 7f8a019a9700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc06b420 0x7f89fc19b2e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00d980 tx=0x7f89ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:27.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.227+0000 7f89f37fe700 1 -- 172.21.15.62:0/1633909770 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec017070 con 0x7f89fc06b420 2024-06-10T00:38:27.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.227+0000 7f89f37fe700 1 -- 172.21.15.62:0/1633909770 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ec005020 con 0x7f89fc06b420 2024-06-10T00:38:27.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.227+0000 7f8a0440e700 1 -- 172.21.15.62:0/1633909770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89fc1a0050 con 0x7f89fc06b420 2024-06-10T00:38:27.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.227+0000 7f89f37fe700 1 -- 172.21.15.62:0/1633909770 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec009bc0 con 0x7f89fc06b420 2024-06-10T00:38:27.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.227+0000 7f8a0440e700 1 -- 172.21.15.62:0/1633909770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89fc1a78d0 con 0x7f89fc06b420 2024-06-10T00:38:27.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.230+0000 7f89f37fe700 1 -- 172.21.15.62:0/1633909770 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f89ec0051d0 con 0x7f89fc06b420 2024-06-10T00:38:27.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.230+0000 7f8a0ec91b80 1 Processor -- start 2024-06-10T00:38:27.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.231+0000 7f8a0ec91b80 1 -- start start 2024-06-10T00:38:27.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.231+0000 7f89f37fe700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f89e80a2eb0 0x7f89e80a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.231+0000 7f8a0ec91b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca7cd0 0x55c0fcca80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.231+0000 7f8a0ec91b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca85f0 0x55c0fcca1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.231+0000 7f8a0ec91b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca2460 0x55c0fcca28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.231+0000 7f8a0ec91b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c0fcbf39a0 con 0x55c0fcca85f0 2024-06-10T00:38:27.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.231+0000 7f8a0ec91b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c0fcb9b8b0 con 0x55c0fcca7cd0 2024-06-10T00:38:27.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.231+0000 7f8a0ec91b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c0fcbef960 con 0x55c0fcca2460 2024-06-10T00:38:27.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.231+0000 7f8a029ab700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f89e80a2eb0 0x7f89e80a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.231+0000 7f8a021aa700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca7cd0 0x55c0fcca80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.231+0000 7f89f37fe700 1 -- 172.21.15.62:0/1633909770 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f89ec0cf6c0 con 0x7f89fc06b420 2024-06-10T00:38:27.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.232+0000 7f8a019a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca85f0 0x55c0fcca1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.232+0000 7f8a021aa700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca7cd0 0x55c0fcca80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:36122/0 (socket says 172.21.15.62:36122) 2024-06-10T00:38:27.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.232+0000 7f8a021aa700 1 -- 172.21.15.62:0/4034674289 learned_addr learned my addr 172.21.15.62:0/4034674289 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:27.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.232+0000 7f8a029ab700 1 --2- 172.21.15.62:0/4034674289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca2460 0x55c0fcca28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.232+0000 7f8a019a9700 1 -- 172.21.15.62:0/4034674289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca2460 msgr2=0x55c0fcca28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:27.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.232+0000 7f8a019a9700 1 --2- 172.21.15.62:0/4034674289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca2460 0x55c0fcca28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.232+0000 7f8a019a9700 1 -- 172.21.15.62:0/4034674289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca7cd0 msgr2=0x55c0fcca80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:27.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.232+0000 7f8a019a9700 1 --2- 172.21.15.62:0/4034674289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca7cd0 0x55c0fcca80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.232+0000 7f8a019a9700 1 -- 172.21.15.62:0/4034674289 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0fcb920e0 con 0x55c0fcca85f0 2024-06-10T00:38:27.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.232+0000 7f8a021aa700 1 --2- 172.21.15.62:0/4034674289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca7cd0 0x55c0fcca80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:38:27.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.232+0000 7f8a019a9700 1 --2- 172.21.15.62:0/4034674289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca85f0 0x55c0fcca1f20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0cca60 tx=0x7f89ec09b2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d5333050d7769 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:27.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.232+0000 7f8a029ab700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f89e80a2eb0 0x7f89e80a5370 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f89f40094f0 tx=0x7f89f4009240 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:27.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.233+0000 7f89f0d70700 1 -- 172.21.15.62:0/4034674289 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec0a44a0 con 0x55c0fcca85f0 2024-06-10T00:38:27.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.233+0000 7f89f0d70700 1 -- 172.21.15.62:0/4034674289 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ec099b30 con 0x55c0fcca85f0 2024-06-10T00:38:27.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.233+0000 7f89f0d70700 1 -- 172.21.15.62:0/4034674289 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec0c05b0 con 0x55c0fcca85f0 2024-06-10T00:38:27.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.234+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/4034674289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca85f0 msgr2=0x55c0fcca1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:27.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.234+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/4034674289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca85f0 0x55c0fcca1f20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0cca60 tx=0x7f89ec09b2a0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.234+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/4034674289 shutdown_connections 2024-06-10T00:38:27.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.234+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/4034674289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca2460 0x55c0fcca28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.234+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/4034674289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca7cd0 0x55c0fcca80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.234+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/4034674289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca85f0 0x55c0fcca1f20 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.234+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/4034674289 >> 172.21.15.62:0/4034674289 conn(0x55c0fcbcce50 msgr2=0x55c0fcc9d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:27.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.234+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/4034674289 shutdown_connections 2024-06-10T00:38:27.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.234+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/4034674289 wait complete. 2024-06-10T00:38:27.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.234+0000 7f8a0ec91b80 1 -- start start 2024-06-10T00:38:27.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.234+0000 7f8a0ec91b80 1 Processor -- start 2024-06-10T00:38:27.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a0ec91b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca2460 0x55c0fccae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a0ec91b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca7cd0 0x55c0fccab640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a0ec91b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca85f0 0x55c0fccabb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a0ec91b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c0fccaefa0 con 0x55c0fcca7cd0 2024-06-10T00:38:27.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a0ec91b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c0fcb93d80 con 0x55c0fcca2460 2024-06-10T00:38:27.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a0ec91b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c0fccaf120 con 0x55c0fcca85f0 2024-06-10T00:38:27.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a0ec91b80 10 client.0 _ll_register_callbacks cb 0x55c0fcb8dab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:27.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a019a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca7cd0 0x55c0fccab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a029ab700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca85f0 0x55c0fccabb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a019a9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca7cd0 0x55c0fccab640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38926/0 (socket says 172.21.15.62:38926) 2024-06-10T00:38:27.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a019a9700 1 -- 172.21.15.62:0/3602122716 learned_addr learned my addr 172.21.15.62:0/3602122716 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:27.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a021aa700 1 --2- 172.21.15.62:0/3602122716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca2460 0x55c0fccae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a029ab700 1 -- 172.21.15.62:0/3602122716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca2460 msgr2=0x55c0fccae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:27.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a029ab700 1 --2- 172.21.15.62:0/3602122716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca2460 0x55c0fccae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a029ab700 1 -- 172.21.15.62:0/3602122716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca7cd0 msgr2=0x55c0fccab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:27.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a029ab700 1 --2- 172.21.15.62:0/3602122716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca7cd0 0x55c0fccab640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.235+0000 7f8a029ab700 1 -- 172.21.15.62:0/3602122716 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0fcb42a60 con 0x55c0fcca85f0 2024-06-10T00:38:27.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.236+0000 7f8a019a9700 1 --2- 172.21.15.62:0/3602122716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca7cd0 0x55c0fccab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:38:27.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.236+0000 7f8a029ab700 1 --2- 172.21.15.62:0/3602122716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca85f0 0x55c0fccabb80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f89f401a000 tx=0x7f89f4004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cc4525fe6eb79c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:27.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.236+0000 7f89e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:38:27.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.236+0000 7f89e5ffb700 1 -- 172.21.15.62:0/3602122716 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f402a060 con 0x55c0fcca85f0 2024-06-10T00:38:27.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.236+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/3602122716 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89ec099b30 con 0x55c0fcca85f0 2024-06-10T00:38:27.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.236+0000 7f89e5ffb700 1 -- 172.21.15.62:0/3602122716 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f89f4025430 con 0x55c0fcca85f0 2024-06-10T00:38:27.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.236+0000 7f89e5ffb700 1 -- 172.21.15.62:0/3602122716 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f89f4031050 con 0x55c0fcca85f0 2024-06-10T00:38:27.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.236+0000 7f8a0ec91b80 10 client.25807 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:27.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.236+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/3602122716 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c0fccb98d0 con 0x55c0fcca85f0 2024-06-10T00:38:27.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.237+0000 7f89e5ffb700 1 -- 172.21.15.62:0/3602122716 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ec099b30 con 0x55c0fcca85f0 2024-06-10T00:38:27.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.237+0000 7f89e5ffb700 1 -- 172.21.15.62:0/3602122716 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f89f4015070 con 0x55c0fcca85f0 2024-06-10T00:38:27.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.237+0000 7f8a0ec91b80 10 client.25807 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:27.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.237+0000 7f8a0ec91b80 10 client.25807 resolve_mds: validated gid 24464 aka daemon mds.b 2024-06-10T00:38:27.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.237+0000 7f8a0ec91b80 20 client.25807 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:27.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.237+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/3602122716 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c0fccbbf10 0x55c0fccbe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.237+0000 7f8a0ec91b80 4 client.25807 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:38:27.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.237+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/3602122716 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0fcc00ea0 con 0x55c0fccbbf10 2024-06-10T00:38:27.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.237+0000 7f8a021aa700 1 --2- 172.21.15.62:0/3602122716 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c0fccbbf10 0x55c0fccbe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.238+0000 7f8a021aa700 1 --2- 172.21.15.62:0/3602122716 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c0fccbbf10 0x55c0fccbe300 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca910d84c811aea server_cookie=14a83bb50f2f3829 in_seq=0 out_seq=0 2024-06-10T00:38:27.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.238+0000 7f89e5ffb700 10 client.25807 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:27.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.238+0000 7f89e5ffb700 1 -- 172.21.15.62:0/3602122716 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c0fcc00ea0 con 0x55c0fccbbf10 2024-06-10T00:38:27.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.239+0000 7f89e5ffb700 10 client.25807 handle_command_reply: tid=0 2024-06-10T00:38:27.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.239+0000 7f8a0ec91b80 1 client.25807 shutdown 2024-06-10T00:38:27.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.239+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/3602122716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca85f0 msgr2=0x55c0fccabb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:27.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.239+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/3602122716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca85f0 0x55c0fccabb80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f89f401a000 tx=0x7f89f4004b40 comp rx=0 tx=0).stop 2024-06-10T00:38:27.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.239+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/3602122716 shutdown_connections 2024-06-10T00:38:27.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.239+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/3602122716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca85f0 0x55c0fccabb80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.239+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/3602122716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca2460 0x55c0fccae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.239+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/3602122716 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c0fccbbf10 0x55c0fccbe300 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.239+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/3602122716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca7cd0 0x55c0fccab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.239+0000 7f89e5ffb700 0 client.25807 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:27.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.239+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/3602122716 >> 172.21.15.62:0/3602122716 conn(0x55c0fcbcce50 msgr2=0x55c0fcc9cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:27.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.239+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/3602122716 shutdown_connections 2024-06-10T00:38:27.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.239+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/3602122716 wait complete. 2024-06-10T00:38:27.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.239+0000 7f8a0ec91b80 20 client.25807 trim_cache size 0 max 16384 2024-06-10T00:38:27.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.259+0000 7fa26449a700 1 -- 172.21.15.62:0/2029976840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa25c199fd0 con 0x7fa25c06ab30 2024-06-10T00:38:27.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.260+0000 7f8a0ec91b80 1 Processor -- start 2024-06-10T00:38:27.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.260+0000 7f8a0ec91b80 1 -- start start 2024-06-10T00:38:27.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.260+0000 7f8a0ec91b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca2460 0x55c0fcca1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.260+0000 7f8a0ec91b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca7cd0 0x55c0fccabbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.260+0000 7f8a0ec91b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca85f0 0x55c0fccc3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.260+0000 7f8a0ec91b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1b0750 con 0x55c0fcca2460 2024-06-10T00:38:27.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.260+0000 7f8a0ec91b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1b2410 con 0x55c0fcca7cd0 2024-06-10T00:38:27.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.260+0000 7f8a0ec91b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1b0e20 con 0x55c0fcca85f0 2024-06-10T00:38:27.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.260+0000 7fa2537fe700 1 -- 172.21.15.62:0/2029976840 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11952 (secure 0 0 0) 0x7fa24c09d2d0 con 0x7fa25c06ab30 2024-06-10T00:38:27.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.260+0000 7f8a021aa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca2460 0x55c0fcca1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.260+0000 7f8a021aa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca2460 0x55c0fcca1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38928/0 (socket says 172.21.15.62:38928) 2024-06-10T00:38:27.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.260+0000 7f8a021aa700 1 -- 172.21.15.62:0/305827879 learned_addr learned my addr 172.21.15.62:0/305827879 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:27.263 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:38:27.263 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:37:35.971947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:38:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.261+0000 7f8a021aa700 1 -- 172.21.15.62:0/305827879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca85f0 msgr2=0x55c0fccc3c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:38:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.261+0000 7f8a021aa700 1 --2- 172.21.15.62:0/305827879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca85f0 0x55c0fccc3c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.261+0000 7f8a021aa700 1 -- 172.21.15.62:0/305827879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca7cd0 msgr2=0x55c0fccabbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.261+0000 7f8a021aa700 1 --2- 172.21.15.62:0/305827879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca7cd0 0x55c0fccabbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.261+0000 7f8a021aa700 1 -- 172.21.15.62:0/305827879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0fcca77d0 con 0x55c0fcca2460 2024-06-10T00:38:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.261+0000 7f8a021aa700 1 --2- 172.21.15.62:0/305827879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca2460 0x55c0fcca1f20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009990 tx=0x7f89f8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf398cadb9dcfd8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.261+0000 7f89e7fff700 1 -- 172.21.15.62:0/305827879 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f80050b0 con 0x55c0fcca2460 2024-06-10T00:38:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.261+0000 7f89e7fff700 1 -- 172.21.15.62:0/305827879 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f8003680 con 0x55c0fcca2460 2024-06-10T00:38:27.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.261+0000 7f89e7fff700 1 -- 172.21.15.62:0/305827879 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f800f6c0 con 0x55c0fcca2460 2024-06-10T00:38:27.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.262+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/305827879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca2460 msgr2=0x55c0fcca1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:27.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.262+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/305827879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca2460 0x55c0fcca1f20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009990 tx=0x7f89f8005040 comp rx=0 tx=0).stop 2024-06-10T00:38:27.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.262+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/305827879 shutdown_connections 2024-06-10T00:38:27.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.262+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/305827879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca2460 0x55c0fcca1f20 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.262+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/305827879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca85f0 0x55c0fccc3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.262+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/305827879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca7cd0 0x55c0fccabbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.262+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/305827879 >> 172.21.15.62:0/305827879 conn(0x55c0fcbcce50 msgr2=0x55c0fcc9cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:27.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.262+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/305827879 shutdown_connections 2024-06-10T00:38:27.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.262+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/305827879 wait complete. 2024-06-10T00:38:27.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.262+0000 7f8a0ec91b80 1 -- start start 2024-06-10T00:38:27.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7f8a0ec91b80 1 Processor -- start 2024-06-10T00:38:27.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7f8a0ec91b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca2460 0x55c0fccb1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7f8a0ec91b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca7cd0 0x55c0fccb18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7f8a0ec91b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca85f0 0x55c0fccb1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7f8a0ec91b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1b1c80 con 0x55c0fcca2460 2024-06-10T00:38:27.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7f8a0ec91b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1b14f0 con 0x55c0fcca85f0 2024-06-10T00:38:27.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7f8a0ec91b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1ae6f0 con 0x55c0fcca7cd0 2024-06-10T00:38:27.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7f8a0ec91b80 10 client.0 _ll_register_callbacks cb 0x55c0fcb8dab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:27.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7f8a021aa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca2460 0x55c0fccb1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7f8a021aa700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca2460 0x55c0fccb1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38930/0 (socket says 172.21.15.62:38930) 2024-06-10T00:38:27.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7f8a021aa700 1 -- 172.21.15.62:0/2775436464 learned_addr learned my addr 172.21.15.62:0/2775436464 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:27.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7f8a029ab700 1 --2- 172.21.15.62:0/2775436464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca85f0 0x55c0fccb1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7f8a019a9700 1 --2- 172.21.15.62:0/2775436464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca7cd0 0x55c0fccb18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7fa26449a700 1 -- 172.21.15.62:0/2029976840 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa2480a2d90 msgr2=0x7fa2480a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:27.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7fa26449a700 1 --2- 172.21.15.62:0/2029976840 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa2480a2d90 0x7fa2480a5250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa25c199be0 tx=0x7fa25800b410 comp rx=0 tx=0).stop 2024-06-10T00:38:27.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7fa26449a700 1 -- 172.21.15.62:0/2029976840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06ab30 msgr2=0x7fa25c19ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:27.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.263+0000 7fa26449a700 1 --2- 172.21.15.62:0/2029976840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06ab30 0x7fa25c19ee20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0038b0 tx=0x7fa24c009f50 comp rx=0 tx=0).stop 2024-06-10T00:38:27.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7fa26449a700 1 -- 172.21.15.62:0/2029976840 shutdown_connections 2024-06-10T00:38:27.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7fa26449a700 1 --2- 172.21.15.62:0/2029976840 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa2480a2d90 0x7fa2480a5250 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7f8a021aa700 1 -- 172.21.15.62:0/2775436464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca7cd0 msgr2=0x55c0fccb18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:27.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7f8a021aa700 1 --2- 172.21.15.62:0/2775436464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca7cd0 0x55c0fccb18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7fa26449a700 1 --2- 172.21.15.62:0/2029976840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa25c108500 0x7fa25c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7fa26449a700 1 --2- 172.21.15.62:0/2029976840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa25c06a170 0x7fa25c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7fa26449a700 1 --2- 172.21.15.62:0/2029976840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa25c06ab30 0x7fa25c19ee20 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7fa26449a700 1 -- 172.21.15.62:0/2029976840 >> 172.21.15.62:0/2029976840 conn(0x7fa25c078340 msgr2=0x7fa25c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:27.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7fa26449a700 1 -- 172.21.15.62:0/2029976840 shutdown_connections 2024-06-10T00:38:27.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7f8a021aa700 1 -- 172.21.15.62:0/2775436464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca85f0 msgr2=0x55c0fccb1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:27.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7fa26449a700 1 -- 172.21.15.62:0/2029976840 wait complete. 2024-06-10T00:38:27.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7f8a021aa700 1 --2- 172.21.15.62:0/2775436464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca85f0 0x55c0fccb1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7f8a021aa700 1 -- 172.21.15.62:0/2775436464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0fcd26c20 con 0x55c0fcca2460 2024-06-10T00:38:27.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7f8a029ab700 1 --2- 172.21.15.62:0/2775436464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca85f0 0x55c0fccb1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:38:27.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7f8a021aa700 1 --2- 172.21.15.62:0/2775436464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca2460 0x55c0fccb1380 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f89f800b2e0 tx=0x7f89f8005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c15242befc2791 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:27.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7f89e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:27.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7f89e5ffb700 1 -- 172.21.15.62:0/2775436464 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8003a40 con 0x55c0fcca2460 2024-06-10T00:38:27.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7f89e5ffb700 1 -- 172.21.15.62:0/2775436464 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f89f8003be0 con 0x55c0fcca2460 2024-06-10T00:38:27.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.264+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/2775436464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f8003680 con 0x55c0fcca2460 2024-06-10T00:38:27.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.265+0000 7f89e5ffb700 1 -- 172.21.15.62:0/2775436464 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f89f80050b0 con 0x55c0fcca2460 2024-06-10T00:38:27.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.265+0000 7f8a0ec91b80 10 client.16314 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:27.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.265+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/2775436464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c0fcd27e50 con 0x55c0fcca2460 2024-06-10T00:38:27.275 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 34 2024-06-10T00:38:27.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.265+0000 7f89e5ffb700 1 -- 172.21.15.62:0/2775436464 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f8003680 con 0x55c0fcca2460 2024-06-10T00:38:27.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.265+0000 7f89e5ffb700 1 -- 172.21.15.62:0/2775436464 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f89f800fc70 con 0x55c0fcca2460 2024-06-10T00:38:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.265+0000 7f8a0ec91b80 10 client.16314 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.265+0000 7f8a0ec91b80 10 client.16314 resolve_mds: validated gid 24464 aka daemon mds.b 2024-06-10T00:38:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.265+0000 7f8a0ec91b80 20 client.16314 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.265+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/2775436464 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c0fcd28590 0x55c0fcd2a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.265+0000 7f8a0ec91b80 4 client.16314 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.2, "await_for": 45.0}] 2024-06-10T00:38:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.265+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/2775436464 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.2, "await_for": 45.0}) -- 0x55c0fcbfb420 con 0x55c0fcd28590 2024-06-10T00:38:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.265+0000 7f8a019a9700 1 --2- 172.21.15.62:0/2775436464 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c0fcd28590 0x55c0fcd2a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:27.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.266+0000 7f89e5ffb700 10 client.16314 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:27.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.266+0000 7f8a019a9700 1 --2- 172.21.15.62:0/2775436464 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c0fcd28590 0x55c0fcd2a9b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f11950bfa5ce9e server_cookie=5912e009681b0136 in_seq=0 out_seq=0 2024-06-10T00:38:27.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f89e5ffb700 1 -- 172.21.15.62:0/2775436464 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c0fcbfb420 con 0x55c0fcd28590 2024-06-10T00:38:27.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f89e5ffb700 10 client.16314 handle_command_reply: tid=0 2024-06-10T00:38:27.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f8a0ec91b80 1 client.16314 shutdown 2024-06-10T00:38:27.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/2775436464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca2460 msgr2=0x55c0fccb1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:27.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/2775436464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca2460 0x55c0fccb1380 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f89f800b2e0 tx=0x7f89f8005980 comp rx=0 tx=0).stop 2024-06-10T00:38:27.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/2775436464 shutdown_connections 2024-06-10T00:38:27.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/2775436464 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c0fcd28590 0x55c0fcd2a9b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/2775436464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c0fcca2460 0x55c0fccb1380 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/2775436464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0fcca85f0 0x55c0fccb1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f8a0ec91b80 1 --2- 172.21.15.62:0/2775436464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0fcca7cd0 0x55c0fccb18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/2775436464 >> 172.21.15.62:0/2775436464 conn(0x55c0fcbcce50 msgr2=0x55c0fcca67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:27.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f89e5ffb700 0 client.16314 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:27.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/2775436464 shutdown_connections 2024-06-10T00:38:27.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f8a0ec91b80 1 -- 172.21.15.62:0/2775436464 wait complete. 2024-06-10T00:38:27.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.401+0000 7f8a0ec91b80 20 client.16314 trim_cache size 0 max 16384 2024-06-10T00:38:27.404 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:38:27.404 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 34, 2024-06-10T00:38:27.404 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24464, 2024-06-10T00:38:27.404 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 10, 2024-06-10T00:38:27.404 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-06-10T00:38:27.405 INFO:teuthology.orchestra.run.smithi062.stdout: "b26818f6": { 2024-06-10T00:38:27.405 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 10, 2024-06-10T00:38:27.405 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-06-10T00:38:27.405 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-06-10T00:38:27.405 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-06-10T00:38:27.405 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-06-10T00:38:27.405 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:38:27.405 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-06-10T00:38:27.406 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 138.2, 2024-06-10T00:38:27.406 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-06-10T00:38:27.406 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-06-10T00:38:27.406 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-06-10T00:38:27.406 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-06-10T00:38:27.406 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-06-10T00:38:27.406 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-06-10T00:38:27.407 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:38:27.407 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:38:27.407 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:38:27.407 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:38:27.407 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:38:27.407 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:38:27.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.402+0000 7f8a0440e700 1 -- 172.21.15.62:0/1633909770 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f89e80a2eb0 msgr2=0x7f89e80a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:27.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.402+0000 7f8a0440e700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f89e80a2eb0 0x7f89e80a5370 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f89f40094f0 tx=0x7f89f4009240 comp rx=0 tx=0).stop 2024-06-10T00:38:27.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.402+0000 7f8a0440e700 1 -- 172.21.15.62:0/1633909770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc06b420 msgr2=0x7f89fc19b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:27.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.402+0000 7f8a0440e700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc06b420 0x7f89fc19b2e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00d980 tx=0x7f89ec00dc90 comp rx=0 tx=0).stop 2024-06-10T00:38:27.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.402+0000 7f8a0440e700 1 -- 172.21.15.62:0/1633909770 shutdown_connections 2024-06-10T00:38:27.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.402+0000 7f8a0440e700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f89e80a2eb0 0x7f89e80a5370 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.402+0000 7f8a0440e700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89fc108520 0x7f89fc19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.402+0000 7f8a0440e700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89fc06aa60 0x7f89fc19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.402+0000 7f8a0440e700 1 --2- 172.21.15.62:0/1633909770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89fc06b420 0x7f89fc19b2e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:27.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.402+0000 7f8a0440e700 1 -- 172.21.15.62:0/1633909770 >> 172.21.15.62:0/1633909770 conn(0x7f89fc0783d0 msgr2=0x7f89fc10d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:27.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.402+0000 7f8a0440e700 1 -- 172.21.15.62:0/1633909770 shutdown_connections 2024-06-10T00:38:27.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:27.402+0000 7f8a0440e700 1 -- 172.21.15.62:0/1633909770 wait complete. 2024-06-10T00:38:27.460 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:27 smithi062 ceph-mon[32390]: pgmap v171: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:38:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:27 smithi105 ceph-mon[36321]: pgmap v171: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:38:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:27 smithi138 ceph-mon[36549]: pgmap v171: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:38:28.493 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:28 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2029976840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:38:28.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:28 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2029976840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:38:28.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:28 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2029976840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:38:29.188 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b26818f6', quiesce took 77.2 seconds. Will release after: -59.0 2024-06-10T00:38:29.188 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:38:29.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:29 smithi105 ceph-mon[36321]: pgmap v172: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:38:29.534 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:29 smithi062 ceph-mon[32390]: pgmap v172: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:38:29.564 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T00:38:29.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:29 smithi138 ceph-mon[36549]: pgmap v172: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:38:29.860 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:38:30.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.331+0000 7f7dae2fd700 1 -- 172.21.15.62:0/1168745989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7da81084f0 msgr2=0x7f7da81088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:30.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.331+0000 7f7dae2fd700 1 --2- 172.21.15.62:0/1168745989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7da81084f0 0x7f7da81088d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009990 tx=0x7f7d98009ca0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.331+0000 7f7dae2fd700 1 -- 172.21.15.62:0/1168745989 shutdown_connections 2024-06-10T00:38:30.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.331+0000 7f7dae2fd700 1 --2- 172.21.15.62:0/1168745989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7da81084f0 0x7f7da81088d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.331+0000 7f7dae2fd700 1 --2- 172.21.15.62:0/1168745989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da806b3c0 0x7f7da810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.331+0000 7f7dae2fd700 1 --2- 172.21.15.62:0/1168745989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7da806aa00 0x7f7da806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.331+0000 7f7dae2fd700 1 -- 172.21.15.62:0/1168745989 >> 172.21.15.62:0/1168745989 conn(0x7f7da8078300 msgr2=0x7f7da8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:30.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.331+0000 7f7dae2fd700 1 -- 172.21.15.62:0/1168745989 shutdown_connections 2024-06-10T00:38:30.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.331+0000 7f7dae2fd700 1 -- 172.21.15.62:0/1168745989 wait complete. 2024-06-10T00:38:30.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.332+0000 7f7dae2fd700 1 Processor -- start 2024-06-10T00:38:30.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.332+0000 7f7dae2fd700 1 -- start start 2024-06-10T00:38:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7dae2fd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da806aa00 0x7f7da819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7dae2fd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7da806b3c0 0x7f7da819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7dae2fd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7da81084f0 0x7f7da8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7dae2fd700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7da8112750 con 0x7f7da81084f0 2024-06-10T00:38:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7dae2fd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81125d0 con 0x7f7da806aa00 2024-06-10T00:38:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7dae2fd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81128d0 con 0x7f7da806b3c0 2024-06-10T00:38:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7dac89a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7da81084f0 0x7f7da8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7dac89a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7da81084f0 0x7f7da8198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41862/0 (socket says 172.21.15.62:41862) 2024-06-10T00:38:30.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7dac89a700 1 -- 172.21.15.62:0/2825405151 learned_addr learned my addr 172.21.15.62:0/2825405151 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:30.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7da77fe700 1 --2- 172.21.15.62:0/2825405151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7da806b3c0 0x7f7da819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7da7fff700 1 --2- 172.21.15.62:0/2825405151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da806aa00 0x7f7da819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7dac89a700 1 -- 172.21.15.62:0/2825405151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7da806b3c0 msgr2=0x7f7da819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:30.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7dac89a700 1 --2- 172.21.15.62:0/2825405151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7da806b3c0 0x7f7da819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7dac89a700 1 -- 172.21.15.62:0/2825405151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da806aa00 msgr2=0x7f7da819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:30.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7dac89a700 1 --2- 172.21.15.62:0/2825405151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da806aa00 0x7f7da819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.333+0000 7f7dac89a700 1 -- 172.21.15.62:0/2825405151 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da81991b0 con 0x7f7da81084f0 2024-06-10T00:38:30.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.334+0000 7f7dac89a700 1 --2- 172.21.15.62:0/2825405151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7da81084f0 0x7f7da8198a50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7da0007ee0 tx=0x7f7da000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:30.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.334+0000 7f7da57fa700 1 -- 172.21.15.62:0/2825405151 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0004bb0 con 0x7f7da81084f0 2024-06-10T00:38:30.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.334+0000 7f7dae2fd700 1 -- 172.21.15.62:0/2825405151 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da81994a0 con 0x7f7da81084f0 2024-06-10T00:38:30.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.334+0000 7f7da57fa700 1 -- 172.21.15.62:0/2825405151 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da0004d50 con 0x7f7da81084f0 2024-06-10T00:38:30.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.334+0000 7f7da57fa700 1 -- 172.21.15.62:0/2825405151 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0005960 con 0x7f7da81084f0 2024-06-10T00:38:30.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.334+0000 7f7dae2fd700 1 -- 172.21.15.62:0/2825405151 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da81a5710 con 0x7f7da81084f0 2024-06-10T00:38:30.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.337+0000 7f7da57fa700 1 -- 172.21.15.62:0/2825405151 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7da001a020 con 0x7f7da81084f0 2024-06-10T00:38:30.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.338+0000 7f7da57fa700 1 --2- 172.21.15.62:0/2825405151 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7d900a2d90 0x7f7d900a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.338+0000 7f7da57fa700 1 -- 172.21.15.62:0/2825405151 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f7da00ceed0 con 0x7f7da81084f0 2024-06-10T00:38:30.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.338+0000 7f7da7fff700 1 --2- 172.21.15.62:0/2825405151 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7d900a2d90 0x7f7d900a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.338+0000 7f7db8b80b80 1 Processor -- start 2024-06-10T00:38:30.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7da7fff700 1 --2- 172.21.15.62:0/2825405151 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7d900a2d90 0x7f7d900a5250 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7da80ffa50 tx=0x7f7d98005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:30.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7db8b80b80 1 -- start start 2024-06-10T00:38:30.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7db8b80b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c8fc0 0x55f8801c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7db8b80b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c98e0 0x55f8801c3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7db8b80b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 0x55f8801c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7db8b80b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f880110880 con 0x55f8801c8fc0 2024-06-10T00:38:30.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7db8b80b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f8800b8790 con 0x55f8801c98e0 2024-06-10T00:38:30.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7db8b80b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f88010c840 con 0x55f8801c3750 2024-06-10T00:38:30.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7da7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c8fc0 0x55f8801c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7da7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c8fc0 0x55f8801c93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41874/0 (socket says 172.21.15.62:41874) 2024-06-10T00:38:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7da7fff700 1 -- 172.21.15.62:0/1728880288 learned_addr learned my addr 172.21.15.62:0/1728880288 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7dac89a700 1 --2- 172.21.15.62:0/1728880288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 0x55f8801c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7da77fe700 1 --2- 172.21.15.62:0/1728880288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c98e0 0x55f8801c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7da7fff700 1 -- 172.21.15.62:0/1728880288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 msgr2=0x55f8801c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7da7fff700 1 --2- 172.21.15.62:0/1728880288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 0x55f8801c3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7da7fff700 1 -- 172.21.15.62:0/1728880288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c98e0 msgr2=0x55f8801c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7da7fff700 1 --2- 172.21.15.62:0/1728880288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c98e0 0x55f8801c3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7da7fff700 1 -- 172.21.15.62:0/1728880288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8800aefc0 con 0x55f8801c8fc0 2024-06-10T00:38:30.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7da77fe700 1 --2- 172.21.15.62:0/1728880288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c98e0 0x55f8801c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:38:30.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.339+0000 7f7da7fff700 1 --2- 172.21.15.62:0/1728880288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c8fc0 0x55f8801c93a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009990 tx=0x7f7d98020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca586cedbd8473e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:30.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.340+0000 7f7d8e7fc700 1 -- 172.21.15.62:0/1728880288 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98003780 con 0x55f8801c8fc0 2024-06-10T00:38:30.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.340+0000 7f7d8e7fc700 1 -- 172.21.15.62:0/1728880288 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d9800bdf0 con 0x55f8801c8fc0 2024-06-10T00:38:30.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.340+0000 7f7d8e7fc700 1 -- 172.21.15.62:0/1728880288 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98024d00 con 0x55f8801c8fc0 2024-06-10T00:38:30.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.340+0000 7f7db8b80b80 1 -- 172.21.15.62:0/1728880288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c8fc0 msgr2=0x55f8801c93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:30.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.340+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/1728880288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c8fc0 0x55f8801c93a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009990 tx=0x7f7d98020000 comp rx=0 tx=0).stop 2024-06-10T00:38:30.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 -- 172.21.15.62:0/1728880288 shutdown_connections 2024-06-10T00:38:30.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/1728880288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c8fc0 0x55f8801c93a0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/1728880288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 0x55f8801c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/1728880288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c98e0 0x55f8801c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 -- 172.21.15.62:0/1728880288 >> 172.21.15.62:0/1728880288 conn(0x55f8800e9d30 msgr2=0x55f8801ba090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:30.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 -- 172.21.15.62:0/1728880288 shutdown_connections 2024-06-10T00:38:30.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 -- 172.21.15.62:0/1728880288 wait complete. 2024-06-10T00:38:30.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 -- start start 2024-06-10T00:38:30.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 Processor -- start 2024-06-10T00:38:30.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 0x55f8801cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c8fc0 0x55f8801cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c98e0 0x55f8801cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f8801d0290 con 0x55f8801c8fc0 2024-06-10T00:38:30.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f8800b0c60 con 0x55f8801c98e0 2024-06-10T00:38:30.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f8801d0410 con 0x55f8801c3750 2024-06-10T00:38:30.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.341+0000 7f7db8b80b80 10 client.0 _ll_register_callbacks cb 0x55f8800aa990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:30.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7da77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c8fc0 0x55f8801cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7da77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c8fc0 0x55f8801cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41880/0 (socket says 172.21.15.62:41880) 2024-06-10T00:38:30.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7da77fe700 1 -- 172.21.15.62:0/585952148 learned_addr learned my addr 172.21.15.62:0/585952148 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:30.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7da7fff700 1 --2- 172.21.15.62:0/585952148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 0x55f8801cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7da77fe700 1 -- 172.21.15.62:0/585952148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 msgr2=0x55f8801cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:30.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7da77fe700 1 --2- 172.21.15.62:0/585952148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 0x55f8801cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7da77fe700 1 -- 172.21.15.62:0/585952148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c98e0 msgr2=0x55f8801cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:30.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7da77fe700 1 --2- 172.21.15.62:0/585952148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c98e0 0x55f8801cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7da77fe700 1 -- 172.21.15.62:0/585952148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88005f940 con 0x55f8801c8fc0 2024-06-10T00:38:30.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7da77fe700 1 --2- 172.21.15.62:0/585952148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c8fc0 0x55f8801cf8b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00b9c0 tx=0x7f7d9c00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbee7af47bde09d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:30.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7d877fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:30.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7d877fe700 1 -- 172.21.15.62:0/585952148 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c004570 con 0x55f8801c8fc0 2024-06-10T00:38:30.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7db8b80b80 1 -- 172.21.15.62:0/585952148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d9800bdf0 con 0x55f8801c8fc0 2024-06-10T00:38:30.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7d877fe700 1 -- 172.21.15.62:0/585952148 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f7d9c012450 con 0x55f8801c8fc0 2024-06-10T00:38:30.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7d877fe700 1 -- 172.21.15.62:0/585952148 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7d9c012650 con 0x55f8801c8fc0 2024-06-10T00:38:30.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7db8b80b80 10 client.16326 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:30.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.342+0000 7f7db8b80b80 1 -- 172.21.15.62:0/585952148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f8801defb0 con 0x55f8801c8fc0 2024-06-10T00:38:30.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.343+0000 7f7d877fe700 1 -- 172.21.15.62:0/585952148 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d9800bdf0 con 0x55f8801c8fc0 2024-06-10T00:38:30.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.343+0000 7f7d877fe700 1 -- 172.21.15.62:0/585952148 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f7d9c01ba90 con 0x55f8801c8fc0 2024-06-10T00:38:30.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.343+0000 7f7db8b80b80 10 client.16326 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:30.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.343+0000 7f7db8b80b80 10 client.16326 resolve_mds: validated gid 24464 aka daemon mds.b 2024-06-10T00:38:30.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.343+0000 7f7db8b80b80 20 client.16326 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:30.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.343+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/585952148 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f8801e1590 0x55f8801e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.343+0000 7f7db8b80b80 4 client.16326 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:38:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.343+0000 7f7db8b80b80 1 -- 172.21.15.62:0/585952148 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f88011dd80 con 0x55f8801e1590 2024-06-10T00:38:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.343+0000 7f7da7fff700 1 --2- 172.21.15.62:0/585952148 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f8801e1590 0x55f8801e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.344+0000 7f7d877fe700 10 client.16326 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:30.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.344+0000 7f7da7fff700 1 --2- 172.21.15.62:0/585952148 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f8801e1590 0x55f8801e39c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0b70090ba4512d server_cookie=eb0e1d062af238fa in_seq=0 out_seq=0 2024-06-10T00:38:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.344+0000 7f7d877fe700 1 -- 172.21.15.62:0/585952148 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f88011dd80 con 0x55f8801e1590 2024-06-10T00:38:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.344+0000 7f7d877fe700 10 client.16326 handle_command_reply: tid=0 2024-06-10T00:38:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.344+0000 7f7db8b80b80 1 client.16326 shutdown 2024-06-10T00:38:30.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.345+0000 7f7db8b80b80 1 -- 172.21.15.62:0/585952148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c8fc0 msgr2=0x55f8801cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:30.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.345+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/585952148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c8fc0 0x55f8801cf8b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00b9c0 tx=0x7f7d9c00bd80 comp rx=0 tx=0).stop 2024-06-10T00:38:30.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.345+0000 7f7db8b80b80 1 -- 172.21.15.62:0/585952148 shutdown_connections 2024-06-10T00:38:30.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.345+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/585952148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c98e0 0x55f8801cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.345+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/585952148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 0x55f8801cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.345+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/585952148 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f8801e1590 0x55f8801e39c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.345+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/585952148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c8fc0 0x55f8801cf8b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.345+0000 7f7d877fe700 0 client.16326 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:30.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.345+0000 7f7db8b80b80 1 -- 172.21.15.62:0/585952148 >> 172.21.15.62:0/585952148 conn(0x55f8800e9d30 msgr2=0x55f8801b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:30.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.345+0000 7f7db8b80b80 1 -- 172.21.15.62:0/585952148 shutdown_connections 2024-06-10T00:38:30.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.345+0000 7f7db8b80b80 1 -- 172.21.15.62:0/585952148 wait complete. 2024-06-10T00:38:30.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.345+0000 7f7db8b80b80 20 client.16326 trim_cache size 0 max 16384 2024-06-10T00:38:30.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.365+0000 7f7db8b80b80 1 Processor -- start 2024-06-10T00:38:30.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.365+0000 7f7db8b80b80 1 -- start start 2024-06-10T00:38:30.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.365+0000 7f7db8b80b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c3750 0x55f8801e9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.365+0000 7f7db8b80b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c8fc0 0x55f8801e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.365+0000 7f7db8b80b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c98e0 0x55f8801ddfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.365+0000 7f7db8b80b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81ac3e0 con 0x55f8801c98e0 2024-06-10T00:38:30.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.365+0000 7f7db8b80b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81ae0a0 con 0x55f8801c3750 2024-06-10T00:38:30.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.365+0000 7f7db8b80b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81acab0 con 0x55f8801c8fc0 2024-06-10T00:38:30.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.366+0000 7f7dac89a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c98e0 0x55f8801ddfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.366+0000 7f7dac89a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c98e0 0x55f8801ddfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41884/0 (socket says 172.21.15.62:41884) 2024-06-10T00:38:30.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.366+0000 7f7dac89a700 1 -- 172.21.15.62:0/52309638 learned_addr learned my addr 172.21.15.62:0/52309638 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:30.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.366+0000 7f7da7fff700 1 --2- 172.21.15.62:0/52309638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c3750 0x55f8801e9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.366+0000 7f7da77fe700 1 --2- 172.21.15.62:0/52309638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c8fc0 0x55f8801e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.366+0000 7f7dac89a700 1 -- 172.21.15.62:0/52309638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c8fc0 msgr2=0x55f8801e97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:30.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.366+0000 7f7dac89a700 1 --2- 172.21.15.62:0/52309638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c8fc0 0x55f8801e97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.366+0000 7f7dac89a700 1 -- 172.21.15.62:0/52309638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c3750 msgr2=0x55f8801e9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:30.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.366+0000 7f7dac89a700 1 --2- 172.21.15.62:0/52309638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c3750 0x55f8801e9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.366+0000 7f7dac89a700 1 -- 172.21.15.62:0/52309638 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8801bb630 con 0x55f8801c98e0 2024-06-10T00:38:30.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.366+0000 7f7dac89a700 1 --2- 172.21.15.62:0/52309638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c98e0 0x55f8801ddfc0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009510 tx=0x7f7da009aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e9315c95abf6e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:30.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.366+0000 7f7d8dffb700 1 -- 172.21.15.62:0/52309638 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da009b740 con 0x55f8801c98e0 2024-06-10T00:38:30.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.366+0000 7f7d8dffb700 1 -- 172.21.15.62:0/52309638 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da009b040 con 0x55f8801c98e0 2024-06-10T00:38:30.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.366+0000 7f7d8dffb700 1 -- 172.21.15.62:0/52309638 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0004f00 con 0x55f8801c98e0 2024-06-10T00:38:30.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.367+0000 7f7db8b80b80 1 -- 172.21.15.62:0/52309638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c98e0 msgr2=0x55f8801ddfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:30.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.367+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/52309638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c98e0 0x55f8801ddfc0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009510 tx=0x7f7da009aeb0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.367+0000 7f7db8b80b80 1 -- 172.21.15.62:0/52309638 shutdown_connections 2024-06-10T00:38:30.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.367+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/52309638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c98e0 0x55f8801ddfc0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.367+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/52309638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c3750 0x55f8801e9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.367+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/52309638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c8fc0 0x55f8801e97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.367+0000 7f7db8b80b80 1 -- 172.21.15.62:0/52309638 >> 172.21.15.62:0/52309638 conn(0x55f8800e9d30 msgr2=0x55f8801c6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:30.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.367+0000 7f7db8b80b80 1 -- 172.21.15.62:0/52309638 shutdown_connections 2024-06-10T00:38:30.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.367+0000 7f7db8b80b80 1 -- 172.21.15.62:0/52309638 wait complete. 2024-06-10T00:38:30.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.367+0000 7f7db8b80b80 1 -- start start 2024-06-10T00:38:30.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7db8b80b80 1 Processor -- start 2024-06-10T00:38:30.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7db8b80b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 0x55f8801de850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7db8b80b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c8fc0 0x55f8801dcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7db8b80b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c98e0 0x55f8801dd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7db8b80b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81ad910 con 0x55f8801c98e0 2024-06-10T00:38:30.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7db8b80b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81ad180 con 0x55f8801c8fc0 2024-06-10T00:38:30.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7db8b80b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81aa380 con 0x55f8801c3750 2024-06-10T00:38:30.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7db8b80b80 10 client.0 _ll_register_callbacks cb 0x55f8800aa990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:30.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7dac89a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c98e0 0x55f8801dd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7dac89a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c98e0 0x55f8801dd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41892/0 (socket says 172.21.15.62:41892) 2024-06-10T00:38:30.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7dac89a700 1 -- 172.21.15.62:0/2928491057 learned_addr learned my addr 172.21.15.62:0/2928491057 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:30.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7da7fff700 1 --2- 172.21.15.62:0/2928491057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 0x55f8801de850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7da77fe700 1 --2- 172.21.15.62:0/2928491057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c8fc0 0x55f8801dcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7dac89a700 1 -- 172.21.15.62:0/2928491057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 msgr2=0x55f8801de850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:30.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7dac89a700 1 --2- 172.21.15.62:0/2928491057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 0x55f8801de850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7dac89a700 1 -- 172.21.15.62:0/2928491057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c8fc0 msgr2=0x55f8801dcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:30.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7dac89a700 1 --2- 172.21.15.62:0/2928491057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c8fc0 0x55f8801dcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7dac89a700 1 -- 172.21.15.62:0/2928491057 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8801dd9b0 con 0x55f8801c98e0 2024-06-10T00:38:30.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7da77fe700 1 --2- 172.21.15.62:0/2928491057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c8fc0 0x55f8801dcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:38:30.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7dac89a700 1 --2- 172.21.15.62:0/2928491057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c98e0 0x55f8801dd190 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7da00cc320 tx=0x7f7da0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32cf8bf9e4a01baf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:30.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.368+0000 7f7d877fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:30.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.369+0000 7f7d877fe700 1 -- 172.21.15.62:0/2928491057 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da009f3f0 con 0x55f8801c98e0 2024-06-10T00:38:30.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.369+0000 7f7d877fe700 1 -- 172.21.15.62:0/2928491057 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f7da009f0d0 con 0x55f8801c98e0 2024-06-10T00:38:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.369+0000 7f7db8b80b80 1 -- 172.21.15.62:0/2928491057 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da009b040 con 0x55f8801c98e0 2024-06-10T00:38:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.369+0000 7f7d877fe700 1 -- 172.21.15.62:0/2928491057 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7da009b2a0 con 0x55f8801c98e0 2024-06-10T00:38:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.369+0000 7f7db8b80b80 10 client.16332 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.369+0000 7f7db8b80b80 1 -- 172.21.15.62:0/2928491057 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f88024d300 con 0x55f8801c98e0 2024-06-10T00:38:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.369+0000 7f7d877fe700 1 -- 172.21.15.62:0/2928491057 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da009b040 con 0x55f8801c98e0 2024-06-10T00:38:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.369+0000 7f7d877fe700 1 -- 172.21.15.62:0/2928491057 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f7da00b7070 con 0x55f8801c98e0 2024-06-10T00:38:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.369+0000 7f7db8b80b80 10 client.16332 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.369+0000 7f7db8b80b80 10 client.16332 resolve_mds: validated gid 24464 aka daemon mds.b 2024-06-10T00:38:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.369+0000 7f7db8b80b80 20 client.16332 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.369+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/2928491057 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f88024da80 0x55f88024fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.369+0000 7f7db8b80b80 4 client.16332 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-06-10T00:38:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.369+0000 7f7db8b80b80 1 -- 172.21.15.62:0/2928491057 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55f880118300 con 0x55f88024da80 2024-06-10T00:38:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.370+0000 7f7da7fff700 1 --2- 172.21.15.62:0/2928491057 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f88024da80 0x55f88024fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.370+0000 7f7da7fff700 1 --2- 172.21.15.62:0/2928491057 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f88024da80 0x55f88024fea0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43096be632f12dd8 server_cookie=7f4eba1c96006e37 in_seq=0 out_seq=0 2024-06-10T00:38:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.370+0000 7f7d877fe700 10 client.16332 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.370+0000 7f7d877fe700 1 -- 172.21.15.62:0/2928491057 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55f880118300 con 0x55f88024da80 2024-06-10T00:38:30.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.370+0000 7f7d877fe700 10 client.16332 handle_command_reply: tid=0 2024-06-10T00:38:30.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.370+0000 7f7db8b80b80 1 client.16332 shutdown 2024-06-10T00:38:30.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7db8b80b80 1 -- 172.21.15.62:0/2928491057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c98e0 msgr2=0x55f8801dd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:30.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/2928491057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c98e0 0x55f8801dd190 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7da00cc320 tx=0x7f7da0004ad0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7db8b80b80 1 -- 172.21.15.62:0/2928491057 shutdown_connections 2024-06-10T00:38:30.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/2928491057 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55f88024da80 0x55f88024fea0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/2928491057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f8801c98e0 0x55f8801dd190 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/2928491057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8801c3750 0x55f8801de850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7db8b80b80 1 --2- 172.21.15.62:0/2928491057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f8801c8fc0 0x55f8801dcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7d877fe700 0 client.16332 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:30.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7db8b80b80 1 -- 172.21.15.62:0/2928491057 >> 172.21.15.62:0/2928491057 conn(0x55f8800e9d30 msgr2=0x55f8801b97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:30.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7db8b80b80 1 -- 172.21.15.62:0/2928491057 shutdown_connections 2024-06-10T00:38:30.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7db8b80b80 1 -- 172.21.15.62:0/2928491057 wait complete. 2024-06-10T00:38:30.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7db8b80b80 20 client.16332 trim_cache size 0 max 16384 2024-06-10T00:38:30.393 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:38:30.393 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 34, 2024-06-10T00:38:30.393 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24464, 2024-06-10T00:38:30.394 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 11, 2024-06-10T00:38:30.394 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": {} 2024-06-10T00:38:30.394 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:38:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7dae2fd700 1 -- 172.21.15.62:0/2825405151 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7d900a2d90 msgr2=0x7f7d900a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7dae2fd700 1 --2- 172.21.15.62:0/2825405151 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7d900a2d90 0x7f7d900a5250 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7da80ffa50 tx=0x7f7d98005960 comp rx=0 tx=0).stop 2024-06-10T00:38:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7dae2fd700 1 -- 172.21.15.62:0/2825405151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7da81084f0 msgr2=0x7f7da8198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.371+0000 7f7dae2fd700 1 --2- 172.21.15.62:0/2825405151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7da81084f0 0x7f7da8198a50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7da0007ee0 tx=0x7f7da000d3b0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.372+0000 7f7dae2fd700 1 -- 172.21.15.62:0/2825405151 shutdown_connections 2024-06-10T00:38:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.372+0000 7f7dae2fd700 1 --2- 172.21.15.62:0/2825405151 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7d900a2d90 0x7f7d900a5250 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.372+0000 7f7dae2fd700 1 --2- 172.21.15.62:0/2825405151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7da81084f0 0x7f7da8198a50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.372+0000 7f7dae2fd700 1 --2- 172.21.15.62:0/2825405151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da806aa00 0x7f7da819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.372+0000 7f7dae2fd700 1 --2- 172.21.15.62:0/2825405151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7da806b3c0 0x7f7da819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.372+0000 7f7dae2fd700 1 -- 172.21.15.62:0/2825405151 >> 172.21.15.62:0/2825405151 conn(0x7f7da8078300 msgr2=0x7f7da8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.372+0000 7f7dae2fd700 1 -- 172.21.15.62:0/2825405151 shutdown_connections 2024-06-10T00:38:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:30.372+0000 7f7dae2fd700 1 -- 172.21.15.62:0/2825405151 wait complete. 2024-06-10T00:38:31.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:31 smithi062 ceph-mon[32390]: pgmap v173: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T00:38:31.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:31 smithi105 ceph-mon[36321]: pgmap v173: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T00:38:31.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:31 smithi138 ceph-mon[36549]: pgmap v173: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T00:38:31.872 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:38:32.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.930+0000 7feac3e4f700 1 -- 172.21.15.62:0/2616046086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feabc102740 msgr2=0x7feabc102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:32.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.930+0000 7feac3e4f700 1 --2- 172.21.15.62:0/2616046086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feabc102740 0x7feabc102b20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7feab800b0d0 tx=0x7feab800b3e0 comp rx=0 tx=0).stop 2024-06-10T00:38:32.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.930+0000 7feac3e4f700 1 -- 172.21.15.62:0/2616046086 shutdown_connections 2024-06-10T00:38:32.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.930+0000 7feac3e4f700 1 --2- 172.21.15.62:0/2616046086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feabc102740 0x7feabc102b20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:32.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.930+0000 7feac3e4f700 1 --2- 172.21.15.62:0/2616046086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feabc107840 0x7feabc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:32.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.930+0000 7feac3e4f700 1 --2- 172.21.15.62:0/2616046086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feabc1030f0 0x7feabc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:32.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.930+0000 7feac3e4f700 1 -- 172.21.15.62:0/2616046086 >> 172.21.15.62:0/2616046086 conn(0x7feabc0795f0 msgr2=0x7feabc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:32.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.930+0000 7feac3e4f700 1 -- 172.21.15.62:0/2616046086 shutdown_connections 2024-06-10T00:38:32.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.930+0000 7feac3e4f700 1 -- 172.21.15.62:0/2616046086 wait complete. 2024-06-10T00:38:32.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.931+0000 7feac3e4f700 1 Processor -- start 2024-06-10T00:38:32.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.931+0000 7feac3e4f700 1 -- start start 2024-06-10T00:38:32.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.931+0000 7feac3e4f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feabc102740 0x7feabc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:32.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.931+0000 7feac3e4f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feabc1030f0 0x7feabc074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:32.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.931+0000 7feac3e4f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feabc107840 0x7feabc075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:32.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.931+0000 7feac3e4f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feabc112760 con 0x7feabc1030f0 2024-06-10T00:38:32.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.931+0000 7feac3e4f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1125e0 con 0x7feabc102740 2024-06-10T00:38:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.931+0000 7feac3e4f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1128e0 con 0x7feabc107840 2024-06-10T00:38:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.931+0000 7feac13ea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feabc1030f0 0x7feabc074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.931+0000 7feac13ea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feabc1030f0 0x7feabc074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41904/0 (socket says 172.21.15.62:41904) 2024-06-10T00:38:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.931+0000 7feac13ea700 1 -- 172.21.15.62:0/2395740458 learned_addr learned my addr 172.21.15.62:0/2395740458 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.932+0000 7feac1beb700 1 --2- 172.21.15.62:0/2395740458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feabc102740 0x7feabc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.932+0000 7feac23ec700 1 --2- 172.21.15.62:0/2395740458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feabc107840 0x7feabc075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.932+0000 7feac13ea700 1 -- 172.21.15.62:0/2395740458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feabc107840 msgr2=0x7feabc075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.932+0000 7feac13ea700 1 --2- 172.21.15.62:0/2395740458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feabc107840 0x7feabc075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:32.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.932+0000 7feac13ea700 1 -- 172.21.15.62:0/2395740458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feabc102740 msgr2=0x7feabc074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:32.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.932+0000 7feac13ea700 1 --2- 172.21.15.62:0/2395740458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feabc102740 0x7feabc074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:32.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.932+0000 7feac13ea700 1 -- 172.21.15.62:0/2395740458 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc1a7110 con 0x7feabc1030f0 2024-06-10T00:38:32.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.932+0000 7feac13ea700 1 --2- 172.21.15.62:0/2395740458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feabc1030f0 0x7feabc074cd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7feaac00c9b0 tx=0x7feaac00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:32.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.932+0000 7feab2ffd700 1 -- 172.21.15.62:0/2395740458 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac007b00 con 0x7feabc1030f0 2024-06-10T00:38:32.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.932+0000 7feab2ffd700 1 -- 172.21.15.62:0/2395740458 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaac007ca0 con 0x7feabc1030f0 2024-06-10T00:38:32.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.932+0000 7feac3e4f700 1 -- 172.21.15.62:0/2395740458 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feabc1a7400 con 0x7feabc1030f0 2024-06-10T00:38:32.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.932+0000 7feab2ffd700 1 -- 172.21.15.62:0/2395740458 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac018730 con 0x7feabc1030f0 2024-06-10T00:38:32.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.932+0000 7feac3e4f700 1 -- 172.21.15.62:0/2395740458 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feabc1a7940 con 0x7feabc1030f0 2024-06-10T00:38:32.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.933+0000 7feac3e4f700 1 -- 172.21.15.62:0/2395740458 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feabc10aa80 con 0x7feabc1030f0 2024-06-10T00:38:32.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.935+0000 7feab2ffd700 1 -- 172.21.15.62:0/2395740458 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7feaac0188d0 con 0x7feabc1030f0 2024-06-10T00:38:32.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.936+0000 7feab2ffd700 1 --2- 172.21.15.62:0/2395740458 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7feaa80a2d90 0x7feaa80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:32.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.936+0000 7feac1beb700 1 --2- 172.21.15.62:0/2395740458 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7feaa80a2d90 0x7feaa80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:32.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.936+0000 7feab2ffd700 1 -- 172.21.15.62:0/2395740458 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7feaac0cfaa0 con 0x7feabc1030f0 2024-06-10T00:38:32.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.937+0000 7feac1beb700 1 --2- 172.21.15.62:0/2395740458 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7feaa80a2d90 0x7feaa80a5250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feab8009fd0 tx=0x7feab800b040 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:32.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:32.940+0000 7feab2ffd700 1 -- 172.21.15.62:0/2395740458 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feaac014030 con 0x7feabc1030f0 2024-06-10T00:38:33.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:33.206+0000 7feac3e4f700 1 -- 172.21.15.62:0/2395740458 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feabc069180 con 0x7feabc1030f0 2024-06-10T00:38:33.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:33.207+0000 7feab2ffd700 1 -- 172.21.15.62:0/2395740458 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11952 (secure 0 0 0) 0x7feaac09c3a0 con 0x7feabc1030f0 2024-06-10T00:38:33.208 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:38:33.208 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:37:35.971947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:38:33.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:33.209+0000 7feac3e4f700 1 -- 172.21.15.62:0/2395740458 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7feaa80a2d90 msgr2=0x7feaa80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:33.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:33.209+0000 7feac3e4f700 1 --2- 172.21.15.62:0/2395740458 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7feaa80a2d90 0x7feaa80a5250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feab8009fd0 tx=0x7feab800b040 comp rx=0 tx=0).stop 2024-06-10T00:38:33.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:33.210+0000 7feac3e4f700 1 -- 172.21.15.62:0/2395740458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feabc1030f0 msgr2=0x7feabc074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:33.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:33.210+0000 7feac3e4f700 1 --2- 172.21.15.62:0/2395740458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feabc1030f0 0x7feabc074cd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7feaac00c9b0 tx=0x7feaac00ccc0 comp rx=0 tx=0).stop 2024-06-10T00:38:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:33.210+0000 7feac3e4f700 1 -- 172.21.15.62:0/2395740458 shutdown_connections 2024-06-10T00:38:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:33.210+0000 7feac3e4f700 1 --2- 172.21.15.62:0/2395740458 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7feaa80a2d90 0x7feaa80a5250 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:33.210+0000 7feac3e4f700 1 --2- 172.21.15.62:0/2395740458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feabc107840 0x7feabc075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:33.210+0000 7feac3e4f700 1 --2- 172.21.15.62:0/2395740458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feabc102740 0x7feabc074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:33.210+0000 7feac3e4f700 1 --2- 172.21.15.62:0/2395740458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feabc1030f0 0x7feabc074cd0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:33.210+0000 7feac3e4f700 1 -- 172.21.15.62:0/2395740458 >> 172.21.15.62:0/2395740458 conn(0x7feabc0795f0 msgr2=0x7feabc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:33.210+0000 7feac3e4f700 1 -- 172.21.15.62:0/2395740458 shutdown_connections 2024-06-10T00:38:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:33.210+0000 7feac3e4f700 1 -- 172.21.15.62:0/2395740458 wait complete. 2024-06-10T00:38:33.212 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 34 2024-06-10T00:38:33.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:33 smithi105 ceph-mon[36321]: pgmap v174: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-06-10T00:38:33.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:33 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2395740458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:38:33.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:33 smithi062 ceph-mon[32390]: pgmap v174: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-06-10T00:38:33.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:33 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2395740458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:38:33.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:33 smithi138 ceph-mon[36549]: pgmap v174: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-06-10T00:38:33.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:33 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2395740458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:38:34.251 INFO:tasks.quiescer:done joining Quiescers 2024-06-10T00:38:34.251 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-06-10T00:38:34.261 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-06-10T00:38:34.261 INFO:teuthology.task.exec:Executing custom commands... 2024-06-10T00:38:34.261 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-06-10T00:38:34.262 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-06-10T00:38:34.305 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-06-10T00:38:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.389+0000 7fcdd8745700 1 -- 172.21.15.62:0/910756279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0101fd0 msgr2=0x7fcdd0105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.389+0000 7fcdd8745700 1 --2- 172.21.15.62:0/910756279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0101fd0 0x7fcdd0105fd0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40099c0 tx=0x7fcdc4009cd0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.389+0000 7fcdd8745700 1 -- 172.21.15.62:0/910756279 shutdown_connections 2024-06-10T00:38:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.389+0000 7fcdd8745700 1 --2- 172.21.15.62:0/910756279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd01066b0 0x7fcdd010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.389+0000 7fcdd8745700 1 --2- 172.21.15.62:0/910756279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdd0101620 0x7fcdd0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.389+0000 7fcdd8745700 1 --2- 172.21.15.62:0/910756279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0101fd0 0x7fcdd0105fd0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.389+0000 7fcdd8745700 1 -- 172.21.15.62:0/910756279 >> 172.21.15.62:0/910756279 conn(0x7fcdd0078360 msgr2=0x7fcdd0078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.389+0000 7fcdd8745700 1 -- 172.21.15.62:0/910756279 shutdown_connections 2024-06-10T00:38:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.389+0000 7fcdd8745700 1 -- 172.21.15.62:0/910756279 wait complete. 2024-06-10T00:38:35.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.389+0000 7fcdd8745700 1 Processor -- start 2024-06-10T00:38:35.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.389+0000 7fcdd8745700 1 -- start start 2024-06-10T00:38:35.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd8745700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0101620 0x7fcdd019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd8745700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdd0101fd0 0x7fcdd019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd8745700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd01066b0 0x7fcdd0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd8745700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0111690 con 0x7fcdd0101620 2024-06-10T00:38:35.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd8745700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0111510 con 0x7fcdd0101fd0 2024-06-10T00:38:35.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd8745700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0111810 con 0x7fcdd01066b0 2024-06-10T00:38:35.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd64e1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0101620 0x7fcdd019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd64e1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0101620 0x7fcdd019e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41932/0 (socket says 172.21.15.62:41932) 2024-06-10T00:38:35.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd64e1700 1 -- 172.21.15.62:0/2268752187 learned_addr learned my addr 172.21.15.62:0/2268752187 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:35.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd6ce2700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd01066b0 0x7fcdd0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd5ce0700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdd0101fd0 0x7fcdd019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd64e1700 1 -- 172.21.15.62:0/2268752187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd01066b0 msgr2=0x7fcdd0198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:35.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd64e1700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd01066b0 0x7fcdd0198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd64e1700 1 -- 172.21.15.62:0/2268752187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdd0101fd0 msgr2=0x7fcdd019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:35.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd64e1700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdd0101fd0 0x7fcdd019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd64e1700 1 -- 172.21.15.62:0/2268752187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd0199200 con 0x7fcdd0101620 2024-06-10T00:38:35.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd5ce0700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdd0101fd0 0x7fcdd019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:38:35.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdd64e1700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0101620 0x7fcdd019e8e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00d980 tx=0x7fcdcc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:35.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.390+0000 7fcdc37fe700 1 -- 172.21.15.62:0/2268752187 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc016070 con 0x7fcdd0101620 2024-06-10T00:38:35.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.391+0000 7fcdc37fe700 1 -- 172.21.15.62:0/2268752187 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdcc004d10 con 0x7fcdd0101620 2024-06-10T00:38:35.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.391+0000 7fcdd8745700 1 -- 172.21.15.62:0/2268752187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd01994f0 con 0x7fcdd0101620 2024-06-10T00:38:35.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.391+0000 7fcdc37fe700 1 -- 172.21.15.62:0/2268752187 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0055f0 con 0x7fcdd0101620 2024-06-10T00:38:35.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.391+0000 7fcdd8745700 1 -- 172.21.15.62:0/2268752187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd01a57b0 con 0x7fcdd0101620 2024-06-10T00:38:35.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.393+0000 7fcdc37fe700 1 -- 172.21.15.62:0/2268752187 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcdcc004750 con 0x7fcdd0101620 2024-06-10T00:38:35.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.394+0000 7fcdc37fe700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fcdbc0a2d90 0x7fcdbc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.394+0000 7fcdd5ce0700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fcdbc0a2d90 0x7fcdbc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.394+0000 7fcdc37fe700 1 -- 172.21.15.62:0/2268752187 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fcdcc0cf630 con 0x7fcdd0101620 2024-06-10T00:38:35.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.394+0000 7fcdd5ce0700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fcdbc0a2d90 0x7fcdbc0a5250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4006010 tx=0x7fcdc4005e80 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:35.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.394+0000 7fcde2fc8b80 1 Processor -- start 2024-06-10T00:38:35.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcde2fc8b80 1 -- start start 2024-06-10T00:38:35.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:35 smithi062 ceph-mon[32390]: pgmap v175: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T00:38:35.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcde2fc8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc2877a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcde2fc8b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2877f60 0x55efc28719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcde2fc8b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2871f20 0x55efc2872340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcde2fc8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55efc27c3810 con 0x55efc2877640 2024-06-10T00:38:35.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcde2fc8b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55efc276b720 con 0x55efc2871f20 2024-06-10T00:38:35.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcde2fc8b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55efc27bf7d0 con 0x55efc2877f60 2024-06-10T00:38:35.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcdd64e1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc2877a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcdd64e1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc2877a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41942/0 (socket says 172.21.15.62:41942) 2024-06-10T00:38:35.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcdd64e1700 1 -- 172.21.15.62:0/3041549623 learned_addr learned my addr 172.21.15.62:0/3041549623 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:35.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcdd5ce0700 1 --2- 172.21.15.62:0/3041549623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2877f60 0x55efc28719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcdd6ce2700 1 --2- 172.21.15.62:0/3041549623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2871f20 0x55efc2872340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcdd64e1700 1 -- 172.21.15.62:0/3041549623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2877f60 msgr2=0x55efc28719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:35.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcdd64e1700 1 --2- 172.21.15.62:0/3041549623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2877f60 0x55efc28719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcdd64e1700 1 -- 172.21.15.62:0/3041549623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2871f20 msgr2=0x55efc2872340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:35.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcdd64e1700 1 --2- 172.21.15.62:0/3041549623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2871f20 0x55efc2872340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.395+0000 7fcdd64e1700 1 -- 172.21.15.62:0/3041549623 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc2761f50 con 0x55efc2877640 2024-06-10T00:38:35.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.396+0000 7fcdd6ce2700 1 --2- 172.21.15.62:0/3041549623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2871f20 0x55efc2872340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:38:35.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.396+0000 7fcdd64e1700 1 --2- 172.21.15.62:0/3041549623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc2877a20 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc097c00 tx=0x7fcdcc09b1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f762618ada5e2312 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:35.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.396+0000 7fcdc0ff9700 1 -- 172.21.15.62:0/3041549623 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc09fb50 con 0x55efc2877640 2024-06-10T00:38:35.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.396+0000 7fcdc0ff9700 1 -- 172.21.15.62:0/3041549623 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdcc0a4410 con 0x55efc2877640 2024-06-10T00:38:35.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.396+0000 7fcdc0ff9700 1 -- 172.21.15.62:0/3041549623 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc09ba00 con 0x55efc2877640 2024-06-10T00:38:35.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.397+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/3041549623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 msgr2=0x55efc2877a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:35.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.397+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/3041549623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc2877a20 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc097c00 tx=0x7fcdcc09b1f0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.397+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/3041549623 shutdown_connections 2024-06-10T00:38:35.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.397+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/3041549623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc2877a20 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.397+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/3041549623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2871f20 0x55efc2872340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.397+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/3041549623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2877f60 0x55efc28719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.397+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/3041549623 >> 172.21.15.62:0/3041549623 conn(0x55efc279ccc0 msgr2=0x55efc286c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:35.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.397+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/3041549623 shutdown_connections 2024-06-10T00:38:35.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.397+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/3041549623 wait complete. 2024-06-10T00:38:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.397+0000 7fcde2fc8b80 1 -- start start 2024-06-10T00:38:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.397+0000 7fcde2fc8b80 1 Processor -- start 2024-06-10T00:38:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcde2fc8b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2871f20 0x55efc287dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcde2fc8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc287b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcde2fc8b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877f60 0x55efc287b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcde2fc8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55efc287e970 con 0x55efc2877640 2024-06-10T00:38:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcde2fc8b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55efc2763bf0 con 0x55efc2877f60 2024-06-10T00:38:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcde2fc8b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55efc287eaf0 con 0x55efc2871f20 2024-06-10T00:38:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcde2fc8b80 10 client.0 _ll_register_callbacks cb 0x55efc275d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcdd5ce0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc287b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcdd64e1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2871f20 0x55efc287dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcdd5ce0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc287b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41948/0 (socket says 172.21.15.62:41948) 2024-06-10T00:38:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcdd5ce0700 1 -- 172.21.15.62:0/1271913513 learned_addr learned my addr 172.21.15.62:0/1271913513 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcdd6ce2700 1 --2- 172.21.15.62:0/1271913513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877f60 0x55efc287b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcdd64e1700 1 -- 172.21.15.62:0/1271913513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877f60 msgr2=0x55efc287b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcdd64e1700 1 --2- 172.21.15.62:0/1271913513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877f60 0x55efc287b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcdd64e1700 1 -- 172.21.15.62:0/1271913513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 msgr2=0x55efc287b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcdd64e1700 1 --2- 172.21.15.62:0/1271913513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc287b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcdd64e1700 1 -- 172.21.15.62:0/1271913513 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc27128d0 con 0x55efc2871f20 2024-06-10T00:38:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.398+0000 7fcdd5ce0700 1 --2- 172.21.15.62:0/1271913513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc287b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:38:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcdd64e1700 1 --2- 172.21.15.62:0/1271913513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2871f20 0x55efc287dca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc097c00 tx=0x7fcdcc097b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=197f0aade1307c1a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcdb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:38:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcdb9ffb700 1 -- 172.21.15.62:0/1271913513 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc064020 con 0x55efc2871f20 2024-06-10T00:38:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1271913513 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdcc0a4410 con 0x55efc2871f20 2024-06-10T00:38:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcdb9ffb700 1 -- 172.21.15.62:0/1271913513 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fcdcc004340 con 0x55efc2871f20 2024-06-10T00:38:35.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcdb9ffb700 1 -- 172.21.15.62:0/1271913513 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcdcc0cf200 con 0x55efc2871f20 2024-06-10T00:38:35.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcde2fc8b80 10 client.25843 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:35.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1271913513 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55efc287e970 con 0x55efc2871f20 2024-06-10T00:38:35.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcdb9ffb700 1 -- 172.21.15.62:0/1271913513 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdcc0a4410 con 0x55efc2871f20 2024-06-10T00:38:35.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcdb9ffb700 1 -- 172.21.15.62:0/1271913513 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7fcdcc084ce0 con 0x55efc2871f20 2024-06-10T00:38:35.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcde2fc8b80 10 client.25843 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:35.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcde2fc8b80 10 client.25843 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:38:35.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcde2fc8b80 20 client.25843 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/1271913513 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55efc288f970 0x55efc2891d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.399+0000 7fcde2fc8b80 4 client.25843 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:38:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.400+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1271913513 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efc27d0d10 con 0x55efc288f970 2024-06-10T00:38:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.400+0000 7fcdd5ce0700 1 --2- 172.21.15.62:0/1271913513 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55efc288f970 0x55efc2891d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.400+0000 7fcdd5ce0700 1 --2- 172.21.15.62:0/1271913513 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55efc288f970 0x55efc2891d60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4e5111489f486f63 server_cookie=3c1603d2a252d4ed in_seq=0 out_seq=0 2024-06-10T00:38:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.400+0000 7fcdb9ffb700 10 client.25843 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:38:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcdb9ffb700 1 -- 172.21.15.62:0/1271913513 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55efc27d0d10 con 0x55efc288f970 2024-06-10T00:38:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcdb9ffb700 10 client.25843 handle_command_reply: tid=0 2024-06-10T00:38:35.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcde2fc8b80 1 client.25843 shutdown 2024-06-10T00:38:35.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1271913513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2871f20 msgr2=0x55efc287dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:35.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/1271913513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2871f20 0x55efc287dca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc097c00 tx=0x7fcdcc097b40 comp rx=0 tx=0).stop 2024-06-10T00:38:35.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1271913513 shutdown_connections 2024-06-10T00:38:35.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/1271913513 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55efc288f970 0x55efc2891d60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/1271913513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877f60 0x55efc287b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/1271913513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2871f20 0x55efc287dca0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcdb9ffb700 0 client.25843 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:38:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/1271913513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc287b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1271913513 >> 172.21.15.62:0/1271913513 conn(0x55efc279ccc0 msgr2=0x55efc28709d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1271913513 shutdown_connections 2024-06-10T00:38:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1271913513 wait complete. 2024-06-10T00:38:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.401+0000 7fcde2fc8b80 20 client.25843 trim_cache size 0 max 16384 2024-06-10T00:38:35.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcde2fc8b80 1 Processor -- start 2024-06-10T00:38:35.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcde2fc8b80 1 -- start start 2024-06-10T00:38:35.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcde2fc8b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2871f20 0x55efc28719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcde2fc8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc287b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcde2fc8b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877f60 0x55efc286c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcde2fc8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd01ac4e0 con 0x55efc2877640 2024-06-10T00:38:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcde2fc8b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd01ae1a0 con 0x55efc2877f60 2024-06-10T00:38:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcde2fc8b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd01acbb0 con 0x55efc2871f20 2024-06-10T00:38:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcdd5ce0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc287b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcdd5ce0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc287b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41950/0 (socket says 172.21.15.62:41950) 2024-06-10T00:38:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcdd5ce0700 1 -- 172.21.15.62:0/4234220128 learned_addr learned my addr 172.21.15.62:0/4234220128 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcdd64e1700 1 --2- 172.21.15.62:0/4234220128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2871f20 0x55efc28719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcdd6ce2700 1 --2- 172.21.15.62:0/4234220128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877f60 0x55efc286c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcdd5ce0700 1 -- 172.21.15.62:0/4234220128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2871f20 msgr2=0x55efc28719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:35.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcdd5ce0700 1 --2- 172.21.15.62:0/4234220128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2871f20 0x55efc28719e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcdd5ce0700 1 -- 172.21.15.62:0/4234220128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877f60 msgr2=0x55efc286c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:35.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcdd5ce0700 1 --2- 172.21.15.62:0/4234220128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877f60 0x55efc286c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcdd5ce0700 1 -- 172.21.15.62:0/4234220128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc2879580 con 0x55efc2877640 2024-06-10T00:38:35.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.417+0000 7fcdd6ce2700 1 --2- 172.21.15.62:0/4234220128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877f60 0x55efc286c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:38:35.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.418+0000 7fcdd5ce0700 1 --2- 172.21.15.62:0/4234220128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc287b340 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4000c00 tx=0x7fcdc401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d9e12b0736682b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.418+0000 7fcdbbfff700 1 -- 172.21.15.62:0/4234220128 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4005310 con 0x55efc2877640 2024-06-10T00:38:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.418+0000 7fcdbbfff700 1 -- 172.21.15.62:0/4234220128 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc4003d40 con 0x55efc2877640 2024-06-10T00:38:35.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.418+0000 7fcdbbfff700 1 -- 172.21.15.62:0/4234220128 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc400fde0 con 0x55efc2877640 2024-06-10T00:38:35.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.418+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/4234220128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 msgr2=0x55efc287b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:35.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.418+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/4234220128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc287b340 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4000c00 tx=0x7fcdc401e000 comp rx=0 tx=0).stop 2024-06-10T00:38:35.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.418+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/4234220128 shutdown_connections 2024-06-10T00:38:35.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.418+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/4234220128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877f60 0x55efc286c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.418+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/4234220128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2871f20 0x55efc28719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.418+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/4234220128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2877640 0x55efc287b340 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.418+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/4234220128 >> 172.21.15.62:0/4234220128 conn(0x55efc279ccc0 msgr2=0x55efc2865e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:35.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.419+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/4234220128 shutdown_connections 2024-06-10T00:38:35.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.419+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/4234220128 wait complete. 2024-06-10T00:38:35.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.419+0000 7fcde2fc8b80 1 -- start start 2024-06-10T00:38:35.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.419+0000 7fcde2fc8b80 1 Processor -- start 2024-06-10T00:38:35.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.419+0000 7fcde2fc8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2871f20 0x55efc28886a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.419+0000 7fcde2fc8b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877640 0x55efc28969b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.419+0000 7fcde2fc8b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2877f60 0x55efc2896ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.419+0000 7fcde2fc8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd01ada10 con 0x55efc2871f20 2024-06-10T00:38:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.419+0000 7fcde2fc8b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd01ad280 con 0x55efc2877640 2024-06-10T00:38:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.419+0000 7fcde2fc8b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd01aa480 con 0x55efc2877f60 2024-06-10T00:38:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.419+0000 7fcdd64e1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2871f20 0x55efc28886a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.419+0000 7fcdd64e1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2871f20 0x55efc28886a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41960/0 (socket says 172.21.15.62:41960) 2024-06-10T00:38:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.419+0000 7fcdd64e1700 1 -- 172.21.15.62:0/1505888250 learned_addr learned my addr 172.21.15.62:0/1505888250 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:35.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.419+0000 7fcde2fc8b80 10 client.0 _ll_register_callbacks cb 0x55efc275d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:35.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcdd64e1700 1 -- 172.21.15.62:0/1505888250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2877f60 msgr2=0x55efc2896ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:35.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcdd5ce0700 1 --2- 172.21.15.62:0/1505888250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877640 0x55efc28969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcdd6ce2700 1 --2- 172.21.15.62:0/1505888250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2877f60 0x55efc2896ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcdd64e1700 1 --2- 172.21.15.62:0/1505888250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2877f60 0x55efc2896ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcdd64e1700 1 -- 172.21.15.62:0/1505888250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877640 msgr2=0x55efc28969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:35.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcdd64e1700 1 --2- 172.21.15.62:0/1505888250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877640 0x55efc28969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcdd64e1700 1 -- 172.21.15.62:0/1505888250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc2897430 con 0x55efc2871f20 2024-06-10T00:38:35.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcdd64e1700 1 --2- 172.21.15.62:0/1505888250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2871f20 0x55efc28886a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc09b730 tx=0x7fcdcc09bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d436169f056988 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcdb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcdb9ffb700 1 -- 172.21.15.62:0/1505888250 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc064070 con 0x55efc2871f20 2024-06-10T00:38:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcdb9ffb700 1 -- 172.21.15.62:0/1505888250 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fcdcc004340 con 0x55efc2871f20 2024-06-10T00:38:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1505888250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc4003d40 con 0x55efc2871f20 2024-06-10T00:38:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcdb9ffb700 1 -- 172.21.15.62:0/1505888250 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcdcc099b60 con 0x55efc2871f20 2024-06-10T00:38:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcde2fc8b80 10 client.16356 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1505888250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55efc2897430 con 0x55efc2871f20 2024-06-10T00:38:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.420+0000 7fcdb9ffb700 1 -- 172.21.15.62:0/1505888250 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc4003d40 con 0x55efc2871f20 2024-06-10T00:38:35.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.421+0000 7fcdb9ffb700 1 -- 172.21.15.62:0/1505888250 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7fcdcc084ce0 con 0x55efc2871f20 2024-06-10T00:38:35.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.421+0000 7fcde2fc8b80 10 client.16356 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:35.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.421+0000 7fcde2fc8b80 10 client.16356 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:38:35.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.421+0000 7fcde2fc8b80 20 client.16356 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:35.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.421+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/1505888250 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55efc28fa270 0x55efc28fc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:35.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.421+0000 7fcde2fc8b80 4 client.16356 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:38:35.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.421+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1505888250 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55efc27cb290 con 0x55efc28fa270 2024-06-10T00:38:35.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.421+0000 7fcdd5ce0700 1 --2- 172.21.15.62:0/1505888250 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55efc28fa270 0x55efc28fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:35.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.421+0000 7fcdd5ce0700 1 --2- 172.21.15.62:0/1505888250 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55efc28fa270 0x55efc28fc690 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7f565c23801e979f server_cookie=232e6a2f6bc165f8 in_seq=0 out_seq=0 2024-06-10T00:38:35.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.421+0000 7fcdb9ffb700 10 client.16356 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:38:35.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.421+0000 7fcdb9ffb700 1 -- 172.21.15.62:0/1505888250 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55efc27cb290 con 0x55efc28fa270 2024-06-10T00:38:35.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.421+0000 7fcdb9ffb700 10 client.16356 handle_command_reply: tid=0 2024-06-10T00:38:35.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.421+0000 7fcde2fc8b80 1 client.16356 shutdown 2024-06-10T00:38:35.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1505888250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2871f20 msgr2=0x55efc28886a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:35.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/1505888250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2871f20 0x55efc28886a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc09b730 tx=0x7fcdcc09bd70 comp rx=0 tx=0).stop 2024-06-10T00:38:35.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1505888250 shutdown_connections 2024-06-10T00:38:35.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/1505888250 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55efc28fa270 0x55efc28fc690 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/1505888250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55efc2871f20 0x55efc28886a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/1505888250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efc2877f60 0x55efc2896ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcdb9ffb700 0 client.16356 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:38:35.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcde2fc8b80 1 --2- 172.21.15.62:0/1505888250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55efc2877640 0x55efc28969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1505888250 >> 172.21.15.62:0/1505888250 conn(0x55efc279ccc0 msgr2=0x55efc2866510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:35.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1505888250 shutdown_connections 2024-06-10T00:38:35.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcde2fc8b80 1 -- 172.21.15.62:0/1505888250 wait complete. 2024-06-10T00:38:35.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcde2fc8b80 20 client.16356 trim_cache size 0 max 16384 2024-06-10T00:38:35.432 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:38:35.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcdd8745700 1 -- 172.21.15.62:0/2268752187 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fcdbc0a2d90 msgr2=0x7fcdbc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:35.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcdd8745700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fcdbc0a2d90 0x7fcdbc0a5250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4006010 tx=0x7fcdc4005e80 comp rx=0 tx=0).stop 2024-06-10T00:38:35.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcdd8745700 1 -- 172.21.15.62:0/2268752187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0101620 msgr2=0x7fcdd019e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:35.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.422+0000 7fcdd8745700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0101620 0x7fcdd019e8e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00d980 tx=0x7fcdcc00dc90 comp rx=0 tx=0).stop 2024-06-10T00:38:35.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.423+0000 7fcdd8745700 1 -- 172.21.15.62:0/2268752187 shutdown_connections 2024-06-10T00:38:35.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.423+0000 7fcdd8745700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fcdbc0a2d90 0x7fcdbc0a5250 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.423+0000 7fcdd8745700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd0101620 0x7fcdd019e8e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.423+0000 7fcdd8745700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd01066b0 0x7fcdd0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.423+0000 7fcdd8745700 1 --2- 172.21.15.62:0/2268752187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdd0101fd0 0x7fcdd019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:35.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.423+0000 7fcdd8745700 1 -- 172.21.15.62:0/2268752187 >> 172.21.15.62:0/2268752187 conn(0x7fcdd0078360 msgr2=0x7fcdd006aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:35.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.423+0000 7fcdd8745700 1 -- 172.21.15.62:0/2268752187 shutdown_connections 2024-06-10T00:38:35.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:35.423+0000 7fcdd8745700 1 -- 172.21.15.62:0/2268752187 wait complete. 2024-06-10T00:38:35.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:35 smithi105 ceph-mon[36321]: pgmap v175: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T00:38:35.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:35 smithi138 ceph-mon[36549]: pgmap v175: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T00:38:36.194 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T00:38:36.451 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:38:37.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:37 smithi105 ceph-mon[36321]: pgmap v176: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-10T00:38:37.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:37 smithi062 ceph-mon[32390]: pgmap v176: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-10T00:38:37.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:37 smithi138 ceph-mon[36549]: pgmap v176: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-10T00:38:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.274+0000 7f29a8608700 1 -- 172.21.15.62:0/1297088968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29a0101390 msgr2=0x7f29a0101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:38.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.274+0000 7f29a8608700 1 --2- 172.21.15.62:0/1297088968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29a0101390 0x7f29a0101810 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2994009990 tx=0x7f2994009ca0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.275+0000 7f29a8608700 1 -- 172.21.15.62:0/1297088968 shutdown_connections 2024-06-10T00:38:38.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.275+0000 7f29a8608700 1 --2- 172.21.15.62:0/1297088968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a0101d50 0x7f29a010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.275+0000 7f29a8608700 1 --2- 172.21.15.62:0/1297088968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29a0107400 0x7f29a01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.275+0000 7f29a8608700 1 --2- 172.21.15.62:0/1297088968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29a0101390 0x7f29a0101810 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.275+0000 7f29a8608700 1 -- 172.21.15.62:0/1297088968 >> 172.21.15.62:0/1297088968 conn(0x7f29a0078340 msgr2=0x7f29a0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:38.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.275+0000 7f29a8608700 1 -- 172.21.15.62:0/1297088968 shutdown_connections 2024-06-10T00:38:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.275+0000 7f29a8608700 1 -- 172.21.15.62:0/1297088968 wait complete. 2024-06-10T00:38:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.275+0000 7f29a8608700 1 Processor -- start 2024-06-10T00:38:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.275+0000 7f29a8608700 1 -- start start 2024-06-10T00:38:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a8608700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29a0101390 0x7f29a0199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a8608700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29a0101d50 0x7f29a019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a8608700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a0107400 0x7f29a01a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a8608700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0111690 con 0x7f29a0101390 2024-06-10T00:38:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a8608700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0111510 con 0x7f29a0107400 2024-06-10T00:38:38.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a8608700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0111810 con 0x7f29a0101d50 2024-06-10T00:38:38.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a63a4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29a0101390 0x7f29a0199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a63a4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29a0101390 0x7f29a0199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50136/0 (socket says 172.21.15.62:50136) 2024-06-10T00:38:38.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a63a4700 1 -- 172.21.15.62:0/3776122279 learned_addr learned my addr 172.21.15.62:0/3776122279 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:38.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29a0101d50 0x7f29a019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a6ba5700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a0107400 0x7f29a01a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a63a4700 1 -- 172.21.15.62:0/3776122279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29a0101d50 msgr2=0x7f29a019a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:38.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a63a4700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29a0101d50 0x7f29a019a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a63a4700 1 -- 172.21.15.62:0/3776122279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a0107400 msgr2=0x7f29a01a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a63a4700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a0107400 0x7f29a01a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a63a4700 1 -- 172.21.15.62:0/3776122279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a01a5150 con 0x7f29a0101390 2024-06-10T00:38:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a6ba5700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a0107400 0x7f29a01a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:38:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a63a4700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29a0101390 0x7f29a0199b90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f299c00e9c0 tx=0x7f299c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29937fe700 1 -- 172.21.15.62:0/3776122279 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c00cbd0 con 0x7f29a0101390 2024-06-10T00:38:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29937fe700 1 -- 172.21.15.62:0/3776122279 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f299c00cd70 con 0x7f29a0101390 2024-06-10T00:38:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.276+0000 7f29a8608700 1 -- 172.21.15.62:0/3776122279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a01a5320 con 0x7f29a0101390 2024-06-10T00:38:38.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.277+0000 7f29937fe700 1 -- 172.21.15.62:0/3776122279 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c010430 con 0x7f29a0101390 2024-06-10T00:38:38.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.277+0000 7f29a8608700 1 -- 172.21.15.62:0/3776122279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a01a5830 con 0x7f29a0101390 2024-06-10T00:38:38.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.279+0000 7f29937fe700 1 -- 172.21.15.62:0/3776122279 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f299c004710 con 0x7f29a0101390 2024-06-10T00:38:38.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.280+0000 7f29937fe700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f298c0a2cf0 0x7f298c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.280+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f298c0a2cf0 0x7f298c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.280+0000 7f29937fe700 1 -- 172.21.15.62:0/3776122279 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f299c0cf9a0 con 0x7f29a0101390 2024-06-10T00:38:38.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.280+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f298c0a2cf0 0x7f298c0a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f299400b2e0 tx=0x7f2994005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:38.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.281+0000 7f29b2e8bb80 1 Processor -- start 2024-06-10T00:38:38.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.281+0000 7f29b2e8bb80 1 -- start start 2024-06-10T00:38:38.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.281+0000 7f29b2e8bb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 0x563bc139a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.281+0000 7f29b2e8bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc139a880 0x563bc1394260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.281+0000 7f29b2e8bb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc13947a0 0x563bc1394bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.281+0000 7f29b2e8bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563bc1286f60 con 0x563bc139a880 2024-06-10T00:38:38.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.281+0000 7f29b2e8bb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563bc12e5850 con 0x563bc13947a0 2024-06-10T00:38:38.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.281+0000 7f29b2e8bb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563bc124d460 con 0x563bc1399f60 2024-06-10T00:38:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.281+0000 7f29a5ba3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc139a880 0x563bc1394260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.281+0000 7f29a5ba3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc139a880 0x563bc1394260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50140/0 (socket says 172.21.15.62:50140) 2024-06-10T00:38:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.281+0000 7f29a5ba3700 1 -- 172.21.15.62:0/3404921769 learned_addr learned my addr 172.21.15.62:0/3404921769 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.281+0000 7f29a6ba5700 1 --2- 172.21.15.62:0/3404921769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc13947a0 0x563bc1394bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.282+0000 7f29a63a4700 1 --2- 172.21.15.62:0/3404921769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 0x563bc139a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.282+0000 7f29a5ba3700 1 -- 172.21.15.62:0/3404921769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 msgr2=0x563bc139a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.282+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/3404921769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 0x563bc139a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.282+0000 7f29a5ba3700 1 -- 172.21.15.62:0/3404921769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc13947a0 msgr2=0x563bc1394bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:38.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.282+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/3404921769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc13947a0 0x563bc1394bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.282+0000 7f29a5ba3700 1 -- 172.21.15.62:0/3404921769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bc12348d0 con 0x563bc139a880 2024-06-10T00:38:38.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.282+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/3404921769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc139a880 0x563bc1394260 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2994015000 tx=0x7f299401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56a875cb4132349 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:38.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.282+0000 7f2990ff9700 1 -- 172.21.15.62:0/3404921769 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994003680 con 0x563bc139a880 2024-06-10T00:38:38.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.282+0000 7f2990ff9700 1 -- 172.21.15.62:0/3404921769 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2994003d00 con 0x563bc139a880 2024-06-10T00:38:38.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.282+0000 7f2990ff9700 1 -- 172.21.15.62:0/3404921769 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299400fd60 con 0x563bc139a880 2024-06-10T00:38:38.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.283+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3404921769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc139a880 msgr2=0x563bc1394260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:38.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.283+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/3404921769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc139a880 0x563bc1394260 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2994015000 tx=0x7f299401f000 comp rx=0 tx=0).stop 2024-06-10T00:38:38.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.283+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3404921769 shutdown_connections 2024-06-10T00:38:38.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.283+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/3404921769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc13947a0 0x563bc1394bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.283+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/3404921769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 0x563bc139a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.283+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/3404921769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc139a880 0x563bc1394260 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.283+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3404921769 >> 172.21.15.62:0/3404921769 conn(0x563bc12c1080 msgr2=0x563bc138f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:38.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.283+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3404921769 shutdown_connections 2024-06-10T00:38:38.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.283+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3404921769 wait complete. 2024-06-10T00:38:38.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29b2e8bb80 1 -- start start 2024-06-10T00:38:38.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29b2e8bb80 1 Processor -- start 2024-06-10T00:38:38.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29b2e8bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc13947a0 0x563bc13a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29b2e8bb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 0x563bc139d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29b2e8bb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 0x563bc139de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29b2e8bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563bc1285fc0 con 0x563bc13947a0 2024-06-10T00:38:38.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29b2e8bb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563bc12e1810 con 0x563bc139a880 2024-06-10T00:38:38.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29b2e8bb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563bc13a11a0 con 0x563bc1399f60 2024-06-10T00:38:38.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29b2e8bb80 10 client.0 _ll_register_callbacks cb 0x563bc127b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:38.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29a63a4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc13947a0 0x563bc13a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29a63a4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc13947a0 0x563bc13a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50150/0 (socket says 172.21.15.62:50150) 2024-06-10T00:38:38.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29a63a4700 1 -- 172.21.15.62:0/1724250825 learned_addr learned my addr 172.21.15.62:0/1724250825 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:38.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/1724250825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 0x563bc139d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29a6ba5700 1 --2- 172.21.15.62:0/1724250825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 0x563bc139de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29a63a4700 1 -- 172.21.15.62:0/1724250825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 msgr2=0x563bc139d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:38.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29a63a4700 1 --2- 172.21.15.62:0/1724250825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 0x563bc139d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29a63a4700 1 -- 172.21.15.62:0/1724250825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 msgr2=0x563bc139de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:38.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29a63a4700 1 --2- 172.21.15.62:0/1724250825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 0x563bc139de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.284+0000 7f29a63a4700 1 -- 172.21.15.62:0/1724250825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bc12ab750 con 0x563bc13947a0 2024-06-10T00:38:38.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f29a63a4700 1 --2- 172.21.15.62:0/1724250825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc13947a0 0x563bc13a05f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f299c009fd0 tx=0x7f299c09c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7ee2f848d6df4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f2989ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f2989ffb700 1 -- 172.21.15.62:0/1724250825 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c0b3070 con 0x563bc13947a0 2024-06-10T00:38:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f2989ffb700 1 -- 172.21.15.62:0/1724250825 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f299c09bd70 con 0x563bc13947a0 2024-06-10T00:38:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/1724250825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2994003d00 con 0x563bc13947a0 2024-06-10T00:38:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f2989ffb700 1 -- 172.21.15.62:0/1724250825 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f299c01f540 con 0x563bc13947a0 2024-06-10T00:38:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f29b2e8bb80 10 client.16368 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/1724250825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563bc13ab9a0 con 0x563bc13947a0 2024-06-10T00:38:38.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f2989ffb700 1 -- 172.21.15.62:0/1724250825 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2994003d00 con 0x563bc13947a0 2024-06-10T00:38:38.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f2989ffb700 1 -- 172.21.15.62:0/1724250825 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f299c09b060 con 0x563bc13947a0 2024-06-10T00:38:38.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f29b2e8bb80 10 client.16368 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:38.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f29b2e8bb80 10 client.16368 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:38:38.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f29b2e8bb80 20 client.16368 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:38.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/1724250825 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x563bc13adfe0 0x563bc13b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f29b2e8bb80 4 client.16368 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:38:38.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.285+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/1724250825 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bc12f2d50 con 0x563bc13adfe0 2024-06-10T00:38:38.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.286+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/1724250825 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x563bc13adfe0 0x563bc13b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.286+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/1724250825 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x563bc13adfe0 0x563bc13b03f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9600396d7976078f server_cookie=a7dd0fb6a4f679d in_seq=0 out_seq=0 2024-06-10T00:38:38.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.286+0000 7f2989ffb700 10 client.16368 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:38:38.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.287+0000 7f2989ffb700 1 -- 172.21.15.62:0/1724250825 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563bc12f2d50 con 0x563bc13adfe0 2024-06-10T00:38:38.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.287+0000 7f2989ffb700 10 client.16368 handle_command_reply: tid=0 2024-06-10T00:38:38.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.287+0000 7f29b2e8bb80 1 client.16368 shutdown 2024-06-10T00:38:38.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.287+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/1724250825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc13947a0 msgr2=0x563bc13a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:38.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.287+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/1724250825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc13947a0 0x563bc13a05f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f299c009fd0 tx=0x7f299c09c350 comp rx=0 tx=0).stop 2024-06-10T00:38:38.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.287+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/1724250825 shutdown_connections 2024-06-10T00:38:38.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.287+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/1724250825 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x563bc13adfe0 0x563bc13b03f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.287+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/1724250825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc13947a0 0x563bc13a05f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.287+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/1724250825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 0x563bc139de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.287+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/1724250825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 0x563bc139d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.288+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/1724250825 >> 172.21.15.62:0/1724250825 conn(0x563bc12c1080 msgr2=0x563bc138e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:38.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.288+0000 7f2989ffb700 0 client.16368 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:38:38.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.288+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/1724250825 shutdown_connections 2024-06-10T00:38:38.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.288+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/1724250825 wait complete. 2024-06-10T00:38:38.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.288+0000 7f29b2e8bb80 20 client.16368 trim_cache size 0 max 16384 2024-06-10T00:38:38.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.303+0000 7f29b2e8bb80 1 Processor -- start 2024-06-10T00:38:38.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.303+0000 7f29b2e8bb80 1 -- start start 2024-06-10T00:38:38.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.303+0000 7f29b2e8bb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc13947a0 0x563bc1394260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.303+0000 7f29b2e8bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc1399f60 0x563bc138ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.303+0000 7f29b2e8bb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 0x563bc138f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.303+0000 7f29b2e8bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01ae240 con 0x563bc1399f60 2024-06-10T00:38:38.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.303+0000 7f29b2e8bb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01aff00 con 0x563bc139a880 2024-06-10T00:38:38.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.303+0000 7f29b2e8bb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01ae910 con 0x563bc13947a0 2024-06-10T00:38:38.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.303+0000 7f29a5ba3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc1399f60 0x563bc138ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.303+0000 7f29a5ba3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc1399f60 0x563bc138ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50158/0 (socket says 172.21.15.62:50158) 2024-06-10T00:38:38.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.303+0000 7f29a5ba3700 1 -- 172.21.15.62:0/3464156634 learned_addr learned my addr 172.21.15.62:0/3464156634 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:38.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.304+0000 7f29a63a4700 1 --2- 172.21.15.62:0/3464156634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc13947a0 0x563bc1394260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.304+0000 7f29a6ba5700 1 --2- 172.21.15.62:0/3464156634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 0x563bc138f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.304+0000 7f29a5ba3700 1 -- 172.21.15.62:0/3464156634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc13947a0 msgr2=0x563bc1394260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:38.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.304+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/3464156634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc13947a0 0x563bc1394260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.304+0000 7f29a5ba3700 1 -- 172.21.15.62:0/3464156634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 msgr2=0x563bc138f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:38.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.304+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/3464156634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 0x563bc138f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.304+0000 7f29a5ba3700 1 -- 172.21.15.62:0/3464156634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bc138b810 con 0x563bc1399f60 2024-06-10T00:38:38.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.304+0000 7f29a6ba5700 1 --2- 172.21.15.62:0/3464156634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 0x563bc138f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:38:38.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.304+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/3464156634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc1399f60 0x563bc138ee80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2994006010 tx=0x7f299400f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6952f371e4c99f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:38.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.304+0000 7f298bfff700 1 -- 172.21.15.62:0/3464156634 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299400f8d0 con 0x563bc1399f60 2024-06-10T00:38:38.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.304+0000 7f298bfff700 1 -- 172.21.15.62:0/3464156634 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2994003680 con 0x563bc1399f60 2024-06-10T00:38:38.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.304+0000 7f298bfff700 1 -- 172.21.15.62:0/3464156634 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994022650 con 0x563bc1399f60 2024-06-10T00:38:38.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3464156634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc1399f60 msgr2=0x563bc138ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:38.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/3464156634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc1399f60 0x563bc138ee80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2994006010 tx=0x7f299400f860 comp rx=0 tx=0).stop 2024-06-10T00:38:38.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3464156634 shutdown_connections 2024-06-10T00:38:38.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/3464156634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 0x563bc138f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/3464156634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc13947a0 0x563bc1394260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/3464156634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc1399f60 0x563bc138ee80 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3464156634 >> 172.21.15.62:0/3464156634 conn(0x563bc12c1080 msgr2=0x563bc139f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:38.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3464156634 shutdown_connections 2024-06-10T00:38:38.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3464156634 wait complete. 2024-06-10T00:38:38.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 -- start start 2024-06-10T00:38:38.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 Processor -- start 2024-06-10T00:38:38.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc13947a0 0x563bc13a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 0x563bc13a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 0x563bc1416c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01af770 con 0x563bc13947a0 2024-06-10T00:38:38.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01aefe0 con 0x563bc139a880 2024-06-10T00:38:38.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.305+0000 7f29b2e8bb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29a01ac1e0 con 0x563bc1399f60 2024-06-10T00:38:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29b2e8bb80 10 client.0 _ll_register_callbacks cb 0x563bc127b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29a63a4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc13947a0 0x563bc13a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29a63a4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc13947a0 0x563bc13a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50168/0 (socket says 172.21.15.62:50168) 2024-06-10T00:38:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29a63a4700 1 -- 172.21.15.62:0/3173866258 learned_addr learned my addr 172.21.15.62:0/3173866258 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/3173866258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 0x563bc13a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29a6ba5700 1 --2- 172.21.15.62:0/3173866258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 0x563bc1416c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29a63a4700 1 -- 172.21.15.62:0/3173866258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 msgr2=0x563bc13a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:38.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29a63a4700 1 --2- 172.21.15.62:0/3173866258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 0x563bc13a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29a63a4700 1 -- 172.21.15.62:0/3173866258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 msgr2=0x563bc1416c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:38.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29a63a4700 1 --2- 172.21.15.62:0/3173866258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 0x563bc1416c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29a63a4700 1 -- 172.21.15.62:0/3173866258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bc14171f0 con 0x563bc13947a0 2024-06-10T00:38:38.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29a63a4700 1 --2- 172.21.15.62:0/3173866258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc13947a0 0x563bc13a1a30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f299c0b1040 tx=0x7f299c09b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508567ff8ac9278c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:38.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f2989ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:38.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f2989ffb700 1 -- 172.21.15.62:0/3173866258 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c0b3070 con 0x563bc13947a0 2024-06-10T00:38:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f2989ffb700 1 -- 172.21.15.62:0/3173866258 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f299c09b170 con 0x563bc13947a0 2024-06-10T00:38:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3173866258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2994003680 con 0x563bc13947a0 2024-06-10T00:38:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f2989ffb700 1 -- 172.21.15.62:0/3173866258 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f299c0a42f0 con 0x563bc13947a0 2024-06-10T00:38:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29b2e8bb80 10 client.16374 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.306+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3173866258 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563bc1418440 con 0x563bc13947a0 2024-06-10T00:38:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.307+0000 7f2989ffb700 1 -- 172.21.15.62:0/3173866258 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2994003680 con 0x563bc13947a0 2024-06-10T00:38:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.307+0000 7f2989ffb700 1 -- 172.21.15.62:0/3173866258 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f299c097860 con 0x563bc13947a0 2024-06-10T00:38:38.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.307+0000 7f29b2e8bb80 10 client.16374 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:38.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.307+0000 7f29b2e8bb80 10 client.16374 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:38:38.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.307+0000 7f29b2e8bb80 20 client.16374 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:38.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.307+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/3173866258 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x563bc1418b60 0x563bc141af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:38.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.307+0000 7f29b2e8bb80 4 client.16374 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:38:38.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.307+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3173866258 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563bc12ed2d0 con 0x563bc1418b60 2024-06-10T00:38:38.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.307+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/3173866258 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x563bc1418b60 0x563bc141af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:38.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.307+0000 7f29a5ba3700 1 --2- 172.21.15.62:0/3173866258 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x563bc1418b60 0x563bc141af80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e231f46472838b0f server_cookie=5920e06216546459 in_seq=0 out_seq=0 2024-06-10T00:38:38.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.307+0000 7f2989ffb700 10 client.16374 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:38:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f2989ffb700 1 -- 172.21.15.62:0/3173866258 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563bc12ed2d0 con 0x563bc1418b60 2024-06-10T00:38:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f2989ffb700 10 client.16374 handle_command_reply: tid=0 2024-06-10T00:38:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f29b2e8bb80 1 client.16374 shutdown 2024-06-10T00:38:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3173866258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc13947a0 msgr2=0x563bc13a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/3173866258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc13947a0 0x563bc13a1a30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f299c0b1040 tx=0x7f299c09b540 comp rx=0 tx=0).stop 2024-06-10T00:38:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3173866258 shutdown_connections 2024-06-10T00:38:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/3173866258 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x563bc1418b60 0x563bc141af80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/3173866258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563bc13947a0 0x563bc13a1a30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/3173866258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bc139a880 0x563bc1416c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f29b2e8bb80 1 --2- 172.21.15.62:0/3173866258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563bc1399f60 0x563bc13a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3173866258 >> 172.21.15.62:0/3173866258 conn(0x563bc12c1080 msgr2=0x563bc1398a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f2989ffb700 0 client.16374 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:38:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3173866258 shutdown_connections 2024-06-10T00:38:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f29b2e8bb80 1 -- 172.21.15.62:0/3173866258 wait complete. 2024-06-10T00:38:38.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.308+0000 7f29b2e8bb80 20 client.16374 trim_cache size 0 max 16384 2024-06-10T00:38:38.319 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:38:38.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.309+0000 7f29a8608700 1 -- 172.21.15.62:0/3776122279 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f298c0a2cf0 msgr2=0x7f298c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:38.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.309+0000 7f29a8608700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f298c0a2cf0 0x7f298c0a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f299400b2e0 tx=0x7f2994005960 comp rx=0 tx=0).stop 2024-06-10T00:38:38.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.309+0000 7f29a8608700 1 -- 172.21.15.62:0/3776122279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29a0101390 msgr2=0x7f29a0199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:38.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.309+0000 7f29a8608700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29a0101390 0x7f29a0199b90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f299c00e9c0 tx=0x7f299c00ecd0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.309+0000 7f29a8608700 1 -- 172.21.15.62:0/3776122279 shutdown_connections 2024-06-10T00:38:38.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.309+0000 7f29a8608700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f298c0a2cf0 0x7f298c0a51b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.309+0000 7f29a8608700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29a0101390 0x7f29a0199b90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.309+0000 7f29a8608700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29a0107400 0x7f29a01a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.309+0000 7f29a8608700 1 --2- 172.21.15.62:0/3776122279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29a0101d50 0x7f29a019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:38.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.309+0000 7f29a8608700 1 -- 172.21.15.62:0/3776122279 >> 172.21.15.62:0/3776122279 conn(0x7f29a0078340 msgr2=0x7f29a00fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:38.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.309+0000 7f29a8608700 1 -- 172.21.15.62:0/3776122279 shutdown_connections 2024-06-10T00:38:38.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:38.309+0000 7f29a8608700 1 -- 172.21.15.62:0/3776122279 wait complete. 2024-06-10T00:38:39.116 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T00:38:39.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:39 smithi062 ceph-mon[32390]: pgmap v177: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:38:39.369 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:38:39.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:39 smithi105 ceph-mon[36321]: pgmap v177: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:38:39.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:39 smithi138 ceph-mon[36549]: pgmap v177: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:38:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.204+0000 7f72d3c81700 1 -- 172.21.15.62:0/3766419781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72cc101090 msgr2=0x7f72cc101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.204+0000 7f72d3c81700 1 --2- 172.21.15.62:0/3766419781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72cc101090 0x7f72cc101470 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009990 tx=0x7f72c8009ca0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.204+0000 7f72d3c81700 1 -- 172.21.15.62:0/3766419781 shutdown_connections 2024-06-10T00:38:41.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.204+0000 7f72d3c81700 1 --2- 172.21.15.62:0/3766419781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72cc101090 0x7f72cc101470 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.204+0000 7f72d3c81700 1 --2- 172.21.15.62:0/3766419781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72cc105f30 0x7f72cc10da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.204+0000 7f72d3c81700 1 --2- 172.21.15.62:0/3766419781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72cc101a40 0x7f72cc105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.204+0000 7f72d3c81700 1 -- 172.21.15.62:0/3766419781 >> 172.21.15.62:0/3766419781 conn(0x7f72cc0766b0 msgr2=0x7f72cc076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.204+0000 7f72d3c81700 1 -- 172.21.15.62:0/3766419781 shutdown_connections 2024-06-10T00:38:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.204+0000 7f72d3c81700 1 -- 172.21.15.62:0/3766419781 wait complete. 2024-06-10T00:38:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.204+0000 7f72d3c81700 1 Processor -- start 2024-06-10T00:38:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d3c81700 1 -- start start 2024-06-10T00:38:41.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d3c81700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72cc101090 0x7f72cc196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d3c81700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72cc101a40 0x7f72cc196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d3c81700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72cc105f30 0x7f72cc197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d3c81700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc110630 con 0x7f72cc105f30 2024-06-10T00:38:41.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d3c81700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1104b0 con 0x7f72cc101090 2024-06-10T00:38:41.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d3c81700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1107b0 con 0x7f72cc101a40 2024-06-10T00:38:41.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d221e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72cc105f30 0x7f72cc197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d1a1d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72cc101090 0x7f72cc196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d1a1d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72cc101090 0x7f72cc196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:56860/0 (socket says 172.21.15.62:56860) 2024-06-10T00:38:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d221e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72cc105f30 0x7f72cc197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50180/0 (socket says 172.21.15.62:50180) 2024-06-10T00:38:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d1a1d700 1 -- 172.21.15.62:0/1787979739 learned_addr learned my addr 172.21.15.62:0/1787979739 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d121c700 1 --2- 172.21.15.62:0/1787979739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72cc101a40 0x7f72cc196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d221e700 1 -- 172.21.15.62:0/1787979739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72cc101a40 msgr2=0x7f72cc196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d221e700 1 --2- 172.21.15.62:0/1787979739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72cc101a40 0x7f72cc196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d221e700 1 -- 172.21.15.62:0/1787979739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72cc101090 msgr2=0x7f72cc196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d221e700 1 --2- 172.21.15.62:0/1787979739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72cc101090 0x7f72cc196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d221e700 1 -- 172.21.15.62:0/1787979739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72cc1a7160 con 0x7f72cc105f30 2024-06-10T00:38:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.205+0000 7f72d1a1d700 1 --2- 172.21.15.62:0/1787979739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72cc101090 0x7f72cc196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:38:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.206+0000 7f72d221e700 1 --2- 172.21.15.62:0/1787979739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72cc105f30 0x7f72cc197290 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f72c400d170 tx=0x7f72c400d480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.206+0000 7f72c2ffd700 1 -- 172.21.15.62:0/1787979739 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c400dd10 con 0x7f72cc105f30 2024-06-10T00:38:41.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.206+0000 7f72d3c81700 1 -- 172.21.15.62:0/1787979739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72cc1a73f0 con 0x7f72cc105f30 2024-06-10T00:38:41.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.206+0000 7f72c2ffd700 1 -- 172.21.15.62:0/1787979739 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c4003a00 con 0x7f72cc105f30 2024-06-10T00:38:41.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.206+0000 7f72c2ffd700 1 -- 172.21.15.62:0/1787979739 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4008bb0 con 0x7f72cc105f30 2024-06-10T00:38:41.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.206+0000 7f72d3c81700 1 -- 172.21.15.62:0/1787979739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72cc1a7930 con 0x7f72cc105f30 2024-06-10T00:38:41.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.208+0000 7f72c2ffd700 1 -- 172.21.15.62:0/1787979739 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f72c401a020 con 0x7f72cc105f30 2024-06-10T00:38:41.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.209+0000 7f72c2ffd700 1 --2- 172.21.15.62:0/1787979739 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f72b80a2d90 0x7f72b80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.209+0000 7f72d1a1d700 1 --2- 172.21.15.62:0/1787979739 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f72b80a2d90 0x7f72b80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.209+0000 7f72c2ffd700 1 -- 172.21.15.62:0/1787979739 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f72c40cf1d0 con 0x7f72cc105f30 2024-06-10T00:38:41.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.210+0000 7f72d1a1d700 1 --2- 172.21.15.62:0/1787979739 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f72b80a2d90 0x7f72b80a5250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f72c800b5c0 tx=0x7f72c8005c00 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:41.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.210+0000 7f72de504b80 1 Processor -- start 2024-06-10T00:38:41.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.210+0000 7f72de504b80 1 -- start start 2024-06-10T00:38:41.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.210+0000 7f72de504b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d990640 0x55f63d990a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.210+0000 7f72de504b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990f60 0x55f63d98a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.210+0000 7f72de504b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d98af20 0x55f63d98b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.210+0000 7f72de504b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f63d8dc810 con 0x55f63d98af20 2024-06-10T00:38:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.210+0000 7f72de504b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f63d884720 con 0x55f63d990f60 2024-06-10T00:38:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.210+0000 7f72de504b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f63d8d87d0 con 0x55f63d990640 2024-06-10T00:38:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.210+0000 7f72d221e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d98af20 0x55f63d98b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.210+0000 7f72d221e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d98af20 0x55f63d98b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50192/0 (socket says 172.21.15.62:50192) 2024-06-10T00:38:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.210+0000 7f72d221e700 1 -- 172.21.15.62:0/2182707858 learned_addr learned my addr 172.21.15.62:0/2182707858 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.211+0000 7f72d1a1d700 1 --2- 172.21.15.62:0/2182707858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d990640 0x55f63d990a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.211+0000 7f72d121c700 1 --2- 172.21.15.62:0/2182707858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990f60 0x55f63d98a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.211+0000 7f72d221e700 1 -- 172.21.15.62:0/2182707858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d990640 msgr2=0x55f63d990a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.211+0000 7f72d221e700 1 --2- 172.21.15.62:0/2182707858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d990640 0x55f63d990a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.211+0000 7f72d221e700 1 -- 172.21.15.62:0/2182707858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990f60 msgr2=0x55f63d98a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.211+0000 7f72d221e700 1 --2- 172.21.15.62:0/2182707858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990f60 0x55f63d98a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.211+0000 7f72d221e700 1 -- 172.21.15.62:0/2182707858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f63d87af50 con 0x55f63d98af20 2024-06-10T00:38:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.211+0000 7f72d221e700 1 --2- 172.21.15.62:0/2182707858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d98af20 0x55f63d98b340 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f72c40060b0 tx=0x7f72c409b240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1e7e467b6f54d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:41.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.211+0000 7f72b7fff700 1 -- 172.21.15.62:0/2182707858 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c409f6f0 con 0x55f63d98af20 2024-06-10T00:38:41.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.211+0000 7f72b7fff700 1 -- 172.21.15.62:0/2182707858 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c40a3fb0 con 0x55f63d98af20 2024-06-10T00:38:41.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.211+0000 7f72b7fff700 1 -- 172.21.15.62:0/2182707858 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c409b5a0 con 0x55f63d98af20 2024-06-10T00:38:41.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.212+0000 7f72de504b80 1 -- 172.21.15.62:0/2182707858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d98af20 msgr2=0x55f63d98b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:41.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.212+0000 7f72de504b80 1 --2- 172.21.15.62:0/2182707858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d98af20 0x55f63d98b340 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f72c40060b0 tx=0x7f72c409b240 comp rx=0 tx=0).stop 2024-06-10T00:38:41.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.212+0000 7f72de504b80 1 -- 172.21.15.62:0/2182707858 shutdown_connections 2024-06-10T00:38:41.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.212+0000 7f72de504b80 1 --2- 172.21.15.62:0/2182707858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d98af20 0x55f63d98b340 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.212+0000 7f72de504b80 1 --2- 172.21.15.62:0/2182707858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d990640 0x55f63d990a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.212+0000 7f72de504b80 1 --2- 172.21.15.62:0/2182707858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990f60 0x55f63d98a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.212+0000 7f72de504b80 1 -- 172.21.15.62:0/2182707858 >> 172.21.15.62:0/2182707858 conn(0x55f63d8b5cc0 msgr2=0x55f63d985850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:41.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.212+0000 7f72de504b80 1 -- 172.21.15.62:0/2182707858 shutdown_connections 2024-06-10T00:38:41.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.212+0000 7f72de504b80 1 -- 172.21.15.62:0/2182707858 wait complete. 2024-06-10T00:38:41.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.212+0000 7f72de504b80 1 -- start start 2024-06-10T00:38:41.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.213+0000 7f72de504b80 1 Processor -- start 2024-06-10T00:38:41.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.213+0000 7f72de504b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d98af20 0x55f63d996d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.213+0000 7f72de504b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990640 0x55f63d9940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.213+0000 7f72de504b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990f60 0x55f63d994620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.213+0000 7f72de504b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f63d9979d0 con 0x55f63d990f60 2024-06-10T00:38:41.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.213+0000 7f72de504b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f63d87cbf0 con 0x55f63d990640 2024-06-10T00:38:41.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.213+0000 7f72de504b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f63d997b50 con 0x55f63d98af20 2024-06-10T00:38:41.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.213+0000 7f72d221e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990f60 0x55f63d994620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.213+0000 7f72d221e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990f60 0x55f63d994620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50202/0 (socket says 172.21.15.62:50202) 2024-06-10T00:38:41.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.213+0000 7f72d221e700 1 -- 172.21.15.62:0/4225476876 learned_addr learned my addr 172.21.15.62:0/4225476876 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:41.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.213+0000 7f72de504b80 10 client.0 _ll_register_callbacks cb 0x55f63d876920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:41.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.213+0000 7f72d121c700 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990640 0x55f63d9940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.213+0000 7f72d1a1d700 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d98af20 0x55f63d996d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72d221e700 1 -- 172.21.15.62:0/4225476876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d98af20 msgr2=0x55f63d996d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72d221e700 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d98af20 0x55f63d996d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72d221e700 1 -- 172.21.15.62:0/4225476876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990640 msgr2=0x55f63d9940e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72d221e700 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990640 0x55f63d9940e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72d221e700 1 -- 172.21.15.62:0/4225476876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f63d82b8d0 con 0x55f63d990f60 2024-06-10T00:38:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72d121c700 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990640 0x55f63d9940e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:38:41.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72d1a1d700 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d98af20 0x55f63d996d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:38:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72d221e700 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990f60 0x55f63d994620 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f72c40977a0 tx=0x7f72c4008610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84326ea70535c46b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72b57fa700 1 -- 172.21.15.62:0/4225476876 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c40194d0 con 0x55f63d990f60 2024-06-10T00:38:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72b57fa700 1 -- 172.21.15.62:0/4225476876 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f72c409b5a0 con 0x55f63d990f60 2024-06-10T00:38:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72de504b80 1 -- 172.21.15.62:0/4225476876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72c40a3fb0 con 0x55f63d990f60 2024-06-10T00:38:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72b57fa700 1 -- 172.21.15.62:0/4225476876 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f72c409b7a0 con 0x55f63d990f60 2024-06-10T00:38:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72de504b80 10 client.16386 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:41.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72de504b80 1 -- 172.21.15.62:0/4225476876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f63d9a23d0 con 0x55f63d990f60 2024-06-10T00:38:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.214+0000 7f72b57fa700 1 -- 172.21.15.62:0/4225476876 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c40a3fb0 con 0x55f63d990f60 2024-06-10T00:38:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.215+0000 7f72b57fa700 1 -- 172.21.15.62:0/4225476876 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f72c409a870 con 0x55f63d990f60 2024-06-10T00:38:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.215+0000 7f72de504b80 10 client.16386 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.215+0000 7f72de504b80 10 client.16386 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:38:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.215+0000 7f72de504b80 20 client.16386 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.215+0000 7f72de504b80 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f63d9a48a0 0x55f63d9a6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.215+0000 7f72de504b80 4 client.16386 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:38:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.215+0000 7f72de504b80 1 -- 172.21.15.62:0/4225476876 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f63d8e9d10 con 0x55f63d9a48a0 2024-06-10T00:38:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.215+0000 7f72d1a1d700 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f63d9a48a0 0x55f63d9a6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.215+0000 7f72d1a1d700 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f63d9a48a0 0x55f63d9a6c90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=734fe1c2b59fc9e5 server_cookie=70a7ed0efedaef23 in_seq=0 out_seq=0 2024-06-10T00:38:41.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.215+0000 7f72b57fa700 10 client.16386 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:38:41.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.216+0000 7f72b57fa700 1 -- 172.21.15.62:0/4225476876 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f63d8e9d10 con 0x55f63d9a48a0 2024-06-10T00:38:41.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.216+0000 7f72b57fa700 10 client.16386 handle_command_reply: tid=0 2024-06-10T00:38:41.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.216+0000 7f72de504b80 1 client.16386 shutdown 2024-06-10T00:38:41.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.216+0000 7f72de504b80 1 -- 172.21.15.62:0/4225476876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990f60 msgr2=0x55f63d994620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:41.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.216+0000 7f72de504b80 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990f60 0x55f63d994620 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f72c40977a0 tx=0x7f72c4008610 comp rx=0 tx=0).stop 2024-06-10T00:38:41.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.217+0000 7f72de504b80 1 -- 172.21.15.62:0/4225476876 shutdown_connections 2024-06-10T00:38:41.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.217+0000 7f72de504b80 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990f60 0x55f63d994620 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.217+0000 7f72de504b80 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f63d9a48a0 0x55f63d9a6c90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.217+0000 7f72de504b80 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d98af20 0x55f63d996d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.217+0000 7f72de504b80 1 --2- 172.21.15.62:0/4225476876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990640 0x55f63d9940e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.217+0000 7f72b57fa700 0 client.16386 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:38:41.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.217+0000 7f72de504b80 1 -- 172.21.15.62:0/4225476876 >> 172.21.15.62:0/4225476876 conn(0x55f63d8b5cc0 msgr2=0x55f63d9899d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:41.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.217+0000 7f72de504b80 1 -- 172.21.15.62:0/4225476876 shutdown_connections 2024-06-10T00:38:41.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.217+0000 7f72de504b80 1 -- 172.21.15.62:0/4225476876 wait complete. 2024-06-10T00:38:41.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.217+0000 7f72de504b80 20 client.16386 trim_cache size 0 max 16384 2024-06-10T00:38:41.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72de504b80 1 Processor -- start 2024-06-10T00:38:41.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72de504b80 1 -- start start 2024-06-10T00:38:41.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72de504b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d98af20 0x55f63d98a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72de504b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990640 0x55f63d97f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72de504b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d990f60 0x55f63d991340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72de504b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1b0930 con 0x55f63d990640 2024-06-10T00:38:41.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72de504b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1b25f0 con 0x55f63d98af20 2024-06-10T00:38:41.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72de504b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1b1000 con 0x55f63d990f60 2024-06-10T00:38:41.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72d121c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990640 0x55f63d97f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72d121c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990640 0x55f63d97f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50206/0 (socket says 172.21.15.62:50206) 2024-06-10T00:38:41.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72d121c700 1 -- 172.21.15.62:0/2781466295 learned_addr learned my addr 172.21.15.62:0/2781466295 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:41.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72d221e700 1 --2- 172.21.15.62:0/2781466295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d990f60 0x55f63d991340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72d1a1d700 1 --2- 172.21.15.62:0/2781466295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d98af20 0x55f63d98a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72d121c700 1 -- 172.21.15.62:0/2781466295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d990f60 msgr2=0x55f63d991340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:41.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72d121c700 1 --2- 172.21.15.62:0/2781466295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d990f60 0x55f63d991340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72d121c700 1 -- 172.21.15.62:0/2781466295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d98af20 msgr2=0x55f63d98a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:41.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72d121c700 1 --2- 172.21.15.62:0/2781466295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d98af20 0x55f63d98a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.233+0000 7f72d121c700 1 -- 172.21.15.62:0/2781466295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f63d992650 con 0x55f63d990640 2024-06-10T00:38:41.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.234+0000 7f72d121c700 1 --2- 172.21.15.62:0/2781466295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990640 0x55f63d97f510 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f72cc074c80 tx=0x7f72bc012780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df58789e554f6ee7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:41.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.234+0000 7f72b77fe700 1 -- 172.21.15.62:0/2781466295 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc020070 con 0x55f63d990640 2024-06-10T00:38:41.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.234+0000 7f72b77fe700 1 -- 172.21.15.62:0/2781466295 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72bc012dc0 con 0x55f63d990640 2024-06-10T00:38:41.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.234+0000 7f72b77fe700 1 -- 172.21.15.62:0/2781466295 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc01b910 con 0x55f63d990640 2024-06-10T00:38:41.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.235+0000 7f72de504b80 1 -- 172.21.15.62:0/2781466295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990640 msgr2=0x55f63d97f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:41.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.235+0000 7f72de504b80 1 --2- 172.21.15.62:0/2781466295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990640 0x55f63d97f510 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f72cc074c80 tx=0x7f72bc012780 comp rx=0 tx=0).stop 2024-06-10T00:38:41.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.235+0000 7f72de504b80 1 -- 172.21.15.62:0/2781466295 shutdown_connections 2024-06-10T00:38:41.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.235+0000 7f72de504b80 1 --2- 172.21.15.62:0/2781466295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d990f60 0x55f63d991340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.235+0000 7f72de504b80 1 --2- 172.21.15.62:0/2781466295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d98af20 0x55f63d98a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.235+0000 7f72de504b80 1 --2- 172.21.15.62:0/2781466295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990640 0x55f63d97f510 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.235+0000 7f72de504b80 1 -- 172.21.15.62:0/2781466295 >> 172.21.15.62:0/2781466295 conn(0x55f63d8b5cc0 msgr2=0x55f63d97ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:41.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.235+0000 7f72de504b80 1 -- 172.21.15.62:0/2781466295 shutdown_connections 2024-06-10T00:38:41.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.235+0000 7f72de504b80 1 -- 172.21.15.62:0/2781466295 wait complete. 2024-06-10T00:38:41.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.235+0000 7f72de504b80 1 -- start start 2024-06-10T00:38:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.235+0000 7f72de504b80 1 Processor -- start 2024-06-10T00:38:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.235+0000 7f72de504b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d98af20 0x55f63d9aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.235+0000 7f72de504b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990640 0x55f63d9a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.235+0000 7f72de504b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990f60 0x55f63d9a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72de504b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1b1e60 con 0x55f63d990640 2024-06-10T00:38:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72de504b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1b16d0 con 0x55f63d990f60 2024-06-10T00:38:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72de504b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1ae8d0 con 0x55f63d98af20 2024-06-10T00:38:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72de504b80 10 client.0 _ll_register_callbacks cb 0x55f63d876920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:41.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72d121c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990640 0x55f63d9a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72d121c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990640 0x55f63d9a74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50220/0 (socket says 172.21.15.62:50220) 2024-06-10T00:38:41.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72d121c700 1 -- 172.21.15.62:0/1591896071 learned_addr learned my addr 172.21.15.62:0/1591896071 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:41.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72d1a1d700 1 --2- 172.21.15.62:0/1591896071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d98af20 0x55f63d9aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72d221e700 1 --2- 172.21.15.62:0/1591896071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990f60 0x55f63d9a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72d121c700 1 -- 172.21.15.62:0/1591896071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d98af20 msgr2=0x55f63d9aa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:41.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72d121c700 1 --2- 172.21.15.62:0/1591896071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d98af20 0x55f63d9aa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72d121c700 1 -- 172.21.15.62:0/1591896071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990f60 msgr2=0x55f63d9a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:41.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72d121c700 1 --2- 172.21.15.62:0/1591896071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990f60 0x55f63d9a7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72d121c700 1 -- 172.21.15.62:0/1591896071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f63d9a7f60 con 0x55f63d990640 2024-06-10T00:38:41.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72d1a1d700 1 --2- 172.21.15.62:0/1591896071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d98af20 0x55f63d9aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:38:41.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72d121c700 1 --2- 172.21.15.62:0/1591896071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990640 0x55f63d9a74e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f72bc005950 tx=0x7f72bc0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e6087e60d79791 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:41.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.236+0000 7f72b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:41.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72b57fa700 1 -- 172.21.15.62:0/1591896071 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc02b060 con 0x55f63d990640 2024-06-10T00:38:41.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72de504b80 1 -- 172.21.15.62:0/1591896071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72bc012dc0 con 0x55f63d990640 2024-06-10T00:38:41.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72b57fa700 1 -- 172.21.15.62:0/1591896071 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f72bc00f420 con 0x55f63d990640 2024-06-10T00:38:41.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72b57fa700 1 -- 172.21.15.62:0/1591896071 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f72bc020040 con 0x55f63d990640 2024-06-10T00:38:41.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72de504b80 10 client.16392 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:41.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72de504b80 1 -- 172.21.15.62:0/1591896071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f63da0ef70 con 0x55f63d990640 2024-06-10T00:38:41.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72b57fa700 1 -- 172.21.15.62:0/1591896071 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72bc012dc0 con 0x55f63d990640 2024-06-10T00:38:41.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72b57fa700 1 -- 172.21.15.62:0/1591896071 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f72bc004c30 con 0x55f63d990640 2024-06-10T00:38:41.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72de504b80 10 client.16392 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:41.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72de504b80 10 client.16392 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:38:41.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72de504b80 20 client.16392 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:41.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72de504b80 1 --2- 172.21.15.62:0/1591896071 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f63da0f660 0x55f63da11a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:41.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72de504b80 4 client.16392 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:38:41.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72de504b80 1 -- 172.21.15.62:0/1591896071 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f63d8e4290 con 0x55f63da0f660 2024-06-10T00:38:41.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.237+0000 7f72d1a1d700 1 --2- 172.21.15.62:0/1591896071 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f63da0f660 0x55f63da11a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:41.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72d1a1d700 1 --2- 172.21.15.62:0/1591896071 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f63da0f660 0x55f63da11a80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e2899195e15a4a4 server_cookie=ee41cfd18c620d96 in_seq=0 out_seq=0 2024-06-10T00:38:41.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72b57fa700 10 client.16392 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:38:41.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72b57fa700 1 -- 172.21.15.62:0/1591896071 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f63d8e4290 con 0x55f63da0f660 2024-06-10T00:38:41.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72b57fa700 10 client.16392 handle_command_reply: tid=0 2024-06-10T00:38:41.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72de504b80 1 client.16392 shutdown 2024-06-10T00:38:41.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72de504b80 1 -- 172.21.15.62:0/1591896071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990640 msgr2=0x55f63d9a74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:41.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72de504b80 1 --2- 172.21.15.62:0/1591896071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990640 0x55f63d9a74e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f72bc005950 tx=0x7f72bc0056d0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72de504b80 1 -- 172.21.15.62:0/1591896071 shutdown_connections 2024-06-10T00:38:41.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72de504b80 1 --2- 172.21.15.62:0/1591896071 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x55f63da0f660 0x55f63da11a80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72de504b80 1 --2- 172.21.15.62:0/1591896071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f63d990f60 0x55f63d9a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72de504b80 1 --2- 172.21.15.62:0/1591896071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63d98af20 0x55f63d9aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72b57fa700 0 client.16392 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:38:41.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72de504b80 1 --2- 172.21.15.62:0/1591896071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f63d990640 0x55f63d9a74e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72de504b80 1 -- 172.21.15.62:0/1591896071 >> 172.21.15.62:0/1591896071 conn(0x55f63d8b5cc0 msgr2=0x55f63d9929d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:41.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72de504b80 1 -- 172.21.15.62:0/1591896071 shutdown_connections 2024-06-10T00:38:41.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72de504b80 1 -- 172.21.15.62:0/1591896071 wait complete. 2024-06-10T00:38:41.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.238+0000 7f72de504b80 20 client.16392 trim_cache size 0 max 16384 2024-06-10T00:38:41.251 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:38:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.239+0000 7f72d3c81700 1 -- 172.21.15.62:0/1787979739 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f72b80a2d90 msgr2=0x7f72b80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.239+0000 7f72d3c81700 1 --2- 172.21.15.62:0/1787979739 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f72b80a2d90 0x7f72b80a5250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f72c800b5c0 tx=0x7f72c8005c00 comp rx=0 tx=0).stop 2024-06-10T00:38:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.239+0000 7f72d3c81700 1 -- 172.21.15.62:0/1787979739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72cc105f30 msgr2=0x7f72cc197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.239+0000 7f72d3c81700 1 --2- 172.21.15.62:0/1787979739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72cc105f30 0x7f72cc197290 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f72c400d170 tx=0x7f72c400d480 comp rx=0 tx=0).stop 2024-06-10T00:38:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.239+0000 7f72d3c81700 1 -- 172.21.15.62:0/1787979739 shutdown_connections 2024-06-10T00:38:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.239+0000 7f72d3c81700 1 --2- 172.21.15.62:0/1787979739 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f72b80a2d90 0x7f72b80a5250 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.239+0000 7f72d3c81700 1 --2- 172.21.15.62:0/1787979739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72cc105f30 0x7f72cc197290 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.239+0000 7f72d3c81700 1 --2- 172.21.15.62:0/1787979739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72cc101090 0x7f72cc196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.239+0000 7f72d3c81700 1 --2- 172.21.15.62:0/1787979739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72cc101a40 0x7f72cc196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.239+0000 7f72d3c81700 1 -- 172.21.15.62:0/1787979739 >> 172.21.15.62:0/1787979739 conn(0x7f72cc0766b0 msgr2=0x7f72cc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.239+0000 7f72d3c81700 1 -- 172.21.15.62:0/1787979739 shutdown_connections 2024-06-10T00:38:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:41.239+0000 7f72d3c81700 1 -- 172.21.15.62:0/1787979739 wait complete. 2024-06-10T00:38:41.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:41 smithi105 ceph-mon[36321]: pgmap v178: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T00:38:41.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:41 smithi062 ceph-mon[32390]: pgmap v178: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T00:38:41.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:41 smithi138 ceph-mon[36549]: pgmap v178: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T00:38:42.066 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T00:38:42.318 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:38:43.463 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:43 smithi062 ceph-mon[32390]: pgmap v179: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T00:38:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:43 smithi105 ceph-mon[36321]: pgmap v179: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T00:38:43.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:43 smithi138 ceph-mon[36549]: pgmap v179: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T00:38:44.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.151+0000 7ff356b46700 1 -- 172.21.15.62:0/1195247976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff350102ed0 msgr2=0x7ff35010f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:44.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.151+0000 7ff356b46700 1 --2- 172.21.15.62:0/1195247976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff350102ed0 0x7ff35010f360 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff34800b0a0 tx=0x7ff34800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.151+0000 7ff356b46700 1 -- 172.21.15.62:0/1195247976 shutdown_connections 2024-06-10T00:38:44.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.151+0000 7ff356b46700 1 --2- 172.21.15.62:0/1195247976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff350102ed0 0x7ff35010f360 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.151+0000 7ff356b46700 1 --2- 172.21.15.62:0/1195247976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff350108520 0x7ff350108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.151+0000 7ff356b46700 1 --2- 172.21.15.62:0/1195247976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff350102510 0x7ff350102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.151+0000 7ff356b46700 1 -- 172.21.15.62:0/1195247976 >> 172.21.15.62:0/1195247976 conn(0x7ff350079560 msgr2=0x7ff350079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.151+0000 7ff356b46700 1 -- 172.21.15.62:0/1195247976 shutdown_connections 2024-06-10T00:38:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.151+0000 7ff356b46700 1 -- 172.21.15.62:0/1195247976 wait complete. 2024-06-10T00:38:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.152+0000 7ff356b46700 1 Processor -- start 2024-06-10T00:38:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.152+0000 7ff356b46700 1 -- start start 2024-06-10T00:38:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.152+0000 7ff356b46700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff350102510 0x7ff35019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.152+0000 7ff356b46700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff350102ed0 0x7ff35019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.152+0000 7ff356b46700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff350108520 0x7ff350198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.152+0000 7ff356b46700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501127a0 con 0x7ff350108520 2024-06-10T00:38:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.152+0000 7ff356b46700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff350112620 con 0x7ff350102510 2024-06-10T00:38:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.152+0000 7ff356b46700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff350112920 con 0x7ff350102ed0 2024-06-10T00:38:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff3550e3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff350108520 0x7ff350198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff3550e3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff350108520 0x7ff350198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50232/0 (socket says 172.21.15.62:50232) 2024-06-10T00:38:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff3550e3700 1 -- 172.21.15.62:0/1864733858 learned_addr learned my addr 172.21.15.62:0/1864733858 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff3548e2700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff350102510 0x7ff35019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff34ffff700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff350102ed0 0x7ff35019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff3550e3700 1 -- 172.21.15.62:0/1864733858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff350102ed0 msgr2=0x7ff35019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:44.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff3550e3700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff350102ed0 0x7ff35019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff3550e3700 1 -- 172.21.15.62:0/1864733858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff350102510 msgr2=0x7ff35019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:44.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff3550e3700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff350102510 0x7ff35019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff3550e3700 1 -- 172.21.15.62:0/1864733858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff350199060 con 0x7ff350108520 2024-06-10T00:38:44.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff34ffff700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff350102ed0 0x7ff35019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:38:44.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff3550e3700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff350108520 0x7ff350198900 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff3480062a0 tx=0x7ff348004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:44.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff34dffb700 1 -- 172.21.15.62:0/1864733858 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348021070 con 0x7ff350108520 2024-06-10T00:38:44.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff356b46700 1 -- 172.21.15.62:0/1864733858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3501992f0 con 0x7ff350108520 2024-06-10T00:38:44.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff34dffb700 1 -- 172.21.15.62:0/1864733858 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff348007c00 con 0x7ff350108520 2024-06-10T00:38:44.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.153+0000 7ff34dffb700 1 -- 172.21.15.62:0/1864733858 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34801d450 con 0x7ff350108520 2024-06-10T00:38:44.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.154+0000 7ff356b46700 1 -- 172.21.15.62:0/1864733858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3501a5610 con 0x7ff350108520 2024-06-10T00:38:44.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.156+0000 7ff34dffb700 1 -- 172.21.15.62:0/1864733858 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff348004020 con 0x7ff350108520 2024-06-10T00:38:44.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.156+0000 7ff34dffb700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff3400a2d90 0x7ff3400a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.157+0000 7ff34dffb700 1 -- 172.21.15.62:0/1864733858 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7ff34800f020 con 0x7ff350108520 2024-06-10T00:38:44.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.157+0000 7ff3548e2700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff3400a2d90 0x7ff3400a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.157+0000 7ff3548e2700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff3400a2d90 0x7ff3400a5250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff350199a40 tx=0x7ff3440073d0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:44.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.157+0000 7ff3613c9b80 1 Processor -- start 2024-06-10T00:38:44.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.157+0000 7ff3613c9b80 1 -- start start 2024-06-10T00:38:44.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff3613c9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6ae850 0x560abc6aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff3613c9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6af170 0x560abc6b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff3613c9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6b3550 0x560abc6b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff3613c9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560abc59cf60 con 0x560abc6af170 2024-06-10T00:38:44.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff3613c9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560abc5fb850 con 0x560abc6b3550 2024-06-10T00:38:44.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff3613c9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560abc563460 con 0x560abc6ae850 2024-06-10T00:38:44.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff34ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6af170 0x560abc6b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff34ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6af170 0x560abc6b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50240/0 (socket says 172.21.15.62:50240) 2024-06-10T00:38:44.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff34ffff700 1 -- 172.21.15.62:0/2034057691 learned_addr learned my addr 172.21.15.62:0/2034057691 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:44.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff3548e2700 1 --2- 172.21.15.62:0/2034057691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6ae850 0x560abc6aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff3550e3700 1 --2- 172.21.15.62:0/2034057691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6b3550 0x560abc6b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff34ffff700 1 -- 172.21.15.62:0/2034057691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6ae850 msgr2=0x560abc6aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:44.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff34ffff700 1 --2- 172.21.15.62:0/2034057691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6ae850 0x560abc6aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff34ffff700 1 -- 172.21.15.62:0/2034057691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6b3550 msgr2=0x560abc6b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:44.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff34ffff700 1 --2- 172.21.15.62:0/2034057691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6b3550 0x560abc6b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff34ffff700 1 -- 172.21.15.62:0/2034057691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abc54a8d0 con 0x560abc6af170 2024-06-10T00:38:44.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff34ffff700 1 --2- 172.21.15.62:0/2034057691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6af170 0x560abc6b3010 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00bd60 tx=0x7ff33c00f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a21836894092b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:44.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.158+0000 7ff33affd700 1 -- 172.21.15.62:0/2034057691 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c00fda0 con 0x560abc6af170 2024-06-10T00:38:44.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.159+0000 7ff33affd700 1 -- 172.21.15.62:0/2034057691 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff33c011920 con 0x560abc6af170 2024-06-10T00:38:44.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.159+0000 7ff33affd700 1 -- 172.21.15.62:0/2034057691 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c010810 con 0x560abc6af170 2024-06-10T00:38:44.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.159+0000 7ff3613c9b80 1 -- 172.21.15.62:0/2034057691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6af170 msgr2=0x560abc6b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:44.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.159+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/2034057691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6af170 0x560abc6b3010 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00bd60 tx=0x7ff33c00f640 comp rx=0 tx=0).stop 2024-06-10T00:38:44.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.159+0000 7ff3613c9b80 1 -- 172.21.15.62:0/2034057691 shutdown_connections 2024-06-10T00:38:44.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.159+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/2034057691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6b3550 0x560abc6b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.159+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/2034057691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6ae850 0x560abc6aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.159+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/2034057691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6af170 0x560abc6b3010 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.159+0000 7ff3613c9b80 1 -- 172.21.15.62:0/2034057691 >> 172.21.15.62:0/2034057691 conn(0x560abc5d7080 msgr2=0x560abc6a5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:44.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.159+0000 7ff3613c9b80 1 -- 172.21.15.62:0/2034057691 shutdown_connections 2024-06-10T00:38:44.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.159+0000 7ff3613c9b80 1 -- 172.21.15.62:0/2034057691 wait complete. 2024-06-10T00:38:44.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.160+0000 7ff3613c9b80 1 -- start start 2024-06-10T00:38:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.160+0000 7ff3613c9b80 1 Processor -- start 2024-06-10T00:38:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.160+0000 7ff3613c9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6ae850 0x560abc6baa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.160+0000 7ff3613c9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6af170 0x560abc6b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.160+0000 7ff3613c9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 0x560abc6b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.160+0000 7ff3613c9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560abc59bfc0 con 0x560abc6ae850 2024-06-10T00:38:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.160+0000 7ff3613c9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560abc5f7810 con 0x560abc6af170 2024-06-10T00:38:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.160+0000 7ff3613c9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560abc6bb680 con 0x560abc6b3550 2024-06-10T00:38:44.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.160+0000 7ff3613c9b80 10 client.0 _ll_register_callbacks cb 0x560abc5915b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.160+0000 7ff3548e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6ae850 0x560abc6baa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.160+0000 7ff3548e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6ae850 0x560abc6baa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50244/0 (socket says 172.21.15.62:50244) 2024-06-10T00:38:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.160+0000 7ff3548e2700 1 -- 172.21.15.62:0/1368506516 learned_addr learned my addr 172.21.15.62:0/1368506516 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff34ffff700 1 --2- 172.21.15.62:0/1368506516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6af170 0x560abc6b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff3550e3700 1 --2- 172.21.15.62:0/1368506516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 0x560abc6b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff3548e2700 1 -- 172.21.15.62:0/1368506516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 msgr2=0x560abc6b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff3548e2700 1 --2- 172.21.15.62:0/1368506516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 0x560abc6b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff3548e2700 1 -- 172.21.15.62:0/1368506516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6af170 msgr2=0x560abc6b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:44.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff3548e2700 1 --2- 172.21.15.62:0/1368506516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6af170 0x560abc6b7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff3548e2700 1 -- 172.21.15.62:0/1368506516 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abc5c1750 con 0x560abc6ae850 2024-06-10T00:38:44.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff3550e3700 1 --2- 172.21.15.62:0/1368506516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 0x560abc6b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:38:44.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff3548e2700 1 --2- 172.21.15.62:0/1368506516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6ae850 0x560abc6baa60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff3440116f0 tx=0x7ff3440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c30ec49ef0ff306 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:44.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff32ffff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:44.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff32ffff700 1 -- 172.21.15.62:0/1368506516 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344004250 con 0x560abc6ae850 2024-06-10T00:38:44.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff3613c9b80 1 -- 172.21.15.62:0/1368506516 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff33c011920 con 0x560abc6ae850 2024-06-10T00:38:44.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff32ffff700 1 -- 172.21.15.62:0/1368506516 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7ff3440043f0 con 0x560abc6ae850 2024-06-10T00:38:44.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff32ffff700 1 -- 172.21.15.62:0/1368506516 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff344030050 con 0x560abc6ae850 2024-06-10T00:38:44.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff3613c9b80 10 client.16404 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:44.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.161+0000 7ff3613c9b80 1 -- 172.21.15.62:0/1368506516 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560abc6c5e90 con 0x560abc6ae850 2024-06-10T00:38:44.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.162+0000 7ff32ffff700 1 -- 172.21.15.62:0/1368506516 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff33c011920 con 0x560abc6ae850 2024-06-10T00:38:44.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.162+0000 7ff32ffff700 1 -- 172.21.15.62:0/1368506516 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7ff34401f670 con 0x560abc6ae850 2024-06-10T00:38:44.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.162+0000 7ff3613c9b80 10 client.16404 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:44.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.162+0000 7ff3613c9b80 10 client.16404 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:38:44.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.162+0000 7ff3613c9b80 20 client.16404 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:44.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.162+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/1368506516 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x560abc6c8520 0x560abc6ca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.162+0000 7ff3613c9b80 4 client.16404 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:38:44.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.162+0000 7ff3613c9b80 1 -- 172.21.15.62:0/1368506516 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560abc608d50 con 0x560abc6c8520 2024-06-10T00:38:44.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.162+0000 7ff34ffff700 1 --2- 172.21.15.62:0/1368506516 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x560abc6c8520 0x560abc6ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.163+0000 7ff34ffff700 1 --2- 172.21.15.62:0/1368506516 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x560abc6c8520 0x560abc6ca960 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7275ce1bde87a5bf server_cookie=f08bf8e5a8d72bb7 in_seq=0 out_seq=0 2024-06-10T00:38:44.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.163+0000 7ff32ffff700 10 client.16404 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:38:44.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.163+0000 7ff32ffff700 1 -- 172.21.15.62:0/1368506516 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560abc608d50 con 0x560abc6c8520 2024-06-10T00:38:44.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.163+0000 7ff32ffff700 10 client.16404 handle_command_reply: tid=0 2024-06-10T00:38:44.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.163+0000 7ff3613c9b80 1 client.16404 shutdown 2024-06-10T00:38:44.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.164+0000 7ff3613c9b80 1 -- 172.21.15.62:0/1368506516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6ae850 msgr2=0x560abc6baa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:44.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.164+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/1368506516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6ae850 0x560abc6baa60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff3440116f0 tx=0x7ff3440047c0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.164+0000 7ff3613c9b80 1 -- 172.21.15.62:0/1368506516 shutdown_connections 2024-06-10T00:38:44.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.164+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/1368506516 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x560abc6c8520 0x560abc6ca960 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.164+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/1368506516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6ae850 0x560abc6baa60 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.164+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/1368506516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 0x560abc6b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.164+0000 7ff32ffff700 0 client.16404 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:38:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.164+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/1368506516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6af170 0x560abc6b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.164+0000 7ff3613c9b80 1 -- 172.21.15.62:0/1368506516 >> 172.21.15.62:0/1368506516 conn(0x560abc5d7080 msgr2=0x560abc6b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.164+0000 7ff3613c9b80 1 -- 172.21.15.62:0/1368506516 shutdown_connections 2024-06-10T00:38:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.164+0000 7ff3613c9b80 1 -- 172.21.15.62:0/1368506516 wait complete. 2024-06-10T00:38:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.164+0000 7ff3613c9b80 20 client.16404 trim_cache size 0 max 16384 2024-06-10T00:38:44.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.179+0000 7ff3613c9b80 1 Processor -- start 2024-06-10T00:38:44.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.179+0000 7ff3613c9b80 1 -- start start 2024-06-10T00:38:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.179+0000 7ff3613c9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6ae850 0x560abc6b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.179+0000 7ff3613c9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6af170 0x560abc6a9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.179+0000 7ff3613c9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 0x560abc6d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.179+0000 7ff3613c9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501ac2f0 con 0x560abc6af170 2024-06-10T00:38:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.179+0000 7ff3613c9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501adfb0 con 0x560abc6ae850 2024-06-10T00:38:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.179+0000 7ff3613c9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501ac9c0 con 0x560abc6b3550 2024-06-10T00:38:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.179+0000 7ff34ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6af170 0x560abc6a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.180+0000 7ff34ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6af170 0x560abc6a9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50248/0 (socket says 172.21.15.62:50248) 2024-06-10T00:38:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.180+0000 7ff34ffff700 1 -- 172.21.15.62:0/3610706919 learned_addr learned my addr 172.21.15.62:0/3610706919 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.180+0000 7ff3550e3700 1 --2- 172.21.15.62:0/3610706919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 0x560abc6d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.180+0000 7ff3548e2700 1 --2- 172.21.15.62:0/3610706919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6ae850 0x560abc6b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.180+0000 7ff34ffff700 1 -- 172.21.15.62:0/3610706919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 msgr2=0x560abc6d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.180+0000 7ff34ffff700 1 --2- 172.21.15.62:0/3610706919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 0x560abc6d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.180+0000 7ff34ffff700 1 -- 172.21.15.62:0/3610706919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6ae850 msgr2=0x560abc6b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.180+0000 7ff34ffff700 1 --2- 172.21.15.62:0/3610706919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6ae850 0x560abc6b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.180+0000 7ff34ffff700 1 -- 172.21.15.62:0/3610706919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abc6a6040 con 0x560abc6af170 2024-06-10T00:38:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.180+0000 7ff34ffff700 1 --2- 172.21.15.62:0/3610706919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6af170 0x560abc6a9960 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00bf20 tx=0x7ff33c00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31e0aeb06492408 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.180+0000 7ff33a7fc700 1 -- 172.21.15.62:0/3610706919 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c005240 con 0x560abc6af170 2024-06-10T00:38:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.180+0000 7ff33a7fc700 1 -- 172.21.15.62:0/3610706919 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff33c004bf0 con 0x560abc6af170 2024-06-10T00:38:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.180+0000 7ff33a7fc700 1 -- 172.21.15.62:0/3610706919 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c027770 con 0x560abc6af170 2024-06-10T00:38:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.181+0000 7ff3613c9b80 1 -- 172.21.15.62:0/3610706919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6af170 msgr2=0x560abc6a9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.181+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/3610706919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6af170 0x560abc6a9960 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00bf20 tx=0x7ff33c00beb0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.181+0000 7ff3613c9b80 1 -- 172.21.15.62:0/3610706919 shutdown_connections 2024-06-10T00:38:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.181+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/3610706919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 0x560abc6d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.181+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/3610706919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6ae850 0x560abc6b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.181+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/3610706919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6af170 0x560abc6a9960 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.181+0000 7ff3613c9b80 1 -- 172.21.15.62:0/3610706919 >> 172.21.15.62:0/3610706919 conn(0x560abc5d7080 msgr2=0x560abc6a9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:44.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.181+0000 7ff3613c9b80 1 -- 172.21.15.62:0/3610706919 shutdown_connections 2024-06-10T00:38:44.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.181+0000 7ff3613c9b80 1 -- 172.21.15.62:0/3610706919 wait complete. 2024-06-10T00:38:44.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.181+0000 7ff3613c9b80 1 -- start start 2024-06-10T00:38:44.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.181+0000 7ff3613c9b80 1 Processor -- start 2024-06-10T00:38:44.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3613c9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6ae850 0x560abc6c0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3613c9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6af170 0x560abc6be1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3613c9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 0x560abc6be730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3613c9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501ad820 con 0x560abc6ae850 2024-06-10T00:38:44.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3613c9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501ad090 con 0x560abc6af170 2024-06-10T00:38:44.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3613c9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3501aa2b0 con 0x560abc6b3550 2024-06-10T00:38:44.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3613c9b80 10 client.0 _ll_register_callbacks cb 0x560abc5915b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:44.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3548e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6ae850 0x560abc6c0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3548e2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6ae850 0x560abc6c0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50264/0 (socket says 172.21.15.62:50264) 2024-06-10T00:38:44.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3548e2700 1 -- 172.21.15.62:0/3374066614 learned_addr learned my addr 172.21.15.62:0/3374066614 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:44.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3550e3700 1 --2- 172.21.15.62:0/3374066614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 0x560abc6be730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff34ffff700 1 --2- 172.21.15.62:0/3374066614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6af170 0x560abc6be1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3548e2700 1 -- 172.21.15.62:0/3374066614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 msgr2=0x560abc6be730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:44.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3548e2700 1 --2- 172.21.15.62:0/3374066614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 0x560abc6be730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3548e2700 1 -- 172.21.15.62:0/3374066614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6af170 msgr2=0x560abc6be1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:44.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3548e2700 1 --2- 172.21.15.62:0/3374066614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6af170 0x560abc6be1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3548e2700 1 -- 172.21.15.62:0/3374066614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abc6becf0 con 0x560abc6ae850 2024-06-10T00:38:44.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3550e3700 1 --2- 172.21.15.62:0/3374066614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 0x560abc6be730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:38:44.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff3548e2700 1 --2- 172.21.15.62:0/3374066614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6ae850 0x560abc6c0e00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff344022040 tx=0x7ff344010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726e1b01eba79233 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:44.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.182+0000 7ff32ffff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:44.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff32ffff700 1 -- 172.21.15.62:0/3374066614 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34401f840 con 0x560abc6ae850 2024-06-10T00:38:44.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff3613c9b80 1 -- 172.21.15.62:0/3374066614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff33c004bf0 con 0x560abc6ae850 2024-06-10T00:38:44.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff32ffff700 1 -- 172.21.15.62:0/3374066614 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7ff344004040 con 0x560abc6ae850 2024-06-10T00:38:44.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff32ffff700 1 -- 172.21.15.62:0/3374066614 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff344004240 con 0x560abc6ae850 2024-06-10T00:38:44.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff3613c9b80 10 client.16410 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:44.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff3613c9b80 1 -- 172.21.15.62:0/3374066614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560abc732fe0 con 0x560abc6ae850 2024-06-10T00:38:44.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff32ffff700 1 -- 172.21.15.62:0/3374066614 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff33c004bf0 con 0x560abc6ae850 2024-06-10T00:38:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff32ffff700 1 -- 172.21.15.62:0/3374066614 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7ff344028500 con 0x560abc6ae850 2024-06-10T00:38:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff3613c9b80 10 client.16410 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff3613c9b80 10 client.16410 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:38:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff3613c9b80 20 client.16410 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/3374066614 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x560abc7336f0 0x560abc735b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff3613c9b80 4 client.16410 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:38:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff3613c9b80 1 -- 172.21.15.62:0/3374066614 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560abc6032d0 con 0x560abc7336f0 2024-06-10T00:38:44.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.183+0000 7ff34ffff700 1 --2- 172.21.15.62:0/3374066614 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x560abc7336f0 0x560abc735b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff34ffff700 1 --2- 172.21.15.62:0/3374066614 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x560abc7336f0 0x560abc735b10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e158cbb013e9aa29 server_cookie=77a30ad303acdcf4 in_seq=0 out_seq=0 2024-06-10T00:38:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff32ffff700 10 client.16410 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:38:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff32ffff700 1 -- 172.21.15.62:0/3374066614 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560abc6032d0 con 0x560abc7336f0 2024-06-10T00:38:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff32ffff700 10 client.16410 handle_command_reply: tid=0 2024-06-10T00:38:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff3613c9b80 1 client.16410 shutdown 2024-06-10T00:38:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff3613c9b80 1 -- 172.21.15.62:0/3374066614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6ae850 msgr2=0x560abc6c0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/3374066614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6ae850 0x560abc6c0e00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff344022040 tx=0x7ff344010ae0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff3613c9b80 1 -- 172.21.15.62:0/3374066614 shutdown_connections 2024-06-10T00:38:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/3374066614 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x560abc7336f0 0x560abc735b10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/3374066614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560abc6ae850 0x560abc6c0e00 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/3374066614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560abc6b3550 0x560abc6be730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff32ffff700 0 client.16410 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:38:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff3613c9b80 1 --2- 172.21.15.62:0/3374066614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560abc6af170 0x560abc6be1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff3613c9b80 1 -- 172.21.15.62:0/3374066614 >> 172.21.15.62:0/3374066614 conn(0x560abc5d7080 msgr2=0x560abc6a9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff3613c9b80 1 -- 172.21.15.62:0/3374066614 shutdown_connections 2024-06-10T00:38:44.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.184+0000 7ff3613c9b80 1 -- 172.21.15.62:0/3374066614 wait complete. 2024-06-10T00:38:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.185+0000 7ff3613c9b80 20 client.16410 trim_cache size 0 max 16384 2024-06-10T00:38:44.195 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:38:44.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.185+0000 7ff356b46700 1 -- 172.21.15.62:0/1864733858 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff3400a2d90 msgr2=0x7ff3400a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.185+0000 7ff356b46700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff3400a2d90 0x7ff3400a5250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff350199a40 tx=0x7ff3440073d0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.185+0000 7ff356b46700 1 -- 172.21.15.62:0/1864733858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff350108520 msgr2=0x7ff350198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.185+0000 7ff356b46700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff350108520 0x7ff350198900 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff3480062a0 tx=0x7ff348004d70 comp rx=0 tx=0).stop 2024-06-10T00:38:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.185+0000 7ff356b46700 1 -- 172.21.15.62:0/1864733858 shutdown_connections 2024-06-10T00:38:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.185+0000 7ff356b46700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff3400a2d90 0x7ff3400a5250 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.185+0000 7ff356b46700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff350108520 0x7ff350198900 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.185+0000 7ff356b46700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff350102510 0x7ff35019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.185+0000 7ff356b46700 1 --2- 172.21.15.62:0/1864733858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff350102ed0 0x7ff35019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:44.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.185+0000 7ff356b46700 1 -- 172.21.15.62:0/1864733858 >> 172.21.15.62:0/1864733858 conn(0x7ff350079560 msgr2=0x7ff350100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:44.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.185+0000 7ff356b46700 1 -- 172.21.15.62:0/1864733858 shutdown_connections 2024-06-10T00:38:44.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:44.185+0000 7ff356b46700 1 -- 172.21.15.62:0/1864733858 wait complete. 2024-06-10T00:38:45.014 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T00:38:45.268 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:38:45.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:45 smithi062 ceph-mon[32390]: pgmap v180: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T00:38:45.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:45 smithi105 ceph-mon[36321]: pgmap v180: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T00:38:45.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:45 smithi138 ceph-mon[36549]: pgmap v180: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-10T00:38:47.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.057+0000 7f8d1a421700 1 -- 172.21.15.62:0/4099283833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d14102ed0 msgr2=0x7f8d1410f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:47.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.057+0000 7f8d1a421700 1 --2- 172.21.15.62:0/4099283833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d14102ed0 0x7f8d1410f350 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00b0a0 tx=0x7f8d0c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.057+0000 7f8d1a421700 1 -- 172.21.15.62:0/4099283833 shutdown_connections 2024-06-10T00:38:47.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.057+0000 7f8d1a421700 1 --2- 172.21.15.62:0/4099283833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d14102ed0 0x7f8d1410f350 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.057+0000 7f8d1a421700 1 --2- 172.21.15.62:0/4099283833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d14108510 0x7f8d141088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.057+0000 7f8d1a421700 1 --2- 172.21.15.62:0/4099283833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d14102510 0x7f8d14102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.057+0000 7f8d1a421700 1 -- 172.21.15.62:0/4099283833 >> 172.21.15.62:0/4099283833 conn(0x7f8d140795b0 msgr2=0x7f8d140799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:47.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.057+0000 7f8d1a421700 1 -- 172.21.15.62:0/4099283833 shutdown_connections 2024-06-10T00:38:47.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.058+0000 7f8d1a421700 1 -- 172.21.15.62:0/4099283833 wait complete. 2024-06-10T00:38:47.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.058+0000 7f8d1a421700 1 Processor -- start 2024-06-10T00:38:47.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.058+0000 7f8d1a421700 1 -- start start 2024-06-10T00:38:47.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.058+0000 7f8d1a421700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d14102510 0x7f8d1419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.058+0000 7f8d1a421700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d14102ed0 0x7f8d1419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d1a421700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d14108510 0x7f8d141989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d1a421700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14112740 con 0x7f8d14108510 2024-06-10T00:38:47.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d1a421700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141125c0 con 0x7f8d14102510 2024-06-10T00:38:47.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d1a421700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141128c0 con 0x7f8d14102ed0 2024-06-10T00:38:47.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d189be700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d14108510 0x7f8d141989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d189be700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d14108510 0x7f8d141989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50286/0 (socket says 172.21.15.62:50286) 2024-06-10T00:38:47.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d189be700 1 -- 172.21.15.62:0/2391477583 learned_addr learned my addr 172.21.15.62:0/2391477583 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:47.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d13fff700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d14102510 0x7f8d1419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d0bfff700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d14102ed0 0x7f8d1419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d189be700 1 -- 172.21.15.62:0/2391477583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d14102ed0 msgr2=0x7f8d1419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:47.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d189be700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d14102ed0 0x7f8d1419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d189be700 1 -- 172.21.15.62:0/2391477583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d14102510 msgr2=0x7f8d1419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:47.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d189be700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d14102510 0x7f8d1419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d189be700 1 -- 172.21.15.62:0/2391477583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d14199120 con 0x7f8d14108510 2024-06-10T00:38:47.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d13fff700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d14102510 0x7f8d1419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:38:47.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d189be700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d14108510 0x7f8d141989c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0062a0 tx=0x7f8d0c004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:47.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d11ffb700 1 -- 172.21.15.62:0/2391477583 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c021070 con 0x7f8d14108510 2024-06-10T00:38:47.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.059+0000 7f8d11ffb700 1 -- 172.21.15.62:0/2391477583 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d0c007c00 con 0x7f8d14108510 2024-06-10T00:38:47.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.060+0000 7f8d1a421700 1 -- 172.21.15.62:0/2391477583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d141993b0 con 0x7f8d14108510 2024-06-10T00:38:47.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.060+0000 7f8d11ffb700 1 -- 172.21.15.62:0/2391477583 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c01d450 con 0x7f8d14108510 2024-06-10T00:38:47.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.060+0000 7f8d1a421700 1 -- 172.21.15.62:0/2391477583 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d141a56d0 con 0x7f8d14108510 2024-06-10T00:38:47.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.062+0000 7f8d11ffb700 1 -- 172.21.15.62:0/2391477583 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8d0c004020 con 0x7f8d14108510 2024-06-10T00:38:47.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.062+0000 7f8d11ffb700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8d000a2d90 0x7f8d000a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.063+0000 7f8d13fff700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8d000a2d90 0x7f8d000a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.063+0000 7f8d11ffb700 1 -- 172.21.15.62:0/2391477583 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f8d0c0d0870 con 0x7f8d14108510 2024-06-10T00:38:47.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.063+0000 7f8d13fff700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8d000a2d90 0x7f8d000a5250 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8d04005ea0 tx=0x7f8d04005e30 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:47.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.063+0000 7f8d24ca4b80 1 Processor -- start 2024-06-10T00:38:47.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.063+0000 7f8d24ca4b80 1 -- start start 2024-06-10T00:38:47.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d24ca4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e2f60 0x5590fe7e3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d24ca4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e3880 0x5590fe7dd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d24ca4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7dd7a0 0x5590fe7ddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d24ca4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5590fe6cff60 con 0x5590fe7dd7a0 2024-06-10T00:38:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d24ca4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5590fe72e850 con 0x5590fe7e3880 2024-06-10T00:38:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d24ca4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5590fe696460 con 0x5590fe7e2f60 2024-06-10T00:38:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d189be700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7dd7a0 0x5590fe7ddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d189be700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7dd7a0 0x5590fe7ddbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50294/0 (socket says 172.21.15.62:50294) 2024-06-10T00:38:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d189be700 1 -- 172.21.15.62:0/1848099867 learned_addr learned my addr 172.21.15.62:0/1848099867 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d0bfff700 1 --2- 172.21.15.62:0/1848099867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e3880 0x5590fe7dd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d13fff700 1 --2- 172.21.15.62:0/1848099867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e2f60 0x5590fe7e3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d189be700 1 -- 172.21.15.62:0/1848099867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e2f60 msgr2=0x5590fe7e3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:47.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d189be700 1 --2- 172.21.15.62:0/1848099867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e2f60 0x5590fe7e3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d189be700 1 -- 172.21.15.62:0/1848099867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e3880 msgr2=0x5590fe7dd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:47.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d189be700 1 --2- 172.21.15.62:0/1848099867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e3880 0x5590fe7dd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d189be700 1 -- 172.21.15.62:0/1848099867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590fe67d8d0 con 0x5590fe7dd7a0 2024-06-10T00:38:47.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d189be700 1 --2- 172.21.15.62:0/1848099867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7dd7a0 0x5590fe7ddbc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c098ed0 tx=0x7f8d0c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bd109a202650d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:47.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.064+0000 7f8d0a7fc700 1 -- 172.21.15.62:0/1848099867 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c02a070 con 0x5590fe7dd7a0 2024-06-10T00:38:47.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.065+0000 7f8d0a7fc700 1 -- 172.21.15.62:0/1848099867 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d0c0a0e20 con 0x5590fe7dd7a0 2024-06-10T00:38:47.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.065+0000 7f8d0a7fc700 1 -- 172.21.15.62:0/1848099867 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c09cf30 con 0x5590fe7dd7a0 2024-06-10T00:38:47.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.065+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1848099867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7dd7a0 msgr2=0x5590fe7ddbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:47.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.065+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/1848099867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7dd7a0 0x5590fe7ddbc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c098ed0 tx=0x7f8d0c0099e0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.065+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1848099867 shutdown_connections 2024-06-10T00:38:47.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.065+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/1848099867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7dd7a0 0x5590fe7ddbc0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.065+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/1848099867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e2f60 0x5590fe7e3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.065+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/1848099867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e3880 0x5590fe7dd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.065+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1848099867 >> 172.21.15.62:0/1848099867 conn(0x5590fe70a080 msgr2=0x5590fe7d8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:47.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.065+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1848099867 shutdown_connections 2024-06-10T00:38:47.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.065+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1848099867 wait complete. 2024-06-10T00:38:47.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.066+0000 7f8d24ca4b80 1 -- start start 2024-06-10T00:38:47.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.066+0000 7f8d24ca4b80 1 Processor -- start 2024-06-10T00:38:47.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d24ca4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7dd7a0 0x5590fe7e9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d24ca4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e2f60 0x5590fe7e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d24ca4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e3880 0x5590fe7e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d24ca4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5590fe6cefc0 con 0x5590fe7dd7a0 2024-06-10T00:38:47.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d24ca4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5590fe72a810 con 0x5590fe7e2f60 2024-06-10T00:38:47.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d24ca4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5590fe7ee420 con 0x5590fe7e3880 2024-06-10T00:38:47.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d24ca4b80 10 client.0 _ll_register_callbacks cb 0x5590fe6c45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:47.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d13fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7dd7a0 0x5590fe7e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d13fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7dd7a0 0x5590fe7e9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50304/0 (socket says 172.21.15.62:50304) 2024-06-10T00:38:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d13fff700 1 -- 172.21.15.62:0/1974613633 learned_addr learned my addr 172.21.15.62:0/1974613633 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d0bfff700 1 --2- 172.21.15.62:0/1974613633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e2f60 0x5590fe7e6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d189be700 1 --2- 172.21.15.62:0/1974613633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e3880 0x5590fe7e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d13fff700 1 -- 172.21.15.62:0/1974613633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e3880 msgr2=0x5590fe7e6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d13fff700 1 --2- 172.21.15.62:0/1974613633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e3880 0x5590fe7e6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d13fff700 1 -- 172.21.15.62:0/1974613633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e2f60 msgr2=0x5590fe7e6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:47.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d13fff700 1 --2- 172.21.15.62:0/1974613633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e2f60 0x5590fe7e6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d13fff700 1 -- 172.21.15.62:0/1974613633 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590fe6f4750 con 0x5590fe7dd7a0 2024-06-10T00:38:47.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d0bfff700 1 --2- 172.21.15.62:0/1974613633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e2f60 0x5590fe7e6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:38:47.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8d13fff700 1 --2- 172.21.15.62:0/1974613633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7dd7a0 0x5590fe7e9700 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8d04010790 tx=0x7f8d04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd6dd09df395748 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:47.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.067+0000 7f8cf37fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:47.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.068+0000 7f8cf37fe700 1 -- 172.21.15.62:0/1974613633 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04004a30 con 0x5590fe7dd7a0 2024-06-10T00:38:47.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.068+0000 7f8cf37fe700 1 -- 172.21.15.62:0/1974613633 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f8d04004bd0 con 0x5590fe7dd7a0 2024-06-10T00:38:47.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.068+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1974613633 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d0c0a0e20 con 0x5590fe7dd7a0 2024-06-10T00:38:47.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.068+0000 7f8cf37fe700 1 -- 172.21.15.62:0/1974613633 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8d0402e050 con 0x5590fe7dd7a0 2024-06-10T00:38:47.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.068+0000 7f8d24ca4b80 10 client.16422 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:47.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.068+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1974613633 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5590fe7f8ce0 con 0x5590fe7dd7a0 2024-06-10T00:38:47.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.068+0000 7f8cf37fe700 1 -- 172.21.15.62:0/1974613633 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d0c0a0e20 con 0x5590fe7dd7a0 2024-06-10T00:38:47.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.068+0000 7f8cf37fe700 1 -- 172.21.15.62:0/1974613633 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f8d0401ea10 con 0x5590fe7dd7a0 2024-06-10T00:38:47.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.068+0000 7f8d24ca4b80 10 client.16422 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:47.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.068+0000 7f8d24ca4b80 10 client.16422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:38:47.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.068+0000 7f8d24ca4b80 20 client.16422 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:47.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.068+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/1974613633 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590fe7fb3f0 0x5590fe7fd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.068+0000 7f8d24ca4b80 4 client.16422 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:38:47.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.068+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1974613633 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590fe73bd50 con 0x5590fe7fb3f0 2024-06-10T00:38:47.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.069+0000 7f8d0bfff700 1 --2- 172.21.15.62:0/1974613633 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590fe7fb3f0 0x5590fe7fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.069+0000 7f8d0bfff700 1 --2- 172.21.15.62:0/1974613633 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590fe7fb3f0 0x5590fe7fd870 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76ec61ab5650c04e server_cookie=4b7f28c0e49dded4 in_seq=0 out_seq=0 2024-06-10T00:38:47.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.069+0000 7f8cf37fe700 10 client.16422 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:47.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.070+0000 7f8cf37fe700 1 -- 172.21.15.62:0/1974613633 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5590fe73bd50 con 0x5590fe7fb3f0 2024-06-10T00:38:47.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.070+0000 7f8cf37fe700 10 client.16422 handle_command_reply: tid=0 2024-06-10T00:38:47.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.070+0000 7f8d24ca4b80 1 client.16422 shutdown 2024-06-10T00:38:47.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.070+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1974613633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7dd7a0 msgr2=0x5590fe7e9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:47.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.070+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/1974613633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7dd7a0 0x5590fe7e9700 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8d04010790 tx=0x7f8d04004750 comp rx=0 tx=0).stop 2024-06-10T00:38:47.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.070+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1974613633 shutdown_connections 2024-06-10T00:38:47.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.070+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/1974613633 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590fe7fb3f0 0x5590fe7fd870 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.070+0000 7f8cf37fe700 0 client.16422 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:47.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.070+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/1974613633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7dd7a0 0x5590fe7e9700 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.070+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/1974613633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e3880 0x5590fe7e6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.070+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/1974613633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e2f60 0x5590fe7e6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.070+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1974613633 >> 172.21.15.62:0/1974613633 conn(0x5590fe70a080 msgr2=0x5590fe7d7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:47.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.070+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1974613633 shutdown_connections 2024-06-10T00:38:47.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.071+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1974613633 wait complete. 2024-06-10T00:38:47.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.071+0000 7f8d24ca4b80 20 client.16422 trim_cache size 0 max 16384 2024-06-10T00:38:47.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.086+0000 7f8d24ca4b80 1 Processor -- start 2024-06-10T00:38:47.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d24ca4b80 1 -- start start 2024-06-10T00:38:47.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d24ca4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7dd7a0 0x5590fe7dd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d24ca4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7e2f60 0x5590fe7f7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d24ca4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e3880 0x5590fe7e8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d24ca4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141ac270 con 0x5590fe7e2f60 2024-06-10T00:38:47.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d24ca4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141adf30 con 0x5590fe7dd7a0 2024-06-10T00:38:47.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d24ca4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141ac940 con 0x5590fe7e3880 2024-06-10T00:38:47.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d0bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7e2f60 0x5590fe7f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d0bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7e2f60 0x5590fe7f7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50312/0 (socket says 172.21.15.62:50312) 2024-06-10T00:38:47.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d0bfff700 1 -- 172.21.15.62:0/1755818868 learned_addr learned my addr 172.21.15.62:0/1755818868 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:47.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d13fff700 1 --2- 172.21.15.62:0/1755818868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7dd7a0 0x5590fe7dd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d189be700 1 --2- 172.21.15.62:0/1755818868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e3880 0x5590fe7e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d0bfff700 1 -- 172.21.15.62:0/1755818868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e3880 msgr2=0x5590fe7e8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:47.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d0bfff700 1 --2- 172.21.15.62:0/1755818868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e3880 0x5590fe7e8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d0bfff700 1 -- 172.21.15.62:0/1755818868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7dd7a0 msgr2=0x5590fe7dd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:47.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d0bfff700 1 --2- 172.21.15.62:0/1755818868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7dd7a0 0x5590fe7dd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d0bfff700 1 -- 172.21.15.62:0/1755818868 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590fe7e3590 con 0x5590fe7e2f60 2024-06-10T00:38:47.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d189be700 1 --2- 172.21.15.62:0/1755818868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e3880 0x5590fe7e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:38:47.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.087+0000 7f8d0bfff700 1 --2- 172.21.15.62:0/1755818868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7e2f60 0x5590fe7f7500 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8d14079bb0 tx=0x7f8cfc00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff88fa66f5c1c13 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:47.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.088+0000 7f8d09ffb700 1 -- 172.21.15.62:0/1755818868 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc003be0 con 0x5590fe7e2f60 2024-06-10T00:38:47.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.088+0000 7f8d09ffb700 1 -- 172.21.15.62:0/1755818868 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cfc003d80 con 0x5590fe7e2f60 2024-06-10T00:38:47.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.088+0000 7f8d09ffb700 1 -- 172.21.15.62:0/1755818868 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc011440 con 0x5590fe7e2f60 2024-06-10T00:38:47.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.088+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1755818868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7e2f60 msgr2=0x5590fe7f7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:47.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.088+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/1755818868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7e2f60 0x5590fe7f7500 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8d14079bb0 tx=0x7f8cfc00b410 comp rx=0 tx=0).stop 2024-06-10T00:38:47.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.088+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1755818868 shutdown_connections 2024-06-10T00:38:47.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.088+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/1755818868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7e3880 0x5590fe7e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.088+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/1755818868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7dd7a0 0x5590fe7dd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.088+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/1755818868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7e2f60 0x5590fe7f7500 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.088+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1755818868 >> 172.21.15.62:0/1755818868 conn(0x5590fe70a080 msgr2=0x5590fe7e8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:47.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.088+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1755818868 shutdown_connections 2024-06-10T00:38:47.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.088+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/1755818868 wait complete. 2024-06-10T00:38:47.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.089+0000 7f8d24ca4b80 1 -- start start 2024-06-10T00:38:47.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.089+0000 7f8d24ca4b80 1 Processor -- start 2024-06-10T00:38:47.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.089+0000 7f8d24ca4b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7dd7a0 0x5590fe7f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.089+0000 7f8d24ca4b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7e2f60 0x5590fe802c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.089+0000 7f8d24ca4b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e3880 0x5590fe803380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.089+0000 7f8d24ca4b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141ad7a0 con 0x5590fe7e2f60 2024-06-10T00:38:47.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.089+0000 7f8d24ca4b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141ad010 con 0x5590fe7e3880 2024-06-10T00:38:47.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.089+0000 7f8d24ca4b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141aa210 con 0x5590fe7dd7a0 2024-06-10T00:38:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.089+0000 7f8d24ca4b80 10 client.0 _ll_register_callbacks cb 0x5590fe6c45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.089+0000 7f8d0bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7e2f60 0x5590fe802c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.089+0000 7f8d0bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7e2f60 0x5590fe802c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50320/0 (socket says 172.21.15.62:50320) 2024-06-10T00:38:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.089+0000 7f8d0bfff700 1 -- 172.21.15.62:0/439037033 learned_addr learned my addr 172.21.15.62:0/439037033 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.089+0000 7f8d189be700 1 --2- 172.21.15.62:0/439037033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e3880 0x5590fe803380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8d13fff700 1 --2- 172.21.15.62:0/439037033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7dd7a0 0x5590fe7f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8d0bfff700 1 -- 172.21.15.62:0/439037033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7dd7a0 msgr2=0x5590fe7f3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8d0bfff700 1 --2- 172.21.15.62:0/439037033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7dd7a0 0x5590fe7f3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8d0bfff700 1 -- 172.21.15.62:0/439037033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e3880 msgr2=0x5590fe803380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8d0bfff700 1 --2- 172.21.15.62:0/439037033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e3880 0x5590fe803380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8d0bfff700 1 -- 172.21.15.62:0/439037033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590fe865110 con 0x5590fe7e2f60 2024-06-10T00:38:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8d0bfff700 1 --2- 172.21.15.62:0/439037033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7e2f60 0x5590fe802c40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc000c00 tx=0x7f8cfc0050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c29c6300498e3e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8cf37fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8cf37fe700 1 -- 172.21.15.62:0/439037033 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc005420 con 0x5590fe7e2f60 2024-06-10T00:38:47.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8cf37fe700 1 -- 172.21.15.62:0/439037033 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f8cfc0055c0 con 0x5590fe7e2f60 2024-06-10T00:38:47.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/439037033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cfc003d80 con 0x5590fe7e2f60 2024-06-10T00:38:47.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8cf37fe700 1 -- 172.21.15.62:0/439037033 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8cfc0047c0 con 0x5590fe7e2f60 2024-06-10T00:38:47.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8d24ca4b80 10 client.16428 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:47.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/439037033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5590fe866260 con 0x5590fe7e2f60 2024-06-10T00:38:47.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.090+0000 7f8cf37fe700 1 -- 172.21.15.62:0/439037033 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cfc003d80 con 0x5590fe7e2f60 2024-06-10T00:38:47.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.091+0000 7f8cf37fe700 1 -- 172.21.15.62:0/439037033 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f8cfc011ae0 con 0x5590fe7e2f60 2024-06-10T00:38:47.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.091+0000 7f8d24ca4b80 10 client.16428 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:47.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.091+0000 7f8d24ca4b80 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:38:47.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.091+0000 7f8d24ca4b80 20 client.16428 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:47.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.091+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/439037033 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590fe866950 0x5590fe868d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:47.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.091+0000 7f8d24ca4b80 4 client.16428 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:38:47.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.091+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/439037033 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590fe7362d0 con 0x5590fe866950 2024-06-10T00:38:47.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.091+0000 7f8d13fff700 1 --2- 172.21.15.62:0/439037033 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590fe866950 0x5590fe868d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:47.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.091+0000 7f8cf37fe700 10 client.16428 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:47.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.091+0000 7f8d13fff700 1 --2- 172.21.15.62:0/439037033 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590fe866950 0x5590fe868d70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fbf96cf5792953 server_cookie=ea9d4595cc156081 in_seq=0 out_seq=0 2024-06-10T00:38:47.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8cf37fe700 1 -- 172.21.15.62:0/439037033 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590fe7362d0 con 0x5590fe866950 2024-06-10T00:38:47.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8cf37fe700 10 client.16428 handle_command_reply: tid=0 2024-06-10T00:38:47.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8d24ca4b80 1 client.16428 shutdown 2024-06-10T00:38:47.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/439037033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7e2f60 msgr2=0x5590fe802c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:47.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/439037033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7e2f60 0x5590fe802c40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc000c00 tx=0x7f8cfc0050d0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/439037033 shutdown_connections 2024-06-10T00:38:47.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/439037033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590fe7e3880 0x5590fe803380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/439037033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590fe7dd7a0 0x5590fe7f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/439037033 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5590fe866950 0x5590fe868d70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8d24ca4b80 1 --2- 172.21.15.62:0/439037033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590fe7e2f60 0x5590fe802c40 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/439037033 >> 172.21.15.62:0/439037033 conn(0x5590fe70a080 msgr2=0x5590fe7e23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:47.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8cf37fe700 0 client.16428 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:47.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/439037033 shutdown_connections 2024-06-10T00:38:47.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8d24ca4b80 1 -- 172.21.15.62:0/439037033 wait complete. 2024-06-10T00:38:47.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.092+0000 7f8d24ca4b80 20 client.16428 trim_cache size 0 max 16384 2024-06-10T00:38:47.103 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:38:47.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.093+0000 7f8d1a421700 1 -- 172.21.15.62:0/2391477583 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8d000a2d90 msgr2=0x7f8d000a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:47.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.093+0000 7f8d1a421700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8d000a2d90 0x7f8d000a5250 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8d04005ea0 tx=0x7f8d04005e30 comp rx=0 tx=0).stop 2024-06-10T00:38:47.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.093+0000 7f8d1a421700 1 -- 172.21.15.62:0/2391477583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d14108510 msgr2=0x7f8d141989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:47.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.093+0000 7f8d1a421700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d14108510 0x7f8d141989c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0062a0 tx=0x7f8d0c004d70 comp rx=0 tx=0).stop 2024-06-10T00:38:47.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.093+0000 7f8d1a421700 1 -- 172.21.15.62:0/2391477583 shutdown_connections 2024-06-10T00:38:47.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.093+0000 7f8d1a421700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8d000a2d90 0x7f8d000a5250 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.093+0000 7f8d1a421700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d14108510 0x7f8d141989c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.093+0000 7f8d1a421700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d14102510 0x7f8d1419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.093+0000 7f8d1a421700 1 --2- 172.21.15.62:0/2391477583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d14102ed0 0x7f8d1419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:47.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.093+0000 7f8d1a421700 1 -- 172.21.15.62:0/2391477583 >> 172.21.15.62:0/2391477583 conn(0x7f8d140795b0 msgr2=0x7f8d141010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:47.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.093+0000 7f8d1a421700 1 -- 172.21.15.62:0/2391477583 shutdown_connections 2024-06-10T00:38:47.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:47.093+0000 7f8d1a421700 1 -- 172.21.15.62:0/2391477583 wait complete. 2024-06-10T00:38:47.357 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:47 smithi062 ceph-mon[32390]: pgmap v181: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 11 op/s 2024-06-10T00:38:47.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:47 smithi105 ceph-mon[36321]: pgmap v181: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 11 op/s 2024-06-10T00:38:47.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:47 smithi138 ceph-mon[36549]: pgmap v181: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 11 op/s 2024-06-10T00:38:48.744 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T00:38:48.744 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T00:38:49.010 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:38:49.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:49 smithi105 ceph-mon[36321]: pgmap v182: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:38:49.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:49 smithi062 ceph-mon[32390]: pgmap v182: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:38:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:49 smithi138 ceph-mon[36549]: pgmap v182: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:38:50.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.811+0000 7f8ef824d700 1 -- 172.21.15.62:0/2319648388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef0100a80 msgr2=0x7f8ef0100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:50.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.811+0000 7f8ef824d700 1 --2- 172.21.15.62:0/2319648388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef0100a80 0x7f8ef0100f00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009990 tx=0x7f8ee0009ca0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.812+0000 7f8ef824d700 1 -- 172.21.15.62:0/2319648388 shutdown_connections 2024-06-10T00:38:50.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.812+0000 7f8ef824d700 1 --2- 172.21.15.62:0/2319648388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ef0101440 0x7f8ef010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.812+0000 7f8ef824d700 1 --2- 172.21.15.62:0/2319648388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef0108510 0x7f8ef01088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.812+0000 7f8ef824d700 1 --2- 172.21.15.62:0/2319648388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef0100a80 0x7f8ef0100f00 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.812+0000 7f8ef824d700 1 -- 172.21.15.62:0/2319648388 >> 172.21.15.62:0/2319648388 conn(0x7f8ef0077a00 msgr2=0x7f8ef00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:50.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.812+0000 7f8ef824d700 1 -- 172.21.15.62:0/2319648388 shutdown_connections 2024-06-10T00:38:50.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.812+0000 7f8ef824d700 1 -- 172.21.15.62:0/2319648388 wait complete. 2024-06-10T00:38:50.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.812+0000 7f8ef824d700 1 Processor -- start 2024-06-10T00:38:50.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.812+0000 7f8ef824d700 1 -- start start 2024-06-10T00:38:50.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef824d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ef0100a80 0x7f8ef0101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef824d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef0101440 0x7f8ef01021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef824d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef0108510 0x7f8ef01026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef824d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01127f0 con 0x7f8ef0108510 2024-06-10T00:38:50.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef824d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0112670 con 0x7f8ef0100a80 2024-06-10T00:38:50.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef824d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0112970 con 0x7f8ef0101440 2024-06-10T00:38:50.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef57e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef0101440 0x7f8ef01021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef67ea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef0108510 0x7f8ef01026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef57e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef0101440 0x7f8ef01021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:36160/0 (socket says 172.21.15.62:36160) 2024-06-10T00:38:50.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef57e8700 1 -- 172.21.15.62:0/177637282 learned_addr learned my addr 172.21.15.62:0/177637282 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:50.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef5fe9700 1 --2- 172.21.15.62:0/177637282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ef0100a80 0x7f8ef0101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef67ea700 1 -- 172.21.15.62:0/177637282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef0101440 msgr2=0x7f8ef01021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:50.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef67ea700 1 --2- 172.21.15.62:0/177637282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef0101440 0x7f8ef01021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef67ea700 1 -- 172.21.15.62:0/177637282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ef0100a80 msgr2=0x7f8ef0101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:50.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef67ea700 1 --2- 172.21.15.62:0/177637282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ef0100a80 0x7f8ef0101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef67ea700 1 -- 172.21.15.62:0/177637282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef01a7130 con 0x7f8ef0108510 2024-06-10T00:38:50.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.813+0000 7f8ef57e8700 1 --2- 172.21.15.62:0/177637282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef0101440 0x7f8ef01021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:38:50.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.814+0000 7f8ef67ea700 1 --2- 172.21.15.62:0/177637282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef0108510 0x7f8ef01026e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800be20 tx=0x7f8ee8009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:50.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.814+0000 7f8ee6ffd700 1 -- 172.21.15.62:0/177637282 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8019070 con 0x7f8ef0108510 2024-06-10T00:38:50.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.814+0000 7f8ee6ffd700 1 -- 172.21.15.62:0/177637282 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee800cb90 con 0x7f8ef0108510 2024-06-10T00:38:50.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.814+0000 7f8ee6ffd700 1 -- 172.21.15.62:0/177637282 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80125a0 con 0x7f8ef0108510 2024-06-10T00:38:50.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.814+0000 7f8ef824d700 1 -- 172.21.15.62:0/177637282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef01a7420 con 0x7f8ef0108510 2024-06-10T00:38:50.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.814+0000 7f8ef824d700 1 -- 172.21.15.62:0/177637282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef01a7910 con 0x7f8ef0108510 2024-06-10T00:38:50.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.816+0000 7f8ee6ffd700 1 -- 172.21.15.62:0/177637282 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8ee801c070 con 0x7f8ef0108510 2024-06-10T00:38:50.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.817+0000 7f8ee6ffd700 1 --2- 172.21.15.62:0/177637282 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8edc0a2d90 0x7f8edc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.817+0000 7f8ef5fe9700 1 --2- 172.21.15.62:0/177637282 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8edc0a2d90 0x7f8edc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.817+0000 7f8ee6ffd700 1 -- 172.21.15.62:0/177637282 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f8ee80cec90 con 0x7f8ef0108510 2024-06-10T00:38:50.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.817+0000 7f8ef5fe9700 1 --2- 172.21.15.62:0/177637282 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8edc0a2d90 0x7f8edc0a5250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00ac50 tx=0x7f8eec00a380 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:50.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.817+0000 7f8f02ad0b80 1 Processor -- start 2024-06-10T00:38:50.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8f02ad0b80 1 -- start start 2024-06-10T00:38:50.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8f02ad0b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fae01fe0 0x5642fae023c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8f02ad0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae02900 0x5642fadfc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8f02ad0b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fadfc820 0x5642fadfcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8f02ad0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5642faceefe0 con 0x5642fae02900 2024-06-10T00:38:50.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8f02ad0b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5642fad4d8d0 con 0x5642fadfc820 2024-06-10T00:38:50.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8f02ad0b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642facb54e0 con 0x5642fae01fe0 2024-06-10T00:38:50.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8ef57e8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae02900 0x5642fadfc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8ef5fe9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fae01fe0 0x5642fae023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8ef57e8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae02900 0x5642fadfc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50656/0 (socket says 172.21.15.62:50656) 2024-06-10T00:38:50.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8ef57e8700 1 -- 172.21.15.62:0/2045166142 learned_addr learned my addr 172.21.15.62:0/2045166142 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:50.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8ef67ea700 1 --2- 172.21.15.62:0/2045166142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fadfc820 0x5642fadfcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8ef57e8700 1 -- 172.21.15.62:0/2045166142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fae01fe0 msgr2=0x5642fae023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:50.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8ef57e8700 1 --2- 172.21.15.62:0/2045166142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fae01fe0 0x5642fae023c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8ef57e8700 1 -- 172.21.15.62:0/2045166142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fadfc820 msgr2=0x5642fadfcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:50.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8ef57e8700 1 --2- 172.21.15.62:0/2045166142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fadfc820 0x5642fadfcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8ef57e8700 1 -- 172.21.15.62:0/2045166142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642fac9c950 con 0x5642fae02900 2024-06-10T00:38:50.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8ef5fe9700 1 --2- 172.21.15.62:0/2045166142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fae01fe0 0x5642fae023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:38:50.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.818+0000 7f8ef57e8700 1 --2- 172.21.15.62:0/2045166142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae02900 0x5642fadfc2e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0003730 tx=0x7f8ee0003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd838bbd388b7eed server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:50.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.819+0000 7f8edbfff700 1 -- 172.21.15.62:0/2045166142 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0028070 con 0x5642fae02900 2024-06-10T00:38:50.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.819+0000 7f8edbfff700 1 -- 172.21.15.62:0/2045166142 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee0003e40 con 0x5642fae02900 2024-06-10T00:38:50.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.819+0000 7f8edbfff700 1 -- 172.21.15.62:0/2045166142 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00236c0 con 0x5642fae02900 2024-06-10T00:38:50.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.819+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/2045166142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae02900 msgr2=0x5642fadfc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:50.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.819+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/2045166142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae02900 0x5642fadfc2e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0003730 tx=0x7f8ee0003760 comp rx=0 tx=0).stop 2024-06-10T00:38:50.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.819+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/2045166142 shutdown_connections 2024-06-10T00:38:50.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.819+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/2045166142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fadfc820 0x5642fadfcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.819+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/2045166142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fae01fe0 0x5642fae023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.819+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/2045166142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae02900 0x5642fadfc2e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.820+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/2045166142 >> 172.21.15.62:0/2045166142 conn(0x5642fad29100 msgr2=0x5642fadf72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:50.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.820+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/2045166142 shutdown_connections 2024-06-10T00:38:50.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.820+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/2045166142 wait complete. 2024-06-10T00:38:50.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.820+0000 7f8f02ad0b80 1 -- start start 2024-06-10T00:38:50.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.820+0000 7f8f02ad0b80 1 Processor -- start 2024-06-10T00:38:50.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.820+0000 7f8f02ad0b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fadfc820 0x5642fae08640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.820+0000 7f8f02ad0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae01fe0 0x5642fae05940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8f02ad0b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fae02900 0x5642fae05e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8f02ad0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5642facee040 con 0x5642fae01fe0 2024-06-10T00:38:50.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8f02ad0b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5642fad49890 con 0x5642fadfc820 2024-06-10T00:38:50.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8f02ad0b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642fae09280 con 0x5642fae02900 2024-06-10T00:38:50.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8f02ad0b80 10 client.0 _ll_register_callbacks cb 0x5642face3630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:50.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8ef57e8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae01fe0 0x5642fae05940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8ef57e8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae01fe0 0x5642fae05940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50662/0 (socket says 172.21.15.62:50662) 2024-06-10T00:38:50.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8ef57e8700 1 -- 172.21.15.62:0/3009627707 learned_addr learned my addr 172.21.15.62:0/3009627707 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:50.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8ef5fe9700 1 --2- 172.21.15.62:0/3009627707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fadfc820 0x5642fae08640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8ef67ea700 1 --2- 172.21.15.62:0/3009627707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fae02900 0x5642fae05e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8ef57e8700 1 -- 172.21.15.62:0/3009627707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fae02900 msgr2=0x5642fae05e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:50.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8ef57e8700 1 --2- 172.21.15.62:0/3009627707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fae02900 0x5642fae05e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8ef57e8700 1 -- 172.21.15.62:0/3009627707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fadfc820 msgr2=0x5642fae08640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:50.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8ef57e8700 1 --2- 172.21.15.62:0/3009627707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fadfc820 0x5642fae08640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8ef57e8700 1 -- 172.21.15.62:0/3009627707 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642fad137d0 con 0x5642fae01fe0 2024-06-10T00:38:50.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8ef57e8700 1 --2- 172.21.15.62:0/3009627707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae01fe0 0x5642fae05940 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005520 tx=0x7f8ee00052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf542b8ce0a08f34 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:50.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8ed97fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:50.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8ed97fa700 1 -- 172.21.15.62:0/3009627707 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0028040 con 0x5642fae01fe0 2024-06-10T00:38:50.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/3009627707 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5642fad5add0 con 0x5642fae01fe0 2024-06-10T00:38:50.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.821+0000 7f8ed97fa700 1 -- 172.21.15.62:0/3009627707 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f8ee0003b60 con 0x5642fae01fe0 2024-06-10T00:38:50.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.822+0000 7f8ed97fa700 1 -- 172.21.15.62:0/3009627707 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8ee003a050 con 0x5642fae01fe0 2024-06-10T00:38:50.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.822+0000 7f8f02ad0b80 10 client.16440 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:50.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.822+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/3009627707 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5642fae139e0 con 0x5642fae01fe0 2024-06-10T00:38:50.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.822+0000 7f8ed97fa700 1 -- 172.21.15.62:0/3009627707 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5642fad5add0 con 0x5642fae01fe0 2024-06-10T00:38:50.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.822+0000 7f8ed97fa700 1 -- 172.21.15.62:0/3009627707 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f8ee002b070 con 0x5642fae01fe0 2024-06-10T00:38:50.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.822+0000 7f8f02ad0b80 10 client.16440 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:50.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.822+0000 7f8f02ad0b80 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:38:50.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.822+0000 7f8f02ad0b80 20 client.16440 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:50.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.822+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/3009627707 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5642fae15eb0 0x5642fae182a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.822+0000 7f8f02ad0b80 4 client.16440 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:38:50.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.822+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/3009627707 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642fad55350 con 0x5642fae15eb0 2024-06-10T00:38:50.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.822+0000 7f8ef5fe9700 1 --2- 172.21.15.62:0/3009627707 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5642fae15eb0 0x5642fae182a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.823+0000 7f8ef5fe9700 1 --2- 172.21.15.62:0/3009627707 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5642fae15eb0 0x5642fae182a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9820aabeb108951 server_cookie=8c48d33c5127606e in_seq=0 out_seq=0 2024-06-10T00:38:50.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.823+0000 7f8ed97fa700 10 client.16440 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:50.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.823+0000 7f8ed97fa700 1 -- 172.21.15.62:0/3009627707 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5642fad55350 con 0x5642fae15eb0 2024-06-10T00:38:50.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.823+0000 7f8ed97fa700 10 client.16440 handle_command_reply: tid=0 2024-06-10T00:38:50.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.823+0000 7f8f02ad0b80 1 client.16440 shutdown 2024-06-10T00:38:50.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.824+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/3009627707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae01fe0 msgr2=0x5642fae05940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:50.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.824+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/3009627707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae01fe0 0x5642fae05940 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005520 tx=0x7f8ee00052c0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.824+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/3009627707 shutdown_connections 2024-06-10T00:38:50.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.824+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/3009627707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fae02900 0x5642fae05e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.824+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/3009627707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fadfc820 0x5642fae08640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.824+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/3009627707 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5642fae15eb0 0x5642fae182a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.824+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/3009627707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae01fe0 0x5642fae05940 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.824+0000 7f8ed97fa700 0 client.16440 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:50.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.824+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/3009627707 >> 172.21.15.62:0/3009627707 conn(0x5642fad29100 msgr2=0x5642fadf6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:50.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.824+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/3009627707 shutdown_connections 2024-06-10T00:38:50.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.824+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/3009627707 wait complete. 2024-06-10T00:38:50.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.824+0000 7f8f02ad0b80 20 client.16440 trim_cache size 0 max 16384 2024-06-10T00:38:50.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8f02ad0b80 1 Processor -- start 2024-06-10T00:38:50.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8f02ad0b80 1 -- start start 2024-06-10T00:38:50.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8f02ad0b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fadfc820 0x5642fae034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8f02ad0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae01fe0 0x5642fadf6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8f02ad0b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fae02900 0x5642fadf7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8f02ad0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01b06d0 con 0x5642fae01fe0 2024-06-10T00:38:50.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8f02ad0b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01b2390 con 0x5642fae02900 2024-06-10T00:38:50.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8f02ad0b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01b0da0 con 0x5642fadfc820 2024-06-10T00:38:50.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8ef57e8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae01fe0 0x5642fadf6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8ef57e8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae01fe0 0x5642fadf6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50672/0 (socket says 172.21.15.62:50672) 2024-06-10T00:38:50.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8ef57e8700 1 -- 172.21.15.62:0/1505820106 learned_addr learned my addr 172.21.15.62:0/1505820106 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:50.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8ef67ea700 1 --2- 172.21.15.62:0/1505820106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fae02900 0x5642fadf7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8ef5fe9700 1 --2- 172.21.15.62:0/1505820106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fadfc820 0x5642fae034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8ef57e8700 1 -- 172.21.15.62:0/1505820106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fadfc820 msgr2=0x5642fae034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:50.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8ef57e8700 1 --2- 172.21.15.62:0/1505820106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fadfc820 0x5642fae034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8ef57e8700 1 -- 172.21.15.62:0/1505820106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fae02900 msgr2=0x5642fadf7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:50.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8ef57e8700 1 --2- 172.21.15.62:0/1505820106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fae02900 0x5642fadf7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8ef57e8700 1 -- 172.21.15.62:0/1505820106 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642fadf8880 con 0x5642fae01fe0 2024-06-10T00:38:50.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8ef67ea700 1 --2- 172.21.15.62:0/1505820106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fae02900 0x5642fadf7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:38:50.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.845+0000 7f8ef5fe9700 1 --2- 172.21.15.62:0/1505820106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fadfc820 0x5642fae034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:38:50.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.846+0000 7f8ef57e8700 1 --2- 172.21.15.62:0/1505820106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae01fe0 0x5642fadf6f00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005550 tx=0x7f8ee000b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72262bc507703347 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:50.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.846+0000 7f8edb7fe700 1 -- 172.21.15.62:0/1505820106 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0028040 con 0x5642fae01fe0 2024-06-10T00:38:50.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.846+0000 7f8edb7fe700 1 -- 172.21.15.62:0/1505820106 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee0031070 con 0x5642fae01fe0 2024-06-10T00:38:50.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.846+0000 7f8edb7fe700 1 -- 172.21.15.62:0/1505820106 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee000f460 con 0x5642fae01fe0 2024-06-10T00:38:50.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.846+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/1505820106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae01fe0 msgr2=0x5642fadf6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:50.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.846+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/1505820106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae01fe0 0x5642fadf6f00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005550 tx=0x7f8ee000b5c0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/1505820106 shutdown_connections 2024-06-10T00:38:50.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/1505820106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fae02900 0x5642fadf7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/1505820106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fadfc820 0x5642fae034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/1505820106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae01fe0 0x5642fadf6f00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/1505820106 >> 172.21.15.62:0/1505820106 conn(0x5642fad29100 msgr2=0x5642fae02da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:50.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/1505820106 shutdown_connections 2024-06-10T00:38:50.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/1505820106 wait complete. 2024-06-10T00:38:50.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 -- start start 2024-06-10T00:38:50.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 Processor -- start 2024-06-10T00:38:50.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fadfc820 0x5642fae194a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fae01fe0 0x5642fae199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae02900 0x5642fae7f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01b1c00 con 0x5642fae02900 2024-06-10T00:38:50.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01b1470 con 0x5642fae01fe0 2024-06-10T00:38:50.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.847+0000 7f8f02ad0b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01ae670 con 0x5642fadfc820 2024-06-10T00:38:50.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8f02ad0b80 10 client.0 _ll_register_callbacks cb 0x5642face3630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:50.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ef67ea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae02900 0x5642fae7f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ef67ea700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae02900 0x5642fae7f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50678/0 (socket says 172.21.15.62:50678) 2024-06-10T00:38:50.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ef67ea700 1 -- 172.21.15.62:0/2401777248 learned_addr learned my addr 172.21.15.62:0/2401777248 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:50.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ef57e8700 1 --2- 172.21.15.62:0/2401777248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fae01fe0 0x5642fae199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ef5fe9700 1 --2- 172.21.15.62:0/2401777248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fadfc820 0x5642fae194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ef67ea700 1 -- 172.21.15.62:0/2401777248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fadfc820 msgr2=0x5642fae194a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:50.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ef67ea700 1 --2- 172.21.15.62:0/2401777248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fadfc820 0x5642fae194a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ef67ea700 1 -- 172.21.15.62:0/2401777248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fae01fe0 msgr2=0x5642fae199e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:50.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ef67ea700 1 --2- 172.21.15.62:0/2401777248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fae01fe0 0x5642fae199e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ef67ea700 1 -- 172.21.15.62:0/2401777248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642fae7fd30 con 0x5642fae02900 2024-06-10T00:38:50.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ef67ea700 1 --2- 172.21.15.62:0/2401777248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae02900 0x5642fae7f7f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800bfd0 tx=0x7f8ee809af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7392eb6f9729327 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:50.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ed97fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:50.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ed97fa700 1 -- 172.21.15.62:0/2401777248 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80b3070 con 0x5642fae02900 2024-06-10T00:38:50.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ed97fa700 1 -- 172.21.15.62:0/2401777248 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f8ee809b370 con 0x5642fae02900 2024-06-10T00:38:50.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/2401777248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee0031070 con 0x5642fae02900 2024-06-10T00:38:50.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8ed97fa700 1 -- 172.21.15.62:0/2401777248 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8ee80a36f0 con 0x5642fae02900 2024-06-10T00:38:50.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8f02ad0b80 10 client.16446 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:50.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.848+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/2401777248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5642fae80f00 con 0x5642fae02900 2024-06-10T00:38:50.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.849+0000 7f8ed97fa700 1 -- 172.21.15.62:0/2401777248 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee0031070 con 0x5642fae02900 2024-06-10T00:38:50.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.849+0000 7f8ed97fa700 1 -- 172.21.15.62:0/2401777248 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f8ee809a360 con 0x5642fae02900 2024-06-10T00:38:50.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.849+0000 7f8f02ad0b80 10 client.16446 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:50.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.849+0000 7f8f02ad0b80 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:38:50.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.849+0000 7f8f02ad0b80 20 client.16446 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:50.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.849+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/2401777248 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5642fae816a0 0x5642fae83ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:50.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.849+0000 7f8f02ad0b80 4 client.16446 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T00:38:50.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.849+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/2401777248 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5642fad08fd0 con 0x5642fae816a0 2024-06-10T00:38:50.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.849+0000 7f8ef5fe9700 1 --2- 172.21.15.62:0/2401777248 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5642fae816a0 0x5642fae83ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:50.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.850+0000 7f8ed97fa700 10 client.16446 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:50.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.850+0000 7f8ef5fe9700 1 --2- 172.21.15.62:0/2401777248 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5642fae816a0 0x5642fae83ac0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=674aa1ae8c50ba23 server_cookie=5fb33b9c61527e8 in_seq=0 out_seq=0 2024-06-10T00:38:50.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8ed97fa700 1 -- 172.21.15.62:0/2401777248 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5642fad08fd0 con 0x5642fae816a0 2024-06-10T00:38:50.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8ed97fa700 10 client.16446 handle_command_reply: tid=0 2024-06-10T00:38:50.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8f02ad0b80 1 client.16446 shutdown 2024-06-10T00:38:50.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/2401777248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae02900 msgr2=0x5642fae7f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:50.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/2401777248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae02900 0x5642fae7f7f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800bfd0 tx=0x7f8ee809af10 comp rx=0 tx=0).stop 2024-06-10T00:38:50.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/2401777248 shutdown_connections 2024-06-10T00:38:50.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/2401777248 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5642fae816a0 0x5642fae83ac0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/2401777248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5642fae02900 0x5642fae7f7f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/2401777248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642fadfc820 0x5642fae194a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8ed97fa700 0 client.16446 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:50.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8f02ad0b80 1 --2- 172.21.15.62:0/2401777248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642fae01fe0 0x5642fae199e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/2401777248 >> 172.21.15.62:0/2401777248 conn(0x5642fad29100 msgr2=0x5642fae07170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:50.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/2401777248 shutdown_connections 2024-06-10T00:38:50.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8f02ad0b80 1 -- 172.21.15.62:0/2401777248 wait complete. 2024-06-10T00:38:50.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.851+0000 7f8f02ad0b80 20 client.16446 trim_cache size 0 max 16384 2024-06-10T00:38:50.862 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:38:50.862 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-06-10T00:38:50.862 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "d0752cc9-2752-42bb-8765-ce6fe8795a1e", 2024-06-10T00:38:50.863 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-06-10T00:38:50.863 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:38:50.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.852+0000 7f8ef824d700 1 -- 172.21.15.62:0/177637282 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8edc0a2d90 msgr2=0x7f8edc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:50.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.852+0000 7f8ef824d700 1 --2- 172.21.15.62:0/177637282 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8edc0a2d90 0x7f8edc0a5250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00ac50 tx=0x7f8eec00a380 comp rx=0 tx=0).stop 2024-06-10T00:38:50.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.852+0000 7f8ef824d700 1 -- 172.21.15.62:0/177637282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef0108510 msgr2=0x7f8ef01026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:50.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.852+0000 7f8ef824d700 1 --2- 172.21.15.62:0/177637282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef0108510 0x7f8ef01026e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800be20 tx=0x7f8ee8009ed0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.852+0000 7f8ef824d700 1 -- 172.21.15.62:0/177637282 shutdown_connections 2024-06-10T00:38:50.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.852+0000 7f8ef824d700 1 --2- 172.21.15.62:0/177637282 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8edc0a2d90 0x7f8edc0a5250 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.852+0000 7f8ef824d700 1 --2- 172.21.15.62:0/177637282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ef0108510 0x7f8ef01026e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.852+0000 7f8ef824d700 1 --2- 172.21.15.62:0/177637282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ef0100a80 0x7f8ef0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.852+0000 7f8ef824d700 1 --2- 172.21.15.62:0/177637282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ef0101440 0x7f8ef01021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:50.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.852+0000 7f8ef824d700 1 -- 172.21.15.62:0/177637282 >> 172.21.15.62:0/177637282 conn(0x7f8ef0077a00 msgr2=0x7f8ef00fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:50.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.852+0000 7f8ef824d700 1 -- 172.21.15.62:0/177637282 shutdown_connections 2024-06-10T00:38:50.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:50.852+0000 7f8ef824d700 1 -- 172.21.15.62:0/177637282 wait complete. 2024-06-10T00:38:51.639 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:38:51.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:51 smithi105 ceph-mon[36321]: pgmap v183: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T00:38:51.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:51 smithi105 ceph-mon[36321]: scrub queued for path: / 2024-06-10T00:38:51.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:51 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:51 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:38:51.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:51 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:38:51.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:51 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:38:51.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:51 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:38:51.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:51 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:51 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:51 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:51 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:51 smithi062 ceph-mon[32390]: pgmap v183: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T00:38:51.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:51 smithi062 ceph-mon[32390]: scrub queued for path: / 2024-06-10T00:38:51.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:51 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:51 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:38:51.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:51 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:38:51.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:51 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:38:51.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:51 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:38:51.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:51 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:51 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:51 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:51 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:51 smithi138 ceph-mon[36549]: pgmap v183: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 0 B/s wr, 8 op/s 2024-06-10T00:38:51.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:51 smithi138 ceph-mon[36549]: scrub queued for path: / 2024-06-10T00:38:51.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:51 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:51 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:38:51.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:51 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:38:51.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:51 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:38:51.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:51 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:38:51.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:51 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:51 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:51 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.875 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:51 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:51.894 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:38:52.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:52 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:38:52.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:52 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:52.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:52 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:38:52.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:52 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:52.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:52 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:38:52.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:52 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:38:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:53 smithi062 ceph-mon[32390]: pgmap v184: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 op/s 2024-06-10T00:38:53.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:53 smithi105 ceph-mon[36321]: pgmap v184: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 op/s 2024-06-10T00:38:53.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.862+0000 7fb412bf9700 1 -- 172.21.15.62:0/1220992077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40c1030f0 msgr2=0x7fb40c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.862+0000 7fb412bf9700 1 --2- 172.21.15.62:0/1220992077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40c1030f0 0x7fb40c107110 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80099c0 tx=0x7fb3f8009cd0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.862+0000 7fb412bf9700 1 -- 172.21.15.62:0/1220992077 shutdown_connections 2024-06-10T00:38:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.862+0000 7fb412bf9700 1 --2- 172.21.15.62:0/1220992077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c107840 0x7fb40c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.862+0000 7fb412bf9700 1 --2- 172.21.15.62:0/1220992077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb40c102740 0x7fb40c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.862+0000 7fb412bf9700 1 --2- 172.21.15.62:0/1220992077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40c1030f0 0x7fb40c107110 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.862+0000 7fb412bf9700 1 -- 172.21.15.62:0/1220992077 >> 172.21.15.62:0/1220992077 conn(0x7fb40c0795f0 msgr2=0x7fb40c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.862+0000 7fb412bf9700 1 -- 172.21.15.62:0/1220992077 shutdown_connections 2024-06-10T00:38:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.862+0000 7fb412bf9700 1 -- 172.21.15.62:0/1220992077 wait complete. 2024-06-10T00:38:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.863+0000 7fb412bf9700 1 Processor -- start 2024-06-10T00:38:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.863+0000 7fb412bf9700 1 -- start start 2024-06-10T00:38:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.863+0000 7fb412bf9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb40c102740 0x7fb40c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.863+0000 7fb412bf9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c1030f0 0x7fb40c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.863+0000 7fb412bf9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40c107840 0x7fb40c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.863+0000 7fb412bf9700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c112750 con 0x7fb40c107840 2024-06-10T00:38:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.863+0000 7fb412bf9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1125d0 con 0x7fb40c102740 2024-06-10T00:38:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.863+0000 7fb412bf9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1128d0 con 0x7fb40c1030f0 2024-06-10T00:38:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.863+0000 7fb411196700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40c107840 0x7fb40c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.863+0000 7fb411196700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40c107840 0x7fb40c075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50684/0 (socket says 172.21.15.62:50684) 2024-06-10T00:38:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.863+0000 7fb411196700 1 -- 172.21.15.62:0/441384968 learned_addr learned my addr 172.21.15.62:0/441384968 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.863+0000 7fb410995700 1 --2- 172.21.15.62:0/441384968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb40c102740 0x7fb40c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.863+0000 7fb40bfff700 1 --2- 172.21.15.62:0/441384968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c1030f0 0x7fb40c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.864+0000 7fb411196700 1 -- 172.21.15.62:0/441384968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c1030f0 msgr2=0x7fb40c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.864+0000 7fb411196700 1 --2- 172.21.15.62:0/441384968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c1030f0 0x7fb40c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.864+0000 7fb411196700 1 -- 172.21.15.62:0/441384968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb40c102740 msgr2=0x7fb40c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.864+0000 7fb411196700 1 --2- 172.21.15.62:0/441384968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb40c102740 0x7fb40c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.864+0000 7fb411196700 1 -- 172.21.15.62:0/441384968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c1a70e0 con 0x7fb40c107840 2024-06-10T00:38:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.864+0000 7fb411196700 1 --2- 172.21.15.62:0/441384968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40c107840 0x7fb40c075210 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb404007ee0 tx=0x7fb40400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:53.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.864+0000 7fb409ffb700 1 -- 172.21.15.62:0/441384968 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404004bb0 con 0x7fb40c107840 2024-06-10T00:38:53.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.864+0000 7fb409ffb700 1 -- 172.21.15.62:0/441384968 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb404004d50 con 0x7fb40c107840 2024-06-10T00:38:53.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.864+0000 7fb409ffb700 1 -- 172.21.15.62:0/441384968 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404005960 con 0x7fb40c107840 2024-06-10T00:38:53.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.864+0000 7fb412bf9700 1 -- 172.21.15.62:0/441384968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c1a73d0 con 0x7fb40c107840 2024-06-10T00:38:53.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.864+0000 7fb412bf9700 1 -- 172.21.15.62:0/441384968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c1a7890 con 0x7fb40c107840 2024-06-10T00:38:53.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.867+0000 7fb409ffb700 1 -- 172.21.15.62:0/441384968 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb40401a020 con 0x7fb40c107840 2024-06-10T00:38:53.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.867+0000 7fb409ffb700 1 --2- 172.21.15.62:0/441384968 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb3fc0a2e10 0x7fb3fc0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.868+0000 7fb410995700 1 --2- 172.21.15.62:0/441384968 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb3fc0a2e10 0x7fb3fc0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.868+0000 7fb409ffb700 1 -- 172.21.15.62:0/441384968 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fb4040cee70 con 0x7fb40c107840 2024-06-10T00:38:53.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.868+0000 7fb410995700 1 --2- 172.21.15.62:0/441384968 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb3fc0a2e10 0x7fb3fc0a52d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb400005950 tx=0x7fb40000b410 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:53.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.868+0000 7fb41d47cb80 1 Processor -- start 2024-06-10T00:38:53.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.868+0000 7fb41d47cb80 1 -- start start 2024-06-10T00:38:53.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.868+0000 7fb41d47cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 0x55a38dd4da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.868+0000 7fb41d47cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd4df60 0x55a38dd479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.868+0000 7fb41d47cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd48340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.868+0000 7fb41d47cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a38dc99810 con 0x55a38dd47f20 2024-06-10T00:38:53.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.868+0000 7fb41d47cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a38dc41720 con 0x55a38dd4df60 2024-06-10T00:38:53.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.868+0000 7fb41d47cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a38dc957d0 con 0x55a38dd4d640 2024-06-10T00:38:53.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.869+0000 7fb411196700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd48340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.869+0000 7fb411196700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd48340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50696/0 (socket says 172.21.15.62:50696) 2024-06-10T00:38:53.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.869+0000 7fb411196700 1 -- 172.21.15.62:0/774438362 learned_addr learned my addr 172.21.15.62:0/774438362 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:53.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.869+0000 7fb410995700 1 --2- 172.21.15.62:0/774438362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 0x55a38dd4da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.869+0000 7fb40bfff700 1 --2- 172.21.15.62:0/774438362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd4df60 0x55a38dd479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.869+0000 7fb411196700 1 -- 172.21.15.62:0/774438362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 msgr2=0x55a38dd4da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:53.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.869+0000 7fb411196700 1 --2- 172.21.15.62:0/774438362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 0x55a38dd4da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.869+0000 7fb411196700 1 -- 172.21.15.62:0/774438362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd4df60 msgr2=0x55a38dd479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:53.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.869+0000 7fb411196700 1 --2- 172.21.15.62:0/774438362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd4df60 0x55a38dd479e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.869+0000 7fb411196700 1 -- 172.21.15.62:0/774438362 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a38dc37f50 con 0x55a38dd47f20 2024-06-10T00:38:53.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.869+0000 7fb411196700 1 --2- 172.21.15.62:0/774438362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd48340 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb4040979b0 tx=0x7fb40409aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75978adc09de7875 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:53.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.869+0000 7fb3f6ffd700 1 -- 172.21.15.62:0/774438362 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40409afe0 con 0x55a38dd47f20 2024-06-10T00:38:53.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.869+0000 7fb3f6ffd700 1 -- 172.21.15.62:0/774438362 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb40409f390 con 0x55a38dd47f20 2024-06-10T00:38:53.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.869+0000 7fb3f6ffd700 1 -- 172.21.15.62:0/774438362 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4040977e0 con 0x55a38dd47f20 2024-06-10T00:38:53.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.870+0000 7fb41d47cb80 1 -- 172.21.15.62:0/774438362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 msgr2=0x55a38dd48340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:53.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.870+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/774438362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd48340 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb4040979b0 tx=0x7fb40409aba0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.870+0000 7fb41d47cb80 1 -- 172.21.15.62:0/774438362 shutdown_connections 2024-06-10T00:38:53.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.870+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/774438362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd48340 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.870+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/774438362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 0x55a38dd4da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.870+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/774438362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd4df60 0x55a38dd479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.870+0000 7fb41d47cb80 1 -- 172.21.15.62:0/774438362 >> 172.21.15.62:0/774438362 conn(0x55a38dc72cc0 msgr2=0x55a38dd42850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:53.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.870+0000 7fb41d47cb80 1 -- 172.21.15.62:0/774438362 shutdown_connections 2024-06-10T00:38:53.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.870+0000 7fb41d47cb80 1 -- 172.21.15.62:0/774438362 wait complete. 2024-06-10T00:38:53.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.870+0000 7fb41d47cb80 1 -- start start 2024-06-10T00:38:53.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.870+0000 7fb41d47cb80 1 Processor -- start 2024-06-10T00:38:53.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb41d47cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd47f20 0x55a38dd53db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb41d47cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 0x55a38dd510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb41d47cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd4df60 0x55a38dd515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb41d47cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a38dd549d0 con 0x55a38dd4df60 2024-06-10T00:38:53.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb41d47cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a38dc39bf0 con 0x55a38dd47f20 2024-06-10T00:38:53.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb41d47cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a38dd54b50 con 0x55a38dd4d640 2024-06-10T00:38:53.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb41d47cb80 10 client.0 _ll_register_callbacks cb 0x55a38dc33920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:53.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb411196700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd4df60 0x55a38dd515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb411196700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd4df60 0x55a38dd515f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50710/0 (socket says 172.21.15.62:50710) 2024-06-10T00:38:53.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb411196700 1 -- 172.21.15.62:0/464574993 learned_addr learned my addr 172.21.15.62:0/464574993 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:53.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb410995700 1 --2- 172.21.15.62:0/464574993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd47f20 0x55a38dd53db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb40bfff700 1 --2- 172.21.15.62:0/464574993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 0x55a38dd510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb411196700 1 -- 172.21.15.62:0/464574993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 msgr2=0x55a38dd510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:53.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb411196700 1 --2- 172.21.15.62:0/464574993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 0x55a38dd510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb411196700 1 -- 172.21.15.62:0/464574993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd47f20 msgr2=0x55a38dd53db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:53.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb411196700 1 --2- 172.21.15.62:0/464574993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd47f20 0x55a38dd53db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.871+0000 7fb411196700 1 -- 172.21.15.62:0/464574993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a38dbe88d0 con 0x55a38dd4df60 2024-06-10T00:38:53.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb411196700 1 --2- 172.21.15.62:0/464574993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd4df60 0x55a38dd515f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb404007fd0 tx=0x7fb4040cc1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c783ae1b8e2bc397 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:53.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb3ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:53.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb3ebfff700 1 -- 172.21.15.62:0/464574993 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4040b3070 con 0x55a38dd4df60 2024-06-10T00:38:53.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb3ebfff700 1 -- 172.21.15.62:0/464574993 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fb404022250 con 0x55a38dd4df60 2024-06-10T00:38:53.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb41d47cb80 1 -- 172.21.15.62:0/464574993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb40409f390 con 0x55a38dd4df60 2024-06-10T00:38:53.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb3ebfff700 1 -- 172.21.15.62:0/464574993 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb404022450 con 0x55a38dd4df60 2024-06-10T00:38:53.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb41d47cb80 10 client.16458 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:53.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb41d47cb80 1 -- 172.21.15.62:0/464574993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a38dd5f3e0 con 0x55a38dd4df60 2024-06-10T00:38:53.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb3ebfff700 1 -- 172.21.15.62:0/464574993 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb40409f390 con 0x55a38dd4df60 2024-06-10T00:38:53.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb3ebfff700 1 -- 172.21.15.62:0/464574993 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7fb404095b50 con 0x55a38dd4df60 2024-06-10T00:38:53.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb41d47cb80 10 client.16458 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:53.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb41d47cb80 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:38:53.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb41d47cb80 20 client.16458 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:53.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/464574993 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a38dd618b0 0x55a38dd63ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb41d47cb80 4 client.16458 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:38:53.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.872+0000 7fb41d47cb80 1 -- 172.21.15.62:0/464574993 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a38dca6d10 con 0x55a38dd618b0 2024-06-10T00:38:53.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.873+0000 7fb410995700 1 --2- 172.21.15.62:0/464574993 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a38dd618b0 0x55a38dd63ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.873+0000 7fb410995700 1 --2- 172.21.15.62:0/464574993 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a38dd618b0 0x55a38dd63ca0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95c305bd96520670 server_cookie=d72acca6991bc8d3 in_seq=0 out_seq=0 2024-06-10T00:38:53.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.873+0000 7fb3ebfff700 10 client.16458 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:53.884 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:53 smithi138 ceph-mon[36549]: pgmap v184: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 op/s 2024-06-10T00:38:53.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.874+0000 7fb3ebfff700 1 -- 172.21.15.62:0/464574993 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a38dca6d10 con 0x55a38dd618b0 2024-06-10T00:38:53.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.874+0000 7fb3ebfff700 10 client.16458 handle_command_reply: tid=0 2024-06-10T00:38:53.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.874+0000 7fb41d47cb80 1 client.16458 shutdown 2024-06-10T00:38:53.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.874+0000 7fb41d47cb80 1 -- 172.21.15.62:0/464574993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd4df60 msgr2=0x55a38dd515f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:53.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.874+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/464574993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd4df60 0x55a38dd515f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb404007fd0 tx=0x7fb4040cc1f0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.874+0000 7fb41d47cb80 1 -- 172.21.15.62:0/464574993 shutdown_connections 2024-06-10T00:38:53.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.874+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/464574993 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a38dd618b0 0x55a38dd63ca0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.874+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/464574993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd4df60 0x55a38dd515f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.874+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/464574993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd47f20 0x55a38dd53db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.874+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/464574993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 0x55a38dd510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.874+0000 7fb41d47cb80 1 -- 172.21.15.62:0/464574993 >> 172.21.15.62:0/464574993 conn(0x55a38dc72cc0 msgr2=0x55a38dd469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:53.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.874+0000 7fb3ebfff700 0 client.16458 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:53.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.875+0000 7fb41d47cb80 1 -- 172.21.15.62:0/464574993 shutdown_connections 2024-06-10T00:38:53.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.875+0000 7fb41d47cb80 1 -- 172.21.15.62:0/464574993 wait complete. 2024-06-10T00:38:53.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.875+0000 7fb41d47cb80 20 client.16458 trim_cache size 0 max 16384 2024-06-10T00:38:53.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.895+0000 7fb41d47cb80 1 Processor -- start 2024-06-10T00:38:53.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.895+0000 7fb41d47cb80 1 -- start start 2024-06-10T00:38:53.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.895+0000 7fb41d47cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.895+0000 7fb41d47cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 0x55a38dd429f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.895+0000 7fb41d47cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd4df60 0x55a38dd3c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.895+0000 7fb41d47cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1b08c0 con 0x55a38dd47f20 2024-06-10T00:38:53.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.895+0000 7fb41d47cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1b2580 con 0x55a38dd4df60 2024-06-10T00:38:53.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.895+0000 7fb41d47cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1b0f90 con 0x55a38dd4d640 2024-06-10T00:38:53.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.895+0000 7fb410995700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.895+0000 7fb410995700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd479e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50724/0 (socket says 172.21.15.62:50724) 2024-06-10T00:38:53.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.895+0000 7fb410995700 1 -- 172.21.15.62:0/3775941247 learned_addr learned my addr 172.21.15.62:0/3775941247 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:53.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.896+0000 7fb40bfff700 1 --2- 172.21.15.62:0/3775941247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 0x55a38dd429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.896+0000 7fb410995700 1 -- 172.21.15.62:0/3775941247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 msgr2=0x55a38dd429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:53.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.896+0000 7fb410995700 1 --2- 172.21.15.62:0/3775941247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 0x55a38dd429f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.896+0000 7fb410995700 1 -- 172.21.15.62:0/3775941247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd4df60 msgr2=0x55a38dd3c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:53.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.896+0000 7fb410995700 1 --2- 172.21.15.62:0/3775941247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd4df60 0x55a38dd3c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.896+0000 7fb410995700 1 -- 172.21.15.62:0/3775941247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a38dd538f0 con 0x55a38dd47f20 2024-06-10T00:38:53.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.896+0000 7fb410995700 1 --2- 172.21.15.62:0/3775941247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd479e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb400011010 tx=0x7fb400013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29387239cceff20 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:53.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.896+0000 7fb3f67fc700 1 -- 172.21.15.62:0/3775941247 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400011d60 con 0x55a38dd47f20 2024-06-10T00:38:53.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.896+0000 7fb3f67fc700 1 -- 172.21.15.62:0/3775941247 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb400004d10 con 0x55a38dd47f20 2024-06-10T00:38:53.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.896+0000 7fb3f67fc700 1 -- 172.21.15.62:0/3775941247 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40002a6d0 con 0x55a38dd47f20 2024-06-10T00:38:53.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.897+0000 7fb41d47cb80 1 -- 172.21.15.62:0/3775941247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 msgr2=0x55a38dd479e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:53.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.897+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/3775941247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd479e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb400011010 tx=0x7fb400013000 comp rx=0 tx=0).stop 2024-06-10T00:38:53.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.897+0000 7fb41d47cb80 1 -- 172.21.15.62:0/3775941247 shutdown_connections 2024-06-10T00:38:53.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.897+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/3775941247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd479e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.897+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/3775941247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd4df60 0x55a38dd3c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.897+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/3775941247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4d640 0x55a38dd429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.897+0000 7fb41d47cb80 1 -- 172.21.15.62:0/3775941247 >> 172.21.15.62:0/3775941247 conn(0x55a38dc72cc0 msgr2=0x55a38dd4e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:53.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.897+0000 7fb41d47cb80 1 -- 172.21.15.62:0/3775941247 shutdown_connections 2024-06-10T00:38:53.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.897+0000 7fb41d47cb80 1 -- 172.21.15.62:0/3775941247 wait complete. 2024-06-10T00:38:53.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.897+0000 7fb41d47cb80 1 -- start start 2024-06-10T00:38:53.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.897+0000 7fb41d47cb80 1 Processor -- start 2024-06-10T00:38:53.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb41d47cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd64870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb41d47cb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd4d640 0x55a38dd64db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb41d47cb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4df60 0x55a38dd652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb41d47cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1b1df0 con 0x55a38dd47f20 2024-06-10T00:38:53.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb41d47cb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1b1660 con 0x55a38dd4d640 2024-06-10T00:38:53.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb41d47cb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1ae860 con 0x55a38dd4df60 2024-06-10T00:38:53.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb41d47cb80 10 client.0 _ll_register_callbacks cb 0x55a38dc33920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:38:53.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb410995700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd64870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb410995700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd64870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50734/0 (socket says 172.21.15.62:50734) 2024-06-10T00:38:53.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb410995700 1 -- 172.21.15.62:0/2185579800 learned_addr learned my addr 172.21.15.62:0/2185579800 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:38:53.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb40bfff700 1 --2- 172.21.15.62:0/2185579800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd4d640 0x55a38dd64db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb410995700 1 -- 172.21.15.62:0/2185579800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4df60 msgr2=0x55a38dd652f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:38:53.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb410995700 1 --2- 172.21.15.62:0/2185579800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4df60 0x55a38dd652f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb410995700 1 -- 172.21.15.62:0/2185579800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd4d640 msgr2=0x55a38dd64db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:38:53.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb410995700 1 --2- 172.21.15.62:0/2185579800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd4d640 0x55a38dd64db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb410995700 1 -- 172.21.15.62:0/2185579800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a38ddcb3c0 con 0x55a38dd47f20 2024-06-10T00:38:53.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb410995700 1 --2- 172.21.15.62:0/2185579800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd64870 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb400005560 tx=0x7fb400012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88600f8d13bbfd30 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:38:53.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.898+0000 7fb3ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:38:53.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb3ebfff700 1 -- 172.21.15.62:0/2185579800 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400004b60 con 0x55a38dd47f20 2024-06-10T00:38:53.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb3ebfff700 1 -- 172.21.15.62:0/2185579800 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fb400004ec0 con 0x55a38dd47f20 2024-06-10T00:38:53.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb41d47cb80 1 -- 172.21.15.62:0/2185579800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb400004d10 con 0x55a38dd47f20 2024-06-10T00:38:53.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb3ebfff700 1 -- 172.21.15.62:0/2185579800 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb400047050 con 0x55a38dd47f20 2024-06-10T00:38:53.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb41d47cb80 10 client.16464 fetch_fsmap learned FSMap version 34 2024-06-10T00:38:53.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb41d47cb80 1 -- 172.21.15.62:0/2185579800 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a38ddcc6b0 con 0x55a38dd47f20 2024-06-10T00:38:53.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb3ebfff700 1 -- 172.21.15.62:0/2185579800 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb400004d10 con 0x55a38dd47f20 2024-06-10T00:38:53.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb3ebfff700 1 -- 172.21.15.62:0/2185579800 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7fb40003e470 con 0x55a38dd47f20 2024-06-10T00:38:53.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb41d47cb80 10 client.16464 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T00:38:53.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb41d47cb80 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:38:53.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb41d47cb80 20 client.16464 populate_metadata read hostname 'smithi062' 2024-06-10T00:38:53.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/2185579800 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a38ddcce20 0x55a38ddcf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:38:53.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb41d47cb80 4 client.16464 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:38:53.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb41d47cb80 1 -- 172.21.15.62:0/2185579800 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a38dca1290 con 0x55a38ddcce20 2024-06-10T00:38:53.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.899+0000 7fb40bfff700 1 --2- 172.21.15.62:0/2185579800 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a38ddcce20 0x55a38ddcf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:38:53.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.900+0000 7fb40bfff700 1 --2- 172.21.15.62:0/2185579800 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a38ddcce20 0x55a38ddcf210 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af26955cdf9884bc server_cookie=e5f81298eb3d6941 in_seq=0 out_seq=0 2024-06-10T00:38:53.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.900+0000 7fb3ebfff700 10 client.16464 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:53.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.900+0000 7fb3ebfff700 1 -- 172.21.15.62:0/2185579800 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a38dca1290 con 0x55a38ddcce20 2024-06-10T00:38:53.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.900+0000 7fb3ebfff700 10 client.16464 handle_command_reply: tid=0 2024-06-10T00:38:53.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.900+0000 7fb41d47cb80 1 client.16464 shutdown 2024-06-10T00:38:53.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.900+0000 7fb41d47cb80 1 -- 172.21.15.62:0/2185579800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 msgr2=0x55a38dd64870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:53.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.900+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/2185579800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd64870 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb400005560 tx=0x7fb400012b70 comp rx=0 tx=0).stop 2024-06-10T00:38:53.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.900+0000 7fb41d47cb80 1 -- 172.21.15.62:0/2185579800 shutdown_connections 2024-06-10T00:38:53.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.900+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/2185579800 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55a38ddcce20 0x55a38ddcf210 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.900+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/2185579800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a38dd47f20 0x55a38dd64870 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.900+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/2185579800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a38dd4df60 0x55a38dd652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.900+0000 7fb41d47cb80 1 --2- 172.21.15.62:0/2185579800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a38dd4d640 0x55a38dd64db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.900+0000 7fb3ebfff700 0 client.16464 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:38:53.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.900+0000 7fb41d47cb80 1 -- 172.21.15.62:0/2185579800 >> 172.21.15.62:0/2185579800 conn(0x55a38dc72cc0 msgr2=0x55a38dd3c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:53.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb41d47cb80 1 -- 172.21.15.62:0/2185579800 shutdown_connections 2024-06-10T00:38:53.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb41d47cb80 1 -- 172.21.15.62:0/2185579800 wait complete. 2024-06-10T00:38:53.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb41d47cb80 20 client.16464 trim_cache size 0 max 16384 2024-06-10T00:38:53.924 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:38:53.924 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T00:38:53.925 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-06-10T00:38:53.925 INFO:teuthology.orchestra.run.smithi062.stdout: "d0752cc9-2752-42bb-8765-ce6fe8795a1e": { 2024-06-10T00:38:53.925 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-06-10T00:38:53.926 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "d0752cc9-2752-42bb-8765-ce6fe8795a1e", 2024-06-10T00:38:53.926 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-06-10T00:38:53.926 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:38:53.926 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:38:53.927 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:38:53.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb412bf9700 1 -- 172.21.15.62:0/441384968 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb3fc0a2e10 msgr2=0x7fb3fc0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:53.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb412bf9700 1 --2- 172.21.15.62:0/441384968 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb3fc0a2e10 0x7fb3fc0a52d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb400005950 tx=0x7fb40000b410 comp rx=0 tx=0).stop 2024-06-10T00:38:53.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb412bf9700 1 -- 172.21.15.62:0/441384968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40c107840 msgr2=0x7fb40c075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:38:53.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb412bf9700 1 --2- 172.21.15.62:0/441384968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40c107840 0x7fb40c075210 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb404007ee0 tx=0x7fb40400d3b0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb412bf9700 1 -- 172.21.15.62:0/441384968 shutdown_connections 2024-06-10T00:38:53.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb412bf9700 1 --2- 172.21.15.62:0/441384968 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb3fc0a2e10 0x7fb3fc0a52d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb412bf9700 1 --2- 172.21.15.62:0/441384968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb40c107840 0x7fb40c075210 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb412bf9700 1 --2- 172.21.15.62:0/441384968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb40c102740 0x7fb40c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb412bf9700 1 --2- 172.21.15.62:0/441384968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c1030f0 0x7fb40c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:38:53.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb412bf9700 1 -- 172.21.15.62:0/441384968 >> 172.21.15.62:0/441384968 conn(0x7fb40c0795f0 msgr2=0x7fb40c108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:38:53.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb412bf9700 1 -- 172.21.15.62:0/441384968 shutdown_connections 2024-06-10T00:38:53.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:38:53.901+0000 7fb412bf9700 1 -- 172.21.15.62:0/441384968 wait complete. 2024-06-10T00:38:54.731 INFO:tasks.cephfs.filesystem:scrub status for tag:d0752cc9-2752-42bb-8765-ce6fe8795a1e - {'path': '/', 'tag': 'd0752cc9-2752-42bb-8765-ce6fe8795a1e', 'options': 'recursive,force'} 2024-06-10T00:38:55.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:55 smithi105 ceph-mon[36321]: pgmap v185: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9 op/s 2024-06-10T00:38:55.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:55 smithi062 ceph-mon[32390]: pgmap v185: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9 op/s 2024-06-10T00:38:55.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:55 smithi138 ceph-mon[36549]: pgmap v185: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9 op/s 2024-06-10T00:38:57.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:57 smithi105 ceph-mon[36321]: pgmap v186: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 op/s 2024-06-10T00:38:57.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:57 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:38:57.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:57 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:38:57.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:57 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:38:57.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:57 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:38:57.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:57 smithi062 ceph-mon[32390]: pgmap v186: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 op/s 2024-06-10T00:38:57.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:57 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:38:57.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:57 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:38:57.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:57 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:38:57.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:57 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:38:57.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:57 smithi138 ceph-mon[36549]: pgmap v186: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 op/s 2024-06-10T00:38:57.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:57 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:38:57.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:57 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:38:57.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:57 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:38:57.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:57 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:38:58.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:58 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:38:58.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:58 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:38:58.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:58 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:38:59.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:38:59 smithi105 ceph-mon[36321]: pgmap v187: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 op/s 2024-06-10T00:38:59.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:38:59 smithi062 ceph-mon[32390]: pgmap v187: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 op/s 2024-06-10T00:38:59.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:38:59 smithi138 ceph-mon[36549]: pgmap v187: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 op/s 2024-06-10T00:39:01.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:01 smithi105 ceph-mon[36321]: pgmap v188: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9 op/s 2024-06-10T00:39:01.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:01 smithi062 ceph-mon[32390]: pgmap v188: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9 op/s 2024-06-10T00:39:01.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:01 smithi138 ceph-mon[36549]: pgmap v188: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9 op/s 2024-06-10T00:39:03.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:03 smithi105 ceph-mon[36321]: pgmap v189: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-10T00:39:03.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:03 smithi062 ceph-mon[32390]: pgmap v189: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-10T00:39:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:03 smithi138 ceph-mon[36549]: pgmap v189: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-10T00:39:05.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:05 smithi105 ceph-mon[36321]: pgmap v190: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-06-10T00:39:05.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:05 smithi062 ceph-mon[32390]: pgmap v190: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-06-10T00:39:05.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:05 smithi138 ceph-mon[36549]: pgmap v190: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-06-10T00:39:07.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:07 smithi105 ceph-mon[36321]: pgmap v191: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-06-10T00:39:07.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:07 smithi062 ceph-mon[32390]: pgmap v191: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-06-10T00:39:07.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:07 smithi138 ceph-mon[36549]: pgmap v191: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-06-10T00:39:09.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:09 smithi105 ceph-mon[36321]: pgmap v192: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-06-10T00:39:09.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:09 smithi062 ceph-mon[32390]: pgmap v192: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-06-10T00:39:09.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:09 smithi138 ceph-mon[36549]: pgmap v192: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-06-10T00:39:11.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:11 smithi105 ceph-mon[36321]: pgmap v193: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-06-10T00:39:11.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:11 smithi062 ceph-mon[32390]: pgmap v193: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-06-10T00:39:11.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:11 smithi138 ceph-mon[36549]: pgmap v193: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-06-10T00:39:13.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:13 smithi105 ceph-mon[36321]: pgmap v194: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-06-10T00:39:13.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:13 smithi062 ceph-mon[32390]: pgmap v194: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-06-10T00:39:13.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:13 smithi138 ceph-mon[36549]: pgmap v194: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-06-10T00:39:15.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:15 smithi105 ceph-mon[36321]: pgmap v195: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-06-10T00:39:15.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:15 smithi062 ceph-mon[32390]: pgmap v195: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-06-10T00:39:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:15 smithi138 ceph-mon[36549]: pgmap v195: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-06-10T00:39:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:17 smithi105 ceph-mon[36321]: pgmap v196: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-10T00:39:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:17 smithi062 ceph-mon[32390]: pgmap v196: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-10T00:39:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:17 smithi138 ceph-mon[36549]: pgmap v196: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-10T00:39:18.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:18 smithi105 ceph-mon[36321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:39:18.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:18 smithi062 ceph-mon[32390]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:39:18.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:18 smithi138 ceph-mon[36549]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:39:19.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:19 smithi105 ceph-mon[36321]: pgmap v197: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:39:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:19 smithi062 ceph-mon[32390]: pgmap v197: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:39:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:19 smithi138 ceph-mon[36549]: pgmap v197: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:39:21.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:21 smithi062 ceph-mon[32390]: pgmap v198: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T00:39:21.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:21 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:39:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:21 smithi138 ceph-mon[36549]: pgmap v198: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T00:39:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:21 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:39:22.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:21 smithi105 ceph-mon[36321]: pgmap v198: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-06-10T00:39:22.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:21 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:39:22.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:22 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:39:22.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:22 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:39:22.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:22 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:39:22.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:22 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:39:22.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:22 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:39:22.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:22 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:39:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:22 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:39:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:22 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:39:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:22 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:39:23.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:23 smithi062 ceph-mon[32390]: pgmap v199: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T00:39:23.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:23 smithi062 ceph-mon[32390]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:39:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:23 smithi138 ceph-mon[36549]: pgmap v199: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T00:39:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:23 smithi138 ceph-mon[36549]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:39:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:23 smithi105 ceph-mon[36321]: pgmap v199: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-06-10T00:39:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:23 smithi105 ceph-mon[36321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:39:24.732 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:39:24.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:24 smithi062 ceph-mon[32390]: pgmap v200: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:24.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:24 smithi138 ceph-mon[36549]: pgmap v200: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:39:25.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:24 smithi105 ceph-mon[36321]: pgmap v200: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:26.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.800+0000 7f5cc1943700 1 -- 172.21.15.62:0/9362480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cbc107840 msgr2=0x7f5cbc10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:26.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.800+0000 7f5cc1943700 1 --2- 172.21.15.62:0/9362480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cbc107840 0x7f5cbc10f3c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400b0d0 tx=0x7f5cb400b3e0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.800+0000 7f5cc1943700 1 -- 172.21.15.62:0/9362480 shutdown_connections 2024-06-10T00:39:26.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.800+0000 7f5cc1943700 1 --2- 172.21.15.62:0/9362480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cbc107840 0x7f5cbc10f3c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.800+0000 7f5cc1943700 1 --2- 172.21.15.62:0/9362480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc102740 0x7f5cbc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.800+0000 7f5cc1943700 1 --2- 172.21.15.62:0/9362480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cbc1030f0 0x7f5cbc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.800+0000 7f5cc1943700 1 -- 172.21.15.62:0/9362480 >> 172.21.15.62:0/9362480 conn(0x7f5cbc0795f0 msgr2=0x7f5cbc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:26.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.800+0000 7f5cc1943700 1 -- 172.21.15.62:0/9362480 shutdown_connections 2024-06-10T00:39:26.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.800+0000 7f5cc1943700 1 -- 172.21.15.62:0/9362480 wait complete. 2024-06-10T00:39:26.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.800+0000 7f5cc1943700 1 Processor -- start 2024-06-10T00:39:26.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cc1943700 1 -- start start 2024-06-10T00:39:26.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cc1943700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cbc102740 0x7f5cbc19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cc1943700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cbc1030f0 0x7f5cbc19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cc1943700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc107840 0x7f5cbc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cc1943700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc112760 con 0x7f5cbc102740 2024-06-10T00:39:26.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cc1943700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc1125e0 con 0x7f5cbc1030f0 2024-06-10T00:39:26.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cc1943700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc1128e0 con 0x7f5cbc107840 2024-06-10T00:39:26.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cbaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cbc102740 0x7f5cbc19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cbaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cbc102740 0x7f5cbc19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45434/0 (socket says 172.21.15.62:45434) 2024-06-10T00:39:26.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cbaffd700 1 -- 172.21.15.62:0/2492995167 learned_addr learned my addr 172.21.15.62:0/2492995167 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:26.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cbb7fe700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc107840 0x7f5cbc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cb27fc700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cbc1030f0 0x7f5cbc19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cbaffd700 1 -- 172.21.15.62:0/2492995167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc107840 msgr2=0x7f5cbc198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:26.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc107840 0x7f5cbc198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cbaffd700 1 -- 172.21.15.62:0/2492995167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cbc1030f0 msgr2=0x7f5cbc19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:26.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cbc1030f0 0x7f5cbc19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cbaffd700 1 -- 172.21.15.62:0/2492995167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cbc1990c0 con 0x7f5cbc102740 2024-06-10T00:39:26.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cbb7fe700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc107840 0x7f5cbc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:39:26.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.801+0000 7f5cb27fc700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cbc1030f0 0x7f5cbc19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:39:26.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.802+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cbc102740 0x7f5cbc19e7a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00d980 tx=0x7f5cac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:26.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.802+0000 7f5cc0941700 1 -- 172.21.15.62:0/2492995167 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac016070 con 0x7f5cbc102740 2024-06-10T00:39:26.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.802+0000 7f5cc0941700 1 -- 172.21.15.62:0/2492995167 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cac004500 con 0x7f5cbc102740 2024-06-10T00:39:26.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.802+0000 7f5cc1943700 1 -- 172.21.15.62:0/2492995167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cbc1993b0 con 0x7f5cbc102740 2024-06-10T00:39:26.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.802+0000 7f5cc0941700 1 -- 172.21.15.62:0/2492995167 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac004d00 con 0x7f5cbc102740 2024-06-10T00:39:26.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.802+0000 7f5cc1943700 1 -- 172.21.15.62:0/2492995167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cbc1a5670 con 0x7f5cbc102740 2024-06-10T00:39:26.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.804+0000 7f5cc0941700 1 -- 172.21.15.62:0/2492995167 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5cac00bbf0 con 0x7f5cbc102740 2024-06-10T00:39:26.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.805+0000 7f5cc0941700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5ca80a2d90 0x7f5ca80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.805+0000 7f5cb27fc700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5ca80a2d90 0x7f5ca80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.805+0000 7f5cc0941700 1 -- 172.21.15.62:0/2492995167 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f5cac0cf6d0 con 0x7f5cbc102740 2024-06-10T00:39:26.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.805+0000 7f5cb27fc700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5ca80a2d90 0x7f5ca80a5250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc101f90 tx=0x7f5ca4009550 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:26.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.805+0000 7f5ccc1c6b80 1 Processor -- start 2024-06-10T00:39:26.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5ccc1c6b80 1 -- start start 2024-06-10T00:39:26.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5ccc1c6b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 0x55be3504da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5ccc1c6b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be3504df60 0x55be350479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5ccc1c6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be35047f20 0x55be35048340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5ccc1c6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55be34f99810 con 0x55be35047f20 2024-06-10T00:39:26.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5ccc1c6b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55be34f41720 con 0x55be3504df60 2024-06-10T00:39:26.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5ccc1c6b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55be34f957d0 con 0x55be3504d640 2024-06-10T00:39:26.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5cbb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be35047f20 0x55be35048340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5cbb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be35047f20 0x55be35048340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45448/0 (socket says 172.21.15.62:45448) 2024-06-10T00:39:26.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5cbb7fe700 1 -- 172.21.15.62:0/2563529628 learned_addr learned my addr 172.21.15.62:0/2563529628 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:26.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/2563529628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 0x55be3504da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5cb27fc700 1 --2- 172.21.15.62:0/2563529628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be3504df60 0x55be350479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5cbb7fe700 1 -- 172.21.15.62:0/2563529628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 msgr2=0x55be3504da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:26.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5cbb7fe700 1 --2- 172.21.15.62:0/2563529628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 0x55be3504da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5cbb7fe700 1 -- 172.21.15.62:0/2563529628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be3504df60 msgr2=0x55be350479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:26.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5cbb7fe700 1 --2- 172.21.15.62:0/2563529628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be3504df60 0x55be350479e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5cbb7fe700 1 -- 172.21.15.62:0/2563529628 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be34f37f50 con 0x55be35047f20 2024-06-10T00:39:26.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5cb27fc700 1 --2- 172.21.15.62:0/2563529628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be3504df60 0x55be350479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:26.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.806+0000 7f5cbb7fe700 1 --2- 172.21.15.62:0/2563529628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be35047f20 0x55be35048340 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009dc0 tx=0x7f5cb4009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b12797347b787e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:26.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.807+0000 7f5cb1ffb700 1 -- 172.21.15.62:0/2563529628 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4021020 con 0x55be35047f20 2024-06-10T00:39:26.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.807+0000 7f5cb1ffb700 1 -- 172.21.15.62:0/2563529628 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb4009350 con 0x55be35047f20 2024-06-10T00:39:26.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.807+0000 7f5cb1ffb700 1 -- 172.21.15.62:0/2563529628 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40074e0 con 0x55be35047f20 2024-06-10T00:39:26.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.807+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/2563529628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be35047f20 msgr2=0x55be35048340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:26.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.807+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/2563529628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be35047f20 0x55be35048340 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009dc0 tx=0x7f5cb4009df0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.807+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/2563529628 shutdown_connections 2024-06-10T00:39:26.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.807+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/2563529628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be35047f20 0x55be35048340 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.807+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/2563529628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 0x55be3504da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.807+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/2563529628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be3504df60 0x55be350479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.807+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/2563529628 >> 172.21.15.62:0/2563529628 conn(0x55be34f72cc0 msgr2=0x55be35042850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:26.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.807+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/2563529628 shutdown_connections 2024-06-10T00:39:26.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.807+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/2563529628 wait complete. 2024-06-10T00:39:26.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.808+0000 7f5ccc1c6b80 1 -- start start 2024-06-10T00:39:26.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.808+0000 7f5ccc1c6b80 1 Processor -- start 2024-06-10T00:39:26.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.808+0000 7f5ccc1c6b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be35047f20 0x55be35053c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.808+0000 7f5ccc1c6b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 0x55be35050f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.808+0000 7f5ccc1c6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be3504df60 0x55be350514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.808+0000 7f5ccc1c6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55be35054970 con 0x55be3504df60 2024-06-10T00:39:26.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.808+0000 7f5ccc1c6b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55be34f39bf0 con 0x55be35047f20 2024-06-10T00:39:26.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.808+0000 7f5ccc1c6b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55be35054af0 con 0x55be3504d640 2024-06-10T00:39:26.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5ccc1c6b80 10 client.0 _ll_register_callbacks cb 0x55be34f33920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:26.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5cbb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be3504df60 0x55be350514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5cbb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be3504df60 0x55be350514b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45450/0 (socket says 172.21.15.62:45450) 2024-06-10T00:39:26.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5cbb7fe700 1 -- 172.21.15.62:0/546830319 learned_addr learned my addr 172.21.15.62:0/546830319 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:26.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/546830319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be35047f20 0x55be35053c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5cb27fc700 1 --2- 172.21.15.62:0/546830319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 0x55be35050f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5cbb7fe700 1 -- 172.21.15.62:0/546830319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 msgr2=0x55be35050f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5cbb7fe700 1 --2- 172.21.15.62:0/546830319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 0x55be35050f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5cbb7fe700 1 -- 172.21.15.62:0/546830319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be35047f20 msgr2=0x55be35053c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5cbb7fe700 1 --2- 172.21.15.62:0/546830319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be35047f20 0x55be35053c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5cbb7fe700 1 -- 172.21.15.62:0/546830319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be34ee88d0 con 0x55be3504df60 2024-06-10T00:39:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/546830319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be35047f20 0x55be35053c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5cbb7fe700 1 --2- 172.21.15.62:0/546830319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be3504df60 0x55be350514b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009d90 tx=0x7f5cb4004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf87ba4271c6e9d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5c9affd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:26.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5c9affd700 1 -- 172.21.15.62:0/546830319 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4021040 con 0x55be3504df60 2024-06-10T00:39:26.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5c9affd700 1 -- 172.21.15.62:0/546830319 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f5cb4005010 con 0x55be3504df60 2024-06-10T00:39:26.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.809+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/546830319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb4009350 con 0x55be3504df60 2024-06-10T00:39:26.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.810+0000 7f5c9affd700 1 -- 172.21.15.62:0/546830319 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5cb40074a0 con 0x55be3504df60 2024-06-10T00:39:26.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.810+0000 7f5ccc1c6b80 10 client.16476 fetch_fsmap learned FSMap version 36 2024-06-10T00:39:26.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.810+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/546830319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55be3505f240 con 0x55be3504df60 2024-06-10T00:39:26.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.810+0000 7f5c9affd700 1 -- 172.21.15.62:0/546830319 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb4009350 con 0x55be3504df60 2024-06-10T00:39:26.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.810+0000 7f5c9affd700 1 -- 172.21.15.62:0/546830319 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f5cb4011070 con 0x55be3504df60 2024-06-10T00:39:26.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.810+0000 7f5ccc1c6b80 10 client.16476 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T00:39:26.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.810+0000 7f5ccc1c6b80 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:39:26.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.810+0000 7f5ccc1c6b80 20 client.16476 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:26.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.810+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/546830319 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55be35061710 0x55be35063b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.810+0000 7f5ccc1c6b80 4 client.16476 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:39:26.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.810+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/546830319 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be34fa6d10 con 0x55be35061710 2024-06-10T00:39:26.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.810+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/546830319 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55be35061710 0x55be35063b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.811+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/546830319 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55be35061710 0x55be35063b00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d86df1c0d297a46 server_cookie=8dbd3880cca080d in_seq=0 out_seq=0 2024-06-10T00:39:26.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.811+0000 7f5c9affd700 10 client.16476 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:26.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.811+0000 7f5c9affd700 1 -- 172.21.15.62:0/546830319 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55be34fa6d10 con 0x55be35061710 2024-06-10T00:39:26.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.812+0000 7f5c9affd700 10 client.16476 handle_command_reply: tid=0 2024-06-10T00:39:26.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.812+0000 7f5ccc1c6b80 1 client.16476 shutdown 2024-06-10T00:39:26.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.812+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/546830319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be3504df60 msgr2=0x55be350514b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:26.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.812+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/546830319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be3504df60 0x55be350514b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009d90 tx=0x7f5cb4004d30 comp rx=0 tx=0).stop 2024-06-10T00:39:26.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.812+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/546830319 shutdown_connections 2024-06-10T00:39:26.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.812+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/546830319 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55be35061710 0x55be35063b00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.812+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/546830319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be3504df60 0x55be350514b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.812+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/546830319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be35047f20 0x55be35053c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.812+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/546830319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 0x55be35050f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.812+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/546830319 >> 172.21.15.62:0/546830319 conn(0x55be34f72cc0 msgr2=0x55be350469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:26.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.812+0000 7f5c9affd700 0 client.16476 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:26.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.812+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/546830319 shutdown_connections 2024-06-10T00:39:26.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.812+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/546830319 wait complete. 2024-06-10T00:39:26.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.812+0000 7f5ccc1c6b80 20 client.16476 trim_cache size 0 max 16384 2024-06-10T00:39:26.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5ccc1c6b80 1 Processor -- start 2024-06-10T00:39:26.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5ccc1c6b80 1 -- start start 2024-06-10T00:39:26.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5ccc1c6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be35047f20 0x55be350479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5ccc1c6b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be3504d640 0x55be3504e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5ccc1c6b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504df60 0x55be3504e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5ccc1c6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc1ae430 con 0x55be35047f20 2024-06-10T00:39:26.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5ccc1c6b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc1b0030 con 0x55be3504d640 2024-06-10T00:39:26.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5ccc1c6b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc1aeb00 con 0x55be3504df60 2024-06-10T00:39:26.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5cbaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be35047f20 0x55be350479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5cbaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be35047f20 0x55be350479e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45458/0 (socket says 172.21.15.62:45458) 2024-06-10T00:39:26.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5cbaffd700 1 -- 172.21.15.62:0/3904027738 learned_addr learned my addr 172.21.15.62:0/3904027738 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:26.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5cbb7fe700 1 --2- 172.21.15.62:0/3904027738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504df60 0x55be3504e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5cb27fc700 1 --2- 172.21.15.62:0/3904027738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be3504d640 0x55be3504e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5cbaffd700 1 -- 172.21.15.62:0/3904027738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504df60 msgr2=0x55be3504e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:26.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/3904027738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504df60 0x55be3504e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5cbaffd700 1 -- 172.21.15.62:0/3904027738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be3504d640 msgr2=0x55be3504e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:26.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/3904027738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be3504d640 0x55be3504e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5cbaffd700 1 -- 172.21.15.62:0/3904027738 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be350538b0 con 0x55be35047f20 2024-06-10T00:39:26.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5cbb7fe700 1 --2- 172.21.15.62:0/3904027738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504df60 0x55be3504e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:26.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.833+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/3904027738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be35047f20 0x55be350479e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0cd070 tx=0x7f5cac086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08fcbca5edcee75 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:26.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.834+0000 7f5cb17fa700 1 -- 172.21.15.62:0/3904027738 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0b9070 con 0x55be35047f20 2024-06-10T00:39:26.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.834+0000 7f5cb17fa700 1 -- 172.21.15.62:0/3904027738 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cac0a44b0 con 0x55be35047f20 2024-06-10T00:39:26.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.834+0000 7f5cb17fa700 1 -- 172.21.15.62:0/3904027738 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0b3030 con 0x55be35047f20 2024-06-10T00:39:26.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.834+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/3904027738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be35047f20 msgr2=0x55be350479e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:26.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.834+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/3904027738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be35047f20 0x55be350479e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0cd070 tx=0x7f5cac086040 comp rx=0 tx=0).stop 2024-06-10T00:39:26.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.834+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/3904027738 shutdown_connections 2024-06-10T00:39:26.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.834+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/3904027738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be35047f20 0x55be350479e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.834+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/3904027738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504df60 0x55be3504e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.834+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/3904027738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be3504d640 0x55be3504e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.834+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/3904027738 >> 172.21.15.62:0/3904027738 conn(0x55be34f72cc0 msgr2=0x55be350426f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:26.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.834+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/3904027738 shutdown_connections 2024-06-10T00:39:26.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.834+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/3904027738 wait complete. 2024-06-10T00:39:26.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.835+0000 7f5ccc1c6b80 1 -- start start 2024-06-10T00:39:26.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.835+0000 7f5ccc1c6b80 1 Processor -- start 2024-06-10T00:39:26.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.835+0000 7f5ccc1c6b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be35047f20 0x55be35066db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.835+0000 7f5ccc1c6b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 0x55be35064150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.835+0000 7f5ccc1c6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be3504df60 0x55be35064690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.835+0000 7f5ccc1c6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc1af960 con 0x55be3504df60 2024-06-10T00:39:26.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.835+0000 7f5ccc1c6b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc1af1d0 con 0x55be35047f20 2024-06-10T00:39:26.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.835+0000 7f5ccc1c6b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc1ac3d0 con 0x55be3504d640 2024-06-10T00:39:26.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.835+0000 7f5ccc1c6b80 10 client.0 _ll_register_callbacks cb 0x55be34f33920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:26.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.835+0000 7f5cbb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be3504df60 0x55be35064690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.835+0000 7f5cbb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be3504df60 0x55be35064690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45472/0 (socket says 172.21.15.62:45472) 2024-06-10T00:39:26.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.835+0000 7f5cbb7fe700 1 -- 172.21.15.62:0/230699055 learned_addr learned my addr 172.21.15.62:0/230699055 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:26.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.835+0000 7f5cb27fc700 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 0x55be35064150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.835+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be35047f20 0x55be35066db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5cbb7fe700 1 -- 172.21.15.62:0/230699055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 msgr2=0x55be35064150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:26.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5cbb7fe700 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 0x55be35064150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5cbb7fe700 1 -- 172.21.15.62:0/230699055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be35047f20 msgr2=0x55be35066db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5cbb7fe700 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be35047f20 0x55be35066db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5cbb7fe700 1 -- 172.21.15.62:0/230699055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be35064da0 con 0x55be3504df60 2024-06-10T00:39:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5cb27fc700 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 0x55be35064150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be35047f20 0x55be35066db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5cbb7fe700 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be3504df60 0x55be35064690 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40056d0 tx=0x7f5cb4030f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494081472ece0542 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5c9affd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5c9affd700 1 -- 172.21.15.62:0/230699055 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4021070 con 0x55be3504df60 2024-06-10T00:39:26.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5c9affd700 1 -- 172.21.15.62:0/230699055 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f5cb40090d0 con 0x55be3504df60 2024-06-10T00:39:26.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/230699055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cac0a44b0 con 0x55be3504df60 2024-06-10T00:39:26.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5c9affd700 1 -- 172.21.15.62:0/230699055 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5cb402fa40 con 0x55be3504df60 2024-06-10T00:39:26.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5ccc1c6b80 10 client.16482 fetch_fsmap learned FSMap version 36 2024-06-10T00:39:26.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/230699055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55be350d03b0 con 0x55be3504df60 2024-06-10T00:39:26.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.836+0000 7f5c9affd700 1 -- 172.21.15.62:0/230699055 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cac0a44b0 con 0x55be3504df60 2024-06-10T00:39:26.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.837+0000 7f5c9affd700 1 -- 172.21.15.62:0/230699055 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f5cb4011040 con 0x55be3504df60 2024-06-10T00:39:26.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.837+0000 7f5ccc1c6b80 10 client.16482 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T00:39:26.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.837+0000 7f5ccc1c6b80 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:39:26.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.837+0000 7f5ccc1c6b80 20 client.16482 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:26.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.837+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55be350d07c0 0x55be350d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:26.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.837+0000 7f5ccc1c6b80 4 client.16482 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:39:26.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.837+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/230699055 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55be34fa1290 con 0x55be350d07c0 2024-06-10T00:39:26.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.837+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55be350d07c0 0x55be350d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:26.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.837+0000 7f5c9affd700 10 client.16482 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:26.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.837+0000 7f5cbaffd700 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55be350d07c0 0x55be350d2bb0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85beccf44c0a6f43 server_cookie=21bb19987e2f157b in_seq=0 out_seq=0 2024-06-10T00:39:26.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.837+0000 7f5c9affd700 1 -- 172.21.15.62:0/230699055 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55be34fa1290 con 0x55be350d07c0 2024-06-10T00:39:26.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.837+0000 7f5c9affd700 10 client.16482 handle_command_reply: tid=0 2024-06-10T00:39:26.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.837+0000 7f5ccc1c6b80 1 client.16482 shutdown 2024-06-10T00:39:26.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/230699055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be3504df60 msgr2=0x55be35064690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:26.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be3504df60 0x55be35064690 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40056d0 tx=0x7f5cb4030f70 comp rx=0 tx=0).stop 2024-06-10T00:39:26.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/230699055 shutdown_connections 2024-06-10T00:39:26.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55be350d07c0 0x55be350d2bb0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55be3504df60 0x55be35064690 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be35047f20 0x55be35066db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5ccc1c6b80 1 --2- 172.21.15.62:0/230699055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be3504d640 0x55be35064150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/230699055 >> 172.21.15.62:0/230699055 conn(0x55be34f72cc0 msgr2=0x55be3503be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:26.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5c9affd700 0 client.16482 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:26.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/230699055 shutdown_connections 2024-06-10T00:39:26.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5ccc1c6b80 1 -- 172.21.15.62:0/230699055 wait complete. 2024-06-10T00:39:26.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5ccc1c6b80 20 client.16482 trim_cache size 0 max 16384 2024-06-10T00:39:26.849 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:39:26.849 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-06-10T00:39:26.850 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-06-10T00:39:26.850 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:39:26.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5cc1943700 1 -- 172.21.15.62:0/2492995167 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5ca80a2d90 msgr2=0x7f5ca80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:26.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5cc1943700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5ca80a2d90 0x7f5ca80a5250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc101f90 tx=0x7f5ca4009550 comp rx=0 tx=0).stop 2024-06-10T00:39:26.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5cc1943700 1 -- 172.21.15.62:0/2492995167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cbc102740 msgr2=0x7f5cbc19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:26.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.838+0000 7f5cc1943700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cbc102740 0x7f5cbc19e7a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00d980 tx=0x7f5cac00dd40 comp rx=0 tx=0).stop 2024-06-10T00:39:26.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.839+0000 7f5cc1943700 1 -- 172.21.15.62:0/2492995167 shutdown_connections 2024-06-10T00:39:26.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.839+0000 7f5cc1943700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5ca80a2d90 0x7f5ca80a5250 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.839+0000 7f5cc1943700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5cbc102740 0x7f5cbc19e7a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.839+0000 7f5cc1943700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc107840 0x7f5cbc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.839+0000 7f5cc1943700 1 --2- 172.21.15.62:0/2492995167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5cbc1030f0 0x7f5cbc19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:26.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.839+0000 7f5cc1943700 1 -- 172.21.15.62:0/2492995167 >> 172.21.15.62:0/2492995167 conn(0x7f5cbc0795f0 msgr2=0x7f5cbc100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:26.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.839+0000 7f5cc1943700 1 -- 172.21.15.62:0/2492995167 shutdown_connections 2024-06-10T00:39:26.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:26.839+0000 7f5cc1943700 1 -- 172.21.15.62:0/2492995167 wait complete. 2024-06-10T00:39:27.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:27 smithi062 ceph-mon[32390]: pgmap v201: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:39:27.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:27 smithi138 ceph-mon[36549]: pgmap v201: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:39:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:27 smithi105 ceph-mon[36321]: pgmap v201: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:39:27.738 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T00:39:27.739 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:39:28.037 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:39:29.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:29 smithi062 ceph-mon[32390]: pgmap v202: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:39:29.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:29 smithi138 ceph-mon[36549]: pgmap v202: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:39:29.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:29 smithi105 ceph-mon[36321]: pgmap v202: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:39:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.819+0000 7f9fedc41700 1 -- 172.21.15.62:0/3158001856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe8101610 msgr2=0x7f9fe81019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.819+0000 7f9fedc41700 1 --2- 172.21.15.62:0/3158001856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe8101610 0x7f9fe81019f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80099c0 tx=0x7f9fd8009cd0 comp rx=0 tx=0).stop 2024-06-10T00:39:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.820+0000 7f9fedc41700 1 -- 172.21.15.62:0/3158001856 shutdown_connections 2024-06-10T00:39:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.820+0000 7f9fedc41700 1 --2- 172.21.15.62:0/3158001856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe8101610 0x7f9fe81019f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.820+0000 7f9fedc41700 1 --2- 172.21.15.62:0/3158001856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fe81066f0 0x7f9fe810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.820+0000 7f9fedc41700 1 --2- 172.21.15.62:0/3158001856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fe8101fc0 0x7f9fe8106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:29.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.820+0000 7f9fedc41700 1 -- 172.21.15.62:0/3158001856 >> 172.21.15.62:0/3158001856 conn(0x7f9fe8078430 msgr2=0x7f9fe8078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:29.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.820+0000 7f9fedc41700 1 -- 172.21.15.62:0/3158001856 shutdown_connections 2024-06-10T00:39:29.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.820+0000 7f9fedc41700 1 -- 172.21.15.62:0/3158001856 wait complete. 2024-06-10T00:39:29.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.820+0000 7f9fedc41700 1 Processor -- start 2024-06-10T00:39:29.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.820+0000 7f9fedc41700 1 -- start start 2024-06-10T00:39:29.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fedc41700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fe8101610 0x7f9fe819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:29.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fedc41700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fe8101fc0 0x7f9fe819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:29.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fedc41700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe81066f0 0x7f9fe8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:29.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fedc41700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe81116d0 con 0x7f9fe81066f0 2024-06-10T00:39:29.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fedc41700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe8111550 con 0x7f9fe8101610 2024-06-10T00:39:29.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fedc41700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe8111850 con 0x7f9fe8101fc0 2024-06-10T00:39:29.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fe7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe81066f0 0x7f9fe8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:29.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fe7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe81066f0 0x7f9fe8198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49896/0 (socket says 172.21.15.62:49896) 2024-06-10T00:39:29.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fe7fff700 1 -- 172.21.15.62:0/1826542969 learned_addr learned my addr 172.21.15.62:0/1826542969 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:29.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fe77fe700 1 --2- 172.21.15.62:0/1826542969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fe8101610 0x7f9fe819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:29.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fdeffd700 1 --2- 172.21.15.62:0/1826542969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fe8101fc0 0x7f9fe819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:29.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fe7fff700 1 -- 172.21.15.62:0/1826542969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fe8101fc0 msgr2=0x7f9fe819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:29.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fe7fff700 1 --2- 172.21.15.62:0/1826542969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fe8101fc0 0x7f9fe819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:29.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fe7fff700 1 -- 172.21.15.62:0/1826542969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fe8101610 msgr2=0x7f9fe819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:29.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fe7fff700 1 --2- 172.21.15.62:0/1826542969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fe8101610 0x7f9fe819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:29.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.821+0000 7f9fe7fff700 1 -- 172.21.15.62:0/1826542969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe81991f0 con 0x7f9fe81066f0 2024-06-10T00:39:29.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.822+0000 7f9fe7fff700 1 --2- 172.21.15.62:0/1826542969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe81066f0 0x7f9fe8198a90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0007ee0 tx=0x7f9fe000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:29.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.822+0000 7f9fe57fa700 1 -- 172.21.15.62:0/1826542969 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0004bb0 con 0x7f9fe81066f0 2024-06-10T00:39:29.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.822+0000 7f9fe57fa700 1 -- 172.21.15.62:0/1826542969 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe0004d50 con 0x7f9fe81066f0 2024-06-10T00:39:29.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.822+0000 7f9fedc41700 1 -- 172.21.15.62:0/1826542969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe81994e0 con 0x7f9fe81066f0 2024-06-10T00:39:29.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.822+0000 7f9fe57fa700 1 -- 172.21.15.62:0/1826542969 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0005960 con 0x7f9fe81066f0 2024-06-10T00:39:29.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.822+0000 7f9fedc41700 1 -- 172.21.15.62:0/1826542969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe81a57a0 con 0x7f9fe81066f0 2024-06-10T00:39:29.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.823+0000 7f9fedc41700 1 -- 172.21.15.62:0/1826542969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe8069180 con 0x7f9fe81066f0 2024-06-10T00:39:29.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.824+0000 7f9fe57fa700 1 -- 172.21.15.62:0/1826542969 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9fe001a020 con 0x7f9fe81066f0 2024-06-10T00:39:29.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.825+0000 7f9fe57fa700 1 --2- 172.21.15.62:0/1826542969 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f9fd40a2d90 0x7f9fd40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:29.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.825+0000 7f9fe77fe700 1 --2- 172.21.15.62:0/1826542969 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f9fd40a2d90 0x7f9fd40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:29.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.825+0000 7f9fe57fa700 1 -- 172.21.15.62:0/1826542969 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f9fe00cf480 con 0x7f9fe81066f0 2024-06-10T00:39:29.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.825+0000 7f9fe77fe700 1 --2- 172.21.15.62:0/1826542969 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f9fd40a2d90 0x7f9fd40a5250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8000c00 tx=0x7f9fd8005fb0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:29.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:29.828+0000 7f9fe57fa700 1 -- 172.21.15.62:0/1826542969 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9fe0010070 con 0x7f9fe81066f0 2024-06-10T00:39:30.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:30.079+0000 7f9fedc41700 1 -- 172.21.15.62:0/1826542969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fe802dca0 con 0x7f9fe81066f0 2024-06-10T00:39:30.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:30.080+0000 7f9fe57fa700 1 -- 172.21.15.62:0/1826542969 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11945 (secure 0 0 0) 0x7f9fe009bd90 con 0x7f9fe81066f0 2024-06-10T00:39:30.081 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:39:30.081 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:39:21.957878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24433,24464,24418,24424,24431]},"id":1}]} 2024-06-10T00:39:30.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:30.083+0000 7f9fedc41700 1 -- 172.21.15.62:0/1826542969 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f9fd40a2d90 msgr2=0x7f9fd40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:30.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:30.083+0000 7f9fedc41700 1 --2- 172.21.15.62:0/1826542969 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f9fd40a2d90 0x7f9fd40a5250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8000c00 tx=0x7f9fd8005fb0 comp rx=0 tx=0).stop 2024-06-10T00:39:30.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:30.083+0000 7f9fedc41700 1 -- 172.21.15.62:0/1826542969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe81066f0 msgr2=0x7f9fe8198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:30.083+0000 7f9fedc41700 1 --2- 172.21.15.62:0/1826542969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe81066f0 0x7f9fe8198a90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0007ee0 tx=0x7f9fe000d3b0 comp rx=0 tx=0).stop 2024-06-10T00:39:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:30.083+0000 7f9fedc41700 1 -- 172.21.15.62:0/1826542969 shutdown_connections 2024-06-10T00:39:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:30.083+0000 7f9fedc41700 1 --2- 172.21.15.62:0/1826542969 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f9fd40a2d90 0x7f9fd40a5250 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:30.083+0000 7f9fedc41700 1 --2- 172.21.15.62:0/1826542969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fe81066f0 0x7f9fe8198a90 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:30.083+0000 7f9fedc41700 1 --2- 172.21.15.62:0/1826542969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fe8101610 0x7f9fe819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:30.083+0000 7f9fedc41700 1 --2- 172.21.15.62:0/1826542969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fe8101fc0 0x7f9fe819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:30.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:30.083+0000 7f9fedc41700 1 -- 172.21.15.62:0/1826542969 >> 172.21.15.62:0/1826542969 conn(0x7f9fe8078430 msgr2=0x7f9fe806aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:30.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:30.083+0000 7f9fedc41700 1 -- 172.21.15.62:0/1826542969 shutdown_connections 2024-06-10T00:39:30.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:30.083+0000 7f9fedc41700 1 -- 172.21.15.62:0/1826542969 wait complete. 2024-06-10T00:39:30.085 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 36 2024-06-10T00:39:30.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:30 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1826542969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:39:30.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:30 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1826542969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:39:30.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:30 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1826542969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:39:30.825 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T00:39:31.080 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:39:31.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:31 smithi138 ceph-mon[36549]: pgmap v203: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:31.456 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:31 smithi062 ceph-mon[32390]: pgmap v203: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:31.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:31 smithi105 ceph-mon[36321]: pgmap v203: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:32.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.918+0000 7f0851ce3700 1 -- 172.21.15.62:0/1835005060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f084c102740 msgr2=0x7f084c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:32.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.918+0000 7f0851ce3700 1 --2- 172.21.15.62:0/1835005060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f084c102740 0x7f084c102b20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f083c0099c0 tx=0x7f083c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.918+0000 7f0851ce3700 1 -- 172.21.15.62:0/1835005060 shutdown_connections 2024-06-10T00:39:32.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.918+0000 7f0851ce3700 1 --2- 172.21.15.62:0/1835005060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f084c102740 0x7f084c102b20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.918+0000 7f0851ce3700 1 --2- 172.21.15.62:0/1835005060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c107840 0x7f084c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.918+0000 7f0851ce3700 1 --2- 172.21.15.62:0/1835005060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f084c1030f0 0x7f084c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.918+0000 7f0851ce3700 1 -- 172.21.15.62:0/1835005060 >> 172.21.15.62:0/1835005060 conn(0x7f084c0795f0 msgr2=0x7f084c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:32.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.918+0000 7f0851ce3700 1 -- 172.21.15.62:0/1835005060 shutdown_connections 2024-06-10T00:39:32.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.918+0000 7f0851ce3700 1 -- 172.21.15.62:0/1835005060 wait complete. 2024-06-10T00:39:32.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.918+0000 7f0851ce3700 1 Processor -- start 2024-06-10T00:39:32.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f0851ce3700 1 -- start start 2024-06-10T00:39:32.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f0851ce3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f084c102740 0x7f084c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f0851ce3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f084c1030f0 0x7f084c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f0851ce3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c107840 0x7f084c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f0851ce3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f084c112780 con 0x7f084c102740 2024-06-10T00:39:32.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f0851ce3700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f084c112600 con 0x7f084c107840 2024-06-10T00:39:32.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f0851ce3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f084c112900 con 0x7f084c1030f0 2024-06-10T00:39:32.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f084b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f084c102740 0x7f084c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f084b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f084c102740 0x7f084c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49924/0 (socket says 172.21.15.62:49924) 2024-06-10T00:39:32.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f084b7fe700 1 -- 172.21.15.62:0/551337332 learned_addr learned my addr 172.21.15.62:0/551337332 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:32.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f084affd700 1 --2- 172.21.15.62:0/551337332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f084c1030f0 0x7f084c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f084bfff700 1 --2- 172.21.15.62:0/551337332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c107840 0x7f084c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f084b7fe700 1 -- 172.21.15.62:0/551337332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f084c1030f0 msgr2=0x7f084c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:32.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f084b7fe700 1 --2- 172.21.15.62:0/551337332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f084c1030f0 0x7f084c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f084b7fe700 1 -- 172.21.15.62:0/551337332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c107840 msgr2=0x7f084c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:32.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f084b7fe700 1 --2- 172.21.15.62:0/551337332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c107840 0x7f084c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.919+0000 7f084b7fe700 1 -- 172.21.15.62:0/551337332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c1a7110 con 0x7f084c102740 2024-06-10T00:39:32.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.920+0000 7f084b7fe700 1 --2- 172.21.15.62:0/551337332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f084c102740 0x7f084c074790 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f083c0059d0 tx=0x7f083c0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:32.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.920+0000 7f0848ff9700 1 -- 172.21.15.62:0/551337332 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c027070 con 0x7f084c102740 2024-06-10T00:39:32.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.920+0000 7f0848ff9700 1 -- 172.21.15.62:0/551337332 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f083c004050 con 0x7f084c102740 2024-06-10T00:39:32.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.920+0000 7f0851ce3700 1 -- 172.21.15.62:0/551337332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084c1a73a0 con 0x7f084c102740 2024-06-10T00:39:32.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.920+0000 7f0848ff9700 1 -- 172.21.15.62:0/551337332 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c022db0 con 0x7f084c102740 2024-06-10T00:39:32.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.920+0000 7f0851ce3700 1 -- 172.21.15.62:0/551337332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084c1a77f0 con 0x7f084c102740 2024-06-10T00:39:32.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.922+0000 7f0848ff9700 1 -- 172.21.15.62:0/551337332 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f083c02a070 con 0x7f084c102740 2024-06-10T00:39:32.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.923+0000 7f0848ff9700 1 --2- 172.21.15.62:0/551337332 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f08340a2d90 0x7f08340a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.923+0000 7f084affd700 1 --2- 172.21.15.62:0/551337332 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f08340a2d90 0x7f08340a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.923+0000 7f0848ff9700 1 -- 172.21.15.62:0/551337332 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f083c0d0580 con 0x7f084c102740 2024-06-10T00:39:32.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.923+0000 7f084affd700 1 --2- 172.21.15.62:0/551337332 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f08340a2d90 0x7f08340a5250 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f084c101fd0 tx=0x7f084000a300 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:32.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f085c566b80 1 Processor -- start 2024-06-10T00:39:32.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f085c566b80 1 -- start start 2024-06-10T00:39:32.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f085c566b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4640 0x55da20ca4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f085c566b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4f60 0x55da20c9e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f085c566b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 0x55da20c9f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f085c566b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55da20bf0810 con 0x55da20ca4640 2024-06-10T00:39:32.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f085c566b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55da20b98720 con 0x55da20c9ef20 2024-06-10T00:39:32.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f085c566b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da20bec7d0 con 0x55da20ca4f60 2024-06-10T00:39:32.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f084b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4640 0x55da20ca4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f084b7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4640 0x55da20ca4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49934/0 (socket says 172.21.15.62:49934) 2024-06-10T00:39:32.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f084b7fe700 1 -- 172.21.15.62:0/2896414981 learned_addr learned my addr 172.21.15.62:0/2896414981 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:32.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f084affd700 1 --2- 172.21.15.62:0/2896414981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4f60 0x55da20c9e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f084b7fe700 1 -- 172.21.15.62:0/2896414981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4f60 msgr2=0x55da20c9e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f084b7fe700 1 --2- 172.21.15.62:0/2896414981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4f60 0x55da20c9e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f084b7fe700 1 -- 172.21.15.62:0/2896414981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 msgr2=0x55da20c9f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f084bfff700 1 --2- 172.21.15.62:0/2896414981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 0x55da20c9f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f084b7fe700 1 --2- 172.21.15.62:0/2896414981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 0x55da20c9f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.924+0000 7f084b7fe700 1 -- 172.21.15.62:0/2896414981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da20b8ef50 con 0x55da20ca4640 2024-06-10T00:39:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.925+0000 7f084bfff700 1 --2- 172.21.15.62:0/2896414981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 0x55da20c9f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:39:32.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.925+0000 7f084b7fe700 1 --2- 172.21.15.62:0/2896414981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4640 0x55da20ca4a20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f083c099150 tx=0x7f083c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5e6873aebe4cb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:32.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.925+0000 7f0831ffb700 1 -- 172.21.15.62:0/2896414981 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c09c780 con 0x55da20ca4640 2024-06-10T00:39:32.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.925+0000 7f0831ffb700 1 -- 172.21.15.62:0/2896414981 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f083c0a0b30 con 0x55da20ca4640 2024-06-10T00:39:32.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.925+0000 7f0831ffb700 1 -- 172.21.15.62:0/2896414981 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c0d01b0 con 0x55da20ca4640 2024-06-10T00:39:32.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.926+0000 7f085c566b80 1 -- 172.21.15.62:0/2896414981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4640 msgr2=0x55da20ca4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:32.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.926+0000 7f085c566b80 1 --2- 172.21.15.62:0/2896414981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4640 0x55da20ca4a20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f083c099150 tx=0x7f083c009310 comp rx=0 tx=0).stop 2024-06-10T00:39:32.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.926+0000 7f085c566b80 1 -- 172.21.15.62:0/2896414981 shutdown_connections 2024-06-10T00:39:32.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.926+0000 7f085c566b80 1 --2- 172.21.15.62:0/2896414981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4640 0x55da20ca4a20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.926+0000 7f085c566b80 1 --2- 172.21.15.62:0/2896414981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 0x55da20c9f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.926+0000 7f085c566b80 1 --2- 172.21.15.62:0/2896414981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4f60 0x55da20c9e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.926+0000 7f085c566b80 1 -- 172.21.15.62:0/2896414981 >> 172.21.15.62:0/2896414981 conn(0x55da20bc9cc0 msgr2=0x55da20c99850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:32.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.926+0000 7f085c566b80 1 -- 172.21.15.62:0/2896414981 shutdown_connections 2024-06-10T00:39:32.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.926+0000 7f085c566b80 1 -- 172.21.15.62:0/2896414981 wait complete. 2024-06-10T00:39:32.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.926+0000 7f085c566b80 1 -- start start 2024-06-10T00:39:32.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.926+0000 7f085c566b80 1 Processor -- start 2024-06-10T00:39:32.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f085c566b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20c9ef20 0x55da20caad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f085c566b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4640 0x55da20ca8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f085c566b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20ca4f60 0x55da20ca85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f085c566b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55da20cab6f0 con 0x55da20ca4640 2024-06-10T00:39:32.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f085c566b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55da20b90bf0 con 0x55da20ca4f60 2024-06-10T00:39:32.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f085c566b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da20cab870 con 0x55da20c9ef20 2024-06-10T00:39:32.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f085c566b80 10 client.0 _ll_register_callbacks cb 0x55da20b8a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:32.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f084affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4640 0x55da20ca8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f084affd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4640 0x55da20ca8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49940/0 (socket says 172.21.15.62:49940) 2024-06-10T00:39:32.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f084affd700 1 -- 172.21.15.62:0/3425120912 learned_addr learned my addr 172.21.15.62:0/3425120912 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:32.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f084bfff700 1 --2- 172.21.15.62:0/3425120912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20ca4f60 0x55da20ca85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f084b7fe700 1 --2- 172.21.15.62:0/3425120912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20c9ef20 0x55da20caad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f084affd700 1 -- 172.21.15.62:0/3425120912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20c9ef20 msgr2=0x55da20caad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:32.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f084affd700 1 --2- 172.21.15.62:0/3425120912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20c9ef20 0x55da20caad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f084affd700 1 -- 172.21.15.62:0/3425120912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20ca4f60 msgr2=0x55da20ca85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:32.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f084affd700 1 --2- 172.21.15.62:0/3425120912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20ca4f60 0x55da20ca85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f084affd700 1 -- 172.21.15.62:0/3425120912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da20b3f8d0 con 0x55da20ca4640 2024-06-10T00:39:32.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f084b7fe700 1 --2- 172.21.15.62:0/3425120912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20c9ef20 0x55da20caad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:32.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f084affd700 1 --2- 172.21.15.62:0/3425120912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4640 0x55da20ca8060 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f084001da90 tx=0x7f08400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fea863a1c7cd9ee server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:32.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f082affd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:32.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.927+0000 7f082affd700 1 -- 172.21.15.62:0/3425120912 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840016070 con 0x55da20ca4640 2024-06-10T00:39:32.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.928+0000 7f085c566b80 1 -- 172.21.15.62:0/3425120912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f083c0a0b30 con 0x55da20ca4640 2024-06-10T00:39:32.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.928+0000 7f082affd700 1 -- 172.21.15.62:0/3425120912 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f0840010840 con 0x55da20ca4640 2024-06-10T00:39:32.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.928+0000 7f082affd700 1 -- 172.21.15.62:0/3425120912 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0840032050 con 0x55da20ca4640 2024-06-10T00:39:32.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.928+0000 7f085c566b80 10 client.16500 fetch_fsmap learned FSMap version 36 2024-06-10T00:39:32.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.928+0000 7f085c566b80 1 -- 172.21.15.62:0/3425120912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55da20cb5fe0 con 0x55da20ca4640 2024-06-10T00:39:32.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.928+0000 7f082affd700 1 -- 172.21.15.62:0/3425120912 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08400273f0 con 0x55da20ca4640 2024-06-10T00:39:32.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.928+0000 7f082affd700 1 -- 172.21.15.62:0/3425120912 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f084001b420 con 0x55da20ca4640 2024-06-10T00:39:32.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.928+0000 7f085c566b80 10 client.16500 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T00:39:32.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.928+0000 7f085c566b80 10 client.16500 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:39:32.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.928+0000 7f085c566b80 20 client.16500 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:32.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.928+0000 7f085c566b80 1 --2- 172.21.15.62:0/3425120912 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55da20cb8620 0x55da20cbaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.928+0000 7f085c566b80 4 client.16500 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:39:32.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.928+0000 7f085c566b80 1 -- 172.21.15.62:0/3425120912 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f083c0a0b30 con 0x55da20cb8620 2024-06-10T00:39:32.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.929+0000 7f084b7fe700 1 --2- 172.21.15.62:0/3425120912 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55da20cb8620 0x55da20cbaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.929+0000 7f084b7fe700 1 --2- 172.21.15.62:0/3425120912 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55da20cb8620 0x55da20cbaa10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1ff2b74f6ff70de7 server_cookie=c4958e2bdafeb535 in_seq=0 out_seq=0 2024-06-10T00:39:32.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.929+0000 7f082affd700 10 client.16500 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:39:32.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f082affd700 1 -- 172.21.15.62:0/3425120912 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f083c0a0b30 con 0x55da20cb8620 2024-06-10T00:39:32.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f082affd700 10 client.16500 handle_command_reply: tid=0 2024-06-10T00:39:32.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f085c566b80 1 client.16500 shutdown 2024-06-10T00:39:32.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f085c566b80 1 -- 172.21.15.62:0/3425120912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4640 msgr2=0x55da20ca8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:32.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f085c566b80 1 --2- 172.21.15.62:0/3425120912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4640 0x55da20ca8060 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f084001da90 tx=0x7f08400047c0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f085c566b80 1 -- 172.21.15.62:0/3425120912 shutdown_connections 2024-06-10T00:39:32.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f085c566b80 1 --2- 172.21.15.62:0/3425120912 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55da20cb8620 0x55da20cbaa10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f085c566b80 1 --2- 172.21.15.62:0/3425120912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20ca4f60 0x55da20ca85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f085c566b80 1 --2- 172.21.15.62:0/3425120912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20c9ef20 0x55da20caad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f085c566b80 1 --2- 172.21.15.62:0/3425120912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4640 0x55da20ca8060 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f085c566b80 1 -- 172.21.15.62:0/3425120912 >> 172.21.15.62:0/3425120912 conn(0x55da20bc9cc0 msgr2=0x55da20c9d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:32.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f082affd700 0 client.16500 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:39:32.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f085c566b80 1 -- 172.21.15.62:0/3425120912 shutdown_connections 2024-06-10T00:39:32.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f085c566b80 1 -- 172.21.15.62:0/3425120912 wait complete. 2024-06-10T00:39:32.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.930+0000 7f085c566b80 20 client.16500 trim_cache size 0 max 16384 2024-06-10T00:39:32.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f085c566b80 1 Processor -- start 2024-06-10T00:39:32.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f085c566b80 1 -- start start 2024-06-10T00:39:32.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f085c566b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 0x55da20c9e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f085c566b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4640 0x55da20ca20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f085c566b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4f60 0x55da20ca2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f085c566b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1b0710 con 0x55da20ca4f60 2024-06-10T00:39:32.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f085c566b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1b2310 con 0x55da20c9ef20 2024-06-10T00:39:32.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f085c566b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1b0de0 con 0x55da20ca4640 2024-06-10T00:39:32.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f084bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4f60 0x55da20ca2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f084bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4f60 0x55da20ca2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49954/0 (socket says 172.21.15.62:49954) 2024-06-10T00:39:32.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f084bfff700 1 -- 172.21.15.62:0/480743150 learned_addr learned my addr 172.21.15.62:0/480743150 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:32.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f084affd700 1 --2- 172.21.15.62:0/480743150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4640 0x55da20ca20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f084b7fe700 1 --2- 172.21.15.62:0/480743150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 0x55da20c9e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f084bfff700 1 -- 172.21.15.62:0/480743150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4640 msgr2=0x55da20ca20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:32.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f084bfff700 1 --2- 172.21.15.62:0/480743150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4640 0x55da20ca20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f084bfff700 1 -- 172.21.15.62:0/480743150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 msgr2=0x55da20c9e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:32.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f084bfff700 1 --2- 172.21.15.62:0/480743150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 0x55da20c9e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f084bfff700 1 -- 172.21.15.62:0/480743150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da20ca2b40 con 0x55da20ca4f60 2024-06-10T00:39:32.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.946+0000 7f084bfff700 1 --2- 172.21.15.62:0/480743150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4f60 0x55da20ca2600 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0844009c20 tx=0x7f0844009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f151ecd72de444d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:32.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.947+0000 7f08317fa700 1 -- 172.21.15.62:0/480743150 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084401b020 con 0x55da20ca4f60 2024-06-10T00:39:32.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.947+0000 7f08317fa700 1 -- 172.21.15.62:0/480743150 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0844007cc0 con 0x55da20ca4f60 2024-06-10T00:39:32.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.947+0000 7f08317fa700 1 -- 172.21.15.62:0/480743150 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844016750 con 0x55da20ca4f60 2024-06-10T00:39:32.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.947+0000 7f085c566b80 1 -- 172.21.15.62:0/480743150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4f60 msgr2=0x55da20ca2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:32.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.947+0000 7f085c566b80 1 --2- 172.21.15.62:0/480743150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4f60 0x55da20ca2600 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0844009c20 tx=0x7f0844009040 comp rx=0 tx=0).stop 2024-06-10T00:39:32.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.947+0000 7f085c566b80 1 -- 172.21.15.62:0/480743150 shutdown_connections 2024-06-10T00:39:32.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.947+0000 7f085c566b80 1 --2- 172.21.15.62:0/480743150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4f60 0x55da20ca2600 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.947+0000 7f085c566b80 1 --2- 172.21.15.62:0/480743150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 0x55da20c9e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.947+0000 7f085c566b80 1 --2- 172.21.15.62:0/480743150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4640 0x55da20ca20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.947+0000 7f085c566b80 1 -- 172.21.15.62:0/480743150 >> 172.21.15.62:0/480743150 conn(0x55da20bc9cc0 msgr2=0x55da20ca6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:32.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.947+0000 7f085c566b80 1 -- 172.21.15.62:0/480743150 shutdown_connections 2024-06-10T00:39:32.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.947+0000 7f085c566b80 1 -- 172.21.15.62:0/480743150 wait complete. 2024-06-10T00:39:32.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.948+0000 7f085c566b80 1 -- start start 2024-06-10T00:39:32.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.948+0000 7f085c566b80 1 Processor -- start 2024-06-10T00:39:32.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.948+0000 7f085c566b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 0x55da20cb0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.948+0000 7f085c566b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4640 0x55da20cb11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.948+0000 7f085c566b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4f60 0x55da20cbf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.948+0000 7f085c566b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1b1c40 con 0x55da20ca4f60 2024-06-10T00:39:32.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.948+0000 7f085c566b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1b14b0 con 0x55da20c9ef20 2024-06-10T00:39:32.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.948+0000 7f085c566b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1ae6b0 con 0x55da20ca4640 2024-06-10T00:39:32.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.948+0000 7f085c566b80 10 client.0 _ll_register_callbacks cb 0x55da20b8a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:32.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.948+0000 7f084bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4f60 0x55da20cbf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.948+0000 7f084bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4f60 0x55da20cbf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49970/0 (socket says 172.21.15.62:49970) 2024-06-10T00:39:32.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.948+0000 7f084bfff700 1 -- 172.21.15.62:0/462081059 learned_addr learned my addr 172.21.15.62:0/462081059 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:32.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f084b7fe700 1 --2- 172.21.15.62:0/462081059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 0x55da20cb0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f084affd700 1 --2- 172.21.15.62:0/462081059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4640 0x55da20cb11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f084bfff700 1 -- 172.21.15.62:0/462081059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4640 msgr2=0x55da20cb11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:32.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f084bfff700 1 --2- 172.21.15.62:0/462081059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4640 0x55da20cb11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f084bfff700 1 -- 172.21.15.62:0/462081059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 msgr2=0x55da20cb0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:32.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f084bfff700 1 --2- 172.21.15.62:0/462081059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 0x55da20cb0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f084bfff700 1 -- 172.21.15.62:0/462081059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da20cbfd50 con 0x55da20ca4f60 2024-06-10T00:39:32.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f084affd700 1 --2- 172.21.15.62:0/462081059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4640 0x55da20cb11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:39:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f084bfff700 1 --2- 172.21.15.62:0/462081059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4f60 0x55da20cbf690 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f0844009bf0 tx=0x7f0844005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb6a9631b7b4fc1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f082affd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f082affd700 1 -- 172.21.15.62:0/462081059 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844020630 con 0x55da20ca4f60 2024-06-10T00:39:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f082affd700 1 -- 172.21.15.62:0/462081059 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f08440207d0 con 0x55da20ca4f60 2024-06-10T00:39:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f085c566b80 1 -- 172.21.15.62:0/462081059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0844007cc0 con 0x55da20ca4f60 2024-06-10T00:39:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f082affd700 1 -- 172.21.15.62:0/462081059 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0844033050 con 0x55da20ca4f60 2024-06-10T00:39:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f085c566b80 10 client.16506 fetch_fsmap learned FSMap version 36 2024-06-10T00:39:32.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.949+0000 7f085c566b80 1 -- 172.21.15.62:0/462081059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55da20d22c10 con 0x55da20ca4f60 2024-06-10T00:39:32.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.950+0000 7f082affd700 1 -- 172.21.15.62:0/462081059 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0844007cc0 con 0x55da20ca4f60 2024-06-10T00:39:32.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.950+0000 7f082affd700 1 -- 172.21.15.62:0/462081059 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f0844028020 con 0x55da20ca4f60 2024-06-10T00:39:32.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.950+0000 7f085c566b80 10 client.16506 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T00:39:32.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.950+0000 7f085c566b80 10 client.16506 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:39:32.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.950+0000 7f085c566b80 20 client.16506 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:32.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.950+0000 7f085c566b80 1 --2- 172.21.15.62:0/462081059 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55da20d23240 0x55da20d25660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:32.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.950+0000 7f085c566b80 4 client.16506 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:39:32.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.950+0000 7f085c566b80 1 -- 172.21.15.62:0/462081059 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da20bfdd10 con 0x55da20d23240 2024-06-10T00:39:32.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.950+0000 7f084b7fe700 1 --2- 172.21.15.62:0/462081059 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55da20d23240 0x55da20d25660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:32.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f082affd700 10 client.16506 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:39:32.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f084b7fe700 1 --2- 172.21.15.62:0/462081059 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55da20d23240 0x55da20d25660 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b898248fb6b30c2a server_cookie=c783df0327405d02 in_seq=0 out_seq=0 2024-06-10T00:39:32.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f082affd700 1 -- 172.21.15.62:0/462081059 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da20bfdd10 con 0x55da20d23240 2024-06-10T00:39:32.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f082affd700 10 client.16506 handle_command_reply: tid=0 2024-06-10T00:39:32.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f085c566b80 1 client.16506 shutdown 2024-06-10T00:39:32.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f085c566b80 1 -- 172.21.15.62:0/462081059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4f60 msgr2=0x55da20cbf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:32.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f085c566b80 1 --2- 172.21.15.62:0/462081059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4f60 0x55da20cbf690 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f0844009bf0 tx=0x7f0844005ae0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f085c566b80 1 -- 172.21.15.62:0/462081059 shutdown_connections 2024-06-10T00:39:32.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f085c566b80 1 --2- 172.21.15.62:0/462081059 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55da20d23240 0x55da20d25660 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f085c566b80 1 --2- 172.21.15.62:0/462081059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da20ca4f60 0x55da20cbf690 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f085c566b80 1 --2- 172.21.15.62:0/462081059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55da20c9ef20 0x55da20cb0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f085c566b80 1 --2- 172.21.15.62:0/462081059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da20ca4640 0x55da20cb11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f085c566b80 1 -- 172.21.15.62:0/462081059 >> 172.21.15.62:0/462081059 conn(0x55da20bc9cc0 msgr2=0x55da20ca6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f082affd700 0 client.16506 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:39:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f085c566b80 1 -- 172.21.15.62:0/462081059 shutdown_connections 2024-06-10T00:39:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f085c566b80 1 -- 172.21.15.62:0/462081059 wait complete. 2024-06-10T00:39:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.951+0000 7f085c566b80 20 client.16506 trim_cache size 0 max 16384 2024-06-10T00:39:32.963 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:39:32.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.952+0000 7f0851ce3700 1 -- 172.21.15.62:0/551337332 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f08340a2d90 msgr2=0x7f08340a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:32.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.952+0000 7f0851ce3700 1 --2- 172.21.15.62:0/551337332 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f08340a2d90 0x7f08340a5250 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f084c101fd0 tx=0x7f084000a300 comp rx=0 tx=0).stop 2024-06-10T00:39:32.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.952+0000 7f0851ce3700 1 -- 172.21.15.62:0/551337332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f084c102740 msgr2=0x7f084c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:32.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.952+0000 7f0851ce3700 1 --2- 172.21.15.62:0/551337332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f084c102740 0x7f084c074790 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f083c0059d0 tx=0x7f083c0038b0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.952+0000 7f0851ce3700 1 -- 172.21.15.62:0/551337332 shutdown_connections 2024-06-10T00:39:32.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.952+0000 7f0851ce3700 1 --2- 172.21.15.62:0/551337332 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f08340a2d90 0x7f08340a5250 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.952+0000 7f0851ce3700 1 --2- 172.21.15.62:0/551337332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f084c102740 0x7f084c074790 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.952+0000 7f0851ce3700 1 --2- 172.21.15.62:0/551337332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c107840 0x7f084c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.952+0000 7f0851ce3700 1 --2- 172.21.15.62:0/551337332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f084c1030f0 0x7f084c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:32.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.952+0000 7f0851ce3700 1 -- 172.21.15.62:0/551337332 >> 172.21.15.62:0/551337332 conn(0x7f084c0795f0 msgr2=0x7f084c108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:32.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.952+0000 7f0851ce3700 1 -- 172.21.15.62:0/551337332 shutdown_connections 2024-06-10T00:39:32.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:32.952+0000 7f0851ce3700 1 -- 172.21.15.62:0/551337332 wait complete. 2024-06-10T00:39:33.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:33 smithi062 ceph-mon[32390]: pgmap v204: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T00:39:33.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:33 smithi138 ceph-mon[36549]: pgmap v204: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T00:39:33.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:33 smithi105 ceph-mon[36321]: pgmap v204: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T00:39:33.679 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T00:39:33.931 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:39:35.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:35 smithi062 ceph-mon[32390]: pgmap v205: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:35.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:35 smithi105 ceph-mon[36321]: pgmap v205: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:35.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:35 smithi138 ceph-mon[36549]: pgmap v205: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:35.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.718+0000 7fe7c5e8b700 1 -- 172.21.15.62:0/1181617736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c0102770 msgr2=0x7fe7c0102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:35.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.718+0000 7fe7c5e8b700 1 --2- 172.21.15.62:0/1181617736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c0102770 0x7fe7c0102b50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe7b00099c0 tx=0x7fe7b0009cd0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.718+0000 7fe7c5e8b700 1 -- 172.21.15.62:0/1181617736 shutdown_connections 2024-06-10T00:39:35.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.718+0000 7fe7c5e8b700 1 --2- 172.21.15.62:0/1181617736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c0102770 0x7fe7c0102b50 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.718+0000 7fe7c5e8b700 1 --2- 172.21.15.62:0/1181617736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7c0107860 0x7fe7c010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.718+0000 7fe7c5e8b700 1 --2- 172.21.15.62:0/1181617736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c0103120 0x7fe7c0107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.718+0000 7fe7c5e8b700 1 -- 172.21.15.62:0/1181617736 >> 172.21.15.62:0/1181617736 conn(0x7fe7c0079600 msgr2=0x7fe7c0079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:35.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.718+0000 7fe7c5e8b700 1 -- 172.21.15.62:0/1181617736 shutdown_connections 2024-06-10T00:39:35.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.718+0000 7fe7c5e8b700 1 -- 172.21.15.62:0/1181617736 wait complete. 2024-06-10T00:39:35.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7c5e8b700 1 Processor -- start 2024-06-10T00:39:35.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7c5e8b700 1 -- start start 2024-06-10T00:39:35.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7c5e8b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7c0102770 0x7fe7c0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7c5e8b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c0103120 0x7fe7c0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7c5e8b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c0107860 0x7fe7c0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7c5e8b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01127a0 con 0x7fe7c0103120 2024-06-10T00:39:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7c5e8b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0112620 con 0x7fe7c0102770 2024-06-10T00:39:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7c5e8b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0112920 con 0x7fe7c0107860 2024-06-10T00:39:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7beffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c0103120 0x7fe7c0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7beffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c0103120 0x7fe7c0074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49992/0 (socket says 172.21.15.62:49992) 2024-06-10T00:39:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7beffd700 1 -- 172.21.15.62:0/3838768502 learned_addr learned my addr 172.21.15.62:0/3838768502 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:35.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7bf7fe700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7c0102770 0x7fe7c0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7bffff700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c0107860 0x7fe7c0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7beffd700 1 -- 172.21.15.62:0/3838768502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c0107860 msgr2=0x7fe7c0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:35.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7beffd700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c0107860 0x7fe7c0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.719+0000 7fe7beffd700 1 -- 172.21.15.62:0/3838768502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7c0102770 msgr2=0x7fe7c0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:35.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.720+0000 7fe7beffd700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7c0102770 0x7fe7c0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.720+0000 7fe7beffd700 1 -- 172.21.15.62:0/3838768502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c01a70d0 con 0x7fe7c0103120 2024-06-10T00:39:35.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.720+0000 7fe7bffff700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c0107860 0x7fe7c0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:39:35.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.720+0000 7fe7beffd700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c0103120 0x7fe7c0074cd0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400d980 tx=0x7fe7b400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:35.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.720+0000 7fe7bcff9700 1 -- 172.21.15.62:0/3838768502 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4017070 con 0x7fe7c0103120 2024-06-10T00:39:35.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.720+0000 7fe7bcff9700 1 -- 172.21.15.62:0/3838768502 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b4004d10 con 0x7fe7c0103120 2024-06-10T00:39:35.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.720+0000 7fe7bcff9700 1 -- 172.21.15.62:0/3838768502 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40055c0 con 0x7fe7c0103120 2024-06-10T00:39:35.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.720+0000 7fe7c5e8b700 1 -- 172.21.15.62:0/3838768502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c01a73c0 con 0x7fe7c0103120 2024-06-10T00:39:35.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.720+0000 7fe7c5e8b700 1 -- 172.21.15.62:0/3838768502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c01a78b0 con 0x7fe7c0103120 2024-06-10T00:39:35.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.722+0000 7fe7bcff9700 1 -- 172.21.15.62:0/3838768502 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe7b4004750 con 0x7fe7c0103120 2024-06-10T00:39:35.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.723+0000 7fe7bcff9700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe7a80a2d90 0x7fe7a80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.723+0000 7fe7bcff9700 1 -- 172.21.15.62:0/3838768502 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fe7b40cf7b0 con 0x7fe7c0103120 2024-06-10T00:39:35.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.723+0000 7fe7bf7fe700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe7a80a2d90 0x7fe7a80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.724+0000 7fe7bf7fe700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe7a80a2d90 0x7fe7a80a5250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0000c00 tx=0x7fe7b0005fd0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:35.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.724+0000 7fe7d070eb80 1 Processor -- start 2024-06-10T00:39:35.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.724+0000 7fe7d070eb80 1 -- start start 2024-06-10T00:39:35.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.724+0000 7fe7d070eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987ca3f60 0x556987ca4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.724+0000 7fe7d070eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca4880 0x556987c9e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.724+0000 7fe7d070eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987c9e7a0 0x556987c9ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.724+0000 7fe7d070eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556987b90f60 con 0x556987ca4880 2024-06-10T00:39:35.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.724+0000 7fe7d070eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556987bef850 con 0x556987c9e7a0 2024-06-10T00:39:35.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.724+0000 7fe7d070eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556987b57460 con 0x556987ca3f60 2024-06-10T00:39:35.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.724+0000 7fe7beffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca4880 0x556987c9e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.724+0000 7fe7beffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca4880 0x556987c9e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49996/0 (socket says 172.21.15.62:49996) 2024-06-10T00:39:35.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.724+0000 7fe7beffd700 1 -- 172.21.15.62:0/2879226437 learned_addr learned my addr 172.21.15.62:0/2879226437 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:35.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.724+0000 7fe7bf7fe700 1 --2- 172.21.15.62:0/2879226437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987ca3f60 0x556987ca4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.724+0000 7fe7bffff700 1 --2- 172.21.15.62:0/2879226437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987c9e7a0 0x556987c9ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.725+0000 7fe7beffd700 1 -- 172.21.15.62:0/2879226437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987ca3f60 msgr2=0x556987ca4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:35.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.725+0000 7fe7beffd700 1 --2- 172.21.15.62:0/2879226437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987ca3f60 0x556987ca4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.725+0000 7fe7beffd700 1 -- 172.21.15.62:0/2879226437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987c9e7a0 msgr2=0x556987c9ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:35.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.725+0000 7fe7beffd700 1 --2- 172.21.15.62:0/2879226437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987c9e7a0 0x556987c9ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.725+0000 7fe7beffd700 1 -- 172.21.15.62:0/2879226437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556987b3e8d0 con 0x556987ca4880 2024-06-10T00:39:35.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.725+0000 7fe7beffd700 1 --2- 172.21.15.62:0/2879226437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca4880 0x556987c9e260 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40982f0 tx=0x7fe7b409b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf68aee925f4622 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:35.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.725+0000 7fe7a5ffb700 1 -- 172.21.15.62:0/2879226437 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4084e60 con 0x556987ca4880 2024-06-10T00:39:35.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.725+0000 7fe7a5ffb700 1 -- 172.21.15.62:0/2879226437 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b400ee10 con 0x556987ca4880 2024-06-10T00:39:35.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.725+0000 7fe7a5ffb700 1 -- 172.21.15.62:0/2879226437 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40cf3c0 con 0x556987ca4880 2024-06-10T00:39:35.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.726+0000 7fe7d070eb80 1 -- 172.21.15.62:0/2879226437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca4880 msgr2=0x556987c9e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:35.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.726+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/2879226437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca4880 0x556987c9e260 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40982f0 tx=0x7fe7b409b570 comp rx=0 tx=0).stop 2024-06-10T00:39:35.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.726+0000 7fe7d070eb80 1 -- 172.21.15.62:0/2879226437 shutdown_connections 2024-06-10T00:39:35.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.726+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/2879226437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987c9e7a0 0x556987c9ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.726+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/2879226437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987ca3f60 0x556987ca4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.726+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/2879226437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca4880 0x556987c9e260 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.726+0000 7fe7d070eb80 1 -- 172.21.15.62:0/2879226437 >> 172.21.15.62:0/2879226437 conn(0x556987bcb080 msgr2=0x556987c99220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.726+0000 7fe7d070eb80 1 -- 172.21.15.62:0/2879226437 shutdown_connections 2024-06-10T00:39:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.726+0000 7fe7d070eb80 1 -- 172.21.15.62:0/2879226437 wait complete. 2024-06-10T00:39:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.726+0000 7fe7d070eb80 1 -- start start 2024-06-10T00:39:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.726+0000 7fe7d070eb80 1 Processor -- start 2024-06-10T00:39:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7d070eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987c9e7a0 0x556987caa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7d070eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca3f60 0x556987ca7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7d070eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987ca4880 0x556987ca7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7d070eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556987b8ffc0 con 0x556987ca3f60 2024-06-10T00:39:35.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7d070eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556987beb810 con 0x556987c9e7a0 2024-06-10T00:39:35.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7d070eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556987cab200 con 0x556987ca4880 2024-06-10T00:39:35.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7d070eb80 10 client.0 _ll_register_callbacks cb 0x556987b855b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:35.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7beffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca3f60 0x556987ca7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7beffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca3f60 0x556987ca7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49998/0 (socket says 172.21.15.62:49998) 2024-06-10T00:39:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7beffd700 1 -- 172.21.15.62:0/2770649034 learned_addr learned my addr 172.21.15.62:0/2770649034 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7bf7fe700 1 --2- 172.21.15.62:0/2770649034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987c9e7a0 0x556987caa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7bffff700 1 --2- 172.21.15.62:0/2770649034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987ca4880 0x556987ca7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7beffd700 1 -- 172.21.15.62:0/2770649034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987ca4880 msgr2=0x556987ca7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:35.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7beffd700 1 --2- 172.21.15.62:0/2770649034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987ca4880 0x556987ca7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7beffd700 1 -- 172.21.15.62:0/2770649034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987c9e7a0 msgr2=0x556987caa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:35.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7beffd700 1 --2- 172.21.15.62:0/2770649034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987c9e7a0 0x556987caa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7beffd700 1 -- 172.21.15.62:0/2770649034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556987bb5750 con 0x556987ca3f60 2024-06-10T00:39:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe7beffd700 1 --2- 172.21.15.62:0/2770649034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca3f60 0x556987ca7910 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe7b409b8b0 tx=0x7fe7b4009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278e368935048a79 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.727+0000 7fe79effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.728+0000 7fe79effd700 1 -- 172.21.15.62:0/2770649034 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40b8070 con 0x556987ca3f60 2024-06-10T00:39:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.728+0000 7fe7d070eb80 1 -- 172.21.15.62:0/2770649034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b400ee10 con 0x556987ca3f60 2024-06-10T00:39:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.728+0000 7fe79effd700 1 -- 172.21.15.62:0/2770649034 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fe7b40ba2c0 con 0x556987ca3f60 2024-06-10T00:39:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.728+0000 7fe79effd700 1 -- 172.21.15.62:0/2770649034 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe7b4084e60 con 0x556987ca3f60 2024-06-10T00:39:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.728+0000 7fe7d070eb80 10 client.16518 fetch_fsmap learned FSMap version 36 2024-06-10T00:39:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.728+0000 7fe7d070eb80 1 -- 172.21.15.62:0/2770649034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556987cb5980 con 0x556987ca3f60 2024-06-10T00:39:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.728+0000 7fe79effd700 1 -- 172.21.15.62:0/2770649034 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b400ee10 con 0x556987ca3f60 2024-06-10T00:39:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.728+0000 7fe79effd700 1 -- 172.21.15.62:0/2770649034 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fe7b409bb80 con 0x556987ca3f60 2024-06-10T00:39:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.728+0000 7fe7d070eb80 10 client.16518 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T00:39:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.728+0000 7fe7d070eb80 10 client.16518 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:39:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.729+0000 7fe7d070eb80 20 client.16518 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.729+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/2770649034 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x556987cb7e50 0x556987cba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.729+0000 7fe7d070eb80 4 client.16518 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:39:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.729+0000 7fe7d070eb80 1 -- 172.21.15.62:0/2770649034 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556987bfcd50 con 0x556987cb7e50 2024-06-10T00:39:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.729+0000 7fe7bf7fe700 1 --2- 172.21.15.62:0/2770649034 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x556987cb7e50 0x556987cba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.729+0000 7fe7bf7fe700 1 --2- 172.21.15.62:0/2770649034 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x556987cb7e50 0x556987cba240 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fea95c7447482e88 server_cookie=b5b98455765ac8cf in_seq=0 out_seq=0 2024-06-10T00:39:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.729+0000 7fe79effd700 10 client.16518 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:39:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.730+0000 7fe79effd700 1 -- 172.21.15.62:0/2770649034 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556987bfcd50 con 0x556987cb7e50 2024-06-10T00:39:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.730+0000 7fe79effd700 10 client.16518 handle_command_reply: tid=0 2024-06-10T00:39:35.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.730+0000 7fe7d070eb80 1 client.16518 shutdown 2024-06-10T00:39:35.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.730+0000 7fe7d070eb80 1 -- 172.21.15.62:0/2770649034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca3f60 msgr2=0x556987ca7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:35.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.730+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/2770649034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca3f60 0x556987ca7910 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe7b409b8b0 tx=0x7fe7b4009e00 comp rx=0 tx=0).stop 2024-06-10T00:39:35.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.730+0000 7fe7d070eb80 1 -- 172.21.15.62:0/2770649034 shutdown_connections 2024-06-10T00:39:35.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.730+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/2770649034 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x556987cb7e50 0x556987cba240 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.730+0000 7fe79effd700 0 client.16518 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:39:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.730+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/2770649034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987ca4880 0x556987ca7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.730+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/2770649034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987c9e7a0 0x556987caa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.730+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/2770649034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca3f60 0x556987ca7910 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.730+0000 7fe7d070eb80 1 -- 172.21.15.62:0/2770649034 >> 172.21.15.62:0/2770649034 conn(0x556987bcb080 msgr2=0x556987c98700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.730+0000 7fe7d070eb80 1 -- 172.21.15.62:0/2770649034 shutdown_connections 2024-06-10T00:39:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.730+0000 7fe7d070eb80 1 -- 172.21.15.62:0/2770649034 wait complete. 2024-06-10T00:39:35.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.731+0000 7fe7d070eb80 20 client.16518 trim_cache size 0 max 16384 2024-06-10T00:39:35.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.746+0000 7fe7d070eb80 1 Processor -- start 2024-06-10T00:39:35.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.746+0000 7fe7d070eb80 1 -- start start 2024-06-10T00:39:35.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.746+0000 7fe7d070eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987c9e7a0 0x556987c9e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.746+0000 7fe7d070eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca3f60 0x556987ca5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.746+0000 7fe7d070eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987ca4880 0x556987caa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.746+0000 7fe7d070eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01b0850 con 0x556987ca3f60 2024-06-10T00:39:35.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.746+0000 7fe7d070eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01b2450 con 0x556987ca4880 2024-06-10T00:39:35.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.746+0000 7fe7d070eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01b0f20 con 0x556987c9e7a0 2024-06-10T00:39:35.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.746+0000 7fe7beffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca3f60 0x556987ca5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.746+0000 7fe7beffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca3f60 0x556987ca5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50004/0 (socket says 172.21.15.62:50004) 2024-06-10T00:39:35.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.746+0000 7fe7beffd700 1 -- 172.21.15.62:0/3191744586 learned_addr learned my addr 172.21.15.62:0/3191744586 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:35.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.746+0000 7fe7bf7fe700 1 --2- 172.21.15.62:0/3191744586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987c9e7a0 0x556987c9e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.747+0000 7fe7beffd700 1 -- 172.21.15.62:0/3191744586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987c9e7a0 msgr2=0x556987c9e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:35.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.747+0000 7fe7beffd700 1 --2- 172.21.15.62:0/3191744586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987c9e7a0 0x556987c9e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.747+0000 7fe7beffd700 1 -- 172.21.15.62:0/3191744586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987ca4880 msgr2=0x556987caa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:35.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.747+0000 7fe7beffd700 1 --2- 172.21.15.62:0/3191744586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987ca4880 0x556987caa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.747+0000 7fe7beffd700 1 -- 172.21.15.62:0/3191744586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556987c9a040 con 0x556987ca3f60 2024-06-10T00:39:35.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.747+0000 7fe7bf7fe700 1 --2- 172.21.15.62:0/3191744586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987c9e7a0 0x556987c9e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:35.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.747+0000 7fe7beffd700 1 --2- 172.21.15.62:0/3191744586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca3f60 0x556987ca5430 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40cd150 tx=0x7fe7b409b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97159fd7bd47d5e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:35.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.747+0000 7fe7a57fa700 1 -- 172.21.15.62:0/3191744586 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40b8070 con 0x556987ca3f60 2024-06-10T00:39:35.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.747+0000 7fe7a57fa700 1 -- 172.21.15.62:0/3191744586 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b40a4590 con 0x556987ca3f60 2024-06-10T00:39:35.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.747+0000 7fe7a57fa700 1 -- 172.21.15.62:0/3191744586 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40860d0 con 0x556987ca3f60 2024-06-10T00:39:35.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.748+0000 7fe7d070eb80 1 -- 172.21.15.62:0/3191744586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca3f60 msgr2=0x556987ca5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:35.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.748+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/3191744586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca3f60 0x556987ca5430 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40cd150 tx=0x7fe7b409b550 comp rx=0 tx=0).stop 2024-06-10T00:39:35.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.748+0000 7fe7d070eb80 1 -- 172.21.15.62:0/3191744586 shutdown_connections 2024-06-10T00:39:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.748+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/3191744586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987ca4880 0x556987caa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.748+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/3191744586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987c9e7a0 0x556987c9e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.748+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/3191744586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca3f60 0x556987ca5430 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.748+0000 7fe7d070eb80 1 -- 172.21.15.62:0/3191744586 >> 172.21.15.62:0/3191744586 conn(0x556987bcb080 msgr2=0x556987ca4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.748+0000 7fe7d070eb80 1 -- 172.21.15.62:0/3191744586 shutdown_connections 2024-06-10T00:39:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.748+0000 7fe7d070eb80 1 -- 172.21.15.62:0/3191744586 wait complete. 2024-06-10T00:39:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.748+0000 7fe7d070eb80 1 -- start start 2024-06-10T00:39:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.748+0000 7fe7d070eb80 1 Processor -- start 2024-06-10T00:39:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.748+0000 7fe7d070eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987c9e7a0 0x556987cbd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.748+0000 7fe7d070eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987ca3f60 0x556987cbab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.748+0000 7fe7d070eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca4880 0x556987cbb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7d070eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01b1d80 con 0x556987ca4880 2024-06-10T00:39:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7d070eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01b15f0 con 0x556987ca3f60 2024-06-10T00:39:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7d070eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01ae7f0 con 0x556987c9e7a0 2024-06-10T00:39:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7d070eb80 10 client.0 _ll_register_callbacks cb 0x556987b855b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7bffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca4880 0x556987cbb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7bffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca4880 0x556987cbb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50014/0 (socket says 172.21.15.62:50014) 2024-06-10T00:39:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7bffff700 1 -- 172.21.15.62:0/4154743297 learned_addr learned my addr 172.21.15.62:0/4154743297 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7bf7fe700 1 --2- 172.21.15.62:0/4154743297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987c9e7a0 0x556987cbd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7beffd700 1 --2- 172.21.15.62:0/4154743297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987ca3f60 0x556987cbab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7bffff700 1 -- 172.21.15.62:0/4154743297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987c9e7a0 msgr2=0x556987cbd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7bffff700 1 --2- 172.21.15.62:0/4154743297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987c9e7a0 0x556987cbd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7bffff700 1 -- 172.21.15.62:0/4154743297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987ca3f60 msgr2=0x556987cbab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:35.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7bffff700 1 --2- 172.21.15.62:0/4154743297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987ca3f60 0x556987cbab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7bffff700 1 -- 172.21.15.62:0/4154743297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556987cbb6f0 con 0x556987ca4880 2024-06-10T00:39:35.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7beffd700 1 --2- 172.21.15.62:0/4154743297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987ca3f60 0x556987cbab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:35.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7bffff700 1 --2- 172.21.15.62:0/4154743297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca4880 0x556987cbb0b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8007660 tx=0x7fe7b8007970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd3abd5461f6271 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:35.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe79effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:35.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe79effd700 1 -- 172.21.15.62:0/4154743297 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b801a020 con 0x556987ca4880 2024-06-10T00:39:35.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe79effd700 1 -- 172.21.15.62:0/4154743297 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fe7b8016610 con 0x556987ca4880 2024-06-10T00:39:35.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.749+0000 7fe7d070eb80 1 -- 172.21.15.62:0/4154743297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b40a4590 con 0x556987ca4880 2024-06-10T00:39:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.750+0000 7fe79effd700 1 -- 172.21.15.62:0/4154743297 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe7b8025050 con 0x556987ca4880 2024-06-10T00:39:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.750+0000 7fe7d070eb80 10 client.16524 fetch_fsmap learned FSMap version 36 2024-06-10T00:39:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.750+0000 7fe7d070eb80 1 -- 172.21.15.62:0/4154743297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556987d22620 con 0x556987ca4880 2024-06-10T00:39:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.750+0000 7fe79effd700 1 -- 172.21.15.62:0/4154743297 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b40a4590 con 0x556987ca4880 2024-06-10T00:39:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.750+0000 7fe79effd700 1 -- 172.21.15.62:0/4154743297 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fe7b801ea10 con 0x556987ca4880 2024-06-10T00:39:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.750+0000 7fe7d070eb80 10 client.16524 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T00:39:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.750+0000 7fe7d070eb80 10 client.16524 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:39:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.750+0000 7fe7d070eb80 20 client.16524 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.750+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/4154743297 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x556987cb7e50 0x556987d24d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.750+0000 7fe7d070eb80 4 client.16524 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:39:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.750+0000 7fe7d070eb80 1 -- 172.21.15.62:0/4154743297 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556987bf72d0 con 0x556987cb7e50 2024-06-10T00:39:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.750+0000 7fe7bf7fe700 1 --2- 172.21.15.62:0/4154743297 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x556987cb7e50 0x556987d24d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe7bf7fe700 1 --2- 172.21.15.62:0/4154743297 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x556987cb7e50 0x556987d24d60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=27aed2c06e621926 server_cookie=e477b9bb4eaa1d8a in_seq=0 out_seq=0 2024-06-10T00:39:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe79effd700 10 client.16524 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:39:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe79effd700 1 -- 172.21.15.62:0/4154743297 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556987bf72d0 con 0x556987cb7e50 2024-06-10T00:39:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe79effd700 10 client.16524 handle_command_reply: tid=0 2024-06-10T00:39:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe7d070eb80 1 client.16524 shutdown 2024-06-10T00:39:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe7d070eb80 1 -- 172.21.15.62:0/4154743297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca4880 msgr2=0x556987cbb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/4154743297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca4880 0x556987cbb0b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8007660 tx=0x7fe7b8007970 comp rx=0 tx=0).stop 2024-06-10T00:39:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe7d070eb80 1 -- 172.21.15.62:0/4154743297 shutdown_connections 2024-06-10T00:39:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/4154743297 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x556987cb7e50 0x556987d24d60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/4154743297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556987ca4880 0x556987cbb0b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/4154743297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556987c9e7a0 0x556987cbd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe7d070eb80 1 --2- 172.21.15.62:0/4154743297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556987ca3f60 0x556987cbab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe7d070eb80 1 -- 172.21.15.62:0/4154743297 >> 172.21.15.62:0/4154743297 conn(0x556987bcb080 msgr2=0x556987ca2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:35.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe79effd700 0 client.16524 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:39:35.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe7d070eb80 1 -- 172.21.15.62:0/4154743297 shutdown_connections 2024-06-10T00:39:35.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe7d070eb80 1 -- 172.21.15.62:0/4154743297 wait complete. 2024-06-10T00:39:35.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.751+0000 7fe7d070eb80 20 client.16524 trim_cache size 0 max 16384 2024-06-10T00:39:35.771 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:39:35.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.752+0000 7fe7c5e8b700 1 -- 172.21.15.62:0/3838768502 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe7a80a2d90 msgr2=0x7fe7a80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:35.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.752+0000 7fe7c5e8b700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe7a80a2d90 0x7fe7a80a5250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0000c00 tx=0x7fe7b0005fd0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.752+0000 7fe7c5e8b700 1 -- 172.21.15.62:0/3838768502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c0103120 msgr2=0x7fe7c0074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:35.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.752+0000 7fe7c5e8b700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c0103120 0x7fe7c0074cd0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400d980 tx=0x7fe7b400dd40 comp rx=0 tx=0).stop 2024-06-10T00:39:35.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.752+0000 7fe7c5e8b700 1 -- 172.21.15.62:0/3838768502 shutdown_connections 2024-06-10T00:39:35.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.752+0000 7fe7c5e8b700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe7a80a2d90 0x7fe7a80a5250 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.752+0000 7fe7c5e8b700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c0107860 0x7fe7c0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.752+0000 7fe7c5e8b700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7c0102770 0x7fe7c0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.752+0000 7fe7c5e8b700 1 --2- 172.21.15.62:0/3838768502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7c0103120 0x7fe7c0074cd0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:35.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.752+0000 7fe7c5e8b700 1 -- 172.21.15.62:0/3838768502 >> 172.21.15.62:0/3838768502 conn(0x7fe7c0079600 msgr2=0x7fe7c0101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:35.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.752+0000 7fe7c5e8b700 1 -- 172.21.15.62:0/3838768502 shutdown_connections 2024-06-10T00:39:35.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:35.752+0000 7fe7c5e8b700 1 -- 172.21.15.62:0/3838768502 wait complete. 2024-06-10T00:39:36.557 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T00:39:36.811 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:39:37.255 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:37 smithi062 ceph-mon[32390]: pgmap v206: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:39:37.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:37 smithi105 ceph-mon[36321]: pgmap v206: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:39:37.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:37 smithi138 ceph-mon[36549]: pgmap v206: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:39:38.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.642+0000 7f7ee662f700 1 -- 172.21.15.62:0/4099046994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee0101620 msgr2=0x7f7ee0101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.642+0000 7f7ee662f700 1 --2- 172.21.15.62:0/4099046994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee0101620 0x7f7ee0101a00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00099c0 tx=0x7f7ed0009cd0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.642+0000 7f7ee662f700 1 -- 172.21.15.62:0/4099046994 shutdown_connections 2024-06-10T00:39:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.642+0000 7f7ee662f700 1 --2- 172.21.15.62:0/4099046994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee0101620 0x7f7ee0101a00 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.642+0000 7f7ee662f700 1 --2- 172.21.15.62:0/4099046994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ee01066b0 0x7f7ee010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.642+0000 7f7ee662f700 1 --2- 172.21.15.62:0/4099046994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ee0101fd0 0x7f7ee0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.642+0000 7f7ee662f700 1 -- 172.21.15.62:0/4099046994 >> 172.21.15.62:0/4099046994 conn(0x7f7ee0078340 msgr2=0x7f7ee0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.642+0000 7f7ee662f700 1 -- 172.21.15.62:0/4099046994 shutdown_connections 2024-06-10T00:39:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.642+0000 7f7ee662f700 1 -- 172.21.15.62:0/4099046994 wait complete. 2024-06-10T00:39:38.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.643+0000 7f7ee662f700 1 Processor -- start 2024-06-10T00:39:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.643+0000 7f7ee662f700 1 -- start start 2024-06-10T00:39:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.643+0000 7f7ee662f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ee0101620 0x7f7ee0199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.643+0000 7f7ee662f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee0101fd0 0x7f7ee0199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.643+0000 7f7ee662f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ee01066b0 0x7f7ee01a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.643+0000 7f7ee662f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01116b0 con 0x7f7ee0101fd0 2024-06-10T00:39:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.643+0000 7f7ee662f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee0111530 con 0x7f7ee0101620 2024-06-10T00:39:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.643+0000 7f7ee662f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee0111830 con 0x7f7ee01066b0 2024-06-10T00:39:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.643+0000 7f7ed7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee0101fd0 0x7f7ee0199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.643+0000 7f7ed7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee0101fd0 0x7f7ee0199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38594/0 (socket says 172.21.15.62:38594) 2024-06-10T00:39:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.643+0000 7f7ed7fff700 1 -- 172.21.15.62:0/897078817 learned_addr learned my addr 172.21.15.62:0/897078817 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.643+0000 7f7edffff700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ee0101620 0x7f7ee0199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.644+0000 7f7ee4bcc700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ee01066b0 0x7f7ee01a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.644+0000 7f7ed7fff700 1 -- 172.21.15.62:0/897078817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ee01066b0 msgr2=0x7f7ee01a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.644+0000 7f7ed7fff700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ee01066b0 0x7f7ee01a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.644+0000 7f7ed7fff700 1 -- 172.21.15.62:0/897078817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ee0101620 msgr2=0x7f7ee0199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:38.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.644+0000 7f7ed7fff700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ee0101620 0x7f7ee0199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.644+0000 7f7ed7fff700 1 -- 172.21.15.62:0/897078817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee01a5050 con 0x7f7ee0101fd0 2024-06-10T00:39:38.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.644+0000 7f7ee4bcc700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ee01066b0 0x7f7ee01a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:39:38.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.644+0000 7f7ed7fff700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee0101fd0 0x7f7ee0199fd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800d950 tx=0x7f7ec800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:38.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.644+0000 7f7eddffb700 1 -- 172.21.15.62:0/897078817 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8017070 con 0x7f7ee0101fd0 2024-06-10T00:39:38.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.644+0000 7f7eddffb700 1 -- 172.21.15.62:0/897078817 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec8005020 con 0x7f7ee0101fd0 2024-06-10T00:39:38.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.644+0000 7f7eddffb700 1 -- 172.21.15.62:0/897078817 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8009b20 con 0x7f7ee0101fd0 2024-06-10T00:39:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.644+0000 7f7ee662f700 1 -- 172.21.15.62:0/897078817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee01a5280 con 0x7f7ee0101fd0 2024-06-10T00:39:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.644+0000 7f7ee662f700 1 -- 172.21.15.62:0/897078817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee01a5740 con 0x7f7ee0101fd0 2024-06-10T00:39:38.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.647+0000 7f7eddffb700 1 -- 172.21.15.62:0/897078817 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7ec80051d0 con 0x7f7ee0101fd0 2024-06-10T00:39:38.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.648+0000 7f7eddffb700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7ecc0a2e10 0x7f7ecc0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.648+0000 7f7edffff700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7ecc0a2e10 0x7f7ecc0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.648+0000 7f7eddffb700 1 -- 172.21.15.62:0/897078817 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f7ec80cf060 con 0x7f7ee0101fd0 2024-06-10T00:39:38.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.648+0000 7f7ef0eb2b80 1 Processor -- start 2024-06-10T00:39:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.648+0000 7f7ef0eb2b80 1 -- start start 2024-06-10T00:39:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.648+0000 7f7edffff700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7ecc0a2e10 0x7f7ecc0a52d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0000c00 tx=0x7f7ed0005fb0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.648+0000 7f7ef0eb2b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a1fd0 0x556abd5a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.648+0000 7f7ef0eb2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a28f0 0x556abd5a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.648+0000 7f7ef0eb2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a6c30 0x556abd5a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.648+0000 7f7ef0eb2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556abd4ef810 con 0x556abd5a6c30 2024-06-10T00:39:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.648+0000 7f7ef0eb2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556abd497720 con 0x556abd5a28f0 2024-06-10T00:39:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.648+0000 7f7ef0eb2b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556abd4eb7d0 con 0x556abd5a1fd0 2024-06-10T00:39:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.649+0000 7f7ee4bcc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a6c30 0x556abd5a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.649+0000 7f7ee4bcc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a6c30 0x556abd5a7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38608/0 (socket says 172.21.15.62:38608) 2024-06-10T00:39:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.649+0000 7f7ee4bcc700 1 -- 172.21.15.62:0/1061626388 learned_addr learned my addr 172.21.15.62:0/1061626388 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.649+0000 7f7ed7fff700 1 --2- 172.21.15.62:0/1061626388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a28f0 0x556abd5a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.649+0000 7f7edffff700 1 --2- 172.21.15.62:0/1061626388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a1fd0 0x556abd5a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.649+0000 7f7ee4bcc700 1 -- 172.21.15.62:0/1061626388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a1fd0 msgr2=0x556abd5a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:38.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.649+0000 7f7ee4bcc700 1 --2- 172.21.15.62:0/1061626388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a1fd0 0x556abd5a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.649+0000 7f7ee4bcc700 1 -- 172.21.15.62:0/1061626388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a28f0 msgr2=0x556abd5a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.649+0000 7f7ee4bcc700 1 --2- 172.21.15.62:0/1061626388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a28f0 0x556abd5a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.649+0000 7f7ee4bcc700 1 -- 172.21.15.62:0/1061626388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556abd48df50 con 0x556abd5a6c30 2024-06-10T00:39:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.649+0000 7f7ee4bcc700 1 --2- 172.21.15.62:0/1061626388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a6c30 0x556abd5a7090 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800d300 tx=0x7f7ed8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f300b5f99b5cb65 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.649+0000 7f7ed67fc700 1 -- 172.21.15.62:0/1061626388 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8016020 con 0x556abd5a6c30 2024-06-10T00:39:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.649+0000 7f7ed67fc700 1 -- 172.21.15.62:0/1061626388 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ed80128d0 con 0x556abd5a6c30 2024-06-10T00:39:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.649+0000 7f7ed67fc700 1 -- 172.21.15.62:0/1061626388 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8011940 con 0x556abd5a6c30 2024-06-10T00:39:38.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.650+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/1061626388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a6c30 msgr2=0x556abd5a7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.650+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/1061626388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a6c30 0x556abd5a7090 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800d300 tx=0x7f7ed8007f90 comp rx=0 tx=0).stop 2024-06-10T00:39:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.650+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/1061626388 shutdown_connections 2024-06-10T00:39:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.650+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/1061626388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a6c30 0x556abd5a7090 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.650+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/1061626388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a1fd0 0x556abd5a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.650+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/1061626388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a28f0 0x556abd5a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.650+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/1061626388 >> 172.21.15.62:0/1061626388 conn(0x556abd4c8cc0 msgr2=0x556abd598850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.650+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/1061626388 shutdown_connections 2024-06-10T00:39:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.650+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/1061626388 wait complete. 2024-06-10T00:39:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.650+0000 7f7ef0eb2b80 1 -- start start 2024-06-10T00:39:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.651+0000 7f7ef0eb2b80 1 Processor -- start 2024-06-10T00:39:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.651+0000 7f7ef0eb2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a1fd0 0x556abd5ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.651+0000 7f7ef0eb2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a28f0 0x556abd5ab470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.651+0000 7f7ef0eb2b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a6c30 0x556abd5ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.651+0000 7f7ef0eb2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556abd5aedd0 con 0x556abd5a1fd0 2024-06-10T00:39:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.651+0000 7f7ef0eb2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556abd48fbf0 con 0x556abd5a28f0 2024-06-10T00:39:38.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.651+0000 7f7ef0eb2b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556abd5aef50 con 0x556abd5a6c30 2024-06-10T00:39:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.651+0000 7f7ef0eb2b80 10 client.0 _ll_register_callbacks cb 0x556abd489920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.651+0000 7f7edffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a1fd0 0x556abd5ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.651+0000 7f7edffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a1fd0 0x556abd5ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38616/0 (socket says 172.21.15.62:38616) 2024-06-10T00:39:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.651+0000 7f7edffff700 1 -- 172.21.15.62:0/2651677356 learned_addr learned my addr 172.21.15.62:0/2651677356 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.651+0000 7f7ed7fff700 1 --2- 172.21.15.62:0/2651677356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a28f0 0x556abd5ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.651+0000 7f7ee4bcc700 1 --2- 172.21.15.62:0/2651677356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a6c30 0x556abd5ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.652+0000 7f7edffff700 1 -- 172.21.15.62:0/2651677356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a6c30 msgr2=0x556abd5ab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.652+0000 7f7edffff700 1 --2- 172.21.15.62:0/2651677356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a6c30 0x556abd5ab9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.652+0000 7f7edffff700 1 -- 172.21.15.62:0/2651677356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a28f0 msgr2=0x556abd5ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.652+0000 7f7edffff700 1 --2- 172.21.15.62:0/2651677356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a28f0 0x556abd5ab470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.652+0000 7f7edffff700 1 -- 172.21.15.62:0/2651677356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556abd43e8d0 con 0x556abd5a1fd0 2024-06-10T00:39:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.652+0000 7f7edffff700 1 --2- 172.21.15.62:0/2651677356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a1fd0 0x556abd5ae100 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00098e0 tx=0x7f7ed0009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549f51f8fd017a60 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.652+0000 7f7ebf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.652+0000 7f7ebf7fe700 1 -- 172.21.15.62:0/2651677356 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0035060 con 0x556abd5a1fd0 2024-06-10T00:39:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.652+0000 7f7ebf7fe700 1 -- 172.21.15.62:0/2651677356 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f7ed0030630 con 0x556abd5a1fd0 2024-06-10T00:39:38.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.652+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/2651677356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed80128d0 con 0x556abd5a1fd0 2024-06-10T00:39:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.652+0000 7f7ebf7fe700 1 -- 172.21.15.62:0/2651677356 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7ed003c050 con 0x556abd5a1fd0 2024-06-10T00:39:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.652+0000 7f7ef0eb2b80 10 client.16536 fetch_fsmap learned FSMap version 36 2024-06-10T00:39:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.652+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/2651677356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556abd5b96d0 con 0x556abd5a1fd0 2024-06-10T00:39:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.652+0000 7f7ebf7fe700 1 -- 172.21.15.62:0/2651677356 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ed80128d0 con 0x556abd5a1fd0 2024-06-10T00:39:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.653+0000 7f7ebf7fe700 1 -- 172.21.15.62:0/2651677356 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f7ed002f070 con 0x556abd5a1fd0 2024-06-10T00:39:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.653+0000 7f7ef0eb2b80 10 client.16536 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T00:39:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.653+0000 7f7ef0eb2b80 10 client.16536 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:39:38.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.653+0000 7f7ef0eb2b80 20 client.16536 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.653+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/2651677356 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556abd5bbd10 0x556abd5be120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.653+0000 7f7ef0eb2b80 4 client.16536 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:39:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.653+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/2651677356 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556abd4fcd10 con 0x556abd5bbd10 2024-06-10T00:39:38.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.653+0000 7f7ed7fff700 1 --2- 172.21.15.62:0/2651677356 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556abd5bbd10 0x556abd5be120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.653+0000 7f7ed7fff700 1 --2- 172.21.15.62:0/2651677356 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556abd5bbd10 0x556abd5be120 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=43b1d15a8f891fb2 server_cookie=1425e0f435107a7f in_seq=0 out_seq=0 2024-06-10T00:39:38.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.653+0000 7f7ebf7fe700 10 client.16536 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:39:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.654+0000 7f7ebf7fe700 1 -- 172.21.15.62:0/2651677356 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556abd4fcd10 con 0x556abd5bbd10 2024-06-10T00:39:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.654+0000 7f7ebf7fe700 10 client.16536 handle_command_reply: tid=0 2024-06-10T00:39:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.654+0000 7f7ef0eb2b80 1 client.16536 shutdown 2024-06-10T00:39:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.654+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/2651677356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a1fd0 msgr2=0x556abd5ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.654+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/2651677356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a1fd0 0x556abd5ae100 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00098e0 tx=0x7f7ed0009cd0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.654+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/2651677356 shutdown_connections 2024-06-10T00:39:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.654+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/2651677356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a1fd0 0x556abd5ae100 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.654+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/2651677356 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556abd5bbd10 0x556abd5be120 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.654+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/2651677356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a6c30 0x556abd5ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.654+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/2651677356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a28f0 0x556abd5ab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.654+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/2651677356 >> 172.21.15.62:0/2651677356 conn(0x556abd4c8cc0 msgr2=0x556abd59c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:38.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.655+0000 7f7ebf7fe700 0 client.16536 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:39:38.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.655+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/2651677356 shutdown_connections 2024-06-10T00:39:38.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.655+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/2651677356 wait complete. 2024-06-10T00:39:38.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.655+0000 7f7ef0eb2b80 20 client.16536 trim_cache size 0 max 16384 2024-06-10T00:39:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7ef0eb2b80 1 Processor -- start 2024-06-10T00:39:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7ef0eb2b80 1 -- start start 2024-06-10T00:39:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7ef0eb2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a1fd0 0x556abd5aad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7ef0eb2b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a28f0 0x556abd591e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7ef0eb2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a6c30 0x556abd592340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7ef0eb2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01ac330 con 0x556abd5a1fd0 2024-06-10T00:39:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7ef0eb2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01adf30 con 0x556abd5a6c30 2024-06-10T00:39:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7ef0eb2b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01aca00 con 0x556abd5a28f0 2024-06-10T00:39:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7edffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a1fd0 0x556abd5aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7edffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a1fd0 0x556abd5aad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38630/0 (socket says 172.21.15.62:38630) 2024-06-10T00:39:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7edffff700 1 -- 172.21.15.62:0/1965446909 learned_addr learned my addr 172.21.15.62:0/1965446909 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:38.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7ee4bcc700 1 --2- 172.21.15.62:0/1965446909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a6c30 0x556abd592340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7ed7fff700 1 --2- 172.21.15.62:0/1965446909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a28f0 0x556abd591e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7edffff700 1 -- 172.21.15.62:0/1965446909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a28f0 msgr2=0x556abd591e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7edffff700 1 --2- 172.21.15.62:0/1965446909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a28f0 0x556abd591e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7edffff700 1 -- 172.21.15.62:0/1965446909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a6c30 msgr2=0x556abd592340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7edffff700 1 --2- 172.21.15.62:0/1965446909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a6c30 0x556abd592340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7edffff700 1 -- 172.21.15.62:0/1965446909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556abd5a9a00 con 0x556abd5a1fd0 2024-06-10T00:39:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7ee4bcc700 1 --2- 172.21.15.62:0/1965446909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a6c30 0x556abd592340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:39:38.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.670+0000 7f7ed7fff700 1 --2- 172.21.15.62:0/1965446909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a28f0 0x556abd591e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:39:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.671+0000 7f7edffff700 1 --2- 172.21.15.62:0/1965446909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a1fd0 0x556abd5aad90 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0024ac0 tx=0x7f7ed0009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab12c1aa0cc6417 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.671+0000 7f7ed5ffb700 1 -- 172.21.15.62:0/1965446909 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0028020 con 0x556abd5a1fd0 2024-06-10T00:39:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.671+0000 7f7ed5ffb700 1 -- 172.21.15.62:0/1965446909 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ed0005960 con 0x556abd5a1fd0 2024-06-10T00:39:38.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.671+0000 7f7ed5ffb700 1 -- 172.21.15.62:0/1965446909 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0035070 con 0x556abd5a1fd0 2024-06-10T00:39:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.671+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/1965446909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a1fd0 msgr2=0x556abd5aad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.671+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/1965446909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a1fd0 0x556abd5aad90 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0024ac0 tx=0x7f7ed0009910 comp rx=0 tx=0).stop 2024-06-10T00:39:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.671+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/1965446909 shutdown_connections 2024-06-10T00:39:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.671+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/1965446909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a1fd0 0x556abd5aad90 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.671+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/1965446909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a6c30 0x556abd592340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.671+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/1965446909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a28f0 0x556abd591e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.671+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/1965446909 >> 172.21.15.62:0/1965446909 conn(0x556abd4c8cc0 msgr2=0x556abd5ab430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.672+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/1965446909 shutdown_connections 2024-06-10T00:39:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.672+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/1965446909 wait complete. 2024-06-10T00:39:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.672+0000 7f7ef0eb2b80 1 -- start start 2024-06-10T00:39:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.672+0000 7f7ef0eb2b80 1 Processor -- start 2024-06-10T00:39:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.672+0000 7f7ef0eb2b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a1fd0 0x556abd61ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.672+0000 7f7ef0eb2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a28f0 0x556abd61f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.672+0000 7f7ef0eb2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a6c30 0x556abd5af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.672+0000 7f7ef0eb2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01ad860 con 0x556abd5a28f0 2024-06-10T00:39:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.672+0000 7f7ef0eb2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01ad0d0 con 0x556abd5a6c30 2024-06-10T00:39:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.672+0000 7f7ef0eb2b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01aa2f0 con 0x556abd5a1fd0 2024-06-10T00:39:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.672+0000 7f7ef0eb2b80 10 client.0 _ll_register_callbacks cb 0x556abd489920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.672+0000 7f7ed7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a28f0 0x556abd61f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ed7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a28f0 0x556abd61f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38644/0 (socket says 172.21.15.62:38644) 2024-06-10T00:39:38.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ed7fff700 1 -- 172.21.15.62:0/3003353002 learned_addr learned my addr 172.21.15.62:0/3003353002 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ee4bcc700 1 --2- 172.21.15.62:0/3003353002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a6c30 0x556abd5af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7edffff700 1 --2- 172.21.15.62:0/3003353002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a1fd0 0x556abd61ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ed7fff700 1 -- 172.21.15.62:0/3003353002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a1fd0 msgr2=0x556abd61ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ed7fff700 1 --2- 172.21.15.62:0/3003353002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a1fd0 0x556abd61ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ed7fff700 1 -- 172.21.15.62:0/3003353002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a6c30 msgr2=0x556abd5af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ed7fff700 1 --2- 172.21.15.62:0/3003353002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a6c30 0x556abd5af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ed7fff700 1 -- 172.21.15.62:0/3003353002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556abd5afff0 con 0x556abd5a28f0 2024-06-10T00:39:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ed7fff700 1 --2- 172.21.15.62:0/3003353002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a28f0 0x556abd61f2d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80cc460 tx=0x7f7ec80bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2431db1979880a4c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:38.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ebf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:38.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ebf7fe700 1 -- 172.21.15.62:0/3003353002 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80b3030 con 0x556abd5a28f0 2024-06-10T00:39:38.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ebf7fe700 1 -- 172.21.15.62:0/3003353002 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f7ec80cc680 con 0x556abd5a28f0 2024-06-10T00:39:38.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/3003353002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556abd4f7290 con 0x556abd5a28f0 2024-06-10T00:39:38.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ebf7fe700 1 -- 172.21.15.62:0/3003353002 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7ec8020040 con 0x556abd5a28f0 2024-06-10T00:39:38.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ef0eb2b80 10 client.16542 fetch_fsmap learned FSMap version 36 2024-06-10T00:39:38.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.673+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/3003353002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556abd6264a0 con 0x556abd5a28f0 2024-06-10T00:39:38.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.674+0000 7f7ebf7fe700 1 -- 172.21.15.62:0/3003353002 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556abd4f7290 con 0x556abd5a28f0 2024-06-10T00:39:38.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.674+0000 7f7ebf7fe700 1 -- 172.21.15.62:0/3003353002 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f7ec809b430 con 0x556abd5a28f0 2024-06-10T00:39:38.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.674+0000 7f7ef0eb2b80 10 client.16542 fetch_fsmap finished waiting for FSMap version 36 2024-06-10T00:39:38.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.674+0000 7f7ef0eb2b80 10 client.16542 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:39:38.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.674+0000 7f7ef0eb2b80 20 client.16542 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:38.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.674+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/3003353002 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556abd626bb0 0x556abd628fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:38.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.674+0000 7f7ef0eb2b80 4 client.16542 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:39:38.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.674+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/3003353002 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556abd371250 con 0x556abd626bb0 2024-06-10T00:39:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.674+0000 7f7edffff700 1 --2- 172.21.15.62:0/3003353002 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556abd626bb0 0x556abd628fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.674+0000 7f7edffff700 1 --2- 172.21.15.62:0/3003353002 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556abd626bb0 0x556abd628fd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c27c5a745a3ed983 server_cookie=c59c9dbb43ffa43e in_seq=0 out_seq=0 2024-06-10T00:39:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.674+0000 7f7ebf7fe700 10 client.16542 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:39:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ebf7fe700 1 -- 172.21.15.62:0/3003353002 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556abd371250 con 0x556abd626bb0 2024-06-10T00:39:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ebf7fe700 10 client.16542 handle_command_reply: tid=0 2024-06-10T00:39:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ef0eb2b80 1 client.16542 shutdown 2024-06-10T00:39:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/3003353002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a28f0 msgr2=0x556abd61f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/3003353002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a28f0 0x556abd61f2d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80cc460 tx=0x7f7ec80bb040 comp rx=0 tx=0).stop 2024-06-10T00:39:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/3003353002 shutdown_connections 2024-06-10T00:39:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/3003353002 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556abd626bb0 0x556abd628fd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/3003353002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556abd5a6c30 0x556abd5af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/3003353002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556abd5a1fd0 0x556abd61ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ebf7fe700 0 client.16542 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:39:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ef0eb2b80 1 --2- 172.21.15.62:0/3003353002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556abd5a28f0 0x556abd61f2d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/3003353002 >> 172.21.15.62:0/3003353002 conn(0x556abd4c8cc0 msgr2=0x556abd591e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:38.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/3003353002 shutdown_connections 2024-06-10T00:39:38.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ef0eb2b80 1 -- 172.21.15.62:0/3003353002 wait complete. 2024-06-10T00:39:38.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.675+0000 7f7ef0eb2b80 20 client.16542 trim_cache size 0 max 16384 2024-06-10T00:39:38.686 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:39:38.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.676+0000 7f7ee662f700 1 -- 172.21.15.62:0/897078817 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7ecc0a2e10 msgr2=0x7f7ecc0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:38.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.676+0000 7f7ee662f700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7ecc0a2e10 0x7f7ecc0a52d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0000c00 tx=0x7f7ed0005fb0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.676+0000 7f7ee662f700 1 -- 172.21.15.62:0/897078817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee0101fd0 msgr2=0x7f7ee0199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:38.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.676+0000 7f7ee662f700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee0101fd0 0x7f7ee0199fd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800d950 tx=0x7f7ec800dc60 comp rx=0 tx=0).stop 2024-06-10T00:39:38.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.676+0000 7f7ee662f700 1 -- 172.21.15.62:0/897078817 shutdown_connections 2024-06-10T00:39:38.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.676+0000 7f7ee662f700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f7ecc0a2e10 0x7f7ecc0a52d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.676+0000 7f7ee662f700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ee01066b0 0x7f7ee01a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.676+0000 7f7ee662f700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ee0101620 0x7f7ee0199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.676+0000 7f7ee662f700 1 --2- 172.21.15.62:0/897078817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ee0101fd0 0x7f7ee0199fd0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:38.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.676+0000 7f7ee662f700 1 -- 172.21.15.62:0/897078817 >> 172.21.15.62:0/897078817 conn(0x7f7ee0078340 msgr2=0x7f7ee0105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:38.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.676+0000 7f7ee662f700 1 -- 172.21.15.62:0/897078817 shutdown_connections 2024-06-10T00:39:38.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:38.676+0000 7f7ee662f700 1 -- 172.21.15.62:0/897078817 wait complete. 2024-06-10T00:39:39.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:39 smithi062 ceph-mon[32390]: pgmap v207: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:39:39.433 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T00:39:39.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:39 smithi105 ceph-mon[36321]: pgmap v207: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:39:39.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:39 smithi138 ceph-mon[36549]: pgmap v207: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:39:39.682 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:39:41.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.370+0000 7f1cd2159700 1 -- 172.21.15.62:0/2355291146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ccc108580 msgr2=0x7f1ccc108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:41.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.370+0000 7f1cd2159700 1 --2- 172.21.15.62:0/2355291146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ccc108580 0x7f1ccc108960 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0099c0 tx=0x7f1cbc009cd0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.371+0000 7f1cd2159700 1 -- 172.21.15.62:0/2355291146 shutdown_connections 2024-06-10T00:39:41.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.371+0000 7f1cd2159700 1 --2- 172.21.15.62:0/2355291146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ccc108580 0x7f1ccc108960 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.371+0000 7f1cd2159700 1 --2- 172.21.15.62:0/2355291146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ccc06b620 0x7f1ccc10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.371+0000 7f1cd2159700 1 --2- 172.21.15.62:0/2355291146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ccc06ac60 0x7f1ccc06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.371+0000 7f1cd2159700 1 -- 172.21.15.62:0/2355291146 >> 172.21.15.62:0/2355291146 conn(0x7f1ccc078430 msgr2=0x7f1ccc078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:41.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.371+0000 7f1cd2159700 1 -- 172.21.15.62:0/2355291146 shutdown_connections 2024-06-10T00:39:41.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.371+0000 7f1cd2159700 1 -- 172.21.15.62:0/2355291146 wait complete. 2024-06-10T00:39:41.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.371+0000 7f1cd2159700 1 Processor -- start 2024-06-10T00:39:41.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1cd2159700 1 -- start start 2024-06-10T00:39:41.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:41 smithi062 ceph-mon[32390]: pgmap v208: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:41.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1cd2159700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ccc06ac60 0x7f1ccc101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1cd2159700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ccc06b620 0x7f1ccc101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1ccb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ccc06ac60 0x7f1ccc101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1cd2159700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ccc108580 0x7f1ccc101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1cd2159700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc112840 con 0x7f1ccc06ac60 2024-06-10T00:39:41.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1cd2159700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1126c0 con 0x7f1ccc108580 2024-06-10T00:39:41.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1ccb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ccc06ac60 0x7f1ccc101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38654/0 (socket says 172.21.15.62:38654) 2024-06-10T00:39:41.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1cd2159700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1129c0 con 0x7f1ccc06b620 2024-06-10T00:39:41.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1ccb7fe700 1 -- 172.21.15.62:0/4229733359 learned_addr learned my addr 172.21.15.62:0/4229733359 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:41.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1ccbfff700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ccc108580 0x7f1ccc101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1ccaffd700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ccc06b620 0x7f1ccc101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1ccb7fe700 1 -- 172.21.15.62:0/4229733359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ccc06b620 msgr2=0x7f1ccc101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:41.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1ccb7fe700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ccc06b620 0x7f1ccc101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1ccb7fe700 1 -- 172.21.15.62:0/4229733359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ccc108580 msgr2=0x7f1ccc101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:41.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1ccb7fe700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ccc108580 0x7f1ccc101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1ccb7fe700 1 -- 172.21.15.62:0/4229733359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc1a7160 con 0x7f1ccc06ac60 2024-06-10T00:39:41.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.372+0000 7f1ccaffd700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ccc06b620 0x7f1ccc101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:41.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.373+0000 7f1ccb7fe700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ccc06ac60 0x7f1ccc101400 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc006010 tx=0x7f1cbc009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:41.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.373+0000 7f1cc8ff9700 1 -- 172.21.15.62:0/4229733359 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc027070 con 0x7f1ccc06ac60 2024-06-10T00:39:41.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.373+0000 7f1cd2159700 1 -- 172.21.15.62:0/4229733359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ccc1a73f0 con 0x7f1ccc06ac60 2024-06-10T00:39:41.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.373+0000 7f1cc8ff9700 1 -- 172.21.15.62:0/4229733359 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cbc003780 con 0x7f1ccc06ac60 2024-06-10T00:39:41.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.373+0000 7f1cc8ff9700 1 -- 172.21.15.62:0/4229733359 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc022da0 con 0x7f1ccc06ac60 2024-06-10T00:39:41.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.373+0000 7f1cd2159700 1 -- 172.21.15.62:0/4229733359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ccc1a7840 con 0x7f1ccc06ac60 2024-06-10T00:39:41.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.376+0000 7f1cc8ff9700 1 -- 172.21.15.62:0/4229733359 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1cbc02a070 con 0x7f1ccc06ac60 2024-06-10T00:39:41.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.377+0000 7f1cc8ff9700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1cb40a2d90 0x7f1cb40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.377+0000 7f1ccaffd700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1cb40a2d90 0x7f1cb40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.377+0000 7f1cc8ff9700 1 -- 172.21.15.62:0/4229733359 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f1cbc0d04f0 con 0x7f1ccc06ac60 2024-06-10T00:39:41.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.377+0000 7f1cdc9dcb80 1 Processor -- start 2024-06-10T00:39:41.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1cdc9dcb80 1 -- start start 2024-06-10T00:39:41.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1ccaffd700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1cb40a2d90 0x7f1cb40a5250 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc078950 tx=0x7f1cc000b5f0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:41.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1cdc9dcb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 0x560736119340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1cdc9dcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x560736113260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1cdc9dcb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607361137a0 0x560736113bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1cdc9dcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560736005f60 con 0x560736119880 2024-06-10T00:39:41.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1cdc9dcb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560736064850 con 0x5607361137a0 2024-06-10T00:39:41.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1cdc9dcb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560735fcc460 con 0x560736118f60 2024-06-10T00:39:41.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1ccaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x560736113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1ccaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x560736113260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38670/0 (socket says 172.21.15.62:38670) 2024-06-10T00:39:41.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1ccaffd700 1 -- 172.21.15.62:0/1034637986 learned_addr learned my addr 172.21.15.62:0/1034637986 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:41.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1ccb7fe700 1 --2- 172.21.15.62:0/1034637986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 0x560736119340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1ccbfff700 1 --2- 172.21.15.62:0/1034637986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607361137a0 0x560736113bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1ccaffd700 1 -- 172.21.15.62:0/1034637986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 msgr2=0x560736119340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:41.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1ccaffd700 1 --2- 172.21.15.62:0/1034637986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 0x560736119340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1ccaffd700 1 -- 172.21.15.62:0/1034637986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607361137a0 msgr2=0x560736113bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:41.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1ccaffd700 1 --2- 172.21.15.62:0/1034637986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607361137a0 0x560736113bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1ccaffd700 1 -- 172.21.15.62:0/1034637986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560735fb38d0 con 0x560736119880 2024-06-10T00:39:41.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1ccb7fe700 1 --2- 172.21.15.62:0/1034637986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 0x560736119340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:41.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.378+0000 7f1ccaffd700 1 --2- 172.21.15.62:0/1034637986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x560736113260 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000fcb0 tx=0x7f1cc0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5a1fa2ba400b7a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:41.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.379+0000 7f1cb1ffb700 1 -- 172.21.15.62:0/1034637986 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0022070 con 0x560736119880 2024-06-10T00:39:41.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.379+0000 7f1cb1ffb700 1 -- 172.21.15.62:0/1034637986 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc001e450 con 0x560736119880 2024-06-10T00:39:41.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.379+0000 7f1cb1ffb700 1 -- 172.21.15.62:0/1034637986 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc001d5f0 con 0x560736119880 2024-06-10T00:39:41.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.379+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/1034637986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 msgr2=0x560736113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:41.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.379+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/1034637986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x560736113260 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000fcb0 tx=0x7f1cc0012000 comp rx=0 tx=0).stop 2024-06-10T00:39:41.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.379+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/1034637986 shutdown_connections 2024-06-10T00:39:41.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.379+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/1034637986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607361137a0 0x560736113bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.379+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/1034637986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 0x560736119340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.379+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/1034637986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x560736113260 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.379+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/1034637986 >> 172.21.15.62:0/1034637986 conn(0x560736040080 msgr2=0x56073610e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:41.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.380+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/1034637986 shutdown_connections 2024-06-10T00:39:41.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.380+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/1034637986 wait complete. 2024-06-10T00:39:41.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.380+0000 7f1cdc9dcb80 1 -- start start 2024-06-10T00:39:41.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.380+0000 7f1cdc9dcb80 1 Processor -- start 2024-06-10T00:39:41.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.380+0000 7f1cdc9dcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607361137a0 0x56073611f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.380+0000 7f1cdc9dcb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 0x56073611c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.380+0000 7f1cdc9dcb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560736119880 0x56073611cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.380+0000 7f1cdc9dcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560736004fc0 con 0x5607361137a0 2024-06-10T00:39:41.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.380+0000 7f1cdc9dcb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560736060810 con 0x560736119880 2024-06-10T00:39:41.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.380+0000 7f1cdc9dcb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560736120260 con 0x560736118f60 2024-06-10T00:39:41.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.380+0000 7f1cdc9dcb80 10 client.0 _ll_register_callbacks cb 0x560735ffa5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1ccb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607361137a0 0x56073611f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1ccb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607361137a0 0x56073611f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38674/0 (socket says 172.21.15.62:38674) 2024-06-10T00:39:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1ccb7fe700 1 -- 172.21.15.62:0/452468495 learned_addr learned my addr 172.21.15.62:0/452468495 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1ccbfff700 1 --2- 172.21.15.62:0/452468495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560736119880 0x56073611cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1ccaffd700 1 --2- 172.21.15.62:0/452468495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 0x56073611c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1ccb7fe700 1 -- 172.21.15.62:0/452468495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 msgr2=0x56073611c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1ccb7fe700 1 --2- 172.21.15.62:0/452468495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 0x56073611c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1ccb7fe700 1 -- 172.21.15.62:0/452468495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560736119880 msgr2=0x56073611cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1ccb7fe700 1 --2- 172.21.15.62:0/452468495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560736119880 0x56073611cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1ccb7fe700 1 -- 172.21.15.62:0/452468495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56073602a750 con 0x5607361137a0 2024-06-10T00:39:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1ccb7fe700 1 --2- 172.21.15.62:0/452468495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607361137a0 0x56073611f5a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0cd8f0 tx=0x7f1cbc09cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52e7614a5dd6676 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1caaffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1caaffd700 1 -- 172.21.15.62:0/452468495 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0d0350 con 0x5607361137a0 2024-06-10T00:39:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1caaffd700 1 -- 172.21.15.62:0/452468495 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f1cbc0b5670 con 0x5607361137a0 2024-06-10T00:39:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/452468495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cc001e450 con 0x5607361137a0 2024-06-10T00:39:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.381+0000 7f1caaffd700 1 -- 172.21.15.62:0/452468495 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1cbc0a53f0 con 0x5607361137a0 2024-06-10T00:39:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.382+0000 7f1cdc9dcb80 10 client.16554 fetch_fsmap learned FSMap version 37 2024-06-10T00:39:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.382+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/452468495 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56073612a990 con 0x5607361137a0 2024-06-10T00:39:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.382+0000 7f1caaffd700 1 -- 172.21.15.62:0/452468495 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc001e450 con 0x5607361137a0 2024-06-10T00:39:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.382+0000 7f1caaffd700 1 -- 172.21.15.62:0/452468495 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f1cbc0a4f60 con 0x5607361137a0 2024-06-10T00:39:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.382+0000 7f1cdc9dcb80 10 client.16554 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T00:39:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.382+0000 7f1cdc9dcb80 10 client.16554 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:39:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.382+0000 7f1cdc9dcb80 20 client.16554 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:41.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.382+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/452468495 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56073612ce60 0x56073612f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.382+0000 7f1cdc9dcb80 4 client.16554 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:39:41.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.382+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/452468495 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560736071d50 con 0x56073612ce60 2024-06-10T00:39:41.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.382+0000 7f1ccaffd700 1 --2- 172.21.15.62:0/452468495 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56073612ce60 0x56073612f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.383+0000 7f1ccaffd700 1 --2- 172.21.15.62:0/452468495 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56073612ce60 0x56073612f2c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51aa8acde054e3ea server_cookie=1d8015c092286cb4 in_seq=0 out_seq=0 2024-06-10T00:39:41.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.383+0000 7f1caaffd700 10 client.16554 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:39:41.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1caaffd700 1 -- 172.21.15.62:0/452468495 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560736071d50 con 0x56073612ce60 2024-06-10T00:39:41.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1caaffd700 10 client.16554 handle_command_reply: tid=0 2024-06-10T00:39:41.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1cdc9dcb80 1 client.16554 shutdown 2024-06-10T00:39:41.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/452468495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607361137a0 msgr2=0x56073611f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:41.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/452468495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607361137a0 0x56073611f5a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0cd8f0 tx=0x7f1cbc09cf10 comp rx=0 tx=0).stop 2024-06-10T00:39:41.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/452468495 shutdown_connections 2024-06-10T00:39:41.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/452468495 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56073612ce60 0x56073612f2c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1caaffd700 0 client.16554 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:39:41.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/452468495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607361137a0 0x56073611f5a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/452468495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560736119880 0x56073611cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/452468495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 0x56073611c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/452468495 >> 172.21.15.62:0/452468495 conn(0x560736040080 msgr2=0x56073610d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:41.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/452468495 shutdown_connections 2024-06-10T00:39:41.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/452468495 wait complete. 2024-06-10T00:39:41.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.384+0000 7f1cdc9dcb80 20 client.16554 trim_cache size 0 max 16384 2024-06-10T00:39:41.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.399+0000 7f1cdc9dcb80 1 Processor -- start 2024-06-10T00:39:41.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.399+0000 7f1cdc9dcb80 1 -- start start 2024-06-10T00:39:41.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1cdc9dcb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607361137a0 0x560736113260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1cdc9dcb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 0x5607361182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1cdc9dcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x56073611f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1cdc9dcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1b0770 con 0x560736119880 2024-06-10T00:39:41.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1cdc9dcb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1b22e0 con 0x5607361137a0 2024-06-10T00:39:41.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1cdc9dcb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1b0e40 con 0x560736118f60 2024-06-10T00:39:41.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1ccbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x56073611f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1ccbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x56073611f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38686/0 (socket says 172.21.15.62:38686) 2024-06-10T00:39:41.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1ccbfff700 1 -- 172.21.15.62:0/2940124744 learned_addr learned my addr 172.21.15.62:0/2940124744 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:41.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1ccaffd700 1 --2- 172.21.15.62:0/2940124744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 0x5607361182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1ccb7fe700 1 --2- 172.21.15.62:0/2940124744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607361137a0 0x560736113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1ccbfff700 1 -- 172.21.15.62:0/2940124744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 msgr2=0x5607361182b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:41.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1ccbfff700 1 --2- 172.21.15.62:0/2940124744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 0x5607361182b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1ccbfff700 1 -- 172.21.15.62:0/2940124744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607361137a0 msgr2=0x560736113260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:41.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1ccbfff700 1 --2- 172.21.15.62:0/2940124744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607361137a0 0x560736113260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1ccbfff700 1 -- 172.21.15.62:0/2940124744 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073610f040 con 0x560736119880 2024-06-10T00:39:41.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1ccb7fe700 1 --2- 172.21.15.62:0/2940124744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607361137a0 0x560736113260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:39:41.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1ccbfff700 1 --2- 172.21.15.62:0/2940124744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x56073611f0e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400a6c0 tx=0x7f1cc4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11fc422d48f0bef server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:41.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.400+0000 7f1cb17fa700 1 -- 172.21.15.62:0/2940124744 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc401c020 con 0x560736119880 2024-06-10T00:39:41.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.401+0000 7f1cb17fa700 1 -- 172.21.15.62:0/2940124744 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc400adf0 con 0x560736119880 2024-06-10T00:39:41.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.401+0000 7f1cb17fa700 1 -- 172.21.15.62:0/2940124744 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40179e0 con 0x560736119880 2024-06-10T00:39:41.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.401+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/2940124744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 msgr2=0x56073611f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:41.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.401+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/2940124744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x56073611f0e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400a6c0 tx=0x7f1cc4009650 comp rx=0 tx=0).stop 2024-06-10T00:39:41.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.401+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/2940124744 shutdown_connections 2024-06-10T00:39:41.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.401+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/2940124744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x56073611f0e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.401+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/2940124744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607361137a0 0x560736113260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.401+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/2940124744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560736118f60 0x5607361182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.401+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/2940124744 >> 172.21.15.62:0/2940124744 conn(0x560736040080 msgr2=0x56073611a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:41.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.401+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/2940124744 shutdown_connections 2024-06-10T00:39:41.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.401+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/2940124744 wait complete. 2024-06-10T00:39:41.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.402+0000 7f1cdc9dcb80 1 -- start start 2024-06-10T00:39:41.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.402+0000 7f1cdc9dcb80 1 Processor -- start 2024-06-10T00:39:41.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.402+0000 7f1cdc9dcb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607361137a0 0x560736134770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.402+0000 7f1cdc9dcb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560736118f60 0x560736134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.402+0000 7f1cdc9dcb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x5607361351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.402+0000 7f1cdc9dcb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1b1c10 con 0x560736119880 2024-06-10T00:39:41.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.402+0000 7f1cdc9dcb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1b1510 con 0x560736118f60 2024-06-10T00:39:41.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.402+0000 7f1cdc9dcb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc1ae710 con 0x5607361137a0 2024-06-10T00:39:41.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.402+0000 7f1cdc9dcb80 10 client.0 _ll_register_callbacks cb 0x56073611bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:41.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.402+0000 7f1ccbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x5607361351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.402+0000 7f1ccbfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x5607361351f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38698/0 (socket says 172.21.15.62:38698) 2024-06-10T00:39:41.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.402+0000 7f1ccbfff700 1 -- 172.21.15.62:0/528073431 learned_addr learned my addr 172.21.15.62:0/528073431 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:41.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.402+0000 7f1ccb7fe700 1 --2- 172.21.15.62:0/528073431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607361137a0 0x560736134770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.402+0000 7f1ccaffd700 1 --2- 172.21.15.62:0/528073431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560736118f60 0x560736134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1ccbfff700 1 -- 172.21.15.62:0/528073431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607361137a0 msgr2=0x560736134770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:41.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1ccbfff700 1 --2- 172.21.15.62:0/528073431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607361137a0 0x560736134770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1ccbfff700 1 -- 172.21.15.62:0/528073431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560736118f60 msgr2=0x560736134cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:41.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1ccbfff700 1 --2- 172.21.15.62:0/528073431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560736118f60 0x560736134cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1ccbfff700 1 -- 172.21.15.62:0/528073431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560736196bb0 con 0x560736119880 2024-06-10T00:39:41.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1ccb7fe700 1 --2- 172.21.15.62:0/528073431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607361137a0 0x560736134770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:41.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1ccbfff700 1 --2- 172.21.15.62:0/528073431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x5607361351f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400a010 tx=0x7f1cc4018c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0089adaa9bd4401 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:41.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1caaffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:41.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1caaffd700 1 -- 172.21.15.62:0/528073431 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4009b00 con 0x560736119880 2024-06-10T00:39:41.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1caaffd700 1 -- 172.21.15.62:0/528073431 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f1cc4017a30 con 0x560736119880 2024-06-10T00:39:41.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/528073431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cc400adf0 con 0x560736119880 2024-06-10T00:39:41.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1caaffd700 1 -- 172.21.15.62:0/528073431 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1cc4036050 con 0x560736119880 2024-06-10T00:39:41.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1cdc9dcb80 10 client.16560 fetch_fsmap learned FSMap version 37 2024-06-10T00:39:41.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/528073431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560736197db0 con 0x560736119880 2024-06-10T00:39:41.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.403+0000 7f1caaffd700 1 -- 172.21.15.62:0/528073431 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc400adf0 con 0x560736119880 2024-06-10T00:39:41.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.404+0000 7f1caaffd700 1 -- 172.21.15.62:0/528073431 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f1cc4022430 con 0x560736119880 2024-06-10T00:39:41.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.404+0000 7f1cdc9dcb80 10 client.16560 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T00:39:41.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.404+0000 7f1cdc9dcb80 10 client.16560 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:39:41.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.404+0000 7f1cdc9dcb80 20 client.16560 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:41.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.404+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/528073431 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56073612ce60 0x56073619a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:41.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.404+0000 7f1cdc9dcb80 4 client.16560 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:39:41.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.404+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/528073431 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56073606c2d0 con 0x56073612ce60 2024-06-10T00:39:41.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.404+0000 7f1ccb7fe700 1 --2- 172.21.15.62:0/528073431 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56073612ce60 0x56073619a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:41.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1ccb7fe700 1 --2- 172.21.15.62:0/528073431 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56073612ce60 0x56073619a410 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f024e666cc95173d server_cookie=43cd5849ad8a6de4 in_seq=0 out_seq=0 2024-06-10T00:39:41.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1caaffd700 10 client.16560 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:39:41.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1caaffd700 1 -- 172.21.15.62:0/528073431 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56073606c2d0 con 0x56073612ce60 2024-06-10T00:39:41.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1caaffd700 10 client.16560 handle_command_reply: tid=0 2024-06-10T00:39:41.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1cdc9dcb80 1 client.16560 shutdown 2024-06-10T00:39:41.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/528073431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 msgr2=0x5607361351f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:41.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/528073431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x5607361351f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400a010 tx=0x7f1cc4018c90 comp rx=0 tx=0).stop 2024-06-10T00:39:41.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/528073431 shutdown_connections 2024-06-10T00:39:41.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/528073431 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x56073612ce60 0x56073619a410 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1caaffd700 0 client.16560 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:39:41.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/528073431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560736119880 0x5607361351f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/528073431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607361137a0 0x560736134770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1cdc9dcb80 1 --2- 172.21.15.62:0/528073431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560736118f60 0x560736134cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/528073431 >> 172.21.15.62:0/528073431 conn(0x560736040080 msgr2=0x56073611c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:41.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/528073431 shutdown_connections 2024-06-10T00:39:41.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1cdc9dcb80 1 -- 172.21.15.62:0/528073431 wait complete. 2024-06-10T00:39:41.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.405+0000 7f1cdc9dcb80 20 client.16560 trim_cache size 0 max 16384 2024-06-10T00:39:41.417 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:39:41.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.406+0000 7f1cd2159700 1 -- 172.21.15.62:0/4229733359 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1cb40a2d90 msgr2=0x7f1cb40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:41.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.406+0000 7f1cd2159700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1cb40a2d90 0x7f1cb40a5250 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc078950 tx=0x7f1cc000b5f0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.406+0000 7f1cd2159700 1 -- 172.21.15.62:0/4229733359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ccc06ac60 msgr2=0x7f1ccc101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:41.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.406+0000 7f1cd2159700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ccc06ac60 0x7f1ccc101400 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc006010 tx=0x7f1cbc009f80 comp rx=0 tx=0).stop 2024-06-10T00:39:41.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.406+0000 7f1cd2159700 1 -- 172.21.15.62:0/4229733359 shutdown_connections 2024-06-10T00:39:41.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.406+0000 7f1cd2159700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1cb40a2d90 0x7f1cb40a5250 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.406+0000 7f1cd2159700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ccc06ac60 0x7f1ccc101400 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.406+0000 7f1cd2159700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ccc108580 0x7f1ccc101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.406+0000 7f1cd2159700 1 --2- 172.21.15.62:0/4229733359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ccc06b620 0x7f1ccc101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:41.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.406+0000 7f1cd2159700 1 -- 172.21.15.62:0/4229733359 >> 172.21.15.62:0/4229733359 conn(0x7f1ccc078430 msgr2=0x7f1ccc0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:41.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.406+0000 7f1cd2159700 1 -- 172.21.15.62:0/4229733359 shutdown_connections 2024-06-10T00:39:41.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:41.406+0000 7f1cd2159700 1 -- 172.21.15.62:0/4229733359 wait complete. 2024-06-10T00:39:41.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:41 smithi105 ceph-mon[36321]: pgmap v208: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:41.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:41 smithi138 ceph-mon[36549]: pgmap v208: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:42.274 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T00:39:42.527 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:39:42.531 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:42 smithi105 ceph-mon[36321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:39:42.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:42 smithi062 ceph-mon[32390]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:39:42.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:42 smithi138 ceph-mon[36549]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:39:43.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:43 smithi062 ceph-mon[32390]: pgmap v209: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T00:39:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:43 smithi105 ceph-mon[36321]: pgmap v209: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T00:39:43.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:43 smithi138 ceph-mon[36549]: pgmap v209: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T00:39:44.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.348+0000 7f32a2265700 1 -- 172.21.15.62:0/1915583279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329c1073f0 msgr2=0x7f329c1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:44.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.348+0000 7f32a2265700 1 --2- 172.21.15.62:0/1915583279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329c1073f0 0x7f329c1077d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f328c009990 tx=0x7f328c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.348+0000 7f32a2265700 1 -- 172.21.15.62:0/1915583279 shutdown_connections 2024-06-10T00:39:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.348+0000 7f32a2265700 1 --2- 172.21.15.62:0/1915583279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329c1073f0 0x7f329c1077d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.348+0000 7f32a2265700 1 --2- 172.21.15.62:0/1915583279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f329c101d40 0x7f329c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.348+0000 7f32a2265700 1 --2- 172.21.15.62:0/1915583279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c101380 0x7f329c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.348+0000 7f32a2265700 1 -- 172.21.15.62:0/1915583279 >> 172.21.15.62:0/1915583279 conn(0x7f329c078300 msgr2=0x7f329c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.348+0000 7f32a2265700 1 -- 172.21.15.62:0/1915583279 shutdown_connections 2024-06-10T00:39:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.348+0000 7f32a2265700 1 -- 172.21.15.62:0/1915583279 wait complete. 2024-06-10T00:39:44.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.349+0000 7f32a2265700 1 Processor -- start 2024-06-10T00:39:44.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.349+0000 7f32a2265700 1 -- start start 2024-06-10T00:39:44.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.349+0000 7f32a2265700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329c101380 0x7f329c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.349+0000 7f32a2265700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c101d40 0x7f329c19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.349+0000 7f32a2265700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f329c1073f0 0x7f329c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.349+0000 7f32a2265700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1116b0 con 0x7f329c101380 2024-06-10T00:39:44.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.349+0000 7f32a2265700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f329c111530 con 0x7f329c1073f0 2024-06-10T00:39:44.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.349+0000 7f32a2265700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f329c111830 con 0x7f329c101d40 2024-06-10T00:39:44.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.349+0000 7f329bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329c101380 0x7f329c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.349+0000 7f329bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329c101380 0x7f329c19e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38706/0 (socket says 172.21.15.62:38706) 2024-06-10T00:39:44.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.349+0000 7f329bfff700 1 -- 172.21.15.62:0/2451865229 learned_addr learned my addr 172.21.15.62:0/2451865229 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:44.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.350+0000 7f329b7fe700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c101d40 0x7f329c19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.350+0000 7f32a0802700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f329c1073f0 0x7f329c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.350+0000 7f329bfff700 1 -- 172.21.15.62:0/2451865229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c101d40 msgr2=0x7f329c19ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:44.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.350+0000 7f329bfff700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c101d40 0x7f329c19ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.350+0000 7f329bfff700 1 -- 172.21.15.62:0/2451865229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f329c1073f0 msgr2=0x7f329c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:44.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.350+0000 7f329bfff700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f329c1073f0 0x7f329c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.350+0000 7f329bfff700 1 -- 172.21.15.62:0/2451865229 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c1991c0 con 0x7f329c101380 2024-06-10T00:39:44.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.350+0000 7f32a0802700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f329c1073f0 0x7f329c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:44.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.350+0000 7f329bfff700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329c101380 0x7f329c19e8a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f328c005910 tx=0x7f328c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:44.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.350+0000 7f32997fa700 1 -- 172.21.15.62:0/2451865229 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c027070 con 0x7f329c101380 2024-06-10T00:39:44.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.350+0000 7f32997fa700 1 -- 172.21.15.62:0/2451865229 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f328c004050 con 0x7f329c101380 2024-06-10T00:39:44.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.350+0000 7f32a2265700 1 -- 172.21.15.62:0/2451865229 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329c199450 con 0x7f329c101380 2024-06-10T00:39:44.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.350+0000 7f32997fa700 1 -- 172.21.15.62:0/2451865229 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c00f460 con 0x7f329c101380 2024-06-10T00:39:44.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.350+0000 7f32a2265700 1 -- 172.21.15.62:0/2451865229 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329c1a5770 con 0x7f329c101380 2024-06-10T00:39:44.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.353+0000 7f32997fa700 1 -- 172.21.15.62:0/2451865229 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f328c02a070 con 0x7f329c101380 2024-06-10T00:39:44.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.354+0000 7f32997fa700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32840a2d90 0x7f32840a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.354+0000 7f32997fa700 1 -- 172.21.15.62:0/2451865229 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f328c0d07b0 con 0x7f329c101380 2024-06-10T00:39:44.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.354+0000 7f329b7fe700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32840a2d90 0x7f32840a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.354+0000 7f32acae8b80 1 Processor -- start 2024-06-10T00:39:44.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.354+0000 7f32acae8b80 1 -- start start 2024-06-10T00:39:44.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f32acae8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b083640 0x564a7b083a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f32acae8b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083f60 0x564a7b07d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f32acae8b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b07df20 0x564a7b07e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f329bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b083640 0x564a7b083a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f329b7fe700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32840a2d90 0x7f32840a5250 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f329c106210 tx=0x7f329000a380 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f329bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b083640 0x564a7b083a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38708/0 (socket says 172.21.15.62:38708) 2024-06-10T00:39:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f329bfff700 1 -- 172.21.15.62:0/3515812051 learned_addr learned my addr 172.21.15.62:0/3515812051 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:44.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f32acae8b80 1 -- 172.21.15.62:0/3515812051 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564a7afcf810 con 0x564a7b083640 2024-06-10T00:39:44.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f32acae8b80 1 -- 172.21.15.62:0/3515812051 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564a7af77720 con 0x564a7b083f60 2024-06-10T00:39:44.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f32acae8b80 1 -- 172.21.15.62:0/3515812051 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564a7afcb7d0 con 0x564a7b07df20 2024-06-10T00:39:44.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f329b7fe700 1 --2- 172.21.15.62:0/3515812051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083f60 0x564a7b07d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f32a0802700 1 --2- 172.21.15.62:0/3515812051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b07df20 0x564a7b07e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f329bfff700 1 -- 172.21.15.62:0/3515812051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b07df20 msgr2=0x564a7b07e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:44.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f329bfff700 1 --2- 172.21.15.62:0/3515812051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b07df20 0x564a7b07e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f329bfff700 1 -- 172.21.15.62:0/3515812051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083f60 msgr2=0x564a7b07d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:44.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f329bfff700 1 --2- 172.21.15.62:0/3515812051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083f60 0x564a7b07d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.355+0000 7f329bfff700 1 -- 172.21.15.62:0/3515812051 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a7af6df50 con 0x564a7b083640 2024-06-10T00:39:44.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.356+0000 7f329bfff700 1 --2- 172.21.15.62:0/3515812051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b083640 0x564a7b083a20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f328c098ed0 tx=0x7f328c09c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b896efaa9978b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:44.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.356+0000 7f32827fc700 1 -- 172.21.15.62:0/3515812051 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c09d170 con 0x564a7b083640 2024-06-10T00:39:44.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.356+0000 7f32827fc700 1 -- 172.21.15.62:0/3515812051 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f328c09ca70 con 0x564a7b083640 2024-06-10T00:39:44.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.356+0000 7f32827fc700 1 -- 172.21.15.62:0/3515812051 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c0d0360 con 0x564a7b083640 2024-06-10T00:39:44.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.356+0000 7f32acae8b80 1 -- 172.21.15.62:0/3515812051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b083640 msgr2=0x564a7b083a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:44.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.356+0000 7f32acae8b80 1 --2- 172.21.15.62:0/3515812051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b083640 0x564a7b083a20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f328c098ed0 tx=0x7f328c09c8c0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.357+0000 7f32acae8b80 1 -- 172.21.15.62:0/3515812051 shutdown_connections 2024-06-10T00:39:44.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.357+0000 7f32acae8b80 1 --2- 172.21.15.62:0/3515812051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b083640 0x564a7b083a20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.357+0000 7f32acae8b80 1 --2- 172.21.15.62:0/3515812051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b07df20 0x564a7b07e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.357+0000 7f32acae8b80 1 --2- 172.21.15.62:0/3515812051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083f60 0x564a7b07d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.357+0000 7f32acae8b80 1 -- 172.21.15.62:0/3515812051 >> 172.21.15.62:0/3515812051 conn(0x564a7afa8cc0 msgr2=0x564a7b078850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:44.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.357+0000 7f32acae8b80 1 -- 172.21.15.62:0/3515812051 shutdown_connections 2024-06-10T00:39:44.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.357+0000 7f32acae8b80 1 -- 172.21.15.62:0/3515812051 wait complete. 2024-06-10T00:39:44.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.357+0000 7f32acae8b80 1 -- start start 2024-06-10T00:39:44.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.357+0000 7f32acae8b80 1 Processor -- start 2024-06-10T00:39:44.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f32acae8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b089c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f32acae8b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083640 0x564a7b087010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f32acae8b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083f60 0x564a7b087550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f32acae8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564a7b08a6f0 con 0x564a7b07df20 2024-06-10T00:39:44.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f32acae8b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564a7af6fbf0 con 0x564a7b083f60 2024-06-10T00:39:44.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f32acae8b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564a7b08a870 con 0x564a7b083640 2024-06-10T00:39:44.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f329bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b089c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f32acae8b80 10 client.0 _ll_register_callbacks cb 0x564a7af69920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:44.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f329bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b089c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38722/0 (socket says 172.21.15.62:38722) 2024-06-10T00:39:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f329bfff700 1 -- 172.21.15.62:0/609452294 learned_addr learned my addr 172.21.15.62:0/609452294 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f329b7fe700 1 --2- 172.21.15.62:0/609452294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083640 0x564a7b087010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f32a0802700 1 --2- 172.21.15.62:0/609452294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083f60 0x564a7b087550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f329bfff700 1 -- 172.21.15.62:0/609452294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083640 msgr2=0x564a7b087010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f329bfff700 1 --2- 172.21.15.62:0/609452294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083640 0x564a7b087010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f329bfff700 1 -- 172.21.15.62:0/609452294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083f60 msgr2=0x564a7b087550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f329bfff700 1 --2- 172.21.15.62:0/609452294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083f60 0x564a7b087550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f329bfff700 1 -- 172.21.15.62:0/609452294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a7af1e8d0 con 0x564a7b07df20 2024-06-10T00:39:44.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f329b7fe700 1 --2- 172.21.15.62:0/609452294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083640 0x564a7b087010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:44.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.358+0000 7f329bfff700 1 --2- 172.21.15.62:0/609452294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b089c70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f328c0a5120 tx=0x7f328c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876b95ceb2c44fa4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:44.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.359+0000 7f327b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:44.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.359+0000 7f327b7fe700 1 -- 172.21.15.62:0/609452294 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c0a5650 con 0x564a7b07df20 2024-06-10T00:39:44.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.359+0000 7f327b7fe700 1 -- 172.21.15.62:0/609452294 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f328c0d0360 con 0x564a7b07df20 2024-06-10T00:39:44.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.359+0000 7f32acae8b80 1 -- 172.21.15.62:0/609452294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f328c09ca70 con 0x564a7b07df20 2024-06-10T00:39:44.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.359+0000 7f327b7fe700 1 -- 172.21.15.62:0/609452294 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f328c0d0560 con 0x564a7b07df20 2024-06-10T00:39:44.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.359+0000 7f32acae8b80 10 client.16572 fetch_fsmap learned FSMap version 37 2024-06-10T00:39:44.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.359+0000 7f32acae8b80 1 -- 172.21.15.62:0/609452294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564a7b08a6f0 con 0x564a7b07df20 2024-06-10T00:39:44.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.359+0000 7f327b7fe700 1 -- 172.21.15.62:0/609452294 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f328c09ca70 con 0x564a7b07df20 2024-06-10T00:39:44.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.359+0000 7f327b7fe700 1 -- 172.21.15.62:0/609452294 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f328c0a30e0 con 0x564a7b07df20 2024-06-10T00:39:44.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.359+0000 7f32acae8b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T00:39:44.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.359+0000 7f32acae8b80 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:39:44.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.359+0000 7f32acae8b80 20 client.16572 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:44.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.359+0000 7f32acae8b80 1 --2- 172.21.15.62:0/609452294 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564a7b097310 0x564a7b099750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.360+0000 7f32acae8b80 4 client.16572 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:39:44.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.360+0000 7f32acae8b80 1 -- 172.21.15.62:0/609452294 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a7afdcd10 con 0x564a7b097310 2024-06-10T00:39:44.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.360+0000 7f329b7fe700 1 --2- 172.21.15.62:0/609452294 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564a7b097310 0x564a7b099750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.360+0000 7f329b7fe700 1 --2- 172.21.15.62:0/609452294 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564a7b097310 0x564a7b099750 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed9e7b1548cf18f1 server_cookie=c3030f55efbe87bc in_seq=0 out_seq=0 2024-06-10T00:39:44.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.360+0000 7f327b7fe700 10 client.16572 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.361+0000 7f327b7fe700 1 -- 172.21.15.62:0/609452294 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564a7afdcd10 con 0x564a7b097310 2024-06-10T00:39:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.361+0000 7f327b7fe700 10 client.16572 handle_command_reply: tid=0 2024-06-10T00:39:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.361+0000 7f32acae8b80 1 client.16572 shutdown 2024-06-10T00:39:44.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.361+0000 7f32acae8b80 1 -- 172.21.15.62:0/609452294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 msgr2=0x564a7b089c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:44.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.361+0000 7f32acae8b80 1 --2- 172.21.15.62:0/609452294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b089c70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f328c0a5120 tx=0x7f328c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.361+0000 7f32acae8b80 1 -- 172.21.15.62:0/609452294 shutdown_connections 2024-06-10T00:39:44.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.361+0000 7f32acae8b80 1 --2- 172.21.15.62:0/609452294 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564a7b097310 0x564a7b099750 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.361+0000 7f32acae8b80 1 --2- 172.21.15.62:0/609452294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b089c70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.361+0000 7f32acae8b80 1 --2- 172.21.15.62:0/609452294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083f60 0x564a7b087550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.361+0000 7f32acae8b80 1 --2- 172.21.15.62:0/609452294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083640 0x564a7b087010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.361+0000 7f32acae8b80 1 -- 172.21.15.62:0/609452294 >> 172.21.15.62:0/609452294 conn(0x564a7afa8cc0 msgr2=0x564a7b0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:44.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.362+0000 7f327b7fe700 0 client.16572 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:44.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.362+0000 7f32acae8b80 1 -- 172.21.15.62:0/609452294 shutdown_connections 2024-06-10T00:39:44.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.362+0000 7f32acae8b80 1 -- 172.21.15.62:0/609452294 wait complete. 2024-06-10T00:39:44.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.362+0000 7f32acae8b80 20 client.16572 trim_cache size 0 max 16384 2024-06-10T00:39:44.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.377+0000 7f32acae8b80 1 Processor -- start 2024-06-10T00:39:44.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.377+0000 7f32acae8b80 1 -- start start 2024-06-10T00:39:44.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.377+0000 7f32acae8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b07d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.377+0000 7f32acae8b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083640 0x564a7b09f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.377+0000 7f32acae8b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083f60 0x564a7b09f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.377+0000 7f32acae8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1ac2a0 con 0x564a7b07df20 2024-06-10T00:39:44.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.377+0000 7f32acae8b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1ade10 con 0x564a7b083640 2024-06-10T00:39:44.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.377+0000 7f32acae8b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1ac970 con 0x564a7b083f60 2024-06-10T00:39:44.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f329bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b07d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f329bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b07d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38738/0 (socket says 172.21.15.62:38738) 2024-06-10T00:39:44.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f329bfff700 1 -- 172.21.15.62:0/937892413 learned_addr learned my addr 172.21.15.62:0/937892413 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:44.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f329b7fe700 1 --2- 172.21.15.62:0/937892413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083640 0x564a7b09f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f329bfff700 1 -- 172.21.15.62:0/937892413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083f60 msgr2=0x564a7b09f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:44.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f32a0802700 1 --2- 172.21.15.62:0/937892413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083f60 0x564a7b09f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f329bfff700 1 --2- 172.21.15.62:0/937892413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083f60 0x564a7b09f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f329bfff700 1 -- 172.21.15.62:0/937892413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083640 msgr2=0x564a7b09f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f329bfff700 1 --2- 172.21.15.62:0/937892413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083640 0x564a7b09f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f329bfff700 1 -- 172.21.15.62:0/937892413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a7b0855a0 con 0x564a7b07df20 2024-06-10T00:39:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f32a0802700 1 --2- 172.21.15.62:0/937892413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083f60 0x564a7b09f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:39:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f329b7fe700 1 --2- 172.21.15.62:0/937892413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083640 0x564a7b09f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:39:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f329bfff700 1 --2- 172.21.15.62:0/937892413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b07d9e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f328c0096f0 tx=0x7f328c0229d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c234a95195e0fdc1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:44.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f3281ffb700 1 -- 172.21.15.62:0/937892413 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c09c5b0 con 0x564a7b07df20 2024-06-10T00:39:44.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f3281ffb700 1 -- 172.21.15.62:0/937892413 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f328c0a51c0 con 0x564a7b07df20 2024-06-10T00:39:44.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.378+0000 7f3281ffb700 1 -- 172.21.15.62:0/937892413 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c0a30b0 con 0x564a7b07df20 2024-06-10T00:39:44.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.379+0000 7f32acae8b80 1 -- 172.21.15.62:0/937892413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 msgr2=0x564a7b07d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:44.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.379+0000 7f32acae8b80 1 --2- 172.21.15.62:0/937892413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b07d9e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f328c0096f0 tx=0x7f328c0229d0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.379+0000 7f32acae8b80 1 -- 172.21.15.62:0/937892413 shutdown_connections 2024-06-10T00:39:44.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.379+0000 7f32acae8b80 1 --2- 172.21.15.62:0/937892413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b07d9e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.379+0000 7f32acae8b80 1 --2- 172.21.15.62:0/937892413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083f60 0x564a7b09f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.379+0000 7f32acae8b80 1 --2- 172.21.15.62:0/937892413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083640 0x564a7b09f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.379+0000 7f32acae8b80 1 -- 172.21.15.62:0/937892413 >> 172.21.15.62:0/937892413 conn(0x564a7afa8cc0 msgr2=0x564a7b085920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.379+0000 7f32acae8b80 1 -- 172.21.15.62:0/937892413 shutdown_connections 2024-06-10T00:39:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.379+0000 7f32acae8b80 1 -- 172.21.15.62:0/937892413 wait complete. 2024-06-10T00:39:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.380+0000 7f32acae8b80 1 -- start start 2024-06-10T00:39:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.380+0000 7f32acae8b80 1 Processor -- start 2024-06-10T00:39:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.380+0000 7f32acae8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b08fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.380+0000 7f32acae8b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083640 0x564a7b090000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.380+0000 7f32acae8b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083f60 0x564a7b09e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.380+0000 7f32acae8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1ad740 con 0x564a7b07df20 2024-06-10T00:39:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.380+0000 7f32acae8b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1ad040 con 0x564a7b083640 2024-06-10T00:39:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.380+0000 7f32acae8b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1aa240 con 0x564a7b083f60 2024-06-10T00:39:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.380+0000 7f32acae8b80 10 client.0 _ll_register_callbacks cb 0x564a7af69920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.381+0000 7f329bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b08fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.381+0000 7f329bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b08fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38740/0 (socket says 172.21.15.62:38740) 2024-06-10T00:39:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.381+0000 7f329bfff700 1 -- 172.21.15.62:0/185269830 learned_addr learned my addr 172.21.15.62:0/185269830 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.381+0000 7f329b7fe700 1 --2- 172.21.15.62:0/185269830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083640 0x564a7b090000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.381+0000 7f32a0802700 1 --2- 172.21.15.62:0/185269830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083f60 0x564a7b09e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.381+0000 7f329b7fe700 1 -- 172.21.15.62:0/185269830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083f60 msgr2=0x564a7b09e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.381+0000 7f329b7fe700 1 --2- 172.21.15.62:0/185269830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083f60 0x564a7b09e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.381+0000 7f329b7fe700 1 -- 172.21.15.62:0/185269830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 msgr2=0x564a7b08fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.381+0000 7f329b7fe700 1 --2- 172.21.15.62:0/185269830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b08fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.381+0000 7f329b7fe700 1 -- 172.21.15.62:0/185269830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a7b09e550 con 0x564a7b083640 2024-06-10T00:39:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.381+0000 7f329bfff700 1 --2- 172.21.15.62:0/185269830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b08fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:39:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.381+0000 7f329b7fe700 1 --2- 172.21.15.62:0/185269830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083640 0x564a7b090000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x564a7b099fc0 tx=0x7f32900047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=554b22329a39182f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.381+0000 7f327b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:39:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.381+0000 7f327b7fe700 1 -- 172.21.15.62:0/185269830 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290016070 con 0x564a7b083640 2024-06-10T00:39:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.382+0000 7f327b7fe700 1 -- 172.21.15.62:0/185269830 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7f3290029470 con 0x564a7b083640 2024-06-10T00:39:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.382+0000 7f32acae8b80 1 -- 172.21.15.62:0/185269830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f328c0a51c0 con 0x564a7b083640 2024-06-10T00:39:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.382+0000 7f327b7fe700 1 -- 172.21.15.62:0/185269830 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3290038050 con 0x564a7b083640 2024-06-10T00:39:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.382+0000 7f32acae8b80 10 client.26006 fetch_fsmap learned FSMap version 37 2024-06-10T00:39:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.382+0000 7f32acae8b80 1 -- 172.21.15.62:0/185269830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564a7b101610 con 0x564a7b083640 2024-06-10T00:39:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.382+0000 7f327b7fe700 1 -- 172.21.15.62:0/185269830 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f328c0a51c0 con 0x564a7b083640 2024-06-10T00:39:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.382+0000 7f327b7fe700 1 -- 172.21.15.62:0/185269830 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7f3290019040 con 0x564a7b083640 2024-06-10T00:39:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.382+0000 7f32acae8b80 10 client.26006 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T00:39:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.382+0000 7f32acae8b80 10 client.26006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:39:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.382+0000 7f32acae8b80 20 client.26006 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.382+0000 7f32acae8b80 1 --2- 172.21.15.62:0/185269830 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564a7b101d20 0x564a7b104140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.382+0000 7f32acae8b80 4 client.26006 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:39:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.382+0000 7f32acae8b80 1 -- 172.21.15.62:0/185269830 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a7afd7290 con 0x564a7b101d20 2024-06-10T00:39:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.383+0000 7f329bfff700 1 --2- 172.21.15.62:0/185269830 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564a7b101d20 0x564a7b104140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.383+0000 7f327b7fe700 10 client.26006 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.383+0000 7f329bfff700 1 --2- 172.21.15.62:0/185269830 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564a7b101d20 0x564a7b104140 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69e4e0053acd054f server_cookie=98f472ed4d7b2997 in_seq=0 out_seq=0 2024-06-10T00:39:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.383+0000 7f327b7fe700 1 -- 172.21.15.62:0/185269830 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a7afd7290 con 0x564a7b101d20 2024-06-10T00:39:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.383+0000 7f327b7fe700 10 client.26006 handle_command_reply: tid=0 2024-06-10T00:39:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.383+0000 7f32acae8b80 1 client.26006 shutdown 2024-06-10T00:39:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.383+0000 7f32acae8b80 1 -- 172.21.15.62:0/185269830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083640 msgr2=0x564a7b090000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.383+0000 7f32acae8b80 1 --2- 172.21.15.62:0/185269830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083640 0x564a7b090000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x564a7b099fc0 tx=0x7f32900047c0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32acae8b80 1 -- 172.21.15.62:0/185269830 shutdown_connections 2024-06-10T00:39:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32acae8b80 1 --2- 172.21.15.62:0/185269830 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564a7b101d20 0x564a7b104140 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32acae8b80 1 --2- 172.21.15.62:0/185269830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564a7b07df20 0x564a7b08fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32acae8b80 1 --2- 172.21.15.62:0/185269830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a7b083f60 0x564a7b09e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f327b7fe700 0 client.26006 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32acae8b80 1 --2- 172.21.15.62:0/185269830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a7b083640 0x564a7b090000 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32acae8b80 1 -- 172.21.15.62:0/185269830 >> 172.21.15.62:0/185269830 conn(0x564a7afa8cc0 msgr2=0x564a7b071e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32acae8b80 1 -- 172.21.15.62:0/185269830 shutdown_connections 2024-06-10T00:39:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32acae8b80 1 -- 172.21.15.62:0/185269830 wait complete. 2024-06-10T00:39:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32acae8b80 20 client.26006 trim_cache size 0 max 16384 2024-06-10T00:39:44.394 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:39:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32a2265700 1 -- 172.21.15.62:0/2451865229 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32840a2d90 msgr2=0x7f32840a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32a2265700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32840a2d90 0x7f32840a5250 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f329c106210 tx=0x7f329000a380 comp rx=0 tx=0).stop 2024-06-10T00:39:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32a2265700 1 -- 172.21.15.62:0/2451865229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329c101380 msgr2=0x7f329c19e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32a2265700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329c101380 0x7f329c19e8a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f328c005910 tx=0x7f328c0047b0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32a2265700 1 -- 172.21.15.62:0/2451865229 shutdown_connections 2024-06-10T00:39:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32a2265700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32840a2d90 0x7f32840a5250 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32a2265700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329c101380 0x7f329c19e8a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32a2265700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f329c1073f0 0x7f329c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32a2265700 1 --2- 172.21.15.62:0/2451865229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f329c101d40 0x7f329c19ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.384+0000 7f32a2265700 1 -- 172.21.15.62:0/2451865229 >> 172.21.15.62:0/2451865229 conn(0x7f329c078300 msgr2=0x7f329c100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:44.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.385+0000 7f32a2265700 1 -- 172.21.15.62:0/2451865229 shutdown_connections 2024-06-10T00:39:44.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:44.385+0000 7f32a2265700 1 -- 172.21.15.62:0/2451865229 wait complete. 2024-06-10T00:39:45.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:45 smithi105 ceph-mon[36321]: pgmap v210: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:45.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:45 smithi062 ceph-mon[32390]: pgmap v210: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:45.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:45 smithi138 ceph-mon[36549]: pgmap v210: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:39:46.266 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T00:39:46.267 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T00:39:46.534 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:39:47.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:47 smithi062 ceph-mon[32390]: pgmap v211: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:39:47.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:47 smithi105 ceph-mon[36321]: pgmap v211: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:39:47.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:47 smithi138 ceph-mon[36549]: pgmap v211: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:39:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.377+0000 7fab02212700 1 -- 172.21.15.62:0/1799101560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faafc102ea0 msgr2=0x7faafc10f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:48.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.377+0000 7fab02212700 1 --2- 172.21.15.62:0/1799101560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faafc102ea0 0x7faafc10f380 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7faaf400b0a0 tx=0x7faaf400b3b0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.377+0000 7fab02212700 1 -- 172.21.15.62:0/1799101560 shutdown_connections 2024-06-10T00:39:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.377+0000 7fab02212700 1 --2- 172.21.15.62:0/1799101560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faafc102ea0 0x7faafc10f380 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.377+0000 7fab02212700 1 --2- 172.21.15.62:0/1799101560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faafc1085f0 0x7faafc1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.377+0000 7fab02212700 1 --2- 172.21.15.62:0/1799101560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faafc1024e0 0x7faafc102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.377+0000 7fab02212700 1 -- 172.21.15.62:0/1799101560 >> 172.21.15.62:0/1799101560 conn(0x7faafc079640 msgr2=0x7faafc079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.378+0000 7fab02212700 1 -- 172.21.15.62:0/1799101560 shutdown_connections 2024-06-10T00:39:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.378+0000 7fab02212700 1 -- 172.21.15.62:0/1799101560 wait complete. 2024-06-10T00:39:48.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.378+0000 7fab02212700 1 Processor -- start 2024-06-10T00:39:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.378+0000 7fab02212700 1 -- start start 2024-06-10T00:39:48.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7fab02212700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faafc1024e0 0x7faafc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7fab02212700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faafc102ea0 0x7faafc074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7fab02212700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faafc1085f0 0x7faafc075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7fab02212700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1127b0 con 0x7faafc102ea0 2024-06-10T00:39:48.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7fab02212700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faafc112630 con 0x7faafc1085f0 2024-06-10T00:39:48.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7fab02212700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faafc112930 con 0x7faafc1024e0 2024-06-10T00:39:48.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7faafaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faafc102ea0 0x7faafc074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7faafaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faafc102ea0 0x7faafc074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53576/0 (socket says 172.21.15.62:53576) 2024-06-10T00:39:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7faafaffd700 1 -- 172.21.15.62:0/601186848 learned_addr learned my addr 172.21.15.62:0/601186848 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7faafb7fe700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faafc1024e0 0x7faafc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7faafbfff700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faafc1085f0 0x7faafc075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7faafaffd700 1 -- 172.21.15.62:0/601186848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faafc1024e0 msgr2=0x7faafc074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:48.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7faafaffd700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faafc1024e0 0x7faafc074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7faafaffd700 1 -- 172.21.15.62:0/601186848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faafc1085f0 msgr2=0x7faafc075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:48.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7faafaffd700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faafc1085f0 0x7faafc075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7faafaffd700 1 -- 172.21.15.62:0/601186848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faafc1a7020 con 0x7faafc102ea0 2024-06-10T00:39:48.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7faafbfff700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faafc1085f0 0x7faafc075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:39:48.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.379+0000 7faafaffd700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faafc102ea0 0x7faafc074cd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7faaf000c9b0 tx=0x7faaf000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:48.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.380+0000 7faaf8ff9700 1 -- 172.21.15.62:0/601186848 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0007b00 con 0x7faafc102ea0 2024-06-10T00:39:48.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.380+0000 7fab02212700 1 -- 172.21.15.62:0/601186848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faafc1a7310 con 0x7faafc102ea0 2024-06-10T00:39:48.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.380+0000 7faaf8ff9700 1 -- 172.21.15.62:0/601186848 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf0007ca0 con 0x7faafc102ea0 2024-06-10T00:39:48.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.380+0000 7faaf8ff9700 1 -- 172.21.15.62:0/601186848 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0018730 con 0x7faafc102ea0 2024-06-10T00:39:48.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.380+0000 7fab02212700 1 -- 172.21.15.62:0/601186848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faafc1a78a0 con 0x7faafc102ea0 2024-06-10T00:39:48.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.382+0000 7faaf8ff9700 1 -- 172.21.15.62:0/601186848 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7faaf00188d0 con 0x7faafc102ea0 2024-06-10T00:39:48.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.383+0000 7faaf8ff9700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7faae40a2d90 0x7faae40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.383+0000 7faafb7fe700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7faae40a2d90 0x7faae40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.383+0000 7faaf8ff9700 1 -- 172.21.15.62:0/601186848 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7faaf00cf810 con 0x7faafc102ea0 2024-06-10T00:39:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.383+0000 7faafb7fe700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7faae40a2d90 0x7faae40a5250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faaec0045b0 tx=0x7faaec00b3c0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.383+0000 7fab0ca95b80 1 Processor -- start 2024-06-10T00:39:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.383+0000 7fab0ca95b80 1 -- start start 2024-06-10T00:39:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7fab0ca95b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b07fe0 0x555eb7b083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7fab0ca95b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b08900 0x555eb7b022e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7fab0ca95b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b02820 0x555eb7b02c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7fab0ca95b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555eb79f4fe0 con 0x555eb7b08900 2024-06-10T00:39:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7fab0ca95b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555eb7a538d0 con 0x555eb7b07fe0 2024-06-10T00:39:48.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7fab0ca95b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555eb79bb4e0 con 0x555eb7b02820 2024-06-10T00:39:48.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7faafaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b08900 0x555eb7b022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7faafaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b08900 0x555eb7b022e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53580/0 (socket says 172.21.15.62:53580) 2024-06-10T00:39:48.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7faafaffd700 1 -- 172.21.15.62:0/1611749167 learned_addr learned my addr 172.21.15.62:0/1611749167 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:48.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7faafb7fe700 1 --2- 172.21.15.62:0/1611749167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b07fe0 0x555eb7b083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7faafbfff700 1 --2- 172.21.15.62:0/1611749167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b02820 0x555eb7b02c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7faafaffd700 1 -- 172.21.15.62:0/1611749167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b02820 msgr2=0x555eb7b02c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:48.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7faafaffd700 1 --2- 172.21.15.62:0/1611749167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b02820 0x555eb7b02c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7faafaffd700 1 -- 172.21.15.62:0/1611749167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b07fe0 msgr2=0x555eb7b083c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:48.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7faafaffd700 1 --2- 172.21.15.62:0/1611749167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b07fe0 0x555eb7b083c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7faafaffd700 1 -- 172.21.15.62:0/1611749167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb79a2950 con 0x555eb7b08900 2024-06-10T00:39:48.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7faafbfff700 1 --2- 172.21.15.62:0/1611749167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b02820 0x555eb7b02c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:39:48.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7faafaffd700 1 --2- 172.21.15.62:0/1611749167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b08900 0x555eb7b022e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7faaf0097de0 tx=0x7faaf009b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b604bb2b0b11b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:48.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.384+0000 7faae1ffb700 1 -- 172.21.15.62:0/1611749167 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf009fd30 con 0x555eb7b08900 2024-06-10T00:39:48.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.385+0000 7faae1ffb700 1 -- 172.21.15.62:0/1611749167 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf00a45f0 con 0x555eb7b08900 2024-06-10T00:39:48.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.385+0000 7faae1ffb700 1 -- 172.21.15.62:0/1611749167 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf009bbe0 con 0x555eb7b08900 2024-06-10T00:39:48.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.385+0000 7fab0ca95b80 1 -- 172.21.15.62:0/1611749167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b08900 msgr2=0x555eb7b022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:48.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.385+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/1611749167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b08900 0x555eb7b022e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7faaf0097de0 tx=0x7faaf009b2e0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.385+0000 7fab0ca95b80 1 -- 172.21.15.62:0/1611749167 shutdown_connections 2024-06-10T00:39:48.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.385+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/1611749167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b02820 0x555eb7b02c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.385+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/1611749167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b07fe0 0x555eb7b083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.385+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/1611749167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b08900 0x555eb7b022e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.385+0000 7fab0ca95b80 1 -- 172.21.15.62:0/1611749167 >> 172.21.15.62:0/1611749167 conn(0x555eb7a2f100 msgr2=0x555eb7afd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.385+0000 7fab0ca95b80 1 -- 172.21.15.62:0/1611749167 shutdown_connections 2024-06-10T00:39:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.385+0000 7fab0ca95b80 1 -- 172.21.15.62:0/1611749167 wait complete. 2024-06-10T00:39:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.386+0000 7fab0ca95b80 1 -- start start 2024-06-10T00:39:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.386+0000 7fab0ca95b80 1 Processor -- start 2024-06-10T00:39:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.386+0000 7fab0ca95b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b02820 0x555eb7b0e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.386+0000 7fab0ca95b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b07fe0 0x555eb7b0ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.386+0000 7fab0ca95b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7b0bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.386+0000 7fab0ca95b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555eb79f4040 con 0x555eb7b02820 2024-06-10T00:39:48.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.386+0000 7fab0ca95b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555eb7a4f890 con 0x555eb7b07fe0 2024-06-10T00:39:48.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.386+0000 7fab0ca95b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555eb7b0f2e0 con 0x555eb7b08900 2024-06-10T00:39:48.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.386+0000 7fab0ca95b80 10 client.0 _ll_register_callbacks cb 0x555eb79e9630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:48.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.386+0000 7faafb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b02820 0x555eb7b0e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.386+0000 7faafb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b02820 0x555eb7b0e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53594/0 (socket says 172.21.15.62:53594) 2024-06-10T00:39:48.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faafb7fe700 1 -- 172.21.15.62:0/680651109 learned_addr learned my addr 172.21.15.62:0/680651109 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:48.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faafbfff700 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7b0bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faafaffd700 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b07fe0 0x555eb7b0ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faafb7fe700 1 -- 172.21.15.62:0/680651109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 msgr2=0x555eb7b0bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:48.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faafb7fe700 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7b0bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faafb7fe700 1 -- 172.21.15.62:0/680651109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b07fe0 msgr2=0x555eb7b0ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:48.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faafb7fe700 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b07fe0 0x555eb7b0ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faafb7fe700 1 -- 172.21.15.62:0/680651109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb7a197d0 con 0x555eb7b02820 2024-06-10T00:39:48.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faafbfff700 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7b0bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:48.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faafaffd700 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b07fe0 0x555eb7b0ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:48.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faafb7fe700 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b02820 0x555eb7b0e710 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7faaec003e20 tx=0x7faaec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891a8a44b66f5d7a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:48.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faadaffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faadaffd700 1 -- 172.21.15.62:0/680651109 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec011ce0 con 0x555eb7b02820 2024-06-10T00:39:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faadaffd700 1 -- 172.21.15.62:0/680651109 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7faaec028470 con 0x555eb7b02820 2024-06-10T00:39:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7fab0ca95b80 1 -- 172.21.15.62:0/680651109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaf00a45f0 con 0x555eb7b02820 2024-06-10T00:39:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7faadaffd700 1 -- 172.21.15.62:0/680651109 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7faaec031050 con 0x555eb7b02820 2024-06-10T00:39:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7fab0ca95b80 10 client.16590 fetch_fsmap learned FSMap version 37 2024-06-10T00:39:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.387+0000 7fab0ca95b80 1 -- 172.21.15.62:0/680651109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555eb7b19b50 con 0x555eb7b02820 2024-06-10T00:39:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.388+0000 7faadaffd700 1 -- 172.21.15.62:0/680651109 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf00a45f0 con 0x555eb7b02820 2024-06-10T00:39:48.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.388+0000 7faadaffd700 1 -- 172.21.15.62:0/680651109 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7faaec01f620 con 0x555eb7b02820 2024-06-10T00:39:48.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.388+0000 7fab0ca95b80 10 client.16590 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T00:39:48.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.388+0000 7fab0ca95b80 10 client.16590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:39:48.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.388+0000 7fab0ca95b80 20 client.16590 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:48.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.388+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555eb7b1c190 0x555eb7b1e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.388+0000 7fab0ca95b80 4 client.16590 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:39:48.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.388+0000 7fab0ca95b80 1 -- 172.21.15.62:0/680651109 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eb7a60dd0 con 0x555eb7b1c190 2024-06-10T00:39:48.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.388+0000 7faafaffd700 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555eb7b1c190 0x555eb7b1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.389+0000 7faafaffd700 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555eb7b1c190 0x555eb7b1e580 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd20dea9bd8adc4 server_cookie=a2492c81daefd596 in_seq=0 out_seq=0 2024-06-10T00:39:48.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.389+0000 7faadaffd700 10 client.16590 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:48.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.389+0000 7faadaffd700 1 -- 172.21.15.62:0/680651109 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555eb7a60dd0 con 0x555eb7b1c190 2024-06-10T00:39:48.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.389+0000 7faadaffd700 10 client.16590 handle_command_reply: tid=0 2024-06-10T00:39:48.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.389+0000 7fab0ca95b80 1 client.16590 shutdown 2024-06-10T00:39:48.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.390+0000 7fab0ca95b80 1 -- 172.21.15.62:0/680651109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b02820 msgr2=0x555eb7b0e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:48.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.390+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b02820 0x555eb7b0e710 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7faaec003e20 tx=0x7faaec0047c0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.390+0000 7fab0ca95b80 1 -- 172.21.15.62:0/680651109 shutdown_connections 2024-06-10T00:39:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.390+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555eb7b1c190 0x555eb7b1e580 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.390+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b02820 0x555eb7b0e710 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.390+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7b0bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.390+0000 7faadaffd700 0 client.16590 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.390+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/680651109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b07fe0 0x555eb7b0ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.390+0000 7fab0ca95b80 1 -- 172.21.15.62:0/680651109 >> 172.21.15.62:0/680651109 conn(0x555eb7a2f100 msgr2=0x555eb7afc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:48.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.390+0000 7fab0ca95b80 1 -- 172.21.15.62:0/680651109 shutdown_connections 2024-06-10T00:39:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.390+0000 7fab0ca95b80 1 -- 172.21.15.62:0/680651109 wait complete. 2024-06-10T00:39:48.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.390+0000 7fab0ca95b80 20 client.16590 trim_cache size 0 max 16384 2024-06-10T00:39:48.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.410+0000 7fab0ca95b80 1 Processor -- start 2024-06-10T00:39:48.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.410+0000 7fab0ca95b80 1 -- start start 2024-06-10T00:39:48.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7fab0ca95b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b02820 0x555eb7b093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7fab0ca95b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b07fe0 0x555eb7b0e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7fab0ca95b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7afcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7fab0ca95b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b0800 con 0x555eb7b07fe0 2024-06-10T00:39:48.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7fab0ca95b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b2370 con 0x555eb7b02820 2024-06-10T00:39:48.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7fab0ca95b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b0ed0 con 0x555eb7b08900 2024-06-10T00:39:48.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faafaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b07fe0 0x555eb7b0e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faafaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b07fe0 0x555eb7b0e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53602/0 (socket says 172.21.15.62:53602) 2024-06-10T00:39:48.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faafaffd700 1 -- 172.21.15.62:0/3785468981 learned_addr learned my addr 172.21.15.62:0/3785468981 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:48.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faafbfff700 1 --2- 172.21.15.62:0/3785468981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7afcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faafb7fe700 1 --2- 172.21.15.62:0/3785468981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b02820 0x555eb7b093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faafaffd700 1 -- 172.21.15.62:0/3785468981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 msgr2=0x555eb7afcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:48.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faafaffd700 1 --2- 172.21.15.62:0/3785468981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7afcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faafaffd700 1 -- 172.21.15.62:0/3785468981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b02820 msgr2=0x555eb7b093b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:48.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faafaffd700 1 --2- 172.21.15.62:0/3785468981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b02820 0x555eb7b093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faafaffd700 1 -- 172.21.15.62:0/3785468981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb7b031a0 con 0x555eb7b07fe0 2024-06-10T00:39:48.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faafbfff700 1 --2- 172.21.15.62:0/3785468981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7afcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:48.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faafaffd700 1 --2- 172.21.15.62:0/3785468981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b07fe0 0x555eb7b0e2c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7faaf000afd0 tx=0x7faaf000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800fac6489c19174 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:48.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faae17fa700 1 -- 172.21.15.62:0/3785468981 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0064070 con 0x555eb7b07fe0 2024-06-10T00:39:48.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faae17fa700 1 -- 172.21.15.62:0/3785468981 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf00a45f0 con 0x555eb7b07fe0 2024-06-10T00:39:48.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.411+0000 7faae17fa700 1 -- 172.21.15.62:0/3785468981 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf009bbe0 con 0x555eb7b07fe0 2024-06-10T00:39:48.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.412+0000 7fab0ca95b80 1 -- 172.21.15.62:0/3785468981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b07fe0 msgr2=0x555eb7b0e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:48.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.412+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/3785468981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b07fe0 0x555eb7b0e2c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7faaf000afd0 tx=0x7faaf000fc30 comp rx=0 tx=0).stop 2024-06-10T00:39:48.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.412+0000 7fab0ca95b80 1 -- 172.21.15.62:0/3785468981 shutdown_connections 2024-06-10T00:39:48.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.412+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/3785468981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7afcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.412+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/3785468981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b02820 0x555eb7b093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.412+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/3785468981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b07fe0 0x555eb7b0e2c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.412+0000 7fab0ca95b80 1 -- 172.21.15.62:0/3785468981 >> 172.21.15.62:0/3785468981 conn(0x555eb7a2f100 msgr2=0x555eb7b07480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:48.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.412+0000 7fab0ca95b80 1 -- 172.21.15.62:0/3785468981 shutdown_connections 2024-06-10T00:39:48.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.412+0000 7fab0ca95b80 1 -- 172.21.15.62:0/3785468981 wait complete. 2024-06-10T00:39:48.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.412+0000 7fab0ca95b80 1 -- start start 2024-06-10T00:39:48.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7fab0ca95b80 1 Processor -- start 2024-06-10T00:39:48.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7fab0ca95b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b02820 0x555eb7b14a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7fab0ca95b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b07fe0 0x555eb7b11540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7fab0ca95b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7b11a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7fab0ca95b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b1ca0 con 0x555eb7b07fe0 2024-06-10T00:39:48.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7fab0ca95b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b15a0 con 0x555eb7b02820 2024-06-10T00:39:48.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7fab0ca95b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1ae7a0 con 0x555eb7b08900 2024-06-10T00:39:48.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7fab0ca95b80 10 client.0 _ll_register_callbacks cb 0x555eb79e9630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:48.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7faafaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b07fe0 0x555eb7b11540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7faafaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b07fe0 0x555eb7b11540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53604/0 (socket says 172.21.15.62:53604) 2024-06-10T00:39:48.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7faafaffd700 1 -- 172.21.15.62:0/3824653663 learned_addr learned my addr 172.21.15.62:0/3824653663 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:48.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7faafbfff700 1 --2- 172.21.15.62:0/3824653663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7b11a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7faafb7fe700 1 --2- 172.21.15.62:0/3824653663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b02820 0x555eb7b14a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7faafaffd700 1 -- 172.21.15.62:0/3824653663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 msgr2=0x555eb7b11a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:48.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7faafaffd700 1 --2- 172.21.15.62:0/3824653663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7b11a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7faafaffd700 1 -- 172.21.15.62:0/3824653663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b02820 msgr2=0x555eb7b14a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:48.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7faafaffd700 1 --2- 172.21.15.62:0/3824653663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b02820 0x555eb7b14a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7faafaffd700 1 -- 172.21.15.62:0/3824653663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb7b12040 con 0x555eb7b07fe0 2024-06-10T00:39:48.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.413+0000 7faafbfff700 1 --2- 172.21.15.62:0/3824653663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7b11a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:48.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7faafaffd700 1 --2- 172.21.15.62:0/3824653663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b07fe0 0x555eb7b11540 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7faaf009b910 tx=0x7faaf009c160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab898a9077acbc6a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7faadaffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7faadaffd700 1 -- 172.21.15.62:0/3824653663 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0064030 con 0x555eb7b07fe0 2024-06-10T00:39:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7faadaffd700 1 -- 172.21.15.62:0/3824653663 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7faaf00ba200 con 0x555eb7b07fe0 2024-06-10T00:39:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7fab0ca95b80 1 -- 172.21.15.62:0/3824653663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaf00a45f0 con 0x555eb7b07fe0 2024-06-10T00:39:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7faadaffd700 1 -- 172.21.15.62:0/3824653663 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7faaf009b2e0 con 0x555eb7b07fe0 2024-06-10T00:39:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7fab0ca95b80 10 client.16596 fetch_fsmap learned FSMap version 37 2024-06-10T00:39:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7fab0ca95b80 1 -- 172.21.15.62:0/3824653663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555eb7b87c40 con 0x555eb7b07fe0 2024-06-10T00:39:48.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7faadaffd700 1 -- 172.21.15.62:0/3824653663 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf0080020 con 0x555eb7b07fe0 2024-06-10T00:39:48.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7faadaffd700 1 -- 172.21.15.62:0/3824653663 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7faaf00a4160 con 0x555eb7b07fe0 2024-06-10T00:39:48.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7fab0ca95b80 10 client.16596 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T00:39:48.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7fab0ca95b80 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:39:48.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7fab0ca95b80 20 client.16596 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:48.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/3824653663 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555eb7b87e70 0x555eb7b8a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:48.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7fab0ca95b80 4 client.16596 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T00:39:48.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.414+0000 7fab0ca95b80 1 -- 172.21.15.62:0/3824653663 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555eb7a5b350 con 0x555eb7b87e70 2024-06-10T00:39:48.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.415+0000 7faafb7fe700 1 --2- 172.21.15.62:0/3824653663 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555eb7b87e70 0x555eb7b8a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:48.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.415+0000 7faadaffd700 10 client.16596 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:48.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.415+0000 7faafb7fe700 1 --2- 172.21.15.62:0/3824653663 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555eb7b87e70 0x555eb7b8a260 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79270abb7fc8d137 server_cookie=49ff729c9b7e893d in_seq=0 out_seq=0 2024-06-10T00:39:48.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.416+0000 7faadaffd700 1 -- 172.21.15.62:0/3824653663 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555eb7a5b350 con 0x555eb7b87e70 2024-06-10T00:39:48.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.416+0000 7faadaffd700 10 client.16596 handle_command_reply: tid=0 2024-06-10T00:39:48.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.416+0000 7fab0ca95b80 1 client.16596 shutdown 2024-06-10T00:39:48.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.416+0000 7fab0ca95b80 1 -- 172.21.15.62:0/3824653663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b07fe0 msgr2=0x555eb7b11540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:48.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.416+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/3824653663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b07fe0 0x555eb7b11540 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7faaf009b910 tx=0x7faaf009c160 comp rx=0 tx=0).stop 2024-06-10T00:39:48.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.416+0000 7fab0ca95b80 1 -- 172.21.15.62:0/3824653663 shutdown_connections 2024-06-10T00:39:48.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.416+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/3824653663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eb7b08900 0x555eb7b11a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.416+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/3824653663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555eb7b02820 0x555eb7b14a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.416+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/3824653663 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555eb7b87e70 0x555eb7b8a260 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.417+0000 7fab0ca95b80 1 --2- 172.21.15.62:0/3824653663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555eb7b07fe0 0x555eb7b11540 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.417+0000 7fab0ca95b80 1 -- 172.21.15.62:0/3824653663 >> 172.21.15.62:0/3824653663 conn(0x555eb7a2f100 msgr2=0x555eb7b06bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:48.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.417+0000 7faadaffd700 0 client.16596 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:48.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.417+0000 7fab0ca95b80 1 -- 172.21.15.62:0/3824653663 shutdown_connections 2024-06-10T00:39:48.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.417+0000 7fab0ca95b80 1 -- 172.21.15.62:0/3824653663 wait complete. 2024-06-10T00:39:48.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.417+0000 7fab0ca95b80 20 client.16596 trim_cache size 0 max 16384 2024-06-10T00:39:48.426 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:39:48.427 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-06-10T00:39:48.427 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "4d7089aa-3d77-4da5-9faf-b97825150439", 2024-06-10T00:39:48.427 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-06-10T00:39:48.427 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:39:48.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.417+0000 7fab02212700 1 -- 172.21.15.62:0/601186848 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7faae40a2d90 msgr2=0x7faae40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:48.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.417+0000 7fab02212700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7faae40a2d90 0x7faae40a5250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faaec0045b0 tx=0x7faaec00b3c0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.417+0000 7fab02212700 1 -- 172.21.15.62:0/601186848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faafc102ea0 msgr2=0x7faafc074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:48.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.417+0000 7fab02212700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faafc102ea0 0x7faafc074cd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7faaf000c9b0 tx=0x7faaf000ccc0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.418+0000 7fab02212700 1 -- 172.21.15.62:0/601186848 shutdown_connections 2024-06-10T00:39:48.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.418+0000 7fab02212700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7faae40a2d90 0x7faae40a5250 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.418+0000 7fab02212700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faafc1085f0 0x7faafc075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.418+0000 7fab02212700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faafc1024e0 0x7faafc074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.418+0000 7fab02212700 1 --2- 172.21.15.62:0/601186848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faafc102ea0 0x7faafc074cd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:48.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.418+0000 7fab02212700 1 -- 172.21.15.62:0/601186848 >> 172.21.15.62:0/601186848 conn(0x7faafc079640 msgr2=0x7faafc100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:48.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.418+0000 7fab02212700 1 -- 172.21.15.62:0/601186848 shutdown_connections 2024-06-10T00:39:48.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:48.418+0000 7fab02212700 1 -- 172.21.15.62:0/601186848 wait complete. 2024-06-10T00:39:49.277 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:39:49.528 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:39:49.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:49 smithi105 ceph-mon[36321]: pgmap v212: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:39:49.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:49 smithi105 ceph-mon[36321]: scrub queued for path: / 2024-06-10T00:39:49.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:49 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:49.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:49 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:39:49.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:49 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:39:49.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:49 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:39:49.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:49 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:39:49.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:49 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:49.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:49 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:49.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:49 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:49.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:49 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:49.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:49 smithi062 ceph-mon[32390]: pgmap v212: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:39:49.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:49 smithi062 ceph-mon[32390]: scrub queued for path: / 2024-06-10T00:39:49.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:49 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:49.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:49 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:39:49.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:49 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:39:49.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:49 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:39:49.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:49 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:39:49.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:49 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:49.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:49 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:49.610 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:49 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:49.610 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:49 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:49.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:49 smithi138 ceph-mon[36549]: pgmap v212: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:39:49.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:49 smithi138 ceph-mon[36549]: scrub queued for path: / 2024-06-10T00:39:49.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:49 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:49.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:49 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:39:49.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:49 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:39:49.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:49 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:39:49.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:49 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:39:49.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:49 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:49.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:49 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:49.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:49 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:49.626 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:49 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:50.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:50 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:39:50.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:50 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:50.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:50 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:39:50.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:50 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:50.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:50 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:39:50.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:50 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:39:51.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.285+0000 7fd484df5700 1 -- 172.21.15.62:0/2927114491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd480101fc0 msgr2=0x7fd480106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:51.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.285+0000 7fd484df5700 1 --2- 172.21.15.62:0/2927114491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd480101fc0 0x7fd480106010 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd4680099c0 tx=0x7fd468009cd0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.285+0000 7fd484df5700 1 -- 172.21.15.62:0/2927114491 shutdown_connections 2024-06-10T00:39:51.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.285+0000 7fd484df5700 1 --2- 172.21.15.62:0/2927114491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4801066f0 0x7fd48010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.285+0000 7fd484df5700 1 --2- 172.21.15.62:0/2927114491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd480101610 0x7fd4801019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.285+0000 7fd484df5700 1 --2- 172.21.15.62:0/2927114491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd480101fc0 0x7fd480106010 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.285+0000 7fd484df5700 1 -- 172.21.15.62:0/2927114491 >> 172.21.15.62:0/2927114491 conn(0x7fd480078430 msgr2=0x7fd480078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:51.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.285+0000 7fd484df5700 1 -- 172.21.15.62:0/2927114491 shutdown_connections 2024-06-10T00:39:51.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.285+0000 7fd484df5700 1 -- 172.21.15.62:0/2927114491 wait complete. 2024-06-10T00:39:51.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd484df5700 1 Processor -- start 2024-06-10T00:39:51.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd484df5700 1 -- start start 2024-06-10T00:39:51.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd484df5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd480101610 0x7fd480198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd484df5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd480101fc0 0x7fd480199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd484df5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4801066f0 0x7fd48019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd484df5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd480111800 con 0x7fd480101610 2024-06-10T00:39:51.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd484df5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd480111680 con 0x7fd480101fc0 2024-06-10T00:39:51.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd484df5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd480111980 con 0x7fd4801066f0 2024-06-10T00:39:51.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd47e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd480101610 0x7fd480198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd47e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd480101610 0x7fd480198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53622/0 (socket says 172.21.15.62:53622) 2024-06-10T00:39:51.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd47e59c700 1 -- 172.21.15.62:0/3259348606 learned_addr learned my addr 172.21.15.62:0/3259348606 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:51.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd47dd9b700 1 --2- 172.21.15.62:0/3259348606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd480101fc0 0x7fd480199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd47ed9d700 1 --2- 172.21.15.62:0/3259348606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4801066f0 0x7fd48019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd47e59c700 1 -- 172.21.15.62:0/3259348606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4801066f0 msgr2=0x7fd48019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:51.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.286+0000 7fd47e59c700 1 --2- 172.21.15.62:0/3259348606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4801066f0 0x7fd48019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.287+0000 7fd47e59c700 1 -- 172.21.15.62:0/3259348606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd480101fc0 msgr2=0x7fd480199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:51.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.287+0000 7fd47e59c700 1 --2- 172.21.15.62:0/3259348606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd480101fc0 0x7fd480199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.287+0000 7fd47e59c700 1 -- 172.21.15.62:0/3259348606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48019de30 con 0x7fd480101610 2024-06-10T00:39:51.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.287+0000 7fd47e59c700 1 --2- 172.21.15.62:0/3259348606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd480101610 0x7fd480198e20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd47000b530 tx=0x7fd47000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:51.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.287+0000 7fd4777fe700 1 -- 172.21.15.62:0/3259348606 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470015020 con 0x7fd480101610 2024-06-10T00:39:51.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.287+0000 7fd4777fe700 1 -- 172.21.15.62:0/3259348606 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd470011d10 con 0x7fd480101610 2024-06-10T00:39:51.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.287+0000 7fd484df5700 1 -- 172.21.15.62:0/3259348606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48019e120 con 0x7fd480101610 2024-06-10T00:39:51.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.287+0000 7fd4777fe700 1 -- 172.21.15.62:0/3259348606 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47000f2e0 con 0x7fd480101610 2024-06-10T00:39:51.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.287+0000 7fd484df5700 1 -- 172.21.15.62:0/3259348606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4801a59a0 con 0x7fd480101610 2024-06-10T00:39:51.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.289+0000 7fd4777fe700 1 -- 172.21.15.62:0/3259348606 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd470018020 con 0x7fd480101610 2024-06-10T00:39:51.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.290+0000 7fd4777fe700 1 --2- 172.21.15.62:0/3259348606 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd46c0a2e10 0x7fd46c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.290+0000 7fd4777fe700 1 -- 172.21.15.62:0/3259348606 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fd4700ced40 con 0x7fd480101610 2024-06-10T00:39:51.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.290+0000 7fd47dd9b700 1 --2- 172.21.15.62:0/3259348606 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd46c0a2e10 0x7fd46c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.291+0000 7fd48f678b80 1 Processor -- start 2024-06-10T00:39:51.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.291+0000 7fd47dd9b700 1 --2- 172.21.15.62:0/3259348606 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd46c0a2e10 0x7fd46c0a52d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd468000c00 tx=0x7fd468005fb0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:51.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.291+0000 7fd48f678b80 1 -- start start 2024-06-10T00:39:51.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.291+0000 7fd48f678b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 0x563eced72750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.291+0000 7fd48f678b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 0x563eced6c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.291+0000 7fd48f678b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced6cbb0 0x563eced6cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.291+0000 7fd48f678b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563ecec5af60 con 0x563eced6cbb0 2024-06-10T00:39:51.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.291+0000 7fd48f678b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563ececb9850 con 0x563eced72370 2024-06-10T00:39:51.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.291+0000 7fd48f678b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563ecec21460 con 0x563eced72c90 2024-06-10T00:39:51.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.291+0000 7fd47ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced6cbb0 0x563eced6cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.291+0000 7fd47ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced6cbb0 0x563eced6cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53634/0 (socket says 172.21.15.62:53634) 2024-06-10T00:39:51.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.291+0000 7fd47ed9d700 1 -- 172.21.15.62:0/4134274774 learned_addr learned my addr 172.21.15.62:0/4134274774 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.292+0000 7fd47dd9b700 1 --2- 172.21.15.62:0/4134274774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 0x563eced6c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.292+0000 7fd47e59c700 1 --2- 172.21.15.62:0/4134274774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 0x563eced72750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.292+0000 7fd47ed9d700 1 -- 172.21.15.62:0/4134274774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 msgr2=0x563eced6c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.292+0000 7fd47ed9d700 1 --2- 172.21.15.62:0/4134274774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 0x563eced6c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.292+0000 7fd47ed9d700 1 -- 172.21.15.62:0/4134274774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 msgr2=0x563eced72750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.292+0000 7fd47ed9d700 1 --2- 172.21.15.62:0/4134274774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 0x563eced72750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.292+0000 7fd47ed9d700 1 -- 172.21.15.62:0/4134274774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ecec088d0 con 0x563eced6cbb0 2024-06-10T00:39:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.292+0000 7fd47dd9b700 1 --2- 172.21.15.62:0/4134274774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 0x563eced6c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.292+0000 7fd47e59c700 1 --2- 172.21.15.62:0/4134274774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 0x563eced72750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.292+0000 7fd47ed9d700 1 --2- 172.21.15.62:0/4134274774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced6cbb0 0x563eced6cff0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd47800a430 tx=0x7fd478009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e02c4e3778fb179 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.292+0000 7fd474ff9700 1 -- 172.21.15.62:0/4134274774 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478013590 con 0x563eced6cbb0 2024-06-10T00:39:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.292+0000 7fd474ff9700 1 -- 172.21.15.62:0/4134274774 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd478019070 con 0x563eced6cbb0 2024-06-10T00:39:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.292+0000 7fd474ff9700 1 -- 172.21.15.62:0/4134274774 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478014bd0 con 0x563eced6cbb0 2024-06-10T00:39:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.293+0000 7fd48f678b80 1 -- 172.21.15.62:0/4134274774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced6cbb0 msgr2=0x563eced6cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.293+0000 7fd48f678b80 1 --2- 172.21.15.62:0/4134274774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced6cbb0 0x563eced6cff0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd47800a430 tx=0x7fd478009650 comp rx=0 tx=0).stop 2024-06-10T00:39:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.293+0000 7fd48f678b80 1 -- 172.21.15.62:0/4134274774 shutdown_connections 2024-06-10T00:39:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.293+0000 7fd48f678b80 1 --2- 172.21.15.62:0/4134274774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced6cbb0 0x563eced6cff0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.293+0000 7fd48f678b80 1 --2- 172.21.15.62:0/4134274774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 0x563eced72750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.293+0000 7fd48f678b80 1 --2- 172.21.15.62:0/4134274774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 0x563eced6c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.293+0000 7fd48f678b80 1 -- 172.21.15.62:0/4134274774 >> 172.21.15.62:0/4134274774 conn(0x563ecec95080 msgr2=0x563eced63220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.293+0000 7fd48f678b80 1 -- 172.21.15.62:0/4134274774 shutdown_connections 2024-06-10T00:39:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.293+0000 7fd48f678b80 1 -- 172.21.15.62:0/4134274774 wait complete. 2024-06-10T00:39:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.293+0000 7fd48f678b80 1 -- start start 2024-06-10T00:39:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.293+0000 7fd48f678b80 1 Processor -- start 2024-06-10T00:39:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd48f678b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced6cbb0 0x563eced78980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd48f678b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 0x563eced75cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd48f678b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 0x563eced76210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd48f678b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563ecec59fc0 con 0x563eced6cbb0 2024-06-10T00:39:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd48f678b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563ececb5810 con 0x563eced72370 2024-06-10T00:39:51.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd48f678b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563eced79680 con 0x563eced72c90 2024-06-10T00:39:51.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd48f678b80 10 client.0 _ll_register_callbacks cb 0x563ecec4f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:51.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd47e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced6cbb0 0x563eced78980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd47e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced6cbb0 0x563eced78980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53642/0 (socket says 172.21.15.62:53642) 2024-06-10T00:39:51.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd47e59c700 1 -- 172.21.15.62:0/252192423 learned_addr learned my addr 172.21.15.62:0/252192423 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:51.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd47dd9b700 1 --2- 172.21.15.62:0/252192423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 0x563eced75cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd47ed9d700 1 --2- 172.21.15.62:0/252192423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 0x563eced76210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd47e59c700 1 -- 172.21.15.62:0/252192423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 msgr2=0x563eced76210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:51.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd47e59c700 1 --2- 172.21.15.62:0/252192423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 0x563eced76210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd47e59c700 1 -- 172.21.15.62:0/252192423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 msgr2=0x563eced75cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:51.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd47e59c700 1 --2- 172.21.15.62:0/252192423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 0x563eced75cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd47e59c700 1 -- 172.21.15.62:0/252192423 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ecec7f750 con 0x563eced6cbb0 2024-06-10T00:39:51.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.294+0000 7fd47ed9d700 1 --2- 172.21.15.62:0/252192423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 0x563eced76210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:39:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.295+0000 7fd47e59c700 1 --2- 172.21.15.62:0/252192423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced6cbb0 0x563eced78980 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd47000b500 tx=0x7fd47009a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2ebb9de585cbdc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.295+0000 7fd465ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.295+0000 7fd465ffb700 1 -- 172.21.15.62:0/252192423 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4700b3020 con 0x563eced6cbb0 2024-06-10T00:39:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.295+0000 7fd465ffb700 1 -- 172.21.15.62:0/252192423 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fd4700105a0 con 0x563eced6cbb0 2024-06-10T00:39:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.295+0000 7fd48f678b80 1 -- 172.21.15.62:0/252192423 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd478019070 con 0x563eced6cbb0 2024-06-10T00:39:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.295+0000 7fd465ffb700 1 -- 172.21.15.62:0/252192423 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd47009f370 con 0x563eced6cbb0 2024-06-10T00:39:51.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.295+0000 7fd48f678b80 10 client.16608 fetch_fsmap learned FSMap version 37 2024-06-10T00:39:51.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.295+0000 7fd48f678b80 1 -- 172.21.15.62:0/252192423 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563eced83e00 con 0x563eced6cbb0 2024-06-10T00:39:51.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.295+0000 7fd465ffb700 1 -- 172.21.15.62:0/252192423 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd478019070 con 0x563eced6cbb0 2024-06-10T00:39:51.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.296+0000 7fd465ffb700 1 -- 172.21.15.62:0/252192423 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fd47009d050 con 0x563eced6cbb0 2024-06-10T00:39:51.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.296+0000 7fd48f678b80 10 client.16608 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T00:39:51.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.296+0000 7fd48f678b80 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:39:51.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.296+0000 7fd48f678b80 20 client.16608 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.296+0000 7fd48f678b80 1 --2- 172.21.15.62:0/252192423 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563eced86440 0x563eced88880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.296+0000 7fd48f678b80 4 client.16608 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:39:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.296+0000 7fd48f678b80 1 -- 172.21.15.62:0/252192423 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ececc6d50 con 0x563eced86440 2024-06-10T00:39:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.296+0000 7fd47dd9b700 1 --2- 172.21.15.62:0/252192423 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563eced86440 0x563eced88880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.296+0000 7fd47dd9b700 1 --2- 172.21.15.62:0/252192423 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563eced86440 0x563eced88880 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd6c470ba9ab838e server_cookie=31c1eb731d8a9e94 in_seq=0 out_seq=0 2024-06-10T00:39:51.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.296+0000 7fd465ffb700 10 client.16608 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:51.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.297+0000 7fd465ffb700 1 -- 172.21.15.62:0/252192423 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563ececc6d50 con 0x563eced86440 2024-06-10T00:39:51.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.297+0000 7fd465ffb700 10 client.16608 handle_command_reply: tid=0 2024-06-10T00:39:51.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.297+0000 7fd48f678b80 1 client.16608 shutdown 2024-06-10T00:39:51.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.297+0000 7fd48f678b80 1 -- 172.21.15.62:0/252192423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced6cbb0 msgr2=0x563eced78980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:51.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.297+0000 7fd48f678b80 1 --2- 172.21.15.62:0/252192423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced6cbb0 0x563eced78980 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd47000b500 tx=0x7fd47009a9a0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.298+0000 7fd48f678b80 1 -- 172.21.15.62:0/252192423 shutdown_connections 2024-06-10T00:39:51.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.298+0000 7fd48f678b80 1 --2- 172.21.15.62:0/252192423 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563eced86440 0x563eced88880 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.298+0000 7fd48f678b80 1 --2- 172.21.15.62:0/252192423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced6cbb0 0x563eced78980 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.298+0000 7fd48f678b80 1 --2- 172.21.15.62:0/252192423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 0x563eced76210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.298+0000 7fd465ffb700 0 client.16608 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:51.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.298+0000 7fd48f678b80 1 --2- 172.21.15.62:0/252192423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 0x563eced75cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.298+0000 7fd48f678b80 1 -- 172.21.15.62:0/252192423 >> 172.21.15.62:0/252192423 conn(0x563ecec95080 msgr2=0x563eced62700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:51.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.298+0000 7fd48f678b80 1 -- 172.21.15.62:0/252192423 shutdown_connections 2024-06-10T00:39:51.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.298+0000 7fd48f678b80 1 -- 172.21.15.62:0/252192423 wait complete. 2024-06-10T00:39:51.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.298+0000 7fd48f678b80 20 client.16608 trim_cache size 0 max 16384 2024-06-10T00:39:51.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.318+0000 7fd48f678b80 1 Processor -- start 2024-06-10T00:39:51.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.318+0000 7fd48f678b80 1 -- start start 2024-06-10T00:39:51.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd48f678b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced6cbb0 0x563eced74e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd48f678b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 0x563eced62e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd48f678b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced72c90 0x563eced633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd48f678b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801ac590 con 0x563eced72c90 2024-06-10T00:39:51.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd48f678b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801ae100 con 0x563eced72370 2024-06-10T00:39:51.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd48f678b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801acc60 con 0x563eced6cbb0 2024-06-10T00:39:51.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd47ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced72c90 0x563eced633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd47ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced72c90 0x563eced633c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53644/0 (socket says 172.21.15.62:53644) 2024-06-10T00:39:51.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd47ed9d700 1 -- 172.21.15.62:0/2701608514 learned_addr learned my addr 172.21.15.62:0/2701608514 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:51.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd47e59c700 1 --2- 172.21.15.62:0/2701608514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced6cbb0 0x563eced74e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd47dd9b700 1 --2- 172.21.15.62:0/2701608514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 0x563eced62e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd47ed9d700 1 -- 172.21.15.62:0/2701608514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced6cbb0 msgr2=0x563eced74e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:51.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd47ed9d700 1 --2- 172.21.15.62:0/2701608514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced6cbb0 0x563eced74e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd47ed9d700 1 -- 172.21.15.62:0/2701608514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 msgr2=0x563eced62e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:51.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd47ed9d700 1 --2- 172.21.15.62:0/2701608514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 0x563eced62e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd47ed9d700 1 -- 172.21.15.62:0/2701608514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eced64800 con 0x563eced72c90 2024-06-10T00:39:51.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd47ed9d700 1 --2- 172.21.15.62:0/2701608514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced72c90 0x563eced633c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd478000f80 tx=0x7fd478009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce58c060101c1fe6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:51.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd467fff700 1 -- 172.21.15.62:0/2701608514 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478013790 con 0x563eced72c90 2024-06-10T00:39:51.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd467fff700 1 -- 172.21.15.62:0/2701608514 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd478019070 con 0x563eced72c90 2024-06-10T00:39:51.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.319+0000 7fd467fff700 1 -- 172.21.15.62:0/2701608514 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478014420 con 0x563eced72c90 2024-06-10T00:39:51.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.320+0000 7fd48f678b80 1 -- 172.21.15.62:0/2701608514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced72c90 msgr2=0x563eced633c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:51.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.320+0000 7fd48f678b80 1 --2- 172.21.15.62:0/2701608514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced72c90 0x563eced633c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd478000f80 tx=0x7fd478009390 comp rx=0 tx=0).stop 2024-06-10T00:39:51.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.320+0000 7fd48f678b80 1 -- 172.21.15.62:0/2701608514 shutdown_connections 2024-06-10T00:39:51.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.320+0000 7fd48f678b80 1 --2- 172.21.15.62:0/2701608514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced72c90 0x563eced633c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.320+0000 7fd48f678b80 1 --2- 172.21.15.62:0/2701608514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced6cbb0 0x563eced74e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.320+0000 7fd48f678b80 1 --2- 172.21.15.62:0/2701608514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced72370 0x563eced62e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.320+0000 7fd48f678b80 1 -- 172.21.15.62:0/2701608514 >> 172.21.15.62:0/2701608514 conn(0x563ecec95080 msgr2=0x563eced736e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:51.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.320+0000 7fd48f678b80 1 -- 172.21.15.62:0/2701608514 shutdown_connections 2024-06-10T00:39:51.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.320+0000 7fd48f678b80 1 -- 172.21.15.62:0/2701608514 wait complete. 2024-06-10T00:39:51.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.320+0000 7fd48f678b80 1 -- start start 2024-06-10T00:39:51.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.320+0000 7fd48f678b80 1 Processor -- start 2024-06-10T00:39:51.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd48f678b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced6cbb0 0x563eced7ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd48f678b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced72370 0x563eced8d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd48f678b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 0x563eced8df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd48f678b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801ada30 con 0x563eced72370 2024-06-10T00:39:51.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd48f678b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801ad330 con 0x563eced6cbb0 2024-06-10T00:39:51.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd48f678b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801aa530 con 0x563eced72c90 2024-06-10T00:39:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd48f678b80 10 client.0 _ll_register_callbacks cb 0x563ecec4f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:39:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd47dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced72370 0x563eced8d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd47dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced72370 0x563eced8d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53646/0 (socket says 172.21.15.62:53646) 2024-06-10T00:39:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd47dd9b700 1 -- 172.21.15.62:0/1381677340 learned_addr learned my addr 172.21.15.62:0/1381677340 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:39:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd47ed9d700 1 --2- 172.21.15.62:0/1381677340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 0x563eced8df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd47e59c700 1 --2- 172.21.15.62:0/1381677340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced6cbb0 0x563eced7ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd47dd9b700 1 -- 172.21.15.62:0/1381677340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 msgr2=0x563eced8df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:51.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd47dd9b700 1 --2- 172.21.15.62:0/1381677340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 0x563eced8df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd47dd9b700 1 -- 172.21.15.62:0/1381677340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced6cbb0 msgr2=0x563eced7ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:39:51.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd47dd9b700 1 --2- 172.21.15.62:0/1381677340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced6cbb0 0x563eced7ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd47dd9b700 1 -- 172.21.15.62:0/1381677340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eced8e5e0 con 0x563eced72370 2024-06-10T00:39:51.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd47e59c700 1 --2- 172.21.15.62:0/1381677340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced6cbb0 0x563eced7ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:39:51.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd47dd9b700 1 --2- 172.21.15.62:0/1381677340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced72370 0x563eced8d9d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd4680099c0 tx=0x7fd468009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b77fc9cb711276 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:39:51.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.321+0000 7fd465ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:39:51.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.322+0000 7fd465ffb700 1 -- 172.21.15.62:0/1381677340 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46800fcd0 con 0x563eced72370 2024-06-10T00:39:51.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.322+0000 7fd465ffb700 1 -- 172.21.15.62:0/1381677340 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9466+0+0 (secure 0 0 0) 0x7fd468024410 con 0x563eced72370 2024-06-10T00:39:51.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.322+0000 7fd48f678b80 1 -- 172.21.15.62:0/1381677340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd478019070 con 0x563eced72370 2024-06-10T00:39:51.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.322+0000 7fd465ffb700 1 -- 172.21.15.62:0/1381677340 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd468024610 con 0x563eced72370 2024-06-10T00:39:51.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.322+0000 7fd48f678b80 10 client.16614 fetch_fsmap learned FSMap version 37 2024-06-10T00:39:51.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.322+0000 7fd48f678b80 1 -- 172.21.15.62:0/1381677340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563ecedf1100 con 0x563eced72370 2024-06-10T00:39:51.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.322+0000 7fd465ffb700 1 -- 172.21.15.62:0/1381677340 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd478019070 con 0x563eced72370 2024-06-10T00:39:51.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.322+0000 7fd465ffb700 1 -- 172.21.15.62:0/1381677340 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fd46803a070 con 0x563eced72370 2024-06-10T00:39:51.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.322+0000 7fd48f678b80 10 client.16614 fetch_fsmap finished waiting for FSMap version 37 2024-06-10T00:39:51.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.322+0000 7fd48f678b80 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:39:51.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.322+0000 7fd48f678b80 20 client.16614 populate_metadata read hostname 'smithi062' 2024-06-10T00:39:51.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.322+0000 7fd48f678b80 1 --2- 172.21.15.62:0/1381677340 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563ecedf1820 0x563ecedf3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:39:51.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.322+0000 7fd48f678b80 4 client.16614 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:39:51.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.322+0000 7fd48f678b80 1 -- 172.21.15.62:0/1381677340 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ececc12d0 con 0x563ecedf1820 2024-06-10T00:39:51.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd47e59c700 1 --2- 172.21.15.62:0/1381677340 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563ecedf1820 0x563ecedf3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:39:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd47e59c700 1 --2- 172.21.15.62:0/1381677340 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563ecedf1820 0x563ecedf3c40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1653b4b749e3e03 server_cookie=ad175c557d032ff4 in_seq=0 out_seq=0 2024-06-10T00:39:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd465ffb700 10 client.16614 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd465ffb700 1 -- 172.21.15.62:0/1381677340 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563ececc12d0 con 0x563ecedf1820 2024-06-10T00:39:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd465ffb700 10 client.16614 handle_command_reply: tid=0 2024-06-10T00:39:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd48f678b80 1 client.16614 shutdown 2024-06-10T00:39:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd48f678b80 1 -- 172.21.15.62:0/1381677340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced72370 msgr2=0x563eced8d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd48f678b80 1 --2- 172.21.15.62:0/1381677340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced72370 0x563eced8d9d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd4680099c0 tx=0x7fd468009cd0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd48f678b80 1 -- 172.21.15.62:0/1381677340 shutdown_connections 2024-06-10T00:39:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd48f678b80 1 --2- 172.21.15.62:0/1381677340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eced72c90 0x563eced8df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd48f678b80 1 --2- 172.21.15.62:0/1381677340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563eced6cbb0 0x563eced7ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd48f678b80 1 --2- 172.21.15.62:0/1381677340 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x563ecedf1820 0x563ecedf3c40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd48f678b80 1 --2- 172.21.15.62:0/1381677340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563eced72370 0x563eced8d9d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd465ffb700 0 client.16614 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:39:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd48f678b80 1 -- 172.21.15.62:0/1381677340 >> 172.21.15.62:0/1381677340 conn(0x563ecec95080 msgr2=0x563eced67250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:51.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd48f678b80 1 -- 172.21.15.62:0/1381677340 shutdown_connections 2024-06-10T00:39:51.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.323+0000 7fd48f678b80 1 -- 172.21.15.62:0/1381677340 wait complete. 2024-06-10T00:39:51.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.324+0000 7fd48f678b80 20 client.16614 trim_cache size 0 max 16384 2024-06-10T00:39:51.335 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:39:51.336 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T00:39:51.336 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-06-10T00:39:51.336 INFO:teuthology.orchestra.run.smithi062.stdout: "4d7089aa-3d77-4da5-9faf-b97825150439": { 2024-06-10T00:39:51.336 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-06-10T00:39:51.336 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "4d7089aa-3d77-4da5-9faf-b97825150439", 2024-06-10T00:39:51.337 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-06-10T00:39:51.337 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:39:51.337 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:39:51.337 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:39:51.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.324+0000 7fd484df5700 1 -- 172.21.15.62:0/3259348606 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd46c0a2e10 msgr2=0x7fd46c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:51.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.324+0000 7fd484df5700 1 --2- 172.21.15.62:0/3259348606 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd46c0a2e10 0x7fd46c0a52d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd468000c00 tx=0x7fd468005fb0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.324+0000 7fd484df5700 1 -- 172.21.15.62:0/3259348606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd480101610 msgr2=0x7fd480198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:39:51.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.324+0000 7fd484df5700 1 --2- 172.21.15.62:0/3259348606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd480101610 0x7fd480198e20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd47000b530 tx=0x7fd47000b840 comp rx=0 tx=0).stop 2024-06-10T00:39:51.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.324+0000 7fd484df5700 1 -- 172.21.15.62:0/3259348606 shutdown_connections 2024-06-10T00:39:51.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.324+0000 7fd484df5700 1 --2- 172.21.15.62:0/3259348606 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd46c0a2e10 0x7fd46c0a52d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.324+0000 7fd484df5700 1 --2- 172.21.15.62:0/3259348606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd480101610 0x7fd480198e20 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.324+0000 7fd484df5700 1 --2- 172.21.15.62:0/3259348606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4801066f0 0x7fd48019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.324+0000 7fd484df5700 1 --2- 172.21.15.62:0/3259348606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd480101fc0 0x7fd480199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:39:51.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.324+0000 7fd484df5700 1 -- 172.21.15.62:0/3259348606 >> 172.21.15.62:0/3259348606 conn(0x7fd480078430 msgr2=0x7fd48006af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:39:51.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.324+0000 7fd484df5700 1 -- 172.21.15.62:0/3259348606 shutdown_connections 2024-06-10T00:39:51.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:39:51.324+0000 7fd484df5700 1 -- 172.21.15.62:0/3259348606 wait complete. 2024-06-10T00:39:51.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:51 smithi062 ceph-mon[32390]: pgmap v213: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:39:51.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:51 smithi105 ceph-mon[36321]: pgmap v213: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:39:51.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:51 smithi138 ceph-mon[36549]: pgmap v213: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:39:52.084 INFO:tasks.cephfs.filesystem:scrub status for tag:4d7089aa-3d77-4da5-9faf-b97825150439 - {'path': '/', 'tag': '4d7089aa-3d77-4da5-9faf-b97825150439', 'options': 'recursive,force'} 2024-06-10T00:39:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:53 smithi105 ceph-mon[36321]: pgmap v214: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-06-10T00:39:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:53 smithi062 ceph-mon[32390]: pgmap v214: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-06-10T00:39:53.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:53 smithi138 ceph-mon[36549]: pgmap v214: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-06-10T00:39:55.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:55 smithi105 ceph-mon[36321]: pgmap v215: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:39:55.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:55 smithi062 ceph-mon[32390]: pgmap v215: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:39:55.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:55 smithi138 ceph-mon[36549]: pgmap v215: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:39:57.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:57 smithi105 ceph-mon[36321]: pgmap v216: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-10T00:39:57.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:57 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:39:57.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:57 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:39:57.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:57 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:39:57.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:57 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:39:57.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:57 smithi062 ceph-mon[32390]: pgmap v216: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-10T00:39:57.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:57 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:39:57.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:57 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:39:57.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:57 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:39:57.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:57 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:39:57.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:57 smithi138 ceph-mon[36549]: pgmap v216: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-10T00:39:57.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:57 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:39:57.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:57 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:39:57.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:57 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:39:57.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:57 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:39:58.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:58 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:39:58.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:58 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:39:58.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:58 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:39:59.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:39:59 smithi105 ceph-mon[36321]: pgmap v217: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-06-10T00:39:59.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:39:59 smithi062 ceph-mon[32390]: pgmap v217: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-06-10T00:39:59.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:39:59 smithi138 ceph-mon[36549]: pgmap v217: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-06-10T00:40:00.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:00 smithi105 ceph-mon[36321]: overall HEALTH_OK 2024-06-10T00:40:00.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:00 smithi062 ceph-mon[32390]: overall HEALTH_OK 2024-06-10T00:40:00.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:00 smithi138 ceph-mon[36549]: overall HEALTH_OK 2024-06-10T00:40:01.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:01 smithi105 ceph-mon[36321]: pgmap v218: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:40:01.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:01 smithi062 ceph-mon[32390]: pgmap v218: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:40:01.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:01 smithi138 ceph-mon[36549]: pgmap v218: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:40:03.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:03 smithi105 ceph-mon[36321]: pgmap v219: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:40:03.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:03 smithi062 ceph-mon[32390]: pgmap v219: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:40:03.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:03 smithi138 ceph-mon[36549]: pgmap v219: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:40:05.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:05 smithi105 ceph-mon[36321]: pgmap v220: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-06-10T00:40:05.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:05 smithi062 ceph-mon[32390]: pgmap v220: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-06-10T00:40:05.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:05 smithi138 ceph-mon[36549]: pgmap v220: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-06-10T00:40:07.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:07 smithi105 ceph-mon[36321]: pgmap v221: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-06-10T00:40:07.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:07 smithi062 ceph-mon[32390]: pgmap v221: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-06-10T00:40:07.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:07 smithi138 ceph-mon[36549]: pgmap v221: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-06-10T00:40:08.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:08 smithi105 ceph-mon[36321]: osdmap e101: 12 total, 12 up, 12 in 2024-06-10T00:40:08.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:08 smithi062 ceph-mon[32390]: osdmap e101: 12 total, 12 up, 12 in 2024-06-10T00:40:08.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:08 smithi138 ceph-mon[36549]: osdmap e101: 12 total, 12 up, 12 in 2024-06-10T00:40:09.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:09 smithi105 ceph-mon[36321]: pgmap v223: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-06-10T00:40:09.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:09 smithi062 ceph-mon[32390]: pgmap v223: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-06-10T00:40:09.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:09 smithi138 ceph-mon[36549]: pgmap v223: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-06-10T00:40:11.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:11 smithi105 ceph-mon[36321]: pgmap v224: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-06-10T00:40:11.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:11 smithi062 ceph-mon[32390]: pgmap v224: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-06-10T00:40:11.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:11 smithi138 ceph-mon[36549]: pgmap v224: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-06-10T00:40:12.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:12 smithi062 ceph-mon[32390]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:40:12.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:12 smithi138 ceph-mon[36549]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:40:13.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:12 smithi105 ceph-mon[36321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-10T00:40:13.831 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:13 smithi062 ceph-mon[32390]: pgmap v225: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-06-10T00:40:13.831 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:13 smithi062 ceph-mon[32390]: osdmap e102: 12 total, 12 up, 12 in 2024-06-10T00:40:13.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:13 smithi138 ceph-mon[36549]: pgmap v225: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-06-10T00:40:13.896 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:13 smithi138 ceph-mon[36549]: osdmap e102: 12 total, 12 up, 12 in 2024-06-10T00:40:14.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:13 smithi105 ceph-mon[36321]: pgmap v225: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-06-10T00:40:14.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:13 smithi105 ceph-mon[36321]: osdmap e102: 12 total, 12 up, 12 in 2024-06-10T00:40:15.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:15 smithi062 ceph-mon[32390]: pgmap v227: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 9 op/s 2024-06-10T00:40:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:15 smithi138 ceph-mon[36549]: pgmap v227: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 9 op/s 2024-06-10T00:40:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:15 smithi105 ceph-mon[36321]: pgmap v227: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 9 op/s 2024-06-10T00:40:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:17 smithi062 ceph-mon[32390]: pgmap v228: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 239 B/s wr, 9 op/s 2024-06-10T00:40:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:17 smithi138 ceph-mon[36549]: pgmap v228: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 239 B/s wr, 9 op/s 2024-06-10T00:40:18.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:17 smithi105 ceph-mon[36321]: pgmap v228: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 239 B/s wr, 9 op/s 2024-06-10T00:40:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:19 smithi062 ceph-mon[32390]: pgmap v229: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:40:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:19 smithi138 ceph-mon[36549]: pgmap v229: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:40:20.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:19 smithi105 ceph-mon[36321]: pgmap v229: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:40:21.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:21 smithi062 ceph-mon[32390]: pgmap v230: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:40:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:21 smithi138 ceph-mon[36549]: pgmap v230: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:40:22.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:21 smithi105 ceph-mon[36321]: pgmap v230: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:40:22.086 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:40:22.368 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:40:22.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:22 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:40:22.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:22 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:40:23.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:22 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:40:23.818 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:23 smithi062 ceph-mon[32390]: pgmap v231: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:40:23.818 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:23 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:40:23.818 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:23 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:40:23.819 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:23 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:40:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:23 smithi138 ceph-mon[36549]: pgmap v231: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:40:23.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:23 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:40:23.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:23 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:40:23.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:23 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:40:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:23 smithi105 ceph-mon[36321]: pgmap v231: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:40:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:23 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:40:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:23 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:40:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:23 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:40:24.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.217+0000 7f6edf82a700 1 -- 172.21.15.62:0/1375441368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed81019d0 msgr2=0x7f6ed8101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.217+0000 7f6edf82a700 1 --2- 172.21.15.62:0/1375441368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed81019d0 0x7f6ed8101db0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800b0d0 tx=0x7f6ec800b3e0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.217+0000 7f6edf82a700 1 -- 172.21.15.62:0/1375441368 shutdown_connections 2024-06-10T00:40:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.217+0000 7f6edf82a700 1 --2- 172.21.15.62:0/1375441368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed81019d0 0x7f6ed8101db0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.217+0000 7f6edf82a700 1 --2- 172.21.15.62:0/1375441368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ed8106720 0x7f6ed810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.217+0000 7f6edf82a700 1 --2- 172.21.15.62:0/1375441368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ed8102380 0x7f6ed8105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.217+0000 7f6edf82a700 1 -- 172.21.15.62:0/1375441368 >> 172.21.15.62:0/1375441368 conn(0x7f6ed8078340 msgr2=0x7f6ed8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.217+0000 7f6edf82a700 1 -- 172.21.15.62:0/1375441368 shutdown_connections 2024-06-10T00:40:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.217+0000 7f6edf82a700 1 -- 172.21.15.62:0/1375441368 wait complete. 2024-06-10T00:40:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.218+0000 7f6edf82a700 1 Processor -- start 2024-06-10T00:40:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.218+0000 7f6edf82a700 1 -- start start 2024-06-10T00:40:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.218+0000 7f6edf82a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ed81019d0 0x7f6ed819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.218+0000 7f6edf82a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed8102380 0x7f6ed819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.218+0000 7f6edf82a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ed8106720 0x7f6ed8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.218+0000 7f6edf82a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8111700 con 0x7f6ed8102380 2024-06-10T00:40:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.218+0000 7f6edf82a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8111580 con 0x7f6ed8106720 2024-06-10T00:40:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.218+0000 7f6edf82a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8111880 con 0x7f6ed81019d0 2024-06-10T00:40:24.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.218+0000 7f6edcdc5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed8102380 0x7f6ed819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.218+0000 7f6edcdc5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed8102380 0x7f6ed819edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43974/0 (socket says 172.21.15.62:43974) 2024-06-10T00:40:24.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.218+0000 7f6edcdc5700 1 -- 172.21.15.62:0/4276359788 learned_addr learned my addr 172.21.15.62:0/4276359788 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:24.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.218+0000 7f6edd5c6700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ed81019d0 0x7f6ed819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.219+0000 7f6edcdc5700 1 -- 172.21.15.62:0/4276359788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ed81019d0 msgr2=0x7f6ed819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:24.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.219+0000 7f6edcdc5700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ed81019d0 0x7f6ed819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.219+0000 7f6edddc7700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ed8106720 0x7f6ed8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.219+0000 7f6edcdc5700 1 -- 172.21.15.62:0/4276359788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ed8106720 msgr2=0x7f6ed8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:24.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.219+0000 7f6edcdc5700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ed8106720 0x7f6ed8198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.219+0000 7f6edcdc5700 1 -- 172.21.15.62:0/4276359788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed8199190 con 0x7f6ed8102380 2024-06-10T00:40:24.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.219+0000 7f6edd5c6700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ed81019d0 0x7f6ed819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:40:24.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.219+0000 7f6edcdc5700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed8102380 0x7f6ed819edb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00c9b0 tx=0x7f6ecc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:24.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.219+0000 7f6ed67fc700 1 -- 172.21.15.62:0/4276359788 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc007aa0 con 0x7f6ed8102380 2024-06-10T00:40:24.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.219+0000 7f6ed67fc700 1 -- 172.21.15.62:0/4276359788 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ecc007c40 con 0x7f6ed8102380 2024-06-10T00:40:24.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.219+0000 7f6ed67fc700 1 -- 172.21.15.62:0/4276359788 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc018730 con 0x7f6ed8102380 2024-06-10T00:40:24.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.219+0000 7f6edf82a700 1 -- 172.21.15.62:0/4276359788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed8199480 con 0x7f6ed8102380 2024-06-10T00:40:24.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.219+0000 7f6edf82a700 1 -- 172.21.15.62:0/4276359788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed81a5740 con 0x7f6ed8102380 2024-06-10T00:40:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.221+0000 7f6ed67fc700 1 -- 172.21.15.62:0/4276359788 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6ecc0188d0 con 0x7f6ed8102380 2024-06-10T00:40:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.222+0000 7f6ed67fc700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6ec00a2d90 0x7f6ec00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.222+0000 7f6edd5c6700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6ec00a2d90 0x7f6ec00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.222+0000 7f6ed67fc700 1 -- 172.21.15.62:0/4276359788 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f6ecc0cf860 con 0x7f6ed8102380 2024-06-10T00:40:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.223+0000 7f6edd5c6700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6ec00a2d90 0x7f6ec00a5250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009850 tx=0x7f6ec80097a0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.223+0000 7f6eea0adb80 1 Processor -- start 2024-06-10T00:40:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.223+0000 7f6eea0adb80 1 -- start start 2024-06-10T00:40:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.223+0000 7f6eea0adb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3261b470 0x555f3261b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.223+0000 7f6eea0adb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32615670 0x555f32615af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.223+0000 7f6eea0adb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f32616030 0x555f32622210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.223+0000 7f6eea0adb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555f32503f60 con 0x555f3261b470 2024-06-10T00:40:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.223+0000 7f6eea0adb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555f32562850 con 0x555f32615670 2024-06-10T00:40:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.223+0000 7f6eea0adb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555f324ca460 con 0x555f32616030 2024-06-10T00:40:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.223+0000 7f6edd5c6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3261b470 0x555f3261b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.224+0000 7f6edd5c6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3261b470 0x555f3261b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43980/0 (socket says 172.21.15.62:43980) 2024-06-10T00:40:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.224+0000 7f6edd5c6700 1 -- 172.21.15.62:0/2902881530 learned_addr learned my addr 172.21.15.62:0/2902881530 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.224+0000 7f6edcdc5700 1 --2- 172.21.15.62:0/2902881530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32615670 0x555f32615af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.224+0000 7f6edddc7700 1 --2- 172.21.15.62:0/2902881530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f32616030 0x555f32622210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.224+0000 7f6edd5c6700 1 -- 172.21.15.62:0/2902881530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f32616030 msgr2=0x555f32622210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.224+0000 7f6edd5c6700 1 --2- 172.21.15.62:0/2902881530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f32616030 0x555f32622210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.224+0000 7f6edd5c6700 1 -- 172.21.15.62:0/2902881530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32615670 msgr2=0x555f32615af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.224+0000 7f6edd5c6700 1 --2- 172.21.15.62:0/2902881530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32615670 0x555f32615af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.224+0000 7f6edd5c6700 1 -- 172.21.15.62:0/2902881530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f324b18d0 con 0x555f3261b470 2024-06-10T00:40:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.224+0000 7f6edcdc5700 1 --2- 172.21.15.62:0/2902881530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32615670 0x555f32615af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:40:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.224+0000 7f6edd5c6700 1 --2- 172.21.15.62:0/2902881530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3261b470 0x555f3261b850 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8015000 tx=0x7f6ec801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e821b5c659c69d7a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.224+0000 7f6ebf7fe700 1 -- 172.21.15.62:0/2902881530 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8007b70 con 0x555f3261b470 2024-06-10T00:40:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.224+0000 7f6ebf7fe700 1 -- 172.21.15.62:0/2902881530 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ec802e070 con 0x555f3261b470 2024-06-10T00:40:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.224+0000 7f6ebf7fe700 1 -- 172.21.15.62:0/2902881530 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80052a0 con 0x555f3261b470 2024-06-10T00:40:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.225+0000 7f6eea0adb80 1 -- 172.21.15.62:0/2902881530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3261b470 msgr2=0x555f3261b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.225+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/2902881530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3261b470 0x555f3261b850 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8015000 tx=0x7f6ec801c000 comp rx=0 tx=0).stop 2024-06-10T00:40:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.225+0000 7f6eea0adb80 1 -- 172.21.15.62:0/2902881530 shutdown_connections 2024-06-10T00:40:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.225+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/2902881530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3261b470 0x555f3261b850 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.225+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/2902881530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f32616030 0x555f32622210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.225+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/2902881530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32615670 0x555f32615af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.225+0000 7f6eea0adb80 1 -- 172.21.15.62:0/2902881530 >> 172.21.15.62:0/2902881530 conn(0x555f3253e080 msgr2=0x555f3260c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:24.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.225+0000 7f6eea0adb80 1 -- 172.21.15.62:0/2902881530 shutdown_connections 2024-06-10T00:40:24.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.225+0000 7f6eea0adb80 1 -- 172.21.15.62:0/2902881530 wait complete. 2024-06-10T00:40:24.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.225+0000 7f6eea0adb80 1 -- start start 2024-06-10T00:40:24.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.226+0000 7f6eea0adb80 1 Processor -- start 2024-06-10T00:40:24.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.226+0000 7f6eea0adb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f32615670 0x555f32621e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.226+0000 7f6eea0adb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 0x555f3261f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.226+0000 7f6eea0adb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f3261b470 0x555f3261f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.226+0000 7f6eea0adb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555f32502fc0 con 0x555f32615670 2024-06-10T00:40:24.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.226+0000 7f6eea0adb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555f3255e810 con 0x555f32616030 2024-06-10T00:40:24.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.226+0000 7f6eea0adb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555f32622750 con 0x555f3261b470 2024-06-10T00:40:24.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.226+0000 7f6eea0adb80 10 client.0 _ll_register_callbacks cb 0x555f324f85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:24.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.226+0000 7f6edd5c6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f32615670 0x555f32621e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.226+0000 7f6edd5c6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f32615670 0x555f32621e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43990/0 (socket says 172.21.15.62:43990) 2024-06-10T00:40:24.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.226+0000 7f6edd5c6700 1 -- 172.21.15.62:0/1604685095 learned_addr learned my addr 172.21.15.62:0/1604685095 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:24.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.226+0000 7f6edddc7700 1 --2- 172.21.15.62:0/1604685095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f3261b470 0x555f3261f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.226+0000 7f6edcdc5700 1 --2- 172.21.15.62:0/1604685095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 0x555f3261f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.227+0000 7f6edd5c6700 1 -- 172.21.15.62:0/1604685095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f3261b470 msgr2=0x555f3261f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:24.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.227+0000 7f6edd5c6700 1 --2- 172.21.15.62:0/1604685095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f3261b470 0x555f3261f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.227+0000 7f6edd5c6700 1 -- 172.21.15.62:0/1604685095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 msgr2=0x555f3261f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:24.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.227+0000 7f6edd5c6700 1 --2- 172.21.15.62:0/1604685095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 0x555f3261f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.227+0000 7f6edd5c6700 1 -- 172.21.15.62:0/1604685095 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f32528750 con 0x555f32615670 2024-06-10T00:40:24.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.227+0000 7f6edd5c6700 1 --2- 172.21.15.62:0/1604685095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f32615670 0x555f32621e50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8000f80 tx=0x7f6ec800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f1be5fe04ce46f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.227+0000 7f6ebcff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.227+0000 7f6ebcff9700 1 -- 172.21.15.62:0/1604685095 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8036070 con 0x555f32615670 2024-06-10T00:40:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.227+0000 7f6eea0adb80 1 -- 172.21.15.62:0/1604685095 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec802e070 con 0x555f32615670 2024-06-10T00:40:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.227+0000 7f6ebcff9700 1 -- 172.21.15.62:0/1604685095 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f6ec801c850 con 0x555f32615670 2024-06-10T00:40:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.227+0000 7f6ebcff9700 1 -- 172.21.15.62:0/1604685095 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6ec8043050 con 0x555f32615670 2024-06-10T00:40:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.227+0000 7f6eea0adb80 10 client.16626 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.227+0000 7f6eea0adb80 1 -- 172.21.15.62:0/1604685095 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555f3262cd50 con 0x555f32615670 2024-06-10T00:40:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.227+0000 7f6ebcff9700 1 -- 172.21.15.62:0/1604685095 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ec802e070 con 0x555f32615670 2024-06-10T00:40:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.228+0000 7f6ebcff9700 1 -- 172.21.15.62:0/1604685095 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f6ec8011070 con 0x555f32615670 2024-06-10T00:40:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.228+0000 7f6eea0adb80 10 client.16626 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.228+0000 7f6eea0adb80 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:40:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.228+0000 7f6eea0adb80 20 client.16626 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.228+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/1604685095 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555f3262f220 0x555f32631610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.228+0000 7f6eea0adb80 4 client.16626 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:40:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.228+0000 7f6eea0adb80 1 -- 172.21.15.62:0/1604685095 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f3256fd50 con 0x555f3262f220 2024-06-10T00:40:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.228+0000 7f6edcdc5700 1 --2- 172.21.15.62:0/1604685095 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555f3262f220 0x555f32631610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.228+0000 7f6edcdc5700 1 --2- 172.21.15.62:0/1604685095 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555f3262f220 0x555f32631610 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ae0a5663995ad8a server_cookie=819dd040393142ff in_seq=0 out_seq=0 2024-06-10T00:40:24.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.228+0000 7f6ebcff9700 10 client.16626 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:24.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.229+0000 7f6ebcff9700 1 -- 172.21.15.62:0/1604685095 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555f3256fd50 con 0x555f3262f220 2024-06-10T00:40:24.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.229+0000 7f6ebcff9700 10 client.16626 handle_command_reply: tid=0 2024-06-10T00:40:24.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.229+0000 7f6eea0adb80 1 client.16626 shutdown 2024-06-10T00:40:24.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.229+0000 7f6eea0adb80 1 -- 172.21.15.62:0/1604685095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f32615670 msgr2=0x555f32621e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:24.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.229+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/1604685095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f32615670 0x555f32621e50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8000f80 tx=0x7f6ec800b860 comp rx=0 tx=0).stop 2024-06-10T00:40:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.229+0000 7f6eea0adb80 1 -- 172.21.15.62:0/1604685095 shutdown_connections 2024-06-10T00:40:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.229+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/1604685095 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555f3262f220 0x555f32631610 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.229+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/1604685095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f32615670 0x555f32621e50 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.229+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/1604685095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f3261b470 0x555f3261f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.229+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/1604685095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 0x555f3261f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.229+0000 7f6ebcff9700 0 client.16626 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.229+0000 7f6eea0adb80 1 -- 172.21.15.62:0/1604685095 >> 172.21.15.62:0/1604685095 conn(0x555f3253e080 msgr2=0x555f3260b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.230+0000 7f6eea0adb80 1 -- 172.21.15.62:0/1604685095 shutdown_connections 2024-06-10T00:40:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.230+0000 7f6eea0adb80 1 -- 172.21.15.62:0/1604685095 wait complete. 2024-06-10T00:40:24.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.230+0000 7f6eea0adb80 20 client.16626 trim_cache size 0 max 16384 2024-06-10T00:40:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6eea0adb80 1 Processor -- start 2024-06-10T00:40:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6eea0adb80 1 -- start start 2024-06-10T00:40:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6eea0adb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f32615670 0x555f32610960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6eea0adb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 0x555f3260be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6eea0adb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3261b470 0x555f3260c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6eea0adb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81ac3c0 con 0x555f3261b470 2024-06-10T00:40:24.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6eea0adb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81adf00 con 0x555f32616030 2024-06-10T00:40:24.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6eea0adb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81aca90 con 0x555f32615670 2024-06-10T00:40:24.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6edddc7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3261b470 0x555f3260c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6edddc7700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3261b470 0x555f3260c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44000/0 (socket says 172.21.15.62:44000) 2024-06-10T00:40:24.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6edddc7700 1 -- 172.21.15.62:0/2018216410 learned_addr learned my addr 172.21.15.62:0/2018216410 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:24.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6edcdc5700 1 --2- 172.21.15.62:0/2018216410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 0x555f3260be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6edd5c6700 1 --2- 172.21.15.62:0/2018216410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f32615670 0x555f32610960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6edddc7700 1 -- 172.21.15.62:0/2018216410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f32615670 msgr2=0x555f32610960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:24.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6edddc7700 1 --2- 172.21.15.62:0/2018216410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f32615670 0x555f32610960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6edddc7700 1 -- 172.21.15.62:0/2018216410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 msgr2=0x555f3260be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:24.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6edddc7700 1 --2- 172.21.15.62:0/2018216410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 0x555f3260be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.251+0000 7f6edddc7700 1 -- 172.21.15.62:0/2018216410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f3261b850 con 0x555f3261b470 2024-06-10T00:40:24.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.252+0000 7f6edcdc5700 1 --2- 172.21.15.62:0/2018216410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 0x555f3260be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:40:24.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.252+0000 7f6edddc7700 1 --2- 172.21.15.62:0/2018216410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3261b470 0x555f3260c3c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0004610 tx=0x7f6ed0003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a726d65f325e03 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:24.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.252+0000 7f6ebeffd700 1 -- 172.21.15.62:0/2018216410 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000e8d0 con 0x555f3261b470 2024-06-10T00:40:24.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.252+0000 7f6ebeffd700 1 -- 172.21.15.62:0/2018216410 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ed0012450 con 0x555f3261b470 2024-06-10T00:40:24.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.252+0000 7f6ebeffd700 1 -- 172.21.15.62:0/2018216410 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed001c6a0 con 0x555f3261b470 2024-06-10T00:40:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.252+0000 7f6eea0adb80 1 -- 172.21.15.62:0/2018216410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3261b470 msgr2=0x555f3260c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.252+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/2018216410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3261b470 0x555f3260c3c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0004610 tx=0x7f6ed0003960 comp rx=0 tx=0).stop 2024-06-10T00:40:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.253+0000 7f6eea0adb80 1 -- 172.21.15.62:0/2018216410 shutdown_connections 2024-06-10T00:40:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.253+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/2018216410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f3261b470 0x555f3260c3c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.253+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/2018216410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f32615670 0x555f32610960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.253+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/2018216410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 0x555f3260be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.253+0000 7f6eea0adb80 1 -- 172.21.15.62:0/2018216410 >> 172.21.15.62:0/2018216410 conn(0x555f3253e080 msgr2=0x555f32610250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.253+0000 7f6eea0adb80 1 -- 172.21.15.62:0/2018216410 shutdown_connections 2024-06-10T00:40:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.253+0000 7f6eea0adb80 1 -- 172.21.15.62:0/2018216410 wait complete. 2024-06-10T00:40:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.253+0000 7f6eea0adb80 1 -- start start 2024-06-10T00:40:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.253+0000 7f6eea0adb80 1 Processor -- start 2024-06-10T00:40:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6eea0adb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f32615670 0x555f32692590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6eea0adb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 0x555f32692ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6eea0adb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f3261b470 0x555f32634370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6eea0adb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81ad830 con 0x555f32615670 2024-06-10T00:40:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6eea0adb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81ad160 con 0x555f32616030 2024-06-10T00:40:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6eea0adb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81aa3a0 con 0x555f3261b470 2024-06-10T00:40:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6eea0adb80 10 client.0 _ll_register_callbacks cb 0x555f324f85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6edd5c6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f32615670 0x555f32692590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6edd5c6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f32615670 0x555f32692590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44010/0 (socket says 172.21.15.62:44010) 2024-06-10T00:40:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6edd5c6700 1 -- 172.21.15.62:0/389338910 learned_addr learned my addr 172.21.15.62:0/389338910 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6edddc7700 1 --2- 172.21.15.62:0/389338910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f3261b470 0x555f32634370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6edcdc5700 1 --2- 172.21.15.62:0/389338910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 0x555f32692ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6edd5c6700 1 -- 172.21.15.62:0/389338910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f3261b470 msgr2=0x555f32634370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6edd5c6700 1 --2- 172.21.15.62:0/389338910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f3261b470 0x555f32634370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6edd5c6700 1 -- 172.21.15.62:0/389338910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 msgr2=0x555f32692ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6edd5c6700 1 --2- 172.21.15.62:0/389338910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 0x555f32692ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6edd5c6700 1 -- 172.21.15.62:0/389338910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f32634a70 con 0x555f32615670 2024-06-10T00:40:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6edcdc5700 1 --2- 172.21.15.62:0/389338910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 0x555f32692ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:40:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6edd5c6700 1 --2- 172.21.15.62:0/389338910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f32615670 0x555f32692590 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8006130 tx=0x7f6ec803a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f81365fda2d8bc7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6ebcff9700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6ebcff9700 1 -- 172.21.15.62:0/389338910 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8036070 con 0x555f32615670 2024-06-10T00:40:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6ebcff9700 1 -- 172.21.15.62:0/389338910 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f6ec803adc0 con 0x555f32615670 2024-06-10T00:40:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.254+0000 7f6eea0adb80 1 -- 172.21.15.62:0/389338910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ed0012450 con 0x555f32615670 2024-06-10T00:40:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.255+0000 7f6ebcff9700 1 -- 172.21.15.62:0/389338910 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6ec8050050 con 0x555f32615670 2024-06-10T00:40:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.255+0000 7f6eea0adb80 10 client.16632 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.255+0000 7f6eea0adb80 1 -- 172.21.15.62:0/389338910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555f32699d40 con 0x555f32615670 2024-06-10T00:40:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.255+0000 7f6ebcff9700 1 -- 172.21.15.62:0/389338910 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ed0012450 con 0x555f32615670 2024-06-10T00:40:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.255+0000 7f6ebcff9700 1 -- 172.21.15.62:0/389338910 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f6ec8011040 con 0x555f32615670 2024-06-10T00:40:24.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.255+0000 7f6eea0adb80 10 client.16632 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:24.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.255+0000 7f6eea0adb80 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:40:24.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.255+0000 7f6eea0adb80 20 client.16632 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:24.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.255+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/389338910 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555f3269a2d0 0x555f3269c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:24.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.255+0000 7f6eea0adb80 4 client.16632 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:40:24.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.255+0000 7f6eea0adb80 1 -- 172.21.15.62:0/389338910 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555f3256a2d0 con 0x555f3269a2d0 2024-06-10T00:40:24.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.255+0000 7f6edcdc5700 1 --2- 172.21.15.62:0/389338910 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555f3269a2d0 0x555f3269c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:24.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6edcdc5700 1 --2- 172.21.15.62:0/389338910 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555f3269a2d0 0x555f3269c6f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb5c293431bd4896 server_cookie=f8e3be0afc18619d in_seq=0 out_seq=0 2024-06-10T00:40:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6ebcff9700 10 client.16632 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6ebcff9700 1 -- 172.21.15.62:0/389338910 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555f3256a2d0 con 0x555f3269a2d0 2024-06-10T00:40:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6ebcff9700 10 client.16632 handle_command_reply: tid=0 2024-06-10T00:40:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6eea0adb80 1 client.16632 shutdown 2024-06-10T00:40:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6eea0adb80 1 -- 172.21.15.62:0/389338910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f32615670 msgr2=0x555f32692590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/389338910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f32615670 0x555f32692590 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8006130 tx=0x7f6ec803a7a0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6eea0adb80 1 -- 172.21.15.62:0/389338910 shutdown_connections 2024-06-10T00:40:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/389338910 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x555f3269a2d0 0x555f3269c6f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/389338910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555f32615670 0x555f32692590 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/389338910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f3261b470 0x555f32634370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6eea0adb80 1 --2- 172.21.15.62:0/389338910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f32616030 0x555f32692ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6eea0adb80 1 -- 172.21.15.62:0/389338910 >> 172.21.15.62:0/389338910 conn(0x555f3253e080 msgr2=0x555f32610ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6ebcff9700 0 client.16632 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6eea0adb80 1 -- 172.21.15.62:0/389338910 shutdown_connections 2024-06-10T00:40:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6eea0adb80 1 -- 172.21.15.62:0/389338910 wait complete. 2024-06-10T00:40:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.256+0000 7f6eea0adb80 20 client.16632 trim_cache size 0 max 16384 2024-06-10T00:40:24.267 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:40:24.267 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-06-10T00:40:24.267 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-06-10T00:40:24.267 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:40:24.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.257+0000 7f6edf82a700 1 -- 172.21.15.62:0/4276359788 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6ec00a2d90 msgr2=0x7f6ec00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:24.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.257+0000 7f6edf82a700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6ec00a2d90 0x7f6ec00a5250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009850 tx=0x7f6ec80097a0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.257+0000 7f6edf82a700 1 -- 172.21.15.62:0/4276359788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed8102380 msgr2=0x7f6ed819edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:24.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.257+0000 7f6edf82a700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed8102380 0x7f6ed819edb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00c9b0 tx=0x7f6ecc00cd70 comp rx=0 tx=0).stop 2024-06-10T00:40:24.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.257+0000 7f6edf82a700 1 -- 172.21.15.62:0/4276359788 shutdown_connections 2024-06-10T00:40:24.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.257+0000 7f6edf82a700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6ec00a2d90 0x7f6ec00a5250 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.257+0000 7f6edf82a700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ed8106720 0x7f6ed8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.257+0000 7f6edf82a700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ed81019d0 0x7f6ed819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.257+0000 7f6edf82a700 1 --2- 172.21.15.62:0/4276359788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ed8102380 0x7f6ed819edb0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:24.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.257+0000 7f6edf82a700 1 -- 172.21.15.62:0/4276359788 >> 172.21.15.62:0/4276359788 conn(0x7f6ed8078340 msgr2=0x7f6ed806ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:24.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.257+0000 7f6edf82a700 1 -- 172.21.15.62:0/4276359788 shutdown_connections 2024-06-10T00:40:24.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:24.257+0000 7f6edf82a700 1 -- 172.21.15.62:0/4276359788 wait complete. 2024-06-10T00:40:24.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:24 smithi062 ceph-mon[32390]: pgmap v232: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 96 B/s wr, 10 op/s 2024-06-10T00:40:24.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:24 smithi138 ceph-mon[36549]: pgmap v232: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 96 B/s wr, 10 op/s 2024-06-10T00:40:25.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:24 smithi105 ceph-mon[36321]: pgmap v232: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 96 B/s wr, 10 op/s 2024-06-10T00:40:25.049 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T00:40:25.049 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:40:25.300 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:40:27.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.131+0000 7fe35cfe0700 1 -- 172.21.15.62:0/4262353353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe358107400 msgr2=0x7fe3581077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:27.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.131+0000 7fe35cfe0700 1 --2- 172.21.15.62:0/4262353353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe358107400 0x7fe3581077e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe348009990 tx=0x7fe348009ca0 comp rx=0 tx=0).stop 2024-06-10T00:40:27.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.132+0000 7fe35cfe0700 1 -- 172.21.15.62:0/4262353353 shutdown_connections 2024-06-10T00:40:27.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.132+0000 7fe35cfe0700 1 --2- 172.21.15.62:0/4262353353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe358107400 0x7fe3581077e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:27.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.132+0000 7fe35cfe0700 1 --2- 172.21.15.62:0/4262353353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe358101d50 0x7fe35810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:27.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.132+0000 7fe35cfe0700 1 --2- 172.21.15.62:0/4262353353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe358101390 0x7fe358101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:27.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.132+0000 7fe35cfe0700 1 -- 172.21.15.62:0/4262353353 >> 172.21.15.62:0/4262353353 conn(0x7fe358078340 msgr2=0x7fe358078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:27.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.132+0000 7fe35cfe0700 1 -- 172.21.15.62:0/4262353353 shutdown_connections 2024-06-10T00:40:27.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.132+0000 7fe35cfe0700 1 -- 172.21.15.62:0/4262353353 wait complete. 2024-06-10T00:40:27.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.132+0000 7fe35cfe0700 1 Processor -- start 2024-06-10T00:40:27.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.132+0000 7fe35cfe0700 1 -- start start 2024-06-10T00:40:27.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe35cfe0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe358101390 0x7fe35819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:27.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe35cfe0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe358101d50 0x7fe35819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:27.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe35cfe0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe358107400 0x7fe358198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:27.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe35cfe0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe358111670 con 0x7fe358107400 2024-06-10T00:40:27.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe35cfe0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe3581114f0 con 0x7fe358101d50 2024-06-10T00:40:27.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe35cfe0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe3581117f0 con 0x7fe358101390 2024-06-10T00:40:27.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe356d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe358107400 0x7fe358198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:27.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe356d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe358107400 0x7fe358198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44026/0 (socket says 172.21.15.62:44026) 2024-06-10T00:40:27.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe356d9d700 1 -- 172.21.15.62:0/1197872016 learned_addr learned my addr 172.21.15.62:0/1197872016 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:27.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe35659c700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe358101390 0x7fe35819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:27.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe355d9b700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe358101d50 0x7fe35819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:27.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe356d9d700 1 -- 172.21.15.62:0/1197872016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe358101390 msgr2=0x7fe35819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:27.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe356d9d700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe358101390 0x7fe35819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:27.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe356d9d700 1 -- 172.21.15.62:0/1197872016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe358101d50 msgr2=0x7fe35819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:27.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe356d9d700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe358101d50 0x7fe35819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:27.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe356d9d700 1 -- 172.21.15.62:0/1197872016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe358199260 con 0x7fe358107400 2024-06-10T00:40:27.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.133+0000 7fe355d9b700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe358101d50 0x7fe35819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:40:27.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.134+0000 7fe356d9d700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe358107400 0x7fe358198b00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe350007ee0 tx=0x7fe35000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:27.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.134+0000 7fe3477fe700 1 -- 172.21.15.62:0/1197872016 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35000ddd0 con 0x7fe358107400 2024-06-10T00:40:27.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.134+0000 7fe35cfe0700 1 -- 172.21.15.62:0/1197872016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe358199550 con 0x7fe358107400 2024-06-10T00:40:27.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.134+0000 7fe3477fe700 1 -- 172.21.15.62:0/1197872016 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe350004020 con 0x7fe358107400 2024-06-10T00:40:27.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.134+0000 7fe3477fe700 1 -- 172.21.15.62:0/1197872016 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350015660 con 0x7fe358107400 2024-06-10T00:40:27.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.134+0000 7fe35cfe0700 1 -- 172.21.15.62:0/1197872016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3581a5810 con 0x7fe358107400 2024-06-10T00:40:27.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.135+0000 7fe35cfe0700 1 -- 172.21.15.62:0/1197872016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe358069180 con 0x7fe358107400 2024-06-10T00:40:27.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.136+0000 7fe3477fe700 1 -- 172.21.15.62:0/1197872016 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe3500041d0 con 0x7fe358107400 2024-06-10T00:40:27.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.137+0000 7fe3477fe700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe3400a2d90 0x7fe3400a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:27.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.137+0000 7fe35659c700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe3400a2d90 0x7fe3400a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:27.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.137+0000 7fe3477fe700 1 -- 172.21.15.62:0/1197872016 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fe3500cfa10 con 0x7fe358107400 2024-06-10T00:40:27.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.137+0000 7fe35659c700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe3400a2d90 0x7fe3400a5250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe3480098e0 tx=0x7fe348005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:27.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.140+0000 7fe3477fe700 1 -- 172.21.15.62:0/1197872016 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe350010070 con 0x7fe358107400 2024-06-10T00:40:27.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.393+0000 7fe35cfe0700 1 -- 172.21.15.62:0/1197872016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3581a5a40 con 0x7fe358107400 2024-06-10T00:40:27.394 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:27 smithi062 ceph-mon[32390]: pgmap v233: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:40:27.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.394+0000 7fe3477fe700 1 -- 172.21.15.62:0/1197872016 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11938 (secure 0 0 0) 0x7fe35009c220 con 0x7fe358107400 2024-06-10T00:40:27.395 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:40:27.395 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:40:10.974660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:40:27.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.396+0000 7fe35cfe0700 1 -- 172.21.15.62:0/1197872016 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe3400a2d90 msgr2=0x7fe3400a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:27.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.396+0000 7fe35cfe0700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe3400a2d90 0x7fe3400a5250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe3480098e0 tx=0x7fe348005960 comp rx=0 tx=0).stop 2024-06-10T00:40:27.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.396+0000 7fe35cfe0700 1 -- 172.21.15.62:0/1197872016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe358107400 msgr2=0x7fe358198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:27.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.396+0000 7fe35cfe0700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe358107400 0x7fe358198b00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe350007ee0 tx=0x7fe35000d3b0 comp rx=0 tx=0).stop 2024-06-10T00:40:27.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.396+0000 7fe35cfe0700 1 -- 172.21.15.62:0/1197872016 shutdown_connections 2024-06-10T00:40:27.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.396+0000 7fe35cfe0700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fe3400a2d90 0x7fe3400a5250 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:27.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.396+0000 7fe35cfe0700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe358107400 0x7fe358198b00 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:27.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.396+0000 7fe35cfe0700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe358101390 0x7fe35819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:27.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.396+0000 7fe35cfe0700 1 --2- 172.21.15.62:0/1197872016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe358101d50 0x7fe35819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:27.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.396+0000 7fe35cfe0700 1 -- 172.21.15.62:0/1197872016 >> 172.21.15.62:0/1197872016 conn(0x7fe358078340 msgr2=0x7fe3581057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:27.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.396+0000 7fe35cfe0700 1 -- 172.21.15.62:0/1197872016 shutdown_connections 2024-06-10T00:40:27.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:27.396+0000 7fe35cfe0700 1 -- 172.21.15.62:0/1197872016 wait complete. 2024-06-10T00:40:27.398 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 38 2024-06-10T00:40:27.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:27 smithi105 ceph-mon[36321]: pgmap v233: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:40:27.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:27 smithi138 ceph-mon[36549]: pgmap v233: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:40:28.289 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:28 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1197872016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:40:28.321 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T00:40:28.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:28 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1197872016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:40:28.576 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:40:28.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:28 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1197872016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:40:29.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:29 smithi105 ceph-mon[36321]: pgmap v234: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-06-10T00:40:29.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:29 smithi062 ceph-mon[32390]: pgmap v234: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-06-10T00:40:29.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:29 smithi138 ceph-mon[36549]: pgmap v234: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-06-10T00:40:30.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.377+0000 7f8137ae0700 1 -- 172.21.15.62:0/889902915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8130102770 msgr2=0x7f8130102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:30.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.377+0000 7f8137ae0700 1 --2- 172.21.15.62:0/889902915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8130102770 0x7f8130102b50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f812c0099c0 tx=0x7f812c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.377+0000 7f8137ae0700 1 -- 172.21.15.62:0/889902915 shutdown_connections 2024-06-10T00:40:30.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.377+0000 7f8137ae0700 1 --2- 172.21.15.62:0/889902915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8130102770 0x7f8130102b50 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.377+0000 7f8137ae0700 1 --2- 172.21.15.62:0/889902915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8130107860 0x7f813010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.377+0000 7f8137ae0700 1 --2- 172.21.15.62:0/889902915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8130103120 0x7f8130107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.377+0000 7f8137ae0700 1 -- 172.21.15.62:0/889902915 >> 172.21.15.62:0/889902915 conn(0x7f8130079600 msgr2=0x7f8130079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:30.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.377+0000 7f8137ae0700 1 -- 172.21.15.62:0/889902915 shutdown_connections 2024-06-10T00:40:30.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.377+0000 7f8137ae0700 1 -- 172.21.15.62:0/889902915 wait complete. 2024-06-10T00:40:30.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.377+0000 7f8137ae0700 1 Processor -- start 2024-06-10T00:40:30.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.377+0000 7f8137ae0700 1 -- start start 2024-06-10T00:40:30.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f8137ae0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8130102770 0x7f813019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f8137ae0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8130103120 0x7f813019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f8137ae0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8130107860 0x7f81301989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f8137ae0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8130112780 con 0x7f8130102770 2024-06-10T00:40:30.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f8137ae0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8130112600 con 0x7f8130107860 2024-06-10T00:40:30.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f8137ae0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8130112900 con 0x7f8130103120 2024-06-10T00:40:30.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f813587c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8130102770 0x7f813019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f813587c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8130102770 0x7f813019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34852/0 (socket says 172.21.15.62:34852) 2024-06-10T00:40:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f813587c700 1 -- 172.21.15.62:0/2266172810 learned_addr learned my addr 172.21.15.62:0/2266172810 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f813507b700 1 --2- 172.21.15.62:0/2266172810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8130103120 0x7f813019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f813607d700 1 --2- 172.21.15.62:0/2266172810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8130107860 0x7f81301989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f813587c700 1 -- 172.21.15.62:0/2266172810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8130103120 msgr2=0x7f813019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f813587c700 1 --2- 172.21.15.62:0/2266172810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8130103120 0x7f813019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f813587c700 1 -- 172.21.15.62:0/2266172810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8130107860 msgr2=0x7f81301989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f813587c700 1 --2- 172.21.15.62:0/2266172810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8130107860 0x7f81301989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f813587c700 1 -- 172.21.15.62:0/2266172810 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8130199140 con 0x7f8130102770 2024-06-10T00:40:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.378+0000 7f813587c700 1 --2- 172.21.15.62:0/2266172810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8130102770 0x7f813019e820 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f812c006010 tx=0x7f812c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.379+0000 7f8122ffd700 1 -- 172.21.15.62:0/2266172810 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c026070 con 0x7f8130102770 2024-06-10T00:40:30.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.379+0000 7f8122ffd700 1 -- 172.21.15.62:0/2266172810 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f812c004cd0 con 0x7f8130102770 2024-06-10T00:40:30.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.379+0000 7f8137ae0700 1 -- 172.21.15.62:0/2266172810 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81301993d0 con 0x7f8130102770 2024-06-10T00:40:30.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.379+0000 7f8122ffd700 1 -- 172.21.15.62:0/2266172810 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c003ba0 con 0x7f8130102770 2024-06-10T00:40:30.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.379+0000 7f8137ae0700 1 -- 172.21.15.62:0/2266172810 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81301a56f0 con 0x7f8130102770 2024-06-10T00:40:30.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.381+0000 7f8122ffd700 1 -- 172.21.15.62:0/2266172810 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f812c029070 con 0x7f8130102770 2024-06-10T00:40:30.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.382+0000 7f8122ffd700 1 --2- 172.21.15.62:0/2266172810 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f811c0a2d90 0x7f811c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.382+0000 7f813507b700 1 --2- 172.21.15.62:0/2266172810 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f811c0a2d90 0x7f811c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.382+0000 7f8122ffd700 1 -- 172.21.15.62:0/2266172810 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f812c0d0190 con 0x7f8130102770 2024-06-10T00:40:30.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.382+0000 7f813507b700 1 --2- 172.21.15.62:0/2266172810 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f811c0a2d90 0x7f811c0a5250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8130102070 tx=0x7f812400b410 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:30.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f8142363b80 1 Processor -- start 2024-06-10T00:40:30.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f8142363b80 1 -- start start 2024-06-10T00:40:30.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f8142363b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb2f60 0x563645fb3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f8142363b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb3880 0x563645fad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f8142363b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 0x563645fadbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f8142363b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563645e9ff60 con 0x563645fb3880 2024-06-10T00:40:30.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f8142363b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563645efe850 con 0x563645fb2f60 2024-06-10T00:40:30.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f8142363b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563645e66460 con 0x563645fad7a0 2024-06-10T00:40:30.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f813507b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb3880 0x563645fad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f813507b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb3880 0x563645fad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34856/0 (socket says 172.21.15.62:34856) 2024-06-10T00:40:30.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f813507b700 1 -- 172.21.15.62:0/2841341549 learned_addr learned my addr 172.21.15.62:0/2841341549 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:30.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f813587c700 1 --2- 172.21.15.62:0/2841341549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb2f60 0x563645fb3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f813607d700 1 --2- 172.21.15.62:0/2841341549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 0x563645fadbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f813507b700 1 -- 172.21.15.62:0/2841341549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 msgr2=0x563645fadbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:30.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f813507b700 1 --2- 172.21.15.62:0/2841341549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 0x563645fadbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f813507b700 1 -- 172.21.15.62:0/2841341549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb2f60 msgr2=0x563645fb3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:30.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f813507b700 1 --2- 172.21.15.62:0/2841341549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb2f60 0x563645fb3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.383+0000 7f813507b700 1 -- 172.21.15.62:0/2841341549 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563645e4d8d0 con 0x563645fb3880 2024-06-10T00:40:30.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.384+0000 7f813507b700 1 --2- 172.21.15.62:0/2841341549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb3880 0x563645fad260 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8124010f70 tx=0x7f8124013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3d48efd20bfd12 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:30.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.384+0000 7f811bfff700 1 -- 172.21.15.62:0/2841341549 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812400be60 con 0x563645fb3880 2024-06-10T00:40:30.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.384+0000 7f811bfff700 1 -- 172.21.15.62:0/2841341549 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8124011db0 con 0x563645fb3880 2024-06-10T00:40:30.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.384+0000 7f811bfff700 1 -- 172.21.15.62:0/2841341549 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812401c7c0 con 0x563645fb3880 2024-06-10T00:40:30.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.385+0000 7f8142363b80 1 -- 172.21.15.62:0/2841341549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb3880 msgr2=0x563645fad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:30.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.385+0000 7f8142363b80 1 --2- 172.21.15.62:0/2841341549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb3880 0x563645fad260 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8124010f70 tx=0x7f8124013000 comp rx=0 tx=0).stop 2024-06-10T00:40:30.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.385+0000 7f8142363b80 1 -- 172.21.15.62:0/2841341549 shutdown_connections 2024-06-10T00:40:30.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.385+0000 7f8142363b80 1 --2- 172.21.15.62:0/2841341549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 0x563645fadbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.385+0000 7f8142363b80 1 --2- 172.21.15.62:0/2841341549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb2f60 0x563645fb3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.385+0000 7f8142363b80 1 --2- 172.21.15.62:0/2841341549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb3880 0x563645fad260 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.385+0000 7f8142363b80 1 -- 172.21.15.62:0/2841341549 >> 172.21.15.62:0/2841341549 conn(0x563645eda080 msgr2=0x563645fa8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.385+0000 7f8142363b80 1 -- 172.21.15.62:0/2841341549 shutdown_connections 2024-06-10T00:40:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.385+0000 7f8142363b80 1 -- 172.21.15.62:0/2841341549 wait complete. 2024-06-10T00:40:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.385+0000 7f8142363b80 1 -- start start 2024-06-10T00:40:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.385+0000 7f8142363b80 1 Processor -- start 2024-06-10T00:40:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f8142363b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fad7a0 0x563645fb9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f8142363b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fb2f60 0x563645fb6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f8142363b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb3880 0x563645fb6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f8142363b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563645e9efc0 con 0x563645fad7a0 2024-06-10T00:40:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f8142363b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563645efa810 con 0x563645fb3880 2024-06-10T00:40:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f8142363b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563645fba260 con 0x563645fb2f60 2024-06-10T00:40:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f8142363b80 10 client.0 _ll_register_callbacks cb 0x563645e945b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f813587c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fad7a0 0x563645fb9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f813587c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fad7a0 0x563645fb9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34862/0 (socket says 172.21.15.62:34862) 2024-06-10T00:40:30.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f813587c700 1 -- 172.21.15.62:0/2811147448 learned_addr learned my addr 172.21.15.62:0/2811147448 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f813607d700 1 --2- 172.21.15.62:0/2811147448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb3880 0x563645fb6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f813507b700 1 --2- 172.21.15.62:0/2811147448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fb2f60 0x563645fb6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f813587c700 1 -- 172.21.15.62:0/2811147448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fb2f60 msgr2=0x563645fb6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f813587c700 1 --2- 172.21.15.62:0/2811147448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fb2f60 0x563645fb6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f813587c700 1 -- 172.21.15.62:0/2811147448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb3880 msgr2=0x563645fb6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f813587c700 1 --2- 172.21.15.62:0/2811147448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb3880 0x563645fb6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f813587c700 1 -- 172.21.15.62:0/2811147448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563645ec4750 con 0x563645fad7a0 2024-06-10T00:40:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f813607d700 1 --2- 172.21.15.62:0/2811147448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb3880 0x563645fb6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:40:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f813587c700 1 --2- 172.21.15.62:0/2811147448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fad7a0 0x563645fb9600 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f812c0059d0 tx=0x7f812c09c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bcef82392f75b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f81197fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.386+0000 7f81197fa700 1 -- 172.21.15.62:0/2811147448 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c09ca30 con 0x563645fad7a0 2024-06-10T00:40:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.387+0000 7f81197fa700 1 -- 172.21.15.62:0/2811147448 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f812c0a4a80 con 0x563645fad7a0 2024-06-10T00:40:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.387+0000 7f8142363b80 1 -- 172.21.15.62:0/2811147448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8124011db0 con 0x563645fad7a0 2024-06-10T00:40:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.387+0000 7f81197fa700 1 -- 172.21.15.62:0/2811147448 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f812c09c330 con 0x563645fad7a0 2024-06-10T00:40:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.387+0000 7f8142363b80 10 client.16650 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.387+0000 7f8142363b80 1 -- 172.21.15.62:0/2811147448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563645fc49f0 con 0x563645fad7a0 2024-06-10T00:40:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.387+0000 7f81197fa700 1 -- 172.21.15.62:0/2811147448 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8124011db0 con 0x563645fad7a0 2024-06-10T00:40:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.387+0000 7f81197fa700 1 -- 172.21.15.62:0/2811147448 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f812c0bc070 con 0x563645fad7a0 2024-06-10T00:40:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.387+0000 7f8142363b80 10 client.16650 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.387+0000 7f8142363b80 10 client.16650 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:40:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.387+0000 7f8142363b80 20 client.16650 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:30.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.387+0000 7f8142363b80 1 --2- 172.21.15.62:0/2811147448 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x563645fc6f60 0x563645fc9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.387+0000 7f8142363b80 4 client.16650 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:40:30.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.387+0000 7f8142363b80 1 -- 172.21.15.62:0/2811147448 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563645f0bd50 con 0x563645fc6f60 2024-06-10T00:40:30.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.387+0000 7f813507b700 1 --2- 172.21.15.62:0/2811147448 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x563645fc6f60 0x563645fc9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.388+0000 7f813507b700 1 --2- 172.21.15.62:0/2811147448 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x563645fc6f60 0x563645fc9390 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=57f6baf22efd826d server_cookie=1039871120bf1035 in_seq=0 out_seq=0 2024-06-10T00:40:30.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.388+0000 7f81197fa700 10 client.16650 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:40:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f81197fa700 1 -- 172.21.15.62:0/2811147448 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563645f0bd50 con 0x563645fc6f60 2024-06-10T00:40:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f81197fa700 10 client.16650 handle_command_reply: tid=0 2024-06-10T00:40:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f8142363b80 1 client.16650 shutdown 2024-06-10T00:40:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f8142363b80 1 -- 172.21.15.62:0/2811147448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fad7a0 msgr2=0x563645fb9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f8142363b80 1 --2- 172.21.15.62:0/2811147448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fad7a0 0x563645fb9600 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f812c0059d0 tx=0x7f812c09c210 comp rx=0 tx=0).stop 2024-06-10T00:40:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f8142363b80 1 -- 172.21.15.62:0/2811147448 shutdown_connections 2024-06-10T00:40:30.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f8142363b80 1 --2- 172.21.15.62:0/2811147448 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x563645fc6f60 0x563645fc9390 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f8142363b80 1 --2- 172.21.15.62:0/2811147448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fad7a0 0x563645fb9600 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f8142363b80 1 --2- 172.21.15.62:0/2811147448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb3880 0x563645fb6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f81197fa700 0 client.16650 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:40:30.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f8142363b80 1 --2- 172.21.15.62:0/2811147448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fb2f60 0x563645fb6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f8142363b80 1 -- 172.21.15.62:0/2811147448 >> 172.21.15.62:0/2811147448 conn(0x563645eda080 msgr2=0x563645fa7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:30.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f8142363b80 1 -- 172.21.15.62:0/2811147448 shutdown_connections 2024-06-10T00:40:30.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f8142363b80 1 -- 172.21.15.62:0/2811147448 wait complete. 2024-06-10T00:40:30.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.389+0000 7f8142363b80 20 client.16650 trim_cache size 0 max 16384 2024-06-10T00:40:30.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.404+0000 7f8142363b80 1 Processor -- start 2024-06-10T00:40:30.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.404+0000 7f8142363b80 1 -- start start 2024-06-10T00:40:30.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f8142363b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 0x563645fad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f8142363b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb2f60 0x563645fb9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f8142363b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb3880 0x563645fa7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f8142363b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f81301ae5a0 con 0x563645fb2f60 2024-06-10T00:40:30.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f8142363b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b00e0 con 0x563645fb3880 2024-06-10T00:40:30.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f8142363b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81301aec70 con 0x563645fad7a0 2024-06-10T00:40:30.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f813507b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb2f60 0x563645fb9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f813507b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb2f60 0x563645fb9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34864/0 (socket says 172.21.15.62:34864) 2024-06-10T00:40:30.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f813507b700 1 -- 172.21.15.62:0/250228130 learned_addr learned my addr 172.21.15.62:0/250228130 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:30.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f813587c700 1 --2- 172.21.15.62:0/250228130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 0x563645fad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f813607d700 1 --2- 172.21.15.62:0/250228130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb3880 0x563645fa7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f813507b700 1 -- 172.21.15.62:0/250228130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 msgr2=0x563645fad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:30.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f813507b700 1 --2- 172.21.15.62:0/250228130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 0x563645fad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f813507b700 1 -- 172.21.15.62:0/250228130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb3880 msgr2=0x563645fa7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:30.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f813507b700 1 --2- 172.21.15.62:0/250228130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb3880 0x563645fa7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f813507b700 1 -- 172.21.15.62:0/250228130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563645fb3680 con 0x563645fb2f60 2024-06-10T00:40:30.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f813607d700 1 --2- 172.21.15.62:0/250228130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb3880 0x563645fa7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:40:30.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f813507b700 1 --2- 172.21.15.62:0/250228130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb2f60 0x563645fb9140 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8124011a00 tx=0x7f8124010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7c2668378db65a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:30.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.405+0000 7f811b7fe700 1 -- 172.21.15.62:0/250228130 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124005240 con 0x563645fb2f60 2024-06-10T00:40:30.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.406+0000 7f811b7fe700 1 -- 172.21.15.62:0/250228130 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8124004b70 con 0x563645fb2f60 2024-06-10T00:40:30.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.406+0000 7f811b7fe700 1 -- 172.21.15.62:0/250228130 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81240336e0 con 0x563645fb2f60 2024-06-10T00:40:30.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.406+0000 7f8142363b80 1 -- 172.21.15.62:0/250228130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb2f60 msgr2=0x563645fb9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:30.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.406+0000 7f8142363b80 1 --2- 172.21.15.62:0/250228130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb2f60 0x563645fb9140 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8124011a00 tx=0x7f8124010fa0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 -- 172.21.15.62:0/250228130 shutdown_connections 2024-06-10T00:40:30.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 --2- 172.21.15.62:0/250228130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb3880 0x563645fa7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 --2- 172.21.15.62:0/250228130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 0x563645fad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 --2- 172.21.15.62:0/250228130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb2f60 0x563645fb9140 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 -- 172.21.15.62:0/250228130 >> 172.21.15.62:0/250228130 conn(0x563645eda080 msgr2=0x563645fb1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:30.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 -- 172.21.15.62:0/250228130 shutdown_connections 2024-06-10T00:40:30.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 -- 172.21.15.62:0/250228130 wait complete. 2024-06-10T00:40:30.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 -- start start 2024-06-10T00:40:30.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 Processor -- start 2024-06-10T00:40:30.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 0x563645fbfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb2f60 0x563645fce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb3880 0x563645fce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f81301afa10 con 0x563645fb3880 2024-06-10T00:40:30.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f81301af340 con 0x563645fb2f60 2024-06-10T00:40:30.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.407+0000 7f8142363b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81301ac560 con 0x563645fad7a0 2024-06-10T00:40:30.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f8142363b80 10 client.0 _ll_register_callbacks cb 0x563645e945b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:30.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f813607d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb3880 0x563645fce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f813607d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb3880 0x563645fce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34870/0 (socket says 172.21.15.62:34870) 2024-06-10T00:40:30.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f813607d700 1 -- 172.21.15.62:0/462226041 learned_addr learned my addr 172.21.15.62:0/462226041 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:30.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f813587c700 1 --2- 172.21.15.62:0/462226041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 0x563645fbfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f813507b700 1 --2- 172.21.15.62:0/462226041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb2f60 0x563645fce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f813607d700 1 -- 172.21.15.62:0/462226041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 msgr2=0x563645fbfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:30.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f813607d700 1 --2- 172.21.15.62:0/462226041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 0x563645fbfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f813607d700 1 -- 172.21.15.62:0/462226041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb2f60 msgr2=0x563645fce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:30.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f813607d700 1 --2- 172.21.15.62:0/462226041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb2f60 0x563645fce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f813607d700 1 -- 172.21.15.62:0/462226041 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563645fcee00 con 0x563645fb3880 2024-06-10T00:40:30.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f813607d700 1 --2- 172.21.15.62:0/462226041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb3880 0x563645fce840 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8128008e30 tx=0x7f8128013520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d20ec3644a1932 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f81197fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f81197fa700 1 -- 172.21.15.62:0/462226041 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812801b070 con 0x563645fb3880 2024-06-10T00:40:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f81197fa700 1 -- 172.21.15.62:0/462226041 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f8128013e10 con 0x563645fb3880 2024-06-10T00:40:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.408+0000 7f8142363b80 1 -- 172.21.15.62:0/462226041 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8124004b70 con 0x563645fb3880 2024-06-10T00:40:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.409+0000 7f81197fa700 1 -- 172.21.15.62:0/462226041 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8128025050 con 0x563645fb3880 2024-06-10T00:40:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.409+0000 7f8142363b80 10 client.16656 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.409+0000 7f8142363b80 1 -- 172.21.15.62:0/462226041 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563645fbe310 con 0x563645fb3880 2024-06-10T00:40:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.409+0000 7f81197fa700 1 -- 172.21.15.62:0/462226041 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8124004b70 con 0x563645fb3880 2024-06-10T00:40:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.409+0000 7f81197fa700 1 -- 172.21.15.62:0/462226041 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f81280043b0 con 0x563645fb3880 2024-06-10T00:40:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.409+0000 7f8142363b80 10 client.16656 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.409+0000 7f8142363b80 10 client.16656 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:40:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.409+0000 7f8142363b80 20 client.16656 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.409+0000 7f8142363b80 1 --2- 172.21.15.62:0/462226041 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x56364603a950 0x56364603cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.409+0000 7f8142363b80 4 client.16656 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:40:30.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.409+0000 7f8142363b80 1 -- 172.21.15.62:0/462226041 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563645f062d0 con 0x56364603a950 2024-06-10T00:40:30.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.409+0000 7f813587c700 1 --2- 172.21.15.62:0/462226041 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x56364603a950 0x56364603cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:30.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f81197fa700 10 client.16656 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:40:30.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f813587c700 1 --2- 172.21.15.62:0/462226041 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x56364603a950 0x56364603cd70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7a865184a68c655f server_cookie=51482d47ee0a48c9 in_seq=0 out_seq=0 2024-06-10T00:40:30.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f81197fa700 1 -- 172.21.15.62:0/462226041 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563645f062d0 con 0x56364603a950 2024-06-10T00:40:30.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f81197fa700 10 client.16656 handle_command_reply: tid=0 2024-06-10T00:40:30.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f8142363b80 1 client.16656 shutdown 2024-06-10T00:40:30.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f8142363b80 1 -- 172.21.15.62:0/462226041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb3880 msgr2=0x563645fce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:30.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f8142363b80 1 --2- 172.21.15.62:0/462226041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb3880 0x563645fce840 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8128008e30 tx=0x7f8128013520 comp rx=0 tx=0).stop 2024-06-10T00:40:30.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f8142363b80 1 -- 172.21.15.62:0/462226041 shutdown_connections 2024-06-10T00:40:30.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f8142363b80 1 --2- 172.21.15.62:0/462226041 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x56364603a950 0x56364603cd70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f8142363b80 1 --2- 172.21.15.62:0/462226041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563645fb3880 0x563645fce840 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f8142363b80 1 --2- 172.21.15.62:0/462226041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563645fad7a0 0x563645fbfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f8142363b80 1 --2- 172.21.15.62:0/462226041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563645fb2f60 0x563645fce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f81197fa700 0 client.16656 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:40:30.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f8142363b80 1 -- 172.21.15.62:0/462226041 >> 172.21.15.62:0/462226041 conn(0x563645eda080 msgr2=0x563645fb22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:30.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f8142363b80 1 -- 172.21.15.62:0/462226041 shutdown_connections 2024-06-10T00:40:30.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f8142363b80 1 -- 172.21.15.62:0/462226041 wait complete. 2024-06-10T00:40:30.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.410+0000 7f8142363b80 20 client.16656 trim_cache size 0 max 16384 2024-06-10T00:40:30.426 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:40:30.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.411+0000 7f8137ae0700 1 -- 172.21.15.62:0/2266172810 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f811c0a2d90 msgr2=0x7f811c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:30.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.411+0000 7f8137ae0700 1 --2- 172.21.15.62:0/2266172810 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f811c0a2d90 0x7f811c0a5250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8130102070 tx=0x7f812400b410 comp rx=0 tx=0).stop 2024-06-10T00:40:30.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.411+0000 7f8137ae0700 1 -- 172.21.15.62:0/2266172810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8130102770 msgr2=0x7f813019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:30.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.411+0000 7f8137ae0700 1 --2- 172.21.15.62:0/2266172810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8130102770 0x7f813019e820 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f812c006010 tx=0x7f812c009f80 comp rx=0 tx=0).stop 2024-06-10T00:40:30.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.411+0000 7f8137ae0700 1 -- 172.21.15.62:0/2266172810 shutdown_connections 2024-06-10T00:40:30.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.411+0000 7f8137ae0700 1 --2- 172.21.15.62:0/2266172810 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f811c0a2d90 0x7f811c0a5250 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.411+0000 7f8137ae0700 1 --2- 172.21.15.62:0/2266172810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8130102770 0x7f813019e820 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.411+0000 7f8137ae0700 1 --2- 172.21.15.62:0/2266172810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8130107860 0x7f81301989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.411+0000 7f8137ae0700 1 --2- 172.21.15.62:0/2266172810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8130103120 0x7f813019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:30.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.411+0000 7f8137ae0700 1 -- 172.21.15.62:0/2266172810 >> 172.21.15.62:0/2266172810 conn(0x7f8130079600 msgr2=0x7f8130100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:30.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.411+0000 7f8137ae0700 1 -- 172.21.15.62:0/2266172810 shutdown_connections 2024-06-10T00:40:30.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:30.411+0000 7f8137ae0700 1 -- 172.21.15.62:0/2266172810 wait complete. 2024-06-10T00:40:31.154 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T00:40:31.405 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:40:31.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:31 smithi105 ceph-mon[36321]: pgmap v235: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T00:40:31.845 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:31 smithi062 ceph-mon[32390]: pgmap v235: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T00:40:31.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:31 smithi138 ceph-mon[36549]: pgmap v235: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T00:40:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.136+0000 7fdf32ba1700 1 -- 172.21.15.62:0/4191577717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c1066f0 msgr2=0x7fdf2c10e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.136+0000 7fdf32ba1700 1 --2- 172.21.15.62:0/4191577717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c1066f0 0x7fdf2c10e2c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400b0d0 tx=0x7fdf2400b3e0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.137+0000 7fdf32ba1700 1 -- 172.21.15.62:0/4191577717 shutdown_connections 2024-06-10T00:40:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.137+0000 7fdf32ba1700 1 --2- 172.21.15.62:0/4191577717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c1066f0 0x7fdf2c10e2c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.137+0000 7fdf32ba1700 1 --2- 172.21.15.62:0/4191577717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf2c101610 0x7fdf2c1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.137+0000 7fdf32ba1700 1 --2- 172.21.15.62:0/4191577717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c101fc0 0x7fdf2c106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.137+0000 7fdf32ba1700 1 -- 172.21.15.62:0/4191577717 >> 172.21.15.62:0/4191577717 conn(0x7fdf2c078430 msgr2=0x7fdf2c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.137+0000 7fdf32ba1700 1 -- 172.21.15.62:0/4191577717 shutdown_connections 2024-06-10T00:40:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.137+0000 7fdf32ba1700 1 -- 172.21.15.62:0/4191577717 wait complete. 2024-06-10T00:40:33.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.137+0000 7fdf32ba1700 1 Processor -- start 2024-06-10T00:40:33.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.137+0000 7fdf32ba1700 1 -- start start 2024-06-10T00:40:33.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf32ba1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf2c101610 0x7fdf2c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf32ba1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c101fc0 0x7fdf2c19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf32ba1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c1066f0 0x7fdf2c197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf32ba1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c111700 con 0x7fdf2c1066f0 2024-06-10T00:40:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf32ba1700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c111580 con 0x7fdf2c101610 2024-06-10T00:40:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf32ba1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c111880 con 0x7fdf2c101fc0 2024-06-10T00:40:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf3113e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c1066f0 0x7fdf2c197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf3113e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c1066f0 0x7fdf2c197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34900/0 (socket says 172.21.15.62:34900) 2024-06-10T00:40:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf3113e700 1 -- 172.21.15.62:0/627157445 learned_addr learned my addr 172.21.15.62:0/627157445 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf3093d700 1 --2- 172.21.15.62:0/627157445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf2c101610 0x7fdf2c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf2bfff700 1 --2- 172.21.15.62:0/627157445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c101fc0 0x7fdf2c19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf3113e700 1 -- 172.21.15.62:0/627157445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c101fc0 msgr2=0x7fdf2c19dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:33.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf3113e700 1 --2- 172.21.15.62:0/627157445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c101fc0 0x7fdf2c19dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf3113e700 1 -- 172.21.15.62:0/627157445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf2c101610 msgr2=0x7fdf2c19d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:33.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf3113e700 1 --2- 172.21.15.62:0/627157445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf2c101610 0x7fdf2c19d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf3113e700 1 -- 172.21.15.62:0/627157445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c1980c0 con 0x7fdf2c1066f0 2024-06-10T00:40:33.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.138+0000 7fdf3113e700 1 --2- 172.21.15.62:0/627157445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c1066f0 0x7fdf2c197960 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fdf24006070 tx=0x7fdf24004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:33.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.139+0000 7fdf29ffb700 1 -- 172.21.15.62:0/627157445 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24021070 con 0x7fdf2c1066f0 2024-06-10T00:40:33.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.139+0000 7fdf29ffb700 1 -- 172.21.15.62:0/627157445 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf24007c80 con 0x7fdf2c1066f0 2024-06-10T00:40:33.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.139+0000 7fdf32ba1700 1 -- 172.21.15.62:0/627157445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2c198350 con 0x7fdf2c1066f0 2024-06-10T00:40:33.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.139+0000 7fdf29ffb700 1 -- 172.21.15.62:0/627157445 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf240053c0 con 0x7fdf2c1066f0 2024-06-10T00:40:33.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.139+0000 7fdf32ba1700 1 -- 172.21.15.62:0/627157445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2c06abc0 con 0x7fdf2c1066f0 2024-06-10T00:40:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.141+0000 7fdf29ffb700 1 -- 172.21.15.62:0/627157445 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdf24024070 con 0x7fdf2c1066f0 2024-06-10T00:40:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.142+0000 7fdf29ffb700 1 --2- 172.21.15.62:0/627157445 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fdf1c0a4fa0 0x7fdf1c0a7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.142+0000 7fdf3093d700 1 --2- 172.21.15.62:0/627157445 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fdf1c0a4fa0 0x7fdf1c0a7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.142+0000 7fdf29ffb700 1 -- 172.21.15.62:0/627157445 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fdf240d09d0 con 0x7fdf2c1066f0 2024-06-10T00:40:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.142+0000 7fdf3093d700 1 --2- 172.21.15.62:0/627157445 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fdf1c0a4fa0 0x7fdf1c0a7460 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005950 tx=0x7fdf200058e0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3d424b80 1 Processor -- start 2024-06-10T00:40:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3d424b80 1 -- start start 2024-06-10T00:40:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3d424b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80e850 0x558a8b80ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3d424b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80f170 0x558a8b813010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3d424b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b813550 0x558a8b813990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3d424b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558a8b6fcf60 con 0x558a8b80e850 2024-06-10T00:40:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3d424b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558a8b75b850 con 0x558a8b80f170 2024-06-10T00:40:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3d424b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a8b6c3460 con 0x558a8b813550 2024-06-10T00:40:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3093d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80e850 0x558a8b80ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3093d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80e850 0x558a8b80ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34910/0 (socket says 172.21.15.62:34910) 2024-06-10T00:40:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3093d700 1 -- 172.21.15.62:0/1602817903 learned_addr learned my addr 172.21.15.62:0/1602817903 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3113e700 1 --2- 172.21.15.62:0/1602817903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b813550 0x558a8b813990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf2bfff700 1 --2- 172.21.15.62:0/1602817903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80f170 0x558a8b813010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3093d700 1 -- 172.21.15.62:0/1602817903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b813550 msgr2=0x558a8b813990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3093d700 1 --2- 172.21.15.62:0/1602817903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b813550 0x558a8b813990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3093d700 1 -- 172.21.15.62:0/1602817903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80f170 msgr2=0x558a8b813010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3093d700 1 --2- 172.21.15.62:0/1602817903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80f170 0x558a8b813010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3093d700 1 -- 172.21.15.62:0/1602817903 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a8b6aa8d0 con 0x558a8b80e850 2024-06-10T00:40:33.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf3113e700 1 --2- 172.21.15.62:0/1602817903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b813550 0x558a8b813990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:40:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.143+0000 7fdf2bfff700 1 --2- 172.21.15.62:0/1602817903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80f170 0x558a8b813010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:40:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.144+0000 7fdf3093d700 1 --2- 172.21.15.62:0/1602817903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80e850 0x558a8b80ec30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000fd40 tx=0x7fdf20012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385ffc6f69ecc3db server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.144+0000 7fdf16ffd700 1 -- 172.21.15.62:0/1602817903 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20022070 con 0x558a8b80e850 2024-06-10T00:40:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.144+0000 7fdf16ffd700 1 -- 172.21.15.62:0/1602817903 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf20011c60 con 0x558a8b80e850 2024-06-10T00:40:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.144+0000 7fdf16ffd700 1 -- 172.21.15.62:0/1602817903 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2001d6a0 con 0x558a8b80e850 2024-06-10T00:40:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.145+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1602817903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80e850 msgr2=0x558a8b80ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.145+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1602817903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80e850 0x558a8b80ec30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000fd40 tx=0x7fdf20012000 comp rx=0 tx=0).stop 2024-06-10T00:40:33.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.145+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1602817903 shutdown_connections 2024-06-10T00:40:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.145+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1602817903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80e850 0x558a8b80ec30 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.145+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1602817903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b813550 0x558a8b813990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.145+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1602817903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80f170 0x558a8b813010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.145+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1602817903 >> 172.21.15.62:0/1602817903 conn(0x558a8b737080 msgr2=0x558a8b805220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.145+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1602817903 shutdown_connections 2024-06-10T00:40:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.145+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1602817903 wait complete. 2024-06-10T00:40:33.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.145+0000 7fdf3d424b80 1 -- start start 2024-06-10T00:40:33.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.145+0000 7fdf3d424b80 1 Processor -- start 2024-06-10T00:40:33.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3d424b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80e850 0x558a8b81aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3d424b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80f170 0x558a8b817d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3d424b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b813550 0x558a8b818250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3d424b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558a8b6fbfc0 con 0x558a8b80e850 2024-06-10T00:40:33.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3d424b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558a8b757810 con 0x558a8b80f170 2024-06-10T00:40:33.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3d424b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a8b81b680 con 0x558a8b813550 2024-06-10T00:40:33.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3d424b80 10 client.0 _ll_register_callbacks cb 0x558a8b6f15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3093d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80e850 0x558a8b81aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3093d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80e850 0x558a8b81aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34916/0 (socket says 172.21.15.62:34916) 2024-06-10T00:40:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3093d700 1 -- 172.21.15.62:0/1465019105 learned_addr learned my addr 172.21.15.62:0/1465019105 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf2bfff700 1 --2- 172.21.15.62:0/1465019105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80f170 0x558a8b817d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3113e700 1 --2- 172.21.15.62:0/1465019105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b813550 0x558a8b818250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3093d700 1 -- 172.21.15.62:0/1465019105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b813550 msgr2=0x558a8b818250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3093d700 1 --2- 172.21.15.62:0/1465019105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b813550 0x558a8b818250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3093d700 1 -- 172.21.15.62:0/1465019105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80f170 msgr2=0x558a8b817d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3093d700 1 --2- 172.21.15.62:0/1465019105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80f170 0x558a8b817d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3093d700 1 -- 172.21.15.62:0/1465019105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a8b721750 con 0x558a8b80e850 2024-06-10T00:40:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf2bfff700 1 --2- 172.21.15.62:0/1465019105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80f170 0x558a8b817d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:40:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf3093d700 1 --2- 172.21.15.62:0/1465019105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80e850 0x558a8b81aa10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdf20010010 tx=0x7fdf200050f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf008f591f98cfb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.146+0000 7fdf0bfff700 1 -- 172.21.15.62:0/1465019105 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20036060 con 0x558a8b80e850 2024-06-10T00:40:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.147+0000 7fdf0bfff700 1 -- 172.21.15.62:0/1465019105 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fdf20005610 con 0x558a8b80e850 2024-06-10T00:40:33.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.147+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1465019105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf20011c60 con 0x558a8b80e850 2024-06-10T00:40:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.147+0000 7fdf0bfff700 1 -- 172.21.15.62:0/1465019105 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdf2003d050 con 0x558a8b80e850 2024-06-10T00:40:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.147+0000 7fdf3d424b80 10 client.16668 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.147+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1465019105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558a8b825d00 con 0x558a8b80e850 2024-06-10T00:40:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.147+0000 7fdf0bfff700 1 -- 172.21.15.62:0/1465019105 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf20011c60 con 0x558a8b80e850 2024-06-10T00:40:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.147+0000 7fdf0bfff700 1 -- 172.21.15.62:0/1465019105 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fdf2001e930 con 0x558a8b80e850 2024-06-10T00:40:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.147+0000 7fdf3d424b80 10 client.16668 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.147+0000 7fdf3d424b80 10 client.16668 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:40:33.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.147+0000 7fdf3d424b80 20 client.16668 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:33.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.147+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1465019105 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558a8b828280 0x558a8b82a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.147+0000 7fdf3d424b80 4 client.16668 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:40:33.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.147+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1465019105 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a8b768d50 con 0x558a8b828280 2024-06-10T00:40:33.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.148+0000 7fdf2bfff700 1 --2- 172.21.15.62:0/1465019105 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558a8b828280 0x558a8b82a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.148+0000 7fdf2bfff700 1 --2- 172.21.15.62:0/1465019105 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558a8b828280 0x558a8b82a670 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=77466025fd8cff93 server_cookie=bf86ed28685f18db in_seq=0 out_seq=0 2024-06-10T00:40:33.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.148+0000 7fdf0bfff700 10 client.16668 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:40:33.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf0bfff700 1 -- 172.21.15.62:0/1465019105 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558a8b768d50 con 0x558a8b828280 2024-06-10T00:40:33.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf0bfff700 10 client.16668 handle_command_reply: tid=0 2024-06-10T00:40:33.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf3d424b80 1 client.16668 shutdown 2024-06-10T00:40:33.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1465019105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80e850 msgr2=0x558a8b81aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:33.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1465019105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80e850 0x558a8b81aa10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdf20010010 tx=0x7fdf200050f0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1465019105 shutdown_connections 2024-06-10T00:40:33.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1465019105 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558a8b828280 0x558a8b82a670 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1465019105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80e850 0x558a8b81aa10 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf0bfff700 0 client.16668 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:40:33.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1465019105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b813550 0x558a8b818250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1465019105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80f170 0x558a8b817d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1465019105 >> 172.21.15.62:0/1465019105 conn(0x558a8b737080 msgr2=0x558a8b814a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:33.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1465019105 shutdown_connections 2024-06-10T00:40:33.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1465019105 wait complete. 2024-06-10T00:40:33.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.149+0000 7fdf3d424b80 20 client.16668 trim_cache size 0 max 16384 2024-06-10T00:40:33.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.164+0000 7fdf3d424b80 1 Processor -- start 2024-06-10T00:40:33.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf3d424b80 1 -- start start 2024-06-10T00:40:33.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf3d424b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b80e850 0x558a8b814940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf3d424b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80f170 0x558a8b819c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf3d424b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b813550 0x558a8b8122c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf3d424b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1ac4c0 con 0x558a8b80f170 2024-06-10T00:40:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf3d424b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1ae000 con 0x558a8b813550 2024-06-10T00:40:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf3d424b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1acb90 con 0x558a8b80e850 2024-06-10T00:40:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf2bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80f170 0x558a8b819c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf2bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80f170 0x558a8b819c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34924/0 (socket says 172.21.15.62:34924) 2024-06-10T00:40:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf2bfff700 1 -- 172.21.15.62:0/3411467291 learned_addr learned my addr 172.21.15.62:0/3411467291 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf3113e700 1 --2- 172.21.15.62:0/3411467291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b813550 0x558a8b8122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf3093d700 1 --2- 172.21.15.62:0/3411467291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b80e850 0x558a8b814940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf2bfff700 1 -- 172.21.15.62:0/3411467291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b80e850 msgr2=0x558a8b814940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf2bfff700 1 --2- 172.21.15.62:0/3411467291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b80e850 0x558a8b814940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf2bfff700 1 -- 172.21.15.62:0/3411467291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b813550 msgr2=0x558a8b8122c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf2bfff700 1 --2- 172.21.15.62:0/3411467291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b813550 0x558a8b8122c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf2bfff700 1 -- 172.21.15.62:0/3411467291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a8b8056a0 con 0x558a8b80f170 2024-06-10T00:40:33.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf3093d700 1 --2- 172.21.15.62:0/3411467291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b80e850 0x558a8b814940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:40:33.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.165+0000 7fdf3113e700 1 --2- 172.21.15.62:0/3411467291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b813550 0x558a8b8122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:40:33.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.166+0000 7fdf2bfff700 1 --2- 172.21.15.62:0/3411467291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80f170 0x558a8b819c50 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c06aa20 tx=0x7fdf1800c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5699d1bf3eccff49 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:33.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.166+0000 7fdf167fc700 1 -- 172.21.15.62:0/3411467291 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1801e020 con 0x558a8b80f170 2024-06-10T00:40:33.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.166+0000 7fdf167fc700 1 -- 172.21.15.62:0/3411467291 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf18005510 con 0x558a8b80f170 2024-06-10T00:40:33.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.166+0000 7fdf167fc700 1 -- 172.21.15.62:0/3411467291 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1801a520 con 0x558a8b80f170 2024-06-10T00:40:33.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.167+0000 7fdf3d424b80 1 -- 172.21.15.62:0/3411467291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80f170 msgr2=0x558a8b819c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:33.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.167+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/3411467291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80f170 0x558a8b819c50 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c06aa20 tx=0x7fdf1800c380 comp rx=0 tx=0).stop 2024-06-10T00:40:33.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.167+0000 7fdf3d424b80 1 -- 172.21.15.62:0/3411467291 shutdown_connections 2024-06-10T00:40:33.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.167+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/3411467291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b813550 0x558a8b8122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.167+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/3411467291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b80e850 0x558a8b814940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.167+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/3411467291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b80f170 0x558a8b819c50 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.167+0000 7fdf3d424b80 1 -- 172.21.15.62:0/3411467291 >> 172.21.15.62:0/3411467291 conn(0x558a8b737080 msgr2=0x558a8b819540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:33.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.167+0000 7fdf3d424b80 1 -- 172.21.15.62:0/3411467291 shutdown_connections 2024-06-10T00:40:33.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.167+0000 7fdf3d424b80 1 -- 172.21.15.62:0/3411467291 wait complete. 2024-06-10T00:40:33.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.167+0000 7fdf3d424b80 1 -- start start 2024-06-10T00:40:33.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.167+0000 7fdf3d424b80 1 Processor -- start 2024-06-10T00:40:33.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3d424b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80e850 0x558a8b820c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3d424b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b80f170 0x558a8b821170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3d424b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b813550 0x558a8b82f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3d424b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1ad930 con 0x558a8b813550 2024-06-10T00:40:33.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3d424b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1ad260 con 0x558a8b80e850 2024-06-10T00:40:33.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3d424b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1aa480 con 0x558a8b80f170 2024-06-10T00:40:33.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3d424b80 10 client.0 _ll_register_callbacks cb 0x558a8b6f15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:33.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3113e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b813550 0x558a8b82f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf2bfff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b80f170 0x558a8b821170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3113e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b813550 0x558a8b82f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34934/0 (socket says 172.21.15.62:34934) 2024-06-10T00:40:33.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3113e700 1 -- 172.21.15.62:0/1136082417 learned_addr learned my addr 172.21.15.62:0/1136082417 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:33.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3093d700 1 --2- 172.21.15.62:0/1136082417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80e850 0x558a8b820c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3113e700 1 -- 172.21.15.62:0/1136082417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b80f170 msgr2=0x558a8b821170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:33.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3113e700 1 --2- 172.21.15.62:0/1136082417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b80f170 0x558a8b821170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3113e700 1 -- 172.21.15.62:0/1136082417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80e850 msgr2=0x558a8b820c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:33.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3113e700 1 --2- 172.21.15.62:0/1136082417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80e850 0x558a8b820c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3113e700 1 -- 172.21.15.62:0/1136082417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a8b82faf0 con 0x558a8b813550 2024-06-10T00:40:33.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.168+0000 7fdf3093d700 1 --2- 172.21.15.62:0/1136082417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80e850 0x558a8b820c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:40:33.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf3113e700 1 --2- 172.21.15.62:0/1136082417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b813550 0x558a8b82f320 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400d010 tx=0x7fdf2409c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbfa51e60e6c2a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:33.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:33.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf0bfff700 1 -- 172.21.15.62:0/1136082417 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2402b040 con 0x558a8b813550 2024-06-10T00:40:33.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1136082417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf18005510 con 0x558a8b813550 2024-06-10T00:40:33.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf0bfff700 1 -- 172.21.15.62:0/1136082417 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7fdf2409cdd0 con 0x558a8b813550 2024-06-10T00:40:33.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf0bfff700 1 -- 172.21.15.62:0/1136082417 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdf2409cb70 con 0x558a8b813550 2024-06-10T00:40:33.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf3d424b80 10 client.16674 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:33.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1136082417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558a8b8205a0 con 0x558a8b813550 2024-06-10T00:40:33.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf0bfff700 1 -- 172.21.15.62:0/1136082417 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf18005510 con 0x558a8b813550 2024-06-10T00:40:33.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf0bfff700 1 -- 172.21.15.62:0/1136082417 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fdf240b5ce0 con 0x558a8b813550 2024-06-10T00:40:33.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf3d424b80 10 client.16674 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:33.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf3d424b80 10 client.16674 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:40:33.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf3d424b80 20 client.16674 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:33.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1136082417 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558a8b828280 0x558a8b895020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:33.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf3d424b80 4 client.16674 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:40:33.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.169+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1136082417 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a8b7632d0 con 0x558a8b828280 2024-06-10T00:40:33.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.170+0000 7fdf3093d700 1 --2- 172.21.15.62:0/1136082417 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558a8b828280 0x558a8b895020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:33.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.170+0000 7fdf3093d700 1 --2- 172.21.15.62:0/1136082417 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558a8b828280 0x558a8b895020 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=708cd19417f50c79 server_cookie=c407823881d32fb in_seq=0 out_seq=0 2024-06-10T00:40:33.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.170+0000 7fdf0bfff700 10 client.16674 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:40:33.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.170+0000 7fdf0bfff700 1 -- 172.21.15.62:0/1136082417 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a8b7632d0 con 0x558a8b828280 2024-06-10T00:40:33.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.170+0000 7fdf0bfff700 10 client.16674 handle_command_reply: tid=0 2024-06-10T00:40:33.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.170+0000 7fdf3d424b80 1 client.16674 shutdown 2024-06-10T00:40:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.171+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1136082417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b813550 msgr2=0x558a8b82f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.171+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1136082417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b813550 0x558a8b82f320 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400d010 tx=0x7fdf2409c730 comp rx=0 tx=0).stop 2024-06-10T00:40:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.171+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1136082417 shutdown_connections 2024-06-10T00:40:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.171+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1136082417 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558a8b828280 0x558a8b895020 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.171+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1136082417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a8b813550 0x558a8b82f320 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.171+0000 7fdf0bfff700 0 client.16674 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:40:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.171+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1136082417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8b80e850 0x558a8b820c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.171+0000 7fdf3d424b80 1 --2- 172.21.15.62:0/1136082417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a8b80f170 0x558a8b821170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.171+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1136082417 >> 172.21.15.62:0/1136082417 conn(0x558a8b737080 msgr2=0x558a8b819540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:33.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.171+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1136082417 shutdown_connections 2024-06-10T00:40:33.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.171+0000 7fdf3d424b80 1 -- 172.21.15.62:0/1136082417 wait complete. 2024-06-10T00:40:33.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.171+0000 7fdf3d424b80 20 client.16674 trim_cache size 0 max 16384 2024-06-10T00:40:33.180 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:40:33.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.172+0000 7fdf32ba1700 1 -- 172.21.15.62:0/627157445 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fdf1c0a4fa0 msgr2=0x7fdf1c0a7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:33.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.172+0000 7fdf32ba1700 1 --2- 172.21.15.62:0/627157445 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fdf1c0a4fa0 0x7fdf1c0a7460 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005950 tx=0x7fdf200058e0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.172+0000 7fdf32ba1700 1 -- 172.21.15.62:0/627157445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c1066f0 msgr2=0x7fdf2c197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:33.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.172+0000 7fdf32ba1700 1 --2- 172.21.15.62:0/627157445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c1066f0 0x7fdf2c197960 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fdf24006070 tx=0x7fdf24004db0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.172+0000 7fdf32ba1700 1 -- 172.21.15.62:0/627157445 shutdown_connections 2024-06-10T00:40:33.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.172+0000 7fdf32ba1700 1 --2- 172.21.15.62:0/627157445 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fdf1c0a4fa0 0x7fdf1c0a7460 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.172+0000 7fdf32ba1700 1 --2- 172.21.15.62:0/627157445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf2c1066f0 0x7fdf2c197960 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.172+0000 7fdf32ba1700 1 --2- 172.21.15.62:0/627157445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf2c101610 0x7fdf2c19d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.172+0000 7fdf32ba1700 1 --2- 172.21.15.62:0/627157445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c101fc0 0x7fdf2c19dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:33.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.172+0000 7fdf32ba1700 1 -- 172.21.15.62:0/627157445 >> 172.21.15.62:0/627157445 conn(0x7fdf2c078430 msgr2=0x7fdf2c0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:33.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.172+0000 7fdf32ba1700 1 -- 172.21.15.62:0/627157445 shutdown_connections 2024-06-10T00:40:33.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:33.172+0000 7fdf32ba1700 1 -- 172.21.15.62:0/627157445 wait complete. 2024-06-10T00:40:33.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:33 smithi105 ceph-mon[36321]: pgmap v236: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T00:40:33.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:33 smithi062 ceph-mon[32390]: pgmap v236: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T00:40:33.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:33 smithi138 ceph-mon[36549]: pgmap v236: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-10T00:40:33.973 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T00:40:34.245 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:40:35.683 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:35 smithi062 ceph-mon[32390]: pgmap v237: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T00:40:35.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:35 smithi105 ceph-mon[36321]: pgmap v237: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T00:40:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:35 smithi138 ceph-mon[36549]: pgmap v237: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-10T00:40:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.914+0000 7f3ce47ab700 1 -- 172.21.15.62:0/853825492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cdc108510 msgr2=0x7f3cdc1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.914+0000 7f3ce47ab700 1 --2- 172.21.15.62:0/853825492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cdc108510 0x7f3cdc1088f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00b0a0 tx=0x7f3ccc00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.915+0000 7f3ce47ab700 1 -- 172.21.15.62:0/853825492 shutdown_connections 2024-06-10T00:40:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.915+0000 7f3ce47ab700 1 --2- 172.21.15.62:0/853825492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cdc108510 0x7f3cdc1088f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.915+0000 7f3ce47ab700 1 --2- 172.21.15.62:0/853825492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cdc101440 0x7f3cdc10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.915+0000 7f3ce47ab700 1 --2- 172.21.15.62:0/853825492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cdc100a80 0x7f3cdc100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.915+0000 7f3ce47ab700 1 -- 172.21.15.62:0/853825492 >> 172.21.15.62:0/853825492 conn(0x7f3cdc077a00 msgr2=0x7f3cdc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.915+0000 7f3ce47ab700 1 -- 172.21.15.62:0/853825492 shutdown_connections 2024-06-10T00:40:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.915+0000 7f3ce47ab700 1 -- 172.21.15.62:0/853825492 wait complete. 2024-06-10T00:40:35.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.915+0000 7f3ce47ab700 1 Processor -- start 2024-06-10T00:40:35.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.915+0000 7f3ce47ab700 1 -- start start 2024-06-10T00:40:35.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce47ab700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cdc100a80 0x7f3cdc101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce47ab700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cdc101440 0x7f3cdc1021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce47ab700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cdc108510 0x7f3cdc1026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce47ab700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1127f0 con 0x7f3cdc108510 2024-06-10T00:40:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce47ab700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc112670 con 0x7f3cdc101440 2024-06-10T00:40:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce47ab700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc112970 con 0x7f3cdc100a80 2024-06-10T00:40:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce2d48700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cdc108510 0x7f3cdc1026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce1d46700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cdc101440 0x7f3cdc1021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce1d46700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cdc101440 0x7f3cdc1021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:60092/0 (socket says 172.21.15.62:60092) 2024-06-10T00:40:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce1d46700 1 -- 172.21.15.62:0/313929622 learned_addr learned my addr 172.21.15.62:0/313929622 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce2d48700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cdc108510 0x7f3cdc1026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34962/0 (socket says 172.21.15.62:34962) 2024-06-10T00:40:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce2547700 1 --2- 172.21.15.62:0/313929622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cdc100a80 0x7f3cdc101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce2d48700 1 -- 172.21.15.62:0/313929622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cdc100a80 msgr2=0x7f3cdc101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce2d48700 1 --2- 172.21.15.62:0/313929622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cdc100a80 0x7f3cdc101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce2d48700 1 -- 172.21.15.62:0/313929622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cdc101440 msgr2=0x7f3cdc1021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce2d48700 1 --2- 172.21.15.62:0/313929622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cdc101440 0x7f3cdc1021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce2d48700 1 -- 172.21.15.62:0/313929622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cdc1a7170 con 0x7f3cdc108510 2024-06-10T00:40:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.916+0000 7f3ce2d48700 1 --2- 172.21.15.62:0/313929622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cdc108510 0x7f3cdc1026e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400b3f0 tx=0x7f3cd400b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.917+0000 7f3cdb7fe700 1 -- 172.21.15.62:0/313929622 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4015020 con 0x7f3cdc108510 2024-06-10T00:40:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.917+0000 7f3cdb7fe700 1 -- 172.21.15.62:0/313929622 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd4011b10 con 0x7f3cdc108510 2024-06-10T00:40:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.917+0000 7f3cdb7fe700 1 -- 172.21.15.62:0/313929622 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4010c50 con 0x7f3cdc108510 2024-06-10T00:40:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.917+0000 7f3ce47ab700 1 -- 172.21.15.62:0/313929622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cdc1a7460 con 0x7f3cdc108510 2024-06-10T00:40:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.917+0000 7f3ce47ab700 1 -- 172.21.15.62:0/313929622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cdc1a7950 con 0x7f3cdc108510 2024-06-10T00:40:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.919+0000 7f3cdb7fe700 1 -- 172.21.15.62:0/313929622 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3cd4018020 con 0x7f3cdc108510 2024-06-10T00:40:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.920+0000 7f3cdb7fe700 1 --2- 172.21.15.62:0/313929622 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3cc40a2e10 0x7f3cc40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.920+0000 7f3ce2547700 1 --2- 172.21.15.62:0/313929622 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3cc40a2e10 0x7f3cc40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.920+0000 7f3cdb7fe700 1 -- 172.21.15.62:0/313929622 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f3cd40cecb0 con 0x7f3cdc108510 2024-06-10T00:40:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.920+0000 7f3ce2547700 1 --2- 172.21.15.62:0/313929622 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3cc40a2e10 0x7f3cc40a52d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009e60 tx=0x7f3ccc00b7b0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3cef02eb80 1 Processor -- start 2024-06-10T00:40:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3cef02eb80 1 -- start start 2024-06-10T00:40:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3cef02eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 0x556416665340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3cef02eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556416665880 0x55641665f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3cef02eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55641665f7a0 0x55641665fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3cef02eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556416551f60 con 0x55641665f7a0 2024-06-10T00:40:35.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3cef02eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5564165b0850 con 0x556416665880 2024-06-10T00:40:35.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3cef02eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556416518460 con 0x556416664f60 2024-06-10T00:40:35.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3ce2d48700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55641665f7a0 0x55641665fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3ce2d48700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55641665f7a0 0x55641665fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34974/0 (socket says 172.21.15.62:34974) 2024-06-10T00:40:35.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3ce2d48700 1 -- 172.21.15.62:0/1324391475 learned_addr learned my addr 172.21.15.62:0/1324391475 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:35.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3ce2547700 1 --2- 172.21.15.62:0/1324391475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 0x556416665340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3ce1d46700 1 --2- 172.21.15.62:0/1324391475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556416665880 0x55641665f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3ce2d48700 1 -- 172.21.15.62:0/1324391475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 msgr2=0x556416665340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:35.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3ce2d48700 1 --2- 172.21.15.62:0/1324391475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 0x556416665340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.921+0000 7f3ce2d48700 1 -- 172.21.15.62:0/1324391475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556416665880 msgr2=0x55641665f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:35.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.922+0000 7f3ce2d48700 1 --2- 172.21.15.62:0/1324391475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556416665880 0x55641665f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.922+0000 7f3ce2d48700 1 -- 172.21.15.62:0/1324391475 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564164ff8d0 con 0x55641665f7a0 2024-06-10T00:40:35.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.922+0000 7f3ce2547700 1 --2- 172.21.15.62:0/1324391475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 0x556416665340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:40:35.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.922+0000 7f3ce1d46700 1 --2- 172.21.15.62:0/1324391475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556416665880 0x55641665f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:40:35.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.922+0000 7f3ce2d48700 1 --2- 172.21.15.62:0/1324391475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55641665f7a0 0x55641665fbc0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3cd409ae60 tx=0x7f3cd40106f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b419f5b1d83de162 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:35.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.922+0000 7f3cd8ff9700 1 -- 172.21.15.62:0/1324391475 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4010380 con 0x55641665f7a0 2024-06-10T00:40:35.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.922+0000 7f3cd8ff9700 1 -- 172.21.15.62:0/1324391475 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd40991d0 con 0x55641665f7a0 2024-06-10T00:40:35.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.922+0000 7f3cd8ff9700 1 -- 172.21.15.62:0/1324391475 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd409b390 con 0x55641665f7a0 2024-06-10T00:40:35.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.923+0000 7f3cef02eb80 1 -- 172.21.15.62:0/1324391475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55641665f7a0 msgr2=0x55641665fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:35.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.923+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/1324391475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55641665f7a0 0x55641665fbc0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3cd409ae60 tx=0x7f3cd40106f0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.923+0000 7f3cef02eb80 1 -- 172.21.15.62:0/1324391475 shutdown_connections 2024-06-10T00:40:35.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.923+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/1324391475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55641665f7a0 0x55641665fbc0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.923+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/1324391475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 0x556416665340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.923+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/1324391475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556416665880 0x55641665f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.923+0000 7f3cef02eb80 1 -- 172.21.15.62:0/1324391475 >> 172.21.15.62:0/1324391475 conn(0x55641658c080 msgr2=0x55641665a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:35.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.923+0000 7f3cef02eb80 1 -- 172.21.15.62:0/1324391475 shutdown_connections 2024-06-10T00:40:35.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.923+0000 7f3cef02eb80 1 -- 172.21.15.62:0/1324391475 wait complete. 2024-06-10T00:40:35.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.923+0000 7f3cef02eb80 1 -- start start 2024-06-10T00:40:35.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.923+0000 7f3cef02eb80 1 Processor -- start 2024-06-10T00:40:35.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3cef02eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55641665f7a0 0x55641666b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3cef02eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 0x5564166689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3cef02eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556416665880 0x556416668ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3cef02eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556416550fc0 con 0x556416665880 2024-06-10T00:40:35.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3cef02eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5564165ac810 con 0x55641665f7a0 2024-06-10T00:40:35.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3cef02eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55641666c260 con 0x556416664f60 2024-06-10T00:40:35.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3cef02eb80 10 client.0 _ll_register_callbacks cb 0x5564165465b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:35.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3ce2d48700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556416665880 0x556416668ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3ce2d48700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556416665880 0x556416668ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34980/0 (socket says 172.21.15.62:34980) 2024-06-10T00:40:35.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3ce2d48700 1 -- 172.21.15.62:0/3561832092 learned_addr learned my addr 172.21.15.62:0/3561832092 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:35.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3ce2547700 1 --2- 172.21.15.62:0/3561832092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55641665f7a0 0x55641666b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3ce1d46700 1 --2- 172.21.15.62:0/3561832092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 0x5564166689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3ce2d48700 1 -- 172.21.15.62:0/3561832092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 msgr2=0x5564166689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:35.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3ce2d48700 1 --2- 172.21.15.62:0/3561832092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 0x5564166689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3ce2d48700 1 -- 172.21.15.62:0/3561832092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55641665f7a0 msgr2=0x55641666b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:35.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3ce2d48700 1 --2- 172.21.15.62:0/3561832092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55641665f7a0 0x55641666b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3ce2d48700 1 -- 172.21.15.62:0/3561832092 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556416576750 con 0x556416665880 2024-06-10T00:40:35.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3ce2d48700 1 --2- 172.21.15.62:0/3561832092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556416665880 0x556416668ee0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3cd409ae10 tx=0x7f3cd4099380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213844fc65918a37 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:35.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3cc1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:35.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3cc1ffb700 1 -- 172.21.15.62:0/3561832092 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd40b6070 con 0x556416665880 2024-06-10T00:40:35.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3cc1ffb700 1 -- 172.21.15.62:0/3561832092 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f3cd409b130 con 0x556416665880 2024-06-10T00:40:35.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.924+0000 7f3cef02eb80 1 -- 172.21.15.62:0/3561832092 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cd40991d0 con 0x556416665880 2024-06-10T00:40:35.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.925+0000 7f3cc1ffb700 1 -- 172.21.15.62:0/3561832092 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3cd4010440 con 0x556416665880 2024-06-10T00:40:35.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.925+0000 7f3cef02eb80 10 client.16686 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:35.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.925+0000 7f3cef02eb80 1 -- 172.21.15.62:0/3561832092 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556416676aa0 con 0x556416665880 2024-06-10T00:40:35.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.925+0000 7f3cc1ffb700 1 -- 172.21.15.62:0/3561832092 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd40991d0 con 0x556416665880 2024-06-10T00:40:35.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.925+0000 7f3cc1ffb700 1 -- 172.21.15.62:0/3561832092 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f3cd409a320 con 0x556416665880 2024-06-10T00:40:35.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.925+0000 7f3cef02eb80 10 client.16686 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:35.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.925+0000 7f3cef02eb80 10 client.16686 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:40:35.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.925+0000 7f3cef02eb80 20 client.16686 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:35.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.925+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/3561832092 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556416678f70 0x55641667b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.925+0000 7f3cef02eb80 4 client.16686 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:40:35.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.925+0000 7f3cef02eb80 1 -- 172.21.15.62:0/3561832092 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564165bdd50 con 0x556416678f70 2024-06-10T00:40:35.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.925+0000 7f3ce2547700 1 --2- 172.21.15.62:0/3561832092 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556416678f70 0x55641667b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.926+0000 7f3ce2547700 1 --2- 172.21.15.62:0/3561832092 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556416678f70 0x55641667b360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e57f75ba901a350 server_cookie=e954508a9799e902 in_seq=0 out_seq=0 2024-06-10T00:40:35.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.926+0000 7f3cc1ffb700 10 client.16686 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:40:35.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.926+0000 7f3cc1ffb700 1 -- 172.21.15.62:0/3561832092 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5564165bdd50 con 0x556416678f70 2024-06-10T00:40:35.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.926+0000 7f3cc1ffb700 10 client.16686 handle_command_reply: tid=0 2024-06-10T00:40:35.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.927+0000 7f3cef02eb80 1 client.16686 shutdown 2024-06-10T00:40:35.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.927+0000 7f3cef02eb80 1 -- 172.21.15.62:0/3561832092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556416665880 msgr2=0x556416668ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:35.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.927+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/3561832092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556416665880 0x556416668ee0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3cd409ae10 tx=0x7f3cd4099380 comp rx=0 tx=0).stop 2024-06-10T00:40:35.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.927+0000 7f3cef02eb80 1 -- 172.21.15.62:0/3561832092 shutdown_connections 2024-06-10T00:40:35.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.927+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/3561832092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556416665880 0x556416668ee0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.927+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/3561832092 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556416678f70 0x55641667b360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.927+0000 7f3cc1ffb700 0 client.16686 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:40:35.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.927+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/3561832092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55641665f7a0 0x55641666b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.927+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/3561832092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 0x5564166689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.927+0000 7f3cef02eb80 1 -- 172.21.15.62:0/3561832092 >> 172.21.15.62:0/3561832092 conn(0x55641658c080 msgr2=0x556416659700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:35.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.927+0000 7f3cef02eb80 1 -- 172.21.15.62:0/3561832092 shutdown_connections 2024-06-10T00:40:35.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.927+0000 7f3cef02eb80 1 -- 172.21.15.62:0/3561832092 wait complete. 2024-06-10T00:40:35.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.927+0000 7f3cef02eb80 20 client.16686 trim_cache size 0 max 16384 2024-06-10T00:40:35.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.942+0000 7f3cef02eb80 1 Processor -- start 2024-06-10T00:40:35.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.942+0000 7f3cef02eb80 1 -- start start 2024-06-10T00:40:35.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.942+0000 7f3cef02eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55641665f7a0 0x55641665f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3cef02eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556416664f60 0x5564166665d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3cef02eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416665880 0x5564166641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3cef02eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1b09a0 con 0x556416664f60 2024-06-10T00:40:35.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3cef02eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1b24e0 con 0x55641665f7a0 2024-06-10T00:40:35.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3cef02eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1b1070 con 0x556416665880 2024-06-10T00:40:35.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3ce1d46700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556416664f60 0x5564166665d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3ce1d46700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556416664f60 0x5564166665d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34994/0 (socket says 172.21.15.62:34994) 2024-06-10T00:40:35.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3ce1d46700 1 -- 172.21.15.62:0/2032888133 learned_addr learned my addr 172.21.15.62:0/2032888133 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:35.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3ce2547700 1 --2- 172.21.15.62:0/2032888133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55641665f7a0 0x55641665f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3ce2d48700 1 --2- 172.21.15.62:0/2032888133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416665880 0x5564166641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3ce1d46700 1 -- 172.21.15.62:0/2032888133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416665880 msgr2=0x5564166641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:35.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3ce1d46700 1 --2- 172.21.15.62:0/2032888133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416665880 0x5564166641e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3ce1d46700 1 -- 172.21.15.62:0/2032888133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55641665f7a0 msgr2=0x55641665f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:35.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3ce1d46700 1 --2- 172.21.15.62:0/2032888133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55641665f7a0 0x55641665f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3ce1d46700 1 -- 172.21.15.62:0/2032888133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55641665b040 con 0x556416664f60 2024-06-10T00:40:35.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3ce1d46700 1 --2- 172.21.15.62:0/2032888133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556416664f60 0x5564166665d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000be20 tx=0x7f3cd0012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538ba7951d210c5d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:35.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.943+0000 7f3cc3fff700 1 -- 172.21.15.62:0/2032888133 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0012bb0 con 0x556416664f60 2024-06-10T00:40:35.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.944+0000 7f3cc3fff700 1 -- 172.21.15.62:0/2032888133 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd0004500 con 0x556416664f60 2024-06-10T00:40:35.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.944+0000 7f3cc3fff700 1 -- 172.21.15.62:0/2032888133 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0010430 con 0x556416664f60 2024-06-10T00:40:35.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.944+0000 7f3cef02eb80 1 -- 172.21.15.62:0/2032888133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556416664f60 msgr2=0x5564166665d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:35.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.944+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/2032888133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556416664f60 0x5564166665d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000be20 tx=0x7f3cd0012730 comp rx=0 tx=0).stop 2024-06-10T00:40:35.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.944+0000 7f3cef02eb80 1 -- 172.21.15.62:0/2032888133 shutdown_connections 2024-06-10T00:40:35.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.944+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/2032888133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416665880 0x5564166641e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.944+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/2032888133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55641665f7a0 0x55641665f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.944+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/2032888133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556416664f60 0x5564166665d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.944+0000 7f3cef02eb80 1 -- 172.21.15.62:0/2032888133 >> 172.21.15.62:0/2032888133 conn(0x55641658c080 msgr2=0x556416663ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:35.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.944+0000 7f3cef02eb80 1 -- 172.21.15.62:0/2032888133 shutdown_connections 2024-06-10T00:40:35.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.945+0000 7f3cef02eb80 1 -- 172.21.15.62:0/2032888133 wait complete. 2024-06-10T00:40:35.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.945+0000 7f3cef02eb80 1 -- start start 2024-06-10T00:40:35.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.945+0000 7f3cef02eb80 1 Processor -- start 2024-06-10T00:40:35.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.945+0000 7f3cef02eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55641665f7a0 0x55641667ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.945+0000 7f3cef02eb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 0x55641667be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.945+0000 7f3cef02eb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556416665880 0x55641667c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.945+0000 7f3cef02eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1b1e10 con 0x55641665f7a0 2024-06-10T00:40:35.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.945+0000 7f3cef02eb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1b1740 con 0x556416665880 2024-06-10T00:40:35.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.945+0000 7f3cef02eb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1ae960 con 0x556416664f60 2024-06-10T00:40:35.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.945+0000 7f3cef02eb80 10 client.0 _ll_register_callbacks cb 0x5564165465b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:35.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3ce2547700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55641665f7a0 0x55641667ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3ce2547700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55641665f7a0 0x55641667ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35004/0 (socket says 172.21.15.62:35004) 2024-06-10T00:40:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3ce2547700 1 -- 172.21.15.62:0/832890212 learned_addr learned my addr 172.21.15.62:0/832890212 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3ce1d46700 1 --2- 172.21.15.62:0/832890212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 0x55641667be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3ce2d48700 1 --2- 172.21.15.62:0/832890212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556416665880 0x55641667c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3ce2547700 1 -- 172.21.15.62:0/832890212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 msgr2=0x55641667be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3ce2547700 1 --2- 172.21.15.62:0/832890212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 0x55641667be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3ce2547700 1 -- 172.21.15.62:0/832890212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556416665880 msgr2=0x55641667c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3ce2547700 1 --2- 172.21.15.62:0/832890212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556416665880 0x55641667c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3ce2547700 1 -- 172.21.15.62:0/832890212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55641667cba0 con 0x55641665f7a0 2024-06-10T00:40:35.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3ce1d46700 1 --2- 172.21.15.62:0/832890212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 0x55641667be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:40:35.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3ce2547700 1 --2- 172.21.15.62:0/832890212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55641665f7a0 0x55641667ead0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00b0a0 tx=0x7f3ccc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fb255a0cb785bd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:35.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3cc1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:35.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3cc1ffb700 1 -- 172.21.15.62:0/832890212 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc007d50 con 0x55641665f7a0 2024-06-10T00:40:35.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3cc1ffb700 1 -- 172.21.15.62:0/832890212 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f3ccc023040 con 0x55641665f7a0 2024-06-10T00:40:35.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.946+0000 7f3cef02eb80 1 -- 172.21.15.62:0/832890212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cd0004500 con 0x55641665f7a0 2024-06-10T00:40:35.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.947+0000 7f3cc1ffb700 1 -- 172.21.15.62:0/832890212 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3ccc042050 con 0x55641665f7a0 2024-06-10T00:40:35.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.947+0000 7f3cef02eb80 10 client.16692 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:35.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.947+0000 7f3cef02eb80 1 -- 172.21.15.62:0/832890212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5564166e3940 con 0x55641665f7a0 2024-06-10T00:40:35.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.947+0000 7f3cc1ffb700 1 -- 172.21.15.62:0/832890212 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd0004500 con 0x55641665f7a0 2024-06-10T00:40:35.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.947+0000 7f3cc1ffb700 1 -- 172.21.15.62:0/832890212 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f3ccc011070 con 0x55641665f7a0 2024-06-10T00:40:35.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.947+0000 7f3cef02eb80 10 client.16692 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:35.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.947+0000 7f3cef02eb80 10 client.16692 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:40:35.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.947+0000 7f3cef02eb80 20 client.16692 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:35.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.947+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/832890212 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556416678f70 0x5564166e6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:35.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.947+0000 7f3cef02eb80 4 client.16692 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:40:35.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.947+0000 7f3cef02eb80 1 -- 172.21.15.62:0/832890212 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564165b82d0 con 0x556416678f70 2024-06-10T00:40:35.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.947+0000 7f3ce1d46700 1 --2- 172.21.15.62:0/832890212 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556416678f70 0x5564166e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:35.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3ce1d46700 1 --2- 172.21.15.62:0/832890212 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556416678f70 0x5564166e6090 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5a689f3babe840d server_cookie=e19deeee08ee945c in_seq=0 out_seq=0 2024-06-10T00:40:35.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cc1ffb700 10 client.16692 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:40:35.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cc1ffb700 1 -- 172.21.15.62:0/832890212 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564165b82d0 con 0x556416678f70 2024-06-10T00:40:35.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cc1ffb700 10 client.16692 handle_command_reply: tid=0 2024-06-10T00:40:35.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cef02eb80 1 client.16692 shutdown 2024-06-10T00:40:35.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cef02eb80 1 -- 172.21.15.62:0/832890212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55641665f7a0 msgr2=0x55641667ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:35.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/832890212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55641665f7a0 0x55641667ead0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00b0a0 tx=0x7f3ccc01d000 comp rx=0 tx=0).stop 2024-06-10T00:40:35.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cef02eb80 1 -- 172.21.15.62:0/832890212 shutdown_connections 2024-06-10T00:40:35.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/832890212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55641665f7a0 0x55641667ead0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/832890212 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x556416678f70 0x5564166e6090 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/832890212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556416665880 0x55641667c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cef02eb80 1 --2- 172.21.15.62:0/832890212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556416664f60 0x55641667be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cef02eb80 1 -- 172.21.15.62:0/832890212 >> 172.21.15.62:0/832890212 conn(0x55641658c080 msgr2=0x55641666a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:35.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cc1ffb700 0 client.16692 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:40:35.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cef02eb80 1 -- 172.21.15.62:0/832890212 shutdown_connections 2024-06-10T00:40:35.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.948+0000 7f3cef02eb80 1 -- 172.21.15.62:0/832890212 wait complete. 2024-06-10T00:40:35.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.949+0000 7f3cef02eb80 20 client.16692 trim_cache size 0 max 16384 2024-06-10T00:40:35.957 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:40:35.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.949+0000 7f3ce47ab700 1 -- 172.21.15.62:0/313929622 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3cc40a2e10 msgr2=0x7f3cc40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:35.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.949+0000 7f3ce47ab700 1 --2- 172.21.15.62:0/313929622 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3cc40a2e10 0x7f3cc40a52d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009e60 tx=0x7f3ccc00b7b0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.949+0000 7f3ce47ab700 1 -- 172.21.15.62:0/313929622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cdc108510 msgr2=0x7f3cdc1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:35.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.949+0000 7f3ce47ab700 1 --2- 172.21.15.62:0/313929622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cdc108510 0x7f3cdc1026e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400b3f0 tx=0x7f3cd400b700 comp rx=0 tx=0).stop 2024-06-10T00:40:35.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.949+0000 7f3ce47ab700 1 -- 172.21.15.62:0/313929622 shutdown_connections 2024-06-10T00:40:35.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.949+0000 7f3ce47ab700 1 --2- 172.21.15.62:0/313929622 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3cc40a2e10 0x7f3cc40a52d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.949+0000 7f3ce47ab700 1 --2- 172.21.15.62:0/313929622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cdc108510 0x7f3cdc1026e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.949+0000 7f3ce47ab700 1 --2- 172.21.15.62:0/313929622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cdc100a80 0x7f3cdc101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.949+0000 7f3ce47ab700 1 --2- 172.21.15.62:0/313929622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3cdc101440 0x7f3cdc1021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:35.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.949+0000 7f3ce47ab700 1 -- 172.21.15.62:0/313929622 >> 172.21.15.62:0/313929622 conn(0x7f3cdc077a00 msgr2=0x7f3cdc0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:35.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.949+0000 7f3ce47ab700 1 -- 172.21.15.62:0/313929622 shutdown_connections 2024-06-10T00:40:35.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:35.949+0000 7f3ce47ab700 1 -- 172.21.15.62:0/313929622 wait complete. 2024-06-10T00:40:36.635 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T00:40:36.889 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:40:37.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:37 smithi062 ceph-mon[32390]: pgmap v238: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-06-10T00:40:37.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:37 smithi105 ceph-mon[36321]: pgmap v238: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-06-10T00:40:37.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:37 smithi138 ceph-mon[36549]: pgmap v238: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-06-10T00:40:38.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.740+0000 7f62dd67f700 1 -- 172.21.15.62:0/3123345258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806ab30 msgr2=0x7f62d810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:38.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.740+0000 7f62dd67f700 1 --2- 172.21.15.62:0/3123345258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806ab30 0x7f62d810f320 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009990 tx=0x7f62c8009ca0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.740+0000 7f62dd67f700 1 -- 172.21.15.62:0/3123345258 shutdown_connections 2024-06-10T00:40:38.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.740+0000 7f62dd67f700 1 --2- 172.21.15.62:0/3123345258 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806ab30 0x7f62d810f320 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.740+0000 7f62dd67f700 1 --2- 172.21.15.62:0/3123345258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62d8108500 0x7f62d81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.740+0000 7f62dd67f700 1 --2- 172.21.15.62:0/3123345258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62d806a170 0x7f62d806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.740+0000 7f62dd67f700 1 -- 172.21.15.62:0/3123345258 >> 172.21.15.62:0/3123345258 conn(0x7f62d8078340 msgr2=0x7f62d8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:38.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.740+0000 7f62dd67f700 1 -- 172.21.15.62:0/3123345258 shutdown_connections 2024-06-10T00:40:38.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.740+0000 7f62dd67f700 1 -- 172.21.15.62:0/3123345258 wait complete. 2024-06-10T00:40:38.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.741+0000 7f62dd67f700 1 Processor -- start 2024-06-10T00:40:38.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.741+0000 7f62dd67f700 1 -- start start 2024-06-10T00:40:38.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62dd67f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62d806a170 0x7f62d819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62dd67f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806ab30 0x7f62d819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62dd67f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62d8108500 0x7f62d81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62dd67f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f62d8112760 con 0x7f62d806ab30 2024-06-10T00:40:38.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62dd67f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81125e0 con 0x7f62d806a170 2024-06-10T00:40:38.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62dd67f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81128e0 con 0x7f62d8108500 2024-06-10T00:40:38.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62d67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806ab30 0x7f62d819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62d67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806ab30 0x7f62d819ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32968/0 (socket says 172.21.15.62:32968) 2024-06-10T00:40:38.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62d67fc700 1 -- 172.21.15.62:0/3668292756 learned_addr learned my addr 172.21.15.62:0/3668292756 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:38.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/3668292756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62d806a170 0x7f62d819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62d77fe700 1 --2- 172.21.15.62:0/3668292756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62d8108500 0x7f62d81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62d67fc700 1 -- 172.21.15.62:0/3668292756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62d8108500 msgr2=0x7f62d81988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:38.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62d67fc700 1 --2- 172.21.15.62:0/3668292756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62d8108500 0x7f62d81988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62d67fc700 1 -- 172.21.15.62:0/3668292756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62d806a170 msgr2=0x7f62d819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:38.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62d67fc700 1 --2- 172.21.15.62:0/3668292756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62d806a170 0x7f62d819e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62d67fc700 1 -- 172.21.15.62:0/3668292756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d8199040 con 0x7f62d806ab30 2024-06-10T00:40:38.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62d67fc700 1 --2- 172.21.15.62:0/3668292756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806ab30 0x7f62d819ec60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f62c000c9b0 tx=0x7f62c000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:38.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.742+0000 7f62cffff700 1 -- 172.21.15.62:0/3668292756 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0007b00 con 0x7f62d806ab30 2024-06-10T00:40:38.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.743+0000 7f62dd67f700 1 -- 172.21.15.62:0/3668292756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d8199330 con 0x7f62d806ab30 2024-06-10T00:40:38.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.743+0000 7f62cffff700 1 -- 172.21.15.62:0/3668292756 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c000ce50 con 0x7f62d806ab30 2024-06-10T00:40:38.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.743+0000 7f62cffff700 1 -- 172.21.15.62:0/3668292756 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0018660 con 0x7f62d806ab30 2024-06-10T00:40:38.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.743+0000 7f62dd67f700 1 -- 172.21.15.62:0/3668292756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d81a55f0 con 0x7f62d806ab30 2024-06-10T00:40:38.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.745+0000 7f62cffff700 1 -- 172.21.15.62:0/3668292756 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f62c000f450 con 0x7f62d806ab30 2024-06-10T00:40:38.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.745+0000 7f62cffff700 1 --2- 172.21.15.62:0/3668292756 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f62c40a2d90 0x7f62c40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.746+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/3668292756 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f62c40a2d90 0x7f62c40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.746+0000 7f62cffff700 1 -- 172.21.15.62:0/3668292756 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f62c00cf790 con 0x7f62d806ab30 2024-06-10T00:40:38.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.746+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/3668292756 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f62c40a2d90 0x7f62c40a5250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f62d8199a20 tx=0x7f62d00079e0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:38.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.746+0000 7f62e7f02b80 1 Processor -- start 2024-06-10T00:40:38.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.746+0000 7f62e7f02b80 1 -- start start 2024-06-10T00:40:38.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62e7f02b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4a640 0x561753f4aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62e7f02b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4af60 0x561753f449e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62e7f02b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f44f20 0x561753f45340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62e7f02b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561753e96810 con 0x561753f4af60 2024-06-10T00:40:38.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62e7f02b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561753e3e720 con 0x561753f4a640 2024-06-10T00:40:38.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62e7f02b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561753e927d0 con 0x561753f44f20 2024-06-10T00:40:38.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62d67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4af60 0x561753f449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62d67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4af60 0x561753f449e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32980/0 (socket says 172.21.15.62:32980) 2024-06-10T00:40:38.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62d67fc700 1 -- 172.21.15.62:0/2205938691 learned_addr learned my addr 172.21.15.62:0/2205938691 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:38.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/2205938691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4a640 0x561753f4aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62d77fe700 1 --2- 172.21.15.62:0/2205938691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f44f20 0x561753f45340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62d67fc700 1 -- 172.21.15.62:0/2205938691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f44f20 msgr2=0x561753f45340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:38.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62d67fc700 1 --2- 172.21.15.62:0/2205938691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f44f20 0x561753f45340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62d67fc700 1 -- 172.21.15.62:0/2205938691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4a640 msgr2=0x561753f4aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:38.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62d67fc700 1 --2- 172.21.15.62:0/2205938691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4a640 0x561753f4aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62d67fc700 1 -- 172.21.15.62:0/2205938691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561753e34f50 con 0x561753f4af60 2024-06-10T00:40:38.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.747+0000 7f62d67fc700 1 --2- 172.21.15.62:0/2205938691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4af60 0x561753f449e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f62c0098270 tx=0x7f62c009c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6608e3be0b1dbfd4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:38.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.748+0000 7f62cd7fa700 1 -- 172.21.15.62:0/2205938691 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c00a4510 con 0x561753f4af60 2024-06-10T00:40:38.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.748+0000 7f62cd7fa700 1 -- 172.21.15.62:0/2205938691 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c0007ca0 con 0x561753f4af60 2024-06-10T00:40:38.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.748+0000 7f62cd7fa700 1 -- 172.21.15.62:0/2205938691 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c00862a0 con 0x561753f4af60 2024-06-10T00:40:38.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.748+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2205938691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4af60 msgr2=0x561753f449e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:38.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.748+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/2205938691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4af60 0x561753f449e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f62c0098270 tx=0x7f62c009c0f0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.748+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2205938691 shutdown_connections 2024-06-10T00:40:38.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.748+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/2205938691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f44f20 0x561753f45340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.748+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/2205938691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4a640 0x561753f4aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.748+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/2205938691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4af60 0x561753f449e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.748+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2205938691 >> 172.21.15.62:0/2205938691 conn(0x561753e6fcc0 msgr2=0x561753f3f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:38.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2205938691 shutdown_connections 2024-06-10T00:40:38.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2205938691 wait complete. 2024-06-10T00:40:38.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62e7f02b80 1 -- start start 2024-06-10T00:40:38.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62e7f02b80 1 Processor -- start 2024-06-10T00:40:38.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62e7f02b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f44f20 0x561753f50d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62e7f02b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4a640 0x561753f4e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62e7f02b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f4af60 0x561753f4e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62e7f02b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561753f519d0 con 0x561753f44f20 2024-06-10T00:40:38.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62e7f02b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561753e36bf0 con 0x561753f4a640 2024-06-10T00:40:38.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62e7f02b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561753f51b50 con 0x561753f4af60 2024-06-10T00:40:38.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62e7f02b80 10 client.0 _ll_register_callbacks cb 0x561753e30920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:38.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f44f20 0x561753f50d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f44f20 0x561753f50d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32994/0 (socket says 172.21.15.62:32994) 2024-06-10T00:40:38.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62d6ffd700 1 -- 172.21.15.62:0/2862861045 learned_addr learned my addr 172.21.15.62:0/2862861045 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:38.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.749+0000 7f62d77fe700 1 --2- 172.21.15.62:0/2862861045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f4af60 0x561753f4e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62d67fc700 1 --2- 172.21.15.62:0/2862861045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4a640 0x561753f4e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62d6ffd700 1 -- 172.21.15.62:0/2862861045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f4af60 msgr2=0x561753f4e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:38.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/2862861045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f4af60 0x561753f4e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62d6ffd700 1 -- 172.21.15.62:0/2862861045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4a640 msgr2=0x561753f4e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:38.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/2862861045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4a640 0x561753f4e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62d6ffd700 1 -- 172.21.15.62:0/2862861045 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561753de58d0 con 0x561753f44f20 2024-06-10T00:40:38.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/2862861045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f44f20 0x561753f50d90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f62d0014d90 tx=0x7f62d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d92597df01bc598 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:38.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62be7fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:38.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62be7fc700 1 -- 172.21.15.62:0/2862861045 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0025070 con 0x561753f44f20 2024-06-10T00:40:38.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62be7fc700 1 -- 172.21.15.62:0/2862861045 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f62d0015aa0 con 0x561753f44f20 2024-06-10T00:40:38.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2862861045 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62c0007ca0 con 0x561753f44f20 2024-06-10T00:40:38.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62be7fc700 1 -- 172.21.15.62:0/2862861045 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f62d0031050 con 0x561753f44f20 2024-06-10T00:40:38.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62e7f02b80 10 client.16704 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:38.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2862861045 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561753f5c400 con 0x561753f44f20 2024-06-10T00:40:38.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.750+0000 7f62be7fc700 1 -- 172.21.15.62:0/2862861045 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c0007ca0 con 0x561753f44f20 2024-06-10T00:40:38.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.751+0000 7f62be7fc700 1 -- 172.21.15.62:0/2862861045 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f62d00293b0 con 0x561753f44f20 2024-06-10T00:40:38.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.751+0000 7f62e7f02b80 10 client.16704 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:38.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.751+0000 7f62e7f02b80 10 client.16704 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:40:38.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.751+0000 7f62e7f02b80 20 client.16704 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:38.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.751+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/2862861045 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561753f5e990 0x561753f60dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.751+0000 7f62e7f02b80 4 client.16704 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:40:38.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.751+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2862861045 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561753ea3d10 con 0x561753f5e990 2024-06-10T00:40:38.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.751+0000 7f62d67fc700 1 --2- 172.21.15.62:0/2862861045 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561753f5e990 0x561753f60dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.751+0000 7f62d67fc700 1 --2- 172.21.15.62:0/2862861045 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561753f5e990 0x561753f60dd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=242f97bed06942b server_cookie=d1965f550843750b in_seq=0 out_seq=0 2024-06-10T00:40:38.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.751+0000 7f62be7fc700 10 client.16704 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:40:38.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.752+0000 7f62be7fc700 1 -- 172.21.15.62:0/2862861045 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561753ea3d10 con 0x561753f5e990 2024-06-10T00:40:38.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.752+0000 7f62be7fc700 10 client.16704 handle_command_reply: tid=0 2024-06-10T00:40:38.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.752+0000 7f62e7f02b80 1 client.16704 shutdown 2024-06-10T00:40:38.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.752+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2862861045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f44f20 msgr2=0x561753f50d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:38.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.752+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/2862861045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f44f20 0x561753f50d90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f62d0014d90 tx=0x7f62d0004ad0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.752+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2862861045 shutdown_connections 2024-06-10T00:40:38.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.752+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/2862861045 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561753f5e990 0x561753f60dd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.752+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/2862861045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f44f20 0x561753f50d90 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.752+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/2862861045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f4af60 0x561753f4e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.752+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/2862861045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4a640 0x561753f4e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.752+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2862861045 >> 172.21.15.62:0/2862861045 conn(0x561753e6fcc0 msgr2=0x561753f3f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:38.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.752+0000 7f62be7fc700 0 client.16704 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:40:38.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.753+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2862861045 shutdown_connections 2024-06-10T00:40:38.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.753+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2862861045 wait complete. 2024-06-10T00:40:38.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.753+0000 7f62e7f02b80 20 client.16704 trim_cache size 0 max 16384 2024-06-10T00:40:38.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.768+0000 7f62e7f02b80 1 Processor -- start 2024-06-10T00:40:38.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.768+0000 7f62e7f02b80 1 -- start start 2024-06-10T00:40:38.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.768+0000 7f62e7f02b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4a640 0x561753f449e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.768+0000 7f62e7f02b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4af60 0x561753f5ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.768+0000 7f62e7f02b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f48800 0x561753f4ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.768+0000 7f62e7f02b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81ae640 con 0x561753f4a640 2024-06-10T00:40:38.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.768+0000 7f62e7f02b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81b0180 con 0x561753f4af60 2024-06-10T00:40:38.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.768+0000 7f62e7f02b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81aed10 con 0x561753f48800 2024-06-10T00:40:38.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.768+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4a640 0x561753f449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.768+0000 7f62d6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4a640 0x561753f449e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33002/0 (socket says 172.21.15.62:33002) 2024-06-10T00:40:38.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.768+0000 7f62d6ffd700 1 -- 172.21.15.62:0/2137124133 learned_addr learned my addr 172.21.15.62:0/2137124133 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:38.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.768+0000 7f62d77fe700 1 --2- 172.21.15.62:0/2137124133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f48800 0x561753f4ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.769+0000 7f62d6ffd700 1 -- 172.21.15.62:0/2137124133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f48800 msgr2=0x561753f4ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:38.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.769+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/2137124133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f48800 0x561753f4ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.769+0000 7f62d6ffd700 1 -- 172.21.15.62:0/2137124133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4af60 msgr2=0x561753f5ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:38.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.769+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/2137124133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4af60 0x561753f5ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.769+0000 7f62d6ffd700 1 -- 172.21.15.62:0/2137124133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561753f4ac70 con 0x561753f4a640 2024-06-10T00:40:38.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.769+0000 7f62d77fe700 1 --2- 172.21.15.62:0/2137124133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f48800 0x561753f4ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:40:38.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.769+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/2137124133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4a640 0x561753f449e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f62d0007ba0 tx=0x7f62d0007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92a243a96bc3a20 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:38.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.769+0000 7f62ccff9700 1 -- 172.21.15.62:0/2137124133 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d003a070 con 0x561753f4a640 2024-06-10T00:40:38.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.769+0000 7f62ccff9700 1 -- 172.21.15.62:0/2137124133 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62d001c040 con 0x561753f4a640 2024-06-10T00:40:38.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.769+0000 7f62ccff9700 1 -- 172.21.15.62:0/2137124133 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0025040 con 0x561753f4a640 2024-06-10T00:40:38.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2137124133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4a640 msgr2=0x561753f449e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:38.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/2137124133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4a640 0x561753f449e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f62d0007ba0 tx=0x7f62d0007f20 comp rx=0 tx=0).stop 2024-06-10T00:40:38.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2137124133 shutdown_connections 2024-06-10T00:40:38.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/2137124133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4a640 0x561753f449e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/2137124133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f48800 0x561753f4ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/2137124133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4af60 0x561753f5ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2137124133 >> 172.21.15.62:0/2137124133 conn(0x561753e6fcc0 msgr2=0x561753f4b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2137124133 shutdown_connections 2024-06-10T00:40:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 -- 172.21.15.62:0/2137124133 wait complete. 2024-06-10T00:40:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 -- start start 2024-06-10T00:40:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 Processor -- start 2024-06-10T00:40:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f48800 0x561753f57420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4a640 0x561753f65e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4af60 0x561753f66370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81afab0 con 0x561753f4a640 2024-06-10T00:40:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81af3e0 con 0x561753f4af60 2024-06-10T00:40:38.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62d81ac600 con 0x561753f48800 2024-06-10T00:40:38.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.770+0000 7f62e7f02b80 10 client.0 _ll_register_callbacks cb 0x561753e30920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:38.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62d67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4a640 0x561753f65e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62d67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4a640 0x561753f65e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33018/0 (socket says 172.21.15.62:33018) 2024-06-10T00:40:38.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62d67fc700 1 -- 172.21.15.62:0/600764642 learned_addr learned my addr 172.21.15.62:0/600764642 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:38.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/600764642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f48800 0x561753f57420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62d77fe700 1 --2- 172.21.15.62:0/600764642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4af60 0x561753f66370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62d67fc700 1 -- 172.21.15.62:0/600764642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f48800 msgr2=0x561753f57420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:38.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62d67fc700 1 --2- 172.21.15.62:0/600764642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f48800 0x561753f57420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62d67fc700 1 -- 172.21.15.62:0/600764642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4af60 msgr2=0x561753f66370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:38.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62d67fc700 1 --2- 172.21.15.62:0/600764642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4af60 0x561753f66370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62d67fc700 1 -- 172.21.15.62:0/600764642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561753f669b0 con 0x561753f4a640 2024-06-10T00:40:38.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62d77fe700 1 --2- 172.21.15.62:0/600764642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4af60 0x561753f66370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:40:38.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62d67fc700 1 --2- 172.21.15.62:0/600764642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4a640 0x561753f65e30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f62c00ccb80 tx=0x7f62c000fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eadf0a3806cf389 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:38.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62be7fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:38.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62be7fc700 1 -- 172.21.15.62:0/600764642 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0099ba0 con 0x561753f4a640 2024-06-10T00:40:38.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62be7fc700 1 -- 172.21.15.62:0/600764642 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f62c00ba1a0 con 0x561753f4a640 2024-06-10T00:40:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62e7f02b80 1 -- 172.21.15.62:0/600764642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62d001c040 con 0x561753f4a640 2024-06-10T00:40:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62be7fc700 1 -- 172.21.15.62:0/600764642 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f62c0097f10 con 0x561753f4a640 2024-06-10T00:40:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62e7f02b80 10 client.16710 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.771+0000 7f62e7f02b80 1 -- 172.21.15.62:0/600764642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561753fc9430 con 0x561753f4a640 2024-06-10T00:40:38.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.772+0000 7f62be7fc700 1 -- 172.21.15.62:0/600764642 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62d001c040 con 0x561753f4a640 2024-06-10T00:40:38.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.772+0000 7f62be7fc700 1 -- 172.21.15.62:0/600764642 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f62c009cbd0 con 0x561753f4a640 2024-06-10T00:40:38.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.772+0000 7f62e7f02b80 10 client.16710 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:38.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.772+0000 7f62e7f02b80 10 client.16710 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:40:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.772+0000 7f62e7f02b80 20 client.16710 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.772+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/600764642 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561753fc9720 0x561753fcbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.772+0000 7f62e7f02b80 4 client.16710 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:40:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.772+0000 7f62e7f02b80 1 -- 172.21.15.62:0/600764642 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561753e9e290 con 0x561753fc9720 2024-06-10T00:40:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.772+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/600764642 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561753fc9720 0x561753fcbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.772+0000 7f62d6ffd700 1 --2- 172.21.15.62:0/600764642 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561753fc9720 0x561753fcbb10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9bd961c449e03bb5 server_cookie=783ad2ccdae199d4 in_seq=0 out_seq=0 2024-06-10T00:40:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.772+0000 7f62be7fc700 10 client.16710 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:40:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62be7fc700 1 -- 172.21.15.62:0/600764642 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561753e9e290 con 0x561753fc9720 2024-06-10T00:40:38.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62be7fc700 10 client.16710 handle_command_reply: tid=0 2024-06-10T00:40:38.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62e7f02b80 1 client.16710 shutdown 2024-06-10T00:40:38.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62e7f02b80 1 -- 172.21.15.62:0/600764642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4a640 msgr2=0x561753f65e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:38.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/600764642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4a640 0x561753f65e30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f62c00ccb80 tx=0x7f62c000fcc0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62e7f02b80 1 -- 172.21.15.62:0/600764642 shutdown_connections 2024-06-10T00:40:38.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/600764642 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x561753fc9720 0x561753fcbb10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/600764642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561753f4af60 0x561753f66370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/600764642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561753f48800 0x561753f57420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62e7f02b80 1 --2- 172.21.15.62:0/600764642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561753f4a640 0x561753f65e30 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62e7f02b80 1 -- 172.21.15.62:0/600764642 >> 172.21.15.62:0/600764642 conn(0x561753e6fcc0 msgr2=0x561753f4b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:38.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62be7fc700 0 client.16710 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:40:38.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62e7f02b80 1 -- 172.21.15.62:0/600764642 shutdown_connections 2024-06-10T00:40:38.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62e7f02b80 1 -- 172.21.15.62:0/600764642 wait complete. 2024-06-10T00:40:38.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.773+0000 7f62e7f02b80 20 client.16710 trim_cache size 0 max 16384 2024-06-10T00:40:38.792 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:40:38.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.774+0000 7f62dd67f700 1 -- 172.21.15.62:0/3668292756 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f62c40a2d90 msgr2=0x7f62c40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:38.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.774+0000 7f62dd67f700 1 --2- 172.21.15.62:0/3668292756 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f62c40a2d90 0x7f62c40a5250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f62d8199a20 tx=0x7f62d00079e0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.774+0000 7f62dd67f700 1 -- 172.21.15.62:0/3668292756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806ab30 msgr2=0x7f62d819ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:38.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.774+0000 7f62dd67f700 1 --2- 172.21.15.62:0/3668292756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806ab30 0x7f62d819ec60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f62c000c9b0 tx=0x7f62c000ccc0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.774+0000 7f62dd67f700 1 -- 172.21.15.62:0/3668292756 shutdown_connections 2024-06-10T00:40:38.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.774+0000 7f62dd67f700 1 --2- 172.21.15.62:0/3668292756 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f62c40a2d90 0x7f62c40a5250 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.774+0000 7f62dd67f700 1 --2- 172.21.15.62:0/3668292756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62d8108500 0x7f62d81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.774+0000 7f62dd67f700 1 --2- 172.21.15.62:0/3668292756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62d806a170 0x7f62d819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.774+0000 7f62dd67f700 1 --2- 172.21.15.62:0/3668292756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806ab30 0x7f62d819ec60 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:38.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.774+0000 7f62dd67f700 1 -- 172.21.15.62:0/3668292756 >> 172.21.15.62:0/3668292756 conn(0x7f62d8078340 msgr2=0x7f62d80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:38.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.774+0000 7f62dd67f700 1 -- 172.21.15.62:0/3668292756 shutdown_connections 2024-06-10T00:40:38.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:38.774+0000 7f62dd67f700 1 -- 172.21.15.62:0/3668292756 wait complete. 2024-06-10T00:40:39.633 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T00:40:39.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:39 smithi105 ceph-mon[36321]: pgmap v239: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:40:39.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:39 smithi062 ceph-mon[32390]: pgmap v239: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:40:39.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:39 smithi138 ceph-mon[36549]: pgmap v239: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:40:39.891 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:40:41.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.580+0000 7f1d34856700 1 -- 172.21.15.62:0/1202252178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d30105330 msgr2=0x7f1d30105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:41.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.580+0000 7f1d34856700 1 --2- 172.21.15.62:0/1202252178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d30105330 0x7f1d30105710 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1d200099c0 tx=0x7f1d20009cd0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:41 smithi062 ceph-mon[32390]: pgmap v240: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:40:41.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.580+0000 7f1d34856700 1 -- 172.21.15.62:0/1202252178 shutdown_connections 2024-06-10T00:40:41.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.580+0000 7f1d34856700 1 --2- 172.21.15.62:0/1202252178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d30105330 0x7f1d30105710 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.580+0000 7f1d34856700 1 --2- 172.21.15.62:0/1202252178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d30107b60 0x7f1d3010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.580+0000 7f1d34856700 1 --2- 172.21.15.62:0/1202252178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d301071a0 0x7f1d30107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.580+0000 7f1d34856700 1 -- 172.21.15.62:0/1202252178 >> 172.21.15.62:0/1202252178 conn(0x7f1d300766b0 msgr2=0x7f1d30076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:41.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.580+0000 7f1d34856700 1 -- 172.21.15.62:0/1202252178 shutdown_connections 2024-06-10T00:40:41.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.580+0000 7f1d34856700 1 -- 172.21.15.62:0/1202252178 wait complete. 2024-06-10T00:40:41.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.581+0000 7f1d34856700 1 Processor -- start 2024-06-10T00:40:41.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.581+0000 7f1d34856700 1 -- start start 2024-06-10T00:40:41.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.581+0000 7f1d34856700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d30105330 0x7f1d301083c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.581+0000 7f1d34856700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d301071a0 0x7f1d30108900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.581+0000 7f1d34856700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d30107b60 0x7f1d30108e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.581+0000 7f1d34856700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30111730 con 0x7f1d30107b60 2024-06-10T00:40:41.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.581+0000 7f1d34856700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301115b0 con 0x7f1d301071a0 2024-06-10T00:40:41.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.581+0000 7f1d34856700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301118b0 con 0x7f1d30105330 2024-06-10T00:40:41.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.581+0000 7f1d2ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d30107b60 0x7f1d30108e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.581+0000 7f1d2ed9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d30107b60 0x7f1d30108e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33036/0 (socket says 172.21.15.62:33036) 2024-06-10T00:40:41.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.581+0000 7f1d2ed9d700 1 -- 172.21.15.62:0/3878698629 learned_addr learned my addr 172.21.15.62:0/3878698629 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:41.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.581+0000 7f1d2dd9b700 1 --2- 172.21.15.62:0/3878698629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d301071a0 0x7f1d30108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.582+0000 7f1d2e59c700 1 --2- 172.21.15.62:0/3878698629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d30105330 0x7f1d301083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.582+0000 7f1d2ed9d700 1 -- 172.21.15.62:0/3878698629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d30105330 msgr2=0x7f1d301083c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:41.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.582+0000 7f1d2ed9d700 1 --2- 172.21.15.62:0/3878698629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d30105330 0x7f1d301083c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.582+0000 7f1d2ed9d700 1 -- 172.21.15.62:0/3878698629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d301071a0 msgr2=0x7f1d30108900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:41.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.582+0000 7f1d2ed9d700 1 --2- 172.21.15.62:0/3878698629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d301071a0 0x7f1d30108900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.582+0000 7f1d2ed9d700 1 -- 172.21.15.62:0/3878698629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d301a7140 con 0x7f1d30107b60 2024-06-10T00:40:41.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.582+0000 7f1d2ed9d700 1 --2- 172.21.15.62:0/3878698629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d30107b60 0x7f1d30108e40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1d28007ee0 tx=0x7f1d2800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:41.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.582+0000 7f1d1f7fe700 1 -- 172.21.15.62:0/3878698629 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2800dbd0 con 0x7f1d30107b60 2024-06-10T00:40:41.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.582+0000 7f1d1f7fe700 1 -- 172.21.15.62:0/3878698629 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d2800dd70 con 0x7f1d30107b60 2024-06-10T00:40:41.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.582+0000 7f1d34856700 1 -- 172.21.15.62:0/3878698629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d301a7430 con 0x7f1d30107b60 2024-06-10T00:40:41.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.582+0000 7f1d1f7fe700 1 -- 172.21.15.62:0/3878698629 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28005a60 con 0x7f1d30107b60 2024-06-10T00:40:41.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.582+0000 7f1d34856700 1 -- 172.21.15.62:0/3878698629 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d301a7970 con 0x7f1d30107b60 2024-06-10T00:40:41.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.584+0000 7f1d1f7fe700 1 -- 172.21.15.62:0/3878698629 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1d2801a020 con 0x7f1d30107b60 2024-06-10T00:40:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.585+0000 7f1d1f7fe700 1 --2- 172.21.15.62:0/3878698629 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1d180a2e10 0x7f1d180a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.585+0000 7f1d2e59c700 1 --2- 172.21.15.62:0/3878698629 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1d180a2e10 0x7f1d180a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.585+0000 7f1d1f7fe700 1 -- 172.21.15.62:0/3878698629 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f1d280ced90 con 0x7f1d30107b60 2024-06-10T00:40:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.586+0000 7f1d2e59c700 1 --2- 172.21.15.62:0/3878698629 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1d180a2e10 0x7f1d180a52d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1d20000c00 tx=0x7f1d20005fd0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.586+0000 7f1d3f0d9b80 1 Processor -- start 2024-06-10T00:40:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.586+0000 7f1d3f0d9b80 1 -- start start 2024-06-10T00:40:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.586+0000 7f1d3f0d9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602640 0x5565d5602a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.586+0000 7f1d3f0d9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d5602f60 0x5565d55fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.586+0000 7f1d3f0d9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d55fcf20 0x5565d55fd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.586+0000 7f1d3f0d9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5565d554e810 con 0x5565d5602f60 2024-06-10T00:40:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.586+0000 7f1d3f0d9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5565d54f6720 con 0x5565d5602640 2024-06-10T00:40:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.586+0000 7f1d3f0d9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5565d554a7d0 con 0x5565d55fcf20 2024-06-10T00:40:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.586+0000 7f1d2dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d5602f60 0x5565d55fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.586+0000 7f1d2dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d5602f60 0x5565d55fc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33052/0 (socket says 172.21.15.62:33052) 2024-06-10T00:40:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.586+0000 7f1d2dd9b700 1 -- 172.21.15.62:0/1167108277 learned_addr learned my addr 172.21.15.62:0/1167108277 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.587+0000 7f1d2e59c700 1 --2- 172.21.15.62:0/1167108277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602640 0x5565d5602a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.587+0000 7f1d2ed9d700 1 --2- 172.21.15.62:0/1167108277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d55fcf20 0x5565d55fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.587+0000 7f1d2dd9b700 1 -- 172.21.15.62:0/1167108277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d55fcf20 msgr2=0x5565d55fd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.587+0000 7f1d2dd9b700 1 --2- 172.21.15.62:0/1167108277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d55fcf20 0x5565d55fd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.587+0000 7f1d2dd9b700 1 -- 172.21.15.62:0/1167108277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602640 msgr2=0x5565d5602a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.587+0000 7f1d2dd9b700 1 --2- 172.21.15.62:0/1167108277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602640 0x5565d5602a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.587+0000 7f1d2dd9b700 1 -- 172.21.15.62:0/1167108277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565d54ecf50 con 0x5565d5602f60 2024-06-10T00:40:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.587+0000 7f1d2ed9d700 1 --2- 172.21.15.62:0/1167108277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d55fcf20 0x5565d55fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:40:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.587+0000 7f1d2dd9b700 1 --2- 172.21.15.62:0/1167108277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d5602f60 0x5565d55fc9e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f1d24007c60 tx=0x7f1d24007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dce239179cb12f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.587+0000 7f1d1cff9700 1 -- 172.21.15.62:0/1167108277 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d24010d50 con 0x5565d5602f60 2024-06-10T00:40:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.587+0000 7f1d1cff9700 1 -- 172.21.15.62:0/1167108277 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d24015070 con 0x5565d5602f60 2024-06-10T00:40:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.587+0000 7f1d1cff9700 1 -- 172.21.15.62:0/1167108277 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2401a470 con 0x5565d5602f60 2024-06-10T00:40:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.588+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/1167108277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d5602f60 msgr2=0x5565d55fc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.588+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/1167108277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d5602f60 0x5565d55fc9e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f1d24007c60 tx=0x7f1d24007f70 comp rx=0 tx=0).stop 2024-06-10T00:40:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.588+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/1167108277 shutdown_connections 2024-06-10T00:40:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.588+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/1167108277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d55fcf20 0x5565d55fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.588+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/1167108277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602640 0x5565d5602a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.588+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/1167108277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d5602f60 0x5565d55fc9e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.588+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/1167108277 >> 172.21.15.62:0/1167108277 conn(0x5565d5527cc0 msgr2=0x5565d55f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:41.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.588+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/1167108277 shutdown_connections 2024-06-10T00:40:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.588+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/1167108277 wait complete. 2024-06-10T00:40:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.588+0000 7f1d3f0d9b80 1 -- start start 2024-06-10T00:40:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d3f0d9b80 1 Processor -- start 2024-06-10T00:40:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d3f0d9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d55fcf20 0x5565d5608cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d3f0d9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602640 0x5565d5605ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d3f0d9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d5602f60 0x5565d5606530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d3f0d9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5565d56099d0 con 0x5565d55fcf20 2024-06-10T00:40:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d3f0d9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5565d54eebf0 con 0x5565d5602640 2024-06-10T00:40:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d3f0d9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5565d5609b50 con 0x5565d5602f60 2024-06-10T00:40:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d3f0d9b80 10 client.0 _ll_register_callbacks cb 0x5565d54e8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d2e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d55fcf20 0x5565d5608cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d2e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d55fcf20 0x5565d5608cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33060/0 (socket says 172.21.15.62:33060) 2024-06-10T00:40:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d2e59c700 1 -- 172.21.15.62:0/2411671099 learned_addr learned my addr 172.21.15.62:0/2411671099 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d2e59c700 1 -- 172.21.15.62:0/2411671099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d5602f60 msgr2=0x5565d5606530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:41.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d2e59c700 1 --2- 172.21.15.62:0/2411671099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d5602f60 0x5565d5606530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d2e59c700 1 -- 172.21.15.62:0/2411671099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602640 msgr2=0x5565d5605ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d2e59c700 1 --2- 172.21.15.62:0/2411671099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602640 0x5565d5605ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d2e59c700 1 -- 172.21.15.62:0/2411671099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565d549d8d0 con 0x5565d55fcf20 2024-06-10T00:40:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d2e59c700 1 --2- 172.21.15.62:0/2411671099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d55fcf20 0x5565d5608cf0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1d200099c0 tx=0x7f1d20009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80219372b82a8592 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:41.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.589+0000 7f1d15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:41.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.590+0000 7f1d15ffb700 1 -- 172.21.15.62:0/2411671099 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20035060 con 0x5565d55fcf20 2024-06-10T00:40:41.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.590+0000 7f1d15ffb700 1 -- 172.21.15.62:0/2411671099 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f1d2000fc50 con 0x5565d55fcf20 2024-06-10T00:40:41.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.590+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2411671099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d24015070 con 0x5565d55fcf20 2024-06-10T00:40:41.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.590+0000 7f1d15ffb700 1 -- 172.21.15.62:0/2411671099 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1d2003c050 con 0x5565d55fcf20 2024-06-10T00:40:41.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.590+0000 7f1d3f0d9b80 10 client.16722 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:41.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.590+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2411671099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5565d5614300 con 0x5565d55fcf20 2024-06-10T00:40:41.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.590+0000 7f1d15ffb700 1 -- 172.21.15.62:0/2411671099 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d24015070 con 0x5565d55fcf20 2024-06-10T00:40:41.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.590+0000 7f1d15ffb700 1 -- 172.21.15.62:0/2411671099 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1d2002f070 con 0x5565d55fcf20 2024-06-10T00:40:41.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.590+0000 7f1d3f0d9b80 10 client.16722 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:41.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.590+0000 7f1d3f0d9b80 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:40:41.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.590+0000 7f1d3f0d9b80 20 client.16722 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:41.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.590+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/2411671099 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5565d5616940 0x5565d5618de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.590+0000 7f1d3f0d9b80 4 client.16722 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:40:41.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.590+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2411671099 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565d555bd10 con 0x5565d5616940 2024-06-10T00:40:41.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.591+0000 7f1d2dd9b700 1 --2- 172.21.15.62:0/2411671099 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5565d5616940 0x5565d5618de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.591+0000 7f1d2dd9b700 1 --2- 172.21.15.62:0/2411671099 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5565d5616940 0x5565d5618de0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecde027713736080 server_cookie=c69aa6358bc4c82c in_seq=0 out_seq=0 2024-06-10T00:40:41.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.591+0000 7f1d15ffb700 10 client.16722 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:41.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d15ffb700 1 -- 172.21.15.62:0/2411671099 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5565d555bd10 con 0x5565d5616940 2024-06-10T00:40:41.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d15ffb700 10 client.16722 handle_command_reply: tid=0 2024-06-10T00:40:41.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d3f0d9b80 1 client.16722 shutdown 2024-06-10T00:40:41.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2411671099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d55fcf20 msgr2=0x5565d5608cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:41.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/2411671099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d55fcf20 0x5565d5608cf0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1d200099c0 tx=0x7f1d20009cd0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2411671099 shutdown_connections 2024-06-10T00:40:41.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/2411671099 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5565d5616940 0x5565d5618de0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/2411671099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d55fcf20 0x5565d5608cf0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/2411671099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d5602f60 0x5565d5606530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/2411671099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602640 0x5565d5605ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2411671099 >> 172.21.15.62:0/2411671099 conn(0x5565d5527cc0 msgr2=0x5565d55fb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:41.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d15ffb700 0 client.16722 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:41.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2411671099 shutdown_connections 2024-06-10T00:40:41.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2411671099 wait complete. 2024-06-10T00:40:41.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.592+0000 7f1d3f0d9b80 20 client.16722 trim_cache size 0 max 16384 2024-06-10T00:40:41.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.607+0000 7f1d3f0d9b80 1 Processor -- start 2024-06-10T00:40:41.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.607+0000 7f1d3f0d9b80 1 -- start start 2024-06-10T00:40:41.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.607+0000 7f1d3f0d9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d55fcf20 0x5565d55fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.607+0000 7f1d3f0d9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d5602640 0x5565d55f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.607+0000 7f1d3f0d9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602f60 0x5565d55f1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.607+0000 7f1d3f0d9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1000eec0 con 0x5565d5602640 2024-06-10T00:40:41.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.607+0000 7f1d3f0d9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1d10010a00 con 0x5565d5602f60 2024-06-10T00:40:41.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.607+0000 7f1d3f0d9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1000f590 con 0x5565d55fcf20 2024-06-10T00:40:41.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.608+0000 7f1d2dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d5602640 0x5565d55f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.608+0000 7f1d2dd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d5602640 0x5565d55f79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33074/0 (socket says 172.21.15.62:33074) 2024-06-10T00:40:41.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.608+0000 7f1d2dd9b700 1 -- 172.21.15.62:0/3219309106 learned_addr learned my addr 172.21.15.62:0/3219309106 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:41.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.608+0000 7f1d2e59c700 1 --2- 172.21.15.62:0/3219309106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d55fcf20 0x5565d55fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.608+0000 7f1d2dd9b700 1 -- 172.21.15.62:0/3219309106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d55fcf20 msgr2=0x5565d55fc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:41.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.608+0000 7f1d2dd9b700 1 --2- 172.21.15.62:0/3219309106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d55fcf20 0x5565d55fc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.608+0000 7f1d2dd9b700 1 -- 172.21.15.62:0/3219309106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602f60 msgr2=0x5565d55f1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:41.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.608+0000 7f1d2dd9b700 1 --2- 172.21.15.62:0/3219309106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602f60 0x5565d55f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.608+0000 7f1d2dd9b700 1 -- 172.21.15.62:0/3219309106 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565d56020f0 con 0x5565d5602640 2024-06-10T00:40:41.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.608+0000 7f1d2dd9b700 1 --2- 172.21.15.62:0/3219309106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d5602640 0x5565d55f79f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1d24003b90 tx=0x7f1d24003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c583612152bda0c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:41.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.608+0000 7f1d17fff700 1 -- 172.21.15.62:0/3219309106 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d24010d50 con 0x5565d5602640 2024-06-10T00:40:41.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.608+0000 7f1d17fff700 1 -- 172.21.15.62:0/3219309106 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d24015040 con 0x5565d5602640 2024-06-10T00:40:41.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.608+0000 7f1d17fff700 1 -- 172.21.15.62:0/3219309106 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2401abc0 con 0x5565d5602640 2024-06-10T00:40:41.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.609+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/3219309106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d5602640 msgr2=0x5565d55f79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:41.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.609+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/3219309106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d5602640 0x5565d55f79f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1d24003b90 tx=0x7f1d24003b20 comp rx=0 tx=0).stop 2024-06-10T00:40:41.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.609+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/3219309106 shutdown_connections 2024-06-10T00:40:41.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.609+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/3219309106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602f60 0x5565d55f1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.609+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/3219309106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d55fcf20 0x5565d55fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.609+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/3219309106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d5602640 0x5565d55f79f0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.609+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/3219309106 >> 172.21.15.62:0/3219309106 conn(0x5565d5527cc0 msgr2=0x5565d55f0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:41.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.609+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/3219309106 shutdown_connections 2024-06-10T00:40:41.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.609+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/3219309106 wait complete. 2024-06-10T00:40:41.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.609+0000 7f1d3f0d9b80 1 -- start start 2024-06-10T00:40:41.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.609+0000 7f1d3f0d9b80 1 Processor -- start 2024-06-10T00:40:41.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d3f0d9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d55fcf20 0x5565d560ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d3f0d9b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d5602640 0x5565d560f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d3f0d9b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602f60 0x5565d561d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d3f0d9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1d10010330 con 0x5565d55fcf20 2024-06-10T00:40:41.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d3f0d9b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1000fc60 con 0x5565d5602f60 2024-06-10T00:40:41.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d3f0d9b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1000ce80 con 0x5565d5602640 2024-06-10T00:40:41.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d3f0d9b80 10 client.0 _ll_register_callbacks cb 0x5565d54e8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:41.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d2e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d55fcf20 0x5565d560ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d2e59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d55fcf20 0x5565d560ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33084/0 (socket says 172.21.15.62:33084) 2024-06-10T00:40:41.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d2e59c700 1 -- 172.21.15.62:0/2255050226 learned_addr learned my addr 172.21.15.62:0/2255050226 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:41.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d2dd9b700 1 --2- 172.21.15.62:0/2255050226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d5602640 0x5565d560f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d2ed9d700 1 --2- 172.21.15.62:0/2255050226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602f60 0x5565d561d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d2e59c700 1 -- 172.21.15.62:0/2255050226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d5602640 msgr2=0x5565d560f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:41.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d2e59c700 1 --2- 172.21.15.62:0/2255050226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d5602640 0x5565d560f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d2e59c700 1 -- 172.21.15.62:0/2255050226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602f60 msgr2=0x5565d561d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:41.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d2e59c700 1 --2- 172.21.15.62:0/2255050226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602f60 0x5565d561d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d2e59c700 1 -- 172.21.15.62:0/2255050226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565d561dd00 con 0x5565d55fcf20 2024-06-10T00:40:41.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d2dd9b700 1 --2- 172.21.15.62:0/2255050226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d5602640 0x5565d560f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:40:41.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d2e59c700 1 --2- 172.21.15.62:0/2255050226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d55fcf20 0x5565d560ee70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1d200038e0 tx=0x7f1d20024a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9fe4ff246a7183 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:41.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:41.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d15ffb700 1 -- 172.21.15.62:0/2255050226 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20028070 con 0x5565d55fcf20 2024-06-10T00:40:41.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d15ffb700 1 -- 172.21.15.62:0/2255050226 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f1d2000fa50 con 0x5565d55fcf20 2024-06-10T00:40:41.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2255050226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d24015040 con 0x5565d55fcf20 2024-06-10T00:40:41.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d15ffb700 1 -- 172.21.15.62:0/2255050226 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1d20035040 con 0x5565d55fcf20 2024-06-10T00:40:41.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.610+0000 7f1d3f0d9b80 10 client.16728 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:41.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.611+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2255050226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5565d5680e80 con 0x5565d55fcf20 2024-06-10T00:40:41.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.611+0000 7f1d15ffb700 1 -- 172.21.15.62:0/2255050226 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d24015040 con 0x5565d55fcf20 2024-06-10T00:40:41.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.611+0000 7f1d15ffb700 1 -- 172.21.15.62:0/2255050226 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1d2002f040 con 0x5565d55fcf20 2024-06-10T00:40:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.611+0000 7f1d3f0d9b80 10 client.16728 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.611+0000 7f1d3f0d9b80 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:40:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.611+0000 7f1d3f0d9b80 20 client.16728 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.611+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/2255050226 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5565d56815b0 0x5565d56839d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.611+0000 7f1d3f0d9b80 4 client.16728 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:40:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.611+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2255050226 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565d5556290 con 0x5565d56815b0 2024-06-10T00:40:41.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.611+0000 7f1d2dd9b700 1 --2- 172.21.15.62:0/2255050226 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5565d56815b0 0x5565d56839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:41.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d2dd9b700 1 --2- 172.21.15.62:0/2255050226 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5565d56815b0 0x5565d56839d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b553368ffeff50a server_cookie=422d13d1b22d3989 in_seq=0 out_seq=0 2024-06-10T00:40:41.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d15ffb700 10 client.16728 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:41.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d15ffb700 1 -- 172.21.15.62:0/2255050226 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565d5556290 con 0x5565d56815b0 2024-06-10T00:40:41.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d15ffb700 10 client.16728 handle_command_reply: tid=0 2024-06-10T00:40:41.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d3f0d9b80 1 client.16728 shutdown 2024-06-10T00:40:41.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2255050226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d55fcf20 msgr2=0x5565d560ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:41.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/2255050226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d55fcf20 0x5565d560ee70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1d200038e0 tx=0x7f1d20024a20 comp rx=0 tx=0).stop 2024-06-10T00:40:41.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2255050226 shutdown_connections 2024-06-10T00:40:41.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/2255050226 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5565d56815b0 0x5565d56839d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/2255050226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5565d55fcf20 0x5565d560ee70 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/2255050226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565d5602f60 0x5565d561d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d3f0d9b80 1 --2- 172.21.15.62:0/2255050226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565d5602640 0x5565d560f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2255050226 >> 172.21.15.62:0/2255050226 conn(0x5565d5527cc0 msgr2=0x5565d5603340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:41.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d15ffb700 0 client.16728 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2255050226 shutdown_connections 2024-06-10T00:40:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d3f0d9b80 1 -- 172.21.15.62:0/2255050226 wait complete. 2024-06-10T00:40:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.612+0000 7f1d3f0d9b80 20 client.16728 trim_cache size 0 max 16384 2024-06-10T00:40:41.625 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:40:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.613+0000 7f1d34856700 1 -- 172.21.15.62:0/3878698629 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1d180a2e10 msgr2=0x7f1d180a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.613+0000 7f1d34856700 1 --2- 172.21.15.62:0/3878698629 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1d180a2e10 0x7f1d180a52d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1d20000c00 tx=0x7f1d20005fd0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.613+0000 7f1d34856700 1 -- 172.21.15.62:0/3878698629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d30107b60 msgr2=0x7f1d30108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:41.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.613+0000 7f1d34856700 1 --2- 172.21.15.62:0/3878698629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d30107b60 0x7f1d30108e40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1d28007ee0 tx=0x7f1d2800d3b0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.613+0000 7f1d34856700 1 -- 172.21.15.62:0/3878698629 shutdown_connections 2024-06-10T00:40:41.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.613+0000 7f1d34856700 1 --2- 172.21.15.62:0/3878698629 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1d180a2e10 0x7f1d180a52d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.613+0000 7f1d34856700 1 --2- 172.21.15.62:0/3878698629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d30107b60 0x7f1d30108e40 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.613+0000 7f1d34856700 1 --2- 172.21.15.62:0/3878698629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d30105330 0x7f1d301083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.613+0000 7f1d34856700 1 --2- 172.21.15.62:0/3878698629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d301071a0 0x7f1d30108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.613+0000 7f1d34856700 1 -- 172.21.15.62:0/3878698629 >> 172.21.15.62:0/3878698629 conn(0x7f1d300766b0 msgr2=0x7f1d300ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.613+0000 7f1d34856700 1 -- 172.21.15.62:0/3878698629 shutdown_connections 2024-06-10T00:40:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:41.613+0000 7f1d34856700 1 -- 172.21.15.62:0/3878698629 wait complete. 2024-06-10T00:40:41.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:41 smithi105 ceph-mon[36321]: pgmap v240: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:40:41.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:41 smithi138 ceph-mon[36549]: pgmap v240: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:40:43.421 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T00:40:43.422 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T00:40:43.686 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:40:43.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:43 smithi105 ceph-mon[36321]: pgmap v241: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:40:43.808 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:43 smithi062 ceph-mon[32390]: pgmap v241: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:40:43.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:43 smithi138 ceph-mon[36549]: pgmap v241: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:40:45.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.422+0000 7f905a868700 1 -- 172.21.15.62:0/3616203831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905406ab10 msgr2=0x7f905410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:45.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.422+0000 7f905a868700 1 --2- 172.21.15.62:0/3616203831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905406ab10 0x7f905410f320 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f904c00b0d0 tx=0x7f904c00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.422+0000 7f905a868700 1 -- 172.21.15.62:0/3616203831 shutdown_connections 2024-06-10T00:40:45.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.422+0000 7f905a868700 1 --2- 172.21.15.62:0/3616203831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905406ab10 0x7f905410f320 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.422+0000 7f905a868700 1 --2- 172.21.15.62:0/3616203831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90541084e0 0x7f90541088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.422+0000 7f905a868700 1 --2- 172.21.15.62:0/3616203831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f905406a170 0x7f905406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.423+0000 7f905a868700 1 -- 172.21.15.62:0/3616203831 >> 172.21.15.62:0/3616203831 conn(0x7f90540782c0 msgr2=0x7f90540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:45.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.423+0000 7f905a868700 1 -- 172.21.15.62:0/3616203831 shutdown_connections 2024-06-10T00:40:45.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.423+0000 7f905a868700 1 -- 172.21.15.62:0/3616203831 wait complete. 2024-06-10T00:40:45.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.423+0000 7f905a868700 1 Processor -- start 2024-06-10T00:40:45.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.423+0000 7f905a868700 1 -- start start 2024-06-10T00:40:45.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f905a868700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905406a170 0x7f9054101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f905a868700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f905406ab10 0x7f90541019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f905a868700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90541084e0 0x7f9054101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f905a868700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9054112760 con 0x7f905406a170 2024-06-10T00:40:45.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f905a868700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90541125e0 con 0x7f90541084e0 2024-06-10T00:40:45.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f905a868700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90541128e0 con 0x7f905406ab10 2024-06-10T00:40:45.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f9053fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905406a170 0x7f9054101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f9053fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905406a170 0x7f9054101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33114/0 (socket says 172.21.15.62:33114) 2024-06-10T00:40:45.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f9053fff700 1 -- 172.21.15.62:0/3819094844 learned_addr learned my addr 172.21.15.62:0/3819094844 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:45.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f9058e05700 1 --2- 172.21.15.62:0/3819094844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90541084e0 0x7f9054101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f90537fe700 1 --2- 172.21.15.62:0/3819094844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f905406ab10 0x7f90541019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f9053fff700 1 -- 172.21.15.62:0/3819094844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f905406ab10 msgr2=0x7f90541019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:45.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f9053fff700 1 --2- 172.21.15.62:0/3819094844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f905406ab10 0x7f90541019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f9053fff700 1 -- 172.21.15.62:0/3819094844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90541084e0 msgr2=0x7f9054101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:45.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f9053fff700 1 --2- 172.21.15.62:0/3819094844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90541084e0 0x7f9054101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f9053fff700 1 -- 172.21.15.62:0/3819094844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90541a70c0 con 0x7f905406a170 2024-06-10T00:40:45.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.424+0000 7f9053fff700 1 --2- 172.21.15.62:0/3819094844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905406a170 0x7f9054101460 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f904400d950 tx=0x7f904400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:45.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.425+0000 7f90517fa700 1 -- 172.21.15.62:0/3819094844 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044016070 con 0x7f905406a170 2024-06-10T00:40:45.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.425+0000 7f905a868700 1 -- 172.21.15.62:0/3819094844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90541a73b0 con 0x7f905406a170 2024-06-10T00:40:45.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.425+0000 7f90517fa700 1 -- 172.21.15.62:0/3819094844 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f904400fd70 con 0x7f905406a170 2024-06-10T00:40:45.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.425+0000 7f90517fa700 1 -- 172.21.15.62:0/3819094844 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904400e610 con 0x7f905406a170 2024-06-10T00:40:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.425+0000 7f905a868700 1 -- 172.21.15.62:0/3819094844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90541a7940 con 0x7f905406a170 2024-06-10T00:40:45.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.427+0000 7f90517fa700 1 -- 172.21.15.62:0/3819094844 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9044019070 con 0x7f905406a170 2024-06-10T00:40:45.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.428+0000 7f90517fa700 1 --2- 172.21.15.62:0/3819094844 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f903c0a2d90 0x7f903c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.428+0000 7f90517fa700 1 -- 172.21.15.62:0/3819094844 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f90440cf070 con 0x7f905406a170 2024-06-10T00:40:45.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.428+0000 7f90537fe700 1 --2- 172.21.15.62:0/3819094844 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f903c0a2d90 0x7f903c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.428+0000 7f90537fe700 1 --2- 172.21.15.62:0/3819094844 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f903c0a2d90 0x7f903c0a5250 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9054078850 tx=0x7f904800a300 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:45.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90650ebb80 1 Processor -- start 2024-06-10T00:40:45.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90650ebb80 1 -- start start 2024-06-10T00:40:45.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90650ebb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524847fe0 0x5635248483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90650ebb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x5635248422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90650ebb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 0x563524842c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90650ebb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563524734fe0 con 0x563524848900 2024-06-10T00:40:45.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90650ebb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5635247938d0 con 0x563524842820 2024-06-10T00:40:45.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90650ebb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5635246fb4e0 con 0x563524847fe0 2024-06-10T00:40:45.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x5635248422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x5635248422e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33130/0 (socket says 172.21.15.62:33130) 2024-06-10T00:40:45.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90537fe700 1 -- 172.21.15.62:0/2782948725 learned_addr learned my addr 172.21.15.62:0/2782948725 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:45.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f9053fff700 1 --2- 172.21.15.62:0/2782948725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524847fe0 0x5635248483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f9058e05700 1 --2- 172.21.15.62:0/2782948725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 0x563524842c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90537fe700 1 -- 172.21.15.62:0/2782948725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524847fe0 msgr2=0x5635248483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:45.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90537fe700 1 --2- 172.21.15.62:0/2782948725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524847fe0 0x5635248483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90537fe700 1 -- 172.21.15.62:0/2782948725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 msgr2=0x563524842c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90537fe700 1 --2- 172.21.15.62:0/2782948725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 0x563524842c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90537fe700 1 -- 172.21.15.62:0/2782948725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635246e2950 con 0x563524848900 2024-06-10T00:40:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f9058e05700 1 --2- 172.21.15.62:0/2782948725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 0x563524842c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:40:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.429+0000 7f90537fe700 1 --2- 172.21.15.62:0/2782948725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x5635248422e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f904800fd70 tx=0x7f9048012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a645fac5dfac3a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.430+0000 7f903a7fc700 1 -- 172.21.15.62:0/2782948725 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048016070 con 0x563524848900 2024-06-10T00:40:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.430+0000 7f903a7fc700 1 -- 172.21.15.62:0/2782948725 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9048010da0 con 0x563524848900 2024-06-10T00:40:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.430+0000 7f903a7fc700 1 -- 172.21.15.62:0/2782948725 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904801c420 con 0x563524848900 2024-06-10T00:40:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.430+0000 7f90650ebb80 1 -- 172.21.15.62:0/2782948725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 msgr2=0x5635248422e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.430+0000 7f90650ebb80 1 --2- 172.21.15.62:0/2782948725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x5635248422e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f904800fd70 tx=0x7f9048012000 comp rx=0 tx=0).stop 2024-06-10T00:40:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.430+0000 7f90650ebb80 1 -- 172.21.15.62:0/2782948725 shutdown_connections 2024-06-10T00:40:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.430+0000 7f90650ebb80 1 --2- 172.21.15.62:0/2782948725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 0x563524842c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.430+0000 7f90650ebb80 1 --2- 172.21.15.62:0/2782948725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524847fe0 0x5635248483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.430+0000 7f90650ebb80 1 --2- 172.21.15.62:0/2782948725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x5635248422e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.430+0000 7f90650ebb80 1 -- 172.21.15.62:0/2782948725 >> 172.21.15.62:0/2782948725 conn(0x56352476f100 msgr2=0x56352483d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.431+0000 7f90650ebb80 1 -- 172.21.15.62:0/2782948725 shutdown_connections 2024-06-10T00:40:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.431+0000 7f90650ebb80 1 -- 172.21.15.62:0/2782948725 wait complete. 2024-06-10T00:40:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.431+0000 7f90650ebb80 1 -- start start 2024-06-10T00:40:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.431+0000 7f90650ebb80 1 Processor -- start 2024-06-10T00:40:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.431+0000 7f90650ebb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 0x56352484e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.431+0000 7f90650ebb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524847fe0 0x56352484b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.431+0000 7f90650ebb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524848900 0x56352484bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.431+0000 7f90650ebb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563524734040 con 0x563524847fe0 2024-06-10T00:40:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.431+0000 7f90650ebb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56352478f890 con 0x563524842820 2024-06-10T00:40:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.431+0000 7f90650ebb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56352484f2e0 con 0x563524848900 2024-06-10T00:40:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.431+0000 7f90650ebb80 10 client.0 _ll_register_callbacks cb 0x563524729630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f90537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524847fe0 0x56352484b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f90537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524847fe0 0x56352484b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33134/0 (socket says 172.21.15.62:33134) 2024-06-10T00:40:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f90537fe700 1 -- 172.21.15.62:0/1587132058 learned_addr learned my addr 172.21.15.62:0/1587132058 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f9053fff700 1 --2- 172.21.15.62:0/1587132058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 0x56352484e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f9058e05700 1 --2- 172.21.15.62:0/1587132058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524848900 0x56352484bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f90537fe700 1 -- 172.21.15.62:0/1587132058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524848900 msgr2=0x56352484bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f90537fe700 1 --2- 172.21.15.62:0/1587132058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524848900 0x56352484bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f90537fe700 1 -- 172.21.15.62:0/1587132058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 msgr2=0x56352484e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f90537fe700 1 --2- 172.21.15.62:0/1587132058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 0x56352484e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f90537fe700 1 -- 172.21.15.62:0/1587132058 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635247597d0 con 0x563524847fe0 2024-06-10T00:40:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f90537fe700 1 --2- 172.21.15.62:0/1587132058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524847fe0 0x56352484b990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f904800ffd0 tx=0x7f90480057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd826297202727f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f90337fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f90337fe700 1 -- 172.21.15.62:0/1587132058 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904801bcc0 con 0x563524847fe0 2024-06-10T00:40:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f90337fe700 1 -- 172.21.15.62:0/1587132058 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f9048005020 con 0x563524847fe0 2024-06-10T00:40:45.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.432+0000 7f90650ebb80 1 -- 172.21.15.62:0/1587132058 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9048010da0 con 0x563524847fe0 2024-06-10T00:40:45.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.433+0000 7f90337fe700 1 -- 172.21.15.62:0/1587132058 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9048039050 con 0x563524847fe0 2024-06-10T00:40:45.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.433+0000 7f90650ebb80 10 client.16740 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:45.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.433+0000 7f90650ebb80 1 -- 172.21.15.62:0/1587132058 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563524859a00 con 0x563524847fe0 2024-06-10T00:40:45.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.433+0000 7f90337fe700 1 -- 172.21.15.62:0/1587132058 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9048010da0 con 0x563524847fe0 2024-06-10T00:40:45.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.433+0000 7f90337fe700 1 -- 172.21.15.62:0/1587132058 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f9048004b60 con 0x563524847fe0 2024-06-10T00:40:45.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.433+0000 7f90650ebb80 10 client.16740 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:45.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.433+0000 7f90650ebb80 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:40:45.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.433+0000 7f90650ebb80 20 client.16740 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:45.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.433+0000 7f90650ebb80 1 --2- 172.21.15.62:0/1587132058 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x56352485bf80 0x56352485e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.433+0000 7f90650ebb80 4 client.16740 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:40:45.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.433+0000 7f90650ebb80 1 -- 172.21.15.62:0/1587132058 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635247a0dd0 con 0x56352485bf80 2024-06-10T00:40:45.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.433+0000 7f9053fff700 1 --2- 172.21.15.62:0/1587132058 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x56352485bf80 0x56352485e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.434+0000 7f9053fff700 1 --2- 172.21.15.62:0/1587132058 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x56352485bf80 0x56352485e370 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcde3ef55279a8fb server_cookie=4fb92cf2cd88af9e in_seq=0 out_seq=0 2024-06-10T00:40:45.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.434+0000 7f90337fe700 10 client.16740 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:45.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.434+0000 7f90337fe700 1 -- 172.21.15.62:0/1587132058 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5635247a0dd0 con 0x56352485bf80 2024-06-10T00:40:45.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.434+0000 7f90337fe700 10 client.16740 handle_command_reply: tid=0 2024-06-10T00:40:45.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.434+0000 7f90650ebb80 1 client.16740 shutdown 2024-06-10T00:40:45.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.435+0000 7f90650ebb80 1 -- 172.21.15.62:0/1587132058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524847fe0 msgr2=0x56352484b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:45.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.435+0000 7f90650ebb80 1 --2- 172.21.15.62:0/1587132058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524847fe0 0x56352484b990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f904800ffd0 tx=0x7f90480057d0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.435+0000 7f90650ebb80 1 -- 172.21.15.62:0/1587132058 shutdown_connections 2024-06-10T00:40:45.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.435+0000 7f90650ebb80 1 --2- 172.21.15.62:0/1587132058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524848900 0x56352484bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.435+0000 7f90650ebb80 1 --2- 172.21.15.62:0/1587132058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 0x56352484e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.435+0000 7f90650ebb80 1 --2- 172.21.15.62:0/1587132058 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x56352485bf80 0x56352485e370 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.435+0000 7f90650ebb80 1 --2- 172.21.15.62:0/1587132058 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524847fe0 0x56352484b990 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.435+0000 7f90650ebb80 1 -- 172.21.15.62:0/1587132058 >> 172.21.15.62:0/1587132058 conn(0x56352476f100 msgr2=0x56352483c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:45.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.435+0000 7f90337fe700 0 client.16740 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:45.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.435+0000 7f90650ebb80 1 -- 172.21.15.62:0/1587132058 shutdown_connections 2024-06-10T00:40:45.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.435+0000 7f90650ebb80 1 -- 172.21.15.62:0/1587132058 wait complete. 2024-06-10T00:40:45.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.435+0000 7f90650ebb80 20 client.16740 trim_cache size 0 max 16384 2024-06-10T00:40:45.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90650ebb80 1 Processor -- start 2024-06-10T00:40:45.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90650ebb80 1 -- start start 2024-06-10T00:40:45.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90650ebb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524842820 0x563524849410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90650ebb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x56352483cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90650ebb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56352483d440 0x5635248412d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90650ebb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f90541b0a00 con 0x563524848900 2024-06-10T00:40:45.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90650ebb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90541b2540 con 0x56352483d440 2024-06-10T00:40:45.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90650ebb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90541b10d0 con 0x563524842820 2024-06-10T00:40:45.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x56352483cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90537fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x56352483cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33150/0 (socket says 172.21.15.62:33150) 2024-06-10T00:40:45.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90537fe700 1 -- 172.21.15.62:0/471544606 learned_addr learned my addr 172.21.15.62:0/471544606 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:45.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f9053fff700 1 --2- 172.21.15.62:0/471544606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524842820 0x563524849410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90537fe700 1 -- 172.21.15.62:0/471544606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524842820 msgr2=0x563524849410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:45.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90537fe700 1 --2- 172.21.15.62:0/471544606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524842820 0x563524849410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f9058e05700 1 --2- 172.21.15.62:0/471544606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56352483d440 0x5635248412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90537fe700 1 -- 172.21.15.62:0/471544606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56352483d440 msgr2=0x5635248412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:45.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90537fe700 1 --2- 172.21.15.62:0/471544606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56352483d440 0x5635248412d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f90537fe700 1 -- 172.21.15.62:0/471544606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563524842620 con 0x563524848900 2024-06-10T00:40:45.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f9053fff700 1 --2- 172.21.15.62:0/471544606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524842820 0x563524849410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:40:45.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.456+0000 7f9058e05700 1 --2- 172.21.15.62:0/471544606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56352483d440 0x5635248412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:40:45.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.457+0000 7f90537fe700 1 --2- 172.21.15.62:0/471544606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x56352483cf00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f904802e520 tx=0x7f9048005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627d68cf24e3e61b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:45.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.457+0000 7f9039ffb700 1 -- 172.21.15.62:0/471544606 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048016040 con 0x563524848900 2024-06-10T00:40:45.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.457+0000 7f9039ffb700 1 -- 172.21.15.62:0/471544606 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9048042070 con 0x563524848900 2024-06-10T00:40:45.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.457+0000 7f9039ffb700 1 -- 172.21.15.62:0/471544606 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904801bc50 con 0x563524848900 2024-06-10T00:40:45.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.458+0000 7f90650ebb80 1 -- 172.21.15.62:0/471544606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 msgr2=0x56352483cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:45.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.458+0000 7f90650ebb80 1 --2- 172.21.15.62:0/471544606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x56352483cf00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f904802e520 tx=0x7f9048005b20 comp rx=0 tx=0).stop 2024-06-10T00:40:45.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.458+0000 7f90650ebb80 1 -- 172.21.15.62:0/471544606 shutdown_connections 2024-06-10T00:40:45.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.458+0000 7f90650ebb80 1 --2- 172.21.15.62:0/471544606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56352483d440 0x5635248412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.458+0000 7f90650ebb80 1 --2- 172.21.15.62:0/471544606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563524842820 0x563524849410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.458+0000 7f90650ebb80 1 --2- 172.21.15.62:0/471544606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x56352483cf00 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.458+0000 7f90650ebb80 1 -- 172.21.15.62:0/471544606 >> 172.21.15.62:0/471544606 conn(0x56352476f100 msgr2=0x56352484da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:45.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.458+0000 7f90650ebb80 1 -- 172.21.15.62:0/471544606 shutdown_connections 2024-06-10T00:40:45.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.458+0000 7f90650ebb80 1 -- 172.21.15.62:0/471544606 wait complete. 2024-06-10T00:40:45.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.458+0000 7f90650ebb80 1 -- start start 2024-06-10T00:40:45.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.458+0000 7f90650ebb80 1 Processor -- start 2024-06-10T00:40:45.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.458+0000 7f90650ebb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352483d440 0x563524860500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.458+0000 7f90650ebb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 0x563524860a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.458+0000 7f90650ebb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x5635248c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f90650ebb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f90541b1e70 con 0x563524848900 2024-06-10T00:40:45.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f90650ebb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90541b17a0 con 0x563524842820 2024-06-10T00:40:45.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f90650ebb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90541ae9c0 con 0x56352483d440 2024-06-10T00:40:45.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f90650ebb80 10 client.0 _ll_register_callbacks cb 0x563524729630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:45.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f9058e05700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x5635248c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f9058e05700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x5635248c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33164/0 (socket says 172.21.15.62:33164) 2024-06-10T00:40:45.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f9058e05700 1 -- 172.21.15.62:0/3081100790 learned_addr learned my addr 172.21.15.62:0/3081100790 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:45.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f9053fff700 1 --2- 172.21.15.62:0/3081100790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352483d440 0x563524860500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f90537fe700 1 --2- 172.21.15.62:0/3081100790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 0x563524860a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f9058e05700 1 -- 172.21.15.62:0/3081100790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352483d440 msgr2=0x563524860500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:45.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f9058e05700 1 --2- 172.21.15.62:0/3081100790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352483d440 0x563524860500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f9058e05700 1 -- 172.21.15.62:0/3081100790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 msgr2=0x563524860a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:45.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f9058e05700 1 --2- 172.21.15.62:0/3081100790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 0x563524860a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f9058e05700 1 -- 172.21.15.62:0/3081100790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635248c6ec0 con 0x563524848900 2024-06-10T00:40:45.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f90537fe700 1 --2- 172.21.15.62:0/3081100790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 0x563524860a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:40:45.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f9058e05700 1 --2- 172.21.15.62:0/3081100790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x5635248c68d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f904c00d010 tx=0x7f904c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6643725c133cf63 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:45.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f90337fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f90337fe700 1 -- 172.21.15.62:0/3081100790 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c0078f0 con 0x563524848900 2024-06-10T00:40:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f90337fe700 1 -- 172.21.15.62:0/3081100790 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f904c007a90 con 0x563524848900 2024-06-10T00:40:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.459+0000 7f90650ebb80 1 -- 172.21.15.62:0/3081100790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9048042070 con 0x563524848900 2024-06-10T00:40:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.460+0000 7f90337fe700 1 -- 172.21.15.62:0/3081100790 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f904c031050 con 0x563524848900 2024-06-10T00:40:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.460+0000 7f90650ebb80 10 client.16746 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.460+0000 7f90650ebb80 1 -- 172.21.15.62:0/3081100790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5635248c8170 con 0x563524848900 2024-06-10T00:40:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.460+0000 7f90337fe700 1 -- 172.21.15.62:0/3081100790 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9048042070 con 0x563524848900 2024-06-10T00:40:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.460+0000 7f90337fe700 1 -- 172.21.15.62:0/3081100790 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f904c011070 con 0x563524848900 2024-06-10T00:40:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.460+0000 7f90650ebb80 10 client.16746 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:45.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.460+0000 7f90650ebb80 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:40:45.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.460+0000 7f90650ebb80 20 client.16746 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:45.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.460+0000 7f90650ebb80 1 --2- 172.21.15.62:0/3081100790 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5635248c8820 0x5635248cac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:45.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.460+0000 7f90650ebb80 4 client.16746 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T00:40:45.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.460+0000 7f90650ebb80 1 -- 172.21.15.62:0/3081100790 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56352479b350 con 0x5635248c8820 2024-06-10T00:40:45.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.460+0000 7f9053fff700 1 --2- 172.21.15.62:0/3081100790 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5635248c8820 0x5635248cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:45.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.461+0000 7f90337fe700 10 client.16746 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:45.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.461+0000 7f9053fff700 1 --2- 172.21.15.62:0/3081100790 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5635248c8820 0x5635248cac40 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f32913144ecab3d server_cookie=fd8bffc3ee5d10de in_seq=0 out_seq=0 2024-06-10T00:40:45.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90337fe700 1 -- 172.21.15.62:0/3081100790 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56352479b350 con 0x5635248c8820 2024-06-10T00:40:45.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90337fe700 10 client.16746 handle_command_reply: tid=0 2024-06-10T00:40:45.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90650ebb80 1 client.16746 shutdown 2024-06-10T00:40:45.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90650ebb80 1 -- 172.21.15.62:0/3081100790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 msgr2=0x5635248c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:45.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90650ebb80 1 --2- 172.21.15.62:0/3081100790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x5635248c68d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f904c00d010 tx=0x7f904c003ce0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90650ebb80 1 -- 172.21.15.62:0/3081100790 shutdown_connections 2024-06-10T00:40:45.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90650ebb80 1 --2- 172.21.15.62:0/3081100790 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5635248c8820 0x5635248cac40 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90650ebb80 1 --2- 172.21.15.62:0/3081100790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563524848900 0x5635248c68d0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90650ebb80 1 --2- 172.21.15.62:0/3081100790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56352483d440 0x563524860500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90337fe700 0 client.16746 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:45.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90650ebb80 1 --2- 172.21.15.62:0/3081100790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563524842820 0x563524860a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90650ebb80 1 -- 172.21.15.62:0/3081100790 >> 172.21.15.62:0/3081100790 conn(0x56352476f100 msgr2=0x56352484da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:45.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90650ebb80 1 -- 172.21.15.62:0/3081100790 shutdown_connections 2024-06-10T00:40:45.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90650ebb80 1 -- 172.21.15.62:0/3081100790 wait complete. 2024-06-10T00:40:45.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.462+0000 7f90650ebb80 20 client.16746 trim_cache size 0 max 16384 2024-06-10T00:40:45.477 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:40:45.477 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-06-10T00:40:45.477 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "51eccb99-9889-4271-bc5d-d5d2dc1a8951", 2024-06-10T00:40:45.477 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-06-10T00:40:45.478 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:40:45.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.463+0000 7f905a868700 1 -- 172.21.15.62:0/3819094844 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f903c0a2d90 msgr2=0x7f903c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:45.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.463+0000 7f905a868700 1 --2- 172.21.15.62:0/3819094844 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f903c0a2d90 0x7f903c0a5250 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9054078850 tx=0x7f904800a300 comp rx=0 tx=0).stop 2024-06-10T00:40:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.463+0000 7f905a868700 1 -- 172.21.15.62:0/3819094844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905406a170 msgr2=0x7f9054101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.463+0000 7f905a868700 1 --2- 172.21.15.62:0/3819094844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905406a170 0x7f9054101460 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f904400d950 tx=0x7f904400dc60 comp rx=0 tx=0).stop 2024-06-10T00:40:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.463+0000 7f905a868700 1 -- 172.21.15.62:0/3819094844 shutdown_connections 2024-06-10T00:40:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.463+0000 7f905a868700 1 --2- 172.21.15.62:0/3819094844 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f903c0a2d90 0x7f903c0a5250 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.463+0000 7f905a868700 1 --2- 172.21.15.62:0/3819094844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f905406a170 0x7f9054101460 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.463+0000 7f905a868700 1 --2- 172.21.15.62:0/3819094844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90541084e0 0x7f9054101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.463+0000 7f905a868700 1 --2- 172.21.15.62:0/3819094844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f905406ab10 0x7f90541019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.463+0000 7f905a868700 1 -- 172.21.15.62:0/3819094844 >> 172.21.15.62:0/3819094844 conn(0x7f90540782c0 msgr2=0x7f90540ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:45.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.463+0000 7f905a868700 1 -- 172.21.15.62:0/3819094844 shutdown_connections 2024-06-10T00:40:45.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:45.463+0000 7f905a868700 1 -- 172.21.15.62:0/3819094844 wait complete. 2024-06-10T00:40:45.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:45 smithi105 ceph-mon[36321]: pgmap v242: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:40:45.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:45 smithi062 ceph-mon[32390]: pgmap v242: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:40:45.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:45 smithi138 ceph-mon[36549]: pgmap v242: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:40:46.148 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:40:46.402 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:40:46.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:46 smithi105 ceph-mon[36321]: scrub queued for path: / 2024-06-10T00:40:46.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:46 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:46 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:40:46.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:46 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:40:46.785 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:46 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:40:46.786 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:46 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:40:46.786 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:46 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:40:46.786 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:46 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.786 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:46 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.786 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:46 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.786 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:46 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.786 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:46 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.789 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:46 smithi062 ceph-mon[32390]: scrub queued for path: / 2024-06-10T00:40:46.789 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:46 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.789 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:46 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:40:46.789 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:46 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:40:46.789 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:46 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:40:46.789 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:46 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:40:46.789 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:46 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:40:46.790 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:46 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.790 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:46 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.790 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:46 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.790 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:46 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.790 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:46 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:46 smithi138 ceph-mon[36549]: scrub queued for path: / 2024-06-10T00:40:46.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:46 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:46 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:40:46.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:46 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:40:46.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:46 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:40:46.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:46 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:40:46.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:46 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:40:46.875 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:46 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.875 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:46 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.875 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:46 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.875 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:46 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:46.875 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:46 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:40:47.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:47 smithi105 ceph-mon[36321]: pgmap v243: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-10T00:40:47.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:47 smithi062 ceph-mon[32390]: pgmap v243: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-10T00:40:47.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:47 smithi138 ceph-mon[36549]: pgmap v243: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-10T00:40:48.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.130+0000 7f70b6710700 1 -- 172.21.15.62:0/1141626507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b01030f0 msgr2=0x7f70b0107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:48.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.130+0000 7f70b6710700 1 --2- 172.21.15.62:0/1141626507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b01030f0 0x7f70b0107110 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f70980099c0 tx=0x7f7098009cd0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.131+0000 7f70b6710700 1 -- 172.21.15.62:0/1141626507 shutdown_connections 2024-06-10T00:40:48.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.131+0000 7f70b6710700 1 --2- 172.21.15.62:0/1141626507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b0107840 0x7f70b010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.131+0000 7f70b6710700 1 --2- 172.21.15.62:0/1141626507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b0102740 0x7f70b0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.131+0000 7f70b6710700 1 --2- 172.21.15.62:0/1141626507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b01030f0 0x7f70b0107110 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.131+0000 7f70b6710700 1 -- 172.21.15.62:0/1141626507 >> 172.21.15.62:0/1141626507 conn(0x7f70b00795f0 msgr2=0x7f70b0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:48.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.131+0000 7f70b6710700 1 -- 172.21.15.62:0/1141626507 shutdown_connections 2024-06-10T00:40:48.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.131+0000 7f70b6710700 1 -- 172.21.15.62:0/1141626507 wait complete. 2024-06-10T00:40:48.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.131+0000 7f70b6710700 1 Processor -- start 2024-06-10T00:40:48.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.131+0000 7f70b6710700 1 -- start start 2024-06-10T00:40:48.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70b6710700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b0102740 0x7f70b0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70b6710700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b01030f0 0x7f70b0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70b6710700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0107840 0x7f70b0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70b6710700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01127a0 con 0x7f70b0107840 2024-06-10T00:40:48.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70b6710700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f70b0112620 con 0x7f70b0102740 2024-06-10T00:40:48.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70b6710700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70b0112920 con 0x7f70b01030f0 2024-06-10T00:40:48.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70b4cad700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0107840 0x7f70b0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70affff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b0102740 0x7f70b0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70a7fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b01030f0 0x7f70b0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70affff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b0102740 0x7f70b0074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:57148/0 (socket says 172.21.15.62:57148) 2024-06-10T00:40:48.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70b4cad700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0107840 0x7f70b0075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50036/0 (socket says 172.21.15.62:50036) 2024-06-10T00:40:48.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70affff700 1 -- 172.21.15.62:0/2297057709 learned_addr learned my addr 172.21.15.62:0/2297057709 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:48.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70b4cad700 1 -- 172.21.15.62:0/2297057709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b01030f0 msgr2=0x7f70b0074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:48.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70b4cad700 1 --2- 172.21.15.62:0/2297057709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b01030f0 0x7f70b0074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70b4cad700 1 -- 172.21.15.62:0/2297057709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b0102740 msgr2=0x7f70b0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:48.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70b4cad700 1 --2- 172.21.15.62:0/2297057709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b0102740 0x7f70b0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70b4cad700 1 -- 172.21.15.62:0/2297057709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b01a70d0 con 0x7f70b0107840 2024-06-10T00:40:48.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.132+0000 7f70b4cad700 1 --2- 172.21.15.62:0/2297057709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0107840 0x7f70b0075210 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f70a8007f10 tx=0x7f70a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:48.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.133+0000 7f70adffb700 1 -- 172.21.15.62:0/2297057709 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8004db0 con 0x7f70b0107840 2024-06-10T00:40:48.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.133+0000 7f70adffb700 1 -- 172.21.15.62:0/2297057709 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a8009e20 con 0x7f70b0107840 2024-06-10T00:40:48.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.133+0000 7f70adffb700 1 -- 172.21.15.62:0/2297057709 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8015400 con 0x7f70b0107840 2024-06-10T00:40:48.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.133+0000 7f70b6710700 1 -- 172.21.15.62:0/2297057709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b01a73c0 con 0x7f70b0107840 2024-06-10T00:40:48.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.133+0000 7f70b6710700 1 -- 172.21.15.62:0/2297057709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b01a7880 con 0x7f70b0107840 2024-06-10T00:40:48.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.135+0000 7f70adffb700 1 -- 172.21.15.62:0/2297057709 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f70a80155a0 con 0x7f70b0107840 2024-06-10T00:40:48.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.136+0000 7f70adffb700 1 --2- 172.21.15.62:0/2297057709 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f709c0a2e10 0x7f709c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.136+0000 7f70adffb700 1 -- 172.21.15.62:0/2297057709 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f70a80cf110 con 0x7f70b0107840 2024-06-10T00:40:48.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.136+0000 7f70affff700 1 --2- 172.21.15.62:0/2297057709 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f709c0a2e10 0x7f709c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.136+0000 7f70c0f93b80 1 Processor -- start 2024-06-10T00:40:48.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.136+0000 7f70affff700 1 --2- 172.21.15.62:0/2297057709 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f709c0a2e10 0x7f709c0a52d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f70a000ad20 tx=0x7f70a0005f90 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:48.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.136+0000 7f70c0f93b80 1 -- start start 2024-06-10T00:40:48.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.136+0000 7f70c0f93b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aab9f60 0x564e5aaba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70c0f93b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aaba880 0x564e5aab4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70c0f93b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5aab4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70c0f93b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564e5a9a6f60 con 0x564e5aab47a0 2024-06-10T00:40:48.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70c0f93b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564e5aa05850 con 0x564e5aaba880 2024-06-10T00:40:48.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70c0f93b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564e5a96d460 con 0x564e5aab9f60 2024-06-10T00:40:48.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70b4cad700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5aab4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70b4cad700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5aab4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50042/0 (socket says 172.21.15.62:50042) 2024-06-10T00:40:48.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70b4cad700 1 -- 172.21.15.62:0/3395472083 learned_addr learned my addr 172.21.15.62:0/3395472083 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:48.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70a7fff700 1 --2- 172.21.15.62:0/3395472083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aaba880 0x564e5aab4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70affff700 1 --2- 172.21.15.62:0/3395472083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aab9f60 0x564e5aaba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70b4cad700 1 -- 172.21.15.62:0/3395472083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aab9f60 msgr2=0x564e5aaba340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:48.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70b4cad700 1 --2- 172.21.15.62:0/3395472083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aab9f60 0x564e5aaba340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70b4cad700 1 -- 172.21.15.62:0/3395472083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aaba880 msgr2=0x564e5aab4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:48.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70b4cad700 1 --2- 172.21.15.62:0/3395472083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aaba880 0x564e5aab4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70b4cad700 1 -- 172.21.15.62:0/3395472083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e5a9548d0 con 0x564e5aab47a0 2024-06-10T00:40:48.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70b4cad700 1 --2- 172.21.15.62:0/3395472083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5aab4bc0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f70a8097680 tx=0x7f70a809b100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520ee7add8de5d11 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:48.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70a67fc700 1 -- 172.21.15.62:0/3395472083 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a80b9e60 con 0x564e5aab47a0 2024-06-10T00:40:48.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70a67fc700 1 -- 172.21.15.62:0/3395472083 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a80a3e90 con 0x564e5aab47a0 2024-06-10T00:40:48.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.137+0000 7f70a67fc700 1 -- 172.21.15.62:0/3395472083 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a809f370 con 0x564e5aab47a0 2024-06-10T00:40:48.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.138+0000 7f70c0f93b80 1 -- 172.21.15.62:0/3395472083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 msgr2=0x564e5aab4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:48.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.138+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/3395472083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5aab4bc0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f70a8097680 tx=0x7f70a809b100 comp rx=0 tx=0).stop 2024-06-10T00:40:48.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.138+0000 7f70c0f93b80 1 -- 172.21.15.62:0/3395472083 shutdown_connections 2024-06-10T00:40:48.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.138+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/3395472083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5aab4bc0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.138+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/3395472083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aab9f60 0x564e5aaba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.138+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/3395472083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aaba880 0x564e5aab4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.138+0000 7f70c0f93b80 1 -- 172.21.15.62:0/3395472083 >> 172.21.15.62:0/3395472083 conn(0x564e5a9e1080 msgr2=0x564e5aaaf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:48.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.138+0000 7f70c0f93b80 1 -- 172.21.15.62:0/3395472083 shutdown_connections 2024-06-10T00:40:48.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.138+0000 7f70c0f93b80 1 -- 172.21.15.62:0/3395472083 wait complete. 2024-06-10T00:40:48.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.139+0000 7f70c0f93b80 1 -- start start 2024-06-10T00:40:48.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.139+0000 7f70c0f93b80 1 Processor -- start 2024-06-10T00:40:48.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.139+0000 7f70c0f93b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aab47a0 0x564e5aac06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.139+0000 7f70c0f93b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 0x564e5aabd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.139+0000 7f70c0f93b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aaba880 0x564e5aabdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.139+0000 7f70c0f93b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564e5a9a5fc0 con 0x564e5aaba880 2024-06-10T00:40:48.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.139+0000 7f70c0f93b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564e5aa01810 con 0x564e5aab9f60 2024-06-10T00:40:48.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.139+0000 7f70c0f93b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564e5aac1260 con 0x564e5aab47a0 2024-06-10T00:40:48.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.139+0000 7f70c0f93b80 10 client.0 _ll_register_callbacks cb 0x564e5a99b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:48.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.139+0000 7f70b4cad700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aaba880 0x564e5aabdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f70b4cad700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aaba880 0x564e5aabdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50048/0 (socket says 172.21.15.62:50048) 2024-06-10T00:40:48.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f70b4cad700 1 -- 172.21.15.62:0/2603161944 learned_addr learned my addr 172.21.15.62:0/2603161944 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:48.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f70a7fff700 1 --2- 172.21.15.62:0/2603161944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 0x564e5aabd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f70affff700 1 --2- 172.21.15.62:0/2603161944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aab47a0 0x564e5aac06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f70b4cad700 1 -- 172.21.15.62:0/2603161944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aab47a0 msgr2=0x564e5aac06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:48.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f70b4cad700 1 --2- 172.21.15.62:0/2603161944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aab47a0 0x564e5aac06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f70b4cad700 1 -- 172.21.15.62:0/2603161944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 msgr2=0x564e5aabd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:48.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f70b4cad700 1 --2- 172.21.15.62:0/2603161944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 0x564e5aabd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f70b4cad700 1 -- 172.21.15.62:0/2603161944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e5a9cb750 con 0x564e5aaba880 2024-06-10T00:40:48.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f70b4cad700 1 --2- 172.21.15.62:0/2603161944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aaba880 0x564e5aabdf20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f70a800d700 tx=0x7f70a800d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d088920a2a49a82 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:48.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f708f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:48.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f708f7fe700 1 -- 172.21.15.62:0/2603161944 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a80b4220 con 0x564e5aaba880 2024-06-10T00:40:48.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f708f7fe700 1 -- 172.21.15.62:0/2603161944 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f70a80bb0c0 con 0x564e5aaba880 2024-06-10T00:40:48.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f70c0f93b80 1 -- 172.21.15.62:0/2603161944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a80a3e90 con 0x564e5aaba880 2024-06-10T00:40:48.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f708f7fe700 1 -- 172.21.15.62:0/2603161944 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f70a80bb2c0 con 0x564e5aaba880 2024-06-10T00:40:48.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f70c0f93b80 10 client.16758 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:48.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.140+0000 7f70c0f93b80 1 -- 172.21.15.62:0/2603161944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564e5aacbaf0 con 0x564e5aaba880 2024-06-10T00:40:48.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.141+0000 7f708f7fe700 1 -- 172.21.15.62:0/2603161944 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a80a3e90 con 0x564e5aaba880 2024-06-10T00:40:48.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.141+0000 7f708f7fe700 1 -- 172.21.15.62:0/2603161944 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f70a80a3a90 con 0x564e5aaba880 2024-06-10T00:40:48.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.141+0000 7f70c0f93b80 10 client.16758 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:48.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.141+0000 7f70c0f93b80 10 client.16758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:40:48.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.141+0000 7f70c0f93b80 20 client.16758 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:48.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.141+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/2603161944 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564e5aacdfc0 0x564e5aad03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.141+0000 7f70c0f93b80 4 client.16758 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:40:48.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.141+0000 7f70c0f93b80 1 -- 172.21.15.62:0/2603161944 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e5aa12d50 con 0x564e5aacdfc0 2024-06-10T00:40:48.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.141+0000 7f70affff700 1 --2- 172.21.15.62:0/2603161944 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564e5aacdfc0 0x564e5aad03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.141+0000 7f708f7fe700 10 client.16758 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:48.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.141+0000 7f70affff700 1 --2- 172.21.15.62:0/2603161944 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564e5aacdfc0 0x564e5aad03b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=424c4adc345d8232 server_cookie=979749c9bc80196e in_seq=0 out_seq=0 2024-06-10T00:40:48.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.142+0000 7f708f7fe700 1 -- 172.21.15.62:0/2603161944 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564e5aa12d50 con 0x564e5aacdfc0 2024-06-10T00:40:48.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.142+0000 7f708f7fe700 10 client.16758 handle_command_reply: tid=0 2024-06-10T00:40:48.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.142+0000 7f70c0f93b80 1 client.16758 shutdown 2024-06-10T00:40:48.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.142+0000 7f70c0f93b80 1 -- 172.21.15.62:0/2603161944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aaba880 msgr2=0x564e5aabdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:48.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.142+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/2603161944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aaba880 0x564e5aabdf20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f70a800d700 tx=0x7f70a800d8a0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.142+0000 7f70c0f93b80 1 -- 172.21.15.62:0/2603161944 shutdown_connections 2024-06-10T00:40:48.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.142+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/2603161944 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564e5aacdfc0 0x564e5aad03b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.142+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/2603161944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aaba880 0x564e5aabdf20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.142+0000 7f708f7fe700 0 client.16758 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:48.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.142+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/2603161944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aab47a0 0x564e5aac06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.142+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/2603161944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 0x564e5aabd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.142+0000 7f70c0f93b80 1 -- 172.21.15.62:0/2603161944 >> 172.21.15.62:0/2603161944 conn(0x564e5a9e1080 msgr2=0x564e5aaae700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:48.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.143+0000 7f70c0f93b80 1 -- 172.21.15.62:0/2603161944 shutdown_connections 2024-06-10T00:40:48.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.143+0000 7f70c0f93b80 1 -- 172.21.15.62:0/2603161944 wait complete. 2024-06-10T00:40:48.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.143+0000 7f70c0f93b80 20 client.16758 trim_cache size 0 max 16384 2024-06-10T00:40:48.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.163+0000 7f70c0f93b80 1 Processor -- start 2024-06-10T00:40:48.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.163+0000 7f70c0f93b80 1 -- start start 2024-06-10T00:40:48.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70c0f93b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5aab4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70c0f93b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 0x564e5aaaee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70c0f93b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aaba880 0x564e5aaaf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70c0f93b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01b0990 con 0x564e5aab47a0 2024-06-10T00:40:48.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70c0f93b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01b24d0 con 0x564e5aab9f60 2024-06-10T00:40:48.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70c0f93b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01b1060 con 0x564e5aaba880 2024-06-10T00:40:48.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70affff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5aab4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70affff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5aab4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50054/0 (socket says 172.21.15.62:50054) 2024-06-10T00:40:48.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70affff700 1 -- 172.21.15.62:0/244168005 learned_addr learned my addr 172.21.15.62:0/244168005 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:48.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70b4cad700 1 --2- 172.21.15.62:0/244168005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aaba880 0x564e5aaaf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70a7fff700 1 --2- 172.21.15.62:0/244168005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 0x564e5aaaee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70affff700 1 -- 172.21.15.62:0/244168005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aaba880 msgr2=0x564e5aaaf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:48.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70affff700 1 --2- 172.21.15.62:0/244168005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aaba880 0x564e5aaaf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70affff700 1 -- 172.21.15.62:0/244168005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 msgr2=0x564e5aaaee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:48.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70affff700 1 --2- 172.21.15.62:0/244168005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 0x564e5aaaee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70affff700 1 -- 172.21.15.62:0/244168005 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e5aab0040 con 0x564e5aab47a0 2024-06-10T00:40:48.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70affff700 1 --2- 172.21.15.62:0/244168005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5aab4260 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f70a000fdd0 tx=0x7f70a0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b13c6b0f2666cc7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:48.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70a5ffb700 1 -- 172.21.15.62:0/244168005 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0016070 con 0x564e5aab47a0 2024-06-10T00:40:48.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70a5ffb700 1 -- 172.21.15.62:0/244168005 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a002c070 con 0x564e5aab47a0 2024-06-10T00:40:48.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.164+0000 7f70a5ffb700 1 -- 172.21.15.62:0/244168005 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a001c640 con 0x564e5aab47a0 2024-06-10T00:40:48.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.165+0000 7f70c0f93b80 1 -- 172.21.15.62:0/244168005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 msgr2=0x564e5aab4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:48.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.165+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/244168005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5aab4260 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f70a000fdd0 tx=0x7f70a0012000 comp rx=0 tx=0).stop 2024-06-10T00:40:48.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.165+0000 7f70c0f93b80 1 -- 172.21.15.62:0/244168005 shutdown_connections 2024-06-10T00:40:48.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.165+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/244168005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5aab4260 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.165+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/244168005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aaba880 0x564e5aaaf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.165+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/244168005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 0x564e5aaaee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.165+0000 7f70c0f93b80 1 -- 172.21.15.62:0/244168005 >> 172.21.15.62:0/244168005 conn(0x564e5a9e1080 msgr2=0x564e5aac9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:48.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.165+0000 7f70c0f93b80 1 -- 172.21.15.62:0/244168005 shutdown_connections 2024-06-10T00:40:48.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.165+0000 7f70c0f93b80 1 -- 172.21.15.62:0/244168005 wait complete. 2024-06-10T00:40:48.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.166+0000 7f70c0f93b80 1 -- start start 2024-06-10T00:40:48.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.166+0000 7f70c0f93b80 1 Processor -- start 2024-06-10T00:40:48.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.166+0000 7f70c0f93b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5ab312d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.166+0000 7f70c0f93b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 0x564e5ab31810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.166+0000 7f70c0f93b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aaba880 0x564e5aad30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.166+0000 7f70c0f93b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01b1e00 con 0x564e5aab47a0 2024-06-10T00:40:48.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.166+0000 7f70c0f93b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01b1730 con 0x564e5aab9f60 2024-06-10T00:40:48.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.166+0000 7f70c0f93b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01ae950 con 0x564e5aaba880 2024-06-10T00:40:48.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.166+0000 7f70c0f93b80 10 client.0 _ll_register_callbacks cb 0x564e5a99b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:40:48.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.166+0000 7f70affff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5ab312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.166+0000 7f70affff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5ab312d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50058/0 (socket says 172.21.15.62:50058) 2024-06-10T00:40:48.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.166+0000 7f70affff700 1 -- 172.21.15.62:0/3572726000 learned_addr learned my addr 172.21.15.62:0/3572726000 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:40:48.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.166+0000 7f70b4cad700 1 --2- 172.21.15.62:0/3572726000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aaba880 0x564e5aad30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.166+0000 7f70a7fff700 1 --2- 172.21.15.62:0/3572726000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 0x564e5ab31810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f70affff700 1 -- 172.21.15.62:0/3572726000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aaba880 msgr2=0x564e5aad30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:48.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f70affff700 1 --2- 172.21.15.62:0/3572726000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aaba880 0x564e5aad30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f70affff700 1 -- 172.21.15.62:0/3572726000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 msgr2=0x564e5ab31810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:40:48.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f70affff700 1 --2- 172.21.15.62:0/3572726000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 0x564e5ab31810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f70affff700 1 -- 172.21.15.62:0/3572726000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e5aad37b0 con 0x564e5aab47a0 2024-06-10T00:40:48.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f70b4cad700 1 --2- 172.21.15.62:0/3572726000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aaba880 0x564e5aad30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:40:48.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f70affff700 1 --2- 172.21.15.62:0/3572726000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5ab312d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f70a0029040 tx=0x7f70a0026720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d89372ce11930d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:40:48.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f708f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:40:48.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f708f7fe700 1 -- 172.21.15.62:0/3572726000 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0011d90 con 0x564e5aab47a0 2024-06-10T00:40:48.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f70c0f93b80 1 -- 172.21.15.62:0/3572726000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a002c070 con 0x564e5aab47a0 2024-06-10T00:40:48.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f708f7fe700 1 -- 172.21.15.62:0/3572726000 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9554+0+0 (secure 0 0 0) 0x7f70a0005040 con 0x564e5aab47a0 2024-06-10T00:40:48.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f708f7fe700 1 -- 172.21.15.62:0/3572726000 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f70a0026950 con 0x564e5aab47a0 2024-06-10T00:40:48.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f70c0f93b80 10 client.16764 fetch_fsmap learned FSMap version 38 2024-06-10T00:40:48.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f70c0f93b80 1 -- 172.21.15.62:0/3572726000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564e5ab38b10 con 0x564e5aab47a0 2024-06-10T00:40:48.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.167+0000 7f708f7fe700 1 -- 172.21.15.62:0/3572726000 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a002c070 con 0x564e5aab47a0 2024-06-10T00:40:48.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.168+0000 7f708f7fe700 1 -- 172.21.15.62:0/3572726000 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f70a001c420 con 0x564e5aab47a0 2024-06-10T00:40:48.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.168+0000 7f70c0f93b80 10 client.16764 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:40:48.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.168+0000 7f70c0f93b80 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:40:48.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.168+0000 7f70c0f93b80 20 client.16764 populate_metadata read hostname 'smithi062' 2024-06-10T00:40:48.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.168+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/3572726000 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564e5aacdfc0 0x564e5ab3b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:40:48.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.168+0000 7f70c0f93b80 4 client.16764 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:40:48.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.168+0000 7f70c0f93b80 1 -- 172.21.15.62:0/3572726000 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e5aa0d2d0 con 0x564e5aacdfc0 2024-06-10T00:40:48.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.168+0000 7f70a7fff700 1 --2- 172.21.15.62:0/3572726000 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564e5aacdfc0 0x564e5ab3b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:40:48.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.168+0000 7f70a7fff700 1 --2- 172.21.15.62:0/3572726000 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564e5aacdfc0 0x564e5ab3b160 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff15bc2d5b978571 server_cookie=a9451f874d1fecd7 in_seq=0 out_seq=0 2024-06-10T00:40:48.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.168+0000 7f708f7fe700 10 client.16764 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:48.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f708f7fe700 1 -- 172.21.15.62:0/3572726000 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564e5aa0d2d0 con 0x564e5aacdfc0 2024-06-10T00:40:48.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f708f7fe700 10 client.16764 handle_command_reply: tid=0 2024-06-10T00:40:48.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f70c0f93b80 1 client.16764 shutdown 2024-06-10T00:40:48.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f70c0f93b80 1 -- 172.21.15.62:0/3572726000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 msgr2=0x564e5ab312d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:48.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/3572726000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5ab312d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f70a0029040 tx=0x7f70a0026720 comp rx=0 tx=0).stop 2024-06-10T00:40:48.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f70c0f93b80 1 -- 172.21.15.62:0/3572726000 shutdown_connections 2024-06-10T00:40:48.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/3572726000 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x564e5aacdfc0 0x564e5ab3b160 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/3572726000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e5aab47a0 0x564e5ab312d0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/3572726000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e5aaba880 0x564e5aad30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f70c0f93b80 1 --2- 172.21.15.62:0/3572726000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e5aab9f60 0x564e5ab31810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f708f7fe700 0 client.16764 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:40:48.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f70c0f93b80 1 -- 172.21.15.62:0/3572726000 >> 172.21.15.62:0/3572726000 conn(0x564e5a9e1080 msgr2=0x564e5aac94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:48.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f70c0f93b80 1 -- 172.21.15.62:0/3572726000 shutdown_connections 2024-06-10T00:40:48.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f70c0f93b80 1 -- 172.21.15.62:0/3572726000 wait complete. 2024-06-10T00:40:48.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.169+0000 7f70c0f93b80 20 client.16764 trim_cache size 0 max 16384 2024-06-10T00:40:48.179 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:40:48.179 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T00:40:48.179 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-06-10T00:40:48.180 INFO:teuthology.orchestra.run.smithi062.stdout: "51eccb99-9889-4271-bc5d-d5d2dc1a8951": { 2024-06-10T00:40:48.180 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-06-10T00:40:48.180 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "51eccb99-9889-4271-bc5d-d5d2dc1a8951", 2024-06-10T00:40:48.180 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-06-10T00:40:48.180 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:40:48.180 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:40:48.180 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:40:48.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.170+0000 7f70b6710700 1 -- 172.21.15.62:0/2297057709 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f709c0a2e10 msgr2=0x7f709c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:48.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.170+0000 7f70b6710700 1 --2- 172.21.15.62:0/2297057709 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f709c0a2e10 0x7f709c0a52d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f70a000ad20 tx=0x7f70a0005f90 comp rx=0 tx=0).stop 2024-06-10T00:40:48.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.170+0000 7f70b6710700 1 -- 172.21.15.62:0/2297057709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0107840 msgr2=0x7f70b0075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:40:48.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.170+0000 7f70b6710700 1 --2- 172.21.15.62:0/2297057709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0107840 0x7f70b0075210 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f70a8007f10 tx=0x7f70a800d3b0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.170+0000 7f70b6710700 1 -- 172.21.15.62:0/2297057709 shutdown_connections 2024-06-10T00:40:48.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.170+0000 7f70b6710700 1 --2- 172.21.15.62:0/2297057709 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f709c0a2e10 0x7f709c0a52d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.170+0000 7f70b6710700 1 --2- 172.21.15.62:0/2297057709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70b0107840 0x7f70b0075210 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.170+0000 7f70b6710700 1 --2- 172.21.15.62:0/2297057709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b0102740 0x7f70b0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.170+0000 7f70b6710700 1 --2- 172.21.15.62:0/2297057709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b01030f0 0x7f70b0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:40:48.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.170+0000 7f70b6710700 1 -- 172.21.15.62:0/2297057709 >> 172.21.15.62:0/2297057709 conn(0x7f70b00795f0 msgr2=0x7f70b0100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:40:48.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.170+0000 7f70b6710700 1 -- 172.21.15.62:0/2297057709 shutdown_connections 2024-06-10T00:40:48.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:40:48.170+0000 7f70b6710700 1 -- 172.21.15.62:0/2297057709 wait complete. 2024-06-10T00:40:48.947 INFO:tasks.cephfs.filesystem:scrub status for tag:51eccb99-9889-4271-bc5d-d5d2dc1a8951 - {'path': '/', 'tag': '51eccb99-9889-4271-bc5d-d5d2dc1a8951', 'options': 'recursive,force'} 2024-06-10T00:40:49.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:49 smithi105 ceph-mon[36321]: pgmap v244: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-06-10T00:40:49.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:49 smithi062 ceph-mon[32390]: pgmap v244: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-06-10T00:40:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:49 smithi138 ceph-mon[36549]: pgmap v244: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-06-10T00:40:51.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:51 smithi105 ceph-mon[36321]: pgmap v245: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:40:51.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:51 smithi062 ceph-mon[32390]: pgmap v245: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:40:51.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:51 smithi138 ceph-mon[36549]: pgmap v245: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:40:52.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:52 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:40:52.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:52 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:40:52.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:52 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:40:52.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:52 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:40:52.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:52 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:40:52.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:52 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:40:52.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:52 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:40:52.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:52 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:40:52.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:52 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:40:52.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:52 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:40:53.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:52 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:40:53.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:52 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:40:53.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:52 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:40:53.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:52 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:40:53.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:52 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:40:53.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:53 smithi062 ceph-mon[32390]: pgmap v246: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-06-10T00:40:53.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:53 smithi138 ceph-mon[36549]: pgmap v246: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-06-10T00:40:54.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:53 smithi105 ceph-mon[36321]: pgmap v246: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-06-10T00:40:55.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:55 smithi062 ceph-mon[32390]: pgmap v247: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:40:55.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:55 smithi138 ceph-mon[36549]: pgmap v247: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:40:56.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:55 smithi105 ceph-mon[36321]: pgmap v247: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:40:57.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:57 smithi062 ceph-mon[32390]: pgmap v248: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-10T00:40:57.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:57 smithi138 ceph-mon[36549]: pgmap v248: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-10T00:40:58.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:57 smithi105 ceph-mon[36321]: pgmap v248: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-10T00:40:59.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:40:59 smithi062 ceph-mon[32390]: pgmap v249: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-06-10T00:40:59.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:40:59 smithi138 ceph-mon[36549]: pgmap v249: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-06-10T00:41:00.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:40:59 smithi105 ceph-mon[36321]: pgmap v249: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-06-10T00:41:01.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:01 smithi062 ceph-mon[32390]: pgmap v250: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:41:01.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:01 smithi138 ceph-mon[36549]: pgmap v250: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:41:02.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:01 smithi105 ceph-mon[36321]: pgmap v250: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:41:03.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:03 smithi062 ceph-mon[32390]: pgmap v251: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:41:03.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:03 smithi138 ceph-mon[36549]: pgmap v251: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:41:04.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:03 smithi105 ceph-mon[36321]: pgmap v251: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:41:05.834 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:05 smithi105 ceph-mon[36321]: pgmap v252: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-06-10T00:41:05.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:05 smithi062 ceph-mon[32390]: pgmap v252: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-06-10T00:41:05.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:05 smithi138 ceph-mon[36549]: pgmap v252: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-06-10T00:41:07.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:07 smithi062 ceph-mon[32390]: pgmap v253: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-06-10T00:41:07.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:07 smithi138 ceph-mon[36549]: pgmap v253: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-06-10T00:41:08.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:07 smithi105 ceph-mon[36321]: pgmap v253: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-06-10T00:41:08.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:08 smithi062 ceph-mon[32390]: osdmap e103: 12 total, 12 up, 12 in 2024-06-10T00:41:08.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:08 smithi062 ceph-mon[32390]: pgmap v255: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-06-10T00:41:08.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:08 smithi138 ceph-mon[36549]: osdmap e103: 12 total, 12 up, 12 in 2024-06-10T00:41:08.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:08 smithi138 ceph-mon[36549]: pgmap v255: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-06-10T00:41:09.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:08 smithi105 ceph-mon[36321]: osdmap e103: 12 total, 12 up, 12 in 2024-06-10T00:41:09.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:08 smithi105 ceph-mon[36321]: pgmap v255: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-06-10T00:41:11.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:11 smithi105 ceph-mon[36321]: pgmap v256: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-06-10T00:41:11.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:11 smithi062 ceph-mon[32390]: pgmap v256: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-06-10T00:41:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:11 smithi138 ceph-mon[36549]: pgmap v256: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-06-10T00:41:13.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:13 smithi105 ceph-mon[36321]: pgmap v257: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-06-10T00:41:13.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:13 smithi062 ceph-mon[32390]: pgmap v257: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-06-10T00:41:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:13 smithi138 ceph-mon[36549]: pgmap v257: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-06-10T00:41:14.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:14 smithi105 ceph-mon[36321]: osdmap e104: 12 total, 12 up, 12 in 2024-06-10T00:41:14.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:14 smithi062 ceph-mon[32390]: osdmap e104: 12 total, 12 up, 12 in 2024-06-10T00:41:14.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:14 smithi138 ceph-mon[36549]: osdmap e104: 12 total, 12 up, 12 in 2024-06-10T00:41:15.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:15 smithi105 ceph-mon[36321]: pgmap v259: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 9 op/s 2024-06-10T00:41:15.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:15 smithi062 ceph-mon[32390]: pgmap v259: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 9 op/s 2024-06-10T00:41:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:15 smithi138 ceph-mon[36549]: pgmap v259: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 9 op/s 2024-06-10T00:41:17.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:17 smithi105 ceph-mon[36321]: pgmap v260: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 240 B/s wr, 9 op/s 2024-06-10T00:41:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:17 smithi062 ceph-mon[32390]: pgmap v260: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 240 B/s wr, 9 op/s 2024-06-10T00:41:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:17 smithi138 ceph-mon[36549]: pgmap v260: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 240 B/s wr, 9 op/s 2024-06-10T00:41:18.948 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:41:19.212 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:41:19.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:19 smithi062 ceph-mon[32390]: pgmap v261: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:41:19.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:19 smithi105 ceph-mon[36321]: pgmap v261: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:41:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:19 smithi138 ceph-mon[36549]: pgmap v261: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:41:21.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.030+0000 7f907c01e700 1 -- 172.21.15.62:0/141592927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074101d30 msgr2=0x7f907410e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:21.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.030+0000 7f907c01e700 1 --2- 172.21.15.62:0/141592927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074101d30 0x7f907410e1d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f906c009990 tx=0x7f906c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.030+0000 7f907c01e700 1 -- 172.21.15.62:0/141592927 shutdown_connections 2024-06-10T00:41:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.030+0000 7f907c01e700 1 --2- 172.21.15.62:0/141592927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074101d30 0x7f907410e1d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.030+0000 7f907c01e700 1 --2- 172.21.15.62:0/141592927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074107420 0x7f9074107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.030+0000 7f907c01e700 1 --2- 172.21.15.62:0/141592927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9074101370 0x7f90741017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.030+0000 7f907c01e700 1 -- 172.21.15.62:0/141592927 >> 172.21.15.62:0/141592927 conn(0x7f9074078340 msgr2=0x7f9074078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.030+0000 7f907c01e700 1 -- 172.21.15.62:0/141592927 shutdown_connections 2024-06-10T00:41:21.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.030+0000 7f907c01e700 1 -- 172.21.15.62:0/141592927 wait complete. 2024-06-10T00:41:21.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.030+0000 7f907c01e700 1 Processor -- start 2024-06-10T00:41:21.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.030+0000 7f907c01e700 1 -- start start 2024-06-10T00:41:21.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907c01e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9074101370 0x7f907419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907c01e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074101d30 0x7f907419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907c01e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074107420 0x7f90741988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907c01e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f90741116a0 con 0x7f9074107420 2024-06-10T00:41:21.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907c01e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9074111520 con 0x7f9074101370 2024-06-10T00:41:21.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907c01e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9074111820 con 0x7f9074101d30 2024-06-10T00:41:21.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907a5bb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074107420 0x7f90741988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907a5bb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074107420 0x7f90741988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58022/0 (socket says 172.21.15.62:58022) 2024-06-10T00:41:21.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907a5bb700 1 -- 172.21.15.62:0/3538834673 learned_addr learned my addr 172.21.15.62:0/3538834673 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:21.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f90795b9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074101d30 0x7f907419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f9079dba700 1 --2- 172.21.15.62:0/3538834673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9074101370 0x7f907419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907a5bb700 1 -- 172.21.15.62:0/3538834673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074101d30 msgr2=0x7f907419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:21.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907a5bb700 1 --2- 172.21.15.62:0/3538834673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074101d30 0x7f907419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907a5bb700 1 -- 172.21.15.62:0/3538834673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9074101370 msgr2=0x7f907419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:21.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907a5bb700 1 --2- 172.21.15.62:0/3538834673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9074101370 0x7f907419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907a5bb700 1 -- 172.21.15.62:0/3538834673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9074199050 con 0x7f9074107420 2024-06-10T00:41:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.031+0000 7f907a5bb700 1 --2- 172.21.15.62:0/3538834673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074107420 0x7f90741988f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f906c0047b0 tx=0x7f906c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.032+0000 7f9072ffd700 1 -- 172.21.15.62:0/3538834673 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c004830 con 0x7f9074107420 2024-06-10T00:41:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.032+0000 7f9072ffd700 1 -- 172.21.15.62:0/3538834673 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c005020 con 0x7f9074107420 2024-06-10T00:41:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.032+0000 7f907c01e700 1 -- 172.21.15.62:0/3538834673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90741992e0 con 0x7f9074107420 2024-06-10T00:41:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.032+0000 7f9072ffd700 1 -- 172.21.15.62:0/3538834673 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c003cc0 con 0x7f9074107420 2024-06-10T00:41:21.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.032+0000 7f907c01e700 1 -- 172.21.15.62:0/3538834673 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90741a5600 con 0x7f9074107420 2024-06-10T00:41:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.035+0000 7f9072ffd700 1 -- 172.21.15.62:0/3538834673 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f906c0049d0 con 0x7f9074107420 2024-06-10T00:41:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.035+0000 7f9072ffd700 1 --2- 172.21.15.62:0/3538834673 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f905c0a2e10 0x7f905c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.035+0000 7f9079dba700 1 --2- 172.21.15.62:0/3538834673 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f905c0a2e10 0x7f905c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.035+0000 7f9072ffd700 1 -- 172.21.15.62:0/3538834673 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f906c0d01b0 con 0x7f9074107420 2024-06-10T00:41:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.036+0000 7f9079dba700 1 --2- 172.21.15.62:0/3538834673 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f905c0a2e10 0x7f905c0a52d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9074199a30 tx=0x7f9064009180 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.036+0000 7f90868a1b80 1 Processor -- start 2024-06-10T00:41:21.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.036+0000 7f90868a1b80 1 -- start start 2024-06-10T00:41:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.036+0000 7f90868a1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ede470 0x560bd2ede850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.036+0000 7f90868a1b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ed8670 0x560bd2ed8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.036+0000 7f90868a1b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed9030 0x560bd2ee5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.036+0000 7f90868a1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560bd2dc6f60 con 0x560bd2ede470 2024-06-10T00:41:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.036+0000 7f90868a1b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560bd2e25850 con 0x560bd2ed8670 2024-06-10T00:41:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.036+0000 7f90868a1b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560bd2d8d460 con 0x560bd2ed9030 2024-06-10T00:41:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f9079dba700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ede470 0x560bd2ede850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f9079dba700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ede470 0x560bd2ede850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58028/0 (socket says 172.21.15.62:58028) 2024-06-10T00:41:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f9079dba700 1 -- 172.21.15.62:0/1508046210 learned_addr learned my addr 172.21.15.62:0/1508046210 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f90795b9700 1 --2- 172.21.15.62:0/1508046210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ed8670 0x560bd2ed8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f907a5bb700 1 --2- 172.21.15.62:0/1508046210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed9030 0x560bd2ee5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f9079dba700 1 -- 172.21.15.62:0/1508046210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed9030 msgr2=0x560bd2ee5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f9079dba700 1 --2- 172.21.15.62:0/1508046210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed9030 0x560bd2ee5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f9079dba700 1 -- 172.21.15.62:0/1508046210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ed8670 msgr2=0x560bd2ed8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f9079dba700 1 --2- 172.21.15.62:0/1508046210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ed8670 0x560bd2ed8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f9079dba700 1 -- 172.21.15.62:0/1508046210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd2d748d0 con 0x560bd2ede470 2024-06-10T00:41:21.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f907a5bb700 1 --2- 172.21.15.62:0/1508046210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed9030 0x560bd2ee5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:21.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f9079dba700 1 --2- 172.21.15.62:0/1508046210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ede470 0x560bd2ede850 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9064009550 tx=0x7f9064013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14560371e4ca783 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:21.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f905bfff700 1 -- 172.21.15.62:0/1508046210 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906401c050 con 0x560bd2ede470 2024-06-10T00:41:21.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f905bfff700 1 -- 172.21.15.62:0/1508046210 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9064026070 con 0x560bd2ede470 2024-06-10T00:41:21.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.037+0000 7f905bfff700 1 -- 172.21.15.62:0/1508046210 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90640156a0 con 0x560bd2ede470 2024-06-10T00:41:21.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.038+0000 7f90868a1b80 1 -- 172.21.15.62:0/1508046210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ede470 msgr2=0x560bd2ede850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:21.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.038+0000 7f90868a1b80 1 --2- 172.21.15.62:0/1508046210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ede470 0x560bd2ede850 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9064009550 tx=0x7f9064013000 comp rx=0 tx=0).stop 2024-06-10T00:41:21.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.038+0000 7f90868a1b80 1 -- 172.21.15.62:0/1508046210 shutdown_connections 2024-06-10T00:41:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.038+0000 7f90868a1b80 1 --2- 172.21.15.62:0/1508046210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ede470 0x560bd2ede850 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.038+0000 7f90868a1b80 1 --2- 172.21.15.62:0/1508046210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed9030 0x560bd2ee5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.038+0000 7f90868a1b80 1 --2- 172.21.15.62:0/1508046210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ed8670 0x560bd2ed8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.038+0000 7f90868a1b80 1 -- 172.21.15.62:0/1508046210 >> 172.21.15.62:0/1508046210 conn(0x560bd2e01080 msgr2=0x560bd2ecf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.038+0000 7f90868a1b80 1 -- 172.21.15.62:0/1508046210 shutdown_connections 2024-06-10T00:41:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.038+0000 7f90868a1b80 1 -- 172.21.15.62:0/1508046210 wait complete. 2024-06-10T00:41:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.039+0000 7f90868a1b80 1 -- start start 2024-06-10T00:41:21.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.039+0000 7f90868a1b80 1 Processor -- start 2024-06-10T00:41:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.039+0000 7f90868a1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ed8670 0x560bd2ee4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.039+0000 7f90868a1b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed9030 0x560bd2ee2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.039+0000 7f90868a1b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ede470 0x560bd2ee26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.039+0000 7f90868a1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560bd2dc5fc0 con 0x560bd2ed8670 2024-06-10T00:41:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.039+0000 7f90868a1b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560bd2e21810 con 0x560bd2ede470 2024-06-10T00:41:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.039+0000 7f90868a1b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560bd2ee5750 con 0x560bd2ed9030 2024-06-10T00:41:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.039+0000 7f90868a1b80 10 client.0 _ll_register_callbacks cb 0x560bd2dbb5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.039+0000 7f9079dba700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ed8670 0x560bd2ee4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.039+0000 7f9079dba700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ed8670 0x560bd2ee4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58038/0 (socket says 172.21.15.62:58038) 2024-06-10T00:41:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.039+0000 7f9079dba700 1 -- 172.21.15.62:0/124487291 learned_addr learned my addr 172.21.15.62:0/124487291 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.039+0000 7f90795b9700 1 --2- 172.21.15.62:0/124487291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed9030 0x560bd2ee2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.039+0000 7f907a5bb700 1 --2- 172.21.15.62:0/124487291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ede470 0x560bd2ee26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f9079dba700 1 -- 172.21.15.62:0/124487291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed9030 msgr2=0x560bd2ee2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f9079dba700 1 --2- 172.21.15.62:0/124487291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed9030 0x560bd2ee2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f9079dba700 1 -- 172.21.15.62:0/124487291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ede470 msgr2=0x560bd2ee26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f9079dba700 1 --2- 172.21.15.62:0/124487291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ede470 0x560bd2ee26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f9079dba700 1 -- 172.21.15.62:0/124487291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd2deb750 con 0x560bd2ed8670 2024-06-10T00:41:21.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f90795b9700 1 --2- 172.21.15.62:0/124487291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed9030 0x560bd2ee2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:41:21.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f9079dba700 1 --2- 172.21.15.62:0/124487291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ed8670 0x560bd2ee4e80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f90640216f0 tx=0x7f9064021720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead73c0485f169d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:21.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f90597fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:21.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f90597fa700 1 -- 172.21.15.62:0/124487291 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064004c90 con 0x560bd2ed8670 2024-06-10T00:41:21.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f90597fa700 1 -- 172.21.15.62:0/124487291 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f9064015440 con 0x560bd2ed8670 2024-06-10T00:41:21.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f90868a1b80 1 -- 172.21.15.62:0/124487291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9064026070 con 0x560bd2ed8670 2024-06-10T00:41:21.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f90597fa700 1 -- 172.21.15.62:0/124487291 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f906403c050 con 0x560bd2ed8670 2024-06-10T00:41:21.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f90868a1b80 10 client.16776 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:21.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f90868a1b80 1 -- 172.21.15.62:0/124487291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560bd2eefd10 con 0x560bd2ed8670 2024-06-10T00:41:21.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.040+0000 7f90597fa700 1 -- 172.21.15.62:0/124487291 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9064026070 con 0x560bd2ed8670 2024-06-10T00:41:21.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.041+0000 7f90597fa700 1 -- 172.21.15.62:0/124487291 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f9064033580 con 0x560bd2ed8670 2024-06-10T00:41:21.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.041+0000 7f90868a1b80 10 client.16776 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:21.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.041+0000 7f90868a1b80 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:41:21.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.041+0000 7f90868a1b80 20 client.16776 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:21.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.041+0000 7f90868a1b80 1 --2- 172.21.15.62:0/124487291 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560bd2ef22f0 0x560bd2ef46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.041+0000 7f90868a1b80 4 client.16776 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:41:21.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.041+0000 7f90868a1b80 1 -- 172.21.15.62:0/124487291 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bd2e32d50 con 0x560bd2ef22f0 2024-06-10T00:41:21.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.041+0000 7f90795b9700 1 --2- 172.21.15.62:0/124487291 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560bd2ef22f0 0x560bd2ef46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.041+0000 7f90597fa700 10 client.16776 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:21.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.041+0000 7f90795b9700 1 --2- 172.21.15.62:0/124487291 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560bd2ef22f0 0x560bd2ef46e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9e87606b19a7ee9 server_cookie=d90cdddc5343665e in_seq=0 out_seq=0 2024-06-10T00:41:21.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.042+0000 7f90597fa700 1 -- 172.21.15.62:0/124487291 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560bd2e32d50 con 0x560bd2ef22f0 2024-06-10T00:41:21.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.042+0000 7f90597fa700 10 client.16776 handle_command_reply: tid=0 2024-06-10T00:41:21.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.042+0000 7f90868a1b80 1 client.16776 shutdown 2024-06-10T00:41:21.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.042+0000 7f90868a1b80 1 -- 172.21.15.62:0/124487291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ed8670 msgr2=0x560bd2ee4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:21.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.042+0000 7f90868a1b80 1 --2- 172.21.15.62:0/124487291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ed8670 0x560bd2ee4e80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f90640216f0 tx=0x7f9064021720 comp rx=0 tx=0).stop 2024-06-10T00:41:21.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.042+0000 7f90868a1b80 1 -- 172.21.15.62:0/124487291 shutdown_connections 2024-06-10T00:41:21.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.042+0000 7f90868a1b80 1 --2- 172.21.15.62:0/124487291 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560bd2ef22f0 0x560bd2ef46e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.042+0000 7f90868a1b80 1 --2- 172.21.15.62:0/124487291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ed8670 0x560bd2ee4e80 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.042+0000 7f90868a1b80 1 --2- 172.21.15.62:0/124487291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ede470 0x560bd2ee26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.042+0000 7f90597fa700 0 client.16776 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:21.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.042+0000 7f90868a1b80 1 --2- 172.21.15.62:0/124487291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed9030 0x560bd2ee2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.042+0000 7f90868a1b80 1 -- 172.21.15.62:0/124487291 >> 172.21.15.62:0/124487291 conn(0x560bd2e01080 msgr2=0x560bd2ece700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:21.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.043+0000 7f90868a1b80 1 -- 172.21.15.62:0/124487291 shutdown_connections 2024-06-10T00:41:21.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.043+0000 7f90868a1b80 1 -- 172.21.15.62:0/124487291 wait complete. 2024-06-10T00:41:21.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.043+0000 7f90868a1b80 20 client.16776 trim_cache size 0 max 16384 2024-06-10T00:41:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.063+0000 7f90868a1b80 1 Processor -- start 2024-06-10T00:41:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.063+0000 7f90868a1b80 1 -- start start 2024-06-10T00:41:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90868a1b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed8670 0x560bd2ee0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90868a1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ed9030 0x560bd2ee40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90868a1b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ede470 0x560bd2ecee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90868a1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f90741ac400 con 0x560bd2ed9030 2024-06-10T00:41:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90868a1b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90741adf40 con 0x560bd2ede470 2024-06-10T00:41:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90868a1b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90741acad0 con 0x560bd2ed8670 2024-06-10T00:41:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90795b9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ed9030 0x560bd2ee40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90795b9700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ed9030 0x560bd2ee40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58044/0 (socket says 172.21.15.62:58044) 2024-06-10T00:41:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90795b9700 1 -- 172.21.15.62:0/1723082201 learned_addr learned my addr 172.21.15.62:0/1723082201 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f9079dba700 1 --2- 172.21.15.62:0/1723082201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed8670 0x560bd2ee0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90795b9700 1 -- 172.21.15.62:0/1723082201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed8670 msgr2=0x560bd2ee0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f907a5bb700 1 --2- 172.21.15.62:0/1723082201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ede470 0x560bd2ecee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90795b9700 1 --2- 172.21.15.62:0/1723082201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed8670 0x560bd2ee0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90795b9700 1 -- 172.21.15.62:0/1723082201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ede470 msgr2=0x560bd2ecee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90795b9700 1 --2- 172.21.15.62:0/1723082201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ede470 0x560bd2ecee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90795b9700 1 -- 172.21.15.62:0/1723082201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd2ecfe40 con 0x560bd2ed9030 2024-06-10T00:41:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f907a5bb700 1 --2- 172.21.15.62:0/1723082201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ede470 0x560bd2ecee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:41:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f9079dba700 1 --2- 172.21.15.62:0/1723082201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed8670 0x560bd2ee0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:41:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.064+0000 7f90795b9700 1 --2- 172.21.15.62:0/1723082201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ed9030 0x560bd2ee40c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9068000c00 tx=0x7f906800a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d75999811d33d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.065+0000 7f905b7fe700 1 -- 172.21.15.62:0/1723082201 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068004310 con 0x560bd2ed9030 2024-06-10T00:41:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.065+0000 7f905b7fe700 1 -- 172.21.15.62:0/1723082201 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90680044b0 con 0x560bd2ed9030 2024-06-10T00:41:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.065+0000 7f905b7fe700 1 -- 172.21.15.62:0/1723082201 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068027b70 con 0x560bd2ed9030 2024-06-10T00:41:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.065+0000 7f90868a1b80 1 -- 172.21.15.62:0/1723082201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ed9030 msgr2=0x560bd2ee40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:21.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.065+0000 7f90868a1b80 1 --2- 172.21.15.62:0/1723082201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ed9030 0x560bd2ee40c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9068000c00 tx=0x7f906800a380 comp rx=0 tx=0).stop 2024-06-10T00:41:21.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.065+0000 7f90868a1b80 1 -- 172.21.15.62:0/1723082201 shutdown_connections 2024-06-10T00:41:21.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.065+0000 7f90868a1b80 1 --2- 172.21.15.62:0/1723082201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ede470 0x560bd2ecee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.065+0000 7f90868a1b80 1 --2- 172.21.15.62:0/1723082201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed8670 0x560bd2ee0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.065+0000 7f90868a1b80 1 --2- 172.21.15.62:0/1723082201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ed9030 0x560bd2ee40c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.065+0000 7f90868a1b80 1 -- 172.21.15.62:0/1723082201 >> 172.21.15.62:0/1723082201 conn(0x560bd2e01080 msgr2=0x560bd2ee39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:21.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.065+0000 7f90868a1b80 1 -- 172.21.15.62:0/1723082201 shutdown_connections 2024-06-10T00:41:21.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.065+0000 7f90868a1b80 1 -- 172.21.15.62:0/1723082201 wait complete. 2024-06-10T00:41:21.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.066+0000 7f90868a1b80 1 -- start start 2024-06-10T00:41:21.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.066+0000 7f90868a1b80 1 Processor -- start 2024-06-10T00:41:21.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.066+0000 7f90868a1b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed8670 0x560bd2ef98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.066+0000 7f90868a1b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ed9030 0x560bd2ef9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.066+0000 7f90868a1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ede470 0x560bd2efa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.066+0000 7f90868a1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f90741ad870 con 0x560bd2ede470 2024-06-10T00:41:21.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.066+0000 7f90868a1b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90741ad1a0 con 0x560bd2ed9030 2024-06-10T00:41:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.066+0000 7f90868a1b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90741aa3c0 con 0x560bd2ed8670 2024-06-10T00:41:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.066+0000 7f90868a1b80 10 client.0 _ll_register_callbacks cb 0x560bd2dbb5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.066+0000 7f907a5bb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ede470 0x560bd2efa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.066+0000 7f907a5bb700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ede470 0x560bd2efa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58048/0 (socket says 172.21.15.62:58048) 2024-06-10T00:41:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.066+0000 7f907a5bb700 1 -- 172.21.15.62:0/212770079 learned_addr learned my addr 172.21.15.62:0/212770079 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.066+0000 7f9079dba700 1 --2- 172.21.15.62:0/212770079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed8670 0x560bd2ef98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f90795b9700 1 --2- 172.21.15.62:0/212770079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ed9030 0x560bd2ef9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f907a5bb700 1 -- 172.21.15.62:0/212770079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed8670 msgr2=0x560bd2ef98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f907a5bb700 1 --2- 172.21.15.62:0/212770079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed8670 0x560bd2ef98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f907a5bb700 1 -- 172.21.15.62:0/212770079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ed9030 msgr2=0x560bd2ef9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f907a5bb700 1 --2- 172.21.15.62:0/212770079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ed9030 0x560bd2ef9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f907a5bb700 1 -- 172.21.15.62:0/212770079 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd2f5bc50 con 0x560bd2ede470 2024-06-10T00:41:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f907a5bb700 1 --2- 172.21.15.62:0/212770079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ede470 0x560bd2efa330 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f906c0cdaf0 tx=0x7f906c09bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693e7efbe70d5d0b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f90597fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f90597fa700 1 -- 172.21.15.62:0/212770079 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c0a0700 con 0x560bd2ede470 2024-06-10T00:41:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f90597fa700 1 -- 172.21.15.62:0/212770079 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f906c0a4aa0 con 0x560bd2ede470 2024-06-10T00:41:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f90868a1b80 1 -- 172.21.15.62:0/212770079 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90680044b0 con 0x560bd2ede470 2024-06-10T00:41:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f90597fa700 1 -- 172.21.15.62:0/212770079 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f906c0a4f30 con 0x560bd2ede470 2024-06-10T00:41:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f90868a1b80 10 client.16782 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f90868a1b80 1 -- 172.21.15.62:0/212770079 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560bd2f5cec0 con 0x560bd2ede470 2024-06-10T00:41:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.067+0000 7f90597fa700 1 -- 172.21.15.62:0/212770079 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90680044b0 con 0x560bd2ede470 2024-06-10T00:41:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.068+0000 7f90597fa700 1 -- 172.21.15.62:0/212770079 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f906c0bc0b0 con 0x560bd2ede470 2024-06-10T00:41:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.068+0000 7f90868a1b80 10 client.16782 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.068+0000 7f90868a1b80 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:41:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.068+0000 7f90868a1b80 20 client.16782 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:21.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.068+0000 7f90868a1b80 1 --2- 172.21.15.62:0/212770079 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560bd2ef22f0 0x560bd2f5f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:21.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.068+0000 7f90868a1b80 4 client.16782 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:41:21.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.068+0000 7f90868a1b80 1 -- 172.21.15.62:0/212770079 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560bd2e2d2d0 con 0x560bd2ef22f0 2024-06-10T00:41:21.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.068+0000 7f9079dba700 1 --2- 172.21.15.62:0/212770079 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560bd2ef22f0 0x560bd2f5f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:21.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.068+0000 7f9079dba700 1 --2- 172.21.15.62:0/212770079 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560bd2ef22f0 0x560bd2f5f660 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34ef146da490a9bf server_cookie=f12f3206de41d48e in_seq=0 out_seq=0 2024-06-10T00:41:21.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.068+0000 7f90597fa700 10 client.16782 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:21.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.068+0000 7f90597fa700 1 -- 172.21.15.62:0/212770079 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560bd2e2d2d0 con 0x560bd2ef22f0 2024-06-10T00:41:21.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.068+0000 7f90597fa700 10 client.16782 handle_command_reply: tid=0 2024-06-10T00:41:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.068+0000 7f90868a1b80 1 client.16782 shutdown 2024-06-10T00:41:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f90868a1b80 1 -- 172.21.15.62:0/212770079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ede470 msgr2=0x560bd2efa330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f90868a1b80 1 --2- 172.21.15.62:0/212770079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ede470 0x560bd2efa330 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f906c0cdaf0 tx=0x7f906c09bd30 comp rx=0 tx=0).stop 2024-06-10T00:41:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f90868a1b80 1 -- 172.21.15.62:0/212770079 shutdown_connections 2024-06-10T00:41:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f90868a1b80 1 --2- 172.21.15.62:0/212770079 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x560bd2ef22f0 0x560bd2f5f660 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f90868a1b80 1 --2- 172.21.15.62:0/212770079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560bd2ede470 0x560bd2efa330 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f90868a1b80 1 --2- 172.21.15.62:0/212770079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd2ed8670 0x560bd2ef98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f90868a1b80 1 --2- 172.21.15.62:0/212770079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560bd2ed9030 0x560bd2ef9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f90597fa700 0 client.16782 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:21.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f90868a1b80 1 -- 172.21.15.62:0/212770079 >> 172.21.15.62:0/212770079 conn(0x560bd2e01080 msgr2=0x560bd2ed3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:21.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f90868a1b80 1 -- 172.21.15.62:0/212770079 shutdown_connections 2024-06-10T00:41:21.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f90868a1b80 1 -- 172.21.15.62:0/212770079 wait complete. 2024-06-10T00:41:21.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f90868a1b80 20 client.16782 trim_cache size 0 max 16384 2024-06-10T00:41:21.080 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:41:21.081 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-06-10T00:41:21.081 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-06-10T00:41:21.081 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:41:21.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f907c01e700 1 -- 172.21.15.62:0/3538834673 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f905c0a2e10 msgr2=0x7f905c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:21.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f907c01e700 1 --2- 172.21.15.62:0/3538834673 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f905c0a2e10 0x7f905c0a52d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9074199a30 tx=0x7f9064009180 comp rx=0 tx=0).stop 2024-06-10T00:41:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f907c01e700 1 -- 172.21.15.62:0/3538834673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074107420 msgr2=0x7f90741988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.069+0000 7f907c01e700 1 --2- 172.21.15.62:0/3538834673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074107420 0x7f90741988f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f906c0047b0 tx=0x7f906c005ef0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.070+0000 7f907c01e700 1 -- 172.21.15.62:0/3538834673 shutdown_connections 2024-06-10T00:41:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.070+0000 7f907c01e700 1 --2- 172.21.15.62:0/3538834673 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f905c0a2e10 0x7f905c0a52d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.070+0000 7f907c01e700 1 --2- 172.21.15.62:0/3538834673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074107420 0x7f90741988f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.070+0000 7f907c01e700 1 --2- 172.21.15.62:0/3538834673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9074101370 0x7f907419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.070+0000 7f907c01e700 1 --2- 172.21.15.62:0/3538834673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074101d30 0x7f907419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:21.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.070+0000 7f907c01e700 1 -- 172.21.15.62:0/3538834673 >> 172.21.15.62:0/3538834673 conn(0x7f9074078340 msgr2=0x7f90741005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:21.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.070+0000 7f907c01e700 1 -- 172.21.15.62:0/3538834673 shutdown_connections 2024-06-10T00:41:21.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:21.070+0000 7f907c01e700 1 -- 172.21.15.62:0/3538834673 wait complete. 2024-06-10T00:41:21.767 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:21 smithi062 ceph-mon[32390]: pgmap v262: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:41:21.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:21 smithi105 ceph-mon[36321]: pgmap v262: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:41:21.797 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T00:41:21.797 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:41:21.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:21 smithi138 ceph-mon[36549]: pgmap v262: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:41:22.049 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:41:23.521 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:23 smithi062 ceph-mon[32390]: pgmap v263: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:41:23.521 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:23 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:41:23.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.764+0000 7f8e39227700 1 -- 172.21.15.62:0/1769395755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e34102770 msgr2=0x7f8e34102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:23.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.764+0000 7f8e39227700 1 --2- 172.21.15.62:0/1769395755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e34102770 0x7f8e34102b50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8e240099c0 tx=0x7f8e24009cd0 comp rx=0 tx=0).stop 2024-06-10T00:41:23.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.764+0000 7f8e39227700 1 -- 172.21.15.62:0/1769395755 shutdown_connections 2024-06-10T00:41:23.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.764+0000 7f8e39227700 1 --2- 172.21.15.62:0/1769395755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e34102770 0x7f8e34102b50 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:23.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.764+0000 7f8e39227700 1 --2- 172.21.15.62:0/1769395755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e34107840 0x7f8e3410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:23.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.764+0000 7f8e39227700 1 --2- 172.21.15.62:0/1769395755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e34103120 0x7f8e34107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:23.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.764+0000 7f8e39227700 1 -- 172.21.15.62:0/1769395755 >> 172.21.15.62:0/1769395755 conn(0x7f8e34079500 msgr2=0x7f8e34079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.764+0000 7f8e39227700 1 -- 172.21.15.62:0/1769395755 shutdown_connections 2024-06-10T00:41:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.764+0000 7f8e39227700 1 -- 172.21.15.62:0/1769395755 wait complete. 2024-06-10T00:41:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.765+0000 7f8e39227700 1 Processor -- start 2024-06-10T00:41:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.765+0000 7f8e39227700 1 -- start start 2024-06-10T00:41:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.765+0000 7f8e39227700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e34102770 0x7f8e3419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.765+0000 7f8e39227700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e34103120 0x7f8e3419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.765+0000 7f8e39227700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e34107840 0x7f8e341989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.765+0000 7f8e39227700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8e34112760 con 0x7f8e34102770 2024-06-10T00:41:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.765+0000 7f8e39227700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341125e0 con 0x7f8e34103120 2024-06-10T00:41:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.765+0000 7f8e39227700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341128e0 con 0x7f8e34107840 2024-06-10T00:41:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.766+0000 7f8e32d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e34102770 0x7f8e3419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.766+0000 7f8e32d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e34102770 0x7f8e3419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58070/0 (socket says 172.21.15.62:58070) 2024-06-10T00:41:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.766+0000 7f8e32d9d700 1 -- 172.21.15.62:0/2605368496 learned_addr learned my addr 172.21.15.62:0/2605368496 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.766+0000 7f8e3259c700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e34103120 0x7f8e3419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.766+0000 7f8e3359e700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e34107840 0x7f8e341989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.766+0000 7f8e32d9d700 1 -- 172.21.15.62:0/2605368496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e34107840 msgr2=0x7f8e341989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.766+0000 7f8e32d9d700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e34107840 0x7f8e341989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.766+0000 7f8e32d9d700 1 -- 172.21.15.62:0/2605368496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e34103120 msgr2=0x7f8e3419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.766+0000 7f8e32d9d700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e34103120 0x7f8e3419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.766+0000 7f8e32d9d700 1 -- 172.21.15.62:0/2605368496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e34199100 con 0x7f8e34102770 2024-06-10T00:41:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.766+0000 7f8e3259c700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e34103120 0x7f8e3419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:41:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.766+0000 7f8e32d9d700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e34102770 0x7f8e3419e7e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8e24006010 tx=0x7f8e24009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.767+0000 7f8e1bfff700 1 -- 172.21.15.62:0/2605368496 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24026070 con 0x7f8e34102770 2024-06-10T00:41:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.767+0000 7f8e39227700 1 -- 172.21.15.62:0/2605368496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e34199390 con 0x7f8e34102770 2024-06-10T00:41:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.767+0000 7f8e1bfff700 1 -- 172.21.15.62:0/2605368496 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e24004cd0 con 0x7f8e34102770 2024-06-10T00:41:23.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.767+0000 7f8e1bfff700 1 -- 172.21.15.62:0/2605368496 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24003ba0 con 0x7f8e34102770 2024-06-10T00:41:23.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.767+0000 7f8e39227700 1 -- 172.21.15.62:0/2605368496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e341a56b0 con 0x7f8e34102770 2024-06-10T00:41:23.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.768+0000 7f8e39227700 1 -- 172.21.15.62:0/2605368496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e34069180 con 0x7f8e34102770 2024-06-10T00:41:23.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.770+0000 7f8e1bfff700 1 -- 172.21.15.62:0/2605368496 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8e24029070 con 0x7f8e34102770 2024-06-10T00:41:23.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.771+0000 7f8e1bfff700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8e1c0a2d90 0x7f8e1c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:23.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.771+0000 7f8e1bfff700 1 -- 172.21.15.62:0/2605368496 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f8e240d0540 con 0x7f8e34102770 2024-06-10T00:41:23.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.771+0000 7f8e3259c700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8e1c0a2d90 0x7f8e1c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:23.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.771+0000 7f8e3259c700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8e1c0a2d90 0x7f8e1c0a5250 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8e34101ff0 tx=0x7f8e280073d0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:23.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:23.777+0000 7f8e1bfff700 1 -- 172.21.15.62:0/2605368496 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8e24030020 con 0x7f8e34102770 2024-06-10T00:41:23.781 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:23 smithi105 ceph-mon[36321]: pgmap v263: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:41:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:23 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:41:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:23 smithi138 ceph-mon[36549]: pgmap v263: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:41:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:23 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:41:24.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:24.031+0000 7f8e39227700 1 -- 172.21.15.62:0/2605368496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e3402dca0 con 0x7f8e34102770 2024-06-10T00:41:24.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:24.032+0000 7f8e1bfff700 1 -- 172.21.15.62:0/2605368496 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11938 (secure 0 0 0) 0x7f8e24013070 con 0x7f8e34102770 2024-06-10T00:41:24.033 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:41:24.034 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:40:10.974660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:41:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:24.034+0000 7f8e39227700 1 -- 172.21.15.62:0/2605368496 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8e1c0a2d90 msgr2=0x7f8e1c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:24.034+0000 7f8e39227700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8e1c0a2d90 0x7f8e1c0a5250 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8e34101ff0 tx=0x7f8e280073d0 comp rx=0 tx=0).stop 2024-06-10T00:41:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:24.034+0000 7f8e39227700 1 -- 172.21.15.62:0/2605368496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e34102770 msgr2=0x7f8e3419e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:24.034+0000 7f8e39227700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e34102770 0x7f8e3419e7e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8e24006010 tx=0x7f8e24009f80 comp rx=0 tx=0).stop 2024-06-10T00:41:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:24.034+0000 7f8e39227700 1 -- 172.21.15.62:0/2605368496 shutdown_connections 2024-06-10T00:41:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:24.034+0000 7f8e39227700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8e1c0a2d90 0x7f8e1c0a5250 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:24.034+0000 7f8e39227700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e34102770 0x7f8e3419e7e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:24.034+0000 7f8e39227700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e34107840 0x7f8e341989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:24.034+0000 7f8e39227700 1 --2- 172.21.15.62:0/2605368496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e34103120 0x7f8e3419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:24.034+0000 7f8e39227700 1 -- 172.21.15.62:0/2605368496 >> 172.21.15.62:0/2605368496 conn(0x7f8e34079500 msgr2=0x7f8e341005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:24.034+0000 7f8e39227700 1 -- 172.21.15.62:0/2605368496 shutdown_connections 2024-06-10T00:41:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:24.034+0000 7f8e39227700 1 -- 172.21.15.62:0/2605368496 wait complete. 2024-06-10T00:41:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 38 2024-06-10T00:41:24.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:24 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:41:24.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:24 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:41:24.737 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:24 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:41:24.738 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:24 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2605368496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:41:24.767 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T00:41:24.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:24 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:41:24.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:24 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:41:24.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:24 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:41:24.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:24 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2605368496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:41:24.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:24 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:41:24.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:24 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:41:24.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:24 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:41:24.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:24 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2605368496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:41:25.016 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:41:25.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:25 smithi105 ceph-mon[36321]: pgmap v264: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 96 B/s wr, 10 op/s 2024-06-10T00:41:25.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:25 smithi062 ceph-mon[32390]: pgmap v264: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 96 B/s wr, 10 op/s 2024-06-10T00:41:25.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:25 smithi138 ceph-mon[36549]: pgmap v264: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 96 B/s wr, 10 op/s 2024-06-10T00:41:26.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.732+0000 7fde00e10700 1 -- 172.21.15.62:0/1449064892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc102740 msgr2=0x7fddfc102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:26.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.732+0000 7fde00e10700 1 --2- 172.21.15.62:0/1449064892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc102740 0x7fddfc102b20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fddec0099c0 tx=0x7fddec009cd0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.732+0000 7fde00e10700 1 -- 172.21.15.62:0/1449064892 shutdown_connections 2024-06-10T00:41:26.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.732+0000 7fde00e10700 1 --2- 172.21.15.62:0/1449064892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc102740 0x7fddfc102b20 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.732+0000 7fde00e10700 1 --2- 172.21.15.62:0/1449064892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddfc107840 0x7fddfc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.732+0000 7fde00e10700 1 --2- 172.21.15.62:0/1449064892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc1030f0 0x7fddfc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.732+0000 7fde00e10700 1 -- 172.21.15.62:0/1449064892 >> 172.21.15.62:0/1449064892 conn(0x7fddfc0795f0 msgr2=0x7fddfc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:26.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.732+0000 7fde00e10700 1 -- 172.21.15.62:0/1449064892 shutdown_connections 2024-06-10T00:41:26.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.732+0000 7fde00e10700 1 -- 172.21.15.62:0/1449064892 wait complete. 2024-06-10T00:41:26.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.733+0000 7fde00e10700 1 Processor -- start 2024-06-10T00:41:26.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.733+0000 7fde00e10700 1 -- start start 2024-06-10T00:41:26.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.733+0000 7fde00e10700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc102740 0x7fddfc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.733+0000 7fde00e10700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc1030f0 0x7fddfc074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.733+0000 7fde00e10700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddfc107840 0x7fddfc075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.733+0000 7fde00e10700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc112760 con 0x7fddfc102740 2024-06-10T00:41:26.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.733+0000 7fde00e10700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1125e0 con 0x7fddfc107840 2024-06-10T00:41:26.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.733+0000 7fde00e10700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1128e0 con 0x7fddfc1030f0 2024-06-10T00:41:26.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.733+0000 7fddfa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc102740 0x7fddfc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.733+0000 7fddfa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc102740 0x7fddfc074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58096/0 (socket says 172.21.15.62:58096) 2024-06-10T00:41:26.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.733+0000 7fddfa59c700 1 -- 172.21.15.62:0/48740401 learned_addr learned my addr 172.21.15.62:0/48740401 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:26.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.733+0000 7fddf9d9b700 1 --2- 172.21.15.62:0/48740401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc1030f0 0x7fddfc074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.734+0000 7fddfad9d700 1 --2- 172.21.15.62:0/48740401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddfc107840 0x7fddfc075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.734+0000 7fddfa59c700 1 -- 172.21.15.62:0/48740401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc1030f0 msgr2=0x7fddfc074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:26.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.734+0000 7fddfa59c700 1 --2- 172.21.15.62:0/48740401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc1030f0 0x7fddfc074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.734+0000 7fddfa59c700 1 -- 172.21.15.62:0/48740401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddfc107840 msgr2=0x7fddfc075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:26.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.734+0000 7fddfa59c700 1 --2- 172.21.15.62:0/48740401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddfc107840 0x7fddfc075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.734+0000 7fddfa59c700 1 -- 172.21.15.62:0/48740401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc1a7190 con 0x7fddfc102740 2024-06-10T00:41:26.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.734+0000 7fddfa59c700 1 --2- 172.21.15.62:0/48740401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc102740 0x7fddfc074790 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fddec006010 tx=0x7fddec009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:26.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.734+0000 7fddeb7fe700 1 -- 172.21.15.62:0/48740401 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec027070 con 0x7fddfc102740 2024-06-10T00:41:26.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.734+0000 7fde00e10700 1 -- 172.21.15.62:0/48740401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddfc1a7420 con 0x7fddfc102740 2024-06-10T00:41:26.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.734+0000 7fddeb7fe700 1 -- 172.21.15.62:0/48740401 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddec003780 con 0x7fddfc102740 2024-06-10T00:41:26.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.734+0000 7fddeb7fe700 1 -- 172.21.15.62:0/48740401 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec022da0 con 0x7fddfc102740 2024-06-10T00:41:26.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.734+0000 7fde00e10700 1 -- 172.21.15.62:0/48740401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddfc1a78d0 con 0x7fddfc102740 2024-06-10T00:41:26.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.737+0000 7fddeb7fe700 1 -- 172.21.15.62:0/48740401 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fddec02a070 con 0x7fddfc102740 2024-06-10T00:41:26.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.738+0000 7fddeb7fe700 1 --2- 172.21.15.62:0/48740401 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fdde40a2d90 0x7fdde40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.738+0000 7fddf9d9b700 1 --2- 172.21.15.62:0/48740401 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fdde40a2d90 0x7fdde40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.738+0000 7fddeb7fe700 1 -- 172.21.15.62:0/48740401 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fddec0d0570 con 0x7fddfc102740 2024-06-10T00:41:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.738+0000 7fddf9d9b700 1 --2- 172.21.15.62:0/48740401 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fdde40a2d90 0x7fdde40a5250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fddfc102030 tx=0x7fddf000b410 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.738+0000 7fde0b693b80 1 Processor -- start 2024-06-10T00:41:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fde0b693b80 1 -- start start 2024-06-10T00:41:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fde0b693b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a142f640 0x55f6a142fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fde0b693b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142ff60 0x55f6a14299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fde0b693b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a1429f20 0x55f6a142a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fde0b693b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f6a137b810 con 0x55f6a142f640 2024-06-10T00:41:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fde0b693b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f6a1323720 con 0x55f6a1429f20 2024-06-10T00:41:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fde0b693b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6a13777d0 con 0x55f6a142ff60 2024-06-10T00:41:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fddfa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a142f640 0x55f6a142fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fddfa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a142f640 0x55f6a142fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58102/0 (socket says 172.21.15.62:58102) 2024-06-10T00:41:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fddfa59c700 1 -- 172.21.15.62:0/1338273850 learned_addr learned my addr 172.21.15.62:0/1338273850 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fddf9d9b700 1 --2- 172.21.15.62:0/1338273850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142ff60 0x55f6a14299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fddfa59c700 1 -- 172.21.15.62:0/1338273850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142ff60 msgr2=0x55f6a14299e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fddfa59c700 1 --2- 172.21.15.62:0/1338273850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142ff60 0x55f6a14299e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fddfa59c700 1 -- 172.21.15.62:0/1338273850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a1429f20 msgr2=0x55f6a142a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:26.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fddfa59c700 1 --2- 172.21.15.62:0/1338273850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a1429f20 0x55f6a142a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.739+0000 7fddfa59c700 1 -- 172.21.15.62:0/1338273850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6a1319f50 con 0x55f6a142f640 2024-06-10T00:41:26.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.740+0000 7fddfa59c700 1 --2- 172.21.15.62:0/1338273850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a142f640 0x55f6a142fa20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fddec099080 tx=0x7fddec09c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef86ac23270772f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:26.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.740+0000 7fdde8ff9700 1 -- 172.21.15.62:0/1338273850 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec09c6b0 con 0x55f6a142f640 2024-06-10T00:41:26.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.740+0000 7fdde8ff9700 1 -- 172.21.15.62:0/1338273850 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddec0a0a60 con 0x55f6a142f640 2024-06-10T00:41:26.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.740+0000 7fdde8ff9700 1 -- 172.21.15.62:0/1338273850 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec09c910 con 0x55f6a142f640 2024-06-10T00:41:26.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.741+0000 7fde0b693b80 1 -- 172.21.15.62:0/1338273850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a142f640 msgr2=0x55f6a142fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.741+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1338273850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a142f640 0x55f6a142fa20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fddec099080 tx=0x7fddec09c1f0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.741+0000 7fde0b693b80 1 -- 172.21.15.62:0/1338273850 shutdown_connections 2024-06-10T00:41:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.741+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1338273850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a142f640 0x55f6a142fa20 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.741+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1338273850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a1429f20 0x55f6a142a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.741+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1338273850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142ff60 0x55f6a14299e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.741+0000 7fde0b693b80 1 -- 172.21.15.62:0/1338273850 >> 172.21.15.62:0/1338273850 conn(0x55f6a1354cc0 msgr2=0x55f6a1424850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.741+0000 7fde0b693b80 1 -- 172.21.15.62:0/1338273850 shutdown_connections 2024-06-10T00:41:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.741+0000 7fde0b693b80 1 -- 172.21.15.62:0/1338273850 wait complete. 2024-06-10T00:41:26.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.741+0000 7fde0b693b80 1 -- start start 2024-06-10T00:41:26.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fde0b693b80 1 Processor -- start 2024-06-10T00:41:26.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fde0b693b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a1429f20 0x55f6a1435dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fde0b693b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a142f640 0x55f6a14330c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fde0b693b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142ff60 0x55f6a1433600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fde0b693b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f6a14366f0 con 0x55f6a1429f20 2024-06-10T00:41:26.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fde0b693b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f6a131bbf0 con 0x55f6a142f640 2024-06-10T00:41:26.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fde0b693b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6a1436870 con 0x55f6a142ff60 2024-06-10T00:41:26.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fde0b693b80 10 client.0 _ll_register_callbacks cb 0x55f6a1315920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:26.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fddfa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a1429f20 0x55f6a1435dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fddfa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a1429f20 0x55f6a1435dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58112/0 (socket says 172.21.15.62:58112) 2024-06-10T00:41:26.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fddfa59c700 1 -- 172.21.15.62:0/1699880160 learned_addr learned my addr 172.21.15.62:0/1699880160 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:26.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fddf9d9b700 1 --2- 172.21.15.62:0/1699880160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a142f640 0x55f6a14330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fddfad9d700 1 --2- 172.21.15.62:0/1699880160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142ff60 0x55f6a1433600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fddfa59c700 1 -- 172.21.15.62:0/1699880160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142ff60 msgr2=0x55f6a1433600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:26.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fddfa59c700 1 --2- 172.21.15.62:0/1699880160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142ff60 0x55f6a1433600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fddfa59c700 1 -- 172.21.15.62:0/1699880160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a142f640 msgr2=0x55f6a14330c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:26.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fddfa59c700 1 --2- 172.21.15.62:0/1699880160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a142f640 0x55f6a14330c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fddfa59c700 1 -- 172.21.15.62:0/1699880160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6a12ca8d0 con 0x55f6a1429f20 2024-06-10T00:41:26.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.742+0000 7fddf9d9b700 1 --2- 172.21.15.62:0/1699880160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a142f640 0x55f6a14330c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:26.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fddfa59c700 1 --2- 172.21.15.62:0/1699880160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a1429f20 0x55f6a1435dc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fddec0059d0 tx=0x7fddec098f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3645ff6384c794 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:26.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fdde1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:26.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fdde1ffb700 1 -- 172.21.15.62:0/1699880160 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec0b5070 con 0x55f6a1429f20 2024-06-10T00:41:26.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fdde1ffb700 1 -- 172.21.15.62:0/1699880160 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fddec09cb20 con 0x55f6a1429f20 2024-06-10T00:41:26.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fde0b693b80 1 -- 172.21.15.62:0/1699880160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddec0a0a60 con 0x55f6a1429f20 2024-06-10T00:41:26.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fdde1ffb700 1 -- 172.21.15.62:0/1699880160 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fddec0a5290 con 0x55f6a1429f20 2024-06-10T00:41:26.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fde0b693b80 10 client.16800 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:26.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fde0b693b80 1 -- 172.21.15.62:0/1699880160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f6a14366f0 con 0x55f6a1429f20 2024-06-10T00:41:26.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fdde1ffb700 1 -- 172.21.15.62:0/1699880160 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddec0a0a60 con 0x55f6a1429f20 2024-06-10T00:41:26.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fdde1ffb700 1 -- 172.21.15.62:0/1699880160 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fddec09bb00 con 0x55f6a1429f20 2024-06-10T00:41:26.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fde0b693b80 10 client.16800 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:26.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fde0b693b80 10 client.16800 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:41:26.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fde0b693b80 20 client.16800 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:26.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1699880160 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f6a1443380 0x55f6a14457a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fde0b693b80 4 client.16800 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:41:26.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.743+0000 7fde0b693b80 1 -- 172.21.15.62:0/1699880160 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6a1388d10 con 0x55f6a1443380 2024-06-10T00:41:26.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.744+0000 7fddf9d9b700 1 --2- 172.21.15.62:0/1699880160 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f6a1443380 0x55f6a14457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.744+0000 7fddf9d9b700 1 --2- 172.21.15.62:0/1699880160 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f6a1443380 0x55f6a14457a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=450337af986e349e server_cookie=81ece0a1456da25a in_seq=0 out_seq=0 2024-06-10T00:41:26.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.744+0000 7fdde1ffb700 10 client.16800 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:41:26.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fdde1ffb700 1 -- 172.21.15.62:0/1699880160 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f6a1388d10 con 0x55f6a1443380 2024-06-10T00:41:26.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fdde1ffb700 10 client.16800 handle_command_reply: tid=0 2024-06-10T00:41:26.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fde0b693b80 1 client.16800 shutdown 2024-06-10T00:41:26.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fde0b693b80 1 -- 172.21.15.62:0/1699880160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a1429f20 msgr2=0x55f6a1435dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:26.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1699880160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a1429f20 0x55f6a1435dc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fddec0059d0 tx=0x7fddec098f50 comp rx=0 tx=0).stop 2024-06-10T00:41:26.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fde0b693b80 1 -- 172.21.15.62:0/1699880160 shutdown_connections 2024-06-10T00:41:26.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1699880160 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f6a1443380 0x55f6a14457a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1699880160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a1429f20 0x55f6a1435dc0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1699880160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142ff60 0x55f6a1433600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1699880160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a142f640 0x55f6a14330c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fde0b693b80 1 -- 172.21.15.62:0/1699880160 >> 172.21.15.62:0/1699880160 conn(0x55f6a1354cc0 msgr2=0x55f6a14289d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:26.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fdde1ffb700 0 client.16800 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:41:26.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fde0b693b80 1 -- 172.21.15.62:0/1699880160 shutdown_connections 2024-06-10T00:41:26.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fde0b693b80 1 -- 172.21.15.62:0/1699880160 wait complete. 2024-06-10T00:41:26.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.745+0000 7fde0b693b80 20 client.16800 trim_cache size 0 max 16384 2024-06-10T00:41:26.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fde0b693b80 1 Processor -- start 2024-06-10T00:41:26.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fde0b693b80 1 -- start start 2024-06-10T00:41:26.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fde0b693b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a1429f20 0x55f6a14299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fde0b693b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a142f640 0x55f6a143f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fde0b693b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142ff60 0x55f6a141e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fde0b693b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1b0900 con 0x55f6a1429f20 2024-06-10T00:41:26.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fde0b693b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1b2440 con 0x55f6a142f640 2024-06-10T00:41:26.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fde0b693b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1b0fd0 con 0x55f6a142ff60 2024-06-10T00:41:26.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fddfa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a1429f20 0x55f6a14299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fddfa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a1429f20 0x55f6a14299e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58122/0 (socket says 172.21.15.62:58122) 2024-06-10T00:41:26.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fddfa59c700 1 -- 172.21.15.62:0/3078828868 learned_addr learned my addr 172.21.15.62:0/3078828868 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:26.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fddf9d9b700 1 --2- 172.21.15.62:0/3078828868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a142f640 0x55f6a143f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fddfa59c700 1 -- 172.21.15.62:0/3078828868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142ff60 msgr2=0x55f6a141e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:26.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fddfa59c700 1 --2- 172.21.15.62:0/3078828868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142ff60 0x55f6a141e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fddfa59c700 1 -- 172.21.15.62:0/3078828868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a142f640 msgr2=0x55f6a143f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:26.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fddfa59c700 1 --2- 172.21.15.62:0/3078828868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a142f640 0x55f6a143f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fddfa59c700 1 -- 172.21.15.62:0/3078828868 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6a1425320 con 0x55f6a1429f20 2024-06-10T00:41:26.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fddf9d9b700 1 --2- 172.21.15.62:0/3078828868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a142f640 0x55f6a143f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:41:26.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.761+0000 7fddfa59c700 1 --2- 172.21.15.62:0/3078828868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a1429f20 0x55f6a14299e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fddec09a960 tx=0x7fddec09c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69189391f4a01240 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:26.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.762+0000 7fdde3fff700 1 -- 172.21.15.62:0/3078828868 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec0b5040 con 0x55f6a1429f20 2024-06-10T00:41:26.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.762+0000 7fdde3fff700 1 -- 172.21.15.62:0/3078828868 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddec0a0a60 con 0x55f6a1429f20 2024-06-10T00:41:26.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.762+0000 7fdde3fff700 1 -- 172.21.15.62:0/3078828868 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec0b0bd0 con 0x55f6a1429f20 2024-06-10T00:41:26.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.762+0000 7fde0b693b80 1 -- 172.21.15.62:0/3078828868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a1429f20 msgr2=0x55f6a14299e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:26.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.762+0000 7fde0b693b80 1 --2- 172.21.15.62:0/3078828868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a1429f20 0x55f6a14299e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fddec09a960 tx=0x7fddec09c530 comp rx=0 tx=0).stop 2024-06-10T00:41:26.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.762+0000 7fde0b693b80 1 -- 172.21.15.62:0/3078828868 shutdown_connections 2024-06-10T00:41:26.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.762+0000 7fde0b693b80 1 --2- 172.21.15.62:0/3078828868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a1429f20 0x55f6a14299e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.762+0000 7fde0b693b80 1 --2- 172.21.15.62:0/3078828868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142ff60 0x55f6a141e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.762+0000 7fde0b693b80 1 --2- 172.21.15.62:0/3078828868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a142f640 0x55f6a143f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.762+0000 7fde0b693b80 1 -- 172.21.15.62:0/3078828868 >> 172.21.15.62:0/3078828868 conn(0x55f6a1354cc0 msgr2=0x55f6a141de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:26.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.762+0000 7fde0b693b80 1 -- 172.21.15.62:0/3078828868 shutdown_connections 2024-06-10T00:41:26.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.762+0000 7fde0b693b80 1 -- 172.21.15.62:0/3078828868 wait complete. 2024-06-10T00:41:26.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.763+0000 7fde0b693b80 1 -- start start 2024-06-10T00:41:26.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.763+0000 7fde0b693b80 1 Processor -- start 2024-06-10T00:41:26.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.763+0000 7fde0b693b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a1429f20 0x55f6a143bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.763+0000 7fde0b693b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142f640 0x55f6a143c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.763+0000 7fde0b693b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a142ff60 0x55f6a144a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.763+0000 7fde0b693b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1b1d70 con 0x55f6a142ff60 2024-06-10T00:41:26.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.763+0000 7fde0b693b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1b16a0 con 0x55f6a1429f20 2024-06-10T00:41:26.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.763+0000 7fde0b693b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1ae8c0 con 0x55f6a142f640 2024-06-10T00:41:26.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.763+0000 7fde0b693b80 10 client.0 _ll_register_callbacks cb 0x55f6a1315920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:26.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.763+0000 7fddfad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a142ff60 0x55f6a144a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.763+0000 7fddfad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a142ff60 0x55f6a144a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58138/0 (socket says 172.21.15.62:58138) 2024-06-10T00:41:26.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.763+0000 7fddfad9d700 1 -- 172.21.15.62:0/1588763275 learned_addr learned my addr 172.21.15.62:0/1588763275 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:26.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.763+0000 7fddfa59c700 1 --2- 172.21.15.62:0/1588763275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a1429f20 0x55f6a143bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fddf9d9b700 1 --2- 172.21.15.62:0/1588763275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142f640 0x55f6a143c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fddfad9d700 1 -- 172.21.15.62:0/1588763275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142f640 msgr2=0x55f6a143c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:26.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fddfad9d700 1 --2- 172.21.15.62:0/1588763275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142f640 0x55f6a143c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fddfad9d700 1 -- 172.21.15.62:0/1588763275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a1429f20 msgr2=0x55f6a143bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fddfad9d700 1 --2- 172.21.15.62:0/1588763275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a1429f20 0x55f6a143bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fddfad9d700 1 -- 172.21.15.62:0/1588763275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6a144ab20 con 0x55f6a142ff60 2024-06-10T00:41:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fddf9d9b700 1 --2- 172.21.15.62:0/1588763275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142f640 0x55f6a143c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:41:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fddfad9d700 1 --2- 172.21.15.62:0/1588763275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a142ff60 0x55f6a144a4b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fddf4008b00 tx=0x7fddf4008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc9a27ec071bdf5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fdde1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fdde1ffb700 1 -- 172.21.15.62:0/1588763275 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf4010a30 con 0x55f6a142ff60 2024-06-10T00:41:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fdde1ffb700 1 -- 172.21.15.62:0/1588763275 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fddf4010bd0 con 0x55f6a142ff60 2024-06-10T00:41:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fde0b693b80 1 -- 172.21.15.62:0/1588763275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddec0a0a60 con 0x55f6a142ff60 2024-06-10T00:41:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fdde1ffb700 1 -- 172.21.15.62:0/1588763275 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fddf4024050 con 0x55f6a142ff60 2024-06-10T00:41:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fde0b693b80 10 client.16806 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fde0b693b80 1 -- 172.21.15.62:0/1588763275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f6a14ad9a0 con 0x55f6a142ff60 2024-06-10T00:41:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fdde1ffb700 1 -- 172.21.15.62:0/1588763275 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddec0a0a60 con 0x55f6a142ff60 2024-06-10T00:41:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fdde1ffb700 1 -- 172.21.15.62:0/1588763275 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fddf40153d0 con 0x55f6a142ff60 2024-06-10T00:41:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fde0b693b80 10 client.16806 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.764+0000 7fde0b693b80 10 client.16806 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:41:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.765+0000 7fde0b693b80 20 client.16806 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:26.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.765+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1588763275 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f6a14ae130 0x55f6a14b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:26.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.765+0000 7fde0b693b80 4 client.16806 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:41:26.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.765+0000 7fde0b693b80 1 -- 172.21.15.62:0/1588763275 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6a1383290 con 0x55f6a14ae130 2024-06-10T00:41:26.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.765+0000 7fddfa59c700 1 --2- 172.21.15.62:0/1588763275 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f6a14ae130 0x55f6a14b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:26.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.765+0000 7fddfa59c700 1 --2- 172.21.15.62:0/1588763275 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f6a14ae130 0x55f6a14b0550 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=59647961c1299e9f server_cookie=b07c8a02bea23017 in_seq=0 out_seq=0 2024-06-10T00:41:26.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.765+0000 7fdde1ffb700 10 client.16806 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:41:26.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fdde1ffb700 1 -- 172.21.15.62:0/1588763275 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6a1383290 con 0x55f6a14ae130 2024-06-10T00:41:26.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fdde1ffb700 10 client.16806 handle_command_reply: tid=0 2024-06-10T00:41:26.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde0b693b80 1 client.16806 shutdown 2024-06-10T00:41:26.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde0b693b80 1 -- 172.21.15.62:0/1588763275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a142ff60 msgr2=0x55f6a144a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:26.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1588763275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a142ff60 0x55f6a144a4b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fddf4008b00 tx=0x7fddf4008ec0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde0b693b80 1 -- 172.21.15.62:0/1588763275 shutdown_connections 2024-06-10T00:41:26.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1588763275 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x55f6a14ae130 0x55f6a14b0550 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1588763275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f6a142ff60 0x55f6a144a4b0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1588763275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f6a1429f20 0x55f6a143bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fdde1ffb700 0 client.16806 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:41:26.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde0b693b80 1 --2- 172.21.15.62:0/1588763275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a142f640 0x55f6a143c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde0b693b80 1 -- 172.21.15.62:0/1588763275 >> 172.21.15.62:0/1588763275 conn(0x55f6a1354cc0 msgr2=0x55f6a142e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:26.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde0b693b80 1 -- 172.21.15.62:0/1588763275 shutdown_connections 2024-06-10T00:41:26.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde0b693b80 1 -- 172.21.15.62:0/1588763275 wait complete. 2024-06-10T00:41:26.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde0b693b80 20 client.16806 trim_cache size 0 max 16384 2024-06-10T00:41:26.777 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:41:26.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde00e10700 1 -- 172.21.15.62:0/48740401 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fdde40a2d90 msgr2=0x7fdde40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:26.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde00e10700 1 --2- 172.21.15.62:0/48740401 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fdde40a2d90 0x7fdde40a5250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fddfc102030 tx=0x7fddf000b410 comp rx=0 tx=0).stop 2024-06-10T00:41:26.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde00e10700 1 -- 172.21.15.62:0/48740401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc102740 msgr2=0x7fddfc074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:26.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.766+0000 7fde00e10700 1 --2- 172.21.15.62:0/48740401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc102740 0x7fddfc074790 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fddec006010 tx=0x7fddec009f80 comp rx=0 tx=0).stop 2024-06-10T00:41:26.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.767+0000 7fde00e10700 1 -- 172.21.15.62:0/48740401 shutdown_connections 2024-06-10T00:41:26.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.767+0000 7fde00e10700 1 --2- 172.21.15.62:0/48740401 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fdde40a2d90 0x7fdde40a5250 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.767+0000 7fde00e10700 1 --2- 172.21.15.62:0/48740401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc102740 0x7fddfc074790 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.767+0000 7fde00e10700 1 --2- 172.21.15.62:0/48740401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddfc107840 0x7fddfc075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.767+0000 7fde00e10700 1 --2- 172.21.15.62:0/48740401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc1030f0 0x7fddfc074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:26.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.767+0000 7fde00e10700 1 -- 172.21.15.62:0/48740401 >> 172.21.15.62:0/48740401 conn(0x7fddfc0795f0 msgr2=0x7fddfc108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:26.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.767+0000 7fde00e10700 1 -- 172.21.15.62:0/48740401 shutdown_connections 2024-06-10T00:41:26.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:26.767+0000 7fde00e10700 1 -- 172.21.15.62:0/48740401 wait complete. 2024-06-10T00:41:27.442 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T00:41:27.692 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:41:27.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:27 smithi062 ceph-mon[32390]: pgmap v265: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:41:27.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:27 smithi138 ceph-mon[36549]: pgmap v265: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:41:28.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:27 smithi105 ceph-mon[36321]: pgmap v265: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:41:29.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.351+0000 7fb4ad4da700 1 -- 172.21.15.62:0/3213591027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4a806b3c0 msgr2=0x7fb4a810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:29.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.351+0000 7fb4ad4da700 1 --2- 172.21.15.62:0/3213591027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4a806b3c0 0x7fb4a810f310 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009990 tx=0x7fb4a0009ca0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.351+0000 7fb4ad4da700 1 -- 172.21.15.62:0/3213591027 shutdown_connections 2024-06-10T00:41:29.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.351+0000 7fb4ad4da700 1 --2- 172.21.15.62:0/3213591027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4a806b3c0 0x7fb4a810f310 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.351+0000 7fb4ad4da700 1 --2- 172.21.15.62:0/3213591027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4a81084f0 0x7fb4a81088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.351+0000 7fb4ad4da700 1 --2- 172.21.15.62:0/3213591027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a806aa00 0x7fb4a806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.351+0000 7fb4ad4da700 1 -- 172.21.15.62:0/3213591027 >> 172.21.15.62:0/3213591027 conn(0x7fb4a8078300 msgr2=0x7fb4a8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:29.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.351+0000 7fb4ad4da700 1 -- 172.21.15.62:0/3213591027 shutdown_connections 2024-06-10T00:41:29.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.351+0000 7fb4ad4da700 1 -- 172.21.15.62:0/3213591027 wait complete. 2024-06-10T00:41:29.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.352+0000 7fb4ad4da700 1 Processor -- start 2024-06-10T00:41:29.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.352+0000 7fb4ad4da700 1 -- start start 2024-06-10T00:41:29.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.352+0000 7fb4ad4da700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4a806aa00 0x7fb4a819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.352+0000 7fb4ad4da700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a806b3c0 0x7fb4a819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.352+0000 7fb4ad4da700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4a81084f0 0x7fb4a8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.352+0000 7fb4ad4da700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81127d0 con 0x7fb4a81084f0 2024-06-10T00:41:29.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.352+0000 7fb4ad4da700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8112650 con 0x7fb4a806aa00 2024-06-10T00:41:29.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.352+0000 7fb4ad4da700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8112950 con 0x7fb4a806b3c0 2024-06-10T00:41:29.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.352+0000 7fb4a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4a81084f0 0x7fb4a8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.352+0000 7fb4a6ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4a806aa00 0x7fb4a819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.352+0000 7fb4a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4a81084f0 0x7fb4a8198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46202/0 (socket says 172.21.15.62:46202) 2024-06-10T00:41:29.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.352+0000 7fb4a77fe700 1 -- 172.21.15.62:0/2346312195 learned_addr learned my addr 172.21.15.62:0/2346312195 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:29.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.352+0000 7fb4a67fc700 1 --2- 172.21.15.62:0/2346312195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a806b3c0 0x7fb4a819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.353+0000 7fb4a77fe700 1 -- 172.21.15.62:0/2346312195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a806b3c0 msgr2=0x7fb4a819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:29.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.353+0000 7fb4a77fe700 1 --2- 172.21.15.62:0/2346312195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a806b3c0 0x7fb4a819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.353+0000 7fb4a77fe700 1 -- 172.21.15.62:0/2346312195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4a806aa00 msgr2=0x7fb4a819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:29.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.353+0000 7fb4a77fe700 1 --2- 172.21.15.62:0/2346312195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4a806aa00 0x7fb4a819e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.353+0000 7fb4a77fe700 1 -- 172.21.15.62:0/2346312195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a8199170 con 0x7fb4a81084f0 2024-06-10T00:41:29.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.353+0000 7fb4a6ffd700 1 --2- 172.21.15.62:0/2346312195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4a806aa00 0x7fb4a819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:41:29.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.353+0000 7fb4a77fe700 1 --2- 172.21.15.62:0/2346312195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4a81084f0 0x7fb4a8198a10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00038b0 tx=0x7fb4a0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:29.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.353+0000 7fb49ffff700 1 -- 172.21.15.62:0/2346312195 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0027070 con 0x7fb4a81084f0 2024-06-10T00:41:29.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.353+0000 7fb4ad4da700 1 -- 172.21.15.62:0/2346312195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a8199400 con 0x7fb4a81084f0 2024-06-10T00:41:29.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.353+0000 7fb49ffff700 1 -- 172.21.15.62:0/2346312195 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a0003e40 con 0x7fb4a81084f0 2024-06-10T00:41:29.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.353+0000 7fb49ffff700 1 -- 172.21.15.62:0/2346312195 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0022cc0 con 0x7fb4a81084f0 2024-06-10T00:41:29.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.353+0000 7fb4ad4da700 1 -- 172.21.15.62:0/2346312195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a81a5720 con 0x7fb4a81084f0 2024-06-10T00:41:29.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.355+0000 7fb49ffff700 1 -- 172.21.15.62:0/2346312195 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb4a002a070 con 0x7fb4a81084f0 2024-06-10T00:41:29.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.356+0000 7fb49ffff700 1 --2- 172.21.15.62:0/2346312195 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb4940a2d90 0x7fb4940a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.356+0000 7fb4a6ffd700 1 --2- 172.21.15.62:0/2346312195 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb4940a2d90 0x7fb4940a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.357+0000 7fb49ffff700 1 -- 172.21.15.62:0/2346312195 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fb4a00d0830 con 0x7fb4a81084f0 2024-06-10T00:41:29.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.357+0000 7fb4b7d5db80 1 Processor -- start 2024-06-10T00:41:29.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.357+0000 7fb4a6ffd700 1 --2- 172.21.15.62:0/2346312195 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb4940a2d90 0x7fb4940a5250 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8199b50 tx=0x7fb4980091b0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:29.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.357+0000 7fb4b7d5db80 1 -- start start 2024-06-10T00:41:29.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.357+0000 7fb4b7d5db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee623340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.357+0000 7fb4b7d5db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee61d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.357+0000 7fb4b7d5db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee61dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.357+0000 7fb4b7d5db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e5ee50ff60 con 0x55e5ee623880 2024-06-10T00:41:29.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.357+0000 7fb4b7d5db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e5ee56e850 con 0x55e5ee61d7a0 2024-06-10T00:41:29.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.357+0000 7fb4b7d5db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e5ee4d6460 con 0x55e5ee622f60 2024-06-10T00:41:29.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.357+0000 7fb4a67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee61d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.358+0000 7fb4a67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee61d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46208/0 (socket says 172.21.15.62:46208) 2024-06-10T00:41:29.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.358+0000 7fb4a67fc700 1 -- 172.21.15.62:0/3756901847 learned_addr learned my addr 172.21.15.62:0/3756901847 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:29.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.358+0000 7fb4a6ffd700 1 --2- 172.21.15.62:0/3756901847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee623340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.358+0000 7fb4a77fe700 1 --2- 172.21.15.62:0/3756901847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee61dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.358+0000 7fb4a67fc700 1 -- 172.21.15.62:0/3756901847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 msgr2=0x55e5ee623340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:29.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.358+0000 7fb4a67fc700 1 --2- 172.21.15.62:0/3756901847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee623340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.358+0000 7fb4a67fc700 1 -- 172.21.15.62:0/3756901847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 msgr2=0x55e5ee61dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:29.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.358+0000 7fb4a67fc700 1 --2- 172.21.15.62:0/3756901847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee61dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.358+0000 7fb4a67fc700 1 -- 172.21.15.62:0/3756901847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ee4bd8d0 con 0x55e5ee623880 2024-06-10T00:41:29.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.358+0000 7fb4a67fc700 1 --2- 172.21.15.62:0/3756901847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee61d260 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb490009ac0 tx=0x7fb490009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd23c34d394a903a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:29.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.358+0000 7fb49d7fa700 1 -- 172.21.15.62:0/3756901847 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490017070 con 0x55e5ee623880 2024-06-10T00:41:29.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.358+0000 7fb49d7fa700 1 -- 172.21.15.62:0/3756901847 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49000edb0 con 0x55e5ee623880 2024-06-10T00:41:29.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.358+0000 7fb49d7fa700 1 -- 172.21.15.62:0/3756901847 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4900123d0 con 0x55e5ee623880 2024-06-10T00:41:29.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.359+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/3756901847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 msgr2=0x55e5ee61d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:29.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.359+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/3756901847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee61d260 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb490009ac0 tx=0x7fb490009dd0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.359+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/3756901847 shutdown_connections 2024-06-10T00:41:29.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.359+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/3756901847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee61dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.359+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/3756901847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee623340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.359+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/3756901847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee61d260 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.359+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/3756901847 >> 172.21.15.62:0/3756901847 conn(0x55e5ee54a080 msgr2=0x55e5ee618220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:29.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.359+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/3756901847 shutdown_connections 2024-06-10T00:41:29.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.359+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/3756901847 wait complete. 2024-06-10T00:41:29.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.360+0000 7fb4b7d5db80 1 -- start start 2024-06-10T00:41:29.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.360+0000 7fb4b7d5db80 1 Processor -- start 2024-06-10T00:41:29.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.360+0000 7fb4b7d5db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee629570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.360+0000 7fb4b7d5db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee622f60 0x55e5ee6268e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.360+0000 7fb4b7d5db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee623880 0x55e5ee626e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.360+0000 7fb4b7d5db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e5ee50efc0 con 0x55e5ee622f60 2024-06-10T00:41:29.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.360+0000 7fb4b7d5db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e5ee56a810 con 0x55e5ee61d7a0 2024-06-10T00:41:29.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.360+0000 7fb4b7d5db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e5ee62a260 con 0x55e5ee623880 2024-06-10T00:41:29.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.360+0000 7fb4b7d5db80 10 client.0 _ll_register_callbacks cb 0x55e5ee5045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:29.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.360+0000 7fb4a67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee622f60 0x55e5ee6268e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.360+0000 7fb4a67fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee622f60 0x55e5ee6268e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46216/0 (socket says 172.21.15.62:46216) 2024-06-10T00:41:29.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.360+0000 7fb4a67fc700 1 -- 172.21.15.62:0/2771831153 learned_addr learned my addr 172.21.15.62:0/2771831153 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:29.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.360+0000 7fb4a77fe700 1 --2- 172.21.15.62:0/2771831153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee623880 0x55e5ee626e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb4a6ffd700 1 --2- 172.21.15.62:0/2771831153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee629570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb4a67fc700 1 -- 172.21.15.62:0/2771831153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee623880 msgr2=0x55e5ee626e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:29.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb4a67fc700 1 --2- 172.21.15.62:0/2771831153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee623880 0x55e5ee626e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb4a67fc700 1 -- 172.21.15.62:0/2771831153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 msgr2=0x55e5ee629570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:29.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb4a67fc700 1 --2- 172.21.15.62:0/2771831153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee629570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb4a67fc700 1 -- 172.21.15.62:0/2771831153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ee534750 con 0x55e5ee622f60 2024-06-10T00:41:29.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb4a6ffd700 1 --2- 172.21.15.62:0/2771831153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee629570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:29.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb4a67fc700 1 --2- 172.21.15.62:0/2771831153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee622f60 0x55e5ee6268e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb490004790 tx=0x7fb49000eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829f83d1e292a096 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:29.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb48e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:29.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb48e7fc700 1 -- 172.21.15.62:0/2771831153 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490017040 con 0x55e5ee622f60 2024-06-10T00:41:29.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb48e7fc700 1 -- 172.21.15.62:0/2771831153 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fb49002a470 con 0x55e5ee622f60 2024-06-10T00:41:29.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/2771831153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb49000edb0 con 0x55e5ee622f60 2024-06-10T00:41:29.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb48e7fc700 1 -- 172.21.15.62:0/2771831153 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb490006df0 con 0x55e5ee622f60 2024-06-10T00:41:29.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb4b7d5db80 10 client.16818 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:29.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.361+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/2771831153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e5ee6349f0 con 0x55e5ee622f60 2024-06-10T00:41:29.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.362+0000 7fb48e7fc700 1 -- 172.21.15.62:0/2771831153 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49000edb0 con 0x55e5ee622f60 2024-06-10T00:41:29.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.362+0000 7fb48e7fc700 1 -- 172.21.15.62:0/2771831153 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fb490021750 con 0x55e5ee622f60 2024-06-10T00:41:29.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.362+0000 7fb4b7d5db80 10 client.16818 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:29.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.362+0000 7fb4b7d5db80 10 client.16818 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:41:29.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.362+0000 7fb4b7d5db80 20 client.16818 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:29.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.362+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/2771831153 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55e5ee637020 0x55e5ee639410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.362+0000 7fb4b7d5db80 4 client.16818 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:41:29.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.362+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/2771831153 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5ee57bd50 con 0x55e5ee637020 2024-06-10T00:41:29.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.362+0000 7fb4a6ffd700 1 --2- 172.21.15.62:0/2771831153 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55e5ee637020 0x55e5ee639410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.363+0000 7fb4a6ffd700 1 --2- 172.21.15.62:0/2771831153 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55e5ee637020 0x55e5ee639410 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2dd6a56a16b1c0f5 server_cookie=5e5bf55355200c06 in_seq=0 out_seq=0 2024-06-10T00:41:29.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.363+0000 7fb48e7fc700 10 client.16818 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:41:29.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.363+0000 7fb48e7fc700 1 -- 172.21.15.62:0/2771831153 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e5ee57bd50 con 0x55e5ee637020 2024-06-10T00:41:29.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.363+0000 7fb48e7fc700 10 client.16818 handle_command_reply: tid=0 2024-06-10T00:41:29.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.363+0000 7fb4b7d5db80 1 client.16818 shutdown 2024-06-10T00:41:29.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.363+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/2771831153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee622f60 msgr2=0x55e5ee6268e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:29.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.363+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/2771831153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee622f60 0x55e5ee6268e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb490004790 tx=0x7fb49000eb00 comp rx=0 tx=0).stop 2024-06-10T00:41:29.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.363+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/2771831153 shutdown_connections 2024-06-10T00:41:29.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.364+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/2771831153 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55e5ee637020 0x55e5ee639410 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.364+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/2771831153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee623880 0x55e5ee626e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.364+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/2771831153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee629570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.364+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/2771831153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee622f60 0x55e5ee6268e0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.364+0000 7fb48e7fc700 0 client.16818 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:41:29.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.364+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/2771831153 >> 172.21.15.62:0/2771831153 conn(0x55e5ee54a080 msgr2=0x55e5ee617700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:29.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.364+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/2771831153 shutdown_connections 2024-06-10T00:41:29.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.364+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/2771831153 wait complete. 2024-06-10T00:41:29.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.364+0000 7fb4b7d5db80 20 client.16818 trim_cache size 0 max 16384 2024-06-10T00:41:29.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.379+0000 7fb4b7d5db80 1 Processor -- start 2024-06-10T00:41:29.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.379+0000 7fb4b7d5db80 1 -- start start 2024-06-10T00:41:29.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.379+0000 7fb4b7d5db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee6261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.379+0000 7fb4b7d5db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee628820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.379+0000 7fb4b7d5db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee617e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.379+0000 7fb4b7d5db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81ac330 con 0x55e5ee623880 2024-06-10T00:41:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.379+0000 7fb4b7d5db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81ade70 con 0x55e5ee61d7a0 2024-06-10T00:41:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.379+0000 7fb4b7d5db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81aca00 con 0x55e5ee622f60 2024-06-10T00:41:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb4a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee617e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb4a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee617e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46232/0 (socket says 172.21.15.62:46232) 2024-06-10T00:41:29.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb4a77fe700 1 -- 172.21.15.62:0/3498520100 learned_addr learned my addr 172.21.15.62:0/3498520100 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:29.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb4a67fc700 1 --2- 172.21.15.62:0/3498520100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee628820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb4a6ffd700 1 --2- 172.21.15.62:0/3498520100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee6261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb4a77fe700 1 -- 172.21.15.62:0/3498520100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 msgr2=0x55e5ee628820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:29.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb4a77fe700 1 --2- 172.21.15.62:0/3498520100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee628820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb4a77fe700 1 -- 172.21.15.62:0/3498520100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 msgr2=0x55e5ee6261e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:29.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb4a77fe700 1 --2- 172.21.15.62:0/3498520100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee6261e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb4a77fe700 1 -- 172.21.15.62:0/3498520100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ee6277f0 con 0x55e5ee623880 2024-06-10T00:41:29.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb4a6ffd700 1 --2- 172.21.15.62:0/3498520100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee6261e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:29.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb4a67fc700 1 --2- 172.21.15.62:0/3498520100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee628820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:29.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb4a77fe700 1 --2- 172.21.15.62:0/3498520100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee617e80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fb4a009cb90 tx=0x7fb4a00a5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ec58e073cd4af4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:29.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb49cff9700 1 -- 172.21.15.62:0/3498520100 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00b0e60 con 0x55e5ee623880 2024-06-10T00:41:29.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb49cff9700 1 -- 172.21.15.62:0/3498520100 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a009de50 con 0x55e5ee623880 2024-06-10T00:41:29.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.380+0000 7fb49cff9700 1 -- 172.21.15.62:0/3498520100 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0022e60 con 0x55e5ee623880 2024-06-10T00:41:29.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.381+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/3498520100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 msgr2=0x55e5ee617e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:29.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.381+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/3498520100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee617e80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fb4a009cb90 tx=0x7fb4a00a5040 comp rx=0 tx=0).stop 2024-06-10T00:41:29.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.381+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/3498520100 shutdown_connections 2024-06-10T00:41:29.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.381+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/3498520100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee617e80 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.381+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/3498520100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee6261e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.381+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/3498520100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee628820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.381+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/3498520100 >> 172.21.15.62:0/3498520100 conn(0x55e5ee54a080 msgr2=0x55e5ee628110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:29.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.381+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/3498520100 shutdown_connections 2024-06-10T00:41:29.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.381+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/3498520100 wait complete. 2024-06-10T00:41:29.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4b7d5db80 1 -- start start 2024-06-10T00:41:29.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4b7d5db80 1 Processor -- start 2024-06-10T00:41:29.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4b7d5db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee699d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4b7d5db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee69a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4b7d5db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee639ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4b7d5db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81ad7a0 con 0x55e5ee623880 2024-06-10T00:41:29.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4b7d5db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81ad0d0 con 0x55e5ee61d7a0 2024-06-10T00:41:29.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4b7d5db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81aa2f0 con 0x55e5ee622f60 2024-06-10T00:41:29.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4b7d5db80 10 client.0 _ll_register_callbacks cb 0x55e5ee61e130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:29.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4a67fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee69a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4a77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee639ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4a67fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee69a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:49006/0 (socket says 172.21.15.62:49006) 2024-06-10T00:41:29.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4a67fc700 1 -- 172.21.15.62:0/213640048 learned_addr learned my addr 172.21.15.62:0/213640048 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:29.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4a6ffd700 1 --2- 172.21.15.62:0/213640048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee699d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4a77fe700 1 -- 172.21.15.62:0/213640048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 msgr2=0x55e5ee69a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:29.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4a77fe700 1 --2- 172.21.15.62:0/213640048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee69a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4a77fe700 1 -- 172.21.15.62:0/213640048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 msgr2=0x55e5ee699d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:29.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4a77fe700 1 --2- 172.21.15.62:0/213640048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee699d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4a77fe700 1 -- 172.21.15.62:0/213640048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ee63a1d0 con 0x55e5ee623880 2024-06-10T00:41:29.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4a6ffd700 1 --2- 172.21.15.62:0/213640048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee699d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:29.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.382+0000 7fb4a67fc700 1 --2- 172.21.15.62:0/213640048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee69a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:41:29.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb4a77fe700 1 --2- 172.21.15.62:0/213640048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee639ad0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x55e5ee622d50 tx=0x7fb4a0098f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efcf26e652e0779 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:29.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb48e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:29.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb48e7fc700 1 -- 172.21.15.62:0/213640048 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0022e60 con 0x55e5ee623880 2024-06-10T00:41:29.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb48e7fc700 1 -- 172.21.15.62:0/213640048 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fb4a00bb300 con 0x55e5ee623880 2024-06-10T00:41:29.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/213640048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a009de50 con 0x55e5ee623880 2024-06-10T00:41:29.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb48e7fc700 1 -- 172.21.15.62:0/213640048 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb4a00a5120 con 0x55e5ee623880 2024-06-10T00:41:29.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb4b7d5db80 10 client.16824 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:29.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/213640048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e5ee6a1540 con 0x55e5ee623880 2024-06-10T00:41:29.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb48e7fc700 1 -- 172.21.15.62:0/213640048 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a009de50 con 0x55e5ee623880 2024-06-10T00:41:29.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb48e7fc700 1 -- 172.21.15.62:0/213640048 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fb4a00b5c60 con 0x55e5ee623880 2024-06-10T00:41:29.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb4b7d5db80 10 client.16824 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:29.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb4b7d5db80 10 client.16824 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:41:29.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb4b7d5db80 20 client.16824 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:29.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/213640048 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55e5ee6a1b90 0x55e5ee6a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:29.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb4b7d5db80 4 client.16824 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:41:29.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.383+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/213640048 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5ee5762d0 con 0x55e5ee6a1b90 2024-06-10T00:41:29.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.384+0000 7fb4a6ffd700 1 --2- 172.21.15.62:0/213640048 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55e5ee6a1b90 0x55e5ee6a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:29.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.384+0000 7fb4a6ffd700 1 --2- 172.21.15.62:0/213640048 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55e5ee6a1b90 0x55e5ee6a3f80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1435de89687306a1 server_cookie=bc334658a3d4aef7 in_seq=0 out_seq=0 2024-06-10T00:41:29.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.384+0000 7fb48e7fc700 10 client.16824 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:41:29.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.384+0000 7fb48e7fc700 1 -- 172.21.15.62:0/213640048 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5ee5762d0 con 0x55e5ee6a1b90 2024-06-10T00:41:29.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.384+0000 7fb48e7fc700 10 client.16824 handle_command_reply: tid=0 2024-06-10T00:41:29.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.384+0000 7fb4b7d5db80 1 client.16824 shutdown 2024-06-10T00:41:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.384+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/213640048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 msgr2=0x55e5ee639ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.384+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/213640048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee639ad0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x55e5ee622d50 tx=0x7fb4a0098f10 comp rx=0 tx=0).stop 2024-06-10T00:41:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.384+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/213640048 shutdown_connections 2024-06-10T00:41:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.384+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/213640048 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55e5ee6a1b90 0x55e5ee6a3f80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.384+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/213640048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5ee623880 0x55e5ee639ad0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.384+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/213640048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5ee61d7a0 0x55e5ee699d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb4b7d5db80 1 --2- 172.21.15.62:0/213640048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e5ee622f60 0x55e5ee69a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb48e7fc700 0 client.16824 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:41:29.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/213640048 >> 172.21.15.62:0/213640048 conn(0x55e5ee54a080 msgr2=0x55e5ee6289a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:29.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/213640048 shutdown_connections 2024-06-10T00:41:29.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb4b7d5db80 1 -- 172.21.15.62:0/213640048 wait complete. 2024-06-10T00:41:29.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb4b7d5db80 20 client.16824 trim_cache size 0 max 16384 2024-06-10T00:41:29.396 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:41:29.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb4ad4da700 1 -- 172.21.15.62:0/2346312195 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb4940a2d90 msgr2=0x7fb4940a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:29.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb4ad4da700 1 --2- 172.21.15.62:0/2346312195 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb4940a2d90 0x7fb4940a5250 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8199b50 tx=0x7fb4980091b0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb4ad4da700 1 -- 172.21.15.62:0/2346312195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4a81084f0 msgr2=0x7fb4a8198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:29.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb4ad4da700 1 --2- 172.21.15.62:0/2346312195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4a81084f0 0x7fb4a8198a10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00038b0 tx=0x7fb4a0005e80 comp rx=0 tx=0).stop 2024-06-10T00:41:29.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb4ad4da700 1 -- 172.21.15.62:0/2346312195 shutdown_connections 2024-06-10T00:41:29.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb4ad4da700 1 --2- 172.21.15.62:0/2346312195 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb4940a2d90 0x7fb4940a5250 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb4ad4da700 1 --2- 172.21.15.62:0/2346312195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4a81084f0 0x7fb4a8198a10 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb4ad4da700 1 --2- 172.21.15.62:0/2346312195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4a806aa00 0x7fb4a819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.385+0000 7fb4ad4da700 1 --2- 172.21.15.62:0/2346312195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a806b3c0 0x7fb4a819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.386+0000 7fb4ad4da700 1 -- 172.21.15.62:0/2346312195 >> 172.21.15.62:0/2346312195 conn(0x7fb4a8078300 msgr2=0x7fb4a80fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.386+0000 7fb4ad4da700 1 -- 172.21.15.62:0/2346312195 shutdown_connections 2024-06-10T00:41:29.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:29.386+0000 7fb4ad4da700 1 -- 172.21.15.62:0/2346312195 wait complete. 2024-06-10T00:41:29.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:29 smithi062 ceph-mon[32390]: pgmap v266: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:41:29.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:29 smithi138 ceph-mon[36549]: pgmap v266: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:41:30.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:29 smithi105 ceph-mon[36321]: pgmap v266: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:41:30.072 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T00:41:30.323 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:41:31.819 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:31 smithi062 ceph-mon[32390]: pgmap v267: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:41:31.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:31 smithi138 ceph-mon[36549]: pgmap v267: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:41:32.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:31 smithi105 ceph-mon[36321]: pgmap v267: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:41:32.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.046+0000 7f25cf21d700 1 -- 172.21.15.62:0/3269974685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25c81020a0 msgr2=0x7f25c810e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:32.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.046+0000 7f25cf21d700 1 --2- 172.21.15.62:0/3269974685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25c81020a0 0x7f25c810e220 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f25c000b0d0 tx=0x7f25c000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.046+0000 7f25cf21d700 1 -- 172.21.15.62:0/3269974685 shutdown_connections 2024-06-10T00:41:32.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.046+0000 7f25cf21d700 1 --2- 172.21.15.62:0/3269974685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25c81020a0 0x7f25c810e220 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.046+0000 7f25cf21d700 1 --2- 172.21.15.62:0/3269974685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c8106410 0x7f25c81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.046+0000 7f25cf21d700 1 --2- 172.21.15.62:0/3269974685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25c81016e0 0x7f25c8101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.046+0000 7f25cf21d700 1 -- 172.21.15.62:0/3269974685 >> 172.21.15.62:0/3269974685 conn(0x7f25c8077200 msgr2=0x7f25c8077610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:32.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.046+0000 7f25cf21d700 1 -- 172.21.15.62:0/3269974685 shutdown_connections 2024-06-10T00:41:32.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.047+0000 7f25cf21d700 1 -- 172.21.15.62:0/3269974685 wait complete. 2024-06-10T00:41:32.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.047+0000 7f25cf21d700 1 Processor -- start 2024-06-10T00:41:32.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.047+0000 7f25cf21d700 1 -- start start 2024-06-10T00:41:32.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25cf21d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c81016e0 0x7f25c81999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25cf21d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25c81020a0 0x7f25c8199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25cf21d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25c8106410 0x7f25c81a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25cf21d700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f25c8111730 con 0x7f25c81020a0 2024-06-10T00:41:32.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25cf21d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81115b0 con 0x7f25c8106410 2024-06-10T00:41:32.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25cf21d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81118b0 con 0x7f25c81016e0 2024-06-10T00:41:32.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25c7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25c81020a0 0x7f25c8199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25c7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25c81020a0 0x7f25c8199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46258/0 (socket says 172.21.15.62:46258) 2024-06-10T00:41:32.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25c7fff700 1 -- 172.21.15.62:0/2495829812 learned_addr learned my addr 172.21.15.62:0/2495829812 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:32.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25ccfb9700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c81016e0 0x7f25c81999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25cd7ba700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25c8106410 0x7f25c81a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25c7fff700 1 -- 172.21.15.62:0/2495829812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c81016e0 msgr2=0x7f25c81999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:32.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25c7fff700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c81016e0 0x7f25c81999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25c7fff700 1 -- 172.21.15.62:0/2495829812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25c8106410 msgr2=0x7f25c81a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:32.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25c7fff700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25c8106410 0x7f25c81a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25c7fff700 1 -- 172.21.15.62:0/2495829812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c81a4fb0 con 0x7f25c81020a0 2024-06-10T00:41:32.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25cd7ba700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25c8106410 0x7f25c81a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:32.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.048+0000 7f25c7fff700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25c81020a0 0x7f25c8199f30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f25b400c920 tx=0x7f25b400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:32.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.049+0000 7f25c5ffb700 1 -- 172.21.15.62:0/2495829812 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b4007b00 con 0x7f25c81020a0 2024-06-10T00:41:32.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.049+0000 7f25cf21d700 1 -- 172.21.15.62:0/2495829812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c81a51e0 con 0x7f25c81020a0 2024-06-10T00:41:32.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.049+0000 7f25c5ffb700 1 -- 172.21.15.62:0/2495829812 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b4004d10 con 0x7f25c81020a0 2024-06-10T00:41:32.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.049+0000 7f25c5ffb700 1 -- 172.21.15.62:0/2495829812 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b4005630 con 0x7f25c81020a0 2024-06-10T00:41:32.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.049+0000 7f25cf21d700 1 -- 172.21.15.62:0/2495829812 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c81a5740 con 0x7f25c81020a0 2024-06-10T00:41:32.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.051+0000 7f25c5ffb700 1 -- 172.21.15.62:0/2495829812 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f25b4004750 con 0x7f25c81020a0 2024-06-10T00:41:32.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.052+0000 7f25c5ffb700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f25b80a2d90 0x7f25b80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.052+0000 7f25ccfb9700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f25b80a2d90 0x7f25b80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.052+0000 7f25c5ffb700 1 -- 172.21.15.62:0/2495829812 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f25b40cf450 con 0x7f25c81020a0 2024-06-10T00:41:32.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.052+0000 7f25ccfb9700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f25b80a2d90 0x7f25b80a5250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f25bc005950 tx=0x7f25bc00a300 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:32.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25d9aa0b80 1 Processor -- start 2024-06-10T00:41:32.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25d9aa0b80 1 -- start start 2024-06-10T00:41:32.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25d9aa0b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a54640 0x559fd9a54a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25d9aa0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a4e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25d9aa0b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a4ef20 0x559fd9a4f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25d9aa0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559fd99a0810 con 0x559fd9a54f60 2024-06-10T00:41:32.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25d9aa0b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559fd9948720 con 0x559fd9a54640 2024-06-10T00:41:32.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25d9aa0b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559fd999c7d0 con 0x559fd9a4ef20 2024-06-10T00:41:32.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25c7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a4e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25c7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a4e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46268/0 (socket says 172.21.15.62:46268) 2024-06-10T00:41:32.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25c7fff700 1 -- 172.21.15.62:0/230141810 learned_addr learned my addr 172.21.15.62:0/230141810 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:32.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25ccfb9700 1 --2- 172.21.15.62:0/230141810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a54640 0x559fd9a54a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25cd7ba700 1 --2- 172.21.15.62:0/230141810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a4ef20 0x559fd9a4f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25c7fff700 1 -- 172.21.15.62:0/230141810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a4ef20 msgr2=0x559fd9a4f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25c7fff700 1 --2- 172.21.15.62:0/230141810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a4ef20 0x559fd9a4f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25c7fff700 1 -- 172.21.15.62:0/230141810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a54640 msgr2=0x559fd9a54a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25c7fff700 1 --2- 172.21.15.62:0/230141810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a54640 0x559fd9a54a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25c7fff700 1 -- 172.21.15.62:0/230141810 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fd993ef50 con 0x559fd9a54f60 2024-06-10T00:41:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.053+0000 7f25ccfb9700 1 --2- 172.21.15.62:0/230141810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a54640 0x559fd9a54a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.054+0000 7f25c7fff700 1 --2- 172.21.15.62:0/230141810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a4e9e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f25b40a3c40 tx=0x7f25b409b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72731213f785336c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:32.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.054+0000 7f25b2ffd700 1 -- 172.21.15.62:0/230141810 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b401d040 con 0x559fd9a54f60 2024-06-10T00:41:32.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.054+0000 7f25b2ffd700 1 -- 172.21.15.62:0/230141810 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b409b500 con 0x559fd9a54f60 2024-06-10T00:41:32.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.054+0000 7f25b2ffd700 1 -- 172.21.15.62:0/230141810 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b409b760 con 0x559fd9a54f60 2024-06-10T00:41:32.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.054+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/230141810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 msgr2=0x559fd9a4e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:32.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.054+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/230141810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a4e9e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f25b40a3c40 tx=0x7f25b409b3a0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/230141810 shutdown_connections 2024-06-10T00:41:32.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/230141810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a4ef20 0x559fd9a4f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/230141810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a54640 0x559fd9a54a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/230141810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a4e9e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/230141810 >> 172.21.15.62:0/230141810 conn(0x559fd9979cc0 msgr2=0x559fd9a49850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:32.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/230141810 shutdown_connections 2024-06-10T00:41:32.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/230141810 wait complete. 2024-06-10T00:41:32.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 -- start start 2024-06-10T00:41:32.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 Processor -- start 2024-06-10T00:41:32.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 0x559fd9a5ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54640 0x559fd9a58040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a58580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559fd9a5b9d0 con 0x559fd9a54f60 2024-06-10T00:41:32.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559fd9940bf0 con 0x559fd9a4ef20 2024-06-10T00:41:32.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.055+0000 7f25d9aa0b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559fd9a5bb50 con 0x559fd9a54640 2024-06-10T00:41:32.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25d9aa0b80 10 client.0 _ll_register_callbacks cb 0x559fd993a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:32.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25cd7ba700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a58580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25cd7ba700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a58580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46274/0 (socket says 172.21.15.62:46274) 2024-06-10T00:41:32.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25cd7ba700 1 -- 172.21.15.62:0/2065806658 learned_addr learned my addr 172.21.15.62:0/2065806658 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:32.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25ccfb9700 1 --2- 172.21.15.62:0/2065806658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 0x559fd9a5ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25c7fff700 1 --2- 172.21.15.62:0/2065806658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54640 0x559fd9a58040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25cd7ba700 1 -- 172.21.15.62:0/2065806658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54640 msgr2=0x559fd9a58040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:32.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25cd7ba700 1 --2- 172.21.15.62:0/2065806658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54640 0x559fd9a58040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25cd7ba700 1 -- 172.21.15.62:0/2065806658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 msgr2=0x559fd9a5ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:32.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25cd7ba700 1 --2- 172.21.15.62:0/2065806658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 0x559fd9a5ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25cd7ba700 1 -- 172.21.15.62:0/2065806658 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fd98ef8d0 con 0x559fd9a54f60 2024-06-10T00:41:32.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25c7fff700 1 --2- 172.21.15.62:0/2065806658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54640 0x559fd9a58040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:32.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25cd7ba700 1 --2- 172.21.15.62:0/2065806658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a58580 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f25c0004cf0 tx=0x7f25c0004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3b856a947e25c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:32.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:32.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25a7fff700 1 -- 172.21.15.62:0/2065806658 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0021070 con 0x559fd9a54f60 2024-06-10T00:41:32.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25a7fff700 1 -- 172.21.15.62:0/2065806658 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f25c00078f0 con 0x559fd9a54f60 2024-06-10T00:41:32.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2065806658 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25b409b500 con 0x559fd9a54f60 2024-06-10T00:41:32.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25a7fff700 1 -- 172.21.15.62:0/2065806658 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f25c002e050 con 0x559fd9a54f60 2024-06-10T00:41:32.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.056+0000 7f25d9aa0b80 10 client.16836 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:32.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.057+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2065806658 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559fd9a66360 con 0x559fd9a54f60 2024-06-10T00:41:32.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.057+0000 7f25a7fff700 1 -- 172.21.15.62:0/2065806658 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b409b500 con 0x559fd9a54f60 2024-06-10T00:41:32.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.057+0000 7f25a7fff700 1 -- 172.21.15.62:0/2065806658 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f25c0011070 con 0x559fd9a54f60 2024-06-10T00:41:32.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.057+0000 7f25d9aa0b80 10 client.16836 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:32.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.057+0000 7f25d9aa0b80 10 client.16836 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:41:32.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.057+0000 7f25d9aa0b80 20 client.16836 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:32.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.057+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2065806658 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x559fd9a689a0 0x559fd9a6ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.057+0000 7f25d9aa0b80 4 client.16836 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:41:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.057+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2065806658 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fd99add10 con 0x559fd9a689a0 2024-06-10T00:41:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.057+0000 7f25ccfb9700 1 --2- 172.21.15.62:0/2065806658 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x559fd9a689a0 0x559fd9a6ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.058+0000 7f25ccfb9700 1 --2- 172.21.15.62:0/2065806658 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x559fd9a689a0 0x559fd9a6ad90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac9ce11c2c8d2457 server_cookie=25f5da14c7aa31a9 in_seq=0 out_seq=0 2024-06-10T00:41:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.058+0000 7f25a7fff700 10 client.16836 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:41:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.058+0000 7f25a7fff700 1 -- 172.21.15.62:0/2065806658 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559fd99add10 con 0x559fd9a689a0 2024-06-10T00:41:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.058+0000 7f25a7fff700 10 client.16836 handle_command_reply: tid=0 2024-06-10T00:41:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.058+0000 7f25d9aa0b80 1 client.16836 shutdown 2024-06-10T00:41:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.059+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2065806658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 msgr2=0x559fd9a58580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.059+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2065806658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a58580 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f25c0004cf0 tx=0x7f25c0004dd0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.059+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2065806658 shutdown_connections 2024-06-10T00:41:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.059+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2065806658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a58580 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.059+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2065806658 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x559fd9a689a0 0x559fd9a6ad90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.059+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2065806658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 0x559fd9a5ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.059+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2065806658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54640 0x559fd9a58040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.059+0000 7f25a7fff700 0 client.16836 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:41:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.059+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2065806658 >> 172.21.15.62:0/2065806658 conn(0x559fd9979cc0 msgr2=0x559fd9a495f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.059+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2065806658 shutdown_connections 2024-06-10T00:41:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.059+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2065806658 wait complete. 2024-06-10T00:41:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.059+0000 7f25d9aa0b80 20 client.16836 trim_cache size 0 max 16384 2024-06-10T00:41:32.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.074+0000 7f25d9aa0b80 1 Processor -- start 2024-06-10T00:41:32.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.074+0000 7f25d9aa0b80 1 -- start start 2024-06-10T00:41:32.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.074+0000 7f25d9aa0b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 0x559fd9a4e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.074+0000 7f25d9aa0b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54640 0x559fd9a70620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.074+0000 7f25d9aa0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a70b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.074+0000 7f25d9aa0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81ae6c0 con 0x559fd9a54f60 2024-06-10T00:41:32.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.074+0000 7f25d9aa0b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81b0200 con 0x559fd9a4ef20 2024-06-10T00:41:32.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.074+0000 7f25d9aa0b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81aed90 con 0x559fd9a54640 2024-06-10T00:41:32.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.074+0000 7f25cd7ba700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a70b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.074+0000 7f25c7fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54640 0x559fd9a70620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.074+0000 7f25c7fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54640 0x559fd9a70620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:49042/0 (socket says 172.21.15.62:49042) 2024-06-10T00:41:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.074+0000 7f25cd7ba700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a70b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46280/0 (socket says 172.21.15.62:46280) 2024-06-10T00:41:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.074+0000 7f25c7fff700 1 -- 172.21.15.62:0/2456547031 learned_addr learned my addr 172.21.15.62:0/2456547031 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.075+0000 7f25ccfb9700 1 --2- 172.21.15.62:0/2456547031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 0x559fd9a4e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.075+0000 7f25c7fff700 1 -- 172.21.15.62:0/2456547031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 msgr2=0x559fd9a4e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.075+0000 7f25c7fff700 1 --2- 172.21.15.62:0/2456547031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 0x559fd9a4e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.075+0000 7f25c7fff700 1 -- 172.21.15.62:0/2456547031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 msgr2=0x559fd9a70b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.075+0000 7f25c7fff700 1 --2- 172.21.15.62:0/2456547031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a70b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.075+0000 7f25c7fff700 1 -- 172.21.15.62:0/2456547031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fd9a4b080 con 0x559fd9a54640 2024-06-10T00:41:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.075+0000 7f25cd7ba700 1 --2- 172.21.15.62:0/2456547031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a70b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:41:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.075+0000 7f25c7fff700 1 --2- 172.21.15.62:0/2456547031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54640 0x559fd9a70620 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f25b4097ed0 tx=0x7f25b400a470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62e397898c56d7ed server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.075+0000 7f25b27fc700 1 -- 172.21.15.62:0/2456547031 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b401d040 con 0x559fd9a54640 2024-06-10T00:41:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.075+0000 7f25b27fc700 1 -- 172.21.15.62:0/2456547031 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b40a4170 con 0x559fd9a54640 2024-06-10T00:41:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.075+0000 7f25b27fc700 1 -- 172.21.15.62:0/2456547031 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b409b760 con 0x559fd9a54640 2024-06-10T00:41:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.076+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2456547031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54640 msgr2=0x559fd9a70620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.076+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2456547031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54640 0x559fd9a70620 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f25b4097ed0 tx=0x7f25b400a470 comp rx=0 tx=0).stop 2024-06-10T00:41:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.076+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2456547031 shutdown_connections 2024-06-10T00:41:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.076+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2456547031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54f60 0x559fd9a70b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.076+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2456547031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 0x559fd9a4e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.076+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2456547031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54640 0x559fd9a70620 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.076+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2456547031 >> 172.21.15.62:0/2456547031 conn(0x559fd9979cc0 msgr2=0x559fd9a56950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.076+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2456547031 shutdown_connections 2024-06-10T00:41:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.076+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2456547031 wait complete. 2024-06-10T00:41:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.076+0000 7f25d9aa0b80 1 -- start start 2024-06-10T00:41:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.076+0000 7f25d9aa0b80 1 Processor -- start 2024-06-10T00:41:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25d9aa0b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 0x559fd9a613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25d9aa0b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54640 0x559fd9a6fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25d9aa0b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54f60 0x559fd9a702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25d9aa0b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81afb30 con 0x559fd9a54640 2024-06-10T00:41:32.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25d9aa0b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81af460 con 0x559fd9a4ef20 2024-06-10T00:41:32.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25d9aa0b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81ac680 con 0x559fd9a54f60 2024-06-10T00:41:32.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25d9aa0b80 10 client.0 _ll_register_callbacks cb 0x559fd993a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:32.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25cd7ba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54f60 0x559fd9a702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25c7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54640 0x559fd9a6fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25cd7ba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54f60 0x559fd9a702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:49048/0 (socket says 172.21.15.62:49048) 2024-06-10T00:41:32.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25cd7ba700 1 -- 172.21.15.62:0/2770155253 learned_addr learned my addr 172.21.15.62:0/2770155253 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25ccfb9700 1 --2- 172.21.15.62:0/2770155253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 0x559fd9a613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25cd7ba700 1 -- 172.21.15.62:0/2770155253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 msgr2=0x559fd9a613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25cd7ba700 1 --2- 172.21.15.62:0/2770155253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 0x559fd9a613f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25cd7ba700 1 -- 172.21.15.62:0/2770155253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54640 msgr2=0x559fd9a6fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25cd7ba700 1 --2- 172.21.15.62:0/2770155253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54640 0x559fd9a6fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25cd7ba700 1 -- 172.21.15.62:0/2770155253 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fd9a709d0 con 0x559fd9a54f60 2024-06-10T00:41:32.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.077+0000 7f25c7fff700 1 --2- 172.21.15.62:0/2770155253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54640 0x559fd9a6fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:41:32.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25cd7ba700 1 --2- 172.21.15.62:0/2770155253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54f60 0x559fd9a702f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009a20 tx=0x7f25c0004f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e31629967bb3c413 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:32.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:41:32.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25a7fff700 1 -- 172.21.15.62:0/2770155253 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0036030 con 0x559fd9a54f60 2024-06-10T00:41:32.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2770155253 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25b40a4170 con 0x559fd9a54f60 2024-06-10T00:41:32.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25a7fff700 1 -- 172.21.15.62:0/2770155253 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f25c002b070 con 0x559fd9a54f60 2024-06-10T00:41:32.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25a7fff700 1 -- 172.21.15.62:0/2770155253 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f25c0040050 con 0x559fd9a54f60 2024-06-10T00:41:32.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25d9aa0b80 10 client.26236 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2770155253 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559fd9ad34c0 con 0x559fd9a54f60 2024-06-10T00:41:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25a7fff700 1 -- 172.21.15.62:0/2770155253 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b40a4170 con 0x559fd9a54f60 2024-06-10T00:41:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25a7fff700 1 -- 172.21.15.62:0/2770155253 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f25c0011050 con 0x559fd9a54f60 2024-06-10T00:41:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25d9aa0b80 10 client.26236 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25d9aa0b80 10 client.26236 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:41:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25d9aa0b80 20 client.26236 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2770155253 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x559fd9ad3b30 0x559fd9ad5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25d9aa0b80 4 client.26236 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:41:32.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.078+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2770155253 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559fd99a8290 con 0x559fd9ad3b30 2024-06-10T00:41:32.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.079+0000 7f25ccfb9700 1 --2- 172.21.15.62:0/2770155253 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x559fd9ad3b30 0x559fd9ad5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:32.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.079+0000 7f25ccfb9700 1 --2- 172.21.15.62:0/2770155253 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x559fd9ad3b30 0x559fd9ad5f50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5dc331e9fcf8926a server_cookie=adf9aa850c701917 in_seq=0 out_seq=0 2024-06-10T00:41:32.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.079+0000 7f25a7fff700 10 client.26236 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:41:32.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.079+0000 7f25a7fff700 1 -- 172.21.15.62:0/2770155253 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559fd99a8290 con 0x559fd9ad3b30 2024-06-10T00:41:32.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.079+0000 7f25a7fff700 10 client.26236 handle_command_reply: tid=0 2024-06-10T00:41:32.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.079+0000 7f25d9aa0b80 1 client.26236 shutdown 2024-06-10T00:41:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.080+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2770155253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54f60 msgr2=0x559fd9a702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.080+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2770155253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54f60 0x559fd9a702f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009a20 tx=0x7f25c0004f60 comp rx=0 tx=0).stop 2024-06-10T00:41:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.080+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2770155253 shutdown_connections 2024-06-10T00:41:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.080+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2770155253 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x559fd9ad3b30 0x559fd9ad5f50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.080+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2770155253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559fd9a54f60 0x559fd9a702f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.080+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2770155253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559fd9a4ef20 0x559fd9a613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.080+0000 7f25a7fff700 0 client.26236 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:41:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.080+0000 7f25d9aa0b80 1 --2- 172.21.15.62:0/2770155253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559fd9a54640 0x559fd9a6fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.080+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2770155253 >> 172.21.15.62:0/2770155253 conn(0x559fd9979cc0 msgr2=0x559fd9a42e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:32.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.080+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2770155253 shutdown_connections 2024-06-10T00:41:32.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.080+0000 7f25d9aa0b80 1 -- 172.21.15.62:0/2770155253 wait complete. 2024-06-10T00:41:32.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.080+0000 7f25d9aa0b80 20 client.26236 trim_cache size 0 max 16384 2024-06-10T00:41:32.090 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:41:32.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.080+0000 7f25cf21d700 1 -- 172.21.15.62:0/2495829812 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f25b80a2d90 msgr2=0x7f25b80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:32.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.080+0000 7f25cf21d700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f25b80a2d90 0x7f25b80a5250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f25bc005950 tx=0x7f25bc00a300 comp rx=0 tx=0).stop 2024-06-10T00:41:32.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.081+0000 7f25cf21d700 1 -- 172.21.15.62:0/2495829812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25c81020a0 msgr2=0x7f25c8199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:32.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.081+0000 7f25cf21d700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25c81020a0 0x7f25c8199f30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f25b400c920 tx=0x7f25b400cc30 comp rx=0 tx=0).stop 2024-06-10T00:41:32.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.081+0000 7f25cf21d700 1 -- 172.21.15.62:0/2495829812 shutdown_connections 2024-06-10T00:41:32.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.081+0000 7f25cf21d700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f25b80a2d90 0x7f25b80a5250 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.081+0000 7f25cf21d700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25c8106410 0x7f25c81a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.081+0000 7f25cf21d700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c81016e0 0x7f25c81999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.081+0000 7f25cf21d700 1 --2- 172.21.15.62:0/2495829812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f25c81020a0 0x7f25c8199f30 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:32.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.081+0000 7f25cf21d700 1 -- 172.21.15.62:0/2495829812 >> 172.21.15.62:0/2495829812 conn(0x7f25c8077200 msgr2=0x7f25c80ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:32.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.081+0000 7f25cf21d700 1 -- 172.21.15.62:0/2495829812 shutdown_connections 2024-06-10T00:41:32.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:32.081+0000 7f25cf21d700 1 -- 172.21.15.62:0/2495829812 wait complete. 2024-06-10T00:41:32.770 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:32 smithi062 ceph-mon[32390]: pgmap v268: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:41:32.800 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T00:41:32.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:32 smithi138 ceph-mon[36549]: pgmap v268: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:41:33.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:32 smithi105 ceph-mon[36321]: pgmap v268: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:41:33.051 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:41:34.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.739+0000 7f66569da700 1 -- 172.21.15.62:0/686664394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650101380 msgr2=0x7f6650101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:34.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.739+0000 7f66569da700 1 --2- 172.21.15.62:0/686664394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650101380 0x7f6650101800 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6644009990 tx=0x7f6644009ca0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.739+0000 7f66569da700 1 -- 172.21.15.62:0/686664394 shutdown_connections 2024-06-10T00:41:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.739+0000 7f66569da700 1 --2- 172.21.15.62:0/686664394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6650101d40 0x7f665010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.739+0000 7f66569da700 1 --2- 172.21.15.62:0/686664394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66501073f0 0x7f66501077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.739+0000 7f66569da700 1 --2- 172.21.15.62:0/686664394 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650101380 0x7f6650101800 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.739+0000 7f66569da700 1 -- 172.21.15.62:0/686664394 >> 172.21.15.62:0/686664394 conn(0x7f6650078300 msgr2=0x7f6650078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.739+0000 7f66569da700 1 -- 172.21.15.62:0/686664394 shutdown_connections 2024-06-10T00:41:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.739+0000 7f66569da700 1 -- 172.21.15.62:0/686664394 wait complete. 2024-06-10T00:41:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.740+0000 7f66569da700 1 Processor -- start 2024-06-10T00:41:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.740+0000 7f66569da700 1 -- start start 2024-06-10T00:41:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.740+0000 7f66569da700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6650101380 0x7f6650199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.740+0000 7f66569da700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650101d40 0x7f6650199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.740+0000 7f66569da700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66501073f0 0x7f66501a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.740+0000 7f66569da700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6650111690 con 0x7f6650101d40 2024-06-10T00:41:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.740+0000 7f66569da700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6650111510 con 0x7f6650101380 2024-06-10T00:41:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.740+0000 7f66569da700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6650111810 con 0x7f66501073f0 2024-06-10T00:41:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.740+0000 7f664f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650101d40 0x7f6650199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.740+0000 7f664f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650101d40 0x7f6650199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46304/0 (socket says 172.21.15.62:46304) 2024-06-10T00:41:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.740+0000 7f664f7fe700 1 -- 172.21.15.62:0/192742530 learned_addr learned my addr 172.21.15.62:0/192742530 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.740+0000 7f6654f77700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66501073f0 0x7f66501a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.741+0000 7f664ffff700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6650101380 0x7f6650199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.741+0000 7f664f7fe700 1 -- 172.21.15.62:0/192742530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66501073f0 msgr2=0x7f66501a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.741+0000 7f664f7fe700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66501073f0 0x7f66501a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.741+0000 7f664f7fe700 1 -- 172.21.15.62:0/192742530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6650101380 msgr2=0x7f6650199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.741+0000 7f664f7fe700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6650101380 0x7f6650199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.741+0000 7f664f7fe700 1 -- 172.21.15.62:0/192742530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66501a5010 con 0x7f6650101d40 2024-06-10T00:41:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.741+0000 7f6654f77700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66501073f0 0x7f66501a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.741+0000 7f664f7fe700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650101d40 0x7f6650199f90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6644000c00 tx=0x7f664400bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.741+0000 7f664d7fa700 1 -- 172.21.15.62:0/192742530 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644027070 con 0x7f6650101d40 2024-06-10T00:41:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.741+0000 7f66569da700 1 -- 172.21.15.62:0/192742530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66501a51e0 con 0x7f6650101d40 2024-06-10T00:41:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.741+0000 7f664d7fa700 1 -- 172.21.15.62:0/192742530 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6644003d70 con 0x7f6650101d40 2024-06-10T00:41:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.741+0000 7f664d7fa700 1 -- 172.21.15.62:0/192742530 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664400f460 con 0x7f6650101d40 2024-06-10T00:41:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.741+0000 7f66569da700 1 -- 172.21.15.62:0/192742530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66501a5690 con 0x7f6650101d40 2024-06-10T00:41:34.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.743+0000 7f664d7fa700 1 -- 172.21.15.62:0/192742530 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f664402a070 con 0x7f6650101d40 2024-06-10T00:41:34.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.745+0000 7f664d7fa700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f66380a2d90 0x7f66380a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.745+0000 7f664ffff700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f66380a2d90 0x7f66380a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.745+0000 7f664d7fa700 1 -- 172.21.15.62:0/192742530 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f66440d07d0 con 0x7f6650101d40 2024-06-10T00:41:34.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.745+0000 7f664ffff700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f66380a2d90 0x7f66380a5250 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6640009bd0 tx=0x7f6640009260 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:34.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.745+0000 7f666125db80 1 Processor -- start 2024-06-10T00:41:34.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.745+0000 7f666125db80 1 -- start start 2024-06-10T00:41:34.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f666125db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 0x55e1a8269340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f666125db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a8269880 0x55e1a8263260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f666125db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a82637a0 0x55e1a8263bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f666125db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e1a8155f60 con 0x55e1a8269880 2024-06-10T00:41:34.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f666125db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e1a81b4850 con 0x55e1a82637a0 2024-06-10T00:41:34.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f666125db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e1a811c460 con 0x55e1a8268f60 2024-06-10T00:41:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f664f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a8269880 0x55e1a8263260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f664f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a8269880 0x55e1a8263260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46320/0 (socket says 172.21.15.62:46320) 2024-06-10T00:41:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f664f7fe700 1 -- 172.21.15.62:0/825542843 learned_addr learned my addr 172.21.15.62:0/825542843 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f664ffff700 1 --2- 172.21.15.62:0/825542843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 0x55e1a8269340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f6654f77700 1 --2- 172.21.15.62:0/825542843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a82637a0 0x55e1a8263bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f664f7fe700 1 -- 172.21.15.62:0/825542843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 msgr2=0x55e1a8269340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f664f7fe700 1 --2- 172.21.15.62:0/825542843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 0x55e1a8269340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f664f7fe700 1 -- 172.21.15.62:0/825542843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a82637a0 msgr2=0x55e1a8263bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f664f7fe700 1 --2- 172.21.15.62:0/825542843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a82637a0 0x55e1a8263bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f664f7fe700 1 -- 172.21.15.62:0/825542843 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a81038d0 con 0x55e1a8269880 2024-06-10T00:41:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f6654f77700 1 --2- 172.21.15.62:0/825542843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a82637a0 0x55e1a8263bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.746+0000 7f664f7fe700 1 --2- 172.21.15.62:0/825542843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a8269880 0x55e1a8263260 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f66440a4fc0 tx=0x7f6644009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb007bdb4d88f38d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.747+0000 7f66367fc700 1 -- 172.21.15.62:0/825542843 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664409c910 con 0x55e1a8269880 2024-06-10T00:41:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.747+0000 7f66367fc700 1 -- 172.21.15.62:0/825542843 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6644085e50 con 0x55e1a8269880 2024-06-10T00:41:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.747+0000 7f66367fc700 1 -- 172.21.15.62:0/825542843 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66440870e0 con 0x55e1a8269880 2024-06-10T00:41:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.747+0000 7f666125db80 1 -- 172.21.15.62:0/825542843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a8269880 msgr2=0x55e1a8263260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.747+0000 7f666125db80 1 --2- 172.21.15.62:0/825542843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a8269880 0x55e1a8263260 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f66440a4fc0 tx=0x7f6644009ca0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.747+0000 7f666125db80 1 -- 172.21.15.62:0/825542843 shutdown_connections 2024-06-10T00:41:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.747+0000 7f666125db80 1 --2- 172.21.15.62:0/825542843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a82637a0 0x55e1a8263bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.747+0000 7f666125db80 1 --2- 172.21.15.62:0/825542843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 0x55e1a8269340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.747+0000 7f666125db80 1 --2- 172.21.15.62:0/825542843 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a8269880 0x55e1a8263260 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.747+0000 7f666125db80 1 -- 172.21.15.62:0/825542843 >> 172.21.15.62:0/825542843 conn(0x55e1a8190080 msgr2=0x55e1a825e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.747+0000 7f666125db80 1 -- 172.21.15.62:0/825542843 shutdown_connections 2024-06-10T00:41:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.748+0000 7f666125db80 1 -- 172.21.15.62:0/825542843 wait complete. 2024-06-10T00:41:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.748+0000 7f666125db80 1 -- start start 2024-06-10T00:41:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.748+0000 7f666125db80 1 Processor -- start 2024-06-10T00:41:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.748+0000 7f666125db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a82637a0 0x55e1a826f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.748+0000 7f666125db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 0x55e1a826c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.748+0000 7f666125db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8269880 0x55e1a826ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.748+0000 7f666125db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e1a8154fc0 con 0x55e1a82637a0 2024-06-10T00:41:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.748+0000 7f666125db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e1a81b0810 con 0x55e1a8269880 2024-06-10T00:41:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f666125db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e1a8270010 con 0x55e1a8268f60 2024-06-10T00:41:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f666125db80 10 client.0 _ll_register_callbacks cb 0x55e1a814a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f664ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a82637a0 0x55e1a826f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f664ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a82637a0 0x55e1a826f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46326/0 (socket says 172.21.15.62:46326) 2024-06-10T00:41:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f664f7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 0x55e1a826c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f664ffff700 1 -- 172.21.15.62:0/1079810122 learned_addr learned my addr 172.21.15.62:0/1079810122 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:34.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f6654f77700 1 --2- 172.21.15.62:0/1079810122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8269880 0x55e1a826ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f664ffff700 1 -- 172.21.15.62:0/1079810122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 msgr2=0x55e1a826c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:34.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f664ffff700 1 --2- 172.21.15.62:0/1079810122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 0x55e1a826c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f664ffff700 1 -- 172.21.15.62:0/1079810122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8269880 msgr2=0x55e1a826ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:34.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f664ffff700 1 --2- 172.21.15.62:0/1079810122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8269880 0x55e1a826ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f664ffff700 1 -- 172.21.15.62:0/1079810122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a817a750 con 0x55e1a82637a0 2024-06-10T00:41:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f664f7fe700 1 --2- 172.21.15.62:0/1079810122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 0x55e1a826c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:41:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f664ffff700 1 --2- 172.21.15.62:0/1079810122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a82637a0 0x55e1a826f5e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f66400118f0 tx=0x7f66400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f7dfea9d3c0f6b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f662f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f662f7fe700 1 -- 172.21.15.62:0/1079810122 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640028670 con 0x55e1a82637a0 2024-06-10T00:41:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.749+0000 7f666125db80 1 -- 172.21.15.62:0/1079810122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6644085e50 con 0x55e1a82637a0 2024-06-10T00:41:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.750+0000 7f662f7fe700 1 -- 172.21.15.62:0/1079810122 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f6640028810 con 0x55e1a82637a0 2024-06-10T00:41:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.750+0000 7f662f7fe700 1 -- 172.21.15.62:0/1079810122 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6640031050 con 0x55e1a82637a0 2024-06-10T00:41:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.750+0000 7f666125db80 10 client.16854 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.750+0000 7f666125db80 1 -- 172.21.15.62:0/1079810122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e1a827a7f0 con 0x55e1a82637a0 2024-06-10T00:41:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.750+0000 7f662f7fe700 1 -- 172.21.15.62:0/1079810122 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6644085e50 con 0x55e1a82637a0 2024-06-10T00:41:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.750+0000 7f662f7fe700 1 -- 172.21.15.62:0/1079810122 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f664001e430 con 0x55e1a82637a0 2024-06-10T00:41:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.750+0000 7f666125db80 10 client.16854 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.750+0000 7f666125db80 10 client.16854 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:41:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.750+0000 7f666125db80 20 client.16854 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:34.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.750+0000 7f666125db80 1 --2- 172.21.15.62:0/1079810122 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55e1a827ce30 0x55e1a827f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.750+0000 7f666125db80 4 client.16854 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:41:34.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.750+0000 7f666125db80 1 -- 172.21.15.62:0/1079810122 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1a81c1d50 con 0x55e1a827ce30 2024-06-10T00:41:34.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.751+0000 7f664f7fe700 1 --2- 172.21.15.62:0/1079810122 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55e1a827ce30 0x55e1a827f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.751+0000 7f662f7fe700 10 client.16854 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:41:34.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.751+0000 7f664f7fe700 1 --2- 172.21.15.62:0/1079810122 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55e1a827ce30 0x55e1a827f220 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c19ede8a43ec7e5 server_cookie=cad407352100a8b in_seq=0 out_seq=0 2024-06-10T00:41:34.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f662f7fe700 1 -- 172.21.15.62:0/1079810122 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e1a81c1d50 con 0x55e1a827ce30 2024-06-10T00:41:34.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f662f7fe700 10 client.16854 handle_command_reply: tid=0 2024-06-10T00:41:34.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f666125db80 1 client.16854 shutdown 2024-06-10T00:41:34.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f666125db80 1 -- 172.21.15.62:0/1079810122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a82637a0 msgr2=0x55e1a826f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:34.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f666125db80 1 --2- 172.21.15.62:0/1079810122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a82637a0 0x55e1a826f5e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f66400118f0 tx=0x7f66400047c0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f666125db80 1 -- 172.21.15.62:0/1079810122 shutdown_connections 2024-06-10T00:41:34.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f666125db80 1 --2- 172.21.15.62:0/1079810122 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55e1a827ce30 0x55e1a827f220 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f666125db80 1 --2- 172.21.15.62:0/1079810122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a82637a0 0x55e1a826f5e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f666125db80 1 --2- 172.21.15.62:0/1079810122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8269880 0x55e1a826ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f666125db80 1 --2- 172.21.15.62:0/1079810122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 0x55e1a826c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f662f7fe700 0 client.16854 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:41:34.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f666125db80 1 -- 172.21.15.62:0/1079810122 >> 172.21.15.62:0/1079810122 conn(0x55e1a8190080 msgr2=0x55e1a825d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:34.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f666125db80 1 -- 172.21.15.62:0/1079810122 shutdown_connections 2024-06-10T00:41:34.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f666125db80 1 -- 172.21.15.62:0/1079810122 wait complete. 2024-06-10T00:41:34.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.752+0000 7f666125db80 20 client.16854 trim_cache size 0 max 16384 2024-06-10T00:41:34.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.768+0000 7f666125db80 1 Processor -- start 2024-06-10T00:41:34.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.768+0000 7f666125db80 1 -- start start 2024-06-10T00:41:34.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.768+0000 7f666125db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a82637a0 0x55e1a826ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.768+0000 7f666125db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8268f60 0x55e1a826e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.768+0000 7f666125db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a8269880 0x55e1a825de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.768+0000 7f666125db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66501ae6b0 con 0x55e1a8269880 2024-06-10T00:41:34.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.768+0000 7f666125db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f66501b01f0 con 0x55e1a8268f60 2024-06-10T00:41:34.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.768+0000 7f666125db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66501aed80 con 0x55e1a82637a0 2024-06-10T00:41:34.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.768+0000 7f6654f77700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a8269880 0x55e1a825de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.768+0000 7f6654f77700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a8269880 0x55e1a825de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46340/0 (socket says 172.21.15.62:46340) 2024-06-10T00:41:34.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.768+0000 7f6654f77700 1 -- 172.21.15.62:0/3408518086 learned_addr learned my addr 172.21.15.62:0/3408518086 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:34.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.768+0000 7f664ffff700 1 --2- 172.21.15.62:0/3408518086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a82637a0 0x55e1a826ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.768+0000 7f664f7fe700 1 --2- 172.21.15.62:0/3408518086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8268f60 0x55e1a826e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.769+0000 7f6654f77700 1 -- 172.21.15.62:0/3408518086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a82637a0 msgr2=0x55e1a826ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:34.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.769+0000 7f6654f77700 1 --2- 172.21.15.62:0/3408518086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a82637a0 0x55e1a826ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.769+0000 7f6654f77700 1 -- 172.21.15.62:0/3408518086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8268f60 msgr2=0x55e1a826e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:34.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.769+0000 7f6654f77700 1 --2- 172.21.15.62:0/3408518086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8268f60 0x55e1a826e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.769+0000 7f6654f77700 1 -- 172.21.15.62:0/3408518086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a827f850 con 0x55e1a8269880 2024-06-10T00:41:34.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.769+0000 7f664f7fe700 1 --2- 172.21.15.62:0/3408518086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8268f60 0x55e1a826e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:34.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.769+0000 7f664ffff700 1 --2- 172.21.15.62:0/3408518086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a82637a0 0x55e1a826ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:34.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.769+0000 7f6654f77700 1 --2- 172.21.15.62:0/3408518086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a8269880 0x55e1a825de80 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6648007670 tx=0x7f664800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861be24e08dfa9a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:34.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.769+0000 7f6635ffb700 1 -- 172.21.15.62:0/3408518086 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664801b070 con 0x55e1a8269880 2024-06-10T00:41:34.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.769+0000 7f6635ffb700 1 -- 172.21.15.62:0/3408518086 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6648017d30 con 0x55e1a8269880 2024-06-10T00:41:34.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.769+0000 7f6635ffb700 1 -- 172.21.15.62:0/3408518086 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648016e60 con 0x55e1a8269880 2024-06-10T00:41:34.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.770+0000 7f666125db80 1 -- 172.21.15.62:0/3408518086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a8269880 msgr2=0x55e1a825de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:34.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.770+0000 7f666125db80 1 --2- 172.21.15.62:0/3408518086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a8269880 0x55e1a825de80 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6648007670 tx=0x7f664800b040 comp rx=0 tx=0).stop 2024-06-10T00:41:34.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.770+0000 7f666125db80 1 -- 172.21.15.62:0/3408518086 shutdown_connections 2024-06-10T00:41:34.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.770+0000 7f666125db80 1 --2- 172.21.15.62:0/3408518086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a8269880 0x55e1a825de80 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.770+0000 7f666125db80 1 --2- 172.21.15.62:0/3408518086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a82637a0 0x55e1a826ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.770+0000 7f666125db80 1 --2- 172.21.15.62:0/3408518086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8268f60 0x55e1a826e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.770+0000 7f666125db80 1 -- 172.21.15.62:0/3408518086 >> 172.21.15.62:0/3408518086 conn(0x55e1a8190080 msgr2=0x55e1a826e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.770+0000 7f666125db80 1 -- 172.21.15.62:0/3408518086 shutdown_connections 2024-06-10T00:41:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.770+0000 7f666125db80 1 -- 172.21.15.62:0/3408518086 wait complete. 2024-06-10T00:41:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.770+0000 7f666125db80 1 -- start start 2024-06-10T00:41:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f666125db80 1 Processor -- start 2024-06-10T00:41:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f666125db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a82637a0 0x55e1a8275480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f666125db80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 0x55e1a82759c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f666125db80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8269880 0x55e1a8283c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f666125db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66501afb20 con 0x55e1a82637a0 2024-06-10T00:41:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f666125db80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f66501af450 con 0x55e1a8269880 2024-06-10T00:41:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f666125db80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66501ac670 con 0x55e1a8268f60 2024-06-10T00:41:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f666125db80 10 client.0 _ll_register_callbacks cb 0x55e1a814a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f664ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a82637a0 0x55e1a8275480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f664ffff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a82637a0 0x55e1a8275480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46354/0 (socket says 172.21.15.62:46354) 2024-06-10T00:41:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f664ffff700 1 -- 172.21.15.62:0/1100293558 learned_addr learned my addr 172.21.15.62:0/1100293558 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f664f7fe700 1 --2- 172.21.15.62:0/1100293558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 0x55e1a82759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f6654f77700 1 --2- 172.21.15.62:0/1100293558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8269880 0x55e1a8283c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f664ffff700 1 -- 172.21.15.62:0/1100293558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 msgr2=0x55e1a82759c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f664ffff700 1 --2- 172.21.15.62:0/1100293558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 0x55e1a82759c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f664ffff700 1 -- 172.21.15.62:0/1100293558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8269880 msgr2=0x55e1a8283c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f664ffff700 1 --2- 172.21.15.62:0/1100293558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8269880 0x55e1a8283c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.771+0000 7f664ffff700 1 -- 172.21.15.62:0/1100293558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a8284280 con 0x55e1a82637a0 2024-06-10T00:41:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f664ffff700 1 --2- 172.21.15.62:0/1100293558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a82637a0 0x55e1a8275480 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f66400097b0 tx=0x7f6640013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e636584c5ae8283e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f662f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f662f7fe700 1 -- 172.21.15.62:0/1100293558 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640010a50 con 0x55e1a82637a0 2024-06-10T00:41:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f666125db80 1 -- 172.21.15.62:0/1100293558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6648017d30 con 0x55e1a82637a0 2024-06-10T00:41:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f662f7fe700 1 -- 172.21.15.62:0/1100293558 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f664001ebf0 con 0x55e1a82637a0 2024-06-10T00:41:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f662f7fe700 1 -- 172.21.15.62:0/1100293558 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f664003e470 con 0x55e1a82637a0 2024-06-10T00:41:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f666125db80 10 client.16860 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f666125db80 1 -- 172.21.15.62:0/1100293558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e1a82e7390 con 0x55e1a82637a0 2024-06-10T00:41:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f662f7fe700 1 -- 172.21.15.62:0/1100293558 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6648017d30 con 0x55e1a82637a0 2024-06-10T00:41:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f662f7fe700 1 -- 172.21.15.62:0/1100293558 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f6640028770 con 0x55e1a82637a0 2024-06-10T00:41:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f666125db80 10 client.16860 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f666125db80 10 client.16860 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:41:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f666125db80 20 client.16860 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f666125db80 1 --2- 172.21.15.62:0/1100293558 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55e1a82e7aa0 0x55e1a82e9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f666125db80 4 client.16860 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:41:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.772+0000 7f666125db80 1 -- 172.21.15.62:0/1100293558 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1a81bc2d0 con 0x55e1a82e7aa0 2024-06-10T00:41:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.773+0000 7f664f7fe700 1 --2- 172.21.15.62:0/1100293558 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55e1a82e7aa0 0x55e1a82e9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:34.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.773+0000 7f662f7fe700 10 client.16860 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:41:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.773+0000 7f664f7fe700 1 --2- 172.21.15.62:0/1100293558 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55e1a82e7aa0 0x55e1a82e9ee0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f98dbc552a5c0355 server_cookie=42a9b3791be616f4 in_seq=0 out_seq=0 2024-06-10T00:41:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.773+0000 7f662f7fe700 1 -- 172.21.15.62:0/1100293558 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1a81bc2d0 con 0x55e1a82e7aa0 2024-06-10T00:41:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.773+0000 7f662f7fe700 10 client.16860 handle_command_reply: tid=0 2024-06-10T00:41:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.773+0000 7f666125db80 1 client.16860 shutdown 2024-06-10T00:41:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.773+0000 7f666125db80 1 -- 172.21.15.62:0/1100293558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a82637a0 msgr2=0x55e1a8275480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.773+0000 7f666125db80 1 --2- 172.21.15.62:0/1100293558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a82637a0 0x55e1a8275480 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f66400097b0 tx=0x7f6640013000 comp rx=0 tx=0).stop 2024-06-10T00:41:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f666125db80 1 -- 172.21.15.62:0/1100293558 shutdown_connections 2024-06-10T00:41:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f666125db80 1 --2- 172.21.15.62:0/1100293558 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x55e1a82e7aa0 0x55e1a82e9ee0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f666125db80 1 --2- 172.21.15.62:0/1100293558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e1a82637a0 0x55e1a8275480 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f662f7fe700 0 client.16860 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:41:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f666125db80 1 --2- 172.21.15.62:0/1100293558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1a8269880 0x55e1a8283c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f666125db80 1 --2- 172.21.15.62:0/1100293558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a8268f60 0x55e1a82759c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f666125db80 1 -- 172.21.15.62:0/1100293558 >> 172.21.15.62:0/1100293558 conn(0x55e1a8190080 msgr2=0x55e1a826e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f666125db80 1 -- 172.21.15.62:0/1100293558 shutdown_connections 2024-06-10T00:41:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f666125db80 1 -- 172.21.15.62:0/1100293558 wait complete. 2024-06-10T00:41:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f666125db80 20 client.16860 trim_cache size 0 max 16384 2024-06-10T00:41:34.786 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:41:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f66569da700 1 -- 172.21.15.62:0/192742530 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f66380a2d90 msgr2=0x7f66380a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f66569da700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f66380a2d90 0x7f66380a5250 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6640009bd0 tx=0x7f6640009260 comp rx=0 tx=0).stop 2024-06-10T00:41:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f66569da700 1 -- 172.21.15.62:0/192742530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650101d40 msgr2=0x7f6650199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:34.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f66569da700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650101d40 0x7f6650199f90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6644000c00 tx=0x7f664400bb60 comp rx=0 tx=0).stop 2024-06-10T00:41:34.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f66569da700 1 -- 172.21.15.62:0/192742530 shutdown_connections 2024-06-10T00:41:34.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f66569da700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f66380a2d90 0x7f66380a5250 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f66569da700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66501073f0 0x7f66501a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f66569da700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6650101380 0x7f6650199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f66569da700 1 --2- 172.21.15.62:0/192742530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6650101d40 0x7f6650199f90 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.774+0000 7f66569da700 1 -- 172.21.15.62:0/192742530 >> 172.21.15.62:0/192742530 conn(0x7f6650078300 msgr2=0x7f66500ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.775+0000 7f66569da700 1 -- 172.21.15.62:0/192742530 shutdown_connections 2024-06-10T00:41:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:34.775+0000 7f66569da700 1 -- 172.21.15.62:0/192742530 wait complete. 2024-06-10T00:41:35.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:35 smithi062 ceph-mon[32390]: pgmap v269: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:41:35.453 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T00:41:35.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:35 smithi105 ceph-mon[36321]: pgmap v269: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:41:35.576 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:35 smithi138 ceph-mon[36549]: pgmap v269: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:41:35.707 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:41:37.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.350+0000 7fd1ba752700 1 -- 172.21.15.62:0/3808790571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b4102e80 msgr2=0x7fd1b410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:37.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.350+0000 7fd1ba752700 1 --2- 172.21.15.62:0/3808790571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b4102e80 0x7fd1b410f330 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00b0a0 tx=0x7fd1ac00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.350+0000 7fd1ba752700 1 -- 172.21.15.62:0/3808790571 shutdown_connections 2024-06-10T00:41:37.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.350+0000 7fd1ba752700 1 --2- 172.21.15.62:0/3808790571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b4102e80 0x7fd1b410f330 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.350+0000 7fd1ba752700 1 --2- 172.21.15.62:0/3808790571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1b41084f0 0x7fd1b41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.350+0000 7fd1ba752700 1 --2- 172.21.15.62:0/3808790571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41024e0 0x7fd1b4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.350+0000 7fd1ba752700 1 -- 172.21.15.62:0/3808790571 >> 172.21.15.62:0/3808790571 conn(0x7fd1b4079530 msgr2=0x7fd1b4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:37.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.350+0000 7fd1ba752700 1 -- 172.21.15.62:0/3808790571 shutdown_connections 2024-06-10T00:41:37.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.350+0000 7fd1ba752700 1 -- 172.21.15.62:0/3808790571 wait complete. 2024-06-10T00:41:37.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.351+0000 7fd1ba752700 1 Processor -- start 2024-06-10T00:41:37.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.351+0000 7fd1ba752700 1 -- start start 2024-06-10T00:41:37.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.351+0000 7fd1ba752700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1b41024e0 0x7fd1b419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.351+0000 7fd1ba752700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b4102e80 0x7fd1b419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.351+0000 7fd1ba752700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41084f0 0x7fd1b4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.351+0000 7fd1ba752700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4112740 con 0x7fd1b4102e80 2024-06-10T00:41:37.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.351+0000 7fd1ba752700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41125c0 con 0x7fd1b41084f0 2024-06-10T00:41:37.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.351+0000 7fd1ba752700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41128c0 con 0x7fd1b41024e0 2024-06-10T00:41:37.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.351+0000 7fd1abfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b4102e80 0x7fd1b419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.351+0000 7fd1abfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b4102e80 0x7fd1b419edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46382/0 (socket says 172.21.15.62:46382) 2024-06-10T00:41:37.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.351+0000 7fd1abfff700 1 -- 172.21.15.62:0/2428912830 learned_addr learned my addr 172.21.15.62:0/2428912830 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:37.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.351+0000 7fd1b8cef700 1 --2- 172.21.15.62:0/2428912830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41084f0 0x7fd1b4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.351+0000 7fd1b3fff700 1 --2- 172.21.15.62:0/2428912830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1b41024e0 0x7fd1b419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.352+0000 7fd1abfff700 1 -- 172.21.15.62:0/2428912830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1b41024e0 msgr2=0x7fd1b419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:37.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.352+0000 7fd1abfff700 1 --2- 172.21.15.62:0/2428912830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1b41024e0 0x7fd1b419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.352+0000 7fd1abfff700 1 -- 172.21.15.62:0/2428912830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41084f0 msgr2=0x7fd1b4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:37.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.352+0000 7fd1abfff700 1 --2- 172.21.15.62:0/2428912830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41084f0 0x7fd1b4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.352+0000 7fd1abfff700 1 -- 172.21.15.62:0/2428912830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b4199200 con 0x7fd1b4102e80 2024-06-10T00:41:37.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.352+0000 7fd1abfff700 1 --2- 172.21.15.62:0/2428912830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b4102e80 0x7fd1b419edd0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00c920 tx=0x7fd19c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:37.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.352+0000 7fd1b1ffb700 1 -- 172.21.15.62:0/2428912830 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c004a30 con 0x7fd1b4102e80 2024-06-10T00:41:37.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.352+0000 7fd1b1ffb700 1 -- 172.21.15.62:0/2428912830 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd19c005500 con 0x7fd1b4102e80 2024-06-10T00:41:37.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.352+0000 7fd1ba752700 1 -- 172.21.15.62:0/2428912830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b41994f0 con 0x7fd1b4102e80 2024-06-10T00:41:37.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.352+0000 7fd1b1ffb700 1 -- 172.21.15.62:0/2428912830 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c007d50 con 0x7fd1b4102e80 2024-06-10T00:41:37.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.352+0000 7fd1ba752700 1 -- 172.21.15.62:0/2428912830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b41a5760 con 0x7fd1b4102e80 2024-06-10T00:41:37.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.354+0000 7fd1b1ffb700 1 -- 172.21.15.62:0/2428912830 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd19c004d60 con 0x7fd1b4102e80 2024-06-10T00:41:37.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.355+0000 7fd1b1ffb700 1 --2- 172.21.15.62:0/2428912830 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd1a00a2d90 0x7fd1a00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.355+0000 7fd1b3fff700 1 --2- 172.21.15.62:0/2428912830 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd1a00a2d90 0x7fd1a00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.355+0000 7fd1b1ffb700 1 -- 172.21.15.62:0/2428912830 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fd19c0cf6f0 con 0x7fd1b4102e80 2024-06-10T00:41:37.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1b3fff700 1 --2- 172.21.15.62:0/2428912830 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd1a00a2d90 0x7fd1a00a5250 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4005950 tx=0x7fd1a400a300 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:37.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1c4fd5b80 1 Processor -- start 2024-06-10T00:41:37.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1c4fd5b80 1 -- start start 2024-06-10T00:41:37.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1c4fd5b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddcf60 0x55c8e3ddd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1c4fd5b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3ddd880 0x55c8e3dd7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1c4fd5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3dd77a0 0x55c8e3dd7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1c4fd5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c8e3cc9f60 con 0x55c8e3dd77a0 2024-06-10T00:41:37.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1c4fd5b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c8e3d28850 con 0x55c8e3ddcf60 2024-06-10T00:41:37.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1c4fd5b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8e3c90460 con 0x55c8e3ddd880 2024-06-10T00:41:37.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1b8cef700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3dd77a0 0x55c8e3dd7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1b8cef700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3dd77a0 0x55c8e3dd7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46384/0 (socket says 172.21.15.62:46384) 2024-06-10T00:41:37.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1b8cef700 1 -- 172.21.15.62:0/611543929 learned_addr learned my addr 172.21.15.62:0/611543929 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:37.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1abfff700 1 --2- 172.21.15.62:0/611543929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3ddd880 0x55c8e3dd7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1b3fff700 1 --2- 172.21.15.62:0/611543929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddcf60 0x55c8e3ddd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1b8cef700 1 -- 172.21.15.62:0/611543929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3ddd880 msgr2=0x55c8e3dd7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:37.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1b8cef700 1 --2- 172.21.15.62:0/611543929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3ddd880 0x55c8e3dd7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1b8cef700 1 -- 172.21.15.62:0/611543929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddcf60 msgr2=0x55c8e3ddd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:37.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1b8cef700 1 --2- 172.21.15.62:0/611543929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddcf60 0x55c8e3ddd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1b8cef700 1 -- 172.21.15.62:0/611543929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e3c778d0 con 0x55c8e3dd77a0 2024-06-10T00:41:37.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.356+0000 7fd1abfff700 1 --2- 172.21.15.62:0/611543929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3ddd880 0x55c8e3dd7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:37.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.357+0000 7fd1b8cef700 1 --2- 172.21.15.62:0/611543929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3dd77a0 0x55c8e3dd7bc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009390 tx=0x7fd1ac0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd8e6e0bed850b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:37.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.357+0000 7fd1aa7fc700 1 -- 172.21.15.62:0/611543929 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac020070 con 0x55c8e3dd77a0 2024-06-10T00:41:37.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.357+0000 7fd1aa7fc700 1 -- 172.21.15.62:0/611543929 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ac003c00 con 0x55c8e3dd77a0 2024-06-10T00:41:37.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.357+0000 7fd1aa7fc700 1 -- 172.21.15.62:0/611543929 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac005280 con 0x55c8e3dd77a0 2024-06-10T00:41:37.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.357+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/611543929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3dd77a0 msgr2=0x55c8e3dd7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:37.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.357+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/611543929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3dd77a0 0x55c8e3dd7bc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009390 tx=0x7fd1ac0093c0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.358+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/611543929 shutdown_connections 2024-06-10T00:41:37.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.358+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/611543929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3dd77a0 0x55c8e3dd7bc0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.358+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/611543929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddcf60 0x55c8e3ddd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.358+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/611543929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3ddd880 0x55c8e3dd7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.358+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/611543929 >> 172.21.15.62:0/611543929 conn(0x55c8e3d04080 msgr2=0x55c8e3dd2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:37.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.358+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/611543929 shutdown_connections 2024-06-10T00:41:37.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.358+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/611543929 wait complete. 2024-06-10T00:41:37.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.358+0000 7fd1c4fd5b80 1 -- start start 2024-06-10T00:41:37.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.358+0000 7fd1c4fd5b80 1 Processor -- start 2024-06-10T00:41:37.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.358+0000 7fd1c4fd5b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3dd77a0 0x55c8e3de3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.358+0000 7fd1c4fd5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3ddcf60 0x55c8e3de0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1c4fd5b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddd880 0x55c8e3de0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1c4fd5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c8e3cc8fc0 con 0x55c8e3ddcf60 2024-06-10T00:41:37.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1c4fd5b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c8e3d24810 con 0x55c8e3ddd880 2024-06-10T00:41:37.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1c4fd5b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8e3de4200 con 0x55c8e3dd77a0 2024-06-10T00:41:37.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1c4fd5b80 10 client.0 _ll_register_callbacks cb 0x55c8e3cbe5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:37.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1abfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3ddcf60 0x55c8e3de0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1abfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3ddcf60 0x55c8e3de0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46396/0 (socket says 172.21.15.62:46396) 2024-06-10T00:41:37.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1abfff700 1 -- 172.21.15.62:0/2618264190 learned_addr learned my addr 172.21.15.62:0/2618264190 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:37.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1b8cef700 1 --2- 172.21.15.62:0/2618264190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddd880 0x55c8e3de0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1b3fff700 1 --2- 172.21.15.62:0/2618264190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3dd77a0 0x55c8e3de3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1abfff700 1 -- 172.21.15.62:0/2618264190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3dd77a0 msgr2=0x55c8e3de3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:37.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1abfff700 1 --2- 172.21.15.62:0/2618264190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3dd77a0 0x55c8e3de3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1abfff700 1 -- 172.21.15.62:0/2618264190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddd880 msgr2=0x55c8e3de0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:37.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1abfff700 1 --2- 172.21.15.62:0/2618264190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddd880 0x55c8e3de0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1abfff700 1 -- 172.21.15.62:0/2618264190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e3cee750 con 0x55c8e3ddcf60 2024-06-10T00:41:37.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1abfff700 1 --2- 172.21.15.62:0/2618264190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3ddcf60 0x55c8e3de0860 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd19c09b790 tx=0x7fd19c09b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6025f857390ff904 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:37.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1937fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:37.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1937fe700 1 -- 172.21.15.62:0/2618264190 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c0b4060 con 0x55c8e3ddcf60 2024-06-10T00:41:37.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1937fe700 1 -- 172.21.15.62:0/2618264190 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fd19c097e70 con 0x55c8e3ddcf60 2024-06-10T00:41:37.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.359+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/2618264190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1ac003c00 con 0x55c8e3ddcf60 2024-06-10T00:41:37.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.360+0000 7fd1937fe700 1 -- 172.21.15.62:0/2618264190 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd19c01d040 con 0x55c8e3ddcf60 2024-06-10T00:41:37.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.360+0000 7fd1c4fd5b80 10 client.16872 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:37.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.360+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/2618264190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c8e3df2d80 con 0x55c8e3ddcf60 2024-06-10T00:41:37.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.360+0000 7fd1937fe700 1 -- 172.21.15.62:0/2618264190 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ac003c00 con 0x55c8e3ddcf60 2024-06-10T00:41:37.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.360+0000 7fd1937fe700 1 -- 172.21.15.62:0/2618264190 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fd19c09ba60 con 0x55c8e3ddcf60 2024-06-10T00:41:37.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.360+0000 7fd1c4fd5b80 10 client.16872 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:37.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.360+0000 7fd1c4fd5b80 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:41:37.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.360+0000 7fd1c4fd5b80 20 client.16872 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:37.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.360+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/2618264190 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c8e3df5310 0x55c8e3df7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.360+0000 7fd1c4fd5b80 4 client.16872 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:41:37.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.360+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/2618264190 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8e3d35d50 con 0x55c8e3df5310 2024-06-10T00:41:37.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.361+0000 7fd1b3fff700 1 --2- 172.21.15.62:0/2618264190 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c8e3df5310 0x55c8e3df7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.361+0000 7fd1b3fff700 1 --2- 172.21.15.62:0/2618264190 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c8e3df5310 0x55c8e3df7730 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc563039ab9abf35 server_cookie=dda19a4142a98f43 in_seq=0 out_seq=0 2024-06-10T00:41:37.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.361+0000 7fd1937fe700 10 client.16872 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:37.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1937fe700 1 -- 172.21.15.62:0/2618264190 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c8e3d35d50 con 0x55c8e3df5310 2024-06-10T00:41:37.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1937fe700 10 client.16872 handle_command_reply: tid=0 2024-06-10T00:41:37.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1c4fd5b80 1 client.16872 shutdown 2024-06-10T00:41:37.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/2618264190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3ddcf60 msgr2=0x55c8e3de0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:37.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/2618264190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3ddcf60 0x55c8e3de0860 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd19c09b790 tx=0x7fd19c09b3c0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/2618264190 shutdown_connections 2024-06-10T00:41:37.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/2618264190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddd880 0x55c8e3de0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/2618264190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3dd77a0 0x55c8e3de3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/2618264190 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c8e3df5310 0x55c8e3df7730 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1937fe700 0 client.16872 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:37.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/2618264190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3ddcf60 0x55c8e3de0860 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/2618264190 >> 172.21.15.62:0/2618264190 conn(0x55c8e3d04080 msgr2=0x55c8e3dd1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:37.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/2618264190 shutdown_connections 2024-06-10T00:41:37.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/2618264190 wait complete. 2024-06-10T00:41:37.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.362+0000 7fd1c4fd5b80 20 client.16872 trim_cache size 0 max 16384 2024-06-10T00:41:37.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.377+0000 7fd1c4fd5b80 1 Processor -- start 2024-06-10T00:41:37.378 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:37 smithi062 ceph-mon[32390]: pgmap v270: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:41:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.377+0000 7fd1c4fd5b80 1 -- start start 2024-06-10T00:41:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1c4fd5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3dd77a0 0x55c8e3dd7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1c4fd5b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddcf60 0x55c8e3ddf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1c4fd5b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3ddd880 0x55c8e3dd1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1c4fd5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41ac380 con 0x55c8e3dd77a0 2024-06-10T00:41:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1c4fd5b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41adec0 con 0x55c8e3ddcf60 2024-06-10T00:41:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1c4fd5b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41aca50 con 0x55c8e3ddd880 2024-06-10T00:41:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1b3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3dd77a0 0x55c8e3dd7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1b3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3dd77a0 0x55c8e3dd7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46408/0 (socket says 172.21.15.62:46408) 2024-06-10T00:41:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1b3fff700 1 -- 172.21.15.62:0/3360831560 learned_addr learned my addr 172.21.15.62:0/3360831560 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1abfff700 1 --2- 172.21.15.62:0/3360831560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddcf60 0x55c8e3ddf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1b3fff700 1 -- 172.21.15.62:0/3360831560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3ddd880 msgr2=0x55c8e3dd1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:37.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1b8cef700 1 --2- 172.21.15.62:0/3360831560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3ddd880 0x55c8e3dd1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1b3fff700 1 --2- 172.21.15.62:0/3360831560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3ddd880 0x55c8e3dd1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1b3fff700 1 -- 172.21.15.62:0/3360831560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddcf60 msgr2=0x55c8e3ddf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:37.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1b3fff700 1 --2- 172.21.15.62:0/3360831560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddcf60 0x55c8e3ddf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1b3fff700 1 -- 172.21.15.62:0/3360831560 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e3ddedd0 con 0x55c8e3dd77a0 2024-06-10T00:41:37.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1b8cef700 1 --2- 172.21.15.62:0/3360831560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3ddd880 0x55c8e3dd1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:41:37.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1abfff700 1 --2- 172.21.15.62:0/3360831560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddcf60 0x55c8e3ddf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:41:37.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.378+0000 7fd1b3fff700 1 --2- 172.21.15.62:0/3360831560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3dd77a0 0x55c8e3dd7260 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x55c8e3df7f70 tx=0x7fd1a4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833572ef4745d837 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:37.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.379+0000 7fd1a9ffb700 1 -- 172.21.15.62:0/3360831560 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4016070 con 0x55c8e3dd77a0 2024-06-10T00:41:37.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.379+0000 7fd1a9ffb700 1 -- 172.21.15.62:0/3360831560 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a402a030 con 0x55c8e3dd77a0 2024-06-10T00:41:37.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.379+0000 7fd1a9ffb700 1 -- 172.21.15.62:0/3360831560 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4010430 con 0x55c8e3dd77a0 2024-06-10T00:41:37.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.379+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/3360831560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3dd77a0 msgr2=0x55c8e3dd7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:37.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.379+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/3360831560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3dd77a0 0x55c8e3dd7260 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x55c8e3df7f70 tx=0x7fd1a4012000 comp rx=0 tx=0).stop 2024-06-10T00:41:37.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.379+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/3360831560 shutdown_connections 2024-06-10T00:41:37.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.379+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/3360831560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3dd77a0 0x55c8e3dd7260 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.379+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/3360831560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3ddd880 0x55c8e3dd1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.379+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/3360831560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddcf60 0x55c8e3ddf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.379+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/3360831560 >> 172.21.15.62:0/3360831560 conn(0x55c8e3d04080 msgr2=0x55c8e3ddf150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:37.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.379+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/3360831560 shutdown_connections 2024-06-10T00:41:37.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.379+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/3360831560 wait complete. 2024-06-10T00:41:37.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.380+0000 7fd1c4fd5b80 1 -- start start 2024-06-10T00:41:37.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.380+0000 7fd1c4fd5b80 1 Processor -- start 2024-06-10T00:41:37.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.380+0000 7fd1c4fd5b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3dd77a0 0x55c8e3de9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.380+0000 7fd1c4fd5b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3ddcf60 0x55c8e3dfc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.380+0000 7fd1c4fd5b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddd880 0x55c8e3dfcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.380+0000 7fd1c4fd5b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41ad7f0 con 0x55c8e3ddcf60 2024-06-10T00:41:37.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.380+0000 7fd1c4fd5b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41ad120 con 0x55c8e3ddd880 2024-06-10T00:41:37.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.380+0000 7fd1c4fd5b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41aa340 con 0x55c8e3dd77a0 2024-06-10T00:41:37.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.380+0000 7fd1c4fd5b80 10 client.0 _ll_register_callbacks cb 0x55c8e3cbe5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:37.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.380+0000 7fd1abfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3ddcf60 0x55c8e3dfc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.380+0000 7fd1abfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3ddcf60 0x55c8e3dfc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46416/0 (socket says 172.21.15.62:46416) 2024-06-10T00:41:37.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.380+0000 7fd1abfff700 1 -- 172.21.15.62:0/484956584 learned_addr learned my addr 172.21.15.62:0/484956584 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:37.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.380+0000 7fd1b8cef700 1 --2- 172.21.15.62:0/484956584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddd880 0x55c8e3dfcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.380+0000 7fd1b3fff700 1 --2- 172.21.15.62:0/484956584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3dd77a0 0x55c8e3de9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.381+0000 7fd1abfff700 1 -- 172.21.15.62:0/484956584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3dd77a0 msgr2=0x55c8e3de9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:37.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.381+0000 7fd1abfff700 1 --2- 172.21.15.62:0/484956584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3dd77a0 0x55c8e3de9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.381+0000 7fd1abfff700 1 -- 172.21.15.62:0/484956584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddd880 msgr2=0x55c8e3dfcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:37.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.381+0000 7fd1abfff700 1 --2- 172.21.15.62:0/484956584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddd880 0x55c8e3dfcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.381+0000 7fd1abfff700 1 -- 172.21.15.62:0/484956584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e3dfd560 con 0x55c8e3ddcf60 2024-06-10T00:41:37.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.381+0000 7fd1abfff700 1 --2- 172.21.15.62:0/484956584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3ddcf60 0x55c8e3dfc850 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd19c09bdd0 tx=0x7fd19c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5bc9c920ca7c667 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:37.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.381+0000 7fd1937fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:37.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.381+0000 7fd1937fe700 1 -- 172.21.15.62:0/484956584 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c09b800 con 0x55c8e3ddcf60 2024-06-10T00:41:37.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.381+0000 7fd1937fe700 1 -- 172.21.15.62:0/484956584 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7fd19c0cf260 con 0x55c8e3ddcf60 2024-06-10T00:41:37.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.381+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/484956584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1a402a030 con 0x55c8e3ddcf60 2024-06-10T00:41:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.381+0000 7fd1937fe700 1 -- 172.21.15.62:0/484956584 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd19c0b4020 con 0x55c8e3ddcf60 2024-06-10T00:41:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.381+0000 7fd1c4fd5b80 10 client.16878 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.381+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/484956584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c8e3e5fed0 con 0x55c8e3ddcf60 2024-06-10T00:41:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.381+0000 7fd1937fe700 1 -- 172.21.15.62:0/484956584 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a402a030 con 0x55c8e3ddcf60 2024-06-10T00:41:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.382+0000 7fd1937fe700 1 -- 172.21.15.62:0/484956584 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fd19c005050 con 0x55c8e3ddcf60 2024-06-10T00:41:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.382+0000 7fd1c4fd5b80 10 client.16878 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.382+0000 7fd1c4fd5b80 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:41:37.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.382+0000 7fd1c4fd5b80 20 client.16878 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.382+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/484956584 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c8e3e605e0 0x55c8e3e62a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.382+0000 7fd1c4fd5b80 4 client.16878 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:41:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.382+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/484956584 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8e3d302d0 con 0x55c8e3e605e0 2024-06-10T00:41:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.382+0000 7fd1b3fff700 1 --2- 172.21.15.62:0/484956584 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c8e3e605e0 0x55c8e3e62a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.382+0000 7fd1b3fff700 1 --2- 172.21.15.62:0/484956584 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c8e3e605e0 0x55c8e3e62a00 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a0e065db5504d8d server_cookie=88eb2cd9ebf0dd01 in_seq=0 out_seq=0 2024-06-10T00:41:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.382+0000 7fd1937fe700 10 client.16878 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1937fe700 1 -- 172.21.15.62:0/484956584 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8e3d302d0 con 0x55c8e3e605e0 2024-06-10T00:41:37.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1937fe700 10 client.16878 handle_command_reply: tid=0 2024-06-10T00:41:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1c4fd5b80 1 client.16878 shutdown 2024-06-10T00:41:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/484956584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3ddcf60 msgr2=0x55c8e3dfc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/484956584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3ddcf60 0x55c8e3dfc850 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd19c09bdd0 tx=0x7fd19c004750 comp rx=0 tx=0).stop 2024-06-10T00:41:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/484956584 shutdown_connections 2024-06-10T00:41:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/484956584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8e3ddd880 0x55c8e3dfcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/484956584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8e3dd77a0 0x55c8e3de9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/484956584 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55c8e3e605e0 0x55c8e3e62a00 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1c4fd5b80 1 --2- 172.21.15.62:0/484956584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c8e3ddcf60 0x55c8e3dfc850 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1937fe700 0 client.16878 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:37.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/484956584 >> 172.21.15.62:0/484956584 conn(0x55c8e3d04080 msgr2=0x55c8e3de2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:37.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/484956584 shutdown_connections 2024-06-10T00:41:37.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1c4fd5b80 1 -- 172.21.15.62:0/484956584 wait complete. 2024-06-10T00:41:37.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.383+0000 7fd1c4fd5b80 20 client.16878 trim_cache size 0 max 16384 2024-06-10T00:41:37.394 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:41:37.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.384+0000 7fd1ba752700 1 -- 172.21.15.62:0/2428912830 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd1a00a2d90 msgr2=0x7fd1a00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:37.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.384+0000 7fd1ba752700 1 --2- 172.21.15.62:0/2428912830 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd1a00a2d90 0x7fd1a00a5250 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4005950 tx=0x7fd1a400a300 comp rx=0 tx=0).stop 2024-06-10T00:41:37.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.384+0000 7fd1ba752700 1 -- 172.21.15.62:0/2428912830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b4102e80 msgr2=0x7fd1b419edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:37.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.384+0000 7fd1ba752700 1 --2- 172.21.15.62:0/2428912830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b4102e80 0x7fd1b419edd0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00c920 tx=0x7fd19c00cc30 comp rx=0 tx=0).stop 2024-06-10T00:41:37.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.384+0000 7fd1ba752700 1 -- 172.21.15.62:0/2428912830 shutdown_connections 2024-06-10T00:41:37.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.384+0000 7fd1ba752700 1 --2- 172.21.15.62:0/2428912830 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd1a00a2d90 0x7fd1a00a5250 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.384+0000 7fd1ba752700 1 --2- 172.21.15.62:0/2428912830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41084f0 0x7fd1b4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.384+0000 7fd1ba752700 1 --2- 172.21.15.62:0/2428912830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1b41024e0 0x7fd1b419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.384+0000 7fd1ba752700 1 --2- 172.21.15.62:0/2428912830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd1b4102e80 0x7fd1b419edd0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:37.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.384+0000 7fd1ba752700 1 -- 172.21.15.62:0/2428912830 >> 172.21.15.62:0/2428912830 conn(0x7fd1b4079530 msgr2=0x7fd1b41005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:37.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.384+0000 7fd1ba752700 1 -- 172.21.15.62:0/2428912830 shutdown_connections 2024-06-10T00:41:37.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:37.384+0000 7fd1ba752700 1 -- 172.21.15.62:0/2428912830 wait complete. 2024-06-10T00:41:37.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:37 smithi105 ceph-mon[36321]: pgmap v270: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:41:37.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:37 smithi138 ceph-mon[36549]: pgmap v270: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:41:39.078 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T00:41:39.079 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T00:41:39.342 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:41:39.465 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:39 smithi062 ceph-mon[32390]: pgmap v271: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:41:39.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:39 smithi105 ceph-mon[36321]: pgmap v271: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:41:39.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:39 smithi138 ceph-mon[36549]: pgmap v271: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:41:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.115+0000 7f364d7d1700 1 -- 172.21.15.62:0/2887612602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36481027a0 msgr2=0x7f3648102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.115+0000 7f364d7d1700 1 --2- 172.21.15.62:0/2887612602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36481027a0 0x7f3648102b80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f363800b0d0 tx=0x7f363800b3e0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.115+0000 7f364d7d1700 1 -- 172.21.15.62:0/2887612602 shutdown_connections 2024-06-10T00:41:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.115+0000 7f364d7d1700 1 --2- 172.21.15.62:0/2887612602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36481027a0 0x7f3648102b80 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.115+0000 7f364d7d1700 1 --2- 172.21.15.62:0/2887612602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3648107870 0x7f364810f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.115+0000 7f364d7d1700 1 --2- 172.21.15.62:0/2887612602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3648103150 0x7f3648107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.115+0000 7f364d7d1700 1 -- 172.21.15.62:0/2887612602 >> 172.21.15.62:0/2887612602 conn(0x7f3648079570 msgr2=0x7f3648079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:41.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.115+0000 7f364d7d1700 1 -- 172.21.15.62:0/2887612602 shutdown_connections 2024-06-10T00:41:41.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.115+0000 7f364d7d1700 1 -- 172.21.15.62:0/2887612602 wait complete. 2024-06-10T00:41:41.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.116+0000 7f364d7d1700 1 Processor -- start 2024-06-10T00:41:41.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.116+0000 7f364d7d1700 1 -- start start 2024-06-10T00:41:41.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.116+0000 7f364d7d1700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36481027a0 0x7f364819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.116+0000 7f364d7d1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3648103150 0x7f364819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.116+0000 7f364d7d1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3648107870 0x7f3648198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.116+0000 7f364d7d1700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3648112800 con 0x7f36481027a0 2024-06-10T00:41:41.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.116+0000 7f364d7d1700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3648112680 con 0x7f3648107870 2024-06-10T00:41:41.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.116+0000 7f364d7d1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3648112980 con 0x7f3648103150 2024-06-10T00:41:41.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.116+0000 7f3646ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36481027a0 0x7f364819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.116+0000 7f3646ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36481027a0 0x7f364819e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32840/0 (socket says 172.21.15.62:32840) 2024-06-10T00:41:41.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.116+0000 7f3646ffd700 1 -- 172.21.15.62:0/374882828 learned_addr learned my addr 172.21.15.62:0/374882828 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:41.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.116+0000 7f36467fc700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3648103150 0x7f364819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.116+0000 7f36477fe700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3648107870 0x7f3648198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.117+0000 7f3646ffd700 1 -- 172.21.15.62:0/374882828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3648103150 msgr2=0x7f364819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:41.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.117+0000 7f3646ffd700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3648103150 0x7f364819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.117+0000 7f3646ffd700 1 -- 172.21.15.62:0/374882828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3648107870 msgr2=0x7f3648198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:41.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.117+0000 7f3646ffd700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3648107870 0x7f3648198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.117+0000 7f3646ffd700 1 -- 172.21.15.62:0/374882828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36481991a0 con 0x7f36481027a0 2024-06-10T00:41:41.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.117+0000 7f36477fe700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3648107870 0x7f3648198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:41:41.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.117+0000 7f3646ffd700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36481027a0 0x7f364819e880 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f363800b820 tx=0x7f3638004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:41.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.117+0000 7f363ffff700 1 -- 172.21.15.62:0/374882828 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638021020 con 0x7f36481027a0 2024-06-10T00:41:41.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.117+0000 7f363ffff700 1 -- 172.21.15.62:0/374882828 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3638007c80 con 0x7f36481027a0 2024-06-10T00:41:41.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.117+0000 7f364d7d1700 1 -- 172.21.15.62:0/374882828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3648199430 con 0x7f36481027a0 2024-06-10T00:41:41.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.117+0000 7f363ffff700 1 -- 172.21.15.62:0/374882828 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638005470 con 0x7f36481027a0 2024-06-10T00:41:41.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.117+0000 7f364d7d1700 1 -- 172.21.15.62:0/374882828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36481a5750 con 0x7f36481027a0 2024-06-10T00:41:41.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.119+0000 7f363ffff700 1 -- 172.21.15.62:0/374882828 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3638024020 con 0x7f36481027a0 2024-06-10T00:41:41.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.120+0000 7f363ffff700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f36340a2e10 0x7f36340a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.120+0000 7f36467fc700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f36340a2e10 0x7f36340a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.120+0000 7f363ffff700 1 -- 172.21.15.62:0/374882828 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f363800f020 con 0x7f36481027a0 2024-06-10T00:41:41.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.120+0000 7f36467fc700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f36340a2e10 0x7f36340a52d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3648108c90 tx=0x7f3630009190 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:41.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.121+0000 7f3658054b80 1 Processor -- start 2024-06-10T00:41:41.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.121+0000 7f3658054b80 1 -- start start 2024-06-10T00:41:41.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.121+0000 7f3658054b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fe13f0 0x556238fe17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.121+0000 7f3658054b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe1d10 0x556238fdb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.121+0000 7f3658054b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fdbc30 0x556238fdc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.121+0000 7f3658054b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556238ec9fe0 con 0x556238fe13f0 2024-06-10T00:41:41.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.121+0000 7f3658054b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556238f288d0 con 0x556238fdbc30 2024-06-10T00:41:41.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.121+0000 7f3658054b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556238e904e0 con 0x556238fe1d10 2024-06-10T00:41:41.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.121+0000 7f3646ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fe13f0 0x556238fe17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.121+0000 7f3646ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fe13f0 0x556238fe17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32850/0 (socket says 172.21.15.62:32850) 2024-06-10T00:41:41.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.121+0000 7f3646ffd700 1 -- 172.21.15.62:0/1583303406 learned_addr learned my addr 172.21.15.62:0/1583303406 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:41.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.121+0000 7f36477fe700 1 --2- 172.21.15.62:0/1583303406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fdbc30 0x556238fdc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.121+0000 7f36467fc700 1 --2- 172.21.15.62:0/1583303406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe1d10 0x556238fdb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.122+0000 7f3646ffd700 1 -- 172.21.15.62:0/1583303406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe1d10 msgr2=0x556238fdb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:41.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.122+0000 7f3646ffd700 1 --2- 172.21.15.62:0/1583303406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe1d10 0x556238fdb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.122+0000 7f3646ffd700 1 -- 172.21.15.62:0/1583303406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fdbc30 msgr2=0x556238fdc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:41.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.122+0000 7f3646ffd700 1 --2- 172.21.15.62:0/1583303406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fdbc30 0x556238fdc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.122+0000 7f3646ffd700 1 -- 172.21.15.62:0/1583303406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556238e77950 con 0x556238fe13f0 2024-06-10T00:41:41.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.122+0000 7f3646ffd700 1 --2- 172.21.15.62:0/1583303406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fe13f0 0x556238fe17d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3638099690 tx=0x7f363800b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc9fe4ee5885b2e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:41.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.122+0000 7f363d7fa700 1 -- 172.21.15.62:0/1583303406 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36380a58a0 con 0x556238fe13f0 2024-06-10T00:41:41.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.122+0000 7f363d7fa700 1 -- 172.21.15.62:0/1583303406 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f363809afc0 con 0x556238fe13f0 2024-06-10T00:41:41.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.122+0000 7f363d7fa700 1 -- 172.21.15.62:0/1583303406 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363809d180 con 0x556238fe13f0 2024-06-10T00:41:41.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.123+0000 7f3658054b80 1 -- 172.21.15.62:0/1583303406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fe13f0 msgr2=0x556238fe17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:41.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.123+0000 7f3658054b80 1 --2- 172.21.15.62:0/1583303406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fe13f0 0x556238fe17d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3638099690 tx=0x7f363800b3e0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.123+0000 7f3658054b80 1 -- 172.21.15.62:0/1583303406 shutdown_connections 2024-06-10T00:41:41.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.123+0000 7f3658054b80 1 --2- 172.21.15.62:0/1583303406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fe13f0 0x556238fe17d0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.123+0000 7f3658054b80 1 --2- 172.21.15.62:0/1583303406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fdbc30 0x556238fdc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.123+0000 7f3658054b80 1 --2- 172.21.15.62:0/1583303406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe1d10 0x556238fdb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.123+0000 7f3658054b80 1 -- 172.21.15.62:0/1583303406 >> 172.21.15.62:0/1583303406 conn(0x556238f04100 msgr2=0x556238fd22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:41.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.123+0000 7f3658054b80 1 -- 172.21.15.62:0/1583303406 shutdown_connections 2024-06-10T00:41:41.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.123+0000 7f3658054b80 1 -- 172.21.15.62:0/1583303406 wait complete. 2024-06-10T00:41:41.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.123+0000 7f3658054b80 1 -- start start 2024-06-10T00:41:41.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.123+0000 7f3658054b80 1 Processor -- start 2024-06-10T00:41:41.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3658054b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fdbc30 0x556238fe7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3658054b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fe13f0 0x556238fe4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3658054b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe1d10 0x556238fe53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3658054b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556238ec9040 con 0x556238fdbc30 2024-06-10T00:41:41.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3658054b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556238f24890 con 0x556238fe13f0 2024-06-10T00:41:41.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3658054b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556238ec9fe0 con 0x556238fe1d10 2024-06-10T00:41:41.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3658054b80 10 client.0 _ll_register_callbacks cb 0x556238ebe630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:41.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3646ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fdbc30 0x556238fe7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3646ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fdbc30 0x556238fe7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32866/0 (socket says 172.21.15.62:32866) 2024-06-10T00:41:41.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3646ffd700 1 -- 172.21.15.62:0/1957356101 learned_addr learned my addr 172.21.15.62:0/1957356101 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:41.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f36467fc700 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fe13f0 0x556238fe4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f36477fe700 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe1d10 0x556238fe53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3646ffd700 1 -- 172.21.15.62:0/1957356101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe1d10 msgr2=0x556238fe53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:41.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3646ffd700 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe1d10 0x556238fe53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3646ffd700 1 -- 172.21.15.62:0/1957356101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fe13f0 msgr2=0x556238fe4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:41.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3646ffd700 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fe13f0 0x556238fe4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3646ffd700 1 -- 172.21.15.62:0/1957356101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556238eee7d0 con 0x556238fdbc30 2024-06-10T00:41:41.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f36477fe700 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe1d10 0x556238fe53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:41.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f36467fc700 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fe13f0 0x556238fe4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:41.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3646ffd700 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fdbc30 0x556238fe7af0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f36380ce3e0 tx=0x7f36380a11e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66c78cefd304545 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:41.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f362e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:41.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f362e7fc700 1 -- 172.21.15.62:0/1957356101 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36380bb030 con 0x556238fdbc30 2024-06-10T00:41:41.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f362e7fc700 1 -- 172.21.15.62:0/1957356101 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f363809d120 con 0x556238fdbc30 2024-06-10T00:41:41.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.124+0000 7f3658054b80 1 -- 172.21.15.62:0/1957356101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f363809afc0 con 0x556238fdbc30 2024-06-10T00:41:41.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.125+0000 7f362e7fc700 1 -- 172.21.15.62:0/1957356101 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f36380a5410 con 0x556238fdbc30 2024-06-10T00:41:41.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.125+0000 7f3658054b80 10 client.16890 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:41.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.125+0000 7f3658054b80 1 -- 172.21.15.62:0/1957356101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556238eee7d0 con 0x556238fdbc30 2024-06-10T00:41:41.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.125+0000 7f362e7fc700 1 -- 172.21.15.62:0/1957356101 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f363809afc0 con 0x556238fdbc30 2024-06-10T00:41:41.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.125+0000 7f362e7fc700 1 -- 172.21.15.62:0/1957356101 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f3638083030 con 0x556238fdbc30 2024-06-10T00:41:41.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.125+0000 7f3658054b80 10 client.16890 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:41.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.125+0000 7f3658054b80 10 client.16890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:41:41.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.125+0000 7f3658054b80 20 client.16890 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:41.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.125+0000 7f3658054b80 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556238ff4e40 0x556238ff7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.125+0000 7f3658054b80 4 client.16890 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:41:41.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.125+0000 7f3658054b80 1 -- 172.21.15.62:0/1957356101 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556238f35dd0 con 0x556238ff4e40 2024-06-10T00:41:41.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.125+0000 7f36467fc700 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556238ff4e40 0x556238ff7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.126+0000 7f36467fc700 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556238ff4e40 0x556238ff7230 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2013157909bd42a server_cookie=5b493483d90a1e64 in_seq=0 out_seq=0 2024-06-10T00:41:41.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.126+0000 7f362e7fc700 10 client.16890 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:41.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f362e7fc700 1 -- 172.21.15.62:0/1957356101 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556238f35dd0 con 0x556238ff4e40 2024-06-10T00:41:41.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f362e7fc700 10 client.16890 handle_command_reply: tid=0 2024-06-10T00:41:41.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f3658054b80 1 client.16890 shutdown 2024-06-10T00:41:41.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f3658054b80 1 -- 172.21.15.62:0/1957356101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fdbc30 msgr2=0x556238fe7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:41.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f3658054b80 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fdbc30 0x556238fe7af0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f36380ce3e0 tx=0x7f36380a11e0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f3658054b80 1 -- 172.21.15.62:0/1957356101 shutdown_connections 2024-06-10T00:41:41.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f3658054b80 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556238ff4e40 0x556238ff7230 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f362e7fc700 0 client.16890 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:41.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f3658054b80 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fdbc30 0x556238fe7af0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f3658054b80 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe1d10 0x556238fe53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f3658054b80 1 --2- 172.21.15.62:0/1957356101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fe13f0 0x556238fe4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f3658054b80 1 -- 172.21.15.62:0/1957356101 >> 172.21.15.62:0/1957356101 conn(0x556238f04100 msgr2=0x556238fd1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:41.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f3658054b80 1 -- 172.21.15.62:0/1957356101 shutdown_connections 2024-06-10T00:41:41.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f3658054b80 1 -- 172.21.15.62:0/1957356101 wait complete. 2024-06-10T00:41:41.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.127+0000 7f3658054b80 20 client.16890 trim_cache size 0 max 16384 2024-06-10T00:41:41.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.148+0000 7f3658054b80 1 Processor -- start 2024-06-10T00:41:41.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.148+0000 7f3658054b80 1 -- start start 2024-06-10T00:41:41.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.148+0000 7f3658054b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fdbc30 0x556238fdf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.148+0000 7f3658054b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe13f0 0x556238fe2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.148+0000 7f3658054b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fe1d10 0x556238fd69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.148+0000 7f3658054b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36481ac530 con 0x556238fe1d10 2024-06-10T00:41:41.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.148+0000 7f3658054b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36481ae070 con 0x556238fdbc30 2024-06-10T00:41:41.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.148+0000 7f3658054b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36481acc00 con 0x556238fe13f0 2024-06-10T00:41:41.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f36477fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fe1d10 0x556238fd69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f36477fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fe1d10 0x556238fd69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32870/0 (socket says 172.21.15.62:32870) 2024-06-10T00:41:41.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f36477fe700 1 -- 172.21.15.62:0/2318644608 learned_addr learned my addr 172.21.15.62:0/2318644608 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:41.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f36467fc700 1 --2- 172.21.15.62:0/2318644608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe13f0 0x556238fe2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f3646ffd700 1 --2- 172.21.15.62:0/2318644608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fdbc30 0x556238fdf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f36477fe700 1 -- 172.21.15.62:0/2318644608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe13f0 msgr2=0x556238fe2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:41.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f36477fe700 1 --2- 172.21.15.62:0/2318644608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe13f0 0x556238fe2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f36477fe700 1 -- 172.21.15.62:0/2318644608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fdbc30 msgr2=0x556238fdf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:41.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f36477fe700 1 --2- 172.21.15.62:0/2318644608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fdbc30 0x556238fdf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f36477fe700 1 -- 172.21.15.62:0/2318644608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556238fd6f20 con 0x556238fe1d10 2024-06-10T00:41:41.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f3646ffd700 1 --2- 172.21.15.62:0/2318644608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fdbc30 0x556238fdf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:41.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f36477fe700 1 --2- 172.21.15.62:0/2318644608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fe1d10 0x556238fd69e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f364000fcb0 tx=0x7f3640011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad5aaf458425b9b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:41.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f363cff9700 1 -- 172.21.15.62:0/2318644608 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364001a440 con 0x556238fe1d10 2024-06-10T00:41:41.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f363cff9700 1 -- 172.21.15.62:0/2318644608 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36400198c0 con 0x556238fe1d10 2024-06-10T00:41:41.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.149+0000 7f363cff9700 1 -- 172.21.15.62:0/2318644608 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364000e9e0 con 0x556238fe1d10 2024-06-10T00:41:41.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.150+0000 7f3658054b80 1 -- 172.21.15.62:0/2318644608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fe1d10 msgr2=0x556238fd69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:41.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.150+0000 7f3658054b80 1 --2- 172.21.15.62:0/2318644608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fe1d10 0x556238fd69e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f364000fcb0 tx=0x7f3640011440 comp rx=0 tx=0).stop 2024-06-10T00:41:41.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.150+0000 7f3658054b80 1 -- 172.21.15.62:0/2318644608 shutdown_connections 2024-06-10T00:41:41.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.150+0000 7f3658054b80 1 --2- 172.21.15.62:0/2318644608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fe1d10 0x556238fd69e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.150+0000 7f3658054b80 1 --2- 172.21.15.62:0/2318644608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fdbc30 0x556238fdf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.150+0000 7f3658054b80 1 --2- 172.21.15.62:0/2318644608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe13f0 0x556238fe2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.150+0000 7f3658054b80 1 -- 172.21.15.62:0/2318644608 >> 172.21.15.62:0/2318644608 conn(0x556238f04100 msgr2=0x556238fd62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:41.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.150+0000 7f3658054b80 1 -- 172.21.15.62:0/2318644608 shutdown_connections 2024-06-10T00:41:41.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.150+0000 7f3658054b80 1 -- 172.21.15.62:0/2318644608 wait complete. 2024-06-10T00:41:41.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.150+0000 7f3658054b80 1 -- start start 2024-06-10T00:41:41.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3658054b80 1 Processor -- start 2024-06-10T00:41:41.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3658054b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fdbc30 0x556238fea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3658054b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe13f0 0x556238feaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3658054b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fe1d10 0x556238feafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3658054b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36481ad9a0 con 0x556238fdbc30 2024-06-10T00:41:41.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3658054b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36481ad2d0 con 0x556238fe1d10 2024-06-10T00:41:41.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3658054b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36481aa510 con 0x556238fe13f0 2024-06-10T00:41:41.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3658054b80 10 client.0 _ll_register_callbacks cb 0x556238ebe630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:41.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3646ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fdbc30 0x556238fea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3646ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fdbc30 0x556238fea550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32880/0 (socket says 172.21.15.62:32880) 2024-06-10T00:41:41.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3646ffd700 1 -- 172.21.15.62:0/1229232264 learned_addr learned my addr 172.21.15.62:0/1229232264 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:41.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f36467fc700 1 --2- 172.21.15.62:0/1229232264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe13f0 0x556238feaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f36477fe700 1 --2- 172.21.15.62:0/1229232264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fe1d10 0x556238feafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3646ffd700 1 -- 172.21.15.62:0/1229232264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe13f0 msgr2=0x556238feaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:41.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3646ffd700 1 --2- 172.21.15.62:0/1229232264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe13f0 0x556238feaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3646ffd700 1 -- 172.21.15.62:0/1229232264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fe1d10 msgr2=0x556238feafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:41.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3646ffd700 1 --2- 172.21.15.62:0/1229232264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fe1d10 0x556238feafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.151+0000 7f3646ffd700 1 -- 172.21.15.62:0/1229232264 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55623905f980 con 0x556238fdbc30 2024-06-10T00:41:41.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.152+0000 7f3646ffd700 1 --2- 172.21.15.62:0/1229232264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fdbc30 0x556238fea550 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f3638009fd0 tx=0x7f363809cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578e23a18b535197 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:41.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.152+0000 7f362e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:41.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.155+0000 7f362e7fc700 1 -- 172.21.15.62:0/1229232264 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36380bb040 con 0x556238fdbc30 2024-06-10T00:41:41.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.155+0000 7f362e7fc700 1 -- 172.21.15.62:0/1229232264 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f363809d120 con 0x556238fdbc30 2024-06-10T00:41:41.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.155+0000 7f3658054b80 1 -- 172.21.15.62:0/1229232264 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36400198c0 con 0x556238fdbc30 2024-06-10T00:41:41.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.155+0000 7f362e7fc700 1 -- 172.21.15.62:0/1229232264 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f36380a5c10 con 0x556238fdbc30 2024-06-10T00:41:41.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.155+0000 7f3658054b80 10 client.16896 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:41.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.155+0000 7f3658054b80 1 -- 172.21.15.62:0/1229232264 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55623905f980 con 0x556238fdbc30 2024-06-10T00:41:41.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.155+0000 7f362e7fc700 1 -- 172.21.15.62:0/1229232264 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36400198c0 con 0x556238fdbc30 2024-06-10T00:41:41.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.156+0000 7f362e7fc700 1 -- 172.21.15.62:0/1229232264 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f3638083030 con 0x556238fdbc30 2024-06-10T00:41:41.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.156+0000 7f3658054b80 10 client.16896 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:41.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.156+0000 7f3658054b80 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:41:41.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.156+0000 7f3658054b80 20 client.16896 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:41.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.156+0000 7f3658054b80 1 --2- 172.21.15.62:0/1229232264 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556238ff4e40 0x556239062e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:41.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.156+0000 7f3658054b80 4 client.16896 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T00:41:41.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.156+0000 7f3658054b80 1 -- 172.21.15.62:0/1229232264 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556238f30350 con 0x556238ff4e40 2024-06-10T00:41:41.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.156+0000 7f36467fc700 1 --2- 172.21.15.62:0/1229232264 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556238ff4e40 0x556239062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:41.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.156+0000 7f36467fc700 1 --2- 172.21.15.62:0/1229232264 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556238ff4e40 0x556239062e10 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0b931ae7a2b898 server_cookie=a11b72e41dbc945a in_seq=0 out_seq=0 2024-06-10T00:41:41.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.156+0000 7f362e7fc700 10 client.16896 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:41.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.157+0000 7f362e7fc700 1 -- 172.21.15.62:0/1229232264 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556238f30350 con 0x556238ff4e40 2024-06-10T00:41:41.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.157+0000 7f362e7fc700 10 client.16896 handle_command_reply: tid=0 2024-06-10T00:41:41.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.157+0000 7f3658054b80 1 client.16896 shutdown 2024-06-10T00:41:41.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.158+0000 7f3658054b80 1 -- 172.21.15.62:0/1229232264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fdbc30 msgr2=0x556238fea550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:41.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.158+0000 7f3658054b80 1 --2- 172.21.15.62:0/1229232264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fdbc30 0x556238fea550 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f3638009fd0 tx=0x7f363809cbc0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.158+0000 7f3658054b80 1 -- 172.21.15.62:0/1229232264 shutdown_connections 2024-06-10T00:41:41.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.158+0000 7f3658054b80 1 --2- 172.21.15.62:0/1229232264 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x556238ff4e40 0x556239062e10 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.158+0000 7f3658054b80 1 --2- 172.21.15.62:0/1229232264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556238fdbc30 0x556238fea550 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.158+0000 7f3658054b80 1 --2- 172.21.15.62:0/1229232264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556238fe1d10 0x556238feafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.158+0000 7f3658054b80 1 --2- 172.21.15.62:0/1229232264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556238fe13f0 0x556238feaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.158+0000 7f3658054b80 1 -- 172.21.15.62:0/1229232264 >> 172.21.15.62:0/1229232264 conn(0x556238f04100 msgr2=0x556238fdff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:41.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.158+0000 7f362e7fc700 0 client.16896 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:41.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.158+0000 7f3658054b80 1 -- 172.21.15.62:0/1229232264 shutdown_connections 2024-06-10T00:41:41.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.158+0000 7f3658054b80 1 -- 172.21.15.62:0/1229232264 wait complete. 2024-06-10T00:41:41.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.158+0000 7f3658054b80 20 client.16896 trim_cache size 0 max 16384 2024-06-10T00:41:41.164 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:41:41.164 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-06-10T00:41:41.164 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "60cc3674-2ce3-4484-93a7-14c58946045a", 2024-06-10T00:41:41.164 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-06-10T00:41:41.164 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:41:41.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.159+0000 7f364d7d1700 1 -- 172.21.15.62:0/374882828 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f36340a2e10 msgr2=0x7f36340a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:41.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.159+0000 7f364d7d1700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f36340a2e10 0x7f36340a52d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3648108c90 tx=0x7f3630009190 comp rx=0 tx=0).stop 2024-06-10T00:41:41.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.159+0000 7f364d7d1700 1 -- 172.21.15.62:0/374882828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36481027a0 msgr2=0x7f364819e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:41.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.159+0000 7f364d7d1700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36481027a0 0x7f364819e880 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f363800b820 tx=0x7f3638004e20 comp rx=0 tx=0).stop 2024-06-10T00:41:41.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.159+0000 7f364d7d1700 1 -- 172.21.15.62:0/374882828 shutdown_connections 2024-06-10T00:41:41.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.159+0000 7f364d7d1700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f36340a2e10 0x7f36340a52d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.159+0000 7f364d7d1700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36481027a0 0x7f364819e880 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.159+0000 7f364d7d1700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3648107870 0x7f3648198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.159+0000 7f364d7d1700 1 --2- 172.21.15.62:0/374882828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3648103150 0x7f364819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:41.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.159+0000 7f364d7d1700 1 -- 172.21.15.62:0/374882828 >> 172.21.15.62:0/374882828 conn(0x7f3648079570 msgr2=0x7f3648101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:41.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.159+0000 7f364d7d1700 1 -- 172.21.15.62:0/374882828 shutdown_connections 2024-06-10T00:41:41.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:41.159+0000 7f364d7d1700 1 -- 172.21.15.62:0/374882828 wait complete. 2024-06-10T00:41:41.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:41 smithi062 ceph-mon[32390]: pgmap v272: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:41:41.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:41 smithi105 ceph-mon[36321]: pgmap v272: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:41:41.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:41 smithi138 ceph-mon[36549]: pgmap v272: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:41:41.829 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:41:42.078 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:41:42.437 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:42 smithi062 ceph-mon[32390]: scrub queued for path: / 2024-06-10T00:41:42.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:42 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:42 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:41:42.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:42 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:41:42.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:42 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:41:42.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:42 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:41:42.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:42 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:41:42.438 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:42 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.439 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:42 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.439 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:42 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.439 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:42 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.439 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:42 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:42 smithi105 ceph-mon[36321]: scrub queued for path: / 2024-06-10T00:41:42.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:42 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:42 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:41:42.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:42 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:41:42.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:42 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:41:42.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:42 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:41:42.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:42 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:41:42.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:42 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:42 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:42 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:42 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.534 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:42 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:42 smithi138 ceph-mon[36549]: scrub queued for path: / 2024-06-10T00:41:42.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:42 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:42 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:41:42.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:42 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:41:42.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:42 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:41:42.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:42 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:41:42.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:42 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:41:42.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:42 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:42 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:42 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:42 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:42.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:42 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:41:43.424 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:43 smithi062 ceph-mon[32390]: pgmap v273: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:41:43.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:43 smithi105 ceph-mon[36321]: pgmap v273: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:41:43.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:43 smithi138 ceph-mon[36549]: pgmap v273: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:41:43.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.806+0000 7f76303f6700 1 -- 172.21.15.62:0/3876188871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7628107840 msgr2=0x7f762810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:43.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.806+0000 7f76303f6700 1 --2- 172.21.15.62:0/3876188871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7628107840 0x7f762810f3c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f762000b0d0 tx=0x7f762000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.806+0000 7f76303f6700 1 -- 172.21.15.62:0/3876188871 shutdown_connections 2024-06-10T00:41:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.806+0000 7f76303f6700 1 --2- 172.21.15.62:0/3876188871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7628107840 0x7f762810f3c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.806+0000 7f76303f6700 1 --2- 172.21.15.62:0/3876188871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7628102740 0x7f7628102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.806+0000 7f76303f6700 1 --2- 172.21.15.62:0/3876188871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76281030f0 0x7f7628107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.806+0000 7f76303f6700 1 -- 172.21.15.62:0/3876188871 >> 172.21.15.62:0/3876188871 conn(0x7f76280795f0 msgr2=0x7f7628079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.806+0000 7f76303f6700 1 -- 172.21.15.62:0/3876188871 shutdown_connections 2024-06-10T00:41:43.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.806+0000 7f76303f6700 1 -- 172.21.15.62:0/3876188871 wait complete. 2024-06-10T00:41:43.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.807+0000 7f76303f6700 1 Processor -- start 2024-06-10T00:41:43.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.807+0000 7f76303f6700 1 -- start start 2024-06-10T00:41:43.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.807+0000 7f76303f6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7628102740 0x7f762819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.807+0000 7f76303f6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76281030f0 0x7f762819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.807+0000 7f76303f6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7628107840 0x7f76281989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.807+0000 7f76303f6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76281127d0 con 0x7f7628102740 2024-06-10T00:41:43.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.807+0000 7f76303f6700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7628112650 con 0x7f76281030f0 2024-06-10T00:41:43.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.807+0000 7f76303f6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7628112950 con 0x7f7628107840 2024-06-10T00:41:43.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.807+0000 7f762e192700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7628102740 0x7f762819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f762e192700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7628102740 0x7f762819e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32902/0 (socket says 172.21.15.62:32902) 2024-06-10T00:41:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f762e192700 1 -- 172.21.15.62:0/1060730484 learned_addr learned my addr 172.21.15.62:0/1060730484 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f762e993700 1 --2- 172.21.15.62:0/1060730484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7628107840 0x7f76281989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f762d991700 1 --2- 172.21.15.62:0/1060730484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76281030f0 0x7f762819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f762e192700 1 -- 172.21.15.62:0/1060730484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7628107840 msgr2=0x7f76281989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f762e192700 1 --2- 172.21.15.62:0/1060730484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7628107840 0x7f76281989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f762e192700 1 -- 172.21.15.62:0/1060730484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76281030f0 msgr2=0x7f762819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:43.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f762e192700 1 --2- 172.21.15.62:0/1060730484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76281030f0 0x7f762819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f762e192700 1 -- 172.21.15.62:0/1060730484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7628199130 con 0x7f7628102740 2024-06-10T00:41:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f762e192700 1 --2- 172.21.15.62:0/1060730484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7628102740 0x7f762819e810 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f762400e9a0 tx=0x7f762400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f761f7fe700 1 -- 172.21.15.62:0/1060730484 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762400cbd0 con 0x7f7628102740 2024-06-10T00:41:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f76303f6700 1 -- 172.21.15.62:0/1060730484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7628199420 con 0x7f7628102740 2024-06-10T00:41:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f761f7fe700 1 -- 172.21.15.62:0/1060730484 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f762400cd70 con 0x7f7628102740 2024-06-10T00:41:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f761f7fe700 1 -- 172.21.15.62:0/1060730484 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624018750 con 0x7f7628102740 2024-06-10T00:41:43.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.808+0000 7f76303f6700 1 -- 172.21.15.62:0/1060730484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76281a56e0 con 0x7f7628102740 2024-06-10T00:41:43.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.811+0000 7f761f7fe700 1 -- 172.21.15.62:0/1060730484 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7624010430 con 0x7f7628102740 2024-06-10T00:41:43.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.811+0000 7f761f7fe700 1 --2- 172.21.15.62:0/1060730484 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f76140a2d90 0x7f76140a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.812+0000 7f762d991700 1 --2- 172.21.15.62:0/1060730484 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f76140a2d90 0x7f76140a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.812+0000 7f761f7fe700 1 -- 172.21.15.62:0/1060730484 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f76240cfb50 con 0x7f7628102740 2024-06-10T00:41:43.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.812+0000 7f763ac79b80 1 Processor -- start 2024-06-10T00:41:43.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.812+0000 7f762d991700 1 --2- 172.21.15.62:0/1060730484 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f76140a2d90 0x7f76140a5250 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7618005950 tx=0x7f76180058e0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:43.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.812+0000 7f763ac79b80 1 -- start start 2024-06-10T00:41:43.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.812+0000 7f763ac79b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7829a70 0x55dcd7829e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.812+0000 7f763ac79b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd782a390 0x55dcd7823df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.812+0000 7f763ac79b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7824330 0x55dcd7824750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.812+0000 7f763ac79b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55dcd7771810 con 0x55dcd7829a70 2024-06-10T00:41:43.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.812+0000 7f763ac79b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dcd7719720 con 0x55dcd7824330 2024-06-10T00:41:43.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.812+0000 7f763ac79b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dcd776d7d0 con 0x55dcd782a390 2024-06-10T00:41:43.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.812+0000 7f762e192700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7829a70 0x55dcd7829e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.813+0000 7f762e192700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7829a70 0x55dcd7829e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32906/0 (socket says 172.21.15.62:32906) 2024-06-10T00:41:43.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.813+0000 7f762e192700 1 -- 172.21.15.62:0/1367474685 learned_addr learned my addr 172.21.15.62:0/1367474685 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:43.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.813+0000 7f762d991700 1 --2- 172.21.15.62:0/1367474685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd782a390 0x55dcd7823df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.813+0000 7f762e993700 1 --2- 172.21.15.62:0/1367474685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7824330 0x55dcd7824750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.813+0000 7f762e192700 1 -- 172.21.15.62:0/1367474685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd782a390 msgr2=0x55dcd7823df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:43.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.813+0000 7f762e192700 1 --2- 172.21.15.62:0/1367474685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd782a390 0x55dcd7823df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.813+0000 7f762e192700 1 -- 172.21.15.62:0/1367474685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7824330 msgr2=0x55dcd7824750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:43.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.813+0000 7f762e192700 1 --2- 172.21.15.62:0/1367474685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7824330 0x55dcd7824750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.813+0000 7f762e192700 1 -- 172.21.15.62:0/1367474685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcd770ff50 con 0x55dcd7829a70 2024-06-10T00:41:43.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.813+0000 7f762d991700 1 --2- 172.21.15.62:0/1367474685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd782a390 0x55dcd7823df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:41:43.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.813+0000 7f762e192700 1 --2- 172.21.15.62:0/1367474685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7829a70 0x55dcd7829e50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7624098630 tx=0x7f762409c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d355f715144c9456 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:43.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.813+0000 7f761cd70700 1 -- 172.21.15.62:0/1367474685 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76240a48d0 con 0x55dcd7829a70 2024-06-10T00:41:43.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.813+0000 7f761cd70700 1 -- 172.21.15.62:0/1367474685 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76240b1070 con 0x55dcd7829a70 2024-06-10T00:41:43.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.813+0000 7f761cd70700 1 -- 172.21.15.62:0/1367474685 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76240a4440 con 0x55dcd7829a70 2024-06-10T00:41:43.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.814+0000 7f763ac79b80 1 -- 172.21.15.62:0/1367474685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7829a70 msgr2=0x55dcd7829e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:43.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.814+0000 7f763ac79b80 1 --2- 172.21.15.62:0/1367474685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7829a70 0x55dcd7829e50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7624098630 tx=0x7f762409c360 comp rx=0 tx=0).stop 2024-06-10T00:41:43.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.814+0000 7f763ac79b80 1 -- 172.21.15.62:0/1367474685 shutdown_connections 2024-06-10T00:41:43.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.814+0000 7f763ac79b80 1 --2- 172.21.15.62:0/1367474685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7829a70 0x55dcd7829e50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.814+0000 7f763ac79b80 1 --2- 172.21.15.62:0/1367474685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7824330 0x55dcd7824750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.814+0000 7f763ac79b80 1 --2- 172.21.15.62:0/1367474685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd782a390 0x55dcd7823df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.814+0000 7f763ac79b80 1 -- 172.21.15.62:0/1367474685 >> 172.21.15.62:0/1367474685 conn(0x55dcd774acc0 msgr2=0x55dcd781a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:43.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.814+0000 7f763ac79b80 1 -- 172.21.15.62:0/1367474685 shutdown_connections 2024-06-10T00:41:43.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.814+0000 7f763ac79b80 1 -- 172.21.15.62:0/1367474685 wait complete. 2024-06-10T00:41:43.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.814+0000 7f763ac79b80 1 -- start start 2024-06-10T00:41:43.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f763ac79b80 1 Processor -- start 2024-06-10T00:41:43.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f763ac79b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7824330 0x55dcd78301b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f763ac79b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7829a70 0x55dcd782d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f763ac79b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd782a390 0x55dcd782d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f763ac79b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55dcd7830dd0 con 0x55dcd7824330 2024-06-10T00:41:43.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f763ac79b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dcd7711bf0 con 0x55dcd7829a70 2024-06-10T00:41:43.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f763ac79b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dcd7830f50 con 0x55dcd782a390 2024-06-10T00:41:43.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f763ac79b80 10 client.0 _ll_register_callbacks cb 0x55dcd770b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:43.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f762e192700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7824330 0x55dcd78301b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f762e192700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7824330 0x55dcd78301b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32908/0 (socket says 172.21.15.62:32908) 2024-06-10T00:41:43.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f762e192700 1 -- 172.21.15.62:0/3096168762 learned_addr learned my addr 172.21.15.62:0/3096168762 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:43.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f762e993700 1 --2- 172.21.15.62:0/3096168762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd782a390 0x55dcd782d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f762d991700 1 --2- 172.21.15.62:0/3096168762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7829a70 0x55dcd782d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f762e192700 1 -- 172.21.15.62:0/3096168762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd782a390 msgr2=0x55dcd782d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:43.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f762e192700 1 --2- 172.21.15.62:0/3096168762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd782a390 0x55dcd782d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f762e192700 1 -- 172.21.15.62:0/3096168762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7829a70 msgr2=0x55dcd782d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:43.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f762e192700 1 --2- 172.21.15.62:0/3096168762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7829a70 0x55dcd782d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f762e192700 1 -- 172.21.15.62:0/3096168762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcd76c08d0 con 0x55dcd7824330 2024-06-10T00:41:43.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f762e192700 1 --2- 172.21.15.62:0/3096168762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7824330 0x55dcd78301b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f762409bbf0 tx=0x7f762409c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e873c55672ee5e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:43.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.815+0000 7f7611ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:43.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.816+0000 7f7611ffb700 1 -- 172.21.15.62:0/3096168762 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76240ba070 con 0x55dcd7824330 2024-06-10T00:41:43.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.816+0000 7f7611ffb700 1 -- 172.21.15.62:0/3096168762 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f76240a4440 con 0x55dcd7824330 2024-06-10T00:41:43.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.816+0000 7f763ac79b80 1 -- 172.21.15.62:0/3096168762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76240b1070 con 0x55dcd7824330 2024-06-10T00:41:43.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.816+0000 7f7611ffb700 1 -- 172.21.15.62:0/3096168762 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f76240b4e60 con 0x55dcd7824330 2024-06-10T00:41:43.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.816+0000 7f763ac79b80 10 client.16908 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:43.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.816+0000 7f763ac79b80 1 -- 172.21.15.62:0/3096168762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55dcd783b790 con 0x55dcd7824330 2024-06-10T00:41:43.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.816+0000 7f7611ffb700 1 -- 172.21.15.62:0/3096168762 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76240b1070 con 0x55dcd7824330 2024-06-10T00:41:43.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.816+0000 7f7611ffb700 1 -- 172.21.15.62:0/3096168762 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f762409ac40 con 0x55dcd7824330 2024-06-10T00:41:43.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.816+0000 7f763ac79b80 10 client.16908 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:43.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.816+0000 7f763ac79b80 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:41:43.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.816+0000 7f763ac79b80 20 client.16908 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:43.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.816+0000 7f763ac79b80 1 --2- 172.21.15.62:0/3096168762 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dcd783dc60 0x55dcd7840050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.816+0000 7f763ac79b80 4 client.16908 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:41:43.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.816+0000 7f763ac79b80 1 -- 172.21.15.62:0/3096168762 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcd777ed10 con 0x55dcd783dc60 2024-06-10T00:41:43.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.817+0000 7f762d991700 1 --2- 172.21.15.62:0/3096168762 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dcd783dc60 0x55dcd7840050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.817+0000 7f762d991700 1 --2- 172.21.15.62:0/3096168762 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dcd783dc60 0x55dcd7840050 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cd7113c43ff0b5a server_cookie=1f0c0ba2236fe5f9 in_seq=0 out_seq=0 2024-06-10T00:41:43.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.817+0000 7f7611ffb700 10 client.16908 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:43.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f7611ffb700 1 -- 172.21.15.62:0/3096168762 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dcd777ed10 con 0x55dcd783dc60 2024-06-10T00:41:43.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f7611ffb700 10 client.16908 handle_command_reply: tid=0 2024-06-10T00:41:43.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f763ac79b80 1 client.16908 shutdown 2024-06-10T00:41:43.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f763ac79b80 1 -- 172.21.15.62:0/3096168762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7824330 msgr2=0x55dcd78301b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:43.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f763ac79b80 1 --2- 172.21.15.62:0/3096168762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7824330 0x55dcd78301b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f762409bbf0 tx=0x7f762409c250 comp rx=0 tx=0).stop 2024-06-10T00:41:43.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f763ac79b80 1 -- 172.21.15.62:0/3096168762 shutdown_connections 2024-06-10T00:41:43.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f763ac79b80 1 --2- 172.21.15.62:0/3096168762 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dcd783dc60 0x55dcd7840050 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f763ac79b80 1 --2- 172.21.15.62:0/3096168762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7824330 0x55dcd78301b0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f763ac79b80 1 --2- 172.21.15.62:0/3096168762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd782a390 0x55dcd782d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f763ac79b80 1 --2- 172.21.15.62:0/3096168762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7829a70 0x55dcd782d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f763ac79b80 1 -- 172.21.15.62:0/3096168762 >> 172.21.15.62:0/3096168762 conn(0x55dcd774acc0 msgr2=0x55dcd781e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:43.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f7611ffb700 0 client.16908 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:43.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f763ac79b80 1 -- 172.21.15.62:0/3096168762 shutdown_connections 2024-06-10T00:41:43.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f763ac79b80 1 -- 172.21.15.62:0/3096168762 wait complete. 2024-06-10T00:41:43.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.818+0000 7f763ac79b80 20 client.16908 trim_cache size 0 max 16384 2024-06-10T00:41:43.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.839+0000 7f763ac79b80 1 Processor -- start 2024-06-10T00:41:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.839+0000 7f763ac79b80 1 -- start start 2024-06-10T00:41:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.839+0000 7f763ac79b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd774acc0 0x55dcd782c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.839+0000 7f763ac79b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7824330 0x55dcd7814690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.839+0000 7f763ac79b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd7829a70 0x55dcd781f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.839+0000 7f763ac79b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76281ac5f0 con 0x55dcd7824330 2024-06-10T00:41:43.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.839+0000 7f763ac79b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76281ae130 con 0x55dcd774acc0 2024-06-10T00:41:43.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.839+0000 7f763ac79b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76281accc0 con 0x55dcd7829a70 2024-06-10T00:41:43.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.839+0000 7f762d991700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7824330 0x55dcd7814690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.840+0000 7f762d991700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7824330 0x55dcd7814690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32912/0 (socket says 172.21.15.62:32912) 2024-06-10T00:41:43.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.840+0000 7f762d991700 1 -- 172.21.15.62:0/1962678269 learned_addr learned my addr 172.21.15.62:0/1962678269 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:43.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.840+0000 7f762e993700 1 --2- 172.21.15.62:0/1962678269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd7829a70 0x55dcd781f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.840+0000 7f762d991700 1 -- 172.21.15.62:0/1962678269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd7829a70 msgr2=0x55dcd781f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:43.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.840+0000 7f762d991700 1 --2- 172.21.15.62:0/1962678269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd7829a70 0x55dcd781f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.840+0000 7f762e192700 1 --2- 172.21.15.62:0/1962678269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd774acc0 0x55dcd782c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.840+0000 7f762d991700 1 -- 172.21.15.62:0/1962678269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd774acc0 msgr2=0x55dcd782c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:43.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.840+0000 7f762d991700 1 --2- 172.21.15.62:0/1962678269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd774acc0 0x55dcd782c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.840+0000 7f762d991700 1 -- 172.21.15.62:0/1962678269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcd781a740 con 0x55dcd7824330 2024-06-10T00:41:43.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.840+0000 7f762e192700 1 --2- 172.21.15.62:0/1962678269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd774acc0 0x55dcd782c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:41:43.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.840+0000 7f762d991700 1 --2- 172.21.15.62:0/1962678269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7824330 0x55dcd7814690 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7628108d00 tx=0x7f7618015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc798669eb1e4b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:43.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.840+0000 7f7613fff700 1 -- 172.21.15.62:0/1962678269 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618021b70 con 0x55dcd7824330 2024-06-10T00:41:43.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.840+0000 7f7613fff700 1 -- 172.21.15.62:0/1962678269 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7618021d10 con 0x55dcd7824330 2024-06-10T00:41:43.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.840+0000 7f7613fff700 1 -- 172.21.15.62:0/1962678269 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618027250 con 0x55dcd7824330 2024-06-10T00:41:43.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 -- 172.21.15.62:0/1962678269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7824330 msgr2=0x55dcd7814690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:43.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 --2- 172.21.15.62:0/1962678269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7824330 0x55dcd7814690 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7628108d00 tx=0x7f7618015000 comp rx=0 tx=0).stop 2024-06-10T00:41:43.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 -- 172.21.15.62:0/1962678269 shutdown_connections 2024-06-10T00:41:43.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 --2- 172.21.15.62:0/1962678269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd7829a70 0x55dcd781f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 --2- 172.21.15.62:0/1962678269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd774acc0 0x55dcd782c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 --2- 172.21.15.62:0/1962678269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7824330 0x55dcd7814690 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 -- 172.21.15.62:0/1962678269 >> 172.21.15.62:0/1962678269 conn(0x55dcd7824a70 msgr2=0x55dcd781e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:43.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 -- 172.21.15.62:0/1962678269 shutdown_connections 2024-06-10T00:41:43.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 -- 172.21.15.62:0/1962678269 wait complete. 2024-06-10T00:41:43.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 -- start start 2024-06-10T00:41:43.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 Processor -- start 2024-06-10T00:41:43.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd774acc0 0x55dcd7843380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7824330 0x55dcd7840770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7829a70 0x55dcd7840cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76281ada60 con 0x55dcd7829a70 2024-06-10T00:41:43.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76281ad390 con 0x55dcd7824330 2024-06-10T00:41:43.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.841+0000 7f763ac79b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76281aa5b0 con 0x55dcd774acc0 2024-06-10T00:41:43.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f763ac79b80 10 client.0 _ll_register_callbacks cb 0x55dcd770b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:41:43.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f762e993700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7829a70 0x55dcd7840cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f762e993700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7829a70 0x55dcd7840cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32920/0 (socket says 172.21.15.62:32920) 2024-06-10T00:41:43.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f762e993700 1 -- 172.21.15.62:0/503500284 learned_addr learned my addr 172.21.15.62:0/503500284 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:41:43.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f762e192700 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd774acc0 0x55dcd7843380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f762d991700 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7824330 0x55dcd7840770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f762e993700 1 -- 172.21.15.62:0/503500284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd774acc0 msgr2=0x55dcd7843380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:43.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f762e993700 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd774acc0 0x55dcd7843380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f762e993700 1 -- 172.21.15.62:0/503500284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7824330 msgr2=0x55dcd7840770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:41:43.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f762e993700 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7824330 0x55dcd7840770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f762e993700 1 -- 172.21.15.62:0/503500284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcd78414a0 con 0x55dcd7829a70 2024-06-10T00:41:43.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f762e192700 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd774acc0 0x55dcd7843380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:41:43.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f762d991700 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7824330 0x55dcd7840770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:41:43.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f762e993700 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7829a70 0x55dcd7840cb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7620015000 tx=0x7f762000bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fba8e5d22c761ba server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:41:43.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f7611ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:41:43.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f7611ffb700 1 -- 172.21.15.62:0/503500284 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7620003cd0 con 0x55dcd7829a70 2024-06-10T00:41:43.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f763ac79b80 1 -- 172.21.15.62:0/503500284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7618021d10 con 0x55dcd7829a70 2024-06-10T00:41:43.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.842+0000 7f7611ffb700 1 -- 172.21.15.62:0/503500284 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9554+0+0 (secure 0 0 0) 0x7f7620005070 con 0x55dcd7829a70 2024-06-10T00:41:43.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.843+0000 7f7611ffb700 1 -- 172.21.15.62:0/503500284 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f76200052b0 con 0x55dcd7829a70 2024-06-10T00:41:43.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.843+0000 7f763ac79b80 10 client.16914 fetch_fsmap learned FSMap version 38 2024-06-10T00:41:43.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.843+0000 7f763ac79b80 1 -- 172.21.15.62:0/503500284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55dcd78a8540 con 0x55dcd7829a70 2024-06-10T00:41:43.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.843+0000 7f7611ffb700 1 -- 172.21.15.62:0/503500284 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7618021d10 con 0x55dcd7829a70 2024-06-10T00:41:43.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.843+0000 7f7611ffb700 1 -- 172.21.15.62:0/503500284 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f7620011070 con 0x55dcd7829a70 2024-06-10T00:41:43.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.843+0000 7f763ac79b80 10 client.16914 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:41:43.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.843+0000 7f763ac79b80 10 client.16914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:41:43.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.843+0000 7f763ac79b80 20 client.16914 populate_metadata read hostname 'smithi062' 2024-06-10T00:41:43.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.843+0000 7f763ac79b80 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dcd782a390 0x55dcd78aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:41:43.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.843+0000 7f763ac79b80 4 client.16914 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:41:43.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.843+0000 7f763ac79b80 1 -- 172.21.15.62:0/503500284 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dcd7779290 con 0x55dcd782a390 2024-06-10T00:41:43.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.843+0000 7f762e192700 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dcd782a390 0x55dcd78aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:41:43.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f762e192700 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dcd782a390 0x55dcd78aac40 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9671d1eabb5454cb server_cookie=b201f70575fbfbdf in_seq=0 out_seq=0 2024-06-10T00:41:43.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f7611ffb700 10 client.16914 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f7611ffb700 1 -- 172.21.15.62:0/503500284 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dcd7779290 con 0x55dcd782a390 2024-06-10T00:41:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f7611ffb700 10 client.16914 handle_command_reply: tid=0 2024-06-10T00:41:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f763ac79b80 1 client.16914 shutdown 2024-06-10T00:41:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f763ac79b80 1 -- 172.21.15.62:0/503500284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7829a70 msgr2=0x55dcd7840cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f763ac79b80 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7829a70 0x55dcd7840cb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7620015000 tx=0x7f762000bec0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f763ac79b80 1 -- 172.21.15.62:0/503500284 shutdown_connections 2024-06-10T00:41:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f763ac79b80 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55dcd782a390 0x55dcd78aac40 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f763ac79b80 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55dcd7829a70 0x55dcd7840cb0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f763ac79b80 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcd774acc0 0x55dcd7843380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f7611ffb700 0 client.16914 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:41:43.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f763ac79b80 1 --2- 172.21.15.62:0/503500284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dcd7824330 0x55dcd7840770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f763ac79b80 1 -- 172.21.15.62:0/503500284 >> 172.21.15.62:0/503500284 conn(0x55dcd7824a70 msgr2=0x55dcd781f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:43.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f763ac79b80 1 -- 172.21.15.62:0/503500284 shutdown_connections 2024-06-10T00:41:43.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f763ac79b80 1 -- 172.21.15.62:0/503500284 wait complete. 2024-06-10T00:41:43.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.844+0000 7f763ac79b80 20 client.16914 trim_cache size 0 max 16384 2024-06-10T00:41:43.861 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:41:43.861 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T00:41:43.861 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-06-10T00:41:43.861 INFO:teuthology.orchestra.run.smithi062.stdout: "60cc3674-2ce3-4484-93a7-14c58946045a": { 2024-06-10T00:41:43.861 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-06-10T00:41:43.861 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "60cc3674-2ce3-4484-93a7-14c58946045a", 2024-06-10T00:41:43.861 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-06-10T00:41:43.862 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:41:43.862 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:41:43.862 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:41:43.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.845+0000 7f76303f6700 1 -- 172.21.15.62:0/1060730484 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f76140a2d90 msgr2=0x7f76140a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:43.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.845+0000 7f76303f6700 1 --2- 172.21.15.62:0/1060730484 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f76140a2d90 0x7f76140a5250 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7618005950 tx=0x7f76180058e0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.845+0000 7f76303f6700 1 -- 172.21.15.62:0/1060730484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7628102740 msgr2=0x7f762819e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:41:43.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.845+0000 7f76303f6700 1 --2- 172.21.15.62:0/1060730484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7628102740 0x7f762819e810 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f762400e9a0 tx=0x7f762400ecb0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.845+0000 7f76303f6700 1 -- 172.21.15.62:0/1060730484 shutdown_connections 2024-06-10T00:41:43.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.845+0000 7f76303f6700 1 --2- 172.21.15.62:0/1060730484 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f76140a2d90 0x7f76140a5250 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.845+0000 7f76303f6700 1 --2- 172.21.15.62:0/1060730484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7628102740 0x7f762819e810 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.845+0000 7f76303f6700 1 --2- 172.21.15.62:0/1060730484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7628107840 0x7f76281989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.845+0000 7f76303f6700 1 --2- 172.21.15.62:0/1060730484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76281030f0 0x7f762819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:41:43.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.845+0000 7f76303f6700 1 -- 172.21.15.62:0/1060730484 >> 172.21.15.62:0/1060730484 conn(0x7f76280795f0 msgr2=0x7f7628100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:41:43.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.845+0000 7f76303f6700 1 -- 172.21.15.62:0/1060730484 shutdown_connections 2024-06-10T00:41:43.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:41:43.845+0000 7f76303f6700 1 -- 172.21.15.62:0/1060730484 wait complete. 2024-06-10T00:41:44.559 INFO:tasks.cephfs.filesystem:scrub status for tag:60cc3674-2ce3-4484-93a7-14c58946045a - {'path': '/', 'tag': '60cc3674-2ce3-4484-93a7-14c58946045a', 'options': 'recursive,force'} 2024-06-10T00:41:45.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:45 smithi105 ceph-mon[36321]: pgmap v274: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:41:45.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:45 smithi062 ceph-mon[32390]: pgmap v274: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:41:45.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:45 smithi138 ceph-mon[36549]: pgmap v274: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:41:47.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:47 smithi105 ceph-mon[36321]: pgmap v275: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-10T00:41:47.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:47 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:41:47.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:47 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:41:47.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:47 smithi062 ceph-mon[32390]: pgmap v275: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-10T00:41:47.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:47 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:41:47.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:47 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:41:47.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:47 smithi138 ceph-mon[36549]: pgmap v275: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-10T00:41:47.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:47 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:41:47.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:47 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:41:48.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:48 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:41:48.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:48 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:41:48.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:48 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:41:48.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:48 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:41:48.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:48 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:41:48.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:48 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:41:48.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:48 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:41:48.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:48 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:41:48.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:48 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:41:49.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:49 smithi105 ceph-mon[36321]: pgmap v276: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-06-10T00:41:49.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:49 smithi062 ceph-mon[32390]: pgmap v276: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-06-10T00:41:49.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:49 smithi138 ceph-mon[36549]: pgmap v276: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 10 op/s 2024-06-10T00:41:51.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:51 smithi105 ceph-mon[36321]: pgmap v277: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:41:51.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:51 smithi062 ceph-mon[32390]: pgmap v277: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:41:51.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:51 smithi138 ceph-mon[36549]: pgmap v277: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:41:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:53 smithi105 ceph-mon[36321]: pgmap v278: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-06-10T00:41:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:53 smithi062 ceph-mon[32390]: pgmap v278: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-06-10T00:41:53.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:53 smithi138 ceph-mon[36549]: pgmap v278: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-06-10T00:41:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:55 smithi105 ceph-mon[36321]: pgmap v279: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:41:55.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:55 smithi062 ceph-mon[32390]: pgmap v279: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:41:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:55 smithi138 ceph-mon[36549]: pgmap v279: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:41:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:57 smithi105 ceph-mon[36321]: pgmap v280: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-10T00:41:57.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:57 smithi062 ceph-mon[32390]: pgmap v280: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-10T00:41:57.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:57 smithi138 ceph-mon[36549]: pgmap v280: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-10T00:41:59.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:41:59 smithi105 ceph-mon[36321]: pgmap v281: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:41:59.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:41:59 smithi062 ceph-mon[32390]: pgmap v281: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:41:59.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:41:59 smithi138 ceph-mon[36549]: pgmap v281: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:42:01.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:01 smithi105 ceph-mon[36321]: pgmap v282: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:42:01.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:01 smithi062 ceph-mon[32390]: pgmap v282: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:42:01.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:01 smithi138 ceph-mon[36549]: pgmap v282: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:42:03.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:03 smithi105 ceph-mon[36321]: pgmap v283: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:42:03.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:03 smithi062 ceph-mon[32390]: pgmap v283: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:42:03.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:03 smithi138 ceph-mon[36549]: pgmap v283: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:42:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:05 smithi105 ceph-mon[36321]: pgmap v284: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-06-10T00:42:05.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:05 smithi062 ceph-mon[32390]: pgmap v284: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-06-10T00:42:05.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:05 smithi138 ceph-mon[36549]: pgmap v284: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-06-10T00:42:07.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:07 smithi105 ceph-mon[36321]: pgmap v285: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-06-10T00:42:07.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:07 smithi062 ceph-mon[32390]: pgmap v285: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-06-10T00:42:07.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:07 smithi138 ceph-mon[36549]: pgmap v285: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-06-10T00:42:08.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:08 smithi105 ceph-mon[36321]: osdmap e105: 12 total, 12 up, 12 in 2024-06-10T00:42:08.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:08 smithi062 ceph-mon[32390]: osdmap e105: 12 total, 12 up, 12 in 2024-06-10T00:42:08.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:08 smithi138 ceph-mon[36549]: osdmap e105: 12 total, 12 up, 12 in 2024-06-10T00:42:09.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:09 smithi105 ceph-mon[36321]: pgmap v287: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:42:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:09 smithi062 ceph-mon[32390]: pgmap v287: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:42:09.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:09 smithi138 ceph-mon[36549]: pgmap v287: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:42:11.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:11 smithi105 ceph-mon[36321]: pgmap v288: 129 pgs: 5 active+clean+snaptrim, 110 active+clean, 14 active+clean+snaptrim_wait; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:42:11.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:11 smithi062 ceph-mon[32390]: pgmap v288: 129 pgs: 5 active+clean+snaptrim, 110 active+clean, 14 active+clean+snaptrim_wait; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:42:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:11 smithi138 ceph-mon[36549]: pgmap v288: 129 pgs: 5 active+clean+snaptrim, 110 active+clean, 14 active+clean+snaptrim_wait; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:42:13.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:13 smithi105 ceph-mon[36321]: pgmap v289: 129 pgs: 8 active+clean+snaptrim, 101 active+clean, 20 active+clean+snaptrim_wait; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-06-10T00:42:13.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:13 smithi062 ceph-mon[32390]: pgmap v289: 129 pgs: 8 active+clean+snaptrim, 101 active+clean, 20 active+clean+snaptrim_wait; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-06-10T00:42:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:13 smithi138 ceph-mon[36549]: pgmap v289: 129 pgs: 8 active+clean+snaptrim, 101 active+clean, 20 active+clean+snaptrim_wait; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-06-10T00:42:14.561 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:42:14.824 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:42:15.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:15 smithi105 ceph-mon[36321]: pgmap v290: 129 pgs: 4 active+clean+snaptrim, 119 active+clean, 6 active+clean+snaptrim_wait; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:42:15.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:15 smithi062 ceph-mon[32390]: pgmap v290: 129 pgs: 4 active+clean+snaptrim, 119 active+clean, 6 active+clean+snaptrim_wait; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:42:15.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:15 smithi138 ceph-mon[36549]: pgmap v290: 129 pgs: 4 active+clean+snaptrim, 119 active+clean, 6 active+clean+snaptrim_wait; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:42:17.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:17 smithi105 ceph-mon[36321]: pgmap v291: 129 pgs: 2 active+clean+snaptrim, 121 active+clean, 6 active+clean+snaptrim_wait; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:42:17.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:17 smithi062 ceph-mon[32390]: pgmap v291: 129 pgs: 2 active+clean+snaptrim, 121 active+clean, 6 active+clean+snaptrim_wait; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:42:17.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:17 smithi138 ceph-mon[36549]: pgmap v291: 129 pgs: 2 active+clean+snaptrim, 121 active+clean, 6 active+clean+snaptrim_wait; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:42:19.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:19 smithi105 ceph-mon[36321]: pgmap v292: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 187 B/s wr, 9 op/s 2024-06-10T00:42:19.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:19 smithi105 ceph-mon[36321]: osdmap e106: 12 total, 12 up, 12 in 2024-06-10T00:42:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:19 smithi062 ceph-mon[32390]: pgmap v292: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 187 B/s wr, 9 op/s 2024-06-10T00:42:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:19 smithi062 ceph-mon[32390]: osdmap e106: 12 total, 12 up, 12 in 2024-06-10T00:42:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:19 smithi138 ceph-mon[36549]: pgmap v292: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 187 B/s wr, 9 op/s 2024-06-10T00:42:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:19 smithi138 ceph-mon[36549]: osdmap e106: 12 total, 12 up, 12 in 2024-06-10T00:42:21.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:21 smithi105 ceph-mon[36321]: pgmap v294: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:42:21.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:21 smithi062 ceph-mon[32390]: pgmap v294: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:42:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:21 smithi138 ceph-mon[36549]: pgmap v294: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:42:23.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:23 smithi105 ceph-mon[36321]: pgmap v295: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:42:23.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:23 smithi062 ceph-mon[32390]: pgmap v295: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:42:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:23 smithi138 ceph-mon[36549]: pgmap v295: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:42:24.706 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:24 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:42:24.707 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:24 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.707 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:24 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.707 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:24 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.707 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:24 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.707 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:24 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.707 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:24 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:24 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:42:24.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:24 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:24 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:24 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:24 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:24 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:24 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:24 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:42:24.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:24 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:24 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:24 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:24 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:24 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:24 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:24.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.956+0000 7f012f1b6700 1 -- 172.21.15.62:0/3443621284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f012806b3c0 msgr2=0x7f012810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:24.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.956+0000 7f012f1b6700 1 --2- 172.21.15.62:0/3443621284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f012806b3c0 0x7f012810f310 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0120009f90 tx=0x7f012000b330 comp rx=0 tx=0).stop 2024-06-10T00:42:24.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.957+0000 7f012f1b6700 1 -- 172.21.15.62:0/3443621284 shutdown_connections 2024-06-10T00:42:24.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.957+0000 7f012f1b6700 1 --2- 172.21.15.62:0/3443621284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f012806b3c0 0x7f012810f310 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.957+0000 7f012f1b6700 1 --2- 172.21.15.62:0/3443621284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01281084f0 0x7f01281088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.957+0000 7f012f1b6700 1 --2- 172.21.15.62:0/3443621284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f012806aa00 0x7f012806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.957+0000 7f012f1b6700 1 -- 172.21.15.62:0/3443621284 >> 172.21.15.62:0/3443621284 conn(0x7f0128078300 msgr2=0x7f0128078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:24.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.957+0000 7f012f1b6700 1 -- 172.21.15.62:0/3443621284 shutdown_connections 2024-06-10T00:42:24.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.957+0000 7f012f1b6700 1 -- 172.21.15.62:0/3443621284 wait complete. 2024-06-10T00:42:24.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.958+0000 7f012f1b6700 1 Processor -- start 2024-06-10T00:42:24.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.958+0000 7f012f1b6700 1 -- start start 2024-06-10T00:42:24.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.958+0000 7f012f1b6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f012806aa00 0x7f012810e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:24.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.958+0000 7f012f1b6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f012806b3c0 0x7f012810eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:24.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.958+0000 7f012f1b6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01281084f0 0x7f01281a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:24.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.958+0000 7f012f1b6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f01281127a0 con 0x7f01281084f0 2024-06-10T00:42:24.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.958+0000 7f012f1b6700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0128112620 con 0x7f012806b3c0 2024-06-10T00:42:24.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.958+0000 7f012f1b6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0128112920 con 0x7f012806aa00 2024-06-10T00:42:24.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.958+0000 7f012cf52700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f012806aa00 0x7f012810e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:24.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.958+0000 7f0127fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f012806b3c0 0x7f012810eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:24.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.958+0000 7f012d753700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01281084f0 0x7f01281a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:24.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.958+0000 7f0127fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f012806b3c0 0x7f012810eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:56032/0 (socket says 172.21.15.62:56032) 2024-06-10T00:42:24.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.958+0000 7f0127fff700 1 -- 172.21.15.62:0/1079830964 learned_addr learned my addr 172.21.15.62:0/1079830964 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:24.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.958+0000 7f012cf52700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f012806aa00 0x7f012810e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:55862/0 (socket says 172.21.15.62:55862) 2024-06-10T00:42:24.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.959+0000 7f0127fff700 1 -- 172.21.15.62:0/1079830964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f012806aa00 msgr2=0x7f012810e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:24.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.959+0000 7f0127fff700 1 --2- 172.21.15.62:0/1079830964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f012806aa00 0x7f012810e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.959+0000 7f0127fff700 1 -- 172.21.15.62:0/1079830964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01281084f0 msgr2=0x7f01281a3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:24.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.959+0000 7f0127fff700 1 --2- 172.21.15.62:0/1079830964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01281084f0 0x7f01281a3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.959+0000 7f0127fff700 1 -- 172.21.15.62:0/1079830964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01281a41c0 con 0x7f012806b3c0 2024-06-10T00:42:24.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.959+0000 7f0127fff700 1 --2- 172.21.15.62:0/1079830964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f012806b3c0 0x7f012810eb40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f011400e9a0 tx=0x7f011400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:24.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.959+0000 7f0125ffb700 1 -- 172.21.15.62:0/1079830964 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011400c9c0 con 0x7f012806b3c0 2024-06-10T00:42:24.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.959+0000 7f0125ffb700 1 -- 172.21.15.62:0/1079830964 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011400cb60 con 0x7f012806b3c0 2024-06-10T00:42:24.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.959+0000 7f012f1b6700 1 -- 172.21.15.62:0/1079830964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01281a44b0 con 0x7f012806b3c0 2024-06-10T00:42:24.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.959+0000 7f0125ffb700 1 -- 172.21.15.62:0/1079830964 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114018660 con 0x7f012806b3c0 2024-06-10T00:42:24.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.960+0000 7f012f1b6700 1 -- 172.21.15.62:0/1079830964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01281abd60 con 0x7f012806b3c0 2024-06-10T00:42:24.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.962+0000 7f0125ffb700 1 -- 172.21.15.62:0/1079830964 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f011400cd10 con 0x7f012806b3c0 2024-06-10T00:42:24.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.963+0000 7f0125ffb700 1 --2- 172.21.15.62:0/1079830964 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f01180a2d90 0x7f01180a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:24.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.963+0000 7f012cf52700 1 --2- 172.21.15.62:0/1079830964 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f01180a2d90 0x7f01180a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:24.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.963+0000 7f0125ffb700 1 -- 172.21.15.62:0/1079830964 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f01140cf9c0 con 0x7f012806b3c0 2024-06-10T00:42:24.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.963+0000 7f012cf52700 1 --2- 172.21.15.62:0/1079830964 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f01180a2d90 0x7f01180a5250 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f011c005fd0 tx=0x7f011c009310 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:24.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0139a39b80 1 Processor -- start 2024-06-10T00:42:24.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0139a39b80 1 -- start start 2024-06-10T00:42:24.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0139a39b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa408f60 0x55cbaa409340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:24.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0139a39b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa409880 0x55cbaa403260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:24.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0139a39b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa4037a0 0x55cbaa403bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:24.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0139a39b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cbaa2f5f60 con 0x55cbaa409880 2024-06-10T00:42:24.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0139a39b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cbaa354850 con 0x55cbaa408f60 2024-06-10T00:42:24.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0139a39b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbaa2bc460 con 0x55cbaa4037a0 2024-06-10T00:42:24.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0127fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa409880 0x55cbaa403260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:24.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f012cf52700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa408f60 0x55cbaa409340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:24.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0127fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa409880 0x55cbaa403260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60162/0 (socket says 172.21.15.62:60162) 2024-06-10T00:42:24.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f012cf52700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa408f60 0x55cbaa409340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:56044/0 (socket says 172.21.15.62:56044) 2024-06-10T00:42:24.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0127fff700 1 -- 172.21.15.62:0/2532681033 learned_addr learned my addr 172.21.15.62:0/2532681033 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:24.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f012d753700 1 --2- 172.21.15.62:0/2532681033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa4037a0 0x55cbaa403bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:24.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0127fff700 1 -- 172.21.15.62:0/2532681033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa4037a0 msgr2=0x55cbaa403bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:24.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0127fff700 1 --2- 172.21.15.62:0/2532681033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa4037a0 0x55cbaa403bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0127fff700 1 -- 172.21.15.62:0/2532681033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa408f60 msgr2=0x55cbaa409340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:24.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0127fff700 1 --2- 172.21.15.62:0/2532681033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa408f60 0x55cbaa409340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.964+0000 7f0127fff700 1 -- 172.21.15.62:0/2532681033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbaa2a38d0 con 0x55cbaa409880 2024-06-10T00:42:24.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.965+0000 7f012cf52700 1 --2- 172.21.15.62:0/2532681033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa408f60 0x55cbaa409340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:42:24.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.965+0000 7f0127fff700 1 --2- 172.21.15.62:0/2532681033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa409880 0x55cbaa403260 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f01140a4210 tx=0x7f011409bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69495e80fac79a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:24.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.965+0000 7f0112ffd700 1 -- 172.21.15.62:0/2532681033 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01140a4740 con 0x55cbaa409880 2024-06-10T00:42:24.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.965+0000 7f0112ffd700 1 -- 172.21.15.62:0/2532681033 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0114099dd0 con 0x55cbaa409880 2024-06-10T00:42:24.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.965+0000 7f0112ffd700 1 -- 172.21.15.62:0/2532681033 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011409b180 con 0x55cbaa409880 2024-06-10T00:42:24.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.966+0000 7f0139a39b80 1 -- 172.21.15.62:0/2532681033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa409880 msgr2=0x55cbaa403260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:24.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.966+0000 7f0139a39b80 1 --2- 172.21.15.62:0/2532681033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa409880 0x55cbaa403260 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f01140a4210 tx=0x7f011409bad0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.966+0000 7f0139a39b80 1 -- 172.21.15.62:0/2532681033 shutdown_connections 2024-06-10T00:42:24.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.966+0000 7f0139a39b80 1 --2- 172.21.15.62:0/2532681033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa4037a0 0x55cbaa403bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.966+0000 7f0139a39b80 1 --2- 172.21.15.62:0/2532681033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa408f60 0x55cbaa409340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.966+0000 7f0139a39b80 1 --2- 172.21.15.62:0/2532681033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa409880 0x55cbaa403260 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.966+0000 7f0139a39b80 1 -- 172.21.15.62:0/2532681033 >> 172.21.15.62:0/2532681033 conn(0x55cbaa330080 msgr2=0x55cbaa3fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:24.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.966+0000 7f0139a39b80 1 -- 172.21.15.62:0/2532681033 shutdown_connections 2024-06-10T00:42:24.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.966+0000 7f0139a39b80 1 -- 172.21.15.62:0/2532681033 wait complete. 2024-06-10T00:42:24.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.966+0000 7f0139a39b80 1 -- start start 2024-06-10T00:42:24.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.966+0000 7f0139a39b80 1 Processor -- start 2024-06-10T00:42:24.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f0139a39b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa4037a0 0x55cbaa40f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:24.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f0139a39b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa408f60 0x55cbaa40c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:24.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f0139a39b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa409880 0x55cbaa40ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:24.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f0139a39b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cbaa2f4fc0 con 0x55cbaa4037a0 2024-06-10T00:42:24.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f0139a39b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cbaa350810 con 0x55cbaa408f60 2024-06-10T00:42:24.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f0139a39b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbaa410260 con 0x55cbaa409880 2024-06-10T00:42:24.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f0139a39b80 10 client.0 _ll_register_callbacks cb 0x55cbaa2ea5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:24.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f012d753700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa409880 0x55cbaa40ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:24.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f012cf52700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa4037a0 0x55cbaa40f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:24.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f012d753700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa409880 0x55cbaa40ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:55888/0 (socket says 172.21.15.62:55888) 2024-06-10T00:42:24.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f012d753700 1 -- 172.21.15.62:0/2960821766 learned_addr learned my addr 172.21.15.62:0/2960821766 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:24.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f0127fff700 1 --2- 172.21.15.62:0/2960821766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa408f60 0x55cbaa40c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:24.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f012d753700 1 -- 172.21.15.62:0/2960821766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa408f60 msgr2=0x55cbaa40c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:24.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f012d753700 1 --2- 172.21.15.62:0/2960821766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa408f60 0x55cbaa40c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f012d753700 1 -- 172.21.15.62:0/2960821766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa4037a0 msgr2=0x55cbaa40f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:24.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f012d753700 1 --2- 172.21.15.62:0/2960821766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa4037a0 0x55cbaa40f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.967+0000 7f012d753700 1 -- 172.21.15.62:0/2960821766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbaa31a750 con 0x55cbaa409880 2024-06-10T00:42:24.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.968+0000 7f012d753700 1 --2- 172.21.15.62:0/2960821766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa409880 0x55cbaa40ceb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0120009d80 tx=0x7f0120003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=161ee84cb5cee59a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:24.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.968+0000 7f0107fff700 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-06-10T00:42:24.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.968+0000 7f0107fff700 1 -- 172.21.15.62:0/2960821766 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012000bcf0 con 0x55cbaa409880 2024-06-10T00:42:24.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.968+0000 7f0139a39b80 1 -- 172.21.15.62:0/2960821766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0114099dd0 con 0x55cbaa409880 2024-06-10T00:42:24.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.968+0000 7f0107fff700 1 -- 172.21.15.62:0/2960821766 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f0120005040 con 0x55cbaa409880 2024-06-10T00:42:24.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.968+0000 7f0107fff700 1 -- 172.21.15.62:0/2960821766 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0120005280 con 0x55cbaa409880 2024-06-10T00:42:24.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.968+0000 7f0139a39b80 10 client.26308 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:24.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.968+0000 7f0139a39b80 1 -- 172.21.15.62:0/2960821766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55cbaa41aae0 con 0x55cbaa409880 2024-06-10T00:42:24.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.968+0000 7f0107fff700 1 -- 172.21.15.62:0/2960821766 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0114099dd0 con 0x55cbaa409880 2024-06-10T00:42:24.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.968+0000 7f0107fff700 1 -- 172.21.15.62:0/2960821766 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f0120011070 con 0x55cbaa409880 2024-06-10T00:42:24.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.968+0000 7f0139a39b80 10 client.26308 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:24.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.968+0000 7f0139a39b80 10 client.26308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:42:24.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.968+0000 7f0139a39b80 20 client.26308 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:24.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.969+0000 7f0139a39b80 1 --2- 172.21.15.62:0/2960821766 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55cbaa41d070 0x55cbaa41f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:24.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.969+0000 7f0139a39b80 4 client.26308 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:42:24.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.969+0000 7f0139a39b80 1 -- 172.21.15.62:0/2960821766 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbaa361d50 con 0x55cbaa41d070 2024-06-10T00:42:24.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.969+0000 7f012cf52700 1 --2- 172.21.15.62:0/2960821766 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55cbaa41d070 0x55cbaa41f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:24.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.969+0000 7f012cf52700 1 --2- 172.21.15.62:0/2960821766 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55cbaa41d070 0x55cbaa41f460 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f9b76a850b3bdf server_cookie=fa3eec7d8938679a in_seq=0 out_seq=0 2024-06-10T00:42:24.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.969+0000 7f0107fff700 10 client.26308 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:24.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.970+0000 7f0107fff700 1 -- 172.21.15.62:0/2960821766 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cbaa361d50 con 0x55cbaa41d070 2024-06-10T00:42:24.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.970+0000 7f0107fff700 10 client.26308 handle_command_reply: tid=0 2024-06-10T00:42:24.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.970+0000 7f0139a39b80 1 client.26308 shutdown 2024-06-10T00:42:24.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.970+0000 7f0139a39b80 1 -- 172.21.15.62:0/2960821766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa409880 msgr2=0x55cbaa40ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:24.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.970+0000 7f0139a39b80 1 --2- 172.21.15.62:0/2960821766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa409880 0x55cbaa40ceb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0120009d80 tx=0x7f0120003f90 comp rx=0 tx=0).stop 2024-06-10T00:42:24.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.970+0000 7f0139a39b80 1 -- 172.21.15.62:0/2960821766 shutdown_connections 2024-06-10T00:42:24.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.970+0000 7f0139a39b80 1 --2- 172.21.15.62:0/2960821766 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55cbaa41d070 0x55cbaa41f460 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.970+0000 7f0139a39b80 1 --2- 172.21.15.62:0/2960821766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa4037a0 0x55cbaa40f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.970+0000 7f0139a39b80 1 --2- 172.21.15.62:0/2960821766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa409880 0x55cbaa40ceb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.970+0000 7f0107fff700 0 client.26308 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:24.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.970+0000 7f0139a39b80 1 --2- 172.21.15.62:0/2960821766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa408f60 0x55cbaa40c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.970+0000 7f0139a39b80 1 -- 172.21.15.62:0/2960821766 >> 172.21.15.62:0/2960821766 conn(0x55cbaa330080 msgr2=0x55cbaa3fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:24.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.970+0000 7f0139a39b80 1 -- 172.21.15.62:0/2960821766 shutdown_connections 2024-06-10T00:42:24.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.970+0000 7f0139a39b80 1 -- 172.21.15.62:0/2960821766 wait complete. 2024-06-10T00:42:24.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.971+0000 7f0139a39b80 20 client.26308 trim_cache size 0 max 16384 2024-06-10T00:42:24.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.992+0000 7f0139a39b80 1 Processor -- start 2024-06-10T00:42:24.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.992+0000 7f0139a39b80 1 -- start start 2024-06-10T00:42:24.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.992+0000 7f0139a39b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa4037a0 0x55cbaa403260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:24.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.992+0000 7f0139a39b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa408f60 0x55cbaa40f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:24.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.992+0000 7f0139a39b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa409880 0x55cbaa3fde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:24.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.992+0000 7f0139a39b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f01281b4cc0 con 0x55cbaa409880 2024-06-10T00:42:24.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.992+0000 7f0139a39b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f01281b6800 con 0x55cbaa4037a0 2024-06-10T00:42:24.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.992+0000 7f0139a39b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f01281b5390 con 0x55cbaa408f60 2024-06-10T00:42:24.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.992+0000 7f012d753700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa409880 0x55cbaa3fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:24.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.992+0000 7f012d753700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa409880 0x55cbaa3fde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60178/0 (socket says 172.21.15.62:60178) 2024-06-10T00:42:24.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.992+0000 7f012d753700 1 -- 172.21.15.62:0/2467112393 learned_addr learned my addr 172.21.15.62:0/2467112393 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:24.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.992+0000 7f0127fff700 1 --2- 172.21.15.62:0/2467112393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa408f60 0x55cbaa40f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.992+0000 7f012cf52700 1 --2- 172.21.15.62:0/2467112393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa4037a0 0x55cbaa403260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.993+0000 7f0127fff700 1 -- 172.21.15.62:0/2467112393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa4037a0 msgr2=0x55cbaa403260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.993+0000 7f0127fff700 1 --2- 172.21.15.62:0/2467112393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa4037a0 0x55cbaa403260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.993+0000 7f0127fff700 1 -- 172.21.15.62:0/2467112393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa409880 msgr2=0x55cbaa3fde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.993+0000 7f0127fff700 1 --2- 172.21.15.62:0/2467112393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa409880 0x55cbaa3fde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.993+0000 7f0127fff700 1 -- 172.21.15.62:0/2467112393 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbaa3fa810 con 0x55cbaa408f60 2024-06-10T00:42:25.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.993+0000 7f0127fff700 1 --2- 172.21.15.62:0/2467112393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa408f60 0x55cbaa40f1b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01140a4210 tx=0x7f01140a0000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f024420b784ac249 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:25.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.993+0000 7f01127fc700 1 -- 172.21.15.62:0/2467112393 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01140b9070 con 0x55cbaa408f60 2024-06-10T00:42:25.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.993+0000 7f01127fc700 1 -- 172.21.15.62:0/2467112393 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01140a4740 con 0x55cbaa408f60 2024-06-10T00:42:25.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.993+0000 7f01127fc700 1 -- 172.21.15.62:0/2467112393 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011409bf00 con 0x55cbaa408f60 2024-06-10T00:42:25.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.994+0000 7f0139a39b80 1 -- 172.21.15.62:0/2467112393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa408f60 msgr2=0x55cbaa40f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:25.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.994+0000 7f0139a39b80 1 --2- 172.21.15.62:0/2467112393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa408f60 0x55cbaa40f1b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01140a4210 tx=0x7f01140a0000 comp rx=0 tx=0).stop 2024-06-10T00:42:25.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.994+0000 7f0139a39b80 1 -- 172.21.15.62:0/2467112393 shutdown_connections 2024-06-10T00:42:25.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.994+0000 7f0139a39b80 1 --2- 172.21.15.62:0/2467112393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa409880 0x55cbaa3fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.994+0000 7f0139a39b80 1 --2- 172.21.15.62:0/2467112393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa4037a0 0x55cbaa403260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.994+0000 7f0139a39b80 1 --2- 172.21.15.62:0/2467112393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa408f60 0x55cbaa40f1b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.994+0000 7f0139a39b80 1 -- 172.21.15.62:0/2467112393 >> 172.21.15.62:0/2467112393 conn(0x55cbaa330080 msgr2=0x55cbaa407ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.994+0000 7f0139a39b80 1 -- 172.21.15.62:0/2467112393 shutdown_connections 2024-06-10T00:42:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.994+0000 7f0139a39b80 1 -- 172.21.15.62:0/2467112393 wait complete. 2024-06-10T00:42:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.995+0000 7f0139a39b80 1 -- start start 2024-06-10T00:42:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.995+0000 7f0139a39b80 1 Processor -- start 2024-06-10T00:42:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.995+0000 7f0139a39b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa4037a0 0x55cbaa4154f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:25.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.995+0000 7f0139a39b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa408f60 0x55cbaa415a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:25.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.995+0000 7f0139a39b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa409880 0x55cbaa4124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:25.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.995+0000 7f0139a39b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f01281b6130 con 0x55cbaa4037a0 2024-06-10T00:42:25.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.995+0000 7f0139a39b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f01281b5a60 con 0x55cbaa409880 2024-06-10T00:42:25.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.995+0000 7f0139a39b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f01281b2c80 con 0x55cbaa408f60 2024-06-10T00:42:25.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.995+0000 7f012cf52700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa4037a0 0x55cbaa4154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:25.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.995+0000 7f0139a39b80 10 client.0 _ll_register_callbacks cb 0x55cbaa2ea5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.995+0000 7f012cf52700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa4037a0 0x55cbaa4154f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60184/0 (socket says 172.21.15.62:60184) 2024-06-10T00:42:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.995+0000 7f012cf52700 1 -- 172.21.15.62:0/724364427 learned_addr learned my addr 172.21.15.62:0/724364427 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.995+0000 7f0127fff700 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa408f60 0x55cbaa415a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.995+0000 7f012d753700 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa409880 0x55cbaa4124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.996+0000 7f012cf52700 1 -- 172.21.15.62:0/724364427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa408f60 msgr2=0x55cbaa415a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.996+0000 7f012cf52700 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa408f60 0x55cbaa415a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.996+0000 7f012cf52700 1 -- 172.21.15.62:0/724364427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa409880 msgr2=0x55cbaa4124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.996+0000 7f012cf52700 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa409880 0x55cbaa4124c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.996+0000 7f012cf52700 1 -- 172.21.15.62:0/724364427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbaa3ff9a0 con 0x55cbaa4037a0 2024-06-10T00:42:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.996+0000 7f012d753700 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa409880 0x55cbaa4124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:42:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.996+0000 7f0127fff700 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa408f60 0x55cbaa415a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:42:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.996+0000 7f012cf52700 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa4037a0 0x55cbaa4154f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x55cbaa41fcd0 tx=0x7f011c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab2ddbd1f2f300a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.996+0000 7f0107fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.996+0000 7f0107fff700 1 -- 172.21.15.62:0/724364427 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c004830 con 0x55cbaa4037a0 2024-06-10T00:42:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.996+0000 7f0107fff700 1 -- 172.21.15.62:0/724364427 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f011c0049d0 con 0x55cbaa4037a0 2024-06-10T00:42:25.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.996+0000 7f0139a39b80 1 -- 172.21.15.62:0/724364427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01140a4740 con 0x55cbaa4037a0 2024-06-10T00:42:25.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.997+0000 7f0107fff700 1 -- 172.21.15.62:0/724364427 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f011c039050 con 0x55cbaa4037a0 2024-06-10T00:42:25.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.997+0000 7f0139a39b80 10 client.16932 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:25.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.997+0000 7f0139a39b80 1 -- 172.21.15.62:0/724364427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55cbaa4882b0 con 0x55cbaa4037a0 2024-06-10T00:42:25.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.997+0000 7f0107fff700 1 -- 172.21.15.62:0/724364427 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01140a4740 con 0x55cbaa4037a0 2024-06-10T00:42:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.997+0000 7f0107fff700 1 -- 172.21.15.62:0/724364427 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f011c02a380 con 0x55cbaa4037a0 2024-06-10T00:42:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.997+0000 7f0139a39b80 10 client.16932 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.997+0000 7f0139a39b80 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:42:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.997+0000 7f0139a39b80 20 client.16932 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.997+0000 7f0139a39b80 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55cbaa4889c0 0x55cbaa48ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.997+0000 7f0139a39b80 4 client.16932 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:42:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.997+0000 7f0139a39b80 1 -- 172.21.15.62:0/724364427 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cbaa35c2d0 con 0x55cbaa4889c0 2024-06-10T00:42:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0127fff700 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55cbaa4889c0 0x55cbaa48ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0127fff700 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55cbaa4889c0 0x55cbaa48ade0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1990e1e387791223 server_cookie=94f9c84ce1b5b19d in_seq=0 out_seq=0 2024-06-10T00:42:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0107fff700 10 client.16932 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0107fff700 1 -- 172.21.15.62:0/724364427 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cbaa35c2d0 con 0x55cbaa4889c0 2024-06-10T00:42:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0107fff700 10 client.16932 handle_command_reply: tid=0 2024-06-10T00:42:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0139a39b80 1 client.16932 shutdown 2024-06-10T00:42:25.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0139a39b80 1 -- 172.21.15.62:0/724364427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa4037a0 msgr2=0x55cbaa4154f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:25.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0139a39b80 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa4037a0 0x55cbaa4154f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x55cbaa41fcd0 tx=0x7f011c0047c0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0139a39b80 1 -- 172.21.15.62:0/724364427 shutdown_connections 2024-06-10T00:42:25.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0139a39b80 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55cbaa4889c0 0x55cbaa48ade0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0139a39b80 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cbaa4037a0 0x55cbaa4154f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0107fff700 0 client.16932 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:25.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0139a39b80 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbaa409880 0x55cbaa4124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0139a39b80 1 --2- 172.21.15.62:0/724364427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbaa408f60 0x55cbaa415a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.998+0000 7f0139a39b80 1 -- 172.21.15.62:0/724364427 >> 172.21.15.62:0/724364427 conn(0x55cbaa330080 msgr2=0x55cbaa408350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:25.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.999+0000 7f0139a39b80 1 -- 172.21.15.62:0/724364427 shutdown_connections 2024-06-10T00:42:25.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.999+0000 7f0139a39b80 1 -- 172.21.15.62:0/724364427 wait complete. 2024-06-10T00:42:25.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.999+0000 7f0139a39b80 20 client.16932 trim_cache size 0 max 16384 2024-06-10T00:42:25.011 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:42:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-06-10T00:42:25.011 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-06-10T00:42:25.011 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:42:25.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.999+0000 7f012f1b6700 1 -- 172.21.15.62:0/1079830964 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f01180a2d90 msgr2=0x7f01180a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:25.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.999+0000 7f012f1b6700 1 --2- 172.21.15.62:0/1079830964 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f01180a2d90 0x7f01180a5250 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f011c005fd0 tx=0x7f011c009310 comp rx=0 tx=0).stop 2024-06-10T00:42:25.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.999+0000 7f012f1b6700 1 -- 172.21.15.62:0/1079830964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f012806b3c0 msgr2=0x7f012810eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:25.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.999+0000 7f012f1b6700 1 --2- 172.21.15.62:0/1079830964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f012806b3c0 0x7f012810eb40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f011400e9a0 tx=0x7f011400ecb0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.999+0000 7f012f1b6700 1 -- 172.21.15.62:0/1079830964 shutdown_connections 2024-06-10T00:42:25.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.999+0000 7f012f1b6700 1 --2- 172.21.15.62:0/1079830964 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f01180a2d90 0x7f01180a5250 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.999+0000 7f012f1b6700 1 --2- 172.21.15.62:0/1079830964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01281084f0 0x7f01281a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.999+0000 7f012f1b6700 1 --2- 172.21.15.62:0/1079830964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f012806aa00 0x7f012810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.999+0000 7f012f1b6700 1 --2- 172.21.15.62:0/1079830964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f012806b3c0 0x7f012810eb40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:25.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:24.999+0000 7f012f1b6700 1 -- 172.21.15.62:0/1079830964 >> 172.21.15.62:0/1079830964 conn(0x7f0128078300 msgr2=0x7f01280ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:25.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:25.000+0000 7f012f1b6700 1 -- 172.21.15.62:0/1079830964 shutdown_connections 2024-06-10T00:42:25.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:25.000+0000 7f012f1b6700 1 -- 172.21.15.62:0/1079830964 wait complete. 2024-06-10T00:42:25.724 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T00:42:25.725 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:42:25.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:25 smithi105 ceph-mon[36321]: pgmap v296: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:42:25.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:25 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:42:25.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:25 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:42:25.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:25 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:25.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:25 smithi062 ceph-mon[32390]: pgmap v296: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:42:25.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:25 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:42:25.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:25 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:42:25.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:25 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:25.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:25 smithi138 ceph-mon[36549]: pgmap v296: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:42:25.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:25 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:42:25.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:25 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:42:25.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:25 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:42:25.972 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:42:27.731 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:27 smithi062 ceph-mon[32390]: pgmap v297: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-10T00:42:27.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.730+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/116446371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c08101390 msgr2=0x7f8c08101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:27.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.730+0000 7f8c0f2e0700 1 --2- 172.21.15.62:0/116446371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c08101390 0x7f8c08101810 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009990 tx=0x7f8bfc009ca0 comp rx=0 tx=0).stop 2024-06-10T00:42:27.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.730+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/116446371 shutdown_connections 2024-06-10T00:42:27.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.730+0000 7f8c0f2e0700 1 --2- 172.21.15.62:0/116446371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c08101d50 0x7f8c0810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:27.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.730+0000 7f8c0f2e0700 1 --2- 172.21.15.62:0/116446371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c08107400 0x7f8c081077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:27.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.730+0000 7f8c0f2e0700 1 --2- 172.21.15.62:0/116446371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c08101390 0x7f8c08101810 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.730+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/116446371 >> 172.21.15.62:0/116446371 conn(0x7f8c08078340 msgr2=0x7f8c08078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.730+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/116446371 shutdown_connections 2024-06-10T00:42:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.731+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/116446371 wait complete. 2024-06-10T00:42:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.731+0000 7f8c0f2e0700 1 Processor -- start 2024-06-10T00:42:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.731+0000 7f8c0f2e0700 1 -- start start 2024-06-10T00:42:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0f2e0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c08101390 0x7f8c08199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0f2e0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c08101d50 0x7f8c08199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0f2e0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c08107400 0x7f8c081a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0d07c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c08101390 0x7f8c08199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0f2e0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8c081116a0 con 0x7f8c08101390 2024-06-10T00:42:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0d07c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c08101390 0x7f8c08199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60192/0 (socket says 172.21.15.62:60192) 2024-06-10T00:42:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0f2e0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c08111520 con 0x7f8c08101d50 2024-06-10T00:42:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0d07c700 1 -- 172.21.15.62:0/3694759261 learned_addr learned my addr 172.21.15.62:0/3694759261 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/3694759261 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c08111820 con 0x7f8c08107400 2024-06-10T00:42:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0c87b700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c08101d50 0x7f8c08199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0d87d700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c08107400 0x7f8c081a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:27.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0d07c700 1 -- 172.21.15.62:0/3694759261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c08107400 msgr2=0x7f8c081a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:27.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0d07c700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c08107400 0x7f8c081a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:27.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0d07c700 1 -- 172.21.15.62:0/3694759261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c08101d50 msgr2=0x7f8c08199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:27.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0d07c700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c08101d50 0x7f8c08199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:27.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0d07c700 1 -- 172.21.15.62:0/3694759261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c081a5020 con 0x7f8c08101390 2024-06-10T00:42:27.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0d87d700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c08107400 0x7f8c081a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:27.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.732+0000 7f8c0d07c700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c08101390 0x7f8c08199a60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400d950 tx=0x7f8c0400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:27.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.733+0000 7f8bfa7fc700 1 -- 172.21.15.62:0/3694759261 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04016070 con 0x7f8c08101390 2024-06-10T00:42:27.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.733+0000 7f8bfa7fc700 1 -- 172.21.15.62:0/3694759261 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c04009e20 con 0x7f8c08101390 2024-06-10T00:42:27.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.733+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/3694759261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c081a5250 con 0x7f8c08101390 2024-06-10T00:42:27.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.733+0000 7f8bfa7fc700 1 -- 172.21.15.62:0/3694759261 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0400f620 con 0x7f8c08101390 2024-06-10T00:42:27.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.733+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/3694759261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c081a5760 con 0x7f8c08101390 2024-06-10T00:42:27.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.734+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/3694759261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c08069180 con 0x7f8c08101390 2024-06-10T00:42:27.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.735+0000 7f8bfa7fc700 1 -- 172.21.15.62:0/3694759261 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8c04019070 con 0x7f8c08101390 2024-06-10T00:42:27.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.735+0000 7f8bfa7fc700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8bf40a2d90 0x7f8bf40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:27.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.736+0000 7f8c0c87b700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8bf40a2d90 0x7f8bf40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:27.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.736+0000 7f8bfa7fc700 1 -- 172.21.15.62:0/3694759261 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f8c040cf3c0 con 0x7f8c08101390 2024-06-10T00:42:27.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.736+0000 7f8c0c87b700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8bf40a2d90 0x7f8bf40a5250 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00b5c0 tx=0x7f8bfc005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:27.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.739+0000 7f8bfa7fc700 1 -- 172.21.15.62:0/3694759261 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8c0409bbd0 con 0x7f8c08101390 2024-06-10T00:42:27.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:27 smithi138 ceph-mon[36549]: pgmap v297: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-10T00:42:27.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.989+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/3694759261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c0802dca0 con 0x7f8c08101390 2024-06-10T00:42:27.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.990+0000 7f8bfa7fc700 1 -- 172.21.15.62:0/3694759261 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11938 (secure 0 0 0) 0x7f8c0409b4d0 con 0x7f8c08101390 2024-06-10T00:42:27.991 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:42:27.991 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:40:10.974660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:42:27.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.992+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/3694759261 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8bf40a2d90 msgr2=0x7f8bf40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:27.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.992+0000 7f8c0f2e0700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8bf40a2d90 0x7f8bf40a5250 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00b5c0 tx=0x7f8bfc005960 comp rx=0 tx=0).stop 2024-06-10T00:42:27.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.992+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/3694759261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c08101390 msgr2=0x7f8c08199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:27.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.992+0000 7f8c0f2e0700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c08101390 0x7f8c08199a60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400d950 tx=0x7f8c0400dc60 comp rx=0 tx=0).stop 2024-06-10T00:42:27.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.993+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/3694759261 shutdown_connections 2024-06-10T00:42:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.993+0000 7f8c0f2e0700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8bf40a2d90 0x7f8bf40a5250 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.993+0000 7f8c0f2e0700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c08101390 0x7f8c08199a60 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.993+0000 7f8c0f2e0700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c08107400 0x7f8c081a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.993+0000 7f8c0f2e0700 1 --2- 172.21.15.62:0/3694759261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c08101d50 0x7f8c08199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.993+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/3694759261 >> 172.21.15.62:0/3694759261 conn(0x7f8c08078340 msgr2=0x7f8c080ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.993+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/3694759261 shutdown_connections 2024-06-10T00:42:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:27.993+0000 7f8c0f2e0700 1 -- 172.21.15.62:0/3694759261 wait complete. 2024-06-10T00:42:27.995 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 38 2024-06-10T00:42:28.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:27 smithi105 ceph-mon[36321]: pgmap v297: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-10T00:42:28.645 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:28 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3694759261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:42:28.674 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T00:42:28.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:28 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3694759261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:42:28.924 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:42:29.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:28 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3694759261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:42:29.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:29 smithi062 ceph-mon[32390]: pgmap v298: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-10T00:42:29.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:29 smithi138 ceph-mon[36549]: pgmap v298: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-10T00:42:30.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:29 smithi105 ceph-mon[36321]: pgmap v298: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-10T00:42:30.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.585+0000 7f034d624700 1 -- 172.21.15.62:0/768321330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0348100ec0 msgr2=0x7f0348101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:30.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.585+0000 7f034d624700 1 --2- 172.21.15.62:0/768321330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0348100ec0 0x7f0348101340 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f033c009990 tx=0x7f033c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.585+0000 7f034d624700 1 -- 172.21.15.62:0/768321330 shutdown_connections 2024-06-10T00:42:30.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.585+0000 7f034d624700 1 --2- 172.21.15.62:0/768321330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0348101880 0x7f034810f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.585+0000 7f034d624700 1 --2- 172.21.15.62:0/768321330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0348108670 0x7f0348108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.585+0000 7f034d624700 1 --2- 172.21.15.62:0/768321330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0348100ec0 0x7f0348101340 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.585+0000 7f034d624700 1 -- 172.21.15.62:0/768321330 >> 172.21.15.62:0/768321330 conn(0x7f0348076600 msgr2=0x7f0348076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.585+0000 7f034d624700 1 -- 172.21.15.62:0/768321330 shutdown_connections 2024-06-10T00:42:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.585+0000 7f034d624700 1 -- 172.21.15.62:0/768321330 wait complete. 2024-06-10T00:42:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.585+0000 7f034d624700 1 Processor -- start 2024-06-10T00:42:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f034d624700 1 -- start start 2024-06-10T00:42:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f034d624700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0348100ec0 0x7f03481020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f034d624700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0348101880 0x7f03481025e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f034d624700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0348108670 0x7f0348102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f034d624700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0348112780 con 0x7f0348101880 2024-06-10T00:42:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f034d624700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0348112600 con 0x7f0348100ec0 2024-06-10T00:42:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f034d624700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0348112900 con 0x7f0348108670 2024-06-10T00:42:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f03467fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0348101880 0x7f03481025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f03467fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0348101880 0x7f03481025e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49522/0 (socket says 172.21.15.62:49522) 2024-06-10T00:42:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f03467fc700 1 -- 172.21.15.62:0/2895583476 learned_addr learned my addr 172.21.15.62:0/2895583476 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f0346ffd700 1 --2- 172.21.15.62:0/2895583476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0348100ec0 0x7f03481020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f03477fe700 1 --2- 172.21.15.62:0/2895583476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0348108670 0x7f0348102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f03467fc700 1 -- 172.21.15.62:0/2895583476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0348108670 msgr2=0x7f0348102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f03467fc700 1 --2- 172.21.15.62:0/2895583476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0348108670 0x7f0348102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f03467fc700 1 -- 172.21.15.62:0/2895583476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0348100ec0 msgr2=0x7f03481020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f03467fc700 1 --2- 172.21.15.62:0/2895583476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0348100ec0 0x7f03481020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f03467fc700 1 -- 172.21.15.62:0/2895583476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03481a7080 con 0x7f0348101880 2024-06-10T00:42:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.586+0000 7f03467fc700 1 --2- 172.21.15.62:0/2895583476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0348101880 0x7f03481025e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f033c000c00 tx=0x7f033c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.587+0000 7f032ffff700 1 -- 172.21.15.62:0/2895583476 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c027070 con 0x7f0348101880 2024-06-10T00:42:30.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.587+0000 7f034d624700 1 -- 172.21.15.62:0/2895583476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03481a7310 con 0x7f0348101880 2024-06-10T00:42:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.587+0000 7f032ffff700 1 -- 172.21.15.62:0/2895583476 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f033c004050 con 0x7f0348101880 2024-06-10T00:42:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.587+0000 7f032ffff700 1 -- 172.21.15.62:0/2895583476 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c00f460 con 0x7f0348101880 2024-06-10T00:42:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.587+0000 7f034d624700 1 -- 172.21.15.62:0/2895583476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03481a77c0 con 0x7f0348101880 2024-06-10T00:42:30.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.590+0000 7f032ffff700 1 -- 172.21.15.62:0/2895583476 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f033c02a070 con 0x7f0348101880 2024-06-10T00:42:30.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.590+0000 7f0357ea7b80 1 Processor -- start 2024-06-10T00:42:30.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.590+0000 7f0357ea7b80 1 -- start start 2024-06-10T00:42:30.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f032ffff700 1 --2- 172.21.15.62:0/2895583476 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f03300a2d90 0x7f03300a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0357ea7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65640 0x560195e65a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0357ea7b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65f60 0x560195e5f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0357ea7b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e5ff20 0x560195e60340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0357ea7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560195db1810 con 0x560195e65640 2024-06-10T00:42:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0357ea7b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560195d59720 con 0x560195e65f60 2024-06-10T00:42:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0357ea7b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560195dad7d0 con 0x560195e5ff20 2024-06-10T00:42:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0346ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65640 0x560195e65a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0346ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65640 0x560195e65a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49534/0 (socket says 172.21.15.62:49534) 2024-06-10T00:42:30.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0346ffd700 1 -- 172.21.15.62:0/1458443497 learned_addr learned my addr 172.21.15.62:0/1458443497 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:30.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f03477fe700 1 --2- 172.21.15.62:0/2895583476 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f03300a2d90 0x7f03300a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f03477fe700 1 --2- 172.21.15.62:0/1458443497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e5ff20 0x560195e60340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f032ffff700 1 -- 172.21.15.62:0/2895583476 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f033c0d08a0 con 0x7f0348101880 2024-06-10T00:42:30.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f03467fc700 1 --2- 172.21.15.62:0/1458443497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65f60 0x560195e5f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0346ffd700 1 -- 172.21.15.62:0/1458443497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e5ff20 msgr2=0x560195e60340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:30.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0346ffd700 1 --2- 172.21.15.62:0/1458443497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e5ff20 0x560195e60340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0346ffd700 1 -- 172.21.15.62:0/1458443497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65f60 msgr2=0x560195e5f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:30.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0346ffd700 1 --2- 172.21.15.62:0/1458443497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65f60 0x560195e5f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0346ffd700 1 -- 172.21.15.62:0/1458443497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560195d4ff50 con 0x560195e65640 2024-06-10T00:42:30.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f03467fc700 1 --2- 172.21.15.62:0/1458443497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65f60 0x560195e5f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:30.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f0346ffd700 1 --2- 172.21.15.62:0/1458443497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65640 0x560195e65a20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f033800ad00 tx=0x7f033800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fe5f24de38ca66 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:30.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.591+0000 7f03477fe700 1 --2- 172.21.15.62:0/2895583476 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f03300a2d90 0x7f03300a5250 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0340007740 tx=0x7f0340007690 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:30.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.592+0000 7f032d7fa700 1 -- 172.21.15.62:0/1458443497 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033800ae50 con 0x560195e65640 2024-06-10T00:42:30.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.592+0000 7f032d7fa700 1 -- 172.21.15.62:0/1458443497 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0338004500 con 0x560195e65640 2024-06-10T00:42:30.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.592+0000 7f032d7fa700 1 -- 172.21.15.62:0/1458443497 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03380186a0 con 0x560195e65640 2024-06-10T00:42:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.592+0000 7f0357ea7b80 1 -- 172.21.15.62:0/1458443497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65640 msgr2=0x560195e65a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.592+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/1458443497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65640 0x560195e65a20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f033800ad00 tx=0x7f033800f6a0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.592+0000 7f0357ea7b80 1 -- 172.21.15.62:0/1458443497 shutdown_connections 2024-06-10T00:42:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.592+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/1458443497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65640 0x560195e65a20 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.592+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/1458443497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e5ff20 0x560195e60340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.592+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/1458443497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65f60 0x560195e5f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.592+0000 7f0357ea7b80 1 -- 172.21.15.62:0/1458443497 >> 172.21.15.62:0/1458443497 conn(0x560195d8acc0 msgr2=0x560195e5a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.592+0000 7f0357ea7b80 1 -- 172.21.15.62:0/1458443497 shutdown_connections 2024-06-10T00:42:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.592+0000 7f0357ea7b80 1 -- 172.21.15.62:0/1458443497 wait complete. 2024-06-10T00:42:30.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.593+0000 7f0357ea7b80 1 -- start start 2024-06-10T00:42:30.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.593+0000 7f0357ea7b80 1 Processor -- start 2024-06-10T00:42:30.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.593+0000 7f0357ea7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e5ff20 0x560195e6bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.593+0000 7f0357ea7b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65640 0x560195e68ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.593+0000 7f0357ea7b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e65f60 0x560195e69430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.593+0000 7f0357ea7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560195e6c9d0 con 0x560195e5ff20 2024-06-10T00:42:30.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.593+0000 7f0357ea7b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560195d51bf0 con 0x560195e65640 2024-06-10T00:42:30.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.593+0000 7f0357ea7b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560195e6cb50 con 0x560195e65f60 2024-06-10T00:42:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.593+0000 7f0357ea7b80 10 client.0 _ll_register_callbacks cb 0x560195d4b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.593+0000 7f0346ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e5ff20 0x560195e6bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f0346ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e5ff20 0x560195e6bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49546/0 (socket says 172.21.15.62:49546) 2024-06-10T00:42:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f0346ffd700 1 -- 172.21.15.62:0/305538467 learned_addr learned my addr 172.21.15.62:0/305538467 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f03477fe700 1 --2- 172.21.15.62:0/305538467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e65f60 0x560195e69430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f03467fc700 1 --2- 172.21.15.62:0/305538467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65640 0x560195e68ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f0346ffd700 1 -- 172.21.15.62:0/305538467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e65f60 msgr2=0x560195e69430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:30.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f0346ffd700 1 --2- 172.21.15.62:0/305538467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e65f60 0x560195e69430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f0346ffd700 1 -- 172.21.15.62:0/305538467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65640 msgr2=0x560195e68ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:30.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f0346ffd700 1 --2- 172.21.15.62:0/305538467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65640 0x560195e68ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f0346ffd700 1 -- 172.21.15.62:0/305538467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560195d008d0 con 0x560195e5ff20 2024-06-10T00:42:30.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f03467fc700 1 --2- 172.21.15.62:0/305538467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65640 0x560195e68ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:30.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f0346ffd700 1 --2- 172.21.15.62:0/305538467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e5ff20 0x560195e6bbf0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f033800acd0 tx=0x7f0338004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b476719ac67e237 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:30.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f032aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:30.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f032aff5700 1 -- 172.21.15.62:0/305538467 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338026020 con 0x560195e5ff20 2024-06-10T00:42:30.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f0357ea7b80 1 -- 172.21.15.62:0/305538467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0338004500 con 0x560195e5ff20 2024-06-10T00:42:30.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f032aff5700 1 -- 172.21.15.62:0/305538467 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f0338005040 con 0x560195e5ff20 2024-06-10T00:42:30.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f032aff5700 1 -- 172.21.15.62:0/305538467 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0338031050 con 0x560195e5ff20 2024-06-10T00:42:30.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f0357ea7b80 10 client.16950 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:30.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.594+0000 7f0357ea7b80 1 -- 172.21.15.62:0/305538467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560195e77230 con 0x560195e5ff20 2024-06-10T00:42:30.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.595+0000 7f032aff5700 1 -- 172.21.15.62:0/305538467 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0338004500 con 0x560195e5ff20 2024-06-10T00:42:30.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.595+0000 7f032aff5700 1 -- 172.21.15.62:0/305538467 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f0338004c80 con 0x560195e5ff20 2024-06-10T00:42:30.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.595+0000 7f0357ea7b80 10 client.16950 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:30.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.595+0000 7f0357ea7b80 10 client.16950 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:42:30.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.595+0000 7f0357ea7b80 20 client.16950 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:30.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.595+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/305538467 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x560195e797b0 0x560195e7bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.595+0000 7f0357ea7b80 4 client.16950 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:42:30.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.595+0000 7f0357ea7b80 1 -- 172.21.15.62:0/305538467 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560195dbed10 con 0x560195e797b0 2024-06-10T00:42:30.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.595+0000 7f03467fc700 1 --2- 172.21.15.62:0/305538467 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x560195e797b0 0x560195e7bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.596+0000 7f03467fc700 1 --2- 172.21.15.62:0/305538467 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x560195e797b0 0x560195e7bba0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5faab8040384e788 server_cookie=16a95bd436f2a1d3 in_seq=0 out_seq=0 2024-06-10T00:42:30.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.596+0000 7f032aff5700 10 client.16950 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:42:30.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f032aff5700 1 -- 172.21.15.62:0/305538467 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560195dbed10 con 0x560195e797b0 2024-06-10T00:42:30.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f032aff5700 10 client.16950 handle_command_reply: tid=0 2024-06-10T00:42:30.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f0357ea7b80 1 client.16950 shutdown 2024-06-10T00:42:30.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f0357ea7b80 1 -- 172.21.15.62:0/305538467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e5ff20 msgr2=0x560195e6bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:30.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/305538467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e5ff20 0x560195e6bbf0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f033800acd0 tx=0x7f0338004050 comp rx=0 tx=0).stop 2024-06-10T00:42:30.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f0357ea7b80 1 -- 172.21.15.62:0/305538467 shutdown_connections 2024-06-10T00:42:30.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/305538467 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x560195e797b0 0x560195e7bba0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/305538467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e5ff20 0x560195e6bbf0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f032aff5700 0 client.16950 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:42:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/305538467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e65f60 0x560195e69430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/305538467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65640 0x560195e68ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f0357ea7b80 1 -- 172.21.15.62:0/305538467 >> 172.21.15.62:0/305538467 conn(0x560195d8acc0 msgr2=0x560195e5e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f0357ea7b80 1 -- 172.21.15.62:0/305538467 shutdown_connections 2024-06-10T00:42:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f0357ea7b80 1 -- 172.21.15.62:0/305538467 wait complete. 2024-06-10T00:42:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.597+0000 7f0357ea7b80 20 client.16950 trim_cache size 0 max 16384 2024-06-10T00:42:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.612+0000 7f0357ea7b80 1 Processor -- start 2024-06-10T00:42:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.612+0000 7f0357ea7b80 1 -- start start 2024-06-10T00:42:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.612+0000 7f0357ea7b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e5ff20 0x560195e5f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.612+0000 7f0357ea7b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65640 0x560195e5a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.612+0000 7f0357ea7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65f60 0x560195e54510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.612+0000 7f0357ea7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f03481b0740 con 0x560195e65f60 2024-06-10T00:42:30.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.612+0000 7f0357ea7b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03481b2280 con 0x560195e65640 2024-06-10T00:42:30.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.612+0000 7f0357ea7b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03481b0e10 con 0x560195e5ff20 2024-06-10T00:42:30.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.612+0000 7f03477fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65f60 0x560195e54510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.612+0000 7f03477fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65f60 0x560195e54510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49560/0 (socket says 172.21.15.62:49560) 2024-06-10T00:42:30.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.612+0000 7f03477fe700 1 -- 172.21.15.62:0/2147620293 learned_addr learned my addr 172.21.15.62:0/2147620293 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:30.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.613+0000 7f0346ffd700 1 --2- 172.21.15.62:0/2147620293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e5ff20 0x560195e5f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.613+0000 7f03467fc700 1 --2- 172.21.15.62:0/2147620293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65640 0x560195e5a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.613+0000 7f03477fe700 1 -- 172.21.15.62:0/2147620293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e5ff20 msgr2=0x560195e5f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:30.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.613+0000 7f03477fe700 1 --2- 172.21.15.62:0/2147620293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e5ff20 0x560195e5f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.613+0000 7f03477fe700 1 -- 172.21.15.62:0/2147620293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65640 msgr2=0x560195e5a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:30.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.613+0000 7f03477fe700 1 --2- 172.21.15.62:0/2147620293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65640 0x560195e5a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.613+0000 7f03477fe700 1 -- 172.21.15.62:0/2147620293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560195e5b520 con 0x560195e65f60 2024-06-10T00:42:30.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.613+0000 7f0346ffd700 1 --2- 172.21.15.62:0/2147620293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e5ff20 0x560195e5f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:30.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.613+0000 7f03477fe700 1 --2- 172.21.15.62:0/2147620293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65f60 0x560195e54510 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0340014790 tx=0x7f0340007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6201472dfbccca12 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:30.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.613+0000 7f032cff9700 1 -- 172.21.15.62:0/2147620293 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340010040 con 0x560195e65f60 2024-06-10T00:42:30.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.613+0000 7f032cff9700 1 -- 172.21.15.62:0/2147620293 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0340004bb0 con 0x560195e65f60 2024-06-10T00:42:30.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.613+0000 7f032cff9700 1 -- 172.21.15.62:0/2147620293 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340005180 con 0x560195e65f60 2024-06-10T00:42:30.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.614+0000 7f0357ea7b80 1 -- 172.21.15.62:0/2147620293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65f60 msgr2=0x560195e54510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:30.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.614+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/2147620293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65f60 0x560195e54510 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0340014790 tx=0x7f0340007950 comp rx=0 tx=0).stop 2024-06-10T00:42:30.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.614+0000 7f0357ea7b80 1 -- 172.21.15.62:0/2147620293 shutdown_connections 2024-06-10T00:42:30.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.614+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/2147620293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65f60 0x560195e54510 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.614+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/2147620293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e5ff20 0x560195e5f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.614+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/2147620293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e65640 0x560195e5a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.614+0000 7f0357ea7b80 1 -- 172.21.15.62:0/2147620293 >> 172.21.15.62:0/2147620293 conn(0x560195d8acc0 msgr2=0x560195e53e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:30.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.614+0000 7f0357ea7b80 1 -- 172.21.15.62:0/2147620293 shutdown_connections 2024-06-10T00:42:30.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.614+0000 7f0357ea7b80 1 -- 172.21.15.62:0/2147620293 wait complete. 2024-06-10T00:42:30.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.614+0000 7f0357ea7b80 1 -- start start 2024-06-10T00:42:30.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.614+0000 7f0357ea7b80 1 Processor -- start 2024-06-10T00:42:30.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.614+0000 7f0357ea7b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e5ff20 0x560195e6cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.614+0000 7f0357ea7b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65640 0x560195e6d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f0357ea7b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e65f60 0x560195ee2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f0357ea7b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f03481b1bb0 con 0x560195e65640 2024-06-10T00:42:30.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f0357ea7b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03481b14e0 con 0x560195e5ff20 2024-06-10T00:42:30.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f0357ea7b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03481ae700 con 0x560195e65f60 2024-06-10T00:42:30.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f0357ea7b80 10 client.0 _ll_register_callbacks cb 0x560195d4b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:30.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f03467fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65640 0x560195e6d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f03467fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65640 0x560195e6d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49568/0 (socket says 172.21.15.62:49568) 2024-06-10T00:42:30.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f03467fc700 1 -- 172.21.15.62:0/334190138 learned_addr learned my addr 172.21.15.62:0/334190138 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:30.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f0346ffd700 1 --2- 172.21.15.62:0/334190138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e5ff20 0x560195e6cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f03467fc700 1 -- 172.21.15.62:0/334190138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e65f60 msgr2=0x560195ee2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:30.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f03467fc700 1 --2- 172.21.15.62:0/334190138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e65f60 0x560195ee2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f03467fc700 1 -- 172.21.15.62:0/334190138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e5ff20 msgr2=0x560195e6cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:30.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f03467fc700 1 --2- 172.21.15.62:0/334190138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e5ff20 0x560195e6cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f03467fc700 1 -- 172.21.15.62:0/334190138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560195e6dca0 con 0x560195e65640 2024-06-10T00:42:30.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f03467fc700 1 --2- 172.21.15.62:0/334190138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65640 0x560195e6d3e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f033c09c9d0 tx=0x7f033c022910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392ec5d8f6c3f9ae server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:30.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f032aff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:30.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f032aff5700 1 -- 172.21.15.62:0/334190138 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c09d140 con 0x560195e65640 2024-06-10T00:42:30.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f032aff5700 1 -- 172.21.15.62:0/334190138 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f033c022570 con 0x560195e65640 2024-06-10T00:42:30.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f0357ea7b80 1 -- 172.21.15.62:0/334190138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0340004bb0 con 0x560195e65640 2024-06-10T00:42:30.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f032aff5700 1 -- 172.21.15.62:0/334190138 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f033c09ca40 con 0x560195e65640 2024-06-10T00:42:30.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f0357ea7b80 10 client.16956 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:30.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.615+0000 7f0357ea7b80 1 -- 172.21.15.62:0/334190138 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560195ee38b0 con 0x560195e65640 2024-06-10T00:42:30.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.616+0000 7f032aff5700 1 -- 172.21.15.62:0/334190138 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0340004bb0 con 0x560195e65640 2024-06-10T00:42:30.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.616+0000 7f032aff5700 1 -- 172.21.15.62:0/334190138 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f033c09cca0 con 0x560195e65640 2024-06-10T00:42:30.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.616+0000 7f0357ea7b80 10 client.16956 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:30.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.616+0000 7f0357ea7b80 10 client.16956 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:42:30.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.616+0000 7f0357ea7b80 20 client.16956 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:30.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.616+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/334190138 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x560195e797b0 0x560195ee5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:30.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.616+0000 7f0357ea7b80 4 client.16956 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:42:30.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.616+0000 7f0357ea7b80 1 -- 172.21.15.62:0/334190138 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560195db9290 con 0x560195e797b0 2024-06-10T00:42:30.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.616+0000 7f0346ffd700 1 --2- 172.21.15.62:0/334190138 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x560195e797b0 0x560195ee5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:30.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.616+0000 7f0346ffd700 1 --2- 172.21.15.62:0/334190138 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x560195e797b0 0x560195ee5fb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=76a75f69723652f3 server_cookie=15a47d276d97bbc5 in_seq=0 out_seq=0 2024-06-10T00:42:30.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.616+0000 7f032aff5700 10 client.16956 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:42:30.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f032aff5700 1 -- 172.21.15.62:0/334190138 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560195db9290 con 0x560195e797b0 2024-06-10T00:42:30.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f032aff5700 10 client.16956 handle_command_reply: tid=0 2024-06-10T00:42:30.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f0357ea7b80 1 client.16956 shutdown 2024-06-10T00:42:30.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f0357ea7b80 1 -- 172.21.15.62:0/334190138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65640 msgr2=0x560195e6d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:30.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/334190138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65640 0x560195e6d3e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f033c09c9d0 tx=0x7f033c022910 comp rx=0 tx=0).stop 2024-06-10T00:42:30.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f0357ea7b80 1 -- 172.21.15.62:0/334190138 shutdown_connections 2024-06-10T00:42:30.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/334190138 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x560195e797b0 0x560195ee5fb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/334190138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560195e65f60 0x560195ee2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f032aff5700 0 client.16956 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:42:30.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/334190138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560195e5ff20 0x560195e6cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f0357ea7b80 1 --2- 172.21.15.62:0/334190138 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560195e65640 0x560195e6d3e0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f0357ea7b80 1 -- 172.21.15.62:0/334190138 >> 172.21.15.62:0/334190138 conn(0x560195d8acc0 msgr2=0x560195e66a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:30.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f0357ea7b80 1 -- 172.21.15.62:0/334190138 shutdown_connections 2024-06-10T00:42:30.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f0357ea7b80 1 -- 172.21.15.62:0/334190138 wait complete. 2024-06-10T00:42:30.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.617+0000 7f0357ea7b80 20 client.16956 trim_cache size 0 max 16384 2024-06-10T00:42:30.628 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:42:30.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.618+0000 7f034d624700 1 -- 172.21.15.62:0/2895583476 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f03300a2d90 msgr2=0x7f03300a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:30.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.618+0000 7f034d624700 1 --2- 172.21.15.62:0/2895583476 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f03300a2d90 0x7f03300a5250 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0340007740 tx=0x7f0340007690 comp rx=0 tx=0).stop 2024-06-10T00:42:30.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.618+0000 7f034d624700 1 -- 172.21.15.62:0/2895583476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0348101880 msgr2=0x7f03481025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:30.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.618+0000 7f034d624700 1 --2- 172.21.15.62:0/2895583476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0348101880 0x7f03481025e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f033c000c00 tx=0x7f033c003680 comp rx=0 tx=0).stop 2024-06-10T00:42:30.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.618+0000 7f034d624700 1 -- 172.21.15.62:0/2895583476 shutdown_connections 2024-06-10T00:42:30.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.618+0000 7f034d624700 1 --2- 172.21.15.62:0/2895583476 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f03300a2d90 0x7f03300a5250 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.618+0000 7f034d624700 1 --2- 172.21.15.62:0/2895583476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0348108670 0x7f0348102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.618+0000 7f034d624700 1 --2- 172.21.15.62:0/2895583476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0348100ec0 0x7f03481020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.618+0000 7f034d624700 1 --2- 172.21.15.62:0/2895583476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0348101880 0x7f03481025e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:30.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.618+0000 7f034d624700 1 -- 172.21.15.62:0/2895583476 >> 172.21.15.62:0/2895583476 conn(0x7f0348076600 msgr2=0x7f03480ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:30.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.618+0000 7f034d624700 1 -- 172.21.15.62:0/2895583476 shutdown_connections 2024-06-10T00:42:30.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:30.618+0000 7f034d624700 1 -- 172.21.15.62:0/2895583476 wait complete. 2024-06-10T00:42:31.342 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T00:42:31.595 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:42:31.827 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:31 smithi062 ceph-mon[32390]: pgmap v299: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-10T00:42:31.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:31 smithi138 ceph-mon[36549]: pgmap v299: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-10T00:42:32.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:31 smithi105 ceph-mon[36321]: pgmap v299: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-10T00:42:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.227+0000 7f424a053700 1 -- 172.21.15.62:0/1839925875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f424406a170 msgr2=0x7f424406a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:33.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.227+0000 7f424a053700 1 --2- 172.21.15.62:0/1839925875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f424406a170 0x7f424406a5f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f422c009990 tx=0x7f422c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.227+0000 7f424a053700 1 -- 172.21.15.62:0/1839925875 shutdown_connections 2024-06-10T00:42:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.227+0000 7f424a053700 1 --2- 172.21.15.62:0/1839925875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f424406ab30 0x7f424410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.227+0000 7f424a053700 1 --2- 172.21.15.62:0/1839925875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4244108500 0x7f42441088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.227+0000 7f424a053700 1 --2- 172.21.15.62:0/1839925875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f424406a170 0x7f424406a5f0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.227+0000 7f424a053700 1 -- 172.21.15.62:0/1839925875 >> 172.21.15.62:0/1839925875 conn(0x7f4244078340 msgr2=0x7f4244078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.227+0000 7f424a053700 1 -- 172.21.15.62:0/1839925875 shutdown_connections 2024-06-10T00:42:33.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.227+0000 7f424a053700 1 -- 172.21.15.62:0/1839925875 wait complete. 2024-06-10T00:42:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.228+0000 7f424a053700 1 Processor -- start 2024-06-10T00:42:33.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.228+0000 7f424a053700 1 -- start start 2024-06-10T00:42:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.228+0000 7f424a053700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f424406a170 0x7f4244101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.228+0000 7f424a053700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f424406ab30 0x7f42441019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.228+0000 7f424a053700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4244108500 0x7f4244101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.228+0000 7f424a053700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4244112780 con 0x7f4244108500 2024-06-10T00:42:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.228+0000 7f424a053700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4244112600 con 0x7f424406a170 2024-06-10T00:42:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.228+0000 7f424a053700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4244112900 con 0x7f424406ab30 2024-06-10T00:42:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f4243fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4244108500 0x7f4244101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f4243fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4244108500 0x7f4244101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49578/0 (socket says 172.21.15.62:49578) 2024-06-10T00:42:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f4243fff700 1 -- 172.21.15.62:0/87826856 learned_addr learned my addr 172.21.15.62:0/87826856 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:33.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f4242ffd700 1 --2- 172.21.15.62:0/87826856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f424406ab30 0x7f42441019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f42437fe700 1 --2- 172.21.15.62:0/87826856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f424406a170 0x7f4244101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f4243fff700 1 -- 172.21.15.62:0/87826856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f424406ab30 msgr2=0x7f42441019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f4243fff700 1 --2- 172.21.15.62:0/87826856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f424406ab30 0x7f42441019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f4243fff700 1 -- 172.21.15.62:0/87826856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f424406a170 msgr2=0x7f4244101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f4243fff700 1 --2- 172.21.15.62:0/87826856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f424406a170 0x7f4244101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f4243fff700 1 -- 172.21.15.62:0/87826856 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42441a71c0 con 0x7f4244108500 2024-06-10T00:42:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f4243fff700 1 --2- 172.21.15.62:0/87826856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4244108500 0x7f4244101ee0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f423c00d980 tx=0x7f423c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:33.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f4240ff9700 1 -- 172.21.15.62:0/87826856 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c017070 con 0x7f4244108500 2024-06-10T00:42:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f424a053700 1 -- 172.21.15.62:0/87826856 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42441a74b0 con 0x7f4244108500 2024-06-10T00:42:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f4240ff9700 1 -- 172.21.15.62:0/87826856 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f423c009e20 con 0x7f4244108500 2024-06-10T00:42:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f4240ff9700 1 -- 172.21.15.62:0/87826856 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c005440 con 0x7f4244108500 2024-06-10T00:42:33.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.229+0000 7f424a053700 1 -- 172.21.15.62:0/87826856 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42441a79a0 con 0x7f4244108500 2024-06-10T00:42:33.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.232+0000 7f4240ff9700 1 -- 172.21.15.62:0/87826856 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f423c0055e0 con 0x7f4244108500 2024-06-10T00:42:33.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.232+0000 7f4240ff9700 1 --2- 172.21.15.62:0/87826856 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f42300a2d90 0x7f42300a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.232+0000 7f42437fe700 1 --2- 172.21.15.62:0/87826856 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f42300a2d90 0x7f42300a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.232+0000 7f4240ff9700 1 -- 172.21.15.62:0/87826856 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f423c0cf820 con 0x7f4244108500 2024-06-10T00:42:33.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.233+0000 7f42437fe700 1 --2- 172.21.15.62:0/87826856 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f42300a2d90 0x7f42300a5250 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4234007700 tx=0x7f4234007690 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:33.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.233+0000 7f42548d6b80 1 Processor -- start 2024-06-10T00:42:33.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.233+0000 7f42548d6b80 1 -- start start 2024-06-10T00:42:33.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.233+0000 7f42548d6b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464640 0x558f08464a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.233+0000 7f42548d6b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464f60 0x558f0845e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.233+0000 7f42548d6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f0845ef20 0x558f0845f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.233+0000 7f42548d6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558f083b0810 con 0x558f0845ef20 2024-06-10T00:42:33.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.233+0000 7f42548d6b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558f08358720 con 0x558f08464640 2024-06-10T00:42:33.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.233+0000 7f42548d6b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558f083ac7d0 con 0x558f08464f60 2024-06-10T00:42:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.233+0000 7f4243fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f0845ef20 0x558f0845f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.234+0000 7f4243fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f0845ef20 0x558f0845f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49582/0 (socket says 172.21.15.62:49582) 2024-06-10T00:42:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.234+0000 7f4243fff700 1 -- 172.21.15.62:0/3573063288 learned_addr learned my addr 172.21.15.62:0/3573063288 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.234+0000 7f4242ffd700 1 --2- 172.21.15.62:0/3573063288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464f60 0x558f0845e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.234+0000 7f42437fe700 1 --2- 172.21.15.62:0/3573063288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464640 0x558f08464a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.234+0000 7f4243fff700 1 -- 172.21.15.62:0/3573063288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464f60 msgr2=0x558f0845e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.234+0000 7f4243fff700 1 --2- 172.21.15.62:0/3573063288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464f60 0x558f0845e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.234+0000 7f4243fff700 1 -- 172.21.15.62:0/3573063288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464640 msgr2=0x558f08464a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:33.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.234+0000 7f4243fff700 1 --2- 172.21.15.62:0/3573063288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464640 0x558f08464a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.234+0000 7f4243fff700 1 -- 172.21.15.62:0/3573063288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f0834ef50 con 0x558f0845ef20 2024-06-10T00:42:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.234+0000 7f42437fe700 1 --2- 172.21.15.62:0/3573063288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464640 0x558f08464a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:42:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.234+0000 7f4243fff700 1 --2- 172.21.15.62:0/3573063288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f0845ef20 0x558f0845f340 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f423c0a4070 tx=0x7f423c09b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc05b80886d6fe22 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.234+0000 7f4239ffb700 1 -- 172.21.15.62:0/3573063288 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c004ae0 con 0x558f0845ef20 2024-06-10T00:42:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.234+0000 7f4239ffb700 1 -- 172.21.15.62:0/3573063288 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f423c0afe50 con 0x558f0845ef20 2024-06-10T00:42:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.234+0000 7f4239ffb700 1 -- 172.21.15.62:0/3573063288 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c0860c0 con 0x558f0845ef20 2024-06-10T00:42:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.235+0000 7f42548d6b80 1 -- 172.21.15.62:0/3573063288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f0845ef20 msgr2=0x558f0845f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.235+0000 7f42548d6b80 1 --2- 172.21.15.62:0/3573063288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f0845ef20 0x558f0845f340 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f423c0a4070 tx=0x7f423c09b930 comp rx=0 tx=0).stop 2024-06-10T00:42:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.235+0000 7f42548d6b80 1 -- 172.21.15.62:0/3573063288 shutdown_connections 2024-06-10T00:42:33.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.235+0000 7f42548d6b80 1 --2- 172.21.15.62:0/3573063288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f0845ef20 0x558f0845f340 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.235+0000 7f42548d6b80 1 --2- 172.21.15.62:0/3573063288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464640 0x558f08464a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.235+0000 7f42548d6b80 1 --2- 172.21.15.62:0/3573063288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464f60 0x558f0845e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.235+0000 7f42548d6b80 1 -- 172.21.15.62:0/3573063288 >> 172.21.15.62:0/3573063288 conn(0x558f08389cc0 msgr2=0x558f08459850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.235+0000 7f42548d6b80 1 -- 172.21.15.62:0/3573063288 shutdown_connections 2024-06-10T00:42:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.235+0000 7f42548d6b80 1 -- 172.21.15.62:0/3573063288 wait complete. 2024-06-10T00:42:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.235+0000 7f42548d6b80 1 -- start start 2024-06-10T00:42:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f42548d6b80 1 Processor -- start 2024-06-10T00:42:33.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f42548d6b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f0845ef20 0x558f0846ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f42548d6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f08464640 0x558f08468080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f42548d6b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464f60 0x558f084685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f42548d6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558f0846b970 con 0x558f08464640 2024-06-10T00:42:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f42548d6b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558f08350bf0 con 0x558f0845ef20 2024-06-10T00:42:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f42548d6b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558f0846baf0 con 0x558f08464f60 2024-06-10T00:42:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f42548d6b80 10 client.0 _ll_register_callbacks cb 0x558f0834a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f4242ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f08464640 0x558f08468080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f4242ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f08464640 0x558f08468080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49590/0 (socket says 172.21.15.62:49590) 2024-06-10T00:42:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f4242ffd700 1 -- 172.21.15.62:0/833998183 learned_addr learned my addr 172.21.15.62:0/833998183 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f42437fe700 1 --2- 172.21.15.62:0/833998183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f0845ef20 0x558f0846ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f4243fff700 1 --2- 172.21.15.62:0/833998183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464f60 0x558f084685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f4242ffd700 1 -- 172.21.15.62:0/833998183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464f60 msgr2=0x558f084685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f4242ffd700 1 --2- 172.21.15.62:0/833998183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464f60 0x558f084685c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f4242ffd700 1 -- 172.21.15.62:0/833998183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f0845ef20 msgr2=0x558f0846ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f4242ffd700 1 --2- 172.21.15.62:0/833998183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f0845ef20 0x558f0846ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.236+0000 7f4242ffd700 1 -- 172.21.15.62:0/833998183 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f082ff8d0 con 0x558f08464640 2024-06-10T00:42:33.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.237+0000 7f4242ffd700 1 --2- 172.21.15.62:0/833998183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f08464640 0x558f08468080 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f422c009990 tx=0x7f422c009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebacadedfceed4ad server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.237+0000 7f4222ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.237+0000 7f4222ffd700 1 -- 172.21.15.62:0/833998183 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c028060 con 0x558f08464640 2024-06-10T00:42:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.237+0000 7f42548d6b80 1 -- 172.21.15.62:0/833998183 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f423c0afe50 con 0x558f08464640 2024-06-10T00:42:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.237+0000 7f4222ffd700 1 -- 172.21.15.62:0/833998183 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f422c0052c0 con 0x558f08464640 2024-06-10T00:42:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.237+0000 7f4222ffd700 1 -- 172.21.15.62:0/833998183 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f422c02f050 con 0x558f08464640 2024-06-10T00:42:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.237+0000 7f42548d6b80 10 client.16968 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.237+0000 7f42548d6b80 1 -- 172.21.15.62:0/833998183 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558f084762f0 con 0x558f08464640 2024-06-10T00:42:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.237+0000 7f4222ffd700 1 -- 172.21.15.62:0/833998183 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f423c0afe50 con 0x558f08464640 2024-06-10T00:42:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.237+0000 7f4222ffd700 1 -- 172.21.15.62:0/833998183 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f422c00f620 con 0x558f08464640 2024-06-10T00:42:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.238+0000 7f42548d6b80 10 client.16968 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.238+0000 7f42548d6b80 10 client.16968 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:42:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.238+0000 7f42548d6b80 20 client.16968 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.238+0000 7f42548d6b80 1 --2- 172.21.15.62:0/833998183 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558f08478880 0x558f0847ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.238+0000 7f42548d6b80 4 client.16968 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:42:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.238+0000 7f42548d6b80 1 -- 172.21.15.62:0/833998183 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f083bdd10 con 0x558f08478880 2024-06-10T00:42:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.238+0000 7f42437fe700 1 --2- 172.21.15.62:0/833998183 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558f08478880 0x558f0847ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.238+0000 7f42437fe700 1 --2- 172.21.15.62:0/833998183 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558f08478880 0x558f0847ac70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8779a0ede684a54e server_cookie=356ca50d09de16db in_seq=0 out_seq=0 2024-06-10T00:42:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.238+0000 7f4222ffd700 10 client.16968 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:42:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.239+0000 7f4222ffd700 1 -- 172.21.15.62:0/833998183 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558f083bdd10 con 0x558f08478880 2024-06-10T00:42:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.239+0000 7f4222ffd700 10 client.16968 handle_command_reply: tid=0 2024-06-10T00:42:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.239+0000 7f42548d6b80 1 client.16968 shutdown 2024-06-10T00:42:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.239+0000 7f42548d6b80 1 -- 172.21.15.62:0/833998183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f08464640 msgr2=0x558f08468080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.239+0000 7f42548d6b80 1 --2- 172.21.15.62:0/833998183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f08464640 0x558f08468080 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f422c009990 tx=0x7f422c009ee0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.239+0000 7f42548d6b80 1 -- 172.21.15.62:0/833998183 shutdown_connections 2024-06-10T00:42:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.239+0000 7f42548d6b80 1 --2- 172.21.15.62:0/833998183 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558f08478880 0x558f0847ac70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.239+0000 7f42548d6b80 1 --2- 172.21.15.62:0/833998183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464f60 0x558f084685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.239+0000 7f42548d6b80 1 --2- 172.21.15.62:0/833998183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f0845ef20 0x558f0846ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.239+0000 7f42548d6b80 1 --2- 172.21.15.62:0/833998183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f08464640 0x558f08468080 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.239+0000 7f42548d6b80 1 -- 172.21.15.62:0/833998183 >> 172.21.15.62:0/833998183 conn(0x558f08389cc0 msgr2=0x558f0845d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.239+0000 7f4222ffd700 0 client.16968 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:42:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.240+0000 7f42548d6b80 1 -- 172.21.15.62:0/833998183 shutdown_connections 2024-06-10T00:42:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.240+0000 7f42548d6b80 1 -- 172.21.15.62:0/833998183 wait complete. 2024-06-10T00:42:33.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.240+0000 7f42548d6b80 20 client.16968 trim_cache size 0 max 16384 2024-06-10T00:42:33.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.256+0000 7f42548d6b80 1 Processor -- start 2024-06-10T00:42:33.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.256+0000 7f42548d6b80 1 -- start start 2024-06-10T00:42:33.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.256+0000 7f42548d6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f0845ef20 0x558f0845e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.256+0000 7f42548d6b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464640 0x558f08474c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.256+0000 7f42548d6b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464f60 0x558f08465cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.256+0000 7f42548d6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f42441b0930 con 0x558f0845ef20 2024-06-10T00:42:33.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.256+0000 7f42548d6b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f42441b2470 con 0x558f08464f60 2024-06-10T00:42:33.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.256+0000 7f42548d6b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42441b1000 con 0x558f08464640 2024-06-10T00:42:33.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f42437fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f0845ef20 0x558f0845e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f42437fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f0845ef20 0x558f0845e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49606/0 (socket says 172.21.15.62:49606) 2024-06-10T00:42:33.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f42437fe700 1 -- 172.21.15.62:0/43340444 learned_addr learned my addr 172.21.15.62:0/43340444 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:33.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f4243fff700 1 --2- 172.21.15.62:0/43340444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464f60 0x558f08465cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f4242ffd700 1 --2- 172.21.15.62:0/43340444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464640 0x558f08474c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f42437fe700 1 -- 172.21.15.62:0/43340444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464640 msgr2=0x558f08474c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:33.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f42437fe700 1 --2- 172.21.15.62:0/43340444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464640 0x558f08474c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f42437fe700 1 -- 172.21.15.62:0/43340444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464f60 msgr2=0x558f08465cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:33.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f42437fe700 1 --2- 172.21.15.62:0/43340444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464f60 0x558f08465cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f42437fe700 1 -- 172.21.15.62:0/43340444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f0845a500 con 0x558f0845ef20 2024-06-10T00:42:33.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f4242ffd700 1 --2- 172.21.15.62:0/43340444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464640 0x558f08474c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:42:33.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f42437fe700 1 --2- 172.21.15.62:0/43340444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f0845ef20 0x558f0845e9e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f42340060b0 tx=0x7f4234011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a655cf67fdf2e3df server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:33.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f42397fa700 1 -- 172.21.15.62:0/43340444 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234032070 con 0x558f0845ef20 2024-06-10T00:42:33.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f42397fa700 1 -- 172.21.15.62:0/43340444 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f423402d450 con 0x558f0845ef20 2024-06-10T00:42:33.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.257+0000 7f42397fa700 1 -- 172.21.15.62:0/43340444 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234024650 con 0x558f0845ef20 2024-06-10T00:42:33.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.258+0000 7f42548d6b80 1 -- 172.21.15.62:0/43340444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f0845ef20 msgr2=0x558f0845e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:33.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.258+0000 7f42548d6b80 1 --2- 172.21.15.62:0/43340444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f0845ef20 0x558f0845e9e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f42340060b0 tx=0x7f4234011000 comp rx=0 tx=0).stop 2024-06-10T00:42:33.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.258+0000 7f42548d6b80 1 -- 172.21.15.62:0/43340444 shutdown_connections 2024-06-10T00:42:33.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.258+0000 7f42548d6b80 1 --2- 172.21.15.62:0/43340444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f0845ef20 0x558f0845e9e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.258+0000 7f42548d6b80 1 --2- 172.21.15.62:0/43340444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464f60 0x558f08465cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.258+0000 7f42548d6b80 1 --2- 172.21.15.62:0/43340444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f08464640 0x558f08474c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.258+0000 7f42548d6b80 1 -- 172.21.15.62:0/43340444 >> 172.21.15.62:0/43340444 conn(0x558f08389cc0 msgr2=0x558f08452e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:33.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.258+0000 7f42548d6b80 1 -- 172.21.15.62:0/43340444 shutdown_connections 2024-06-10T00:42:33.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.258+0000 7f42548d6b80 1 -- 172.21.15.62:0/43340444 wait complete. 2024-06-10T00:42:33.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.259+0000 7f42548d6b80 1 -- start start 2024-06-10T00:42:33.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.259+0000 7f42548d6b80 1 Processor -- start 2024-06-10T00:42:33.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.259+0000 7f42548d6b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f0845ef20 0x558f0846c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.259+0000 7f42548d6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f08464640 0x558f0846c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.259+0000 7f42548d6b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464f60 0x558f084e14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.259+0000 7f42548d6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f42441b1da0 con 0x558f08464640 2024-06-10T00:42:33.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.259+0000 7f42548d6b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f42441b16d0 con 0x558f08464f60 2024-06-10T00:42:33.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.259+0000 7f42548d6b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42441ae8f0 con 0x558f0845ef20 2024-06-10T00:42:33.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.259+0000 7f4242ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f08464640 0x558f0846c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f42548d6b80 10 client.0 _ll_register_callbacks cb 0x558f0834a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:33.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f4242ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f08464640 0x558f0846c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49610/0 (socket says 172.21.15.62:49610) 2024-06-10T00:42:33.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f4242ffd700 1 -- 172.21.15.62:0/2855780893 learned_addr learned my addr 172.21.15.62:0/2855780893 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:33.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f42437fe700 1 --2- 172.21.15.62:0/2855780893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f0845ef20 0x558f0846c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f4243fff700 1 --2- 172.21.15.62:0/2855780893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464f60 0x558f084e14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f4242ffd700 1 -- 172.21.15.62:0/2855780893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f0845ef20 msgr2=0x558f0846c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:33.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f4242ffd700 1 --2- 172.21.15.62:0/2855780893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f0845ef20 0x558f0846c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f4242ffd700 1 -- 172.21.15.62:0/2855780893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464f60 msgr2=0x558f084e14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:33.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f4242ffd700 1 --2- 172.21.15.62:0/2855780893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464f60 0x558f084e14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f4242ffd700 1 -- 172.21.15.62:0/2855780893 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f084e1a50 con 0x558f08464640 2024-06-10T00:42:33.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f4243fff700 1 --2- 172.21.15.62:0/2855780893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464f60 0x558f084e14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:33.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f4242ffd700 1 --2- 172.21.15.62:0/2855780893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f08464640 0x558f0846c840 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f422c006010 tx=0x7f422c009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f41614d9db17cac server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:33.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f4222ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:33.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f4222ffd700 1 -- 172.21.15.62:0/2855780893 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c028070 con 0x558f08464640 2024-06-10T00:42:33.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f42548d6b80 1 -- 172.21.15.62:0/2855780893 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f423402d450 con 0x558f08464640 2024-06-10T00:42:33.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.260+0000 7f4222ffd700 1 -- 172.21.15.62:0/2855780893 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f422c0050b0 con 0x558f08464640 2024-06-10T00:42:33.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.261+0000 7f4222ffd700 1 -- 172.21.15.62:0/2855780893 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f422c005c00 con 0x558f08464640 2024-06-10T00:42:33.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.261+0000 7f42548d6b80 10 client.16974 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:33.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.261+0000 7f42548d6b80 1 -- 172.21.15.62:0/2855780893 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558f084e2cd0 con 0x558f08464640 2024-06-10T00:42:33.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.261+0000 7f4222ffd700 1 -- 172.21.15.62:0/2855780893 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f423402d450 con 0x558f08464640 2024-06-10T00:42:33.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.261+0000 7f4222ffd700 1 -- 172.21.15.62:0/2855780893 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f422c032070 con 0x558f08464640 2024-06-10T00:42:33.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.261+0000 7f42548d6b80 10 client.16974 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:33.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.261+0000 7f42548d6b80 10 client.16974 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:42:33.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.261+0000 7f42548d6b80 20 client.16974 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:33.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.261+0000 7f42548d6b80 1 --2- 172.21.15.62:0/2855780893 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558f084e3290 0x558f084e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:33.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.261+0000 7f42548d6b80 4 client.16974 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:42:33.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.261+0000 7f42548d6b80 1 -- 172.21.15.62:0/2855780893 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f083b8290 con 0x558f084e3290 2024-06-10T00:42:33.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.262+0000 7f42437fe700 1 --2- 172.21.15.62:0/2855780893 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558f084e3290 0x558f084e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:33.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.262+0000 7f42437fe700 1 --2- 172.21.15.62:0/2855780893 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558f084e3290 0x558f084e56b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=169fb212c2498b5a server_cookie=2d5d761ff0773715 in_seq=0 out_seq=0 2024-06-10T00:42:33.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.262+0000 7f4222ffd700 10 client.16974 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:42:33.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.262+0000 7f4222ffd700 1 -- 172.21.15.62:0/2855780893 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f083b8290 con 0x558f084e3290 2024-06-10T00:42:33.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.262+0000 7f4222ffd700 10 client.16974 handle_command_reply: tid=0 2024-06-10T00:42:33.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.262+0000 7f42548d6b80 1 client.16974 shutdown 2024-06-10T00:42:33.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.263+0000 7f42548d6b80 1 -- 172.21.15.62:0/2855780893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f08464640 msgr2=0x558f0846c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:33.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.263+0000 7f42548d6b80 1 --2- 172.21.15.62:0/2855780893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f08464640 0x558f0846c840 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f422c006010 tx=0x7f422c009290 comp rx=0 tx=0).stop 2024-06-10T00:42:33.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.263+0000 7f42548d6b80 1 -- 172.21.15.62:0/2855780893 shutdown_connections 2024-06-10T00:42:33.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.263+0000 7f42548d6b80 1 --2- 172.21.15.62:0/2855780893 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x558f084e3290 0x558f084e56b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.263+0000 7f42548d6b80 1 --2- 172.21.15.62:0/2855780893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f08464f60 0x558f084e14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.263+0000 7f42548d6b80 1 --2- 172.21.15.62:0/2855780893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f0845ef20 0x558f0846c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.263+0000 7f42548d6b80 1 --2- 172.21.15.62:0/2855780893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558f08464640 0x558f0846c840 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.263+0000 7f42548d6b80 1 -- 172.21.15.62:0/2855780893 >> 172.21.15.62:0/2855780893 conn(0x558f08389cc0 msgr2=0x558f08465ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:33.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.263+0000 7f4222ffd700 0 client.16974 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:42:33.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.263+0000 7f42548d6b80 1 -- 172.21.15.62:0/2855780893 shutdown_connections 2024-06-10T00:42:33.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.263+0000 7f42548d6b80 1 -- 172.21.15.62:0/2855780893 wait complete. 2024-06-10T00:42:33.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.263+0000 7f42548d6b80 20 client.16974 trim_cache size 0 max 16384 2024-06-10T00:42:33.274 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:42:33.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.264+0000 7f424a053700 1 -- 172.21.15.62:0/87826856 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f42300a2d90 msgr2=0x7f42300a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:33.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.264+0000 7f424a053700 1 --2- 172.21.15.62:0/87826856 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f42300a2d90 0x7f42300a5250 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4234007700 tx=0x7f4234007690 comp rx=0 tx=0).stop 2024-06-10T00:42:33.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.264+0000 7f424a053700 1 -- 172.21.15.62:0/87826856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4244108500 msgr2=0x7f4244101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:33.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.264+0000 7f424a053700 1 --2- 172.21.15.62:0/87826856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4244108500 0x7f4244101ee0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f423c00d980 tx=0x7f423c00dd40 comp rx=0 tx=0).stop 2024-06-10T00:42:33.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.264+0000 7f424a053700 1 -- 172.21.15.62:0/87826856 shutdown_connections 2024-06-10T00:42:33.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.264+0000 7f424a053700 1 --2- 172.21.15.62:0/87826856 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f42300a2d90 0x7f42300a5250 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.264+0000 7f424a053700 1 --2- 172.21.15.62:0/87826856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4244108500 0x7f4244101ee0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.264+0000 7f424a053700 1 --2- 172.21.15.62:0/87826856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f424406a170 0x7f4244101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.264+0000 7f424a053700 1 --2- 172.21.15.62:0/87826856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f424406ab30 0x7f42441019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:33.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.264+0000 7f424a053700 1 -- 172.21.15.62:0/87826856 >> 172.21.15.62:0/87826856 conn(0x7f4244078340 msgr2=0x7f42441000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:33.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.264+0000 7f424a053700 1 -- 172.21.15.62:0/87826856 shutdown_connections 2024-06-10T00:42:33.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:33.264+0000 7f424a053700 1 -- 172.21.15.62:0/87826856 wait complete. 2024-06-10T00:42:33.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:33 smithi062 ceph-mon[32390]: pgmap v300: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T00:42:33.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:33 smithi138 ceph-mon[36549]: pgmap v300: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T00:42:33.987 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T00:42:34.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:33 smithi105 ceph-mon[36321]: pgmap v300: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T00:42:34.239 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:42:35.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:35 smithi062 ceph-mon[32390]: pgmap v301: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:42:35.833 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:35 smithi105 ceph-mon[36321]: pgmap v301: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:42:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:35 smithi138 ceph-mon[36549]: pgmap v301: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:42:35.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.941+0000 7ff3b086f700 1 -- 172.21.15.62:0/2071922887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3ac0746c0 msgr2=0x7ff3ac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:35.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.941+0000 7ff3b086f700 1 --2- 172.21.15.62:0/2071922887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3ac0746c0 0x7ff3ac074aa0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009990 tx=0x7ff39c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.941+0000 7ff3b086f700 1 -- 172.21.15.62:0/2071922887 shutdown_connections 2024-06-10T00:42:35.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.941+0000 7ff3b086f700 1 --2- 172.21.15.62:0/2071922887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3ac0746c0 0x7ff3ac074aa0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.941+0000 7ff3b086f700 1 --2- 172.21.15.62:0/2071922887 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3ac113690 0x7ff3ac115a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.941+0000 7ff3b086f700 1 --2- 172.21.15.62:0/2071922887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ac074fe0 0x7ff3ac113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.941+0000 7ff3b086f700 1 -- 172.21.15.62:0/2071922887 >> 172.21.15.62:0/2071922887 conn(0x7ff3ac079600 msgr2=0x7ff3ac079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:35.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.941+0000 7ff3b086f700 1 -- 172.21.15.62:0/2071922887 shutdown_connections 2024-06-10T00:42:35.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.941+0000 7ff3b086f700 1 -- 172.21.15.62:0/2071922887 wait complete. 2024-06-10T00:42:35.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.942+0000 7ff3b086f700 1 Processor -- start 2024-06-10T00:42:35.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.942+0000 7ff3b086f700 1 -- start start 2024-06-10T00:42:35.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.942+0000 7ff3b086f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ac0746c0 0x7ff3ac19f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.942+0000 7ff3b086f700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3ac074fe0 0x7ff3ac19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.942+0000 7ff3b086f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3ac113690 0x7ff3ac1a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.942+0000 7ff3b086f700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac118dd0 con 0x7ff3ac074fe0 2024-06-10T00:42:35.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.942+0000 7ff3b086f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac118c50 con 0x7ff3ac113690 2024-06-10T00:42:35.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.942+0000 7ff3b086f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac118f50 con 0x7ff3ac0746c0 2024-06-10T00:42:35.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.942+0000 7ff3a9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3ac074fe0 0x7ff3ac19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.942+0000 7ff3a9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3ac074fe0 0x7ff3ac19f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49640/0 (socket says 172.21.15.62:49640) 2024-06-10T00:42:35.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.942+0000 7ff3a9d9b700 1 -- 172.21.15.62:0/2755736662 learned_addr learned my addr 172.21.15.62:0/2755736662 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:35.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.943+0000 7ff3aa59c700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ac0746c0 0x7ff3ac19f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.943+0000 7ff3aad9d700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3ac113690 0x7ff3ac1a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.943+0000 7ff3a9d9b700 1 -- 172.21.15.62:0/2755736662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ac0746c0 msgr2=0x7ff3ac19f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:35.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.943+0000 7ff3a9d9b700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ac0746c0 0x7ff3ac19f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.943+0000 7ff3a9d9b700 1 -- 172.21.15.62:0/2755736662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3ac113690 msgr2=0x7ff3ac1a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:35.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.943+0000 7ff3a9d9b700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3ac113690 0x7ff3ac1a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.943+0000 7ff3a9d9b700 1 -- 172.21.15.62:0/2755736662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ac1a4230 con 0x7ff3ac074fe0 2024-06-10T00:42:35.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.943+0000 7ff3aa59c700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ac0746c0 0x7ff3ac19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:42:35.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.943+0000 7ff3a9d9b700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3ac074fe0 0x7ff3ac19f760 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000d980 tx=0x7ff3a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:35.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.943+0000 7ff39b7fe700 1 -- 172.21.15.62:0/2755736662 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0016070 con 0x7ff3ac074fe0 2024-06-10T00:42:35.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.943+0000 7ff39b7fe700 1 -- 172.21.15.62:0/2755736662 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a0004d10 con 0x7ff3ac074fe0 2024-06-10T00:42:35.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.943+0000 7ff3b086f700 1 -- 172.21.15.62:0/2755736662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ac1a4520 con 0x7ff3ac074fe0 2024-06-10T00:42:35.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.943+0000 7ff39b7fe700 1 -- 172.21.15.62:0/2755736662 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0005670 con 0x7ff3ac074fe0 2024-06-10T00:42:35.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.943+0000 7ff3b086f700 1 -- 172.21.15.62:0/2755736662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ac1abda0 con 0x7ff3ac074fe0 2024-06-10T00:42:35.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.946+0000 7ff39b7fe700 1 -- 172.21.15.62:0/2755736662 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff3a0004750 con 0x7ff3ac074fe0 2024-06-10T00:42:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.946+0000 7ff39b7fe700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff3940a2d90 0x7ff3940a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.947+0000 7ff3aa59c700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff3940a2d90 0x7ff3940a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.947+0000 7ff39b7fe700 1 -- 172.21.15.62:0/2755736662 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7ff3a00cf5e0 con 0x7ff3ac074fe0 2024-06-10T00:42:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.947+0000 7ff3aa59c700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff3940a2d90 0x7ff3940a5250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00b5c0 tx=0x7ff39c005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.947+0000 7ff3bb0f2b80 1 Processor -- start 2024-06-10T00:42:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3bb0f2b80 1 -- start start 2024-06-10T00:42:35.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3bb0f2b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7640 0x56527e9e7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3bb0f2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7f60 0x56527e9e19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3bb0f2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e1f20 0x56527e9e2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3bb0f2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56527e933810 con 0x56527e9e7f60 2024-06-10T00:42:35.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3bb0f2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56527e8db720 con 0x56527e9e1f20 2024-06-10T00:42:35.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3bb0f2b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56527e92f7d0 con 0x56527e9e7640 2024-06-10T00:42:35.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3a9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7f60 0x56527e9e19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3a9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7f60 0x56527e9e19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49642/0 (socket says 172.21.15.62:49642) 2024-06-10T00:42:35.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3a9d9b700 1 -- 172.21.15.62:0/2603752827 learned_addr learned my addr 172.21.15.62:0/2603752827 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:35.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3aa59c700 1 --2- 172.21.15.62:0/2603752827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7640 0x56527e9e7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3aad9d700 1 --2- 172.21.15.62:0/2603752827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e1f20 0x56527e9e2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3a9d9b700 1 -- 172.21.15.62:0/2603752827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7640 msgr2=0x56527e9e7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:35.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3a9d9b700 1 --2- 172.21.15.62:0/2603752827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7640 0x56527e9e7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3a9d9b700 1 -- 172.21.15.62:0/2603752827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e1f20 msgr2=0x56527e9e2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:35.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3a9d9b700 1 --2- 172.21.15.62:0/2603752827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e1f20 0x56527e9e2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3a9d9b700 1 -- 172.21.15.62:0/2603752827 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56527e8d1f50 con 0x56527e9e7f60 2024-06-10T00:42:35.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.948+0000 7ff3aad9d700 1 --2- 172.21.15.62:0/2603752827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e1f20 0x56527e9e2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:35.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.949+0000 7ff3a9d9b700 1 --2- 172.21.15.62:0/2603752827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7f60 0x56527e9e19e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00a3e30 tx=0x7ff3a009be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d696fab349f1befe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:35.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.949+0000 7ff398ff9700 1 -- 172.21.15.62:0/2603752827 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00860c0 con 0x56527e9e7f60 2024-06-10T00:42:35.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.949+0000 7ff398ff9700 1 -- 172.21.15.62:0/2603752827 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a00a4360 con 0x56527e9e7f60 2024-06-10T00:42:35.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.949+0000 7ff398ff9700 1 -- 172.21.15.62:0/2603752827 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a009b950 con 0x56527e9e7f60 2024-06-10T00:42:35.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.950+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2603752827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7f60 msgr2=0x56527e9e19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:35.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.950+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2603752827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7f60 0x56527e9e19e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00a3e30 tx=0x7ff3a009be70 comp rx=0 tx=0).stop 2024-06-10T00:42:35.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.950+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2603752827 shutdown_connections 2024-06-10T00:42:35.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.950+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2603752827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e1f20 0x56527e9e2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.950+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2603752827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7640 0x56527e9e7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.950+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2603752827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7f60 0x56527e9e19e0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.950+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2603752827 >> 172.21.15.62:0/2603752827 conn(0x56527e90ccc0 msgr2=0x56527e9dc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:35.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.950+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2603752827 shutdown_connections 2024-06-10T00:42:35.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.950+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2603752827 wait complete. 2024-06-10T00:42:35.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.950+0000 7ff3bb0f2b80 1 -- start start 2024-06-10T00:42:35.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3bb0f2b80 1 Processor -- start 2024-06-10T00:42:35.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3bb0f2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e1f20 0x56527e9edd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3bb0f2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e7640 0x56527e9eb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3bb0f2b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7f60 0x56527e9eb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3bb0f2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56527e9ee9d0 con 0x56527e9e1f20 2024-06-10T00:42:35.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3bb0f2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56527e8d3bf0 con 0x56527e9e7640 2024-06-10T00:42:35.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3bb0f2b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56527e9eeb50 con 0x56527e9e7f60 2024-06-10T00:42:35.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3bb0f2b80 10 client.0 _ll_register_callbacks cb 0x56527e8cd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:35.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3aa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e1f20 0x56527e9edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3aa59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e1f20 0x56527e9edd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49644/0 (socket says 172.21.15.62:49644) 2024-06-10T00:42:35.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3aa59c700 1 -- 172.21.15.62:0/2895478732 learned_addr learned my addr 172.21.15.62:0/2895478732 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:35.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3aad9d700 1 --2- 172.21.15.62:0/2895478732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7f60 0x56527e9eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3a9d9b700 1 --2- 172.21.15.62:0/2895478732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e7640 0x56527e9eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3aa59c700 1 -- 172.21.15.62:0/2895478732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7f60 msgr2=0x56527e9eb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:35.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3aa59c700 1 --2- 172.21.15.62:0/2895478732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7f60 0x56527e9eb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3aa59c700 1 -- 172.21.15.62:0/2895478732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e7640 msgr2=0x56527e9eb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:35.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3aa59c700 1 --2- 172.21.15.62:0/2895478732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e7640 0x56527e9eb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.951+0000 7ff3aa59c700 1 -- 172.21.15.62:0/2895478732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56527e8828d0 con 0x56527e9e1f20 2024-06-10T00:42:35.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.952+0000 7ff3aa59c700 1 --2- 172.21.15.62:0/2895478732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e1f20 0x56527e9edd50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7ff39c000c00 tx=0x7ff39c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9519f23fe0792afc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:35.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.952+0000 7ff391ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:35.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.952+0000 7ff391ffb700 1 -- 172.21.15.62:0/2895478732 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00f660 con 0x56527e9e1f20 2024-06-10T00:42:35.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.952+0000 7ff391ffb700 1 -- 172.21.15.62:0/2895478732 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7ff39c00f800 con 0x56527e9e1f20 2024-06-10T00:42:35.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.952+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2895478732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3a00a4360 con 0x56527e9e1f20 2024-06-10T00:42:35.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.952+0000 7ff391ffb700 1 -- 172.21.15.62:0/2895478732 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff39c03d050 con 0x56527e9e1f20 2024-06-10T00:42:35.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.952+0000 7ff3bb0f2b80 10 client.16986 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:35.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.952+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2895478732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56527e9f9380 con 0x56527e9e1f20 2024-06-10T00:42:35.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.952+0000 7ff391ffb700 1 -- 172.21.15.62:0/2895478732 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a00a4360 con 0x56527e9e1f20 2024-06-10T00:42:35.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.952+0000 7ff391ffb700 1 -- 172.21.15.62:0/2895478732 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7ff39c0249c0 con 0x56527e9e1f20 2024-06-10T00:42:35.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.953+0000 7ff3bb0f2b80 10 client.16986 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:35.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.953+0000 7ff3bb0f2b80 10 client.16986 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:42:35.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.953+0000 7ff3bb0f2b80 20 client.16986 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:35.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.953+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2895478732 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x56527e9fb9c0 0x56527e9fddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.953+0000 7ff3bb0f2b80 4 client.16986 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:42:35.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.953+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2895478732 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56527e940d10 con 0x56527e9fb9c0 2024-06-10T00:42:35.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.953+0000 7ff3a9d9b700 1 --2- 172.21.15.62:0/2895478732 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x56527e9fb9c0 0x56527e9fddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.953+0000 7ff3a9d9b700 1 --2- 172.21.15.62:0/2895478732 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x56527e9fb9c0 0x56527e9fddd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89066436e12374ff server_cookie=42997fe6ccd3370a in_seq=0 out_seq=0 2024-06-10T00:42:35.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.953+0000 7ff391ffb700 10 client.16986 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:42:35.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.954+0000 7ff391ffb700 1 -- 172.21.15.62:0/2895478732 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56527e940d10 con 0x56527e9fb9c0 2024-06-10T00:42:35.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.954+0000 7ff391ffb700 10 client.16986 handle_command_reply: tid=0 2024-06-10T00:42:35.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.954+0000 7ff3bb0f2b80 1 client.16986 shutdown 2024-06-10T00:42:35.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.954+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2895478732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e1f20 msgr2=0x56527e9edd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:35.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.954+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2895478732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e1f20 0x56527e9edd50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7ff39c000c00 tx=0x7ff39c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.954+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2895478732 shutdown_connections 2024-06-10T00:42:35.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.954+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2895478732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e1f20 0x56527e9edd50 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.954+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2895478732 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x56527e9fb9c0 0x56527e9fddd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.954+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2895478732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7f60 0x56527e9eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.954+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2895478732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e7640 0x56527e9eb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.954+0000 7ff391ffb700 0 client.16986 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:42:35.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.954+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2895478732 >> 172.21.15.62:0/2895478732 conn(0x56527e90ccc0 msgr2=0x56527e9e09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:35.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.955+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2895478732 shutdown_connections 2024-06-10T00:42:35.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.955+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2895478732 wait complete. 2024-06-10T00:42:35.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.955+0000 7ff3bb0f2b80 20 client.16986 trim_cache size 0 max 16384 2024-06-10T00:42:35.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.970+0000 7ff3bb0f2b80 1 Processor -- start 2024-06-10T00:42:35.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.970+0000 7ff3bb0f2b80 1 -- start start 2024-06-10T00:42:35.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3bb0f2b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e1f20 0x56527e9f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3bb0f2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e7640 0x56527e9e19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3bb0f2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7f60 0x56527e9fddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3bb0f2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b4df0 con 0x56527e9e7f60 2024-06-10T00:42:35.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3bb0f2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b6930 con 0x56527e9e7640 2024-06-10T00:42:35.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3bb0f2b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b54c0 con 0x56527e9e1f20 2024-06-10T00:42:35.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3aad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7f60 0x56527e9fddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3aa59c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e1f20 0x56527e9f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3aa59c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e1f20 0x56527e9f7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:43562/0 (socket says 172.21.15.62:43562) 2024-06-10T00:42:35.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3aad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7f60 0x56527e9fddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49658/0 (socket says 172.21.15.62:49658) 2024-06-10T00:42:35.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3aa59c700 1 -- 172.21.15.62:0/3485906606 learned_addr learned my addr 172.21.15.62:0/3485906606 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:35.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3a9d9b700 1 --2- 172.21.15.62:0/3485906606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e7640 0x56527e9e19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3aad9d700 1 -- 172.21.15.62:0/3485906606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e1f20 msgr2=0x56527e9f7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:35.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3aad9d700 1 --2- 172.21.15.62:0/3485906606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e1f20 0x56527e9f7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3aad9d700 1 -- 172.21.15.62:0/3485906606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e7640 msgr2=0x56527e9e19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:35.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3aad9d700 1 --2- 172.21.15.62:0/3485906606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e7640 0x56527e9e19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3aad9d700 1 -- 172.21.15.62:0/3485906606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56527e9e95c0 con 0x56527e9e7f60 2024-06-10T00:42:35.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3aa59c700 1 --2- 172.21.15.62:0/3485906606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e1f20 0x56527e9f7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:42:35.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.971+0000 7ff3aad9d700 1 --2- 172.21.15.62:0/3485906606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7f60 0x56527e9fddb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009d30 tx=0x7ff3a4009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b437d26caae843 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:35.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.972+0000 7ff393fff700 1 -- 172.21.15.62:0/3485906606 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4007e00 con 0x56527e9e7f60 2024-06-10T00:42:35.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.972+0000 7ff393fff700 1 -- 172.21.15.62:0/3485906606 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a4004bb0 con 0x56527e9e7f60 2024-06-10T00:42:35.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.972+0000 7ff393fff700 1 -- 172.21.15.62:0/3485906606 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4017670 con 0x56527e9e7f60 2024-06-10T00:42:35.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.972+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/3485906606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7f60 msgr2=0x56527e9fddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:35.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.972+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/3485906606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7f60 0x56527e9fddb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009d30 tx=0x7ff3a4009100 comp rx=0 tx=0).stop 2024-06-10T00:42:35.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.972+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/3485906606 shutdown_connections 2024-06-10T00:42:35.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.972+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/3485906606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7f60 0x56527e9fddb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.972+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/3485906606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e1f20 0x56527e9f7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.972+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/3485906606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e7640 0x56527e9e19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.972+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/3485906606 >> 172.21.15.62:0/3485906606 conn(0x56527e90ccc0 msgr2=0x56527e9d5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:35.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.973+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/3485906606 shutdown_connections 2024-06-10T00:42:35.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.973+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/3485906606 wait complete. 2024-06-10T00:42:35.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.973+0000 7ff3bb0f2b80 1 -- start start 2024-06-10T00:42:35.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.973+0000 7ff3bb0f2b80 1 Processor -- start 2024-06-10T00:42:35.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.973+0000 7ff3bb0f2b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e1f20 0x56527e9f4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.973+0000 7ff3bb0f2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7640 0x56527ea02d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.973+0000 7ff3bb0f2b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7f60 0x56527ea03250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.973+0000 7ff3bb0f2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b6260 con 0x56527e9e7640 2024-06-10T00:42:35.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.973+0000 7ff3bb0f2b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b5b90 con 0x56527e9e1f20 2024-06-10T00:42:35.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.973+0000 7ff3bb0f2b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b2db0 con 0x56527e9e7f60 2024-06-10T00:42:35.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.973+0000 7ff3a9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7640 0x56527ea02d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.973+0000 7ff3bb0f2b80 10 client.0 _ll_register_callbacks cb 0x56527e8cd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:35.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff3a9d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7640 0x56527ea02d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49664/0 (socket says 172.21.15.62:49664) 2024-06-10T00:42:35.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff3a9d9b700 1 -- 172.21.15.62:0/2412874338 learned_addr learned my addr 172.21.15.62:0/2412874338 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:35.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff3aad9d700 1 --2- 172.21.15.62:0/2412874338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7f60 0x56527ea03250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff3aa59c700 1 --2- 172.21.15.62:0/2412874338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e1f20 0x56527e9f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff3a9d9b700 1 -- 172.21.15.62:0/2412874338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7f60 msgr2=0x56527ea03250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:35.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff3a9d9b700 1 --2- 172.21.15.62:0/2412874338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7f60 0x56527ea03250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff3a9d9b700 1 -- 172.21.15.62:0/2412874338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e1f20 msgr2=0x56527e9f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:35.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff3a9d9b700 1 --2- 172.21.15.62:0/2412874338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e1f20 0x56527e9f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff3a9d9b700 1 -- 172.21.15.62:0/2412874338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56527ea03790 con 0x56527e9e7640 2024-06-10T00:42:35.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff3a9d9b700 1 --2- 172.21.15.62:0/2412874338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7640 0x56527ea02d10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00cc9d0 tx=0x7ff3a009fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a85a7772f91c30 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:35.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff391ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:35.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff391ffb700 1 -- 172.21.15.62:0/2412874338 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00a4360 con 0x56527e9e7640 2024-06-10T00:42:35.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff391ffb700 1 -- 172.21.15.62:0/2412874338 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7ff3a0004340 con 0x56527e9e7640 2024-06-10T00:42:35.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2412874338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3a4004bb0 con 0x56527e9e7640 2024-06-10T00:42:35.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff391ffb700 1 -- 172.21.15.62:0/2412874338 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff3a0004a40 con 0x56527e9e7640 2024-06-10T00:42:35.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff3bb0f2b80 10 client.16992 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:35.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.974+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2412874338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56527ea663a0 con 0x56527e9e7640 2024-06-10T00:42:35.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.975+0000 7ff391ffb700 1 -- 172.21.15.62:0/2412874338 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3a4004bb0 con 0x56527e9e7640 2024-06-10T00:42:35.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.975+0000 7ff391ffb700 1 -- 172.21.15.62:0/2412874338 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7ff3a009b9e0 con 0x56527e9e7640 2024-06-10T00:42:35.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.975+0000 7ff3bb0f2b80 10 client.16992 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:35.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.975+0000 7ff3bb0f2b80 10 client.16992 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:42:35.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.975+0000 7ff3bb0f2b80 20 client.16992 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:35.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.975+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2412874338 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x56527ea66630 0x56527ea68a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:35.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.975+0000 7ff3bb0f2b80 4 client.16992 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:42:35.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.975+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2412874338 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56527e93b290 con 0x56527ea66630 2024-06-10T00:42:35.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.975+0000 7ff3aa59c700 1 --2- 172.21.15.62:0/2412874338 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x56527ea66630 0x56527ea68a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:35.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.975+0000 7ff3aa59c700 1 --2- 172.21.15.62:0/2412874338 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x56527ea66630 0x56527ea68a20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84cd68c70000724a server_cookie=ad5e11bdc047d84c in_seq=0 out_seq=0 2024-06-10T00:42:35.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.975+0000 7ff391ffb700 10 client.16992 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:42:35.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff391ffb700 1 -- 172.21.15.62:0/2412874338 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56527e93b290 con 0x56527ea66630 2024-06-10T00:42:35.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff391ffb700 10 client.16992 handle_command_reply: tid=0 2024-06-10T00:42:35.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff3bb0f2b80 1 client.16992 shutdown 2024-06-10T00:42:35.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2412874338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7640 msgr2=0x56527ea02d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:35.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2412874338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7640 0x56527ea02d10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00cc9d0 tx=0x7ff3a009fc20 comp rx=0 tx=0).stop 2024-06-10T00:42:35.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2412874338 shutdown_connections 2024-06-10T00:42:35.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2412874338 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x56527ea66630 0x56527ea68a20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2412874338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527e9e7f60 0x56527ea03250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2412874338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56527e9e1f20 0x56527e9f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff391ffb700 0 client.16992 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:42:35.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff3bb0f2b80 1 --2- 172.21.15.62:0/2412874338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56527e9e7640 0x56527ea02d10 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2412874338 >> 172.21.15.62:0/2412874338 conn(0x56527e90ccc0 msgr2=0x56527e9e8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:35.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2412874338 shutdown_connections 2024-06-10T00:42:35.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff3bb0f2b80 1 -- 172.21.15.62:0/2412874338 wait complete. 2024-06-10T00:42:35.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.976+0000 7ff3bb0f2b80 20 client.16992 trim_cache size 0 max 16384 2024-06-10T00:42:35.994 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:42:35.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.977+0000 7ff3b086f700 1 -- 172.21.15.62:0/2755736662 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff3940a2d90 msgr2=0x7ff3940a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:35.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.977+0000 7ff3b086f700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff3940a2d90 0x7ff3940a5250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00b5c0 tx=0x7ff39c005960 comp rx=0 tx=0).stop 2024-06-10T00:42:35.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.977+0000 7ff3b086f700 1 -- 172.21.15.62:0/2755736662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3ac074fe0 msgr2=0x7ff3ac19f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:35.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.977+0000 7ff3b086f700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3ac074fe0 0x7ff3ac19f760 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000d980 tx=0x7ff3a000dc90 comp rx=0 tx=0).stop 2024-06-10T00:42:35.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.977+0000 7ff3b086f700 1 -- 172.21.15.62:0/2755736662 shutdown_connections 2024-06-10T00:42:35.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.977+0000 7ff3b086f700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7ff3940a2d90 0x7ff3940a5250 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.977+0000 7ff3b086f700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3ac113690 0x7ff3ac1a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.977+0000 7ff3b086f700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ac0746c0 0x7ff3ac19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.977+0000 7ff3b086f700 1 --2- 172.21.15.62:0/2755736662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3ac074fe0 0x7ff3ac19f760 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:35.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.977+0000 7ff3b086f700 1 -- 172.21.15.62:0/2755736662 >> 172.21.15.62:0/2755736662 conn(0x7ff3ac079600 msgr2=0x7ff3ac1124c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:35.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.977+0000 7ff3b086f700 1 -- 172.21.15.62:0/2755736662 shutdown_connections 2024-06-10T00:42:35.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:35.977+0000 7ff3b086f700 1 -- 172.21.15.62:0/2755736662 wait complete. 2024-06-10T00:42:36.719 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T00:42:36.970 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:42:37.808 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:37 smithi062 ceph-mon[32390]: pgmap v302: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:42:37.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:37 smithi138 ceph-mon[36549]: pgmap v302: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:42:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:37 smithi105 ceph-mon[36321]: pgmap v302: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:42:38.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.604+0000 7f080d17a700 1 -- 172.21.15.62:0/612918082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0808107860 msgr2=0x7f080810f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:38.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.604+0000 7f080d17a700 1 --2- 172.21.15.62:0/612918082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0808107860 0x7f080810f3e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f080000b0d0 tx=0x7f080000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.604+0000 7f080d17a700 1 -- 172.21.15.62:0/612918082 shutdown_connections 2024-06-10T00:42:38.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.604+0000 7f080d17a700 1 --2- 172.21.15.62:0/612918082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0808107860 0x7f080810f3e0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.604+0000 7f080d17a700 1 --2- 172.21.15.62:0/612918082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0808102770 0x7f0808102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.604+0000 7f080d17a700 1 --2- 172.21.15.62:0/612918082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0808103120 0x7f0808107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.604+0000 7f080d17a700 1 -- 172.21.15.62:0/612918082 >> 172.21.15.62:0/612918082 conn(0x7f0808079600 msgr2=0x7f0808079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:38.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.604+0000 7f080d17a700 1 -- 172.21.15.62:0/612918082 shutdown_connections 2024-06-10T00:42:38.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.604+0000 7f080d17a700 1 -- 172.21.15.62:0/612918082 wait complete. 2024-06-10T00:42:38.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.605+0000 7f080d17a700 1 Processor -- start 2024-06-10T00:42:38.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.605+0000 7f080d17a700 1 -- start start 2024-06-10T00:42:38.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.605+0000 7f080d17a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0808102770 0x7f0808074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.605+0000 7f080d17a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0808103120 0x7f0808074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.605+0000 7f080d17a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0808107860 0x7f0808075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.605+0000 7f080d17a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f08081127a0 con 0x7f0808102770 2024-06-10T00:42:38.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.605+0000 7f080d17a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0808112620 con 0x7f0808107860 2024-06-10T00:42:38.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.605+0000 7f080d17a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0808112920 con 0x7f0808103120 2024-06-10T00:42:38.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.605+0000 7f0806d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0808102770 0x7f0808074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.605+0000 7f0806d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0808102770 0x7f0808074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43726/0 (socket says 172.21.15.62:43726) 2024-06-10T00:42:38.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.605+0000 7f0806d9d700 1 -- 172.21.15.62:0/3969331636 learned_addr learned my addr 172.21.15.62:0/3969331636 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:38.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.605+0000 7f080659c700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0808103120 0x7f0808074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.605+0000 7f080759e700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0808107860 0x7f0808075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.606+0000 7f0806d9d700 1 -- 172.21.15.62:0/3969331636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0808103120 msgr2=0x7f0808074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:38.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.606+0000 7f0806d9d700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0808103120 0x7f0808074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.606+0000 7f0806d9d700 1 -- 172.21.15.62:0/3969331636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0808107860 msgr2=0x7f0808075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:38.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.606+0000 7f0806d9d700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0808107860 0x7f0808075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.606+0000 7f0806d9d700 1 -- 172.21.15.62:0/3969331636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08081a7080 con 0x7f0808102770 2024-06-10T00:42:38.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.606+0000 7f080659c700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0808103120 0x7f0808074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:38.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.606+0000 7f080759e700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0808107860 0x7f0808075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:38.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.606+0000 7f0806d9d700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0808102770 0x7f0808074790 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f07f800d980 tx=0x7f07f800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:38.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.606+0000 7f07effff700 1 -- 172.21.15.62:0/3969331636 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8016070 con 0x7f0808102770 2024-06-10T00:42:38.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.606+0000 7f080d17a700 1 -- 172.21.15.62:0/3969331636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08081a7370 con 0x7f0808102770 2024-06-10T00:42:38.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.606+0000 7f07effff700 1 -- 172.21.15.62:0/3969331636 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07f8004500 con 0x7f0808102770 2024-06-10T00:42:38.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.606+0000 7f07effff700 1 -- 172.21.15.62:0/3969331636 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8004d10 con 0x7f0808102770 2024-06-10T00:42:38.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.606+0000 7f080d17a700 1 -- 172.21.15.62:0/3969331636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08081a7900 con 0x7f0808102770 2024-06-10T00:42:38.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.608+0000 7f07effff700 1 -- 172.21.15.62:0/3969331636 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f07f800bbf0 con 0x7f0808102770 2024-06-10T00:42:38.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.609+0000 7f07effff700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f07f00a2d90 0x7f07f00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.609+0000 7f080659c700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f07f00a2d90 0x7f07f00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.609+0000 7f07effff700 1 -- 172.21.15.62:0/3969331636 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f07f80cf6b0 con 0x7f0808102770 2024-06-10T00:42:38.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.609+0000 7f080659c700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f07f00a2d90 0x7f07f00a5250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0808108bb0 tx=0x7f07fc009260 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:38.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f08179fdb80 1 Processor -- start 2024-06-10T00:42:38.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f08179fdb80 1 -- start start 2024-06-10T00:42:38.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f08179fdb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ebaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f08179fdb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 0x5650a1eb49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f08179fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1eb5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f08179fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5650a1e06810 con 0x5650a1eb4f20 2024-06-10T00:42:38.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f08179fdb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5650a1dae720 con 0x5650a1ebaf60 2024-06-10T00:42:38.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f08179fdb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5650a1e027d0 con 0x5650a1eba640 2024-06-10T00:42:38.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f080759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1eb5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f080759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1eb5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43740/0 (socket says 172.21.15.62:43740) 2024-06-10T00:42:38.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f080759e700 1 -- 172.21.15.62:0/1950472761 learned_addr learned my addr 172.21.15.62:0/1950472761 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:38.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f0806d9d700 1 --2- 172.21.15.62:0/1950472761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ebaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f080659c700 1 --2- 172.21.15.62:0/1950472761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 0x5650a1eb49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f080759e700 1 -- 172.21.15.62:0/1950472761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 msgr2=0x5650a1ebaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:38.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f080759e700 1 --2- 172.21.15.62:0/1950472761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ebaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f080759e700 1 -- 172.21.15.62:0/1950472761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 msgr2=0x5650a1eb49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:38.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f080759e700 1 --2- 172.21.15.62:0/1950472761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 0x5650a1eb49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.610+0000 7f080759e700 1 -- 172.21.15.62:0/1950472761 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a1da4f50 con 0x5650a1eb4f20 2024-06-10T00:42:38.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.611+0000 7f080659c700 1 --2- 172.21.15.62:0/1950472761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 0x5650a1eb49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:38.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.611+0000 7f080759e700 1 --2- 172.21.15.62:0/1950472761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1eb5340 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f08000092e0 tx=0x7f0800009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d9b108c28841cf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:38.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.611+0000 7f07ed7fa700 1 -- 172.21.15.62:0/1950472761 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800020030 con 0x5650a1eb4f20 2024-06-10T00:42:38.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.611+0000 7f07ed7fa700 1 -- 172.21.15.62:0/1950472761 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0800004a50 con 0x5650a1eb4f20 2024-06-10T00:42:38.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.611+0000 7f07ed7fa700 1 -- 172.21.15.62:0/1950472761 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800007b80 con 0x5650a1eb4f20 2024-06-10T00:42:38.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 -- 172.21.15.62:0/1950472761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 msgr2=0x5650a1eb5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:38.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 --2- 172.21.15.62:0/1950472761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1eb5340 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f08000092e0 tx=0x7f0800009310 comp rx=0 tx=0).stop 2024-06-10T00:42:38.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 -- 172.21.15.62:0/1950472761 shutdown_connections 2024-06-10T00:42:38.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 --2- 172.21.15.62:0/1950472761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1eb5340 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 --2- 172.21.15.62:0/1950472761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ebaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 --2- 172.21.15.62:0/1950472761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 0x5650a1eb49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 -- 172.21.15.62:0/1950472761 >> 172.21.15.62:0/1950472761 conn(0x5650a1ddfcc0 msgr2=0x5650a1eaf850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:38.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 -- 172.21.15.62:0/1950472761 shutdown_connections 2024-06-10T00:42:38.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 -- 172.21.15.62:0/1950472761 wait complete. 2024-06-10T00:42:38.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 -- start start 2024-06-10T00:42:38.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 Processor -- start 2024-06-10T00:42:38.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1ec0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ebdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 0x5650a1ebe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5650a1ec1970 con 0x5650a1eb4f20 2024-06-10T00:42:38.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.612+0000 7f08179fdb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5650a1da6bf0 con 0x5650a1ebaf60 2024-06-10T00:42:38.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f08179fdb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5650a1ec1af0 con 0x5650a1eba640 2024-06-10T00:42:38.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f08179fdb80 10 client.0 _ll_register_callbacks cb 0x5650a1da0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:38.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f0806d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1ec0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f0806d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1ec0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43750/0 (socket says 172.21.15.62:43750) 2024-06-10T00:42:38.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f0806d9d700 1 -- 172.21.15.62:0/542101576 learned_addr learned my addr 172.21.15.62:0/542101576 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:38.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f080759e700 1 --2- 172.21.15.62:0/542101576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 0x5650a1ebe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f080659c700 1 --2- 172.21.15.62:0/542101576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ebdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f0806d9d700 1 -- 172.21.15.62:0/542101576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 msgr2=0x5650a1ebdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:38.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f0806d9d700 1 --2- 172.21.15.62:0/542101576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ebdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f0806d9d700 1 -- 172.21.15.62:0/542101576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 msgr2=0x5650a1ebe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:38.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f0806d9d700 1 --2- 172.21.15.62:0/542101576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 0x5650a1ebe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f0806d9d700 1 -- 172.21.15.62:0/542101576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a1d558d0 con 0x5650a1eb4f20 2024-06-10T00:42:38.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f0806d9d700 1 --2- 172.21.15.62:0/542101576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1ec0cd0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f07f8007790 tx=0x7f07f809b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8dc1ab3d14d42f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:38.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f07eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:38.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f07eaff5700 1 -- 172.21.15.62:0/542101576 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f800f5c0 con 0x5650a1eb4f20 2024-06-10T00:42:38.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f07eaff5700 1 -- 172.21.15.62:0/542101576 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f07f809ba20 con 0x5650a1eb4f20 2024-06-10T00:42:38.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f08179fdb80 1 -- 172.21.15.62:0/542101576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0800004a50 con 0x5650a1eb4f20 2024-06-10T00:42:38.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f07eaff5700 1 -- 172.21.15.62:0/542101576 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f07f809fb70 con 0x5650a1eb4f20 2024-06-10T00:42:38.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.613+0000 7f08179fdb80 10 client.17004 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:38.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.614+0000 7f08179fdb80 1 -- 172.21.15.62:0/542101576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5650a1ecc270 con 0x5650a1eb4f20 2024-06-10T00:42:38.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.614+0000 7f07eaff5700 1 -- 172.21.15.62:0/542101576 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0800004a50 con 0x5650a1eb4f20 2024-06-10T00:42:38.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.614+0000 7f07eaff5700 1 -- 172.21.15.62:0/542101576 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f07f80afce0 con 0x5650a1eb4f20 2024-06-10T00:42:38.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.614+0000 7f08179fdb80 10 client.17004 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:38.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.614+0000 7f08179fdb80 10 client.17004 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:42:38.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.614+0000 7f08179fdb80 20 client.17004 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:38.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.614+0000 7f08179fdb80 1 --2- 172.21.15.62:0/542101576 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5650a1ece8b0 0x5650a1ed0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.614+0000 7f08179fdb80 4 client.17004 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:42:38.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.614+0000 7f08179fdb80 1 -- 172.21.15.62:0/542101576 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650a1e13d10 con 0x5650a1ece8b0 2024-06-10T00:42:38.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.614+0000 7f080659c700 1 --2- 172.21.15.62:0/542101576 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5650a1ece8b0 0x5650a1ed0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.615+0000 7f080659c700 1 --2- 172.21.15.62:0/542101576 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5650a1ece8b0 0x5650a1ed0ca0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aba52adea2b0fd12 server_cookie=13891d57cfa6d921 in_seq=0 out_seq=0 2024-06-10T00:42:38.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.615+0000 7f07eaff5700 10 client.17004 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:42:38.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.615+0000 7f07eaff5700 1 -- 172.21.15.62:0/542101576 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5650a1e13d10 con 0x5650a1ece8b0 2024-06-10T00:42:38.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.615+0000 7f07eaff5700 10 client.17004 handle_command_reply: tid=0 2024-06-10T00:42:38.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.615+0000 7f08179fdb80 1 client.17004 shutdown 2024-06-10T00:42:38.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.616+0000 7f08179fdb80 1 -- 172.21.15.62:0/542101576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 msgr2=0x5650a1ec0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:38.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.616+0000 7f08179fdb80 1 --2- 172.21.15.62:0/542101576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1ec0cd0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f07f8007790 tx=0x7f07f809b3f0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.616+0000 7f08179fdb80 1 -- 172.21.15.62:0/542101576 shutdown_connections 2024-06-10T00:42:38.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.616+0000 7f08179fdb80 1 --2- 172.21.15.62:0/542101576 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5650a1ece8b0 0x5650a1ed0ca0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.616+0000 7f08179fdb80 1 --2- 172.21.15.62:0/542101576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1ec0cd0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.616+0000 7f07eaff5700 0 client.17004 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:42:38.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.616+0000 7f08179fdb80 1 --2- 172.21.15.62:0/542101576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 0x5650a1ebe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.616+0000 7f08179fdb80 1 --2- 172.21.15.62:0/542101576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ebdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.616+0000 7f08179fdb80 1 -- 172.21.15.62:0/542101576 >> 172.21.15.62:0/542101576 conn(0x5650a1ddfcc0 msgr2=0x5650a1eb39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:38.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.616+0000 7f08179fdb80 1 -- 172.21.15.62:0/542101576 shutdown_connections 2024-06-10T00:42:38.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.616+0000 7f08179fdb80 1 -- 172.21.15.62:0/542101576 wait complete. 2024-06-10T00:42:38.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.616+0000 7f08179fdb80 20 client.17004 trim_cache size 0 max 16384 2024-06-10T00:42:38.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.631+0000 7f08179fdb80 1 Processor -- start 2024-06-10T00:42:38.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.631+0000 7f08179fdb80 1 -- start start 2024-06-10T00:42:38.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.631+0000 7f08179fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1eb49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.631+0000 7f08179fdb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ebba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.631+0000 7f08179fdb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 0x5650a1ed6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.631+0000 7f08179fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f08081b08d0 con 0x5650a1eb4f20 2024-06-10T00:42:38.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.631+0000 7f08179fdb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f08081b2410 con 0x5650a1ebaf60 2024-06-10T00:42:38.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.631+0000 7f08179fdb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08081b0fa0 con 0x5650a1eba640 2024-06-10T00:42:38.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.631+0000 7f0806d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1eb49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.631+0000 7f0806d9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1eb49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43760/0 (socket says 172.21.15.62:43760) 2024-06-10T00:42:38.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.631+0000 7f0806d9d700 1 -- 172.21.15.62:0/3188378953 learned_addr learned my addr 172.21.15.62:0/3188378953 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:38.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.632+0000 7f0806d9d700 1 -- 172.21.15.62:0/3188378953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 msgr2=0x5650a1ebba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:38.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.632+0000 7f0806d9d700 1 --2- 172.21.15.62:0/3188378953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ebba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.632+0000 7f0806d9d700 1 -- 172.21.15.62:0/3188378953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 msgr2=0x5650a1ed6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:38.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.632+0000 7f0806d9d700 1 --2- 172.21.15.62:0/3188378953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 0x5650a1ed6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.632+0000 7f0806d9d700 1 -- 172.21.15.62:0/3188378953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a1eb0520 con 0x5650a1eb4f20 2024-06-10T00:42:38.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.632+0000 7f0806d9d700 1 --2- 172.21.15.62:0/3188378953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1eb49e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f07f809b4c0 tx=0x7f07f800df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6882d54d2931687 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:38.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.632+0000 7f07ecff9700 1 -- 172.21.15.62:0/3188378953 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8014040 con 0x5650a1eb4f20 2024-06-10T00:42:38.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.632+0000 7f07ecff9700 1 -- 172.21.15.62:0/3188378953 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07f80ba1a0 con 0x5650a1eb4f20 2024-06-10T00:42:38.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.632+0000 7f07ecff9700 1 -- 172.21.15.62:0/3188378953 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f809bc40 con 0x5650a1eb4f20 2024-06-10T00:42:38.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.633+0000 7f08179fdb80 1 -- 172.21.15.62:0/3188378953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 msgr2=0x5650a1eb49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:38.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.633+0000 7f08179fdb80 1 --2- 172.21.15.62:0/3188378953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1eb49e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f07f809b4c0 tx=0x7f07f800df10 comp rx=0 tx=0).stop 2024-06-10T00:42:38.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.633+0000 7f08179fdb80 1 -- 172.21.15.62:0/3188378953 shutdown_connections 2024-06-10T00:42:38.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.633+0000 7f08179fdb80 1 --2- 172.21.15.62:0/3188378953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1eb4f20 0x5650a1eb49e0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.633+0000 7f08179fdb80 1 --2- 172.21.15.62:0/3188378953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1ebaf60 0x5650a1ed6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.633+0000 7f08179fdb80 1 --2- 172.21.15.62:0/3188378953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ebba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.633+0000 7f08179fdb80 1 -- 172.21.15.62:0/3188378953 >> 172.21.15.62:0/3188378953 conn(0x5650a1ddfcc0 msgr2=0x5650a1ebb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:38.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.633+0000 7f08179fdb80 1 -- 172.21.15.62:0/3188378953 shutdown_connections 2024-06-10T00:42:38.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.633+0000 7f08179fdb80 1 -- 172.21.15.62:0/3188378953 wait complete. 2024-06-10T00:42:38.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.633+0000 7f08179fdb80 1 -- start start 2024-06-10T00:42:38.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.633+0000 7f08179fdb80 1 Processor -- start 2024-06-10T00:42:38.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f08179fdb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1eb4f20 0x5650a1ec7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f08179fdb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ed6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f08179fdb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1ebaf60 0x5650a1ed6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f08179fdb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f08081b1d40 con 0x5650a1ebaf60 2024-06-10T00:42:38.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f08179fdb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f08081b1670 con 0x5650a1eb4f20 2024-06-10T00:42:38.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f08179fdb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08081ae890 con 0x5650a1eba640 2024-06-10T00:42:38.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f08179fdb80 10 client.0 _ll_register_callbacks cb 0x5650a1da0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f080759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1ebaf60 0x5650a1ed6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f080759e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1ebaf60 0x5650a1ed6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43768/0 (socket says 172.21.15.62:43768) 2024-06-10T00:42:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f080759e700 1 -- 172.21.15.62:0/3329065797 learned_addr learned my addr 172.21.15.62:0/3329065797 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f0806d9d700 1 --2- 172.21.15.62:0/3329065797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1eb4f20 0x5650a1ec7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f080659c700 1 --2- 172.21.15.62:0/3329065797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ed6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f080759e700 1 -- 172.21.15.62:0/3329065797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 msgr2=0x5650a1ed6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f080759e700 1 --2- 172.21.15.62:0/3329065797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ed6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f080759e700 1 -- 172.21.15.62:0/3329065797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1eb4f20 msgr2=0x5650a1ec7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f080759e700 1 --2- 172.21.15.62:0/3329065797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1eb4f20 0x5650a1ec7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f080759e700 1 -- 172.21.15.62:0/3329065797 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a1eb56f0 con 0x5650a1ebaf60 2024-06-10T00:42:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f0806d9d700 1 --2- 172.21.15.62:0/3329065797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1eb4f20 0x5650a1ec7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:42:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f080759e700 1 --2- 172.21.15.62:0/3329065797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1ebaf60 0x5650a1ed6750 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0800009bf0 tx=0x7f0800009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88c0f00842d68ba server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.634+0000 7f07eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.635+0000 7f07eaff5700 1 -- 172.21.15.62:0/3329065797 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080002c070 con 0x5650a1ebaf60 2024-06-10T00:42:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.635+0000 7f07eaff5700 1 -- 172.21.15.62:0/3329065797 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f0800007790 con 0x5650a1ebaf60 2024-06-10T00:42:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.635+0000 7f08179fdb80 1 -- 172.21.15.62:0/3329065797 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07f80ba1a0 con 0x5650a1ebaf60 2024-06-10T00:42:38.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.635+0000 7f07eaff5700 1 -- 172.21.15.62:0/3329065797 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0800037050 con 0x5650a1ebaf60 2024-06-10T00:42:38.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.635+0000 7f08179fdb80 10 client.17010 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:38.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.635+0000 7f08179fdb80 1 -- 172.21.15.62:0/3329065797 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5650a1f39500 con 0x5650a1ebaf60 2024-06-10T00:42:38.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.635+0000 7f07eaff5700 1 -- 172.21.15.62:0/3329065797 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07f80ba1a0 con 0x5650a1ebaf60 2024-06-10T00:42:38.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.635+0000 7f07eaff5700 1 -- 172.21.15.62:0/3329065797 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f0800011070 con 0x5650a1ebaf60 2024-06-10T00:42:38.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.635+0000 7f08179fdb80 10 client.17010 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:38.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.635+0000 7f08179fdb80 10 client.17010 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:42:38.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.635+0000 7f08179fdb80 20 client.17010 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.635+0000 7f08179fdb80 1 --2- 172.21.15.62:0/3329065797 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5650a1ece8b0 0x5650a1f3b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.635+0000 7f08179fdb80 4 client.17010 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:42:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.635+0000 7f08179fdb80 1 -- 172.21.15.62:0/3329065797 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650a1e0e290 con 0x5650a1ece8b0 2024-06-10T00:42:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f0806d9d700 1 --2- 172.21.15.62:0/3329065797 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5650a1ece8b0 0x5650a1f3b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f0806d9d700 1 --2- 172.21.15.62:0/3329065797 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5650a1ece8b0 0x5650a1f3b6e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6eb72962bb2117c8 server_cookie=27f7b46f4eefa161 in_seq=0 out_seq=0 2024-06-10T00:42:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f07eaff5700 10 client.17010 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:42:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f07eaff5700 1 -- 172.21.15.62:0/3329065797 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650a1e0e290 con 0x5650a1ece8b0 2024-06-10T00:42:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f07eaff5700 10 client.17010 handle_command_reply: tid=0 2024-06-10T00:42:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f08179fdb80 1 client.17010 shutdown 2024-06-10T00:42:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f08179fdb80 1 -- 172.21.15.62:0/3329065797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1ebaf60 msgr2=0x5650a1ed6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f08179fdb80 1 --2- 172.21.15.62:0/3329065797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1ebaf60 0x5650a1ed6750 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0800009bf0 tx=0x7f0800009530 comp rx=0 tx=0).stop 2024-06-10T00:42:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f08179fdb80 1 -- 172.21.15.62:0/3329065797 shutdown_connections 2024-06-10T00:42:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f08179fdb80 1 --2- 172.21.15.62:0/3329065797 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5650a1ece8b0 0x5650a1f3b6e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f08179fdb80 1 --2- 172.21.15.62:0/3329065797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650a1ebaf60 0x5650a1ed6750 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f08179fdb80 1 --2- 172.21.15.62:0/3329065797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650a1eb4f20 0x5650a1ec7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f08179fdb80 1 --2- 172.21.15.62:0/3329065797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5650a1eba640 0x5650a1ed6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f08179fdb80 1 -- 172.21.15.62:0/3329065797 >> 172.21.15.62:0/3329065797 conn(0x5650a1ddfcc0 msgr2=0x5650a1ea9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:38.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.636+0000 7f07eaff5700 0 client.17010 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:42:38.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f08179fdb80 1 -- 172.21.15.62:0/3329065797 shutdown_connections 2024-06-10T00:42:38.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f08179fdb80 1 -- 172.21.15.62:0/3329065797 wait complete. 2024-06-10T00:42:38.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f08179fdb80 20 client.17010 trim_cache size 0 max 16384 2024-06-10T00:42:38.648 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:42:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f080d17a700 1 -- 172.21.15.62:0/3969331636 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f07f00a2d90 msgr2=0x7f07f00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f080d17a700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f07f00a2d90 0x7f07f00a5250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0808108bb0 tx=0x7f07fc009260 comp rx=0 tx=0).stop 2024-06-10T00:42:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f080d17a700 1 -- 172.21.15.62:0/3969331636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0808102770 msgr2=0x7f0808074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f080d17a700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0808102770 0x7f0808074790 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f07f800d980 tx=0x7f07f800dd40 comp rx=0 tx=0).stop 2024-06-10T00:42:38.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f080d17a700 1 -- 172.21.15.62:0/3969331636 shutdown_connections 2024-06-10T00:42:38.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f080d17a700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f07f00a2d90 0x7f07f00a5250 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f080d17a700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0808102770 0x7f0808074790 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f080d17a700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0808107860 0x7f0808075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f080d17a700 1 --2- 172.21.15.62:0/3969331636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0808103120 0x7f0808074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:38.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f080d17a700 1 -- 172.21.15.62:0/3969331636 >> 172.21.15.62:0/3969331636 conn(0x7f0808079600 msgr2=0x7f0808100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:38.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f080d17a700 1 -- 172.21.15.62:0/3969331636 shutdown_connections 2024-06-10T00:42:38.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:38.637+0000 7f080d17a700 1 -- 172.21.15.62:0/3969331636 wait complete. 2024-06-10T00:42:39.325 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T00:42:39.580 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:42:39.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:39 smithi062 ceph-mon[32390]: pgmap v303: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:42:39.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:39 smithi138 ceph-mon[36549]: pgmap v303: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:42:40.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:39 smithi105 ceph-mon[36321]: pgmap v303: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:42:41.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.324+0000 7fb54587b700 1 -- 172.21.15.62:0/1441289479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb540107840 msgr2=0x7fb54010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:41.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.324+0000 7fb54587b700 1 --2- 172.21.15.62:0/1441289479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb540107840 0x7fb54010f3c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb5380099c0 tx=0x7fb538009cd0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.324+0000 7fb54587b700 1 -- 172.21.15.62:0/1441289479 shutdown_connections 2024-06-10T00:42:41.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.324+0000 7fb54587b700 1 --2- 172.21.15.62:0/1441289479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb540107840 0x7fb54010f3c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.324+0000 7fb54587b700 1 --2- 172.21.15.62:0/1441289479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb540102740 0x7fb540102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.324+0000 7fb54587b700 1 --2- 172.21.15.62:0/1441289479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5401030f0 0x7fb540107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.324+0000 7fb54587b700 1 -- 172.21.15.62:0/1441289479 >> 172.21.15.62:0/1441289479 conn(0x7fb5400795f0 msgr2=0x7fb540079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:41.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.324+0000 7fb54587b700 1 -- 172.21.15.62:0/1441289479 shutdown_connections 2024-06-10T00:42:41.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.324+0000 7fb54587b700 1 -- 172.21.15.62:0/1441289479 wait complete. 2024-06-10T00:42:41.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.325+0000 7fb54587b700 1 Processor -- start 2024-06-10T00:42:41.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.325+0000 7fb54587b700 1 -- start start 2024-06-10T00:42:41.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.325+0000 7fb54587b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb540102740 0x7fb5401a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.325+0000 7fb54587b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5401030f0 0x7fb5401a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.325+0000 7fb54587b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb540107840 0x7fb54019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.325+0000 7fb54587b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb540112780 con 0x7fb5401030f0 2024-06-10T00:42:41.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.325+0000 7fb54587b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb540112600 con 0x7fb540107840 2024-06-10T00:42:41.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.325+0000 7fb54587b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb540112900 con 0x7fb540102740 2024-06-10T00:42:41.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.325+0000 7fb53e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5401030f0 0x7fb5401a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.325+0000 7fb53e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5401030f0 0x7fb5401a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43788/0 (socket says 172.21.15.62:43788) 2024-06-10T00:42:41.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.325+0000 7fb53e7fc700 1 -- 172.21.15.62:0/3484237772 learned_addr learned my addr 172.21.15.62:0/3484237772 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:41.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.325+0000 7fb53f7fe700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb540107840 0x7fb54019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.325+0000 7fb53effd700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb540102740 0x7fb5401a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.326+0000 7fb53e7fc700 1 -- 172.21.15.62:0/3484237772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb540102740 msgr2=0x7fb5401a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:41.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.326+0000 7fb53e7fc700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb540102740 0x7fb5401a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.326+0000 7fb53e7fc700 1 -- 172.21.15.62:0/3484237772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb540107840 msgr2=0x7fb54019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:41.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.326+0000 7fb53e7fc700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb540107840 0x7fb54019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.326+0000 7fb53e7fc700 1 -- 172.21.15.62:0/3484237772 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54019b210 con 0x7fb5401030f0 2024-06-10T00:42:41.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.326+0000 7fb53f7fe700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb540107840 0x7fb54019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:41.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.326+0000 7fb53e7fc700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5401030f0 0x7fb5401a0e30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb52800d980 tx=0x7fb52800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:41.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.326+0000 7fb544879700 1 -- 172.21.15.62:0/3484237772 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528017070 con 0x7fb5401030f0 2024-06-10T00:42:41.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.326+0000 7fb54587b700 1 -- 172.21.15.62:0/3484237772 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54019b500 con 0x7fb5401030f0 2024-06-10T00:42:41.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.326+0000 7fb544879700 1 -- 172.21.15.62:0/3484237772 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb528004d10 con 0x7fb5401030f0 2024-06-10T00:42:41.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.326+0000 7fb544879700 1 -- 172.21.15.62:0/3484237772 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5280055c0 con 0x7fb5401030f0 2024-06-10T00:42:41.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.326+0000 7fb54587b700 1 -- 172.21.15.62:0/3484237772 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5401a77c0 con 0x7fb5401030f0 2024-06-10T00:42:41.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.328+0000 7fb544879700 1 -- 172.21.15.62:0/3484237772 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb528004750 con 0x7fb5401030f0 2024-06-10T00:42:41.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.329+0000 7fb544879700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb52c0a2d90 0x7fb52c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.329+0000 7fb53effd700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb52c0a2d90 0x7fb52c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.329+0000 7fb544879700 1 -- 172.21.15.62:0/3484237772 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7fb5280cf810 con 0x7fb5401030f0 2024-06-10T00:42:41.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.330+0000 7fb53effd700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb52c0a2d90 0x7fb52c0a5250 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb54019bbf0 tx=0x7fb530009040 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:41.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.330+0000 7fb5500feb80 1 Processor -- start 2024-06-10T00:42:41.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.330+0000 7fb5500feb80 1 -- start start 2024-06-10T00:42:41.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.330+0000 7fb5500feb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f33a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.330+0000 7fb5500feb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 0x55fe65f2d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.330+0000 7fb5500feb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f2e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.330+0000 7fb5500feb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fe65e7f810 con 0x55fe65f33640 2024-06-10T00:42:41.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.330+0000 7fb5500feb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fe65e27720 con 0x55fe65f33f60 2024-06-10T00:42:41.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.330+0000 7fb5500feb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fe65e7b7d0 con 0x55fe65f2df20 2024-06-10T00:42:41.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.330+0000 7fb53effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f33a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.330+0000 7fb53effd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f33a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43802/0 (socket says 172.21.15.62:43802) 2024-06-10T00:42:41.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.330+0000 7fb53effd700 1 -- 172.21.15.62:0/2536722913 learned_addr learned my addr 172.21.15.62:0/2536722913 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:41.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.331+0000 7fb53f7fe700 1 --2- 172.21.15.62:0/2536722913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f2e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.331+0000 7fb53e7fc700 1 --2- 172.21.15.62:0/2536722913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 0x55fe65f2d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.331+0000 7fb53effd700 1 -- 172.21.15.62:0/2536722913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 msgr2=0x55fe65f2e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:41.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.331+0000 7fb53effd700 1 --2- 172.21.15.62:0/2536722913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f2e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.331+0000 7fb53effd700 1 -- 172.21.15.62:0/2536722913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 msgr2=0x55fe65f2d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:41.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.331+0000 7fb53effd700 1 --2- 172.21.15.62:0/2536722913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 0x55fe65f2d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.331+0000 7fb53effd700 1 -- 172.21.15.62:0/2536722913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe65e1df50 con 0x55fe65f33640 2024-06-10T00:42:41.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.331+0000 7fb53f7fe700 1 --2- 172.21.15.62:0/2536722913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f2e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:42:41.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.331+0000 7fb53effd700 1 --2- 172.21.15.62:0/2536722913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f33a20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb53000ee70 tx=0x7fb530011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c253b086ff92af3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:41.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.331+0000 7fb535ffb700 1 -- 172.21.15.62:0/2536722913 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530025070 con 0x55fe65f33640 2024-06-10T00:42:41.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.331+0000 7fb535ffb700 1 -- 172.21.15.62:0/2536722913 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5300208e0 con 0x55fe65f33640 2024-06-10T00:42:41.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.331+0000 7fb535ffb700 1 -- 172.21.15.62:0/2536722913 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530013670 con 0x55fe65f33640 2024-06-10T00:42:41.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.332+0000 7fb5500feb80 1 -- 172.21.15.62:0/2536722913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 msgr2=0x55fe65f33a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:41.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.332+0000 7fb5500feb80 1 --2- 172.21.15.62:0/2536722913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f33a20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb53000ee70 tx=0x7fb530011000 comp rx=0 tx=0).stop 2024-06-10T00:42:41.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.332+0000 7fb5500feb80 1 -- 172.21.15.62:0/2536722913 shutdown_connections 2024-06-10T00:42:41.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.332+0000 7fb5500feb80 1 --2- 172.21.15.62:0/2536722913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f33a20 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.332+0000 7fb5500feb80 1 --2- 172.21.15.62:0/2536722913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f2e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.332+0000 7fb5500feb80 1 --2- 172.21.15.62:0/2536722913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 0x55fe65f2d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.332+0000 7fb5500feb80 1 -- 172.21.15.62:0/2536722913 >> 172.21.15.62:0/2536722913 conn(0x55fe65e58cc0 msgr2=0x55fe65f28850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:41.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.332+0000 7fb5500feb80 1 -- 172.21.15.62:0/2536722913 shutdown_connections 2024-06-10T00:42:41.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.332+0000 7fb5500feb80 1 -- 172.21.15.62:0/2536722913 wait complete. 2024-06-10T00:42:41.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.332+0000 7fb5500feb80 1 -- start start 2024-06-10T00:42:41.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.332+0000 7fb5500feb80 1 Processor -- start 2024-06-10T00:42:41.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb5500feb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f39cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb5500feb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33640 0x55fe65f36ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb5500feb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33f60 0x55fe65f37530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb5500feb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fe65f3a9d0 con 0x55fe65f33f60 2024-06-10T00:42:41.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb5500feb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fe65e1fbf0 con 0x55fe65f33640 2024-06-10T00:42:41.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb5500feb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fe65f3ab50 con 0x55fe65f2df20 2024-06-10T00:42:41.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb5500feb80 10 client.0 _ll_register_callbacks cb 0x55fe65e19920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:41.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb53f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33f60 0x55fe65f37530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb53f7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33f60 0x55fe65f37530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43818/0 (socket says 172.21.15.62:43818) 2024-06-10T00:42:41.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb53f7fe700 1 -- 172.21.15.62:0/802355757 learned_addr learned my addr 172.21.15.62:0/802355757 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:41.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb53e7fc700 1 --2- 172.21.15.62:0/802355757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33640 0x55fe65f36ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb53effd700 1 --2- 172.21.15.62:0/802355757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f39cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb53f7fe700 1 -- 172.21.15.62:0/802355757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 msgr2=0x55fe65f39cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:41.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb53f7fe700 1 --2- 172.21.15.62:0/802355757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f39cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb53f7fe700 1 -- 172.21.15.62:0/802355757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33640 msgr2=0x55fe65f36ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:41.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb53f7fe700 1 --2- 172.21.15.62:0/802355757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33640 0x55fe65f36ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb53f7fe700 1 -- 172.21.15.62:0/802355757 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe65dce8d0 con 0x55fe65f33f60 2024-06-10T00:42:41.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb53f7fe700 1 --2- 172.21.15.62:0/802355757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33f60 0x55fe65f37530 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb538015000 tx=0x7fb538005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8f5cb8f3e52f0d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:41.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.333+0000 7fb51effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:41.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.334+0000 7fb51effd700 1 -- 172.21.15.62:0/802355757 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538027060 con 0x55fe65f33f60 2024-06-10T00:42:41.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.334+0000 7fb51effd700 1 -- 172.21.15.62:0/802355757 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7fb5380052c0 con 0x55fe65f33f60 2024-06-10T00:42:41.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.334+0000 7fb5500feb80 1 -- 172.21.15.62:0/802355757 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5300208e0 con 0x55fe65f33f60 2024-06-10T00:42:41.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.334+0000 7fb51effd700 1 -- 172.21.15.62:0/802355757 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb53802e050 con 0x55fe65f33f60 2024-06-10T00:42:41.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.334+0000 7fb5500feb80 10 client.17022 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:41.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.334+0000 7fb5500feb80 1 -- 172.21.15.62:0/802355757 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fe65f452f0 con 0x55fe65f33f60 2024-06-10T00:42:41.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.334+0000 7fb51effd700 1 -- 172.21.15.62:0/802355757 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5300208e0 con 0x55fe65f33f60 2024-06-10T00:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.334+0000 7fb51effd700 1 -- 172.21.15.62:0/802355757 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fb538021070 con 0x55fe65f33f60 2024-06-10T00:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.334+0000 7fb5500feb80 10 client.17022 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.334+0000 7fb5500feb80 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.334+0000 7fb5500feb80 20 client.17022 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.334+0000 7fb5500feb80 1 --2- 172.21.15.62:0/802355757 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55fe65f47930 0x55fe65f49d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.334+0000 7fb5500feb80 4 client.17022 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.334+0000 7fb5500feb80 1 -- 172.21.15.62:0/802355757 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe65e8cd10 con 0x55fe65f47930 2024-06-10T00:42:41.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.335+0000 7fb53effd700 1 --2- 172.21.15.62:0/802355757 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55fe65f47930 0x55fe65f49d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.335+0000 7fb51effd700 10 client.17022 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:41.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.335+0000 7fb53effd700 1 --2- 172.21.15.62:0/802355757 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55fe65f47930 0x55fe65f49d20 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aac093dce7579139 server_cookie=ac6d7ebcdf8417b1 in_seq=0 out_seq=0 2024-06-10T00:42:41.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb51effd700 1 -- 172.21.15.62:0/802355757 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fe65e8cd10 con 0x55fe65f47930 2024-06-10T00:42:41.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb51effd700 10 client.17022 handle_command_reply: tid=0 2024-06-10T00:42:41.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb5500feb80 1 client.17022 shutdown 2024-06-10T00:42:41.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb5500feb80 1 -- 172.21.15.62:0/802355757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33f60 msgr2=0x55fe65f37530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:41.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb5500feb80 1 --2- 172.21.15.62:0/802355757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33f60 0x55fe65f37530 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb538015000 tx=0x7fb538005c00 comp rx=0 tx=0).stop 2024-06-10T00:42:41.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb5500feb80 1 -- 172.21.15.62:0/802355757 shutdown_connections 2024-06-10T00:42:41.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb5500feb80 1 --2- 172.21.15.62:0/802355757 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55fe65f47930 0x55fe65f49d20 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb5500feb80 1 --2- 172.21.15.62:0/802355757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33f60 0x55fe65f37530 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb5500feb80 1 --2- 172.21.15.62:0/802355757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f39cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb5500feb80 1 --2- 172.21.15.62:0/802355757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33640 0x55fe65f36ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb5500feb80 1 -- 172.21.15.62:0/802355757 >> 172.21.15.62:0/802355757 conn(0x55fe65e58cc0 msgr2=0x55fe65f2c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:41.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb51effd700 0 client.17022 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:41.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb5500feb80 1 -- 172.21.15.62:0/802355757 shutdown_connections 2024-06-10T00:42:41.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb5500feb80 1 -- 172.21.15.62:0/802355757 wait complete. 2024-06-10T00:42:41.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.336+0000 7fb5500feb80 20 client.17022 trim_cache size 0 max 16384 2024-06-10T00:42:41.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.351+0000 7fb5500feb80 1 Processor -- start 2024-06-10T00:42:41.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb5500feb80 1 -- start start 2024-06-10T00:42:41.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb5500feb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f2d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb5500feb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f36fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb5500feb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 0x55fe65f374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb5500feb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb5401b07c0 con 0x55fe65f33640 2024-06-10T00:42:41.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb5500feb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5401b2300 con 0x55fe65f33f60 2024-06-10T00:42:41.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb5500feb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5401b0e90 con 0x55fe65f2df20 2024-06-10T00:42:41.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb53e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f36fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb53e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f36fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43834/0 (socket says 172.21.15.62:43834) 2024-06-10T00:42:41.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb53e7fc700 1 -- 172.21.15.62:0/3756792557 learned_addr learned my addr 172.21.15.62:0/3756792557 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:41.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb53f7fe700 1 --2- 172.21.15.62:0/3756792557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 0x55fe65f374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb53effd700 1 --2- 172.21.15.62:0/3756792557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f2d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb53e7fc700 1 -- 172.21.15.62:0/3756792557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 msgr2=0x55fe65f2d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:41.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb53e7fc700 1 --2- 172.21.15.62:0/3756792557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f2d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb53e7fc700 1 -- 172.21.15.62:0/3756792557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 msgr2=0x55fe65f374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:41.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb53e7fc700 1 --2- 172.21.15.62:0/3756792557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 0x55fe65f374f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb53e7fc700 1 -- 172.21.15.62:0/3756792557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe65f37a30 con 0x55fe65f33640 2024-06-10T00:42:41.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb53f7fe700 1 --2- 172.21.15.62:0/3756792557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 0x55fe65f374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:41.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.352+0000 7fb53e7fc700 1 --2- 172.21.15.62:0/3756792557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f36fb0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb528097d80 tx=0x7fb528005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9177d8cce097f304 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:41.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.353+0000 7fb5357fa700 1 -- 172.21.15.62:0/3756792557 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5280b4070 con 0x55fe65f33640 2024-06-10T00:42:41.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.353+0000 7fb5357fa700 1 -- 172.21.15.62:0/3756792557 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb52800ee10 con 0x55fe65f33640 2024-06-10T00:42:41.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.353+0000 7fb5357fa700 1 -- 172.21.15.62:0/3756792557 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528084e60 con 0x55fe65f33640 2024-06-10T00:42:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.353+0000 7fb5500feb80 1 -- 172.21.15.62:0/3756792557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 msgr2=0x55fe65f36fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.353+0000 7fb5500feb80 1 --2- 172.21.15.62:0/3756792557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f36fb0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb528097d80 tx=0x7fb528005870 comp rx=0 tx=0).stop 2024-06-10T00:42:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.353+0000 7fb5500feb80 1 -- 172.21.15.62:0/3756792557 shutdown_connections 2024-06-10T00:42:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.353+0000 7fb5500feb80 1 --2- 172.21.15.62:0/3756792557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 0x55fe65f374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.353+0000 7fb5500feb80 1 --2- 172.21.15.62:0/3756792557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f2d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.353+0000 7fb5500feb80 1 --2- 172.21.15.62:0/3756792557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f36fb0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.353+0000 7fb5500feb80 1 -- 172.21.15.62:0/3756792557 >> 172.21.15.62:0/3756792557 conn(0x55fe65e58cc0 msgr2=0x55fe65f358e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.353+0000 7fb5500feb80 1 -- 172.21.15.62:0/3756792557 shutdown_connections 2024-06-10T00:42:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.353+0000 7fb5500feb80 1 -- 172.21.15.62:0/3756792557 wait complete. 2024-06-10T00:42:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.354+0000 7fb5500feb80 1 -- start start 2024-06-10T00:42:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.354+0000 7fb5500feb80 1 Processor -- start 2024-06-10T00:42:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.354+0000 7fb5500feb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f4fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.354+0000 7fb5500feb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f3fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.354+0000 7fb5500feb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 0x55fe65f40380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.354+0000 7fb5500feb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb5401b1c30 con 0x55fe65f33640 2024-06-10T00:42:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.354+0000 7fb5500feb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5401b1560 con 0x55fe65f33f60 2024-06-10T00:42:41.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.354+0000 7fb5500feb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5401ae780 con 0x55fe65f2df20 2024-06-10T00:42:41.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.354+0000 7fb5500feb80 10 client.0 _ll_register_callbacks cb 0x55fe65e19920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:41.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.354+0000 7fb53e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f3fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.354+0000 7fb53e7fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f3fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43848/0 (socket says 172.21.15.62:43848) 2024-06-10T00:42:41.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.354+0000 7fb53e7fc700 1 -- 172.21.15.62:0/825973469 learned_addr learned my addr 172.21.15.62:0/825973469 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:41.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb53effd700 1 --2- 172.21.15.62:0/825973469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f4fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb53e7fc700 1 -- 172.21.15.62:0/825973469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 msgr2=0x55fe65f4fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:41.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb53e7fc700 1 --2- 172.21.15.62:0/825973469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f4fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb53e7fc700 1 -- 172.21.15.62:0/825973469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 msgr2=0x55fe65f40380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:41.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb53e7fc700 1 --2- 172.21.15.62:0/825973469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 0x55fe65f40380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb53e7fc700 1 -- 172.21.15.62:0/825973469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe65f4e570 con 0x55fe65f33640 2024-06-10T00:42:41.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb53e7fc700 1 --2- 172.21.15.62:0/825973469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f3fe40 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb5280cd150 tx=0x7fb528099e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcbc4cb6e9dc2bc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:41.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb51effd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:41.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb51effd700 1 -- 172.21.15.62:0/825973469 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5280b8070 con 0x55fe65f33640 2024-06-10T00:42:41.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb51effd700 1 -- 172.21.15.62:0/825973469 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7fb528004a40 con 0x55fe65f33640 2024-06-10T00:42:41.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb5500feb80 1 -- 172.21.15.62:0/825973469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52800ee10 con 0x55fe65f33640 2024-06-10T00:42:41.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb51effd700 1 -- 172.21.15.62:0/825973469 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb5280b4040 con 0x55fe65f33640 2024-06-10T00:42:41.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb5500feb80 10 client.17028 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:41.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb5500feb80 1 -- 172.21.15.62:0/825973469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fe65fb16a0 con 0x55fe65f33640 2024-06-10T00:42:41.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.355+0000 7fb51effd700 1 -- 172.21.15.62:0/825973469 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb52800ee10 con 0x55fe65f33640 2024-06-10T00:42:41.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.356+0000 7fb51effd700 1 -- 172.21.15.62:0/825973469 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fb5280a4100 con 0x55fe65f33640 2024-06-10T00:42:41.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.356+0000 7fb5500feb80 10 client.17028 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:41.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.356+0000 7fb5500feb80 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:42:41.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.356+0000 7fb5500feb80 20 client.17028 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:41.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.356+0000 7fb5500feb80 1 --2- 172.21.15.62:0/825973469 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55fe65fb1cd0 0x55fe65fb40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:41.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.356+0000 7fb5500feb80 4 client.17028 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:42:41.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.356+0000 7fb5500feb80 1 -- 172.21.15.62:0/825973469 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe65e87290 con 0x55fe65fb1cd0 2024-06-10T00:42:41.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.356+0000 7fb53effd700 1 --2- 172.21.15.62:0/825973469 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55fe65fb1cd0 0x55fe65fb40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:41.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.356+0000 7fb53effd700 1 --2- 172.21.15.62:0/825973469 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55fe65fb1cd0 0x55fe65fb40f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b774fa84e40542d1 server_cookie=78047973fb733ae2 in_seq=0 out_seq=0 2024-06-10T00:42:41.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.356+0000 7fb51effd700 10 client.17028 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:41.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.356+0000 7fb51effd700 1 -- 172.21.15.62:0/825973469 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe65e87290 con 0x55fe65fb1cd0 2024-06-10T00:42:41.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.356+0000 7fb51effd700 10 client.17028 handle_command_reply: tid=0 2024-06-10T00:42:41.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.356+0000 7fb5500feb80 1 client.17028 shutdown 2024-06-10T00:42:41.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.357+0000 7fb5500feb80 1 -- 172.21.15.62:0/825973469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 msgr2=0x55fe65f3fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:41.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.357+0000 7fb5500feb80 1 --2- 172.21.15.62:0/825973469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f3fe40 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb5280cd150 tx=0x7fb528099e30 comp rx=0 tx=0).stop 2024-06-10T00:42:41.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.357+0000 7fb5500feb80 1 -- 172.21.15.62:0/825973469 shutdown_connections 2024-06-10T00:42:41.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.357+0000 7fb5500feb80 1 --2- 172.21.15.62:0/825973469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe65f33f60 0x55fe65f40380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.357+0000 7fb5500feb80 1 --2- 172.21.15.62:0/825973469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe65f2df20 0x55fe65f4fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.357+0000 7fb5500feb80 1 --2- 172.21.15.62:0/825973469 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55fe65fb1cd0 0x55fe65fb40f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.357+0000 7fb5500feb80 1 --2- 172.21.15.62:0/825973469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe65f33640 0x55fe65f3fe40 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.357+0000 7fb5500feb80 1 -- 172.21.15.62:0/825973469 >> 172.21.15.62:0/825973469 conn(0x55fe65e58cc0 msgr2=0x55fe65f358a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:41.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.357+0000 7fb51effd700 0 client.17028 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:41.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.357+0000 7fb5500feb80 1 -- 172.21.15.62:0/825973469 shutdown_connections 2024-06-10T00:42:41.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.357+0000 7fb5500feb80 1 -- 172.21.15.62:0/825973469 wait complete. 2024-06-10T00:42:41.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.357+0000 7fb5500feb80 20 client.17028 trim_cache size 0 max 16384 2024-06-10T00:42:41.368 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:42:41.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.358+0000 7fb54587b700 1 -- 172.21.15.62:0/3484237772 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb52c0a2d90 msgr2=0x7fb52c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.358+0000 7fb54587b700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb52c0a2d90 0x7fb52c0a5250 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb54019bbf0 tx=0x7fb530009040 comp rx=0 tx=0).stop 2024-06-10T00:42:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.358+0000 7fb54587b700 1 -- 172.21.15.62:0/3484237772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5401030f0 msgr2=0x7fb5401a0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.358+0000 7fb54587b700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5401030f0 0x7fb5401a0e30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb52800d980 tx=0x7fb52800dd40 comp rx=0 tx=0).stop 2024-06-10T00:42:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.358+0000 7fb54587b700 1 -- 172.21.15.62:0/3484237772 shutdown_connections 2024-06-10T00:42:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.358+0000 7fb54587b700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fb52c0a2d90 0x7fb52c0a5250 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.358+0000 7fb54587b700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb540107840 0x7fb54019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.358+0000 7fb54587b700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb540102740 0x7fb5401a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.358+0000 7fb54587b700 1 --2- 172.21.15.62:0/3484237772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5401030f0 0x7fb5401a0e30 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:41.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.358+0000 7fb54587b700 1 -- 172.21.15.62:0/3484237772 >> 172.21.15.62:0/3484237772 conn(0x7fb5400795f0 msgr2=0x7fb5401005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:41.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.358+0000 7fb54587b700 1 -- 172.21.15.62:0/3484237772 shutdown_connections 2024-06-10T00:42:41.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:41.358+0000 7fb54587b700 1 -- 172.21.15.62:0/3484237772 wait complete. 2024-06-10T00:42:41.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:41 smithi062 ceph-mon[32390]: pgmap v304: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:42:41.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:41 smithi138 ceph-mon[36549]: pgmap v304: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:42:42.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:41 smithi105 ceph-mon[36321]: pgmap v304: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:42:42.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:42 smithi062 ceph-mon[32390]: pgmap v305: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T00:42:42.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:42 smithi138 ceph-mon[36549]: pgmap v305: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T00:42:43.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:42 smithi105 ceph-mon[36321]: pgmap v305: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-10T00:42:43.051 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T00:42:43.052 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T00:42:43.315 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:42:45.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.001+0000 7f90b1514700 1 -- 172.21.15.62:0/2084422202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac06abc0 msgr2=0x7f90ac06b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:45.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.001+0000 7f90b1514700 1 --2- 172.21.15.62:0/2084422202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac06abc0 0x7f90ac06b040 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f909c009990 tx=0x7f909c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.002+0000 7f90b1514700 1 -- 172.21.15.62:0/2084422202 shutdown_connections 2024-06-10T00:42:45.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.002+0000 7f90b1514700 1 --2- 172.21.15.62:0/2084422202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ac06b580 0x7f90ac10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.002+0000 7f90b1514700 1 --2- 172.21.15.62:0/2084422202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90ac108500 0x7f90ac1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.002+0000 7f90b1514700 1 --2- 172.21.15.62:0/2084422202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac06abc0 0x7f90ac06b040 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.002+0000 7f90b1514700 1 -- 172.21.15.62:0/2084422202 >> 172.21.15.62:0/2084422202 conn(0x7f90ac078340 msgr2=0x7f90ac078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:45.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.002+0000 7f90b1514700 1 -- 172.21.15.62:0/2084422202 shutdown_connections 2024-06-10T00:42:45.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.002+0000 7f90b1514700 1 -- 172.21.15.62:0/2084422202 wait complete. 2024-06-10T00:42:45.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.002+0000 7f90b1514700 1 Processor -- start 2024-06-10T00:42:45.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.002+0000 7f90b1514700 1 -- start start 2024-06-10T00:42:45.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90b1514700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90ac06abc0 0x7f90ac198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90b1514700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ac06b580 0x7f90ac199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90b1514700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac108500 0x7f90ac19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90b1514700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac1127e0 con 0x7f90ac108500 2024-06-10T00:42:45.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90b1514700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac112660 con 0x7f90ac06abc0 2024-06-10T00:42:45.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90b1514700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac112960 con 0x7f90ac06b580 2024-06-10T00:42:45.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90ab7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac108500 0x7f90ac19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90ab7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac108500 0x7f90ac19d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43870/0 (socket says 172.21.15.62:43870) 2024-06-10T00:42:45.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90ab7fe700 1 -- 172.21.15.62:0/2145462230 learned_addr learned my addr 172.21.15.62:0/2145462230 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:45.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90aaffd700 1 --2- 172.21.15.62:0/2145462230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90ac06abc0 0x7f90ac198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90aa7fc700 1 --2- 172.21.15.62:0/2145462230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ac06b580 0x7f90ac199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90ab7fe700 1 -- 172.21.15.62:0/2145462230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ac06b580 msgr2=0x7f90ac199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:45.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90ab7fe700 1 --2- 172.21.15.62:0/2145462230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ac06b580 0x7f90ac199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90ab7fe700 1 -- 172.21.15.62:0/2145462230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90ac06abc0 msgr2=0x7f90ac198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:45.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90ab7fe700 1 --2- 172.21.15.62:0/2145462230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90ac06abc0 0x7f90ac198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90ab7fe700 1 -- 172.21.15.62:0/2145462230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac19dd40 con 0x7f90ac108500 2024-06-10T00:42:45.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f90ab7fe700 1 --2- 172.21.15.62:0/2145462230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac108500 0x7f90ac19d600 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f90a4007ee0 tx=0x7f90a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:45.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.003+0000 7f9093fff700 1 -- 172.21.15.62:0/2145462230 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4004bb0 con 0x7f90ac108500 2024-06-10T00:42:45.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.004+0000 7f9093fff700 1 -- 172.21.15.62:0/2145462230 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a4004d50 con 0x7f90ac108500 2024-06-10T00:42:45.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.004+0000 7f90b1514700 1 -- 172.21.15.62:0/2145462230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ac19e030 con 0x7f90ac108500 2024-06-10T00:42:45.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.004+0000 7f9093fff700 1 -- 172.21.15.62:0/2145462230 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4005960 con 0x7f90ac108500 2024-06-10T00:42:45.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.004+0000 7f90b1514700 1 -- 172.21.15.62:0/2145462230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ac1a58b0 con 0x7f90ac108500 2024-06-10T00:42:45.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.006+0000 7f9093fff700 1 -- 172.21.15.62:0/2145462230 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f90a401a020 con 0x7f90ac108500 2024-06-10T00:42:45.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.007+0000 7f9093fff700 1 --2- 172.21.15.62:0/2145462230 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f90940a2d90 0x7f90940a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.007+0000 7f90aaffd700 1 --2- 172.21.15.62:0/2145462230 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f90940a2d90 0x7f90940a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.007+0000 7f9093fff700 1 -- 172.21.15.62:0/2145462230 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f90a40cef40 con 0x7f90ac108500 2024-06-10T00:42:45.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.007+0000 7f90aaffd700 1 --2- 172.21.15.62:0/2145462230 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f90940a2d90 0x7f90940a5250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009a10 tx=0x7f90a0009190 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:45.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.007+0000 7f90bbd97b80 1 Processor -- start 2024-06-10T00:42:45.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90bbd97b80 1 -- start start 2024-06-10T00:42:45.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90bbd97b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9ac46c0 0x5616c9ac4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90bbd97b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 0x5616c9abea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90bbd97b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9abefa0 0x5616c9abf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90aaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9ac46c0 0x5616c9ac4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90aaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9ac46c0 0x5616c9ac4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43882/0 (socket says 172.21.15.62:43882) 2024-06-10T00:42:45.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90bbd97b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5616c9a10890 con 0x5616c9ac46c0 2024-06-10T00:42:45.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90aaffd700 1 -- 172.21.15.62:0/1929313696 learned_addr learned my addr 172.21.15.62:0/1929313696 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:45.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90bbd97b80 1 -- 172.21.15.62:0/1929313696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5616c99b87a0 con 0x5616c9abefa0 2024-06-10T00:42:45.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90bbd97b80 1 -- 172.21.15.62:0/1929313696 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5616c9a0c850 con 0x5616c9ac4fe0 2024-06-10T00:42:45.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90aa7fc700 1 --2- 172.21.15.62:0/1929313696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 0x5616c9abea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90ab7fe700 1 --2- 172.21.15.62:0/1929313696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9abefa0 0x5616c9abf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90aaffd700 1 -- 172.21.15.62:0/1929313696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 msgr2=0x5616c9abea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:45.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90aaffd700 1 --2- 172.21.15.62:0/1929313696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 0x5616c9abea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90aaffd700 1 -- 172.21.15.62:0/1929313696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9abefa0 msgr2=0x5616c9abf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:45.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90aaffd700 1 --2- 172.21.15.62:0/1929313696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9abefa0 0x5616c9abf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90aaffd700 1 -- 172.21.15.62:0/1929313696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616c99aefd0 con 0x5616c9ac46c0 2024-06-10T00:42:45.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90aa7fc700 1 --2- 172.21.15.62:0/1929313696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 0x5616c9abea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:42:45.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.008+0000 7f90ab7fe700 1 --2- 172.21.15.62:0/1929313696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9abefa0 0x5616c9abf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:45.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.009+0000 7f90aaffd700 1 --2- 172.21.15.62:0/1929313696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9ac46c0 0x5616c9ac4aa0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f90a000ea00 tx=0x7f90a0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15470a0f00aeecd0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:45.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.009+0000 7f90917fa700 1 -- 172.21.15.62:0/1929313696 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a001b040 con 0x5616c9ac46c0 2024-06-10T00:42:45.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.009+0000 7f90917fa700 1 -- 172.21.15.62:0/1929313696 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a0024070 con 0x5616c9ac46c0 2024-06-10T00:42:45.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.009+0000 7f90917fa700 1 -- 172.21.15.62:0/1929313696 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0013810 con 0x5616c9ac46c0 2024-06-10T00:42:45.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.010+0000 7f90bbd97b80 1 -- 172.21.15.62:0/1929313696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9ac46c0 msgr2=0x5616c9ac4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:45.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.010+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/1929313696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9ac46c0 0x5616c9ac4aa0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f90a000ea00 tx=0x7f90a0011000 comp rx=0 tx=0).stop 2024-06-10T00:42:45.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.010+0000 7f90bbd97b80 1 -- 172.21.15.62:0/1929313696 shutdown_connections 2024-06-10T00:42:45.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.010+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/1929313696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9ac46c0 0x5616c9ac4aa0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.010+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/1929313696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9abefa0 0x5616c9abf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.010+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/1929313696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 0x5616c9abea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.010+0000 7f90bbd97b80 1 -- 172.21.15.62:0/1929313696 >> 172.21.15.62:0/1929313696 conn(0x5616c99e9d40 msgr2=0x5616c9ab98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:45.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.010+0000 7f90bbd97b80 1 -- 172.21.15.62:0/1929313696 shutdown_connections 2024-06-10T00:42:45.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.010+0000 7f90bbd97b80 1 -- 172.21.15.62:0/1929313696 wait complete. 2024-06-10T00:42:45.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.010+0000 7f90bbd97b80 1 -- start start 2024-06-10T00:42:45.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.010+0000 7f90bbd97b80 1 Processor -- start 2024-06-10T00:42:45.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90bbd97b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9abefa0 0x5616c9acad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90bbd97b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9ac46c0 0x5616c9ac8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90bbd97b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 0x5616c9ac85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90bbd97b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5616c9acba50 con 0x5616c9abefa0 2024-06-10T00:42:45.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90bbd97b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5616c99b0c70 con 0x5616c9ac46c0 2024-06-10T00:42:45.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90bbd97b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5616c9acbbd0 con 0x5616c9ac4fe0 2024-06-10T00:42:45.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90bbd97b80 10 client.0 _ll_register_callbacks cb 0x5616c99aa9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:45.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90aaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9abefa0 0x5616c9acad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90aaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9abefa0 0x5616c9acad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43892/0 (socket says 172.21.15.62:43892) 2024-06-10T00:42:45.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90aaffd700 1 -- 172.21.15.62:0/2894246299 learned_addr learned my addr 172.21.15.62:0/2894246299 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:45.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90aa7fc700 1 --2- 172.21.15.62:0/2894246299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9ac46c0 0x5616c9ac8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90ab7fe700 1 --2- 172.21.15.62:0/2894246299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 0x5616c9ac85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90aaffd700 1 -- 172.21.15.62:0/2894246299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 msgr2=0x5616c9ac85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:45.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90aaffd700 1 --2- 172.21.15.62:0/2894246299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 0x5616c9ac85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90aaffd700 1 -- 172.21.15.62:0/2894246299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9ac46c0 msgr2=0x5616c9ac8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:45.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90aaffd700 1 --2- 172.21.15.62:0/2894246299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9ac46c0 0x5616c9ac8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90aaffd700 1 -- 172.21.15.62:0/2894246299 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616c995f950 con 0x5616c9abefa0 2024-06-10T00:42:45.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90ab7fe700 1 --2- 172.21.15.62:0/2894246299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 0x5616c9ac85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:45.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f90aaffd700 1 --2- 172.21.15.62:0/2894246299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9abefa0 0x5616c9acad60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009330 tx=0x7f90a00159f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d116bcfb934aba29 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:45.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.011+0000 7f908eff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:45.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.012+0000 7f908eff5700 1 -- 172.21.15.62:0/2894246299 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a002d070 con 0x5616c9abefa0 2024-06-10T00:42:45.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.012+0000 7f908eff5700 1 -- 172.21.15.62:0/2894246299 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f90a0029810 con 0x5616c9abefa0 2024-06-10T00:42:45.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.012+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2894246299 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a0024070 con 0x5616c9abefa0 2024-06-10T00:42:45.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.012+0000 7f908eff5700 1 -- 172.21.15.62:0/2894246299 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f90a003b050 con 0x5616c9abefa0 2024-06-10T00:42:45.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.012+0000 7f90bbd97b80 10 client.17040 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:45.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.012+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2894246299 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5616c9ad6360 con 0x5616c9abefa0 2024-06-10T00:42:45.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.012+0000 7f908eff5700 1 -- 172.21.15.62:0/2894246299 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a0024070 con 0x5616c9abefa0 2024-06-10T00:42:45.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.012+0000 7f908eff5700 1 -- 172.21.15.62:0/2894246299 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f90a001e030 con 0x5616c9abefa0 2024-06-10T00:42:45.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.012+0000 7f90bbd97b80 10 client.17040 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:45.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.012+0000 7f90bbd97b80 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:42:45.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.012+0000 7f90bbd97b80 20 client.17040 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:45.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.013+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/2894246299 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5616c9ad8940 0x5616c9adad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.013+0000 7f90bbd97b80 4 client.17040 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:42:45.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.013+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2894246299 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616c9a1dd90 con 0x5616c9ad8940 2024-06-10T00:42:45.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.013+0000 7f90aa7fc700 1 --2- 172.21.15.62:0/2894246299 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5616c9ad8940 0x5616c9adad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.013+0000 7f90aa7fc700 1 --2- 172.21.15.62:0/2894246299 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5616c9ad8940 0x5616c9adad30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c49ffadeb3a1b9fe server_cookie=1bcba7023f54ec63 in_seq=0 out_seq=0 2024-06-10T00:42:45.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.013+0000 7f908eff5700 10 client.17040 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:45.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f908eff5700 1 -- 172.21.15.62:0/2894246299 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5616c9a1dd90 con 0x5616c9ad8940 2024-06-10T00:42:45.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f908eff5700 10 client.17040 handle_command_reply: tid=0 2024-06-10T00:42:45.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f90bbd97b80 1 client.17040 shutdown 2024-06-10T00:42:45.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2894246299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9abefa0 msgr2=0x5616c9acad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:45.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/2894246299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9abefa0 0x5616c9acad60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009330 tx=0x7f90a00159f0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2894246299 shutdown_connections 2024-06-10T00:42:45.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/2894246299 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5616c9ad8940 0x5616c9adad30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/2894246299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9abefa0 0x5616c9acad60 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/2894246299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 0x5616c9ac85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/2894246299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9ac46c0 0x5616c9ac8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2894246299 >> 172.21.15.62:0/2894246299 conn(0x5616c99e9d40 msgr2=0x5616c9abda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:45.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f908eff5700 0 client.17040 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:45.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2894246299 shutdown_connections 2024-06-10T00:42:45.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2894246299 wait complete. 2024-06-10T00:42:45.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.014+0000 7f90bbd97b80 20 client.17040 trim_cache size 0 max 16384 2024-06-10T00:42:45.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.035+0000 7f90bbd97b80 1 Processor -- start 2024-06-10T00:42:45.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.035+0000 7f90bbd97b80 1 -- start start 2024-06-10T00:42:45.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90bbd97b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9abefa0 0x5616c9abea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90bbd97b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac46c0 0x5616c9ab3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90bbd97b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9ac4fe0 0x5616c9add970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90bbd97b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac1ae860 con 0x5616c9ac4fe0 2024-06-10T00:42:45.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90bbd97b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac1b03a0 con 0x5616c9abefa0 2024-06-10T00:42:45.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90bbd97b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac1aef30 con 0x5616c9ac46c0 2024-06-10T00:42:45.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90ab7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9ac4fe0 0x5616c9add970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90ab7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9ac4fe0 0x5616c9add970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43894/0 (socket says 172.21.15.62:43894) 2024-06-10T00:42:45.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90ab7fe700 1 -- 172.21.15.62:0/3723834449 learned_addr learned my addr 172.21.15.62:0/3723834449 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:45.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90aaffd700 1 --2- 172.21.15.62:0/3723834449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9abefa0 0x5616c9abea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90aa7fc700 1 --2- 172.21.15.62:0/3723834449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac46c0 0x5616c9ab3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90ab7fe700 1 -- 172.21.15.62:0/3723834449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac46c0 msgr2=0x5616c9ab3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:45.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90ab7fe700 1 --2- 172.21.15.62:0/3723834449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac46c0 0x5616c9ab3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90ab7fe700 1 -- 172.21.15.62:0/3723834449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9abefa0 msgr2=0x5616c9abea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:45.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90ab7fe700 1 --2- 172.21.15.62:0/3723834449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9abefa0 0x5616c9abea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90ab7fe700 1 -- 172.21.15.62:0/3723834449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616c9ac65f0 con 0x5616c9ac4fe0 2024-06-10T00:42:45.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f90ab7fe700 1 --2- 172.21.15.62:0/3723834449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9ac4fe0 0x5616c9add970 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f90a4097a20 tx=0x7f90a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8838a19c5b14241 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:45.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f9090ff9700 1 -- 172.21.15.62:0/3723834449 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a409b050 con 0x5616c9ac4fe0 2024-06-10T00:42:45.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f9090ff9700 1 -- 172.21.15.62:0/3723834449 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a409f400 con 0x5616c9ac4fe0 2024-06-10T00:42:45.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.036+0000 7f9090ff9700 1 -- 172.21.15.62:0/3723834449 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a40cc690 con 0x5616c9ac4fe0 2024-06-10T00:42:45.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.037+0000 7f90bbd97b80 1 -- 172.21.15.62:0/3723834449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9ac4fe0 msgr2=0x5616c9add970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:45.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.037+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/3723834449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9ac4fe0 0x5616c9add970 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f90a4097a20 tx=0x7f90a400b040 comp rx=0 tx=0).stop 2024-06-10T00:42:45.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.037+0000 7f90bbd97b80 1 -- 172.21.15.62:0/3723834449 shutdown_connections 2024-06-10T00:42:45.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.037+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/3723834449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9ac4fe0 0x5616c9add970 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.037+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/3723834449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9abefa0 0x5616c9abea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.037+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/3723834449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac46c0 0x5616c9ab3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.037+0000 7f90bbd97b80 1 -- 172.21.15.62:0/3723834449 >> 172.21.15.62:0/3723834449 conn(0x5616c99e9d40 msgr2=0x5616c9ac5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.037+0000 7f90bbd97b80 1 -- 172.21.15.62:0/3723834449 shutdown_connections 2024-06-10T00:42:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.037+0000 7f90bbd97b80 1 -- 172.21.15.62:0/3723834449 wait complete. 2024-06-10T00:42:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.037+0000 7f90bbd97b80 1 -- start start 2024-06-10T00:42:45.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90bbd97b80 1 Processor -- start 2024-06-10T00:42:45.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90bbd97b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9abefa0 0x5616c9adba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90bbd97b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9ac46c0 0x5616c9adbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90bbd97b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 0x5616c9b46690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90bbd97b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac1afcd0 con 0x5616c9abefa0 2024-06-10T00:42:45.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90bbd97b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac1af600 con 0x5616c9ac46c0 2024-06-10T00:42:45.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90bbd97b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac1ac820 con 0x5616c9ac4fe0 2024-06-10T00:42:45.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90bbd97b80 10 client.0 _ll_register_callbacks cb 0x5616c99aa9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:45.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90aaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9abefa0 0x5616c9adba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90aaffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9abefa0 0x5616c9adba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43908/0 (socket says 172.21.15.62:43908) 2024-06-10T00:42:45.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90aaffd700 1 -- 172.21.15.62:0/2895606344 learned_addr learned my addr 172.21.15.62:0/2895606344 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:45.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90ab7fe700 1 --2- 172.21.15.62:0/2895606344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 0x5616c9b46690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90aa7fc700 1 --2- 172.21.15.62:0/2895606344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9ac46c0 0x5616c9adbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90aaffd700 1 -- 172.21.15.62:0/2895606344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 msgr2=0x5616c9b46690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:45.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90aaffd700 1 --2- 172.21.15.62:0/2895606344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 0x5616c9b46690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90aaffd700 1 -- 172.21.15.62:0/2895606344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9ac46c0 msgr2=0x5616c9adbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:45.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90aaffd700 1 --2- 172.21.15.62:0/2895606344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9ac46c0 0x5616c9adbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90aaffd700 1 -- 172.21.15.62:0/2895606344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616c9adc870 con 0x5616c9abefa0 2024-06-10T00:42:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f90aaffd700 1 --2- 172.21.15.62:0/2895606344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9abefa0 0x5616c9adba70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f90a00096e0 tx=0x7f90a00327c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e081412789b7c457 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f908eff5700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.038+0000 7f908eff5700 1 -- 172.21.15.62:0/2895606344 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0044060 con 0x5616c9abefa0 2024-06-10T00:42:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.039+0000 7f908eff5700 1 -- 172.21.15.62:0/2895606344 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f90a003f430 con 0x5616c9abefa0 2024-06-10T00:42:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.039+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2895606344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a409f400 con 0x5616c9abefa0 2024-06-10T00:42:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.039+0000 7f908eff5700 1 -- 172.21.15.62:0/2895606344 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f90a002d050 con 0x5616c9abefa0 2024-06-10T00:42:45.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.039+0000 7f90bbd97b80 10 client.17046 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.039+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2895606344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5616c9b47d10 con 0x5616c9abefa0 2024-06-10T00:42:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.039+0000 7f908eff5700 1 -- 172.21.15.62:0/2895606344 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a409f400 con 0x5616c9abefa0 2024-06-10T00:42:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.039+0000 7f908eff5700 1 -- 172.21.15.62:0/2895606344 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f90a001e040 con 0x5616c9abefa0 2024-06-10T00:42:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.039+0000 7f90bbd97b80 10 client.17046 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.039+0000 7f90bbd97b80 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:42:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.039+0000 7f90bbd97b80 20 client.17046 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.039+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/2895606344 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5616c9b48390 0x5616c9b4a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.039+0000 7f90bbd97b80 4 client.17046 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T00:42:45.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.039+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2895606344 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5616c9a18310 con 0x5616c9b48390 2024-06-10T00:42:45.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.039+0000 7f90aa7fc700 1 --2- 172.21.15.62:0/2895606344 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5616c9b48390 0x5616c9b4a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:45.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.040+0000 7f90aa7fc700 1 --2- 172.21.15.62:0/2895606344 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5616c9b48390 0x5616c9b4a7b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55f273afe973ee0 server_cookie=bbfc357966476fa3 in_seq=0 out_seq=0 2024-06-10T00:42:45.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.040+0000 7f908eff5700 10 client.17046 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:45.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f908eff5700 1 -- 172.21.15.62:0/2895606344 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5616c9a18310 con 0x5616c9b48390 2024-06-10T00:42:45.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f908eff5700 10 client.17046 handle_command_reply: tid=0 2024-06-10T00:42:45.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f90bbd97b80 1 client.17046 shutdown 2024-06-10T00:42:45.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2895606344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9abefa0 msgr2=0x5616c9adba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:45.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/2895606344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9abefa0 0x5616c9adba70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f90a00096e0 tx=0x7f90a00327c0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2895606344 shutdown_connections 2024-06-10T00:42:45.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/2895606344 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5616c9b48390 0x5616c9b4a7b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/2895606344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616c9abefa0 0x5616c9adba70 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/2895606344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616c9ac4fe0 0x5616c9b46690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f908eff5700 0 client.17046 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:45.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f90bbd97b80 1 --2- 172.21.15.62:0/2895606344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616c9ac46c0 0x5616c9adbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2895606344 >> 172.21.15.62:0/2895606344 conn(0x5616c99e9d40 msgr2=0x5616c9ac3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:45.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2895606344 shutdown_connections 2024-06-10T00:42:45.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f90bbd97b80 1 -- 172.21.15.62:0/2895606344 wait complete. 2024-06-10T00:42:45.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.041+0000 7f90bbd97b80 20 client.17046 trim_cache size 0 max 16384 2024-06-10T00:42:45.051 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:42:45.051 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-06-10T00:42:45.051 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "57d759f9-2b8b-40f4-b2b5-b9b9a5898fb6", 2024-06-10T00:42:45.051 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-06-10T00:42:45.052 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:42:45.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.042+0000 7f90b1514700 1 -- 172.21.15.62:0/2145462230 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f90940a2d90 msgr2=0x7f90940a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:45.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.042+0000 7f90b1514700 1 --2- 172.21.15.62:0/2145462230 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f90940a2d90 0x7f90940a5250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009a10 tx=0x7f90a0009190 comp rx=0 tx=0).stop 2024-06-10T00:42:45.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.042+0000 7f90b1514700 1 -- 172.21.15.62:0/2145462230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac108500 msgr2=0x7f90ac19d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:45.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.042+0000 7f90b1514700 1 --2- 172.21.15.62:0/2145462230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac108500 0x7f90ac19d600 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f90a4007ee0 tx=0x7f90a400d3b0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.042+0000 7f90b1514700 1 -- 172.21.15.62:0/2145462230 shutdown_connections 2024-06-10T00:42:45.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.042+0000 7f90b1514700 1 --2- 172.21.15.62:0/2145462230 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f90940a2d90 0x7f90940a5250 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.042+0000 7f90b1514700 1 --2- 172.21.15.62:0/2145462230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ac108500 0x7f90ac19d600 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.042+0000 7f90b1514700 1 --2- 172.21.15.62:0/2145462230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90ac06abc0 0x7f90ac198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.042+0000 7f90b1514700 1 --2- 172.21.15.62:0/2145462230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ac06b580 0x7f90ac199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:45.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.042+0000 7f90b1514700 1 -- 172.21.15.62:0/2145462230 >> 172.21.15.62:0/2145462230 conn(0x7f90ac078340 msgr2=0x7f90ac1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:45.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.042+0000 7f90b1514700 1 -- 172.21.15.62:0/2145462230 shutdown_connections 2024-06-10T00:42:45.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:45.042+0000 7f90b1514700 1 -- 172.21.15.62:0/2145462230 wait complete. 2024-06-10T00:42:45.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:45 smithi062 ceph-mon[32390]: pgmap v306: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:42:45.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:45 smithi062 ceph-mon[32390]: scrub queued for path: / 2024-06-10T00:42:45.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:45 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:45.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:45 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:42:45.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:45 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:42:45.359 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:45 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:45.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:45 smithi105 ceph-mon[36321]: pgmap v306: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:42:45.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:45 smithi105 ceph-mon[36321]: scrub queued for path: / 2024-06-10T00:42:45.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:45 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:45.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:45 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:42:45.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:45 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:42:45.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:45 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:45.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:45 smithi138 ceph-mon[36549]: pgmap v306: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:42:45.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:45 smithi138 ceph-mon[36549]: scrub queued for path: / 2024-06-10T00:42:45.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:45 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:45.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:45 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:42:45.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:45 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:42:45.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:45 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:45.744 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:42:45.994 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:42:46.418 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:46 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:42:46.418 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:46 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:42:46.418 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:46 smithi062 ceph-mon[32390]: scrub summary: active paths [/] 2024-06-10T00:42:46.418 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:46 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:46.419 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:46 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:46.419 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:46 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:46.419 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:46 smithi062 ceph-mon[32390]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:46.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:46 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:42:46.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:46 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:42:46.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:46 smithi105 ceph-mon[36321]: scrub summary: active paths [/] 2024-06-10T00:42:46.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:46 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:46.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:46 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:46.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:46 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:46.533 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:46 smithi105 ceph-mon[36321]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:46.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:46 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:42:46.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:46 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:42:46.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:46 smithi138 ceph-mon[36549]: scrub summary: active paths [/] 2024-06-10T00:42:46.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:46 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:46.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:46 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:46.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:46 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:46.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:46 smithi138 ceph-mon[36549]: scrub summary: idle+waiting paths [/] 2024-06-10T00:42:47.456 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:47 smithi062 ceph-mon[32390]: pgmap v307: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-10T00:42:47.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:47 smithi105 ceph-mon[36321]: pgmap v307: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-10T00:42:47.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:47 smithi138 ceph-mon[36549]: pgmap v307: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-10T00:42:47.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.686+0000 7f21e4bd2700 1 -- 172.21.15.62:0/2008758769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e0107400 msgr2=0x7f21e01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:47.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.686+0000 7f21e4bd2700 1 --2- 172.21.15.62:0/2008758769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e0107400 0x7f21e01077e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f21d000b0a0 tx=0x7f21d000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.686+0000 7f21e4bd2700 1 -- 172.21.15.62:0/2008758769 shutdown_connections 2024-06-10T00:42:47.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.686+0000 7f21e4bd2700 1 --2- 172.21.15.62:0/2008758769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e0107400 0x7f21e01077e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.686+0000 7f21e4bd2700 1 --2- 172.21.15.62:0/2008758769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e0101d50 0x7f21e010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.686+0000 7f21e4bd2700 1 --2- 172.21.15.62:0/2008758769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21e0101390 0x7f21e0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.686+0000 7f21e4bd2700 1 -- 172.21.15.62:0/2008758769 >> 172.21.15.62:0/2008758769 conn(0x7f21e0078340 msgr2=0x7f21e0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:47.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.686+0000 7f21e4bd2700 1 -- 172.21.15.62:0/2008758769 shutdown_connections 2024-06-10T00:42:47.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.686+0000 7f21e4bd2700 1 -- 172.21.15.62:0/2008758769 wait complete. 2024-06-10T00:42:47.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.687+0000 7f21e4bd2700 1 Processor -- start 2024-06-10T00:42:47.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.687+0000 7f21e4bd2700 1 -- start start 2024-06-10T00:42:47.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.687+0000 7f21e4bd2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21e0101390 0x7f21e019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.687+0000 7f21e4bd2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e0101d50 0x7f21e019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.687+0000 7f21e4bd2700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e0107400 0x7f21e0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.687+0000 7f21e4bd2700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01116f0 con 0x7f21e0107400 2024-06-10T00:42:47.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.687+0000 7f21e4bd2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f21e0111570 con 0x7f21e0101390 2024-06-10T00:42:47.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.687+0000 7f21e4bd2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21e0111870 con 0x7f21e0101d50 2024-06-10T00:42:47.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21ded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e0107400 0x7f21e0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21ded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e0107400 0x7f21e0198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43922/0 (socket says 172.21.15.62:43922) 2024-06-10T00:42:47.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21ded9d700 1 -- 172.21.15.62:0/705902345 learned_addr learned my addr 172.21.15.62:0/705902345 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:47.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21ddd9b700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e0101d50 0x7f21e019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21de59c700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21e0101390 0x7f21e019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21ded9d700 1 -- 172.21.15.62:0/705902345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e0101d50 msgr2=0x7f21e019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:47.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21ded9d700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e0101d50 0x7f21e019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21ded9d700 1 -- 172.21.15.62:0/705902345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21e0101390 msgr2=0x7f21e019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:47.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21ded9d700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21e0101390 0x7f21e019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21ded9d700 1 -- 172.21.15.62:0/705902345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e0199250 con 0x7f21e0107400 2024-06-10T00:42:47.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21de59c700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21e0101390 0x7f21e019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:47.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21ded9d700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e0107400 0x7f21e0198af0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f21d800e940 tx=0x7f21d800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:47.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21d77fe700 1 -- 172.21.15.62:0/705902345 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d800cbd0 con 0x7f21e0107400 2024-06-10T00:42:47.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21d77fe700 1 -- 172.21.15.62:0/705902345 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d800ee90 con 0x7f21e0107400 2024-06-10T00:42:47.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21e4bd2700 1 -- 172.21.15.62:0/705902345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e0199540 con 0x7f21e0107400 2024-06-10T00:42:47.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21d77fe700 1 -- 172.21.15.62:0/705902345 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d80056c0 con 0x7f21e0107400 2024-06-10T00:42:47.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.688+0000 7f21e4bd2700 1 -- 172.21.15.62:0/705902345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e01a57b0 con 0x7f21e0107400 2024-06-10T00:42:47.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.691+0000 7f21d77fe700 1 -- 172.21.15.62:0/705902345 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f21d8004750 con 0x7f21e0107400 2024-06-10T00:42:47.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.691+0000 7f21d77fe700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f21cc0a2d90 0x7f21cc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.691+0000 7f21de59c700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f21cc0a2d90 0x7f21cc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.692+0000 7f21d77fe700 1 -- 172.21.15.62:0/705902345 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f21d80cfab0 con 0x7f21e0107400 2024-06-10T00:42:47.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.692+0000 7f21de59c700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f21cc0a2d90 0x7f21cc0a5250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f21d0006130 tx=0x7f21d0006080 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:47.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.692+0000 7f21ef455b80 1 Processor -- start 2024-06-10T00:42:47.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.692+0000 7f21ef455b80 1 -- start start 2024-06-10T00:42:47.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.692+0000 7f21ef455b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9376370 0x55e2c9376750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.692+0000 7f21ef455b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376c90 0x55e2c9370670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.692+0000 7f21ef455b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9370bb0 0x55e2c9370fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.692+0000 7f21ef455b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e2c925ef60 con 0x55e2c9370bb0 2024-06-10T00:42:47.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.692+0000 7f21ef455b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e2c92bd850 con 0x55e2c9376c90 2024-06-10T00:42:47.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21ef455b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e2c9225460 con 0x55e2c9376370 2024-06-10T00:42:47.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21ded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9370bb0 0x55e2c9370fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21ded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9370bb0 0x55e2c9370fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43934/0 (socket says 172.21.15.62:43934) 2024-06-10T00:42:47.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21ded9d700 1 -- 172.21.15.62:0/4046652453 learned_addr learned my addr 172.21.15.62:0/4046652453 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:47.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21de59c700 1 --2- 172.21.15.62:0/4046652453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9376370 0x55e2c9376750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21ddd9b700 1 --2- 172.21.15.62:0/4046652453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376c90 0x55e2c9370670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21ded9d700 1 -- 172.21.15.62:0/4046652453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9376370 msgr2=0x55e2c9376750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:47.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21ded9d700 1 --2- 172.21.15.62:0/4046652453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9376370 0x55e2c9376750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21ded9d700 1 -- 172.21.15.62:0/4046652453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376c90 msgr2=0x55e2c9370670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:47.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21ded9d700 1 --2- 172.21.15.62:0/4046652453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376c90 0x55e2c9370670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21ded9d700 1 -- 172.21.15.62:0/4046652453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2c920c8d0 con 0x55e2c9370bb0 2024-06-10T00:42:47.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21ded9d700 1 --2- 172.21.15.62:0/4046652453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9370bb0 0x55e2c9370fd0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f21d8098020 tx=0x7f21d809c2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0c5d34ed48d7a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:47.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21d4ff9700 1 -- 172.21.15.62:0/4046652453 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d80a4830 con 0x55e2c9370bb0 2024-06-10T00:42:47.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21d4ff9700 1 -- 172.21.15.62:0/4046652453 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d8099ec0 con 0x55e2c9370bb0 2024-06-10T00:42:47.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.693+0000 7f21d4ff9700 1 -- 172.21.15.62:0/4046652453 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d80a4600 con 0x55e2c9370bb0 2024-06-10T00:42:47.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.694+0000 7f21ef455b80 1 -- 172.21.15.62:0/4046652453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9370bb0 msgr2=0x55e2c9370fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:47.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.694+0000 7f21ef455b80 1 --2- 172.21.15.62:0/4046652453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9370bb0 0x55e2c9370fd0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f21d8098020 tx=0x7f21d809c2c0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.694+0000 7f21ef455b80 1 -- 172.21.15.62:0/4046652453 shutdown_connections 2024-06-10T00:42:47.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.694+0000 7f21ef455b80 1 --2- 172.21.15.62:0/4046652453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9370bb0 0x55e2c9370fd0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.694+0000 7f21ef455b80 1 --2- 172.21.15.62:0/4046652453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9376370 0x55e2c9376750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.694+0000 7f21ef455b80 1 --2- 172.21.15.62:0/4046652453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376c90 0x55e2c9370670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.694+0000 7f21ef455b80 1 -- 172.21.15.62:0/4046652453 >> 172.21.15.62:0/4046652453 conn(0x55e2c9299080 msgr2=0x55e2c9367220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:47.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.694+0000 7f21ef455b80 1 -- 172.21.15.62:0/4046652453 shutdown_connections 2024-06-10T00:42:47.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.694+0000 7f21ef455b80 1 -- 172.21.15.62:0/4046652453 wait complete. 2024-06-10T00:42:47.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.695+0000 7f21ef455b80 1 -- start start 2024-06-10T00:42:47.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.695+0000 7f21ef455b80 1 Processor -- start 2024-06-10T00:42:47.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.695+0000 7f21ef455b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9370bb0 0x55e2c937cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.695+0000 7f21ef455b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376370 0x55e2c9379db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.695+0000 7f21ef455b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9376c90 0x55e2c937a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.695+0000 7f21ef455b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e2c925dfc0 con 0x55e2c9370bb0 2024-06-10T00:42:47.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.695+0000 7f21ef455b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e2c92b9810 con 0x55e2c9376370 2024-06-10T00:42:47.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.695+0000 7f21ef455b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e2c937d670 con 0x55e2c9376c90 2024-06-10T00:42:47.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.695+0000 7f21ef455b80 10 client.0 _ll_register_callbacks cb 0x55e2c92535b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:47.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.695+0000 7f21de59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9370bb0 0x55e2c937cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.695+0000 7f21de59c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9370bb0 0x55e2c937cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43942/0 (socket says 172.21.15.62:43942) 2024-06-10T00:42:47.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.695+0000 7f21de59c700 1 -- 172.21.15.62:0/485989982 learned_addr learned my addr 172.21.15.62:0/485989982 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:47.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.695+0000 7f21ddd9b700 1 --2- 172.21.15.62:0/485989982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376370 0x55e2c9379db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.696+0000 7f21ded9d700 1 --2- 172.21.15.62:0/485989982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9376c90 0x55e2c937a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.696+0000 7f21de59c700 1 -- 172.21.15.62:0/485989982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9376c90 msgr2=0x55e2c937a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:47.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.696+0000 7f21de59c700 1 --2- 172.21.15.62:0/485989982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9376c90 0x55e2c937a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.696+0000 7f21de59c700 1 -- 172.21.15.62:0/485989982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376370 msgr2=0x55e2c9379db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:47.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.696+0000 7f21de59c700 1 --2- 172.21.15.62:0/485989982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376370 0x55e2c9379db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.696+0000 7f21de59c700 1 -- 172.21.15.62:0/485989982 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2c9283750 con 0x55e2c9370bb0 2024-06-10T00:42:47.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.696+0000 7f21de59c700 1 --2- 172.21.15.62:0/485989982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9370bb0 0x55e2c937cab0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f21d000b0a0 tx=0x7f21d00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4095928435372f00 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:47.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.696+0000 7f21c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:47.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.696+0000 7f21c5ffb700 1 -- 172.21.15.62:0/485989982 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d002e030 con 0x55e2c9370bb0 2024-06-10T00:42:47.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.696+0000 7f21c5ffb700 1 -- 172.21.15.62:0/485989982 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f21d001c440 con 0x55e2c9370bb0 2024-06-10T00:42:47.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.696+0000 7f21ef455b80 1 -- 172.21.15.62:0/485989982 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21d8099ec0 con 0x55e2c9370bb0 2024-06-10T00:42:47.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.696+0000 7f21c5ffb700 1 -- 172.21.15.62:0/485989982 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f21d003b050 con 0x55e2c9370bb0 2024-06-10T00:42:47.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.696+0000 7f21ef455b80 10 client.17058 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:47.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.696+0000 7f21ef455b80 1 -- 172.21.15.62:0/485989982 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e2c9387eb0 con 0x55e2c9370bb0 2024-06-10T00:42:47.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.697+0000 7f21c5ffb700 1 -- 172.21.15.62:0/485989982 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d8099ec0 con 0x55e2c9370bb0 2024-06-10T00:42:47.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.697+0000 7f21c5ffb700 1 -- 172.21.15.62:0/485989982 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f21d0011070 con 0x55e2c9370bb0 2024-06-10T00:42:47.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.697+0000 7f21ef455b80 10 client.17058 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:47.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.697+0000 7f21ef455b80 10 client.17058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:42:47.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.697+0000 7f21ef455b80 20 client.17058 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:47.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.697+0000 7f21ef455b80 1 --2- 172.21.15.62:0/485989982 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55e2c938a7c0 0x55e2c938cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.697+0000 7f21ef455b80 4 client.17058 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:42:47.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.697+0000 7f21ef455b80 1 -- 172.21.15.62:0/485989982 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2c92cad50 con 0x55e2c938a7c0 2024-06-10T00:42:47.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.697+0000 7f21ddd9b700 1 --2- 172.21.15.62:0/485989982 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55e2c938a7c0 0x55e2c938cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.697+0000 7f21ddd9b700 1 --2- 172.21.15.62:0/485989982 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55e2c938a7c0 0x55e2c938cbb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42d3a1919cdb73b4 server_cookie=277b11ed63c062f3 in_seq=0 out_seq=0 2024-06-10T00:42:47.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.697+0000 7f21c5ffb700 10 client.17058 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:47.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.698+0000 7f21c5ffb700 1 -- 172.21.15.62:0/485989982 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e2c92cad50 con 0x55e2c938a7c0 2024-06-10T00:42:47.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.698+0000 7f21c5ffb700 10 client.17058 handle_command_reply: tid=0 2024-06-10T00:42:47.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.698+0000 7f21ef455b80 1 client.17058 shutdown 2024-06-10T00:42:47.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.698+0000 7f21ef455b80 1 -- 172.21.15.62:0/485989982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9370bb0 msgr2=0x55e2c937cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:47.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.698+0000 7f21ef455b80 1 --2- 172.21.15.62:0/485989982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9370bb0 0x55e2c937cab0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f21d000b0a0 tx=0x7f21d00099e0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.698+0000 7f21ef455b80 1 -- 172.21.15.62:0/485989982 shutdown_connections 2024-06-10T00:42:47.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.698+0000 7f21ef455b80 1 --2- 172.21.15.62:0/485989982 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55e2c938a7c0 0x55e2c938cbb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.698+0000 7f21ef455b80 1 --2- 172.21.15.62:0/485989982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9370bb0 0x55e2c937cab0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.698+0000 7f21ef455b80 1 --2- 172.21.15.62:0/485989982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9376c90 0x55e2c937a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.698+0000 7f21ef455b80 1 --2- 172.21.15.62:0/485989982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376370 0x55e2c9379db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.698+0000 7f21ef455b80 1 -- 172.21.15.62:0/485989982 >> 172.21.15.62:0/485989982 conn(0x55e2c9299080 msgr2=0x55e2c9366700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:47.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.698+0000 7f21c5ffb700 0 client.17058 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:47.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.698+0000 7f21ef455b80 1 -- 172.21.15.62:0/485989982 shutdown_connections 2024-06-10T00:42:47.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.699+0000 7f21ef455b80 1 -- 172.21.15.62:0/485989982 wait complete. 2024-06-10T00:42:47.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.699+0000 7f21ef455b80 20 client.17058 trim_cache size 0 max 16384 2024-06-10T00:42:47.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.719+0000 7f21ef455b80 1 Processor -- start 2024-06-10T00:42:47.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.719+0000 7f21ef455b80 1 -- start start 2024-06-10T00:42:47.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.719+0000 7f21ef455b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9370bb0 0x55e2c9386790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.719+0000 7f21ef455b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9376370 0x55e2c937c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.719+0000 7f21ef455b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376c90 0x55e2c9366e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.719+0000 7f21ef455b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01ac4d0 con 0x55e2c9376370 2024-06-10T00:42:47.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.719+0000 7f21ef455b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01ae010 con 0x55e2c9376c90 2024-06-10T00:42:47.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.719+0000 7f21ef455b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01acba0 con 0x55e2c9370bb0 2024-06-10T00:42:47.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.719+0000 7f21ddd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9376370 0x55e2c937c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.719+0000 7f21ddd9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9376370 0x55e2c937c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43954/0 (socket says 172.21.15.62:43954) 2024-06-10T00:42:47.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.719+0000 7f21ddd9b700 1 -- 172.21.15.62:0/380240070 learned_addr learned my addr 172.21.15.62:0/380240070 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:47.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.720+0000 7f21de59c700 1 --2- 172.21.15.62:0/380240070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9370bb0 0x55e2c9386790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.720+0000 7f21ddd9b700 1 -- 172.21.15.62:0/380240070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9370bb0 msgr2=0x55e2c9386790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:47.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.720+0000 7f21ded9d700 1 --2- 172.21.15.62:0/380240070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376c90 0x55e2c9366e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.720+0000 7f21ddd9b700 1 --2- 172.21.15.62:0/380240070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9370bb0 0x55e2c9386790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.720+0000 7f21ddd9b700 1 -- 172.21.15.62:0/380240070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376c90 msgr2=0x55e2c9366e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:47.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.720+0000 7f21ddd9b700 1 --2- 172.21.15.62:0/380240070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376c90 0x55e2c9366e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.720+0000 7f21ddd9b700 1 -- 172.21.15.62:0/380240070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2c9378320 con 0x55e2c9376370 2024-06-10T00:42:47.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.720+0000 7f21ddd9b700 1 --2- 172.21.15.62:0/380240070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9376370 0x55e2c937c5f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009d70 tx=0x7f21c8009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f7c43adb0af944 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:47.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.720+0000 7f21c7fff700 1 -- 172.21.15.62:0/380240070 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c8025070 con 0x55e2c9376370 2024-06-10T00:42:47.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.720+0000 7f21c7fff700 1 -- 172.21.15.62:0/380240070 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c8015d70 con 0x55e2c9376370 2024-06-10T00:42:47.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.720+0000 7f21c7fff700 1 -- 172.21.15.62:0/380240070 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c801ee10 con 0x55e2c9376370 2024-06-10T00:42:47.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.721+0000 7f21ef455b80 1 -- 172.21.15.62:0/380240070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9376370 msgr2=0x55e2c937c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:47.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.721+0000 7f21ef455b80 1 --2- 172.21.15.62:0/380240070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9376370 0x55e2c937c5f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009d70 tx=0x7f21c8009190 comp rx=0 tx=0).stop 2024-06-10T00:42:47.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.721+0000 7f21ef455b80 1 -- 172.21.15.62:0/380240070 shutdown_connections 2024-06-10T00:42:47.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.721+0000 7f21ef455b80 1 --2- 172.21.15.62:0/380240070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376c90 0x55e2c9366e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.721+0000 7f21ef455b80 1 --2- 172.21.15.62:0/380240070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9370bb0 0x55e2c9386790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.721+0000 7f21ef455b80 1 --2- 172.21.15.62:0/380240070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9376370 0x55e2c937c5f0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.721+0000 7f21ef455b80 1 -- 172.21.15.62:0/380240070 >> 172.21.15.62:0/380240070 conn(0x55e2c9299080 msgr2=0x55e2c9375720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:47.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.721+0000 7f21ef455b80 1 -- 172.21.15.62:0/380240070 shutdown_connections 2024-06-10T00:42:47.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.721+0000 7f21ef455b80 1 -- 172.21.15.62:0/380240070 wait complete. 2024-06-10T00:42:47.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ef455b80 1 -- start start 2024-06-10T00:42:47.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ef455b80 1 Processor -- start 2024-06-10T00:42:47.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ef455b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9370bb0 0x55e2c9382ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ef455b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376370 0x55e2c9383220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ef455b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9376c90 0x55e2c9391960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ef455b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01ad940 con 0x55e2c9376c90 2024-06-10T00:42:47.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ef455b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01ad270 con 0x55e2c9376370 2024-06-10T00:42:47.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ef455b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01aa490 con 0x55e2c9370bb0 2024-06-10T00:42:47.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ef455b80 10 client.0 _ll_register_callbacks cb 0x55e2c92535b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:42:47.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ded9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9376c90 0x55e2c9391960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ddd9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376370 0x55e2c9383220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ddd9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376370 0x55e2c9383220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:33400/0 (socket says 172.21.15.62:33400) 2024-06-10T00:42:47.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ddd9b700 1 -- 172.21.15.62:0/3872694904 learned_addr learned my addr 172.21.15.62:0/3872694904 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:42:47.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21de59c700 1 --2- 172.21.15.62:0/3872694904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9370bb0 0x55e2c9382ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ded9d700 1 -- 172.21.15.62:0/3872694904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9370bb0 msgr2=0x55e2c9382ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:47.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ded9d700 1 --2- 172.21.15.62:0/3872694904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9370bb0 0x55e2c9382ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.722+0000 7f21ded9d700 1 -- 172.21.15.62:0/3872694904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376370 msgr2=0x55e2c9383220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:42:47.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.723+0000 7f21ded9d700 1 --2- 172.21.15.62:0/3872694904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376370 0x55e2c9383220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.723+0000 7f21ded9d700 1 -- 172.21.15.62:0/3872694904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2c93921a0 con 0x55e2c9376c90 2024-06-10T00:42:47.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.723+0000 7f21de59c700 1 --2- 172.21.15.62:0/3872694904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9370bb0 0x55e2c9382ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:42:47.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.723+0000 7f21ddd9b700 1 --2- 172.21.15.62:0/3872694904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376370 0x55e2c9383220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:42:47.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.723+0000 7f21ded9d700 1 --2- 172.21.15.62:0/3872694904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9376c90 0x55e2c9391960 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f21d800c5b0 tx=0x7f21d800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d441c2fd01d158c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:42:47.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.723+0000 7f21c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:42:47.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.723+0000 7f21c5ffb700 1 -- 172.21.15.62:0/3872694904 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d80ba050 con 0x55e2c9376c90 2024-06-10T00:42:47.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.723+0000 7f21c5ffb700 1 -- 172.21.15.62:0/3872694904 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f21d809be20 con 0x55e2c9376c90 2024-06-10T00:42:47.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.723+0000 7f21ef455b80 1 -- 172.21.15.62:0/3872694904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21c8015d70 con 0x55e2c9376c90 2024-06-10T00:42:47.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.723+0000 7f21c5ffb700 1 -- 172.21.15.62:0/3872694904 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f21d80a4830 con 0x55e2c9376c90 2024-06-10T00:42:47.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.723+0000 7f21ef455b80 10 client.17064 fetch_fsmap learned FSMap version 38 2024-06-10T00:42:47.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.723+0000 7f21ef455b80 1 -- 172.21.15.62:0/3872694904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e2c93f4e20 con 0x55e2c9376c90 2024-06-10T00:42:47.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.723+0000 7f21c5ffb700 1 -- 172.21.15.62:0/3872694904 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c8015d70 con 0x55e2c9376c90 2024-06-10T00:42:47.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.724+0000 7f21c5ffb700 1 -- 172.21.15.62:0/3872694904 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f21d809b010 con 0x55e2c9376c90 2024-06-10T00:42:47.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.724+0000 7f21ef455b80 10 client.17064 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:42:47.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.724+0000 7f21ef455b80 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:42:47.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.724+0000 7f21ef455b80 20 client.17064 populate_metadata read hostname 'smithi062' 2024-06-10T00:42:47.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.724+0000 7f21ef455b80 1 --2- 172.21.15.62:0/3872694904 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55e2c93f50a0 0x55e2c93f7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:42:47.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.724+0000 7f21ef455b80 4 client.17064 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:42:47.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.724+0000 7f21ef455b80 1 -- 172.21.15.62:0/3872694904 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e2c92c52d0 con 0x55e2c93f50a0 2024-06-10T00:42:47.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.724+0000 7f21de59c700 1 --2- 172.21.15.62:0/3872694904 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55e2c93f50a0 0x55e2c93f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:42:47.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.724+0000 7f21c5ffb700 10 client.17064 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:47.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.724+0000 7f21de59c700 1 --2- 172.21.15.62:0/3872694904 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55e2c93f50a0 0x55e2c93f7490 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a77cfa515e3a8767 server_cookie=23ffcfd9cf4f0998 in_seq=0 out_seq=0 2024-06-10T00:42:47.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21c5ffb700 1 -- 172.21.15.62:0/3872694904 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e2c92c52d0 con 0x55e2c93f50a0 2024-06-10T00:42:47.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21c5ffb700 10 client.17064 handle_command_reply: tid=0 2024-06-10T00:42:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21ef455b80 1 client.17064 shutdown 2024-06-10T00:42:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21ef455b80 1 -- 172.21.15.62:0/3872694904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9376c90 msgr2=0x55e2c9391960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21ef455b80 1 --2- 172.21.15.62:0/3872694904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9376c90 0x55e2c9391960 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f21d800c5b0 tx=0x7f21d800cf90 comp rx=0 tx=0).stop 2024-06-10T00:42:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21ef455b80 1 -- 172.21.15.62:0/3872694904 shutdown_connections 2024-06-10T00:42:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21ef455b80 1 --2- 172.21.15.62:0/3872694904 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x55e2c93f50a0 0x55e2c93f7490 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21ef455b80 1 --2- 172.21.15.62:0/3872694904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2c9376c90 0x55e2c9391960 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21ef455b80 1 --2- 172.21.15.62:0/3872694904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2c9370bb0 0x55e2c9382ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21ef455b80 1 --2- 172.21.15.62:0/3872694904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e2c9376370 0x55e2c9383220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21c5ffb700 0 client.17064 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:42:47.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21ef455b80 1 -- 172.21.15.62:0/3872694904 >> 172.21.15.62:0/3872694904 conn(0x55e2c9299080 msgr2=0x55e2c9374e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:47.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21ef455b80 1 -- 172.21.15.62:0/3872694904 shutdown_connections 2024-06-10T00:42:47.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21ef455b80 1 -- 172.21.15.62:0/3872694904 wait complete. 2024-06-10T00:42:47.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.725+0000 7f21ef455b80 20 client.17064 trim_cache size 0 max 16384 2024-06-10T00:42:47.734 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:42:47.735 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T00:42:47.735 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-06-10T00:42:47.735 INFO:teuthology.orchestra.run.smithi062.stdout: "57d759f9-2b8b-40f4-b2b5-b9b9a5898fb6": { 2024-06-10T00:42:47.735 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-06-10T00:42:47.735 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "57d759f9-2b8b-40f4-b2b5-b9b9a5898fb6", 2024-06-10T00:42:47.735 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,force" 2024-06-10T00:42:47.735 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:42:47.735 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:42:47.736 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:42:47.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.726+0000 7f21e4bd2700 1 -- 172.21.15.62:0/705902345 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f21cc0a2d90 msgr2=0x7f21cc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:47.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.726+0000 7f21e4bd2700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f21cc0a2d90 0x7f21cc0a5250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f21d0006130 tx=0x7f21d0006080 comp rx=0 tx=0).stop 2024-06-10T00:42:47.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.726+0000 7f21e4bd2700 1 -- 172.21.15.62:0/705902345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e0107400 msgr2=0x7f21e0198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:42:47.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.726+0000 7f21e4bd2700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e0107400 0x7f21e0198af0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f21d800e940 tx=0x7f21d800ec50 comp rx=0 tx=0).stop 2024-06-10T00:42:47.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.726+0000 7f21e4bd2700 1 -- 172.21.15.62:0/705902345 shutdown_connections 2024-06-10T00:42:47.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.726+0000 7f21e4bd2700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f21cc0a2d90 0x7f21cc0a5250 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.726+0000 7f21e4bd2700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21e0107400 0x7f21e0198af0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.726+0000 7f21e4bd2700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21e0101390 0x7f21e019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.726+0000 7f21e4bd2700 1 --2- 172.21.15.62:0/705902345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e0101d50 0x7f21e019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:42:47.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.726+0000 7f21e4bd2700 1 -- 172.21.15.62:0/705902345 >> 172.21.15.62:0/705902345 conn(0x7f21e0078340 msgr2=0x7f21e01006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:42:47.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.726+0000 7f21e4bd2700 1 -- 172.21.15.62:0/705902345 shutdown_connections 2024-06-10T00:42:47.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:42:47.726+0000 7f21e4bd2700 1 -- 172.21.15.62:0/705902345 wait complete. 2024-06-10T00:42:48.397 INFO:tasks.cephfs.filesystem:scrub status for tag:57d759f9-2b8b-40f4-b2b5-b9b9a5898fb6 - {'path': '/', 'tag': '57d759f9-2b8b-40f4-b2b5-b9b9a5898fb6', 'options': 'recursive,force'} 2024-06-10T00:42:49.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:49 smithi105 ceph-mon[36321]: pgmap v308: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:42:49.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:49 smithi062 ceph-mon[32390]: pgmap v308: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:42:49.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:49 smithi138 ceph-mon[36549]: pgmap v308: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:42:51.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:51 smithi105 ceph-mon[36321]: pgmap v309: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:42:51.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:51 smithi062 ceph-mon[32390]: pgmap v309: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:42:51.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:51 smithi138 ceph-mon[36549]: pgmap v309: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:42:52.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:52 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:42:52.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:52 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:42:52.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:52 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:42:52.619 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:52 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:42:52.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:52 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:42:52.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:52 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:42:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:53 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:42:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:53 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:42:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:53 smithi105 ceph-mon[36321]: scrub summary: idle 2024-06-10T00:42:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:53 smithi105 ceph-mon[36321]: pgmap v310: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-06-10T00:42:53.610 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:53 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:42:53.610 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:53 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:42:53.611 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:53 smithi062 ceph-mon[32390]: scrub summary: idle 2024-06-10T00:42:53.611 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:53 smithi062 ceph-mon[32390]: pgmap v310: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-06-10T00:42:53.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:53 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:42:53.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:53 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:42:53.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:53 smithi138 ceph-mon[36549]: scrub summary: idle 2024-06-10T00:42:53.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:53 smithi138 ceph-mon[36549]: pgmap v310: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 op/s 2024-06-10T00:42:55.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:55 smithi105 ceph-mon[36321]: pgmap v311: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:42:55.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:55 smithi062 ceph-mon[32390]: pgmap v311: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:42:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:55 smithi138 ceph-mon[36549]: pgmap v311: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:42:57.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:57 smithi105 ceph-mon[36321]: pgmap v312: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-10T00:42:57.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:57 smithi062 ceph-mon[32390]: pgmap v312: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-10T00:42:57.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:57 smithi138 ceph-mon[36549]: pgmap v312: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-10T00:42:59.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:42:59 smithi105 ceph-mon[36321]: pgmap v313: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:42:59.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:42:59 smithi062 ceph-mon[32390]: pgmap v313: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:42:59.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:42:59 smithi138 ceph-mon[36549]: pgmap v313: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:43:01.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:01 smithi105 ceph-mon[36321]: pgmap v314: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:43:01.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:01 smithi062 ceph-mon[32390]: pgmap v314: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:43:01.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:01 smithi138 ceph-mon[36549]: pgmap v314: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-06-10T00:43:03.378 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-06-10T00:43:03.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.501+0000 7f1a78d9e640 1 -- 172.21.15.62:0/4283767009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a741066a0 msgr2=0x7f1a74106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:03.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.501+0000 7f1a78d9e640 1 --2- 172.21.15.62:0/4283767009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a741066a0 0x7f1a74106a80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009930 tx=0x7f1a6402a420 comp rx=0 tx=0).stop 2024-06-10T00:43:03.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.501+0000 7f1a78d9e640 1 -- 172.21.15.62:0/4283767009 shutdown_connections 2024-06-10T00:43:03.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.501+0000 7f1a78d9e640 1 --2- 172.21.15.62:0/4283767009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a74101050 0x7f1a7410d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.501+0000 7f1a78d9e640 1 --2- 172.21.15.62:0/4283767009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a74100690 0x7f1a74100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.501+0000 7f1a78d9e640 1 --2- 172.21.15.62:0/4283767009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a741066a0 0x7f1a74106a80 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.501+0000 7f1a78d9e640 1 -- 172.21.15.62:0/4283767009 >> 172.21.15.62:0/4283767009 conn(0x7f1a740fc1c0 msgr2=0x7f1a740fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:03.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.501+0000 7f1a78d9e640 1 -- 172.21.15.62:0/4283767009 shutdown_connections 2024-06-10T00:43:03.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.501+0000 7f1a78d9e640 1 -- 172.21.15.62:0/4283767009 wait complete. 2024-06-10T00:43:03.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.503+0000 7f1a78d9e640 1 Processor -- start 2024-06-10T00:43:03.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.503+0000 7f1a78d9e640 1 -- start start 2024-06-10T00:43:03.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.503+0000 7f1a78d9e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a74100690 0x7f1a74196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:03.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.503+0000 7f1a78d9e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a74101050 0x7f1a74197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:03.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.503+0000 7f1a78d9e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a741066a0 0x7f1a7419b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:03.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.503+0000 7f1a78d9e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1a74110a70 con 0x7f1a74101050 2024-06-10T00:43:03.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.503+0000 7f1a78d9e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a741108f0 con 0x7f1a741066a0 2024-06-10T00:43:03.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.503+0000 7f1a78d9e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a74110bf0 con 0x7f1a74100690 2024-06-10T00:43:03.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a72ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a74101050 0x7f1a74197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:03.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a72ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a74101050 0x7f1a74197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44656/0 (socket says 172.21.15.62:44656) 2024-06-10T00:43:03.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a72ffd640 1 -- 172.21.15.62:0/2948388497 learned_addr learned my addr 172.21.15.62:0/2948388497 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:03.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a737fe640 1 --2- 172.21.15.62:0/2948388497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a74100690 0x7f1a74196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:03.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a73fff640 1 --2- 172.21.15.62:0/2948388497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a741066a0 0x7f1a7419b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:03.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a72ffd640 1 -- 172.21.15.62:0/2948388497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a74100690 msgr2=0x7f1a74196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:03.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a72ffd640 1 --2- 172.21.15.62:0/2948388497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a74100690 0x7f1a74196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a72ffd640 1 -- 172.21.15.62:0/2948388497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a741066a0 msgr2=0x7f1a7419b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:03.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a72ffd640 1 --2- 172.21.15.62:0/2948388497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a741066a0 0x7f1a7419b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a72ffd640 1 -- 172.21.15.62:0/2948388497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7419bfc0 con 0x7f1a74101050 2024-06-10T00:43:03.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a72ffd640 1 --2- 172.21.15.62:0/2948388497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a74101050 0x7f1a74197510 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800d950 tx=0x7f1a6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:03.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a70ff9640 1 -- 172.21.15.62:0/2948388497 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68017070 con 0x7f1a74101050 2024-06-10T00:43:03.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a70ff9640 1 -- 172.21.15.62:0/2948388497 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a68009c30 con 0x7f1a74101050 2024-06-10T00:43:03.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a78d9e640 1 -- 172.21.15.62:0/2948388497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7419c2b0 con 0x7f1a74101050 2024-06-10T00:43:03.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.504+0000 7f1a70ff9640 1 -- 172.21.15.62:0/2948388497 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6800f6b0 con 0x7f1a74101050 2024-06-10T00:43:03.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.505+0000 7f1a78d9e640 1 -- 172.21.15.62:0/2948388497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a741a3b40 con 0x7f1a74101050 2024-06-10T00:43:03.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.505+0000 7f1a78d9e640 1 -- 172.21.15.62:0/2948388497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a7404ffb0 con 0x7f1a74101050 2024-06-10T00:43:03.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.507+0000 7f1a70ff9640 1 -- 172.21.15.62:0/2948388497 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1a680109d0 con 0x7f1a74101050 2024-06-10T00:43:03.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.508+0000 7f1a70ff9640 1 --2- 172.21.15.62:0/2948388497 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1a5c0a2cf0 0x7f1a5c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:03.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.508+0000 7f1a70ff9640 1 -- 172.21.15.62:0/2948388497 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f1a680cf470 con 0x7f1a74101050 2024-06-10T00:43:03.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.508+0000 7f1a737fe640 1 --2- 172.21.15.62:0/2948388497 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1a5c0a2cf0 0x7f1a5c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:03.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.508+0000 7f1a737fe640 1 --2- 172.21.15.62:0/2948388497 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1a5c0a2cf0 0x7f1a5c0a51b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009900 tx=0x7f1a640058d0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:03.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.511+0000 7f1a70ff9640 1 -- 172.21.15.62:0/2948388497 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1a68009de0 con 0x7f1a74101050 2024-06-10T00:43:03.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:03 smithi105 ceph-mon[36321]: pgmap v315: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:43:03.607 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:03 smithi062 ceph-mon[32390]: pgmap v315: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:43:03.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:03 smithi138 ceph-mon[36549]: pgmap v315: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:43:03.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.720+0000 7f1a78d9e640 1 -- 172.21.15.62:0/2948388497 --> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f1a74061de0 con 0x7f1a5c0a2cf0 2024-06-10T00:43:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.728+0000 7f1a70ff9640 1 -- 172.21.15.62:0/2948388497 <== mgr.15333 v2:172.21.15.62:6800/518034449 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f1a74061de0 con 0x7f1a5c0a2cf0 2024-06-10T00:43:03.728 INFO:teuthology.orchestra.run.smithi062.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-06-10T00:00:00", "created": "2024-06-10T00:33:01", "first": "2024-06-10T00:34:00", "last": "2024-06-10T00:43:00", "last_pruned": "2024-06-10T00:43:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-06-10T00:43:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.730+0000 7f1a78d9e640 1 -- 172.21.15.62:0/2948388497 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1a5c0a2cf0 msgr2=0x7f1a5c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.730+0000 7f1a78d9e640 1 --2- 172.21.15.62:0/2948388497 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1a5c0a2cf0 0x7f1a5c0a51b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009900 tx=0x7f1a640058d0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.730+0000 7f1a78d9e640 1 -- 172.21.15.62:0/2948388497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a74101050 msgr2=0x7f1a74197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.730+0000 7f1a78d9e640 1 --2- 172.21.15.62:0/2948388497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a74101050 0x7f1a74197510 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800d950 tx=0x7f1a6800de10 comp rx=0 tx=0).stop 2024-06-10T00:43:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.730+0000 7f1a78d9e640 1 -- 172.21.15.62:0/2948388497 shutdown_connections 2024-06-10T00:43:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.730+0000 7f1a78d9e640 1 --2- 172.21.15.62:0/2948388497 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f1a5c0a2cf0 0x7f1a5c0a51b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.730+0000 7f1a78d9e640 1 --2- 172.21.15.62:0/2948388497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a741066a0 0x7f1a7419b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.730+0000 7f1a78d9e640 1 --2- 172.21.15.62:0/2948388497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a74101050 0x7f1a74197510 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.730+0000 7f1a78d9e640 1 --2- 172.21.15.62:0/2948388497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a74100690 0x7f1a74196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.730+0000 7f1a78d9e640 1 -- 172.21.15.62:0/2948388497 >> 172.21.15.62:0/2948388497 conn(0x7f1a740fc1c0 msgr2=0x7f1a740fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:03.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.730+0000 7f1a78d9e640 1 -- 172.21.15.62:0/2948388497 shutdown_connections 2024-06-10T00:43:03.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.730+0000 7f1a78d9e640 1 -- 172.21.15.62:0/2948388497 wait complete. 2024-06-10T00:43:03.742 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-06-10T00:43:03.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.850+0000 7f05f8b61640 1 -- 172.21.15.62:0/3009275613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05f41010d0 msgr2=0x7f05f410de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:03.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.850+0000 7f05f8b61640 1 --2- 172.21.15.62:0/3009275613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05f41010d0 0x7f05f410de20 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009900 tx=0x7f05e802a410 comp rx=0 tx=0).stop 2024-06-10T00:43:03.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.850+0000 7f05f8b61640 1 -- 172.21.15.62:0/3009275613 shutdown_connections 2024-06-10T00:43:03.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.850+0000 7f05f8b61640 1 --2- 172.21.15.62:0/3009275613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f410e360 0x7f05f4110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.850+0000 7f05f8b61640 1 --2- 172.21.15.62:0/3009275613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05f41010d0 0x7f05f410de20 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.850+0000 7f05f8b61640 1 --2- 172.21.15.62:0/3009275613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05f41007b0 0x7f05f4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.850+0000 7f05f8b61640 1 -- 172.21.15.62:0/3009275613 >> 172.21.15.62:0/3009275613 conn(0x7f05f4077a20 msgr2=0x7f05f4077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:03.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.850+0000 7f05f8b61640 1 -- 172.21.15.62:0/3009275613 shutdown_connections 2024-06-10T00:43:03.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.850+0000 7f05f8b61640 1 -- 172.21.15.62:0/3009275613 wait complete. 2024-06-10T00:43:03.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.851+0000 7f05f8b61640 1 Processor -- start 2024-06-10T00:43:03.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.851+0000 7f05f8b61640 1 -- start start 2024-06-10T00:43:03.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.851+0000 7f05f8b61640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05f41007b0 0x7f05f419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:03.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.851+0000 7f05f8b61640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f41010d0 0x7f05f419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:03.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.851+0000 7f05f8b61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05f410e360 0x7f05f419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:03.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.851+0000 7f05f8b61640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f05f4113f90 con 0x7f05f41007b0 2024-06-10T00:43:03.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.851+0000 7f05f8b61640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f05f4113e10 con 0x7f05f410e360 2024-06-10T00:43:03.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.851+0000 7f05f8b61640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05f4114110 con 0x7f05f41010d0 2024-06-10T00:43:03.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.851+0000 7f05f259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05f41007b0 0x7f05f419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:03.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.851+0000 7f05f259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05f41007b0 0x7f05f419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44676/0 (socket says 172.21.15.62:44676) 2024-06-10T00:43:03.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.851+0000 7f05f259c640 1 -- 172.21.15.62:0/2340066853 learned_addr learned my addr 172.21.15.62:0/2340066853 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:03.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05f1d9b640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f41010d0 0x7f05f419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:03.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05f2d9d640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05f410e360 0x7f05f419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:03.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05f259c640 1 -- 172.21.15.62:0/2340066853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f41010d0 msgr2=0x7f05f419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:03.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05f259c640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f41010d0 0x7f05f419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05f259c640 1 -- 172.21.15.62:0/2340066853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05f410e360 msgr2=0x7f05f419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:03.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05f259c640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05f410e360 0x7f05f419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:03.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05f259c640 1 -- 172.21.15.62:0/2340066853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f419f500 con 0x7f05f41007b0 2024-06-10T00:43:03.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05f1d9b640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f41010d0 0x7f05f419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:43:03.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05f2d9d640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05f410e360 0x7f05f419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:43:03.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05f259c640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05f41007b0 0x7f05f419a510 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f05e400e9f0 tx=0x7f05e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:03.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05e37fe640 1 -- 172.21.15.62:0/2340066853 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e400cd80 con 0x7f05f41007b0 2024-06-10T00:43:03.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05e37fe640 1 -- 172.21.15.62:0/2340066853 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e4010910 con 0x7f05f41007b0 2024-06-10T00:43:03.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05e37fe640 1 -- 172.21.15.62:0/2340066853 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4018990 con 0x7f05f41007b0 2024-06-10T00:43:03.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05f8b61640 1 -- 172.21.15.62:0/2340066853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f419f7f0 con 0x7f05f41007b0 2024-06-10T00:43:03.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.852+0000 7f05f8b61640 1 -- 172.21.15.62:0/2340066853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f41a7080 con 0x7f05f41007b0 2024-06-10T00:43:03.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.853+0000 7f05f8b61640 1 -- 172.21.15.62:0/2340066853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f404f760 con 0x7f05f41007b0 2024-06-10T00:43:03.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.855+0000 7f05e37fe640 1 -- 172.21.15.62:0/2340066853 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f05e4010ac0 con 0x7f05f41007b0 2024-06-10T00:43:03.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.855+0000 7f05e37fe640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f05dc0a2e20 0x7f05dc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:03.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.856+0000 7f05e37fe640 1 -- 172.21.15.62:0/2340066853 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9554+0+0 (secure 0 0 0) 0x7f05e40d0020 con 0x7f05f41007b0 2024-06-10T00:43:03.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.856+0000 7f05f1d9b640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f05dc0a2e20 0x7f05dc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:03.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.856+0000 7f05f1d9b640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f05dc0a2e20 0x7f05dc0a52e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f05f41021f0 tx=0x7f05e8005a20 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:03.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:03.860+0000 7f05e37fe640 1 -- 172.21.15.62:0/2340066853 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f05e409c830 con 0x7f05f41007b0 2024-06-10T00:43:04.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:04.069+0000 7f05f8b61640 1 -- 172.21.15.62:0/2340066853 --> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f05f4061de0 con 0x7f05dc0a2e20 2024-06-10T00:43:04.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:04.077+0000 7f05e37fe640 1 -- 172.21.15.62:0/2340066853 <== mgr.15333 v2:172.21.15.62:6800/518034449 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f05f4061de0 con 0x7f05dc0a2e20 2024-06-10T00:43:04.077 INFO:teuthology.orchestra.run.smithi062.stdout:/ 1m 6m3h 2024-06-10T00:43:04.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:04.079+0000 7f05f8b61640 1 -- 172.21.15.62:0/2340066853 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f05dc0a2e20 msgr2=0x7f05dc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:04.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:04.079+0000 7f05f8b61640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f05dc0a2e20 0x7f05dc0a52e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f05f41021f0 tx=0x7f05e8005a20 comp rx=0 tx=0).stop 2024-06-10T00:43:04.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:04.079+0000 7f05f8b61640 1 -- 172.21.15.62:0/2340066853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05f41007b0 msgr2=0x7f05f419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:04.079+0000 7f05f8b61640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05f41007b0 0x7f05f419a510 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f05e400e9f0 tx=0x7f05e400eeb0 comp rx=0 tx=0).stop 2024-06-10T00:43:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:04.079+0000 7f05f8b61640 1 -- 172.21.15.62:0/2340066853 shutdown_connections 2024-06-10T00:43:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:04.079+0000 7f05f8b61640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f05dc0a2e20 0x7f05dc0a52e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:04.079+0000 7f05f8b61640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05f410e360 0x7f05f419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:04.079+0000 7f05f8b61640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f41010d0 0x7f05f419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:04.079+0000 7f05f8b61640 1 --2- 172.21.15.62:0/2340066853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05f41007b0 0x7f05f419a510 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:04.079+0000 7f05f8b61640 1 -- 172.21.15.62:0/2340066853 >> 172.21.15.62:0/2340066853 conn(0x7f05f4077a20 msgr2=0x7f05f410f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:04.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:04.079+0000 7f05f8b61640 1 -- 172.21.15.62:0/2340066853 shutdown_connections 2024-06-10T00:43:04.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:04.079+0000 7f05f8b61640 1 -- 172.21.15.62:0/2340066853 wait complete. 2024-06-10T00:43:04.090 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-06-10T00:43:04.101 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-06-10T00:43:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:05 smithi105 ceph-mon[36321]: from='client.17070 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:43:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:05 smithi105 ceph-mon[36321]: from='client.17076 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:43:05.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:05 smithi105 ceph-mon[36321]: pgmap v316: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-06-10T00:43:05.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:05 smithi062 ceph-mon[32390]: from='client.17070 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:43:05.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:05 smithi062 ceph-mon[32390]: from='client.17076 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:43:05.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:05 smithi062 ceph-mon[32390]: pgmap v316: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-06-10T00:43:05.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:05 smithi138 ceph-mon[36549]: from='client.17070 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:43:05.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:05 smithi138 ceph-mon[36549]: from='client.17076 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:43:05.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:05 smithi138 ceph-mon[36549]: pgmap v316: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-06-10T00:43:07.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:07 smithi105 ceph-mon[36321]: pgmap v317: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-06-10T00:43:07.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:07 smithi062 ceph-mon[32390]: pgmap v317: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-06-10T00:43:07.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:07 smithi138 ceph-mon[36549]: pgmap v317: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-06-10T00:43:08.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:08 smithi105 ceph-mon[36321]: osdmap e107: 12 total, 12 up, 12 in 2024-06-10T00:43:08.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:08 smithi062 ceph-mon[32390]: osdmap e107: 12 total, 12 up, 12 in 2024-06-10T00:43:08.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:08 smithi138 ceph-mon[36549]: osdmap e107: 12 total, 12 up, 12 in 2024-06-10T00:43:09.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:09 smithi105 ceph-mon[36321]: pgmap v319: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:43:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:09 smithi062 ceph-mon[32390]: pgmap v319: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:43:09.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:09 smithi138 ceph-mon[36549]: pgmap v319: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:43:11.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:11 smithi105 ceph-mon[36321]: pgmap v320: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-06-10T00:43:11.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:11 smithi062 ceph-mon[32390]: pgmap v320: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-06-10T00:43:11.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:11 smithi138 ceph-mon[36549]: pgmap v320: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-06-10T00:43:13.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:13 smithi105 ceph-mon[36321]: pgmap v321: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-06-10T00:43:13.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:13 smithi062 ceph-mon[32390]: pgmap v321: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-06-10T00:43:13.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:13 smithi138 ceph-mon[36549]: pgmap v321: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-06-10T00:43:14.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:14 smithi062 ceph-mon[32390]: osdmap e108: 12 total, 12 up, 12 in 2024-06-10T00:43:14.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:14 smithi138 ceph-mon[36549]: osdmap e108: 12 total, 12 up, 12 in 2024-06-10T00:43:15.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:14 smithi105 ceph-mon[36321]: osdmap e108: 12 total, 12 up, 12 in 2024-06-10T00:43:15.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:15 smithi062 ceph-mon[32390]: pgmap v323: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 127 B/s wr, 12 op/s 2024-06-10T00:43:15.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:15 smithi138 ceph-mon[36549]: pgmap v323: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 127 B/s wr, 12 op/s 2024-06-10T00:43:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:15 smithi105 ceph-mon[36321]: pgmap v323: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 127 B/s wr, 12 op/s 2024-06-10T00:43:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:17 smithi062 ceph-mon[32390]: pgmap v324: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 230 B/s wr, 13 op/s 2024-06-10T00:43:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:17 smithi138 ceph-mon[36549]: pgmap v324: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 230 B/s wr, 13 op/s 2024-06-10T00:43:18.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:17 smithi105 ceph-mon[36321]: pgmap v324: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 230 B/s wr, 13 op/s 2024-06-10T00:43:18.398 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:43:18.666 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:43:19.794 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:19 smithi062 ceph-mon[32390]: pgmap v325: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 204 B/s wr, 13 op/s 2024-06-10T00:43:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:19 smithi138 ceph-mon[36549]: pgmap v325: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 204 B/s wr, 13 op/s 2024-06-10T00:43:20.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:19 smithi105 ceph-mon[36321]: pgmap v325: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 204 B/s wr, 13 op/s 2024-06-10T00:43:20.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.464+0000 7fec3e79c700 1 -- 172.21.15.62:0/802615192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec38106410 msgr2=0x7fec381067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:20.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.464+0000 7fec3e79c700 1 --2- 172.21.15.62:0/802615192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec38106410 0x7fec381067f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fec280099c0 tx=0x7fec28009cd0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.464+0000 7fec3e79c700 1 -- 172.21.15.62:0/802615192 shutdown_connections 2024-06-10T00:43:20.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.464+0000 7fec3e79c700 1 --2- 172.21.15.62:0/802615192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec38106410 0x7fec381067f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.464+0000 7fec3e79c700 1 --2- 172.21.15.62:0/802615192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec381020a0 0x7fec3810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.464+0000 7fec3e79c700 1 --2- 172.21.15.62:0/802615192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec381016e0 0x7fec38101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.464+0000 7fec3e79c700 1 -- 172.21.15.62:0/802615192 >> 172.21.15.62:0/802615192 conn(0x7fec38077200 msgr2=0x7fec38077610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:20.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.464+0000 7fec3e79c700 1 -- 172.21.15.62:0/802615192 shutdown_connections 2024-06-10T00:43:20.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.464+0000 7fec3e79c700 1 -- 172.21.15.62:0/802615192 wait complete. 2024-06-10T00:43:20.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.465+0000 7fec3e79c700 1 Processor -- start 2024-06-10T00:43:20.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.465+0000 7fec3e79c700 1 -- start start 2024-06-10T00:43:20.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.465+0000 7fec3e79c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec381016e0 0x7fec38199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.465+0000 7fec3e79c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec381020a0 0x7fec38199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.465+0000 7fec3e79c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec38106410 0x7fec381a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.465+0000 7fec3e79c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fec38111730 con 0x7fec381016e0 2024-06-10T00:43:20.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.465+0000 7fec3e79c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fec381115b0 con 0x7fec38106410 2024-06-10T00:43:20.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.465+0000 7fec3e79c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec381118b0 con 0x7fec381020a0 2024-06-10T00:43:20.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.465+0000 7fec37fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec381016e0 0x7fec38199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.465+0000 7fec37fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec381016e0 0x7fec38199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59520/0 (socket says 172.21.15.62:59520) 2024-06-10T00:43:20.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.465+0000 7fec37fff700 1 -- 172.21.15.62:0/4282475679 learned_addr learned my addr 172.21.15.62:0/4282475679 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:20.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.465+0000 7fec2ffff700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec381020a0 0x7fec38199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.465+0000 7fec3cd39700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec38106410 0x7fec381a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.466+0000 7fec37fff700 1 -- 172.21.15.62:0/4282475679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec381020a0 msgr2=0x7fec38199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:20.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.466+0000 7fec37fff700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec381020a0 0x7fec38199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.466+0000 7fec37fff700 1 -- 172.21.15.62:0/4282475679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec38106410 msgr2=0x7fec381a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:20.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.466+0000 7fec37fff700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec38106410 0x7fec381a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.466+0000 7fec37fff700 1 -- 172.21.15.62:0/4282475679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec381a4ff0 con 0x7fec381016e0 2024-06-10T00:43:20.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.466+0000 7fec3cd39700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec38106410 0x7fec381a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:43:20.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.466+0000 7fec37fff700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec381016e0 0x7fec38199a30 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fec280059d0 tx=0x7fec280038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:20.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.466+0000 7fec35ffb700 1 -- 172.21.15.62:0/4282475679 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28027070 con 0x7fec381016e0 2024-06-10T00:43:20.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.466+0000 7fec35ffb700 1 -- 172.21.15.62:0/4282475679 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec28004050 con 0x7fec381016e0 2024-06-10T00:43:20.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.466+0000 7fec3e79c700 1 -- 172.21.15.62:0/4282475679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec381a51c0 con 0x7fec381016e0 2024-06-10T00:43:20.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.466+0000 7fec35ffb700 1 -- 172.21.15.62:0/4282475679 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28022db0 con 0x7fec381016e0 2024-06-10T00:43:20.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.466+0000 7fec3e79c700 1 -- 172.21.15.62:0/4282475679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec381a55b0 con 0x7fec381016e0 2024-06-10T00:43:20.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.468+0000 7fec35ffb700 1 -- 172.21.15.62:0/4282475679 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fec2802a070 con 0x7fec381016e0 2024-06-10T00:43:20.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.469+0000 7fec35ffb700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fec240a2d90 0x7fec240a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.469+0000 7fec2ffff700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fec240a2d90 0x7fec240a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.470+0000 7fec35ffb700 1 -- 172.21.15.62:0/4282475679 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fec280d0670 con 0x7fec381016e0 2024-06-10T00:43:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.470+0000 7fec2ffff700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fec240a2d90 0x7fec240a5250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fec38077690 tx=0x7fec2000a380 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.470+0000 7fec4901fb80 1 Processor -- start 2024-06-10T00:43:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.470+0000 7fec4901fb80 1 -- start start 2024-06-10T00:43:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.470+0000 7fec4901fb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e460f60 0x562d1e461340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.470+0000 7fec4901fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e461880 0x562d1e45b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.470+0000 7fec4901fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e45b7a0 0x562d1e45bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.470+0000 7fec4901fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562d1e34df60 con 0x562d1e45b7a0 2024-06-10T00:43:20.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.470+0000 7fec4901fb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562d1e3ac850 con 0x562d1e461880 2024-06-10T00:43:20.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.470+0000 7fec4901fb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562d1e314460 con 0x562d1e460f60 2024-06-10T00:43:20.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.470+0000 7fec3cd39700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e45b7a0 0x562d1e45bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.470+0000 7fec3cd39700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e45b7a0 0x562d1e45bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59524/0 (socket says 172.21.15.62:59524) 2024-06-10T00:43:20.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.470+0000 7fec3cd39700 1 -- 172.21.15.62:0/146755090 learned_addr learned my addr 172.21.15.62:0/146755090 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:20.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.471+0000 7fec2ffff700 1 --2- 172.21.15.62:0/146755090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e461880 0x562d1e45b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.471+0000 7fec37fff700 1 --2- 172.21.15.62:0/146755090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e460f60 0x562d1e461340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.471+0000 7fec3cd39700 1 -- 172.21.15.62:0/146755090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e460f60 msgr2=0x562d1e461340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:20.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.471+0000 7fec3cd39700 1 --2- 172.21.15.62:0/146755090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e460f60 0x562d1e461340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.471+0000 7fec3cd39700 1 -- 172.21.15.62:0/146755090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e461880 msgr2=0x562d1e45b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:20.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.471+0000 7fec3cd39700 1 --2- 172.21.15.62:0/146755090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e461880 0x562d1e45b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.471+0000 7fec3cd39700 1 -- 172.21.15.62:0/146755090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1e2fb8d0 con 0x562d1e45b7a0 2024-06-10T00:43:20.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.471+0000 7fec3cd39700 1 --2- 172.21.15.62:0/146755090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e45b7a0 0x562d1e45bbc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fec30009dd0 tx=0x7fec300077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292cdaa3dad98b4b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:20.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.471+0000 7fec2e7fc700 1 -- 172.21.15.62:0/146755090 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30004bb0 con 0x562d1e45b7a0 2024-06-10T00:43:20.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.471+0000 7fec2e7fc700 1 -- 172.21.15.62:0/146755090 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec30004d50 con 0x562d1e45b7a0 2024-06-10T00:43:20.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.471+0000 7fec2e7fc700 1 -- 172.21.15.62:0/146755090 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30014700 con 0x562d1e45b7a0 2024-06-10T00:43:20.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.472+0000 7fec4901fb80 1 -- 172.21.15.62:0/146755090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e45b7a0 msgr2=0x562d1e45bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:20.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.472+0000 7fec4901fb80 1 --2- 172.21.15.62:0/146755090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e45b7a0 0x562d1e45bbc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fec30009dd0 tx=0x7fec300077e0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.472+0000 7fec4901fb80 1 -- 172.21.15.62:0/146755090 shutdown_connections 2024-06-10T00:43:20.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.472+0000 7fec4901fb80 1 --2- 172.21.15.62:0/146755090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e45b7a0 0x562d1e45bbc0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.472+0000 7fec4901fb80 1 --2- 172.21.15.62:0/146755090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e460f60 0x562d1e461340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.472+0000 7fec4901fb80 1 --2- 172.21.15.62:0/146755090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e461880 0x562d1e45b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.472+0000 7fec4901fb80 1 -- 172.21.15.62:0/146755090 >> 172.21.15.62:0/146755090 conn(0x562d1e388080 msgr2=0x562d1e456220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.472+0000 7fec4901fb80 1 -- 172.21.15.62:0/146755090 shutdown_connections 2024-06-10T00:43:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.472+0000 7fec4901fb80 1 -- 172.21.15.62:0/146755090 wait complete. 2024-06-10T00:43:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.472+0000 7fec4901fb80 1 -- start start 2024-06-10T00:43:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec4901fb80 1 Processor -- start 2024-06-10T00:43:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec4901fb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e45b7a0 0x562d1e467570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec4901fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 0x562d1e464870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec4901fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e461880 0x562d1e464db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec4901fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562d1e34cfc0 con 0x562d1e461880 2024-06-10T00:43:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec4901fb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562d1e3a8810 con 0x562d1e460f60 2024-06-10T00:43:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec4901fb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562d1e468260 con 0x562d1e45b7a0 2024-06-10T00:43:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec4901fb80 10 client.0 _ll_register_callbacks cb 0x562d1e3425b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:43:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec3cd39700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e461880 0x562d1e464db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec3cd39700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e461880 0x562d1e464db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59534/0 (socket says 172.21.15.62:59534) 2024-06-10T00:43:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec3cd39700 1 -- 172.21.15.62:0/3236596908 learned_addr learned my addr 172.21.15.62:0/3236596908 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec37fff700 1 --2- 172.21.15.62:0/3236596908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e45b7a0 0x562d1e467570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec2ffff700 1 --2- 172.21.15.62:0/3236596908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 0x562d1e464870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec3cd39700 1 -- 172.21.15.62:0/3236596908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e45b7a0 msgr2=0x562d1e467570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec3cd39700 1 --2- 172.21.15.62:0/3236596908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e45b7a0 0x562d1e467570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec3cd39700 1 -- 172.21.15.62:0/3236596908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 msgr2=0x562d1e464870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec3cd39700 1 --2- 172.21.15.62:0/3236596908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 0x562d1e464870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec3cd39700 1 -- 172.21.15.62:0/3236596908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1e372750 con 0x562d1e461880 2024-06-10T00:43:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.473+0000 7fec2ffff700 1 --2- 172.21.15.62:0/3236596908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 0x562d1e464870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:43:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec3cd39700 1 --2- 172.21.15.62:0/3236596908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e461880 0x562d1e464db0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fec30005bd0 tx=0x7fec30005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224b0812ced27352 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:20.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec177fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:43:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec177fe700 1 -- 172.21.15.62:0/3236596908 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30022020 con 0x562d1e461880 2024-06-10T00:43:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec177fe700 1 -- 172.21.15.62:0/3236596908 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fec3001e420 con 0x562d1e461880 2024-06-10T00:43:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec4901fb80 1 -- 172.21.15.62:0/3236596908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec30004d50 con 0x562d1e461880 2024-06-10T00:43:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec177fe700 1 -- 172.21.15.62:0/3236596908 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fec3002d050 con 0x562d1e461880 2024-06-10T00:43:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec4901fb80 10 client.17088 fetch_fsmap learned FSMap version 38 2024-06-10T00:43:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec4901fb80 1 -- 172.21.15.62:0/3236596908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562d1e4729a0 con 0x562d1e461880 2024-06-10T00:43:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec177fe700 1 -- 172.21.15.62:0/3236596908 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec30004d50 con 0x562d1e461880 2024-06-10T00:43:20.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec177fe700 1 -- 172.21.15.62:0/3236596908 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fec300145a0 con 0x562d1e461880 2024-06-10T00:43:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec4901fb80 10 client.17088 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:43:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec4901fb80 10 client.17088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:43:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec4901fb80 20 client.17088 populate_metadata read hostname 'smithi062' 2024-06-10T00:43:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec4901fb80 1 --2- 172.21.15.62:0/3236596908 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x562d1e474f80 0x562d1e477370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec4901fb80 4 client.17088 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:43:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.474+0000 7fec4901fb80 1 -- 172.21.15.62:0/3236596908 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d1e3b9d50 con 0x562d1e474f80 2024-06-10T00:43:20.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.475+0000 7fec37fff700 1 --2- 172.21.15.62:0/3236596908 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x562d1e474f80 0x562d1e477370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.475+0000 7fec37fff700 1 --2- 172.21.15.62:0/3236596908 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x562d1e474f80 0x562d1e477370 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=559b09cb94d555bb server_cookie=3e3027249df5097 in_seq=0 out_seq=0 2024-06-10T00:43:20.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.475+0000 7fec177fe700 10 client.17088 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:43:20.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec177fe700 1 -- 172.21.15.62:0/3236596908 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562d1e3b9d50 con 0x562d1e474f80 2024-06-10T00:43:20.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec177fe700 10 client.17088 handle_command_reply: tid=0 2024-06-10T00:43:20.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec4901fb80 1 client.17088 shutdown 2024-06-10T00:43:20.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec4901fb80 1 -- 172.21.15.62:0/3236596908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e461880 msgr2=0x562d1e464db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:20.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec4901fb80 1 --2- 172.21.15.62:0/3236596908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e461880 0x562d1e464db0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fec30005bd0 tx=0x7fec30005610 comp rx=0 tx=0).stop 2024-06-10T00:43:20.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec4901fb80 1 -- 172.21.15.62:0/3236596908 shutdown_connections 2024-06-10T00:43:20.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec4901fb80 1 --2- 172.21.15.62:0/3236596908 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x562d1e474f80 0x562d1e477370 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec4901fb80 1 --2- 172.21.15.62:0/3236596908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e461880 0x562d1e464db0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec4901fb80 1 --2- 172.21.15.62:0/3236596908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e45b7a0 0x562d1e467570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec4901fb80 1 --2- 172.21.15.62:0/3236596908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 0x562d1e464870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec4901fb80 1 -- 172.21.15.62:0/3236596908 >> 172.21.15.62:0/3236596908 conn(0x562d1e388080 msgr2=0x562d1e455700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:20.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec177fe700 0 client.17088 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:43:20.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec4901fb80 1 -- 172.21.15.62:0/3236596908 shutdown_connections 2024-06-10T00:43:20.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec4901fb80 1 -- 172.21.15.62:0/3236596908 wait complete. 2024-06-10T00:43:20.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.476+0000 7fec4901fb80 20 client.17088 trim_cache size 0 max 16384 2024-06-10T00:43:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.497+0000 7fec4901fb80 1 Processor -- start 2024-06-10T00:43:20.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.497+0000 7fec4901fb80 1 -- start start 2024-06-10T00:43:20.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.497+0000 7fec4901fb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e45b7a0 0x562d1e45b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.497+0000 7fec4901fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 0x562d1e455e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.497+0000 7fec4901fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e461880 0x562d1e4563c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.497+0000 7fec4901fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fec381ae500 con 0x562d1e461880 2024-06-10T00:43:20.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.497+0000 7fec4901fb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fec381b0040 con 0x562d1e460f60 2024-06-10T00:43:20.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.497+0000 7fec4901fb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec381aebd0 con 0x562d1e45b7a0 2024-06-10T00:43:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.497+0000 7fec3cd39700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e461880 0x562d1e4563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.497+0000 7fec3cd39700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e461880 0x562d1e4563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59538/0 (socket says 172.21.15.62:59538) 2024-06-10T00:43:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.497+0000 7fec3cd39700 1 -- 172.21.15.62:0/1946294087 learned_addr learned my addr 172.21.15.62:0/1946294087 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.497+0000 7fec2ffff700 1 --2- 172.21.15.62:0/1946294087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 0x562d1e455e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.498+0000 7fec37fff700 1 --2- 172.21.15.62:0/1946294087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e45b7a0 0x562d1e45b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.498+0000 7fec3cd39700 1 -- 172.21.15.62:0/1946294087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e45b7a0 msgr2=0x562d1e45b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.498+0000 7fec3cd39700 1 --2- 172.21.15.62:0/1946294087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e45b7a0 0x562d1e45b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.498+0000 7fec3cd39700 1 -- 172.21.15.62:0/1946294087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 msgr2=0x562d1e455e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.498+0000 7fec3cd39700 1 --2- 172.21.15.62:0/1946294087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 0x562d1e455e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.498+0000 7fec3cd39700 1 -- 172.21.15.62:0/1946294087 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1e465130 con 0x562d1e461880 2024-06-10T00:43:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.498+0000 7fec2ffff700 1 --2- 172.21.15.62:0/1946294087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 0x562d1e455e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:43:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.498+0000 7fec3cd39700 1 --2- 172.21.15.62:0/1946294087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e461880 0x562d1e4563c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fec30009710 tx=0x7fec30009130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49762a1fb3b79d8e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.498+0000 7fec2dffb700 1 -- 172.21.15.62:0/1946294087 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30022040 con 0x562d1e461880 2024-06-10T00:43:20.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.498+0000 7fec2dffb700 1 -- 172.21.15.62:0/1946294087 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec30004ce0 con 0x562d1e461880 2024-06-10T00:43:20.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.498+0000 7fec2dffb700 1 -- 172.21.15.62:0/1946294087 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3001e840 con 0x562d1e461880 2024-06-10T00:43:20.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.499+0000 7fec4901fb80 1 -- 172.21.15.62:0/1946294087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e461880 msgr2=0x562d1e4563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:20.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.499+0000 7fec4901fb80 1 --2- 172.21.15.62:0/1946294087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e461880 0x562d1e4563c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fec30009710 tx=0x7fec30009130 comp rx=0 tx=0).stop 2024-06-10T00:43:20.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.499+0000 7fec4901fb80 1 -- 172.21.15.62:0/1946294087 shutdown_connections 2024-06-10T00:43:20.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.499+0000 7fec4901fb80 1 --2- 172.21.15.62:0/1946294087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e461880 0x562d1e4563c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.499+0000 7fec4901fb80 1 --2- 172.21.15.62:0/1946294087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e45b7a0 0x562d1e45b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.499+0000 7fec4901fb80 1 --2- 172.21.15.62:0/1946294087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 0x562d1e455e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.499+0000 7fec4901fb80 1 -- 172.21.15.62:0/1946294087 >> 172.21.15.62:0/1946294087 conn(0x562d1e388080 msgr2=0x562d1e466930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:20.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.499+0000 7fec4901fb80 1 -- 172.21.15.62:0/1946294087 shutdown_connections 2024-06-10T00:43:20.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.499+0000 7fec4901fb80 1 -- 172.21.15.62:0/1946294087 wait complete. 2024-06-10T00:43:20.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.499+0000 7fec4901fb80 1 -- start start 2024-06-10T00:43:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.499+0000 7fec4901fb80 1 Processor -- start 2024-06-10T00:43:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec4901fb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e45b7a0 0x562d1e46d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec4901fb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 0x562d1e46de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec4901fb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e461880 0x562d1e47c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec4901fb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fec381af970 con 0x562d1e45b7a0 2024-06-10T00:43:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec4901fb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fec381af2a0 con 0x562d1e460f60 2024-06-10T00:43:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec4901fb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec381ac4c0 con 0x562d1e461880 2024-06-10T00:43:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec4901fb80 10 client.0 _ll_register_callbacks cb 0x562d1e3425b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:43:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec37fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e45b7a0 0x562d1e46d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec37fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e45b7a0 0x562d1e46d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59552/0 (socket says 172.21.15.62:59552) 2024-06-10T00:43:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec37fff700 1 -- 172.21.15.62:0/2968390383 learned_addr learned my addr 172.21.15.62:0/2968390383 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec2ffff700 1 --2- 172.21.15.62:0/2968390383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 0x562d1e46de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec3cd39700 1 --2- 172.21.15.62:0/2968390383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e461880 0x562d1e47c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec37fff700 1 -- 172.21.15.62:0/2968390383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e461880 msgr2=0x562d1e47c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:20.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec37fff700 1 --2- 172.21.15.62:0/2968390383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e461880 0x562d1e47c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec37fff700 1 -- 172.21.15.62:0/2968390383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 msgr2=0x562d1e46de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:20.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec37fff700 1 --2- 172.21.15.62:0/2968390383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 0x562d1e46de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec37fff700 1 -- 172.21.15.62:0/2968390383 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1e47c8d0 con 0x562d1e45b7a0 2024-06-10T00:43:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec3cd39700 1 --2- 172.21.15.62:0/2968390383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e461880 0x562d1e47c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:43:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec37fff700 1 --2- 172.21.15.62:0/2968390383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e45b7a0 0x562d1e46d8f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fec280098e0 tx=0x7fec2801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef08273fc474dfd3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec177fe700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:43:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec177fe700 1 -- 172.21.15.62:0/2968390383 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec280a0b60 con 0x562d1e45b7a0 2024-06-10T00:43:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec177fe700 1 -- 172.21.15.62:0/2968390383 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fec280cdbd0 con 0x562d1e45b7a0 2024-06-10T00:43:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.500+0000 7fec4901fb80 1 -- 172.21.15.62:0/2968390383 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec30004ce0 con 0x562d1e45b7a0 2024-06-10T00:43:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.501+0000 7fec177fe700 1 -- 172.21.15.62:0/2968390383 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fec280a5390 con 0x562d1e45b7a0 2024-06-10T00:43:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.501+0000 7fec4901fb80 10 client.17094 fetch_fsmap learned FSMap version 38 2024-06-10T00:43:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.501+0000 7fec4901fb80 1 -- 172.21.15.62:0/2968390383 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562d1e4df5e0 con 0x562d1e45b7a0 2024-06-10T00:43:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.501+0000 7fec177fe700 1 -- 172.21.15.62:0/2968390383 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec30004ce0 con 0x562d1e45b7a0 2024-06-10T00:43:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.501+0000 7fec177fe700 1 -- 172.21.15.62:0/2968390383 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fec2809caa0 con 0x562d1e45b7a0 2024-06-10T00:43:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.501+0000 7fec4901fb80 10 client.17094 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:43:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.501+0000 7fec4901fb80 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:43:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.501+0000 7fec4901fb80 20 client.17094 populate_metadata read hostname 'smithi062' 2024-06-10T00:43:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.501+0000 7fec4901fb80 1 --2- 172.21.15.62:0/2968390383 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x562d1e4dfc90 0x562d1e4e20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.501+0000 7fec4901fb80 4 client.17094 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:43:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.501+0000 7fec4901fb80 1 -- 172.21.15.62:0/2968390383 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562d1e3b42d0 con 0x562d1e4dfc90 2024-06-10T00:43:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.501+0000 7fec2ffff700 1 --2- 172.21.15.62:0/2968390383 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x562d1e4dfc90 0x562d1e4e20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:20.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec2ffff700 1 --2- 172.21.15.62:0/2968390383 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x562d1e4dfc90 0x562d1e4e20b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c378cbf1550cc9 server_cookie=3db56b15ac3e7c5 in_seq=0 out_seq=0 2024-06-10T00:43:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec177fe700 10 client.17094 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:43:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec177fe700 1 -- 172.21.15.62:0/2968390383 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562d1e3b42d0 con 0x562d1e4dfc90 2024-06-10T00:43:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec177fe700 10 client.17094 handle_command_reply: tid=0 2024-06-10T00:43:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec4901fb80 1 client.17094 shutdown 2024-06-10T00:43:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec4901fb80 1 -- 172.21.15.62:0/2968390383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e45b7a0 msgr2=0x562d1e46d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec4901fb80 1 --2- 172.21.15.62:0/2968390383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e45b7a0 0x562d1e46d8f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fec280098e0 tx=0x7fec2801a040 comp rx=0 tx=0).stop 2024-06-10T00:43:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec4901fb80 1 -- 172.21.15.62:0/2968390383 shutdown_connections 2024-06-10T00:43:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec4901fb80 1 --2- 172.21.15.62:0/2968390383 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x562d1e4dfc90 0x562d1e4e20b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec177fe700 0 client.17094 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:43:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec4901fb80 1 --2- 172.21.15.62:0/2968390383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d1e45b7a0 0x562d1e46d8f0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec4901fb80 1 --2- 172.21.15.62:0/2968390383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d1e461880 0x562d1e47c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec4901fb80 1 --2- 172.21.15.62:0/2968390383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562d1e460f60 0x562d1e46de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec4901fb80 1 -- 172.21.15.62:0/2968390383 >> 172.21.15.62:0/2968390383 conn(0x562d1e388080 msgr2=0x562d1e463030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec4901fb80 1 -- 172.21.15.62:0/2968390383 shutdown_connections 2024-06-10T00:43:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec4901fb80 1 -- 172.21.15.62:0/2968390383 wait complete. 2024-06-10T00:43:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.502+0000 7fec4901fb80 20 client.17094 trim_cache size 0 max 16384 2024-06-10T00:43:20.516 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:43:20.517 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-06-10T00:43:20.517 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-06-10T00:43:20.517 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:43:20.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.503+0000 7fec3e79c700 1 -- 172.21.15.62:0/4282475679 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fec240a2d90 msgr2=0x7fec240a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:20.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.503+0000 7fec3e79c700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fec240a2d90 0x7fec240a5250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fec38077690 tx=0x7fec2000a380 comp rx=0 tx=0).stop 2024-06-10T00:43:20.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.503+0000 7fec3e79c700 1 -- 172.21.15.62:0/4282475679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec381016e0 msgr2=0x7fec38199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:20.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.503+0000 7fec3e79c700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec381016e0 0x7fec38199a30 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fec280059d0 tx=0x7fec280038b0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.503+0000 7fec3e79c700 1 -- 172.21.15.62:0/4282475679 shutdown_connections 2024-06-10T00:43:20.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.503+0000 7fec3e79c700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fec240a2d90 0x7fec240a5250 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.503+0000 7fec3e79c700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec381016e0 0x7fec38199a30 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.503+0000 7fec3e79c700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec38106410 0x7fec381a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.503+0000 7fec3e79c700 1 --2- 172.21.15.62:0/4282475679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec381020a0 0x7fec38199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.503+0000 7fec3e79c700 1 -- 172.21.15.62:0/4282475679 >> 172.21.15.62:0/4282475679 conn(0x7fec38077200 msgr2=0x7fec380ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.503+0000 7fec3e79c700 1 -- 172.21.15.62:0/4282475679 shutdown_connections 2024-06-10T00:43:20.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:20.503+0000 7fec3e79c700 1 -- 172.21.15.62:0/4282475679 wait complete. 2024-06-10T00:43:21.204 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T00:43:21.205 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:43:21.458 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:43:21.847 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:21 smithi062 ceph-mon[32390]: pgmap v326: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:43:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:21 smithi138 ceph-mon[36549]: pgmap v326: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:43:22.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:21 smithi105 ceph-mon[36321]: pgmap v326: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:43:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.186+0000 7f8b32fd6700 1 -- 172.21.15.62:0/377270322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2c108500 msgr2=0x7f8b2c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.186+0000 7f8b32fd6700 1 --2- 172.21.15.62:0/377270322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2c108500 0x7f8b2c1088e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009990 tx=0x7f8b14009ca0 comp rx=0 tx=0).stop 2024-06-10T00:43:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.186+0000 7f8b32fd6700 1 -- 172.21.15.62:0/377270322 shutdown_connections 2024-06-10T00:43:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.186+0000 7f8b32fd6700 1 --2- 172.21.15.62:0/377270322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2c108500 0x7f8b2c1088e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.186+0000 7f8b32fd6700 1 --2- 172.21.15.62:0/377270322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b2c06b5a0 0x7f8b2c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.186+0000 7f8b32fd6700 1 --2- 172.21.15.62:0/377270322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b2c06abe0 0x7f8b2c06b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.186+0000 7f8b32fd6700 1 -- 172.21.15.62:0/377270322 >> 172.21.15.62:0/377270322 conn(0x7f8b2c078360 msgr2=0x7f8b2c078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.186+0000 7f8b32fd6700 1 -- 172.21.15.62:0/377270322 shutdown_connections 2024-06-10T00:43:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.186+0000 7f8b32fd6700 1 -- 172.21.15.62:0/377270322 wait complete. 2024-06-10T00:43:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.187+0000 7f8b32fd6700 1 Processor -- start 2024-06-10T00:43:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.187+0000 7f8b32fd6700 1 -- start start 2024-06-10T00:43:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.187+0000 7f8b32fd6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b2c06abe0 0x7f8b2c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.187+0000 7f8b32fd6700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2c06b5a0 0x7f8b2c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.187+0000 7f8b32fd6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b2c108500 0x7f8b2c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.187+0000 7f8b32fd6700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c112780 con 0x7f8b2c06b5a0 2024-06-10T00:43:23.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.187+0000 7f8b32fd6700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c112600 con 0x7f8b2c108500 2024-06-10T00:43:23.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.187+0000 7f8b32fd6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c112900 con 0x7f8b2c06abe0 2024-06-10T00:43:23.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.187+0000 7f8b2bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2c06b5a0 0x7f8b2c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:23.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.187+0000 7f8b2bfff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2c06b5a0 0x7f8b2c19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59566/0 (socket says 172.21.15.62:59566) 2024-06-10T00:43:23.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.187+0000 7f8b2bfff700 1 -- 172.21.15.62:0/3502458699 learned_addr learned my addr 172.21.15.62:0/3502458699 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:23.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.188+0000 7f8b31573700 1 --2- 172.21.15.62:0/3502458699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b2c108500 0x7f8b2c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:23.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.188+0000 7f8b30d72700 1 --2- 172.21.15.62:0/3502458699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b2c06abe0 0x7f8b2c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:23.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.188+0000 7f8b2bfff700 1 -- 172.21.15.62:0/3502458699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b2c06abe0 msgr2=0x7f8b2c19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:23.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.188+0000 7f8b2bfff700 1 --2- 172.21.15.62:0/3502458699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b2c06abe0 0x7f8b2c19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:23.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.188+0000 7f8b2bfff700 1 -- 172.21.15.62:0/3502458699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b2c108500 msgr2=0x7f8b2c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:23.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.188+0000 7f8b2bfff700 1 --2- 172.21.15.62:0/3502458699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b2c108500 0x7f8b2c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:23.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.188+0000 7f8b2bfff700 1 -- 172.21.15.62:0/3502458699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c1991e0 con 0x7f8b2c06b5a0 2024-06-10T00:43:23.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.188+0000 7f8b2bfff700 1 --2- 172.21.15.62:0/3502458699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2c06b5a0 0x7f8b2c19ee00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00d980 tx=0x7f8b1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:23.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.188+0000 7f8b29ffb700 1 -- 172.21.15.62:0/3502458699 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c017070 con 0x7f8b2c06b5a0 2024-06-10T00:43:23.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.188+0000 7f8b29ffb700 1 -- 172.21.15.62:0/3502458699 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b1c005020 con 0x7f8b2c06b5a0 2024-06-10T00:43:23.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.188+0000 7f8b32fd6700 1 -- 172.21.15.62:0/3502458699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c1994d0 con 0x7f8b2c06b5a0 2024-06-10T00:43:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.188+0000 7f8b29ffb700 1 -- 172.21.15.62:0/3502458699 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c009bc0 con 0x7f8b2c06b5a0 2024-06-10T00:43:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.188+0000 7f8b32fd6700 1 -- 172.21.15.62:0/3502458699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c1a5790 con 0x7f8b2c06b5a0 2024-06-10T00:43:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.189+0000 7f8b32fd6700 1 -- 172.21.15.62:0/3502458699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2c10a9e0 con 0x7f8b2c06b5a0 2024-06-10T00:43:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.190+0000 7f8b29ffb700 1 -- 172.21.15.62:0/3502458699 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8b1c0051d0 con 0x7f8b2c06b5a0 2024-06-10T00:43:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.191+0000 7f8b29ffb700 1 --2- 172.21.15.62:0/3502458699 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8b180a2e10 0x7f8b180a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.191+0000 7f8b30d72700 1 --2- 172.21.15.62:0/3502458699 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8b180a2e10 0x7f8b180a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.191+0000 7f8b29ffb700 1 -- 172.21.15.62:0/3502458699 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f8b1c0cf720 con 0x7f8b2c06b5a0 2024-06-10T00:43:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.192+0000 7f8b30d72700 1 --2- 172.21.15.62:0/3502458699 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8b180a2e10 0x7f8b180a52d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b14006010 tx=0x7f8b1400b540 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:23.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.196+0000 7f8b29ffb700 1 -- 172.21.15.62:0/3502458699 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8b1c020020 con 0x7f8b2c06b5a0 2024-06-10T00:43:23.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.452+0000 7f8b32fd6700 1 -- 172.21.15.62:0/3502458699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b2c069180 con 0x7f8b2c06b5a0 2024-06-10T00:43:23.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.453+0000 7f8b29ffb700 1 -- 172.21.15.62:0/3502458699 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11938 (secure 0 0 0) 0x7f8b1c09bf30 con 0x7f8b2c06b5a0 2024-06-10T00:43:23.454 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:43:23.454 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:40:10.974660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:43:23.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.455+0000 7f8b32fd6700 1 -- 172.21.15.62:0/3502458699 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8b180a2e10 msgr2=0x7f8b180a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:23.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.455+0000 7f8b32fd6700 1 --2- 172.21.15.62:0/3502458699 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8b180a2e10 0x7f8b180a52d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b14006010 tx=0x7f8b1400b540 comp rx=0 tx=0).stop 2024-06-10T00:43:23.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.455+0000 7f8b32fd6700 1 -- 172.21.15.62:0/3502458699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2c06b5a0 msgr2=0x7f8b2c19ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:23.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.455+0000 7f8b32fd6700 1 --2- 172.21.15.62:0/3502458699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2c06b5a0 0x7f8b2c19ee00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00d980 tx=0x7f8b1c00dc90 comp rx=0 tx=0).stop 2024-06-10T00:43:23.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.455+0000 7f8b32fd6700 1 -- 172.21.15.62:0/3502458699 shutdown_connections 2024-06-10T00:43:23.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.455+0000 7f8b32fd6700 1 --2- 172.21.15.62:0/3502458699 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8b180a2e10 0x7f8b180a52d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:23.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.455+0000 7f8b32fd6700 1 --2- 172.21.15.62:0/3502458699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b2c108500 0x7f8b2c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:23.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.455+0000 7f8b32fd6700 1 --2- 172.21.15.62:0/3502458699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b2c06abe0 0x7f8b2c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:23.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.455+0000 7f8b32fd6700 1 --2- 172.21.15.62:0/3502458699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b2c06b5a0 0x7f8b2c19ee00 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:23.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.455+0000 7f8b32fd6700 1 -- 172.21.15.62:0/3502458699 >> 172.21.15.62:0/3502458699 conn(0x7f8b2c078360 msgr2=0x7f8b2c100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:23.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.455+0000 7f8b32fd6700 1 -- 172.21.15.62:0/3502458699 shutdown_connections 2024-06-10T00:43:23.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:23.455+0000 7f8b32fd6700 1 -- 172.21.15.62:0/3502458699 wait complete. 2024-06-10T00:43:23.457 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 38 2024-06-10T00:43:23.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:23 smithi062 ceph-mon[32390]: pgmap v327: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:43:23.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:23 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/3502458699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:43:23.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:23 smithi138 ceph-mon[36549]: pgmap v327: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:43:23.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:23 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/3502458699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:43:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:23 smithi105 ceph-mon[36321]: pgmap v327: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-06-10T00:43:24.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:23 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/3502458699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:43:24.179 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-10T00:43:24.434 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:43:25.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:25 smithi062 ceph-mon[32390]: pgmap v328: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 96 B/s wr, 10 op/s 2024-06-10T00:43:25.834 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:25 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:43:25.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:25 smithi138 ceph-mon[36549]: pgmap v328: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 96 B/s wr, 10 op/s 2024-06-10T00:43:25.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:25 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:43:26.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:25 smithi105 ceph-mon[36321]: pgmap v328: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 96 B/s wr, 10 op/s 2024-06-10T00:43:26.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:25 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:43:26.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.209+0000 7f1380c4c700 1 -- 172.21.15.62:0/3214956149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f137c101620 msgr2=0x7f137c101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:26.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.209+0000 7f1380c4c700 1 --2- 172.21.15.62:0/3214956149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f137c101620 0x7f137c101a00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f136c0099c0 tx=0x7f136c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.209+0000 7f1380c4c700 1 -- 172.21.15.62:0/3214956149 shutdown_connections 2024-06-10T00:43:26.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.209+0000 7f1380c4c700 1 --2- 172.21.15.62:0/3214956149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f137c101620 0x7f137c101a00 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.209+0000 7f1380c4c700 1 --2- 172.21.15.62:0/3214956149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f137c1066b0 0x7f137c10e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.209+0000 7f1380c4c700 1 --2- 172.21.15.62:0/3214956149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f137c101fd0 0x7f137c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.209+0000 7f1380c4c700 1 -- 172.21.15.62:0/3214956149 >> 172.21.15.62:0/3214956149 conn(0x7f137c078340 msgr2=0x7f137c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:26.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.209+0000 7f1380c4c700 1 -- 172.21.15.62:0/3214956149 shutdown_connections 2024-06-10T00:43:26.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.209+0000 7f1380c4c700 1 -- 172.21.15.62:0/3214956149 wait complete. 2024-06-10T00:43:26.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f1380c4c700 1 Processor -- start 2024-06-10T00:43:26.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f1380c4c700 1 -- start start 2024-06-10T00:43:26.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f1380c4c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f137c101620 0x7f137c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f1380c4c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f137c101fd0 0x7f137c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f1380c4c700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f137c1066b0 0x7f137c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f1380c4c700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f137c111690 con 0x7f137c1066b0 2024-06-10T00:43:26.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f1380c4c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f137c111510 con 0x7f137c101620 2024-06-10T00:43:26.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f1380c4c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f137c111810 con 0x7f137c101fd0 2024-06-10T00:43:26.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f137ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f137c1066b0 0x7f137c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f137ad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f137c1066b0 0x7f137c198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59584/0 (socket says 172.21.15.62:59584) 2024-06-10T00:43:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f137c101fd0 0x7f137c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f137ad9d700 1 -- 172.21.15.62:0/3261621778 learned_addr learned my addr 172.21.15.62:0/3261621778 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f137a59c700 1 --2- 172.21.15.62:0/3261621778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f137c101620 0x7f137c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f137ad9d700 1 -- 172.21.15.62:0/3261621778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f137c101fd0 msgr2=0x7f137c19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f137ad9d700 1 --2- 172.21.15.62:0/3261621778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f137c101fd0 0x7f137c19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f137ad9d700 1 -- 172.21.15.62:0/3261621778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f137c101620 msgr2=0x7f137c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f137ad9d700 1 --2- 172.21.15.62:0/3261621778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f137c101620 0x7f137c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f137ad9d700 1 -- 172.21.15.62:0/3261621778 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137c1991a0 con 0x7f137c1066b0 2024-06-10T00:43:26.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.210+0000 7f1379d9b700 1 --2- 172.21.15.62:0/3261621778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f137c101fd0 0x7f137c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:43:26.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.211+0000 7f137ad9d700 1 --2- 172.21.15.62:0/3261621778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f137c1066b0 0x7f137c198a40 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1374007ee0 tx=0x7f137400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:26.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.211+0000 7f13737fe700 1 -- 172.21.15.62:0/3261621778 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137400dd20 con 0x7f137c1066b0 2024-06-10T00:43:26.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.211+0000 7f13737fe700 1 -- 172.21.15.62:0/3261621778 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1374004020 con 0x7f137c1066b0 2024-06-10T00:43:26.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.211+0000 7f1380c4c700 1 -- 172.21.15.62:0/3261621778 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137c199490 con 0x7f137c1066b0 2024-06-10T00:43:26.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.211+0000 7f13737fe700 1 -- 172.21.15.62:0/3261621778 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374015660 con 0x7f137c1066b0 2024-06-10T00:43:26.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.211+0000 7f1380c4c700 1 -- 172.21.15.62:0/3261621778 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f137c1a5750 con 0x7f137c1066b0 2024-06-10T00:43:26.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.213+0000 7f13737fe700 1 -- 172.21.15.62:0/3261621778 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f13740041d0 con 0x7f137c1066b0 2024-06-10T00:43:26.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.214+0000 7f13737fe700 1 --2- 172.21.15.62:0/3261621778 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f13680a2d90 0x7f13680a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.214+0000 7f137a59c700 1 --2- 172.21.15.62:0/3261621778 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f13680a2d90 0x7f13680a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.214+0000 7f13737fe700 1 -- 172.21.15.62:0/3261621778 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f13740cf210 con 0x7f137c1066b0 2024-06-10T00:43:26.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.214+0000 7f137a59c700 1 --2- 172.21.15.62:0/3261621778 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f13680a2d90 0x7f13680a5250 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f136c000c00 tx=0x7f136c005fb0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:26.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.215+0000 7f138b4cfb80 1 Processor -- start 2024-06-10T00:43:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.215+0000 7f138b4cfb80 1 -- start start 2024-06-10T00:43:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.215+0000 7f138b4cfb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74640 0x5555a5b74a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.215+0000 7f138b4cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74f60 0x5555a5b6e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.215+0000 7f138b4cfb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b6f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.215+0000 7f138b4cfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5555a5ac0810 con 0x5555a5b74f60 2024-06-10T00:43:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.215+0000 7f138b4cfb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5555a5a68720 con 0x5555a5b74640 2024-06-10T00:43:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.215+0000 7f138b4cfb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5555a5abc7d0 con 0x5555a5b6ef20 2024-06-10T00:43:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.215+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74f60 0x5555a5b6e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.215+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74f60 0x5555a5b6e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59586/0 (socket says 172.21.15.62:59586) 2024-06-10T00:43:26.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.215+0000 7f1379d9b700 1 -- 172.21.15.62:0/3480056642 learned_addr learned my addr 172.21.15.62:0/3480056642 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:26.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.215+0000 7f137ad9d700 1 --2- 172.21.15.62:0/3480056642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b6f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.215+0000 7f137a59c700 1 --2- 172.21.15.62:0/3480056642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74640 0x5555a5b74a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.216+0000 7f1379d9b700 1 -- 172.21.15.62:0/3480056642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 msgr2=0x5555a5b6f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:26.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.216+0000 7f1379d9b700 1 --2- 172.21.15.62:0/3480056642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b6f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.216+0000 7f1379d9b700 1 -- 172.21.15.62:0/3480056642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74640 msgr2=0x5555a5b74a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:26.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.216+0000 7f1379d9b700 1 --2- 172.21.15.62:0/3480056642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74640 0x5555a5b74a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.216+0000 7f1379d9b700 1 -- 172.21.15.62:0/3480056642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555a5a5ef50 con 0x5555a5b74f60 2024-06-10T00:43:26.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.216+0000 7f137a59c700 1 --2- 172.21.15.62:0/3480056642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74640 0x5555a5b74a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:43:26.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.216+0000 7f1379d9b700 1 --2- 172.21.15.62:0/3480056642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74f60 0x5555a5b6e9e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f136400f440 tx=0x7f136400f750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e9ef42a10f7e02 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:26.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.216+0000 7f1370ff9700 1 -- 172.21.15.62:0/3480056642 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364004a30 con 0x5555a5b74f60 2024-06-10T00:43:26.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.216+0000 7f1370ff9700 1 -- 172.21.15.62:0/3480056642 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1364005500 con 0x5555a5b74f60 2024-06-10T00:43:26.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.216+0000 7f1370ff9700 1 -- 172.21.15.62:0/3480056642 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13640189e0 con 0x5555a5b74f60 2024-06-10T00:43:26.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.217+0000 7f138b4cfb80 1 -- 172.21.15.62:0/3480056642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74f60 msgr2=0x5555a5b6e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:26.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.217+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/3480056642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74f60 0x5555a5b6e9e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f136400f440 tx=0x7f136400f750 comp rx=0 tx=0).stop 2024-06-10T00:43:26.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.217+0000 7f138b4cfb80 1 -- 172.21.15.62:0/3480056642 shutdown_connections 2024-06-10T00:43:26.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.217+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/3480056642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b6f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.217+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/3480056642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74640 0x5555a5b74a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.217+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/3480056642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74f60 0x5555a5b6e9e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.217+0000 7f138b4cfb80 1 -- 172.21.15.62:0/3480056642 >> 172.21.15.62:0/3480056642 conn(0x5555a5a99cc0 msgr2=0x5555a5b69850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:26.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.217+0000 7f138b4cfb80 1 -- 172.21.15.62:0/3480056642 shutdown_connections 2024-06-10T00:43:26.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.217+0000 7f138b4cfb80 1 -- 172.21.15.62:0/3480056642 wait complete. 2024-06-10T00:43:26.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.217+0000 7f138b4cfb80 1 -- start start 2024-06-10T00:43:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.217+0000 7f138b4cfb80 1 Processor -- start 2024-06-10T00:43:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f138b4cfb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b7aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f138b4cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b78010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f138b4cfb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 0x5555a5b78550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f138b4cfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5555a5b7b9d0 con 0x5555a5b74640 2024-06-10T00:43:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f138b4cfb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5555a5a60bf0 con 0x5555a5b74f60 2024-06-10T00:43:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f138b4cfb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5555a5b7bb50 con 0x5555a5b6ef20 2024-06-10T00:43:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f138b4cfb80 10 client.0 _ll_register_callbacks cb 0x5555a5a5a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:43:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b78010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b78010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59592/0 (socket says 172.21.15.62:59592) 2024-06-10T00:43:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f1379d9b700 1 -- 172.21.15.62:0/1935517510 learned_addr learned my addr 172.21.15.62:0/1935517510 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f137a59c700 1 --2- 172.21.15.62:0/1935517510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b7aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f137ad9d700 1 --2- 172.21.15.62:0/1935517510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 0x5555a5b78550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f1379d9b700 1 -- 172.21.15.62:0/1935517510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 msgr2=0x5555a5b7aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f1379d9b700 1 --2- 172.21.15.62:0/1935517510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b7aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f1379d9b700 1 -- 172.21.15.62:0/1935517510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 msgr2=0x5555a5b78550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f1379d9b700 1 --2- 172.21.15.62:0/1935517510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 0x5555a5b78550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f1379d9b700 1 -- 172.21.15.62:0/1935517510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555a5a0f8d0 con 0x5555a5b74640 2024-06-10T00:43:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f137ad9d700 1 --2- 172.21.15.62:0/1935517510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 0x5555a5b78550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:43:26.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f1379d9b700 1 --2- 172.21.15.62:0/1935517510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b78010 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1364018620 tx=0x7f13640058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d393cb19c9aa6d08 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:26.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f1361ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:43:26.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.218+0000 7f1361ffb700 1 -- 172.21.15.62:0/1935517510 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364004960 con 0x5555a5b74640 2024-06-10T00:43:26.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.219+0000 7f1361ffb700 1 -- 172.21.15.62:0/1935517510 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f1364004b00 con 0x5555a5b74640 2024-06-10T00:43:26.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.219+0000 7f138b4cfb80 1 -- 172.21.15.62:0/1935517510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1364005500 con 0x5555a5b74640 2024-06-10T00:43:26.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.219+0000 7f1361ffb700 1 -- 172.21.15.62:0/1935517510 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1364004020 con 0x5555a5b74640 2024-06-10T00:43:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.219+0000 7f138b4cfb80 10 client.17112 fetch_fsmap learned FSMap version 38 2024-06-10T00:43:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.219+0000 7f138b4cfb80 1 -- 172.21.15.62:0/1935517510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5555a5b86330 con 0x5555a5b74640 2024-06-10T00:43:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.219+0000 7f1361ffb700 1 -- 172.21.15.62:0/1935517510 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1364005500 con 0x5555a5b74640 2024-06-10T00:43:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.219+0000 7f1361ffb700 1 -- 172.21.15.62:0/1935517510 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1364003680 con 0x5555a5b74640 2024-06-10T00:43:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.219+0000 7f138b4cfb80 10 client.17112 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:43:26.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.219+0000 7f138b4cfb80 10 client.17112 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:43:26.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.219+0000 7f138b4cfb80 20 client.17112 populate_metadata read hostname 'smithi062' 2024-06-10T00:43:26.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.219+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/1935517510 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x5555a5b88970 0x5555a5b8ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.219+0000 7f138b4cfb80 4 client.17112 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:43:26.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.219+0000 7f138b4cfb80 1 -- 172.21.15.62:0/1935517510 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555a5acdd10 con 0x5555a5b88970 2024-06-10T00:43:26.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.219+0000 7f137a59c700 1 --2- 172.21.15.62:0/1935517510 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x5555a5b88970 0x5555a5b8ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.220+0000 7f137a59c700 1 --2- 172.21.15.62:0/1935517510 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x5555a5b88970 0x5555a5b8ad60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=21614dcef0f1ceba server_cookie=24776329a2418f33 in_seq=0 out_seq=0 2024-06-10T00:43:26.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.220+0000 7f1361ffb700 10 client.17112 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:43:26.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.220+0000 7f1361ffb700 1 -- 172.21.15.62:0/1935517510 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5555a5acdd10 con 0x5555a5b88970 2024-06-10T00:43:26.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.220+0000 7f1361ffb700 10 client.17112 handle_command_reply: tid=0 2024-06-10T00:43:26.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.220+0000 7f138b4cfb80 1 client.17112 shutdown 2024-06-10T00:43:26.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.221+0000 7f138b4cfb80 1 -- 172.21.15.62:0/1935517510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 msgr2=0x5555a5b78010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:26.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.221+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/1935517510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b78010 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1364018620 tx=0x7f13640058c0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.221+0000 7f138b4cfb80 1 -- 172.21.15.62:0/1935517510 shutdown_connections 2024-06-10T00:43:26.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.221+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/1935517510 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x5555a5b88970 0x5555a5b8ad60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.221+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/1935517510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 0x5555a5b78550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.221+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/1935517510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b7aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.221+0000 7f1361ffb700 0 client.17112 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:43:26.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.221+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/1935517510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b78010 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.221+0000 7f138b4cfb80 1 -- 172.21.15.62:0/1935517510 >> 172.21.15.62:0/1935517510 conn(0x5555a5a99cc0 msgr2=0x5555a5b6d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:26.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.221+0000 7f138b4cfb80 1 -- 172.21.15.62:0/1935517510 shutdown_connections 2024-06-10T00:43:26.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.221+0000 7f138b4cfb80 1 -- 172.21.15.62:0/1935517510 wait complete. 2024-06-10T00:43:26.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.221+0000 7f138b4cfb80 20 client.17112 trim_cache size 0 max 16384 2024-06-10T00:43:26.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.236+0000 7f138b4cfb80 1 Processor -- start 2024-06-10T00:43:26.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.236+0000 7f138b4cfb80 1 -- start start 2024-06-10T00:43:26.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.236+0000 7f138b4cfb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b84ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.236+0000 7f138b4cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b6e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.236+0000 7f138b4cfb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 0x5555a5b75b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.236+0000 7f138b4cfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f137c1ac520 con 0x5555a5b74640 2024-06-10T00:43:26.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.236+0000 7f138b4cfb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f137c1ae060 con 0x5555a5b74f60 2024-06-10T00:43:26.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.236+0000 7f138b4cfb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f137c1acbf0 con 0x5555a5b6ef20 2024-06-10T00:43:26.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b6e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b6e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59606/0 (socket says 172.21.15.62:59606) 2024-06-10T00:43:26.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f1379d9b700 1 -- 172.21.15.62:0/3809790732 learned_addr learned my addr 172.21.15.62:0/3809790732 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:26.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f137a59c700 1 --2- 172.21.15.62:0/3809790732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b84ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f1379d9b700 1 -- 172.21.15.62:0/3809790732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 msgr2=0x5555a5b84ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:26.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f1379d9b700 1 --2- 172.21.15.62:0/3809790732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b84ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f1379d9b700 1 -- 172.21.15.62:0/3809790732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 msgr2=0x5555a5b75b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:26.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f137ad9d700 1 --2- 172.21.15.62:0/3809790732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 0x5555a5b75b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f1379d9b700 1 --2- 172.21.15.62:0/3809790732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 0x5555a5b75b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f1379d9b700 1 -- 172.21.15.62:0/3809790732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555a5b78930 con 0x5555a5b74640 2024-06-10T00:43:26.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f137a59c700 1 --2- 172.21.15.62:0/3809790732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b84ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:43:26.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f137ad9d700 1 --2- 172.21.15.62:0/3809790732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 0x5555a5b75b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:43:26.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f1379d9b700 1 --2- 172.21.15.62:0/3809790732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b6e9e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1364005950 tx=0x7f1364005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7cbf3187e86650 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:26.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f1363fff700 1 -- 172.21.15.62:0/3809790732 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364005450 con 0x5555a5b74640 2024-06-10T00:43:26.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f1363fff700 1 -- 172.21.15.62:0/3809790732 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1364003cc0 con 0x5555a5b74640 2024-06-10T00:43:26.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.237+0000 7f1363fff700 1 -- 172.21.15.62:0/3809790732 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364018650 con 0x5555a5b74640 2024-06-10T00:43:26.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.238+0000 7f138b4cfb80 1 -- 172.21.15.62:0/3809790732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 msgr2=0x5555a5b6e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:26.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.238+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/3809790732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b6e9e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1364005950 tx=0x7f1364005df0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.238+0000 7f138b4cfb80 1 -- 172.21.15.62:0/3809790732 shutdown_connections 2024-06-10T00:43:26.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.238+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/3809790732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 0x5555a5b75b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.238+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/3809790732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b84ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.238+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/3809790732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b6e9e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.238+0000 7f138b4cfb80 1 -- 172.21.15.62:0/3809790732 >> 172.21.15.62:0/3809790732 conn(0x5555a5a99cc0 msgr2=0x5555a5b75400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:26.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.238+0000 7f138b4cfb80 1 -- 172.21.15.62:0/3809790732 shutdown_connections 2024-06-10T00:43:26.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.238+0000 7f138b4cfb80 1 -- 172.21.15.62:0/3809790732 wait complete. 2024-06-10T00:43:26.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.238+0000 7f138b4cfb80 1 -- start start 2024-06-10T00:43:26.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f138b4cfb80 1 Processor -- start 2024-06-10T00:43:26.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f138b4cfb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b7dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f138b4cfb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b7e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f138b4cfb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 0x5555a5b7e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f138b4cfb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f137c1ad990 con 0x5555a5b74640 2024-06-10T00:43:26.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f138b4cfb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f137c1ad2c0 con 0x5555a5b74f60 2024-06-10T00:43:26.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f138b4cfb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f137c1aa4e0 con 0x5555a5b6ef20 2024-06-10T00:43:26.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f138b4cfb80 10 client.0 _ll_register_callbacks cb 0x7f1374009480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:43:26.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b7e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f1379d9b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b7e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59610/0 (socket says 172.21.15.62:59610) 2024-06-10T00:43:26.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f137a59c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b7dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f1379d9b700 1 -- 172.21.15.62:0/2656121471 learned_addr learned my addr 172.21.15.62:0/2656121471 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:26.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f137ad9d700 1 --2- 172.21.15.62:0/2656121471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 0x5555a5b7e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f1379d9b700 1 -- 172.21.15.62:0/2656121471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 msgr2=0x5555a5b7dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:26.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f1379d9b700 1 --2- 172.21.15.62:0/2656121471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b7dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f1379d9b700 1 -- 172.21.15.62:0/2656121471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 msgr2=0x5555a5b7e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:26.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f1379d9b700 1 --2- 172.21.15.62:0/2656121471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 0x5555a5b7e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.239+0000 7f1379d9b700 1 -- 172.21.15.62:0/2656121471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555a5bf1450 con 0x5555a5b74640 2024-06-10T00:43:26.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.240+0000 7f1379d9b700 1 --2- 172.21.15.62:0/2656121471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b7e320 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1364005ec0 tx=0x7f1364005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729744eb07ffeab7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:26.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.240+0000 7f1361ffb700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:43:26.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.240+0000 7f1361ffb700 1 -- 172.21.15.62:0/2656121471 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13640043d0 con 0x5555a5b74640 2024-06-10T00:43:26.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.240+0000 7f1361ffb700 1 -- 172.21.15.62:0/2656121471 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f136403e070 con 0x5555a5b74640 2024-06-10T00:43:26.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.240+0000 7f138b4cfb80 1 -- 172.21.15.62:0/2656121471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1364003cc0 con 0x5555a5b74640 2024-06-10T00:43:26.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.240+0000 7f1361ffb700 1 -- 172.21.15.62:0/2656121471 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f136404b050 con 0x5555a5b74640 2024-06-10T00:43:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.240+0000 7f138b4cfb80 10 client.17118 fetch_fsmap learned FSMap version 38 2024-06-10T00:43:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.240+0000 7f138b4cfb80 1 -- 172.21.15.62:0/2656121471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5555a5bf2640 con 0x5555a5b74640 2024-06-10T00:43:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.240+0000 7f1361ffb700 1 -- 172.21.15.62:0/2656121471 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1364003cc0 con 0x5555a5b74640 2024-06-10T00:43:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.240+0000 7f1361ffb700 1 -- 172.21.15.62:0/2656121471 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f13640205d0 con 0x5555a5b74640 2024-06-10T00:43:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.241+0000 7f138b4cfb80 10 client.17118 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:43:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.241+0000 7f138b4cfb80 10 client.17118 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-06-10T00:43:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.241+0000 7f138b4cfb80 20 client.17118 populate_metadata read hostname 'smithi062' 2024-06-10T00:43:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.241+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/2656121471 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x5555a5bf2c00 0x5555a5bf5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.241+0000 7f138b4cfb80 4 client.17118 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:43:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.241+0000 7f138b4cfb80 1 -- 172.21.15.62:0/2656121471 --> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555a5ac8290 con 0x5555a5bf2c00 2024-06-10T00:43:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.241+0000 7f137a59c700 1 --2- 172.21.15.62:0/2656121471 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x5555a5bf2c00 0x5555a5bf5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.241+0000 7f137a59c700 1 --2- 172.21.15.62:0/2656121471 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x5555a5bf2c00 0x5555a5bf5020 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c738fbb1bf6469a server_cookie=cec2fb4e0af8f31f in_seq=0 out_seq=0 2024-06-10T00:43:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.241+0000 7f1361ffb700 10 client.17118 ms_handle_connect on v2:172.21.15.138:6834/331298815 2024-06-10T00:43:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.241+0000 7f1361ffb700 1 -- 172.21.15.62:0/2656121471 <== mds.4 v2:172.21.15.138:6834/331298815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555a5ac8290 con 0x5555a5bf2c00 2024-06-10T00:43:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.241+0000 7f1361ffb700 10 client.17118 handle_command_reply: tid=0 2024-06-10T00:43:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.241+0000 7f138b4cfb80 1 client.17118 shutdown 2024-06-10T00:43:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f138b4cfb80 1 -- 172.21.15.62:0/2656121471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 msgr2=0x5555a5b7e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/2656121471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b7e320 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1364005ec0 tx=0x7f1364005c30 comp rx=0 tx=0).stop 2024-06-10T00:43:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f138b4cfb80 1 -- 172.21.15.62:0/2656121471 shutdown_connections 2024-06-10T00:43:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/2656121471 >> [v2:172.21.15.138:6834/331298815,v1:172.21.15.138:6835/331298815] conn(0x5555a5bf2c00 0x5555a5bf5020 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/2656121471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555a5b74f60 0x5555a5b7e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/2656121471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555a5b6ef20 0x5555a5b7dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f138b4cfb80 1 --2- 172.21.15.62:0/2656121471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5555a5b74640 0x5555a5b7e320 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f138b4cfb80 1 -- 172.21.15.62:0/2656121471 >> 172.21.15.62:0/2656121471 conn(0x5555a5a99cc0 msgr2=0x5555a5b62e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:26.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f1361ffb700 0 client.17118 ms_handle_reset on v2:172.21.15.138:6834/331298815 2024-06-10T00:43:26.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f138b4cfb80 1 -- 172.21.15.62:0/2656121471 shutdown_connections 2024-06-10T00:43:26.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f138b4cfb80 1 -- 172.21.15.62:0/2656121471 wait complete. 2024-06-10T00:43:26.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f138b4cfb80 20 client.17118 trim_cache size 0 max 16384 2024-06-10T00:43:26.255 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:43:26.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f1380c4c700 1 -- 172.21.15.62:0/3261621778 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f13680a2d90 msgr2=0x7f13680a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:26.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f1380c4c700 1 --2- 172.21.15.62:0/3261621778 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f13680a2d90 0x7f13680a5250 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f136c000c00 tx=0x7f136c005fb0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f1380c4c700 1 -- 172.21.15.62:0/3261621778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f137c1066b0 msgr2=0x7f137c198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.242+0000 7f1380c4c700 1 --2- 172.21.15.62:0/3261621778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f137c1066b0 0x7f137c198a40 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1374007ee0 tx=0x7f137400d3b0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.243+0000 7f1380c4c700 1 -- 172.21.15.62:0/3261621778 shutdown_connections 2024-06-10T00:43:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.243+0000 7f1380c4c700 1 --2- 172.21.15.62:0/3261621778 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f13680a2d90 0x7f13680a5250 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.243+0000 7f1380c4c700 1 --2- 172.21.15.62:0/3261621778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f137c1066b0 0x7f137c198a40 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.243+0000 7f1380c4c700 1 --2- 172.21.15.62:0/3261621778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f137c101620 0x7f137c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.243+0000 7f1380c4c700 1 --2- 172.21.15.62:0/3261621778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f137c101fd0 0x7f137c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.243+0000 7f1380c4c700 1 -- 172.21.15.62:0/3261621778 >> 172.21.15.62:0/3261621778 conn(0x7f137c078340 msgr2=0x7f137c06af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.243+0000 7f1380c4c700 1 -- 172.21.15.62:0/3261621778 shutdown_connections 2024-06-10T00:43:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:26.243+0000 7f1380c4c700 1 -- 172.21.15.62:0/3261621778 wait complete. 2024-06-10T00:43:26.901 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-10T00:43:27.655 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:43:27.804 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:27 smithi062 ceph-mon[32390]: pgmap v329: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:43:27.804 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:27 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:27.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:27 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:27.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:27 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:27.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:27 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:27.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:27 smithi138 ceph-mon[36549]: pgmap v329: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:43:27.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:27 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:27.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:27 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:27.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:27 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:27.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:27 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:28.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:27 smithi105 ceph-mon[36321]: pgmap v329: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-10T00:43:28.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:27 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:28.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:27 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:28.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:27 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:28.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:27 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:28.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:28 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:28 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:28 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:28 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:28 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-06-10T00:43:28.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:28 smithi062 ceph-mon[32390]: Unable to set osd_memory_target on smithi138 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T00:43:28.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:28 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:28 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:28 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:28 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:28 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:28 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:28 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:28 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:28 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-06-10T00:43:28.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:28 smithi138 ceph-mon[36549]: Unable to set osd_memory_target on smithi138 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T00:43:28.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:28 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:28 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.875 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:28 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:28.875 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:28 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:29.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:28 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:29.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:28 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:29.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:28 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:29.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:28 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:29.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:28 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-06-10T00:43:29.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:28 smithi105 ceph-mon[36321]: Unable to set osd_memory_target on smithi138 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-06-10T00:43:29.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:28 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:29.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:28 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:29.034 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:28 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:29.034 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:28 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:29.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:29 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-06-10T00:43:29.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:29 smithi062 ceph-mon[32390]: Unable to set osd_memory_target on smithi105 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-06-10T00:43:29.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:29 smithi062 ceph-mon[32390]: pgmap v330: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:43:29.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.576+0000 7f4d77fd5700 1 -- 172.21.15.62:0/606874543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70103120 msgr2=0x7f4d70107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:29.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.576+0000 7f4d77fd5700 1 --2- 172.21.15.62:0/606874543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70103120 0x7f4d70107130 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4d600099c0 tx=0x7f4d60009cd0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.576+0000 7f4d77fd5700 1 -- 172.21.15.62:0/606874543 shutdown_connections 2024-06-10T00:43:29.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.576+0000 7f4d77fd5700 1 --2- 172.21.15.62:0/606874543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d70107860 0x7f4d7010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.576+0000 7f4d77fd5700 1 --2- 172.21.15.62:0/606874543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d70102770 0x7f4d70102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.576+0000 7f4d77fd5700 1 --2- 172.21.15.62:0/606874543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70103120 0x7f4d70107130 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.576+0000 7f4d77fd5700 1 -- 172.21.15.62:0/606874543 >> 172.21.15.62:0/606874543 conn(0x7f4d70079600 msgr2=0x7f4d70079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:29.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.576+0000 7f4d77fd5700 1 -- 172.21.15.62:0/606874543 shutdown_connections 2024-06-10T00:43:29.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.576+0000 7f4d77fd5700 1 -- 172.21.15.62:0/606874543 wait complete. 2024-06-10T00:43:29.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.577+0000 7f4d77fd5700 1 Processor -- start 2024-06-10T00:43:29.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.577+0000 7f4d77fd5700 1 -- start start 2024-06-10T00:43:29.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.577+0000 7f4d77fd5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d70102770 0x7f4d70074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.577+0000 7f4d77fd5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70103120 0x7f4d70074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.577+0000 7f4d77fd5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d70107860 0x7f4d70075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.577+0000 7f4d77fd5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4d70112770 con 0x7f4d70103120 2024-06-10T00:43:29.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.577+0000 7f4d77fd5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701125f0 con 0x7f4d70107860 2024-06-10T00:43:29.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.577+0000 7f4d77fd5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701128f0 con 0x7f4d70102770 2024-06-10T00:43:29.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d75570700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70103120 0x7f4d70074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d75570700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70103120 0x7f4d70074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37126/0 (socket says 172.21.15.62:37126) 2024-06-10T00:43:29.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d75570700 1 -- 172.21.15.62:0/2232148898 learned_addr learned my addr 172.21.15.62:0/2232148898 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:29.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d75d71700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d70102770 0x7f4d70074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d76572700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d70107860 0x7f4d70075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d75570700 1 -- 172.21.15.62:0/2232148898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d70102770 msgr2=0x7f4d70074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:29.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d75570700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d70102770 0x7f4d70074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d75570700 1 -- 172.21.15.62:0/2232148898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d70107860 msgr2=0x7f4d70075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:29.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d75570700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d70107860 0x7f4d70075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d75570700 1 -- 172.21.15.62:0/2232148898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d701a70f0 con 0x7f4d70103120 2024-06-10T00:43:29.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d76572700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d70107860 0x7f4d70075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:43:29.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d75570700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70103120 0x7f4d70074cd0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4d60006010 tx=0x7f4d60009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:29.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d66ffd700 1 -- 172.21.15.62:0/2232148898 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60027070 con 0x7f4d70103120 2024-06-10T00:43:29.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d77fd5700 1 -- 172.21.15.62:0/2232148898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d701a7380 con 0x7f4d70103120 2024-06-10T00:43:29.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d66ffd700 1 -- 172.21.15.62:0/2232148898 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d60003780 con 0x7f4d70103120 2024-06-10T00:43:29.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.578+0000 7f4d66ffd700 1 -- 172.21.15.62:0/2232148898 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60022da0 con 0x7f4d70103120 2024-06-10T00:43:29.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.579+0000 7f4d77fd5700 1 -- 172.21.15.62:0/2232148898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d701a77d0 con 0x7f4d70103120 2024-06-10T00:43:29.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.581+0000 7f4d66ffd700 1 -- 172.21.15.62:0/2232148898 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4d6002a070 con 0x7f4d70103120 2024-06-10T00:43:29.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.582+0000 7f4d66ffd700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4d5c0a2d90 0x7f4d5c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.582+0000 7f4d75d71700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4d5c0a2d90 0x7f4d5c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.582+0000 7f4d66ffd700 1 -- 172.21.15.62:0/2232148898 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f4d600d06d0 con 0x7f4d70103120 2024-06-10T00:43:29.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.582+0000 7f4d75d71700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4d5c0a2d90 0x7f4d5c0a5250 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c005950 tx=0x7f4d6c00b410 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:29.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.582+0000 7f4d82858b80 1 Processor -- start 2024-06-10T00:43:29.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d82858b80 1 -- start start 2024-06-10T00:43:29.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d82858b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c7bf60 0x55ac12c7c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d82858b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7c880 0x55ac12c76260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d82858b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c767a0 0x55ac12c76bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d82858b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ac12b68f60 con 0x55ac12c7bf60 2024-06-10T00:43:29.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d82858b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ac12bc7850 con 0x55ac12c767a0 2024-06-10T00:43:29.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d82858b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ac12b2f460 con 0x55ac12c7c880 2024-06-10T00:43:29.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d75d71700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c7bf60 0x55ac12c7c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d75d71700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c7bf60 0x55ac12c7c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37138/0 (socket says 172.21.15.62:37138) 2024-06-10T00:43:29.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d75d71700 1 -- 172.21.15.62:0/1844950382 learned_addr learned my addr 172.21.15.62:0/1844950382 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:29.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d75570700 1 --2- 172.21.15.62:0/1844950382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7c880 0x55ac12c76260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d76572700 1 --2- 172.21.15.62:0/1844950382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c767a0 0x55ac12c76bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d75d71700 1 -- 172.21.15.62:0/1844950382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7c880 msgr2=0x55ac12c76260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:29.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d75d71700 1 --2- 172.21.15.62:0/1844950382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7c880 0x55ac12c76260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d75d71700 1 -- 172.21.15.62:0/1844950382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c767a0 msgr2=0x55ac12c76bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:29.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d75d71700 1 --2- 172.21.15.62:0/1844950382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c767a0 0x55ac12c76bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.583+0000 7f4d75d71700 1 -- 172.21.15.62:0/1844950382 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac12b168d0 con 0x55ac12c7bf60 2024-06-10T00:43:29.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.584+0000 7f4d75d71700 1 --2- 172.21.15.62:0/1844950382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c7bf60 0x55ac12c7c340 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c010fa0 tx=0x7f4d6c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95d0bce51f33295 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:29.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.584+0000 7f4d5bfff700 1 -- 172.21.15.62:0/1844950382 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c00be60 con 0x55ac12c7bf60 2024-06-10T00:43:29.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.584+0000 7f4d5bfff700 1 -- 172.21.15.62:0/1844950382 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d6c01eb00 con 0x55ac12c7bf60 2024-06-10T00:43:29.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.584+0000 7f4d5bfff700 1 -- 172.21.15.62:0/1844950382 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c01cc80 con 0x55ac12c7bf60 2024-06-10T00:43:29.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.585+0000 7f4d82858b80 1 -- 172.21.15.62:0/1844950382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c7bf60 msgr2=0x55ac12c7c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:29.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.585+0000 7f4d82858b80 1 --2- 172.21.15.62:0/1844950382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c7bf60 0x55ac12c7c340 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c010fa0 tx=0x7f4d6c013000 comp rx=0 tx=0).stop 2024-06-10T00:43:29.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.585+0000 7f4d82858b80 1 -- 172.21.15.62:0/1844950382 shutdown_connections 2024-06-10T00:43:29.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.585+0000 7f4d82858b80 1 --2- 172.21.15.62:0/1844950382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c7bf60 0x55ac12c7c340 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.585+0000 7f4d82858b80 1 --2- 172.21.15.62:0/1844950382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c767a0 0x55ac12c76bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.585+0000 7f4d82858b80 1 --2- 172.21.15.62:0/1844950382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7c880 0x55ac12c76260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.585+0000 7f4d82858b80 1 -- 172.21.15.62:0/1844950382 >> 172.21.15.62:0/1844950382 conn(0x55ac12ba3080 msgr2=0x55ac12c71220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:29.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.585+0000 7f4d82858b80 1 -- 172.21.15.62:0/1844950382 shutdown_connections 2024-06-10T00:43:29.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.585+0000 7f4d82858b80 1 -- 172.21.15.62:0/1844950382 wait complete. 2024-06-10T00:43:29.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.585+0000 7f4d82858b80 1 -- start start 2024-06-10T00:43:29.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.586+0000 7f4d82858b80 1 Processor -- start 2024-06-10T00:43:29.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.586+0000 7f4d82858b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c767a0 0x55ac12c82610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.586+0000 7f4d82858b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c7bf60 0x55ac12c7f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.586+0000 7f4d82858b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c7fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.586+0000 7f4d82858b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ac12b67fc0 con 0x55ac12c7bf60 2024-06-10T00:43:29.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.586+0000 7f4d82858b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ac12bc3810 con 0x55ac12c7c880 2024-06-10T00:43:29.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.586+0000 7f4d82858b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ac12c83260 con 0x55ac12c767a0 2024-06-10T00:43:29.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.586+0000 7f4d82858b80 10 client.0 _ll_register_callbacks cb 0x55ac12b5d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:43:29.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.586+0000 7f4d76572700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c7fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.586+0000 7f4d75570700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c7bf60 0x55ac12c7f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.586+0000 7f4d75d71700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c767a0 0x55ac12c82610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.586+0000 7f4d76572700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c7fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:50526/0 (socket says 172.21.15.62:50526) 2024-06-10T00:43:29.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.586+0000 7f4d76572700 1 -- 172.21.15.62:0/2724955766 learned_addr learned my addr 172.21.15.62:0/2724955766 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:29.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d76572700 1 -- 172.21.15.62:0/2724955766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c767a0 msgr2=0x55ac12c82610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:29.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d76572700 1 --2- 172.21.15.62:0/2724955766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c767a0 0x55ac12c82610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d76572700 1 -- 172.21.15.62:0/2724955766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c7bf60 msgr2=0x55ac12c7f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:29.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d76572700 1 --2- 172.21.15.62:0/2724955766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c7bf60 0x55ac12c7f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d76572700 1 -- 172.21.15.62:0/2724955766 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac12b8d750 con 0x55ac12c7c880 2024-06-10T00:43:29.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d75d71700 1 --2- 172.21.15.62:0/2724955766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c767a0 0x55ac12c82610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:43:29.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d75570700 1 --2- 172.21.15.62:0/2724955766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c7bf60 0x55ac12c7f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:43:29.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d76572700 1 --2- 172.21.15.62:0/2724955766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c7fe50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4d68007940 tx=0x7f4d68007c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e1d7466a7167672 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:29.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d597fa700 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-06-10T00:43:29.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d597fa700 1 -- 172.21.15.62:0/2724955766 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68019020 con 0x55ac12c7c880 2024-06-10T00:43:29.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d82858b80 1 -- 172.21.15.62:0/2724955766 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d6c01eb00 con 0x55ac12c7c880 2024-06-10T00:43:29.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d597fa700 1 -- 172.21.15.62:0/2724955766 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f4d68010670 con 0x55ac12c7c880 2024-06-10T00:43:29.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d597fa700 1 -- 172.21.15.62:0/2724955766 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4d68024050 con 0x55ac12c7c880 2024-06-10T00:43:29.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d82858b80 10 client.26405 fetch_fsmap learned FSMap version 38 2024-06-10T00:43:29.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d82858b80 1 -- 172.21.15.62:0/2724955766 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ac12c8dac0 con 0x55ac12c7c880 2024-06-10T00:43:29.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.587+0000 7f4d597fa700 1 -- 172.21.15.62:0/2724955766 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d6c01eb00 con 0x55ac12c7c880 2024-06-10T00:43:29.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.588+0000 7f4d597fa700 1 -- 172.21.15.62:0/2724955766 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f4d68015bb0 con 0x55ac12c7c880 2024-06-10T00:43:29.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.588+0000 7f4d82858b80 10 client.26405 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:43:29.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.588+0000 7f4d82858b80 10 client.26405 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:43:29.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.588+0000 7f4d82858b80 20 client.26405 populate_metadata read hostname 'smithi062' 2024-06-10T00:43:29.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.588+0000 7f4d82858b80 1 --2- 172.21.15.62:0/2724955766 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55ac12c900a0 0x55ac12c92490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.588+0000 7f4d82858b80 4 client.26405 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:43:29.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.588+0000 7f4d82858b80 1 -- 172.21.15.62:0/2724955766 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac12bd4d50 con 0x55ac12c900a0 2024-06-10T00:43:29.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.588+0000 7f4d75d71700 1 --2- 172.21.15.62:0/2724955766 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55ac12c900a0 0x55ac12c92490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.589+0000 7f4d597fa700 10 client.26405 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:43:29.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.589+0000 7f4d75d71700 1 --2- 172.21.15.62:0/2724955766 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55ac12c900a0 0x55ac12c92490 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e98e616a5f0569fe server_cookie=d163d94e1f1321d1 in_seq=0 out_seq=0 2024-06-10T00:43:29.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.589+0000 7f4d597fa700 1 -- 172.21.15.62:0/2724955766 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ac12bd4d50 con 0x55ac12c900a0 2024-06-10T00:43:29.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.589+0000 7f4d597fa700 10 client.26405 handle_command_reply: tid=0 2024-06-10T00:43:29.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.589+0000 7f4d82858b80 1 client.26405 shutdown 2024-06-10T00:43:29.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.590+0000 7f4d82858b80 1 -- 172.21.15.62:0/2724955766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 msgr2=0x55ac12c7fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:29.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.590+0000 7f4d82858b80 1 --2- 172.21.15.62:0/2724955766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c7fe50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4d68007940 tx=0x7f4d68007c50 comp rx=0 tx=0).stop 2024-06-10T00:43:29.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.590+0000 7f4d82858b80 1 -- 172.21.15.62:0/2724955766 shutdown_connections 2024-06-10T00:43:29.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.590+0000 7f4d82858b80 1 --2- 172.21.15.62:0/2724955766 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55ac12c900a0 0x55ac12c92490 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.590+0000 7f4d82858b80 1 --2- 172.21.15.62:0/2724955766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c7fe50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.590+0000 7f4d82858b80 1 --2- 172.21.15.62:0/2724955766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c767a0 0x55ac12c82610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.590+0000 7f4d82858b80 1 --2- 172.21.15.62:0/2724955766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c7bf60 0x55ac12c7f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.590+0000 7f4d597fa700 0 client.26405 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:43:29.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.590+0000 7f4d82858b80 1 -- 172.21.15.62:0/2724955766 >> 172.21.15.62:0/2724955766 conn(0x55ac12ba3080 msgr2=0x55ac12c70700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:29.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.590+0000 7f4d82858b80 1 -- 172.21.15.62:0/2724955766 shutdown_connections 2024-06-10T00:43:29.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.590+0000 7f4d82858b80 1 -- 172.21.15.62:0/2724955766 wait complete. 2024-06-10T00:43:29.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.590+0000 7f4d82858b80 20 client.26405 trim_cache size 0 max 16384 2024-06-10T00:43:29.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d82858b80 1 Processor -- start 2024-06-10T00:43:29.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d82858b80 1 -- start start 2024-06-10T00:43:29.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d82858b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c767a0 0x55ac12c82150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d82858b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7bf60 0x55ac12c70e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d82858b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d82858b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701ac400 con 0x55ac12c767a0 2024-06-10T00:43:29.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d82858b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701adf40 con 0x55ac12c7c880 2024-06-10T00:43:29.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d82858b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701acad0 con 0x55ac12c7bf60 2024-06-10T00:43:29.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d75d71700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c767a0 0x55ac12c82150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d75d71700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c767a0 0x55ac12c82150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37156/0 (socket says 172.21.15.62:37156) 2024-06-10T00:43:29.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d75d71700 1 -- 172.21.15.62:0/2724786541 learned_addr learned my addr 172.21.15.62:0/2724786541 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:29.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d75570700 1 --2- 172.21.15.62:0/2724786541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7bf60 0x55ac12c70e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d76572700 1 --2- 172.21.15.62:0/2724786541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d75d71700 1 -- 172.21.15.62:0/2724786541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7bf60 msgr2=0x55ac12c70e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:29.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d75d71700 1 --2- 172.21.15.62:0/2724786541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7bf60 0x55ac12c70e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d75d71700 1 -- 172.21.15.62:0/2724786541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 msgr2=0x55ac12c713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:29.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d75d71700 1 --2- 172.21.15.62:0/2724786541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.606+0000 7f4d75d71700 1 -- 172.21.15.62:0/2724786541 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac12c7de80 con 0x55ac12c767a0 2024-06-10T00:43:29.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.607+0000 7f4d75d71700 1 --2- 172.21.15.62:0/2724786541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c767a0 0x55ac12c82150 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c010f40 tx=0x7f4d6c02a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46f3188ee802276 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:29.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.607+0000 7f4d5b7fe700 1 -- 172.21.15.62:0/2724786541 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c01c640 con 0x55ac12c767a0 2024-06-10T00:43:29.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.607+0000 7f4d5b7fe700 1 -- 172.21.15.62:0/2724786541 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d6c005040 con 0x55ac12c767a0 2024-06-10T00:43:29.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.607+0000 7f4d5b7fe700 1 -- 172.21.15.62:0/2724786541 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0049a0 con 0x55ac12c767a0 2024-06-10T00:43:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.608+0000 7f4d82858b80 1 -- 172.21.15.62:0/2724786541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c767a0 msgr2=0x55ac12c82150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.608+0000 7f4d82858b80 1 --2- 172.21.15.62:0/2724786541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c767a0 0x55ac12c82150 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c010f40 tx=0x7f4d6c02a040 comp rx=0 tx=0).stop 2024-06-10T00:43:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.608+0000 7f4d82858b80 1 -- 172.21.15.62:0/2724786541 shutdown_connections 2024-06-10T00:43:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.608+0000 7f4d82858b80 1 --2- 172.21.15.62:0/2724786541 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c767a0 0x55ac12c82150 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.608+0000 7f4d82858b80 1 --2- 172.21.15.62:0/2724786541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.608+0000 7f4d82858b80 1 --2- 172.21.15.62:0/2724786541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7bf60 0x55ac12c70e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.608+0000 7f4d82858b80 1 -- 172.21.15.62:0/2724786541 >> 172.21.15.62:0/2724786541 conn(0x55ac12ba3080 msgr2=0x55ac12c7a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:29.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.608+0000 7f4d82858b80 1 -- 172.21.15.62:0/2724786541 shutdown_connections 2024-06-10T00:43:29.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.608+0000 7f4d82858b80 1 -- 172.21.15.62:0/2724786541 wait complete. 2024-06-10T00:43:29.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.608+0000 7f4d82858b80 1 -- start start 2024-06-10T00:43:29.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.608+0000 7f4d82858b80 1 Processor -- start 2024-06-10T00:43:29.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d82858b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c767a0 0x55ac12c88900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d82858b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7bf60 0x55ac12c88e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d82858b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c97500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d82858b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701ad870 con 0x55ac12c767a0 2024-06-10T00:43:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d82858b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701ad1a0 con 0x55ac12c7c880 2024-06-10T00:43:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d82858b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701aa3c0 con 0x55ac12c7bf60 2024-06-10T00:43:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d82858b80 10 client.0 _ll_register_callbacks cb 0x55ac12b5d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:43:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d75d71700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c767a0 0x55ac12c88900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d75d71700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c767a0 0x55ac12c88900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37164/0 (socket says 172.21.15.62:37164) 2024-06-10T00:43:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d75d71700 1 -- 172.21.15.62:0/3535876073 learned_addr learned my addr 172.21.15.62:0/3535876073 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d76572700 1 --2- 172.21.15.62:0/3535876073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c97500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d75570700 1 --2- 172.21.15.62:0/3535876073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7bf60 0x55ac12c88e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d75d71700 1 -- 172.21.15.62:0/3535876073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7bf60 msgr2=0x55ac12c88e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:29.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d75d71700 1 --2- 172.21.15.62:0/3535876073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7bf60 0x55ac12c88e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d75d71700 1 -- 172.21.15.62:0/3535876073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 msgr2=0x55ac12c97500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:29.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d75d71700 1 --2- 172.21.15.62:0/3535876073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c97500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d75d71700 1 -- 172.21.15.62:0/3535876073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac12c97d40 con 0x55ac12c767a0 2024-06-10T00:43:29.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.609+0000 7f4d76572700 1 --2- 172.21.15.62:0/3535876073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c97500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:43:29.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.610+0000 7f4d75d71700 1 --2- 172.21.15.62:0/3535876073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c767a0 0x55ac12c88900 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c006010 tx=0x7f4d6c010a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205fe942f017193a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:29.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.610+0000 7f4d597fa700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:43:29.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.610+0000 7f4d597fa700 1 -- 172.21.15.62:0/3535876073 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c005570 con 0x55ac12c767a0 2024-06-10T00:43:29.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.610+0000 7f4d82858b80 1 -- 172.21.15.62:0/3535876073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d6c005040 con 0x55ac12c767a0 2024-06-10T00:43:29.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.610+0000 7f4d597fa700 1 -- 172.21.15.62:0/3535876073 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f4d6c004690 con 0x55ac12c767a0 2024-06-10T00:43:29.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.610+0000 7f4d597fa700 1 -- 172.21.15.62:0/3535876073 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4d6c004890 con 0x55ac12c767a0 2024-06-10T00:43:29.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.610+0000 7f4d82858b80 10 client.17136 fetch_fsmap learned FSMap version 38 2024-06-10T00:43:29.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.610+0000 7f4d82858b80 1 -- 172.21.15.62:0/3535876073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ac12cfaa10 con 0x55ac12c767a0 2024-06-10T00:43:29.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.610+0000 7f4d597fa700 1 -- 172.21.15.62:0/3535876073 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d6c005040 con 0x55ac12c767a0 2024-06-10T00:43:29.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.610+0000 7f4d597fa700 1 -- 172.21.15.62:0/3535876073 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f4d6c01cc60 con 0x55ac12c767a0 2024-06-10T00:43:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.611+0000 7f4d82858b80 10 client.17136 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:43:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.611+0000 7f4d82858b80 10 client.17136 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-10T00:43:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.611+0000 7f4d82858b80 20 client.17136 populate_metadata read hostname 'smithi062' 2024-06-10T00:43:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.611+0000 7f4d82858b80 1 --2- 172.21.15.62:0/3535876073 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55ac12cfaf10 0x55ac12cfd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.611+0000 7f4d82858b80 4 client.17136 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:43:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.611+0000 7f4d82858b80 1 -- 172.21.15.62:0/3535876073 --> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac12bcf2d0 con 0x55ac12cfaf10 2024-06-10T00:43:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.611+0000 7f4d75570700 1 --2- 172.21.15.62:0/3535876073 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55ac12cfaf10 0x55ac12cfd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.611+0000 7f4d597fa700 10 client.17136 ms_handle_connect on v2:172.21.15.138:6836/3880084398 2024-06-10T00:43:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.611+0000 7f4d75570700 1 --2- 172.21.15.62:0/3535876073 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55ac12cfaf10 0x55ac12cfd300 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef2d72a5e4d1dd75 server_cookie=8980110147bac5df in_seq=0 out_seq=0 2024-06-10T00:43:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.611+0000 7f4d597fa700 1 -- 172.21.15.62:0/3535876073 <== mds.3 v2:172.21.15.138:6836/3880084398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac12bcf2d0 con 0x55ac12cfaf10 2024-06-10T00:43:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.611+0000 7f4d597fa700 10 client.17136 handle_command_reply: tid=0 2024-06-10T00:43:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.611+0000 7f4d82858b80 1 client.17136 shutdown 2024-06-10T00:43:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.612+0000 7f4d82858b80 1 -- 172.21.15.62:0/3535876073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c767a0 msgr2=0x55ac12c88900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.612+0000 7f4d82858b80 1 --2- 172.21.15.62:0/3535876073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c767a0 0x55ac12c88900 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c006010 tx=0x7f4d6c010a60 comp rx=0 tx=0).stop 2024-06-10T00:43:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.612+0000 7f4d82858b80 1 -- 172.21.15.62:0/3535876073 shutdown_connections 2024-06-10T00:43:29.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.612+0000 7f4d82858b80 1 --2- 172.21.15.62:0/3535876073 >> [v2:172.21.15.138:6836/3880084398,v1:172.21.15.138:6837/3880084398] conn(0x55ac12cfaf10 0x55ac12cfd300 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.612+0000 7f4d82858b80 1 --2- 172.21.15.62:0/3535876073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac12c767a0 0x55ac12c88900 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.612+0000 7f4d82858b80 1 --2- 172.21.15.62:0/3535876073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac12c7c880 0x55ac12c97500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.612+0000 7f4d597fa700 0 client.17136 ms_handle_reset on v2:172.21.15.138:6836/3880084398 2024-06-10T00:43:29.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.612+0000 7f4d82858b80 1 --2- 172.21.15.62:0/3535876073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12c7bf60 0x55ac12c88e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.612+0000 7f4d82858b80 1 -- 172.21.15.62:0/3535876073 >> 172.21.15.62:0/3535876073 conn(0x55ac12ba3080 msgr2=0x55ac12c75ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:29.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.612+0000 7f4d82858b80 1 -- 172.21.15.62:0/3535876073 shutdown_connections 2024-06-10T00:43:29.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.612+0000 7f4d82858b80 1 -- 172.21.15.62:0/3535876073 wait complete. 2024-06-10T00:43:29.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.612+0000 7f4d82858b80 20 client.17136 trim_cache size 0 max 16384 2024-06-10T00:43:29.621 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:43:29.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.613+0000 7f4d77fd5700 1 -- 172.21.15.62:0/2232148898 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4d5c0a2d90 msgr2=0x7f4d5c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:29.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.613+0000 7f4d77fd5700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4d5c0a2d90 0x7f4d5c0a5250 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c005950 tx=0x7f4d6c00b410 comp rx=0 tx=0).stop 2024-06-10T00:43:29.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.613+0000 7f4d77fd5700 1 -- 172.21.15.62:0/2232148898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70103120 msgr2=0x7f4d70074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:29.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.613+0000 7f4d77fd5700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70103120 0x7f4d70074cd0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4d60006010 tx=0x7f4d60009f80 comp rx=0 tx=0).stop 2024-06-10T00:43:29.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.613+0000 7f4d77fd5700 1 -- 172.21.15.62:0/2232148898 shutdown_connections 2024-06-10T00:43:29.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.613+0000 7f4d77fd5700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f4d5c0a2d90 0x7f4d5c0a5250 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.613+0000 7f4d77fd5700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d70107860 0x7f4d70075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.613+0000 7f4d77fd5700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d70102770 0x7f4d70074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.613+0000 7f4d77fd5700 1 --2- 172.21.15.62:0/2232148898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d70103120 0x7f4d70074cd0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:29.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.613+0000 7f4d77fd5700 1 -- 172.21.15.62:0/2232148898 >> 172.21.15.62:0/2232148898 conn(0x7f4d70079600 msgr2=0x7f4d70100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:29.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.613+0000 7f4d77fd5700 1 -- 172.21.15.62:0/2232148898 shutdown_connections 2024-06-10T00:43:29.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:29.613+0000 7f4d77fd5700 1 -- 172.21.15.62:0/2232148898 wait complete. 2024-06-10T00:43:29.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:29 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-06-10T00:43:29.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:29 smithi138 ceph-mon[36549]: Unable to set osd_memory_target on smithi105 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-06-10T00:43:29.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:29 smithi138 ceph-mon[36549]: pgmap v330: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:43:30.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:29 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-06-10T00:43:30.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:29 smithi105 ceph-mon[36321]: Unable to set osd_memory_target on smithi105 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-06-10T00:43:30.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:29 smithi105 ceph-mon[36321]: pgmap v330: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:43:30.435 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T00:43:30.701 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:43:31.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:31 smithi062 ceph-mon[32390]: pgmap v331: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:31.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:31 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:31.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:31 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:31.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:31 smithi138 ceph-mon[36549]: pgmap v331: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:31.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:31 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:31.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:31 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:32.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:31 smithi105 ceph-mon[36321]: pgmap v331: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:32.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:31 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:32.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:31 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.511+0000 7f86ee24b700 1 -- 172.21.15.62:0/2689490696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86e806b3c0 msgr2=0x7f86e810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.511+0000 7f86ee24b700 1 --2- 172.21.15.62:0/2689490696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86e806b3c0 0x7f86e810f310 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b0a0 tx=0x7f86e000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.511+0000 7f86ee24b700 1 -- 172.21.15.62:0/2689490696 shutdown_connections 2024-06-10T00:43:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.511+0000 7f86ee24b700 1 --2- 172.21.15.62:0/2689490696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86e806b3c0 0x7f86e810f310 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.511+0000 7f86ee24b700 1 --2- 172.21.15.62:0/2689490696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e81084f0 0x7f86e81088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.511+0000 7f86ee24b700 1 --2- 172.21.15.62:0/2689490696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e806aa00 0x7f86e806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.511+0000 7f86ee24b700 1 -- 172.21.15.62:0/2689490696 >> 172.21.15.62:0/2689490696 conn(0x7f86e8078300 msgr2=0x7f86e8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.511+0000 7f86ee24b700 1 -- 172.21.15.62:0/2689490696 shutdown_connections 2024-06-10T00:43:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.511+0000 7f86ee24b700 1 -- 172.21.15.62:0/2689490696 wait complete. 2024-06-10T00:43:32.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.512+0000 7f86ee24b700 1 Processor -- start 2024-06-10T00:43:32.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.512+0000 7f86ee24b700 1 -- start start 2024-06-10T00:43:32.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86ee24b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e806aa00 0x7f86e819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86ee24b700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86e806b3c0 0x7f86e819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86ee24b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e81084f0 0x7f86e8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86ee24b700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f86e8112770 con 0x7f86e806b3c0 2024-06-10T00:43:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86ee24b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86e81125f0 con 0x7f86e81084f0 2024-06-10T00:43:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86ee24b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86e81128f0 con 0x7f86e806aa00 2024-06-10T00:43:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86e806b3c0 0x7f86e819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86e806b3c0 0x7f86e819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37192/0 (socket says 172.21.15.62:37192) 2024-06-10T00:43:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86e6ffd700 1 -- 172.21.15.62:0/1245466681 learned_addr learned my addr 172.21.15.62:0/1245466681 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86e77fe700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e806aa00 0x7f86e819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86e7fff700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e81084f0 0x7f86e8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86e6ffd700 1 -- 172.21.15.62:0/1245466681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e806aa00 msgr2=0x7f86e819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86e6ffd700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e806aa00 0x7f86e819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86e6ffd700 1 -- 172.21.15.62:0/1245466681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e81084f0 msgr2=0x7f86e8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86e6ffd700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e81084f0 0x7f86e8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.513+0000 7f86e6ffd700 1 -- 172.21.15.62:0/1245466681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e8199190 con 0x7f86e806b3c0 2024-06-10T00:43:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.514+0000 7f86e7fff700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e81084f0 0x7f86e8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:43:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.514+0000 7f86e6ffd700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86e806b3c0 0x7f86e819ed60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f86d000d980 tx=0x7f86d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:32.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.514+0000 7f86e4ff9700 1 -- 172.21.15.62:0/1245466681 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0017070 con 0x7f86e806b3c0 2024-06-10T00:43:32.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.514+0000 7f86ee24b700 1 -- 172.21.15.62:0/1245466681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e8199480 con 0x7f86e806b3c0 2024-06-10T00:43:32.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.514+0000 7f86e4ff9700 1 -- 172.21.15.62:0/1245466681 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d0005020 con 0x7f86e806b3c0 2024-06-10T00:43:32.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.514+0000 7f86ee24b700 1 -- 172.21.15.62:0/1245466681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e8101380 con 0x7f86e806b3c0 2024-06-10T00:43:32.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.514+0000 7f86e4ff9700 1 -- 172.21.15.62:0/1245466681 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d000fbf0 con 0x7f86e806b3c0 2024-06-10T00:43:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.517+0000 7f86e4ff9700 1 -- 172.21.15.62:0/1245466681 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f86d0010460 con 0x7f86e806b3c0 2024-06-10T00:43:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.518+0000 7f86e4ff9700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f86d40a2d90 0x7f86d40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.518+0000 7f86e77fe700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f86d40a2d90 0x7f86d40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.519+0000 7f86e4ff9700 1 -- 172.21.15.62:0/1245466681 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f86d00cf660 con 0x7f86e806b3c0 2024-06-10T00:43:32.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.519+0000 7f86e77fe700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f86d40a2d90 0x7f86d40a5250 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86e8107ca0 tx=0x7f86d80073d0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:32.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.519+0000 7f86f8aceb80 1 Processor -- start 2024-06-10T00:43:32.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.519+0000 7f86f8aceb80 1 -- start start 2024-06-10T00:43:32.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.519+0000 7f86f8aceb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df1f60 0x565515df2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.519+0000 7f86f8aceb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df2880 0x565515dec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.519+0000 7f86f8aceb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515dec7a0 0x565515decbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.519+0000 7f86f8aceb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565515cdef60 con 0x565515df2880 2024-06-10T00:43:32.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.519+0000 7f86f8aceb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565515d3d850 con 0x565515dec7a0 2024-06-10T00:43:32.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.519+0000 7f86f8aceb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565515ca5460 con 0x565515df1f60 2024-06-10T00:43:32.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df2880 0x565515dec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df2880 0x565515dec260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37200/0 (socket says 172.21.15.62:37200) 2024-06-10T00:43:32.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86e6ffd700 1 -- 172.21.15.62:0/1087749855 learned_addr learned my addr 172.21.15.62:0/1087749855 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:32.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86e7fff700 1 --2- 172.21.15.62:0/1087749855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515dec7a0 0x565515decbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86e77fe700 1 --2- 172.21.15.62:0/1087749855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df1f60 0x565515df2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86e6ffd700 1 -- 172.21.15.62:0/1087749855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df1f60 msgr2=0x565515df2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:32.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86e6ffd700 1 --2- 172.21.15.62:0/1087749855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df1f60 0x565515df2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86e6ffd700 1 -- 172.21.15.62:0/1087749855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515dec7a0 msgr2=0x565515decbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:32.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86e6ffd700 1 --2- 172.21.15.62:0/1087749855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515dec7a0 0x565515decbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86e6ffd700 1 -- 172.21.15.62:0/1087749855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565515c8c8d0 con 0x565515df2880 2024-06-10T00:43:32.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86e7fff700 1 --2- 172.21.15.62:0/1087749855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515dec7a0 0x565515decbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:43:32.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86e6ffd700 1 --2- 172.21.15.62:0/1087749855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df2880 0x565515dec260 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f86d0098140 tx=0x7f86d009bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766ee30efa401110 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:32.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86ddffb700 1 -- 172.21.15.62:0/1087749855 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d00a43e0 con 0x565515df2880 2024-06-10T00:43:32.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86ddffb700 1 -- 172.21.15.62:0/1087749855 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d0099a70 con 0x565515df2880 2024-06-10T00:43:32.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.520+0000 7f86ddffb700 1 -- 172.21.15.62:0/1087749855 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0022300 con 0x565515df2880 2024-06-10T00:43:32.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.521+0000 7f86f8aceb80 1 -- 172.21.15.62:0/1087749855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df2880 msgr2=0x565515dec260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:32.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.521+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/1087749855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df2880 0x565515dec260 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f86d0098140 tx=0x7f86d009bfc0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.521+0000 7f86f8aceb80 1 -- 172.21.15.62:0/1087749855 shutdown_connections 2024-06-10T00:43:32.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.521+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/1087749855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515dec7a0 0x565515decbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.521+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/1087749855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df1f60 0x565515df2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.521+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/1087749855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df2880 0x565515dec260 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.521+0000 7f86f8aceb80 1 -- 172.21.15.62:0/1087749855 >> 172.21.15.62:0/1087749855 conn(0x565515d19080 msgr2=0x565515de7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:32.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.521+0000 7f86f8aceb80 1 -- 172.21.15.62:0/1087749855 shutdown_connections 2024-06-10T00:43:32.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.521+0000 7f86f8aceb80 1 -- 172.21.15.62:0/1087749855 wait complete. 2024-06-10T00:43:32.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.522+0000 7f86f8aceb80 1 -- start start 2024-06-10T00:43:32.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.522+0000 7f86f8aceb80 1 Processor -- start 2024-06-10T00:43:32.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.522+0000 7f86f8aceb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515dec7a0 0x565515df8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.522+0000 7f86f8aceb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df1f60 0x565515df5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.522+0000 7f86f8aceb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515df2880 0x565515df5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.522+0000 7f86f8aceb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565515cddfc0 con 0x565515df1f60 2024-06-10T00:43:32.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.522+0000 7f86f8aceb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565515d39810 con 0x565515df2880 2024-06-10T00:43:32.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.522+0000 7f86f8aceb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565515df9260 con 0x565515dec7a0 2024-06-10T00:43:32.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.522+0000 7f86f8aceb80 10 client.0 _ll_register_callbacks cb 0x565515cd35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:43:32.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.522+0000 7f86e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df1f60 0x565515df5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df1f60 0x565515df5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37208/0 (socket says 172.21.15.62:37208) 2024-06-10T00:43:32.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86e6ffd700 1 -- 172.21.15.62:0/541872584 learned_addr learned my addr 172.21.15.62:0/541872584 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:32.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86e77fe700 1 --2- 172.21.15.62:0/541872584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515dec7a0 0x565515df8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86e7fff700 1 --2- 172.21.15.62:0/541872584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515df2880 0x565515df5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86e6ffd700 1 -- 172.21.15.62:0/541872584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515dec7a0 msgr2=0x565515df8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:32.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86e6ffd700 1 --2- 172.21.15.62:0/541872584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515dec7a0 0x565515df8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86e6ffd700 1 -- 172.21.15.62:0/541872584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515df2880 msgr2=0x565515df5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:32.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86e6ffd700 1 --2- 172.21.15.62:0/541872584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515df2880 0x565515df5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86e6ffd700 1 -- 172.21.15.62:0/541872584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565515d03750 con 0x565515df1f60 2024-06-10T00:43:32.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86e6ffd700 1 --2- 172.21.15.62:0/541872584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df1f60 0x565515df5930 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f86d00ccfa0 tx=0x7f86d000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55e5f7f182adf1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:32.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:43:32.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86c6ffd700 1 -- 172.21.15.62:0/541872584 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0067060 con 0x565515df1f60 2024-06-10T00:43:32.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86c6ffd700 1 -- 172.21.15.62:0/541872584 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f86d0097de0 con 0x565515df1f60 2024-06-10T00:43:32.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86f8aceb80 1 -- 172.21.15.62:0/541872584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86d0099a70 con 0x565515df1f60 2024-06-10T00:43:32.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86c6ffd700 1 -- 172.21.15.62:0/541872584 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f86d0081050 con 0x565515df1f60 2024-06-10T00:43:32.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86f8aceb80 10 client.17148 fetch_fsmap learned FSMap version 38 2024-06-10T00:43:32.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.523+0000 7f86f8aceb80 1 -- 172.21.15.62:0/541872584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x565515e03a60 con 0x565515df1f60 2024-06-10T00:43:32.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.524+0000 7f86c6ffd700 1 -- 172.21.15.62:0/541872584 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d0099a70 con 0x565515df1f60 2024-06-10T00:43:32.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.524+0000 7f86c6ffd700 1 -- 172.21.15.62:0/541872584 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f86d009b9d0 con 0x565515df1f60 2024-06-10T00:43:32.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.524+0000 7f86f8aceb80 10 client.17148 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:43:32.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.524+0000 7f86f8aceb80 10 client.17148 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:43:32.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.524+0000 7f86f8aceb80 20 client.17148 populate_metadata read hostname 'smithi062' 2024-06-10T00:43:32.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.524+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/541872584 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x565515e05f30 0x565515e08320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.524+0000 7f86f8aceb80 4 client.17148 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:43:32.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.524+0000 7f86f8aceb80 1 -- 172.21.15.62:0/541872584 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565515d4ad50 con 0x565515e05f30 2024-06-10T00:43:32.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.524+0000 7f86e77fe700 1 --2- 172.21.15.62:0/541872584 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x565515e05f30 0x565515e08320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.525+0000 7f86e77fe700 1 --2- 172.21.15.62:0/541872584 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x565515e05f30 0x565515e08320 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=125b26d9f33defb server_cookie=2cfce2efe93b1ad2 in_seq=0 out_seq=0 2024-06-10T00:43:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.525+0000 7f86c6ffd700 10 client.17148 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:43:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.525+0000 7f86c6ffd700 1 -- 172.21.15.62:0/541872584 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565515d4ad50 con 0x565515e05f30 2024-06-10T00:43:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.525+0000 7f86c6ffd700 10 client.17148 handle_command_reply: tid=0 2024-06-10T00:43:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.525+0000 7f86f8aceb80 1 client.17148 shutdown 2024-06-10T00:43:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.526+0000 7f86f8aceb80 1 -- 172.21.15.62:0/541872584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df1f60 msgr2=0x565515df5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.526+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/541872584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df1f60 0x565515df5930 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f86d00ccfa0 tx=0x7f86d000f690 comp rx=0 tx=0).stop 2024-06-10T00:43:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.526+0000 7f86f8aceb80 1 -- 172.21.15.62:0/541872584 shutdown_connections 2024-06-10T00:43:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.526+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/541872584 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x565515e05f30 0x565515e08320 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.526+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/541872584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515df2880 0x565515df5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.526+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/541872584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515dec7a0 0x565515df8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.526+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/541872584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df1f60 0x565515df5930 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.526+0000 7f86f8aceb80 1 -- 172.21.15.62:0/541872584 >> 172.21.15.62:0/541872584 conn(0x565515d19080 msgr2=0x565515de6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.526+0000 7f86c6ffd700 0 client.17148 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:43:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.526+0000 7f86f8aceb80 1 -- 172.21.15.62:0/541872584 shutdown_connections 2024-06-10T00:43:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.526+0000 7f86f8aceb80 1 -- 172.21.15.62:0/541872584 wait complete. 2024-06-10T00:43:32.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.526+0000 7f86f8aceb80 20 client.17148 trim_cache size 0 max 16384 2024-06-10T00:43:32.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86f8aceb80 1 Processor -- start 2024-06-10T00:43:32.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86f8aceb80 1 -- start start 2024-06-10T00:43:32.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86f8aceb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515dec7a0 0x565515df3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86f8aceb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515df1f60 0x565515de6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86f8aceb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df2880 0x565515de73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86f8aceb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f86e81b08e0 con 0x565515dec7a0 2024-06-10T00:43:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86f8aceb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86e81b2420 con 0x565515df1f60 2024-06-10T00:43:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86f8aceb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86e81b0fb0 con 0x565515df2880 2024-06-10T00:43:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86e77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515dec7a0 0x565515df3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86e77fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515dec7a0 0x565515df3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37214/0 (socket says 172.21.15.62:37214) 2024-06-10T00:43:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86e77fe700 1 -- 172.21.15.62:0/2166321162 learned_addr learned my addr 172.21.15.62:0/2166321162 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86e6ffd700 1 --2- 172.21.15.62:0/2166321162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515df1f60 0x565515de6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86e7fff700 1 --2- 172.21.15.62:0/2166321162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df2880 0x565515de73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86e77fe700 1 -- 172.21.15.62:0/2166321162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df2880 msgr2=0x565515de73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86e77fe700 1 --2- 172.21.15.62:0/2166321162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df2880 0x565515de73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86e77fe700 1 -- 172.21.15.62:0/2166321162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515df1f60 msgr2=0x565515de6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.542+0000 7f86e77fe700 1 --2- 172.21.15.62:0/2166321162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515df1f60 0x565515de6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.543+0000 7f86e77fe700 1 -- 172.21.15.62:0/2166321162 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565515de3810 con 0x565515dec7a0 2024-06-10T00:43:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.543+0000 7f86e7fff700 1 --2- 172.21.15.62:0/2166321162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df2880 0x565515de73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:43:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.543+0000 7f86e6ffd700 1 --2- 172.21.15.62:0/2166321162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515df1f60 0x565515de6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:43:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.543+0000 7f86e77fe700 1 --2- 172.21.15.62:0/2166321162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515dec7a0 0x565515df3390 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f86d8005fd0 tx=0x7f86d8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63960ab09114121 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:32.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.543+0000 7f86dd7fa700 1 -- 172.21.15.62:0/2166321162 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d802e070 con 0x565515dec7a0 2024-06-10T00:43:32.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.543+0000 7f86dd7fa700 1 -- 172.21.15.62:0/2166321162 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d8004550 con 0x565515dec7a0 2024-06-10T00:43:32.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.543+0000 7f86dd7fa700 1 -- 172.21.15.62:0/2166321162 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8011440 con 0x565515dec7a0 2024-06-10T00:43:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.544+0000 7f86f8aceb80 1 -- 172.21.15.62:0/2166321162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515dec7a0 msgr2=0x565515df3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.544+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/2166321162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515dec7a0 0x565515df3390 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f86d8005fd0 tx=0x7f86d8013000 comp rx=0 tx=0).stop 2024-06-10T00:43:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.544+0000 7f86f8aceb80 1 -- 172.21.15.62:0/2166321162 shutdown_connections 2024-06-10T00:43:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.544+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/2166321162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515dec7a0 0x565515df3390 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.544+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/2166321162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df2880 0x565515de73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.544+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/2166321162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515df1f60 0x565515de6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.544+0000 7f86f8aceb80 1 -- 172.21.15.62:0/2166321162 >> 172.21.15.62:0/2166321162 conn(0x565515d19080 msgr2=0x565515df79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:32.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.544+0000 7f86f8aceb80 1 -- 172.21.15.62:0/2166321162 shutdown_connections 2024-06-10T00:43:32.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.544+0000 7f86f8aceb80 1 -- 172.21.15.62:0/2166321162 wait complete. 2024-06-10T00:43:32.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.544+0000 7f86f8aceb80 1 -- start start 2024-06-10T00:43:32.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.544+0000 7f86f8aceb80 1 Processor -- start 2024-06-10T00:43:32.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86f8aceb80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515dec7a0 0x565515e07f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86f8aceb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df1f60 0x565515e084c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86f8aceb80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df2880 0x565515e08a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86f8aceb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f86e81b1d50 con 0x565515df1f60 2024-06-10T00:43:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86f8aceb80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86e81b1680 con 0x565515dec7a0 2024-06-10T00:43:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86f8aceb80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86e81ae8a0 con 0x565515df2880 2024-06-10T00:43:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86f8aceb80 10 client.0 _ll_register_callbacks cb 0x565515cd35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:43:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df1f60 0x565515e084c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86e6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df1f60 0x565515e084c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37230/0 (socket says 172.21.15.62:37230) 2024-06-10T00:43:32.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86e6ffd700 1 -- 172.21.15.62:0/599728259 learned_addr learned my addr 172.21.15.62:0/599728259 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86e77fe700 1 --2- 172.21.15.62:0/599728259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515dec7a0 0x565515e07f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86e7fff700 1 --2- 172.21.15.62:0/599728259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df2880 0x565515e08a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86e6ffd700 1 -- 172.21.15.62:0/599728259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df2880 msgr2=0x565515e08a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86e6ffd700 1 --2- 172.21.15.62:0/599728259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df2880 0x565515e08a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86e6ffd700 1 -- 172.21.15.62:0/599728259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515dec7a0 msgr2=0x565515e07f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86e6ffd700 1 --2- 172.21.15.62:0/599728259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515dec7a0 0x565515e07f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.545+0000 7f86e6ffd700 1 -- 172.21.15.62:0/599728259 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565515e6eac0 con 0x565515df1f60 2024-06-10T00:43:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86e6ffd700 1 --2- 172.21.15.62:0/599728259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df1f60 0x565515e084c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f86d009b700 tx=0x7f86d0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9e9e060ebad9b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:43:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86c6ffd700 1 -- 172.21.15.62:0/599728259 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0067040 con 0x565515df1f60 2024-06-10T00:43:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86c6ffd700 1 -- 172.21.15.62:0/599728259 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f86d00051d0 con 0x565515df1f60 2024-06-10T00:43:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86f8aceb80 1 -- 172.21.15.62:0/599728259 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565515d452d0 con 0x565515df1f60 2024-06-10T00:43:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86c6ffd700 1 -- 172.21.15.62:0/599728259 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f86d0097a10 con 0x565515df1f60 2024-06-10T00:43:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86f8aceb80 10 client.17154 fetch_fsmap learned FSMap version 38 2024-06-10T00:43:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86f8aceb80 1 -- 172.21.15.62:0/599728259 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x565515e6fd10 con 0x565515df1f60 2024-06-10T00:43:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86c6ffd700 1 -- 172.21.15.62:0/599728259 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x565515d452d0 con 0x565515df1f60 2024-06-10T00:43:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86c6ffd700 1 -- 172.21.15.62:0/599728259 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f86d0097de0 con 0x565515df1f60 2024-06-10T00:43:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86f8aceb80 10 client.17154 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:43:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86f8aceb80 10 client.17154 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-10T00:43:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86f8aceb80 20 client.17154 populate_metadata read hostname 'smithi062' 2024-06-10T00:43:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/599728259 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x565515e70200 0x565515e72620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86f8aceb80 4 client.17154 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:43:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.546+0000 7f86f8aceb80 1 -- 172.21.15.62:0/599728259 --> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565515cf8f50 con 0x565515e70200 2024-06-10T00:43:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.547+0000 7f86e77fe700 1 --2- 172.21.15.62:0/599728259 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x565515e70200 0x565515e72620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.547+0000 7f86c6ffd700 10 client.17154 ms_handle_connect on v2:172.21.15.105:6836/4270590303 2024-06-10T00:43:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.547+0000 7f86e77fe700 1 --2- 172.21.15.62:0/599728259 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x565515e70200 0x565515e72620 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=815dc67261aa0331 server_cookie=c71b82c421e17561 in_seq=0 out_seq=0 2024-06-10T00:43:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.547+0000 7f86c6ffd700 1 -- 172.21.15.62:0/599728259 <== mds.2 v2:172.21.15.105:6836/4270590303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565515cf8f50 con 0x565515e70200 2024-06-10T00:43:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.547+0000 7f86c6ffd700 10 client.17154 handle_command_reply: tid=0 2024-06-10T00:43:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.547+0000 7f86f8aceb80 1 client.17154 shutdown 2024-06-10T00:43:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86f8aceb80 1 -- 172.21.15.62:0/599728259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df1f60 msgr2=0x565515e084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/599728259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df1f60 0x565515e084c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f86d009b700 tx=0x7f86d0005b00 comp rx=0 tx=0).stop 2024-06-10T00:43:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86f8aceb80 1 -- 172.21.15.62:0/599728259 shutdown_connections 2024-06-10T00:43:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/599728259 >> [v2:172.21.15.105:6836/4270590303,v1:172.21.15.105:6837/4270590303] conn(0x565515e70200 0x565515e72620 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/599728259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565515df2880 0x565515e08a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/599728259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565515dec7a0 0x565515e07f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86f8aceb80 1 --2- 172.21.15.62:0/599728259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565515df1f60 0x565515e084c0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86c6ffd700 0 client.17154 ms_handle_reset on v2:172.21.15.105:6836/4270590303 2024-06-10T00:43:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86f8aceb80 1 -- 172.21.15.62:0/599728259 >> 172.21.15.62:0/599728259 conn(0x565515d19080 msgr2=0x565515df79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86f8aceb80 1 -- 172.21.15.62:0/599728259 shutdown_connections 2024-06-10T00:43:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86f8aceb80 1 -- 172.21.15.62:0/599728259 wait complete. 2024-06-10T00:43:32.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86f8aceb80 20 client.17154 trim_cache size 0 max 16384 2024-06-10T00:43:32.558 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:43:32.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86ee24b700 1 -- 172.21.15.62:0/1245466681 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f86d40a2d90 msgr2=0x7f86d40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86ee24b700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f86d40a2d90 0x7f86d40a5250 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86e8107ca0 tx=0x7f86d80073d0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86ee24b700 1 -- 172.21.15.62:0/1245466681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86e806b3c0 msgr2=0x7f86e819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.548+0000 7f86ee24b700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86e806b3c0 0x7f86e819ed60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f86d000d980 tx=0x7f86d000dc90 comp rx=0 tx=0).stop 2024-06-10T00:43:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.549+0000 7f86ee24b700 1 -- 172.21.15.62:0/1245466681 shutdown_connections 2024-06-10T00:43:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.549+0000 7f86ee24b700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f86d40a2d90 0x7f86d40a5250 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.549+0000 7f86ee24b700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e81084f0 0x7f86e8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.549+0000 7f86ee24b700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e806aa00 0x7f86e819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.549+0000 7f86ee24b700 1 --2- 172.21.15.62:0/1245466681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86e806b3c0 0x7f86e819ed60 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.549+0000 7f86ee24b700 1 -- 172.21.15.62:0/1245466681 >> 172.21.15.62:0/1245466681 conn(0x7f86e8078300 msgr2=0x7f86e8100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.549+0000 7f86ee24b700 1 -- 172.21.15.62:0/1245466681 shutdown_connections 2024-06-10T00:43:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:32.549+0000 7f86ee24b700 1 -- 172.21.15.62:0/1245466681 wait complete. 2024-06-10T00:43:32.564 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:32 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:32.567 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:32 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:32.567 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:32 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:32.567 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:32 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:32.567 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:32 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:43:32.567 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:32 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:43:32.568 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:32 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:32.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:32 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:32.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:32 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:32.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:32 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:32.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:32 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:32.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:32 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:43:32.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:32 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:43:32.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:32 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:33.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:32 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:33.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:32 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:33.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:32 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:33.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:32 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:43:33.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:32 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:43:33.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:32 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:43:33.033 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:32 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:43:33.427 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T00:43:33.681 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:43:33.803 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:33 smithi062 ceph-mon[32390]: Adjusting osd_memory_target on smithi062 to 176.1M 2024-06-10T00:43:33.803 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:33 smithi062 ceph-mon[32390]: Unable to set osd_memory_target on smithi062 to 184742297: error parsing value: Value '184742297' is below minimum 939524096 2024-06-10T00:43:33.803 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:33 smithi062 ceph-mon[32390]: pgmap v332: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:43:33.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:33 smithi138 ceph-mon[36549]: Adjusting osd_memory_target on smithi062 to 176.1M 2024-06-10T00:43:33.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:33 smithi138 ceph-mon[36549]: Unable to set osd_memory_target on smithi062 to 184742297: error parsing value: Value '184742297' is below minimum 939524096 2024-06-10T00:43:33.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:33 smithi138 ceph-mon[36549]: pgmap v332: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:43:34.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:33 smithi105 ceph-mon[36321]: Adjusting osd_memory_target on smithi062 to 176.1M 2024-06-10T00:43:34.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:33 smithi105 ceph-mon[36321]: Unable to set osd_memory_target on smithi062 to 184742297: error parsing value: Value '184742297' is below minimum 939524096 2024-06-10T00:43:34.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:33 smithi105 ceph-mon[36321]: pgmap v332: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:43:35.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.507+0000 7f268f230700 1 -- 172.21.15.62:0/3376156722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26881030f0 msgr2=0x7f2688107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:35.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.507+0000 7f268f230700 1 --2- 172.21.15.62:0/3376156722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26881030f0 0x7f2688107110 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f26740099c0 tx=0x7f2674009cd0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.508+0000 7f268f230700 1 -- 172.21.15.62:0/3376156722 shutdown_connections 2024-06-10T00:43:35.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.508+0000 7f268f230700 1 --2- 172.21.15.62:0/3376156722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2688107840 0x7f268810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.508+0000 7f268f230700 1 --2- 172.21.15.62:0/3376156722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2688102740 0x7f2688102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.508+0000 7f268f230700 1 --2- 172.21.15.62:0/3376156722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26881030f0 0x7f2688107110 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.508+0000 7f268f230700 1 -- 172.21.15.62:0/3376156722 >> 172.21.15.62:0/3376156722 conn(0x7f26880795f0 msgr2=0x7f2688079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:35.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.508+0000 7f268f230700 1 -- 172.21.15.62:0/3376156722 shutdown_connections 2024-06-10T00:43:35.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.508+0000 7f268f230700 1 -- 172.21.15.62:0/3376156722 wait complete. 2024-06-10T00:43:35.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.508+0000 7f268f230700 1 Processor -- start 2024-06-10T00:43:35.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.508+0000 7f268f230700 1 -- start start 2024-06-10T00:43:35.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268f230700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688102740 0x7f268819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268f230700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26881030f0 0x7f268819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268f230700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2688107840 0x7f2688198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268f230700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f26881127a0 con 0x7f2688102740 2024-06-10T00:43:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268f230700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2688112620 con 0x7f26881030f0 2024-06-10T00:43:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268f230700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2688112920 con 0x7f2688107840 2024-06-10T00:43:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268cfcc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688102740 0x7f268819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268cfcc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688102740 0x7f268819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37246/0 (socket says 172.21.15.62:37246) 2024-06-10T00:43:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268cfcc700 1 -- 172.21.15.62:0/373888272 learned_addr learned my addr 172.21.15.62:0/373888272 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:35.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f2687fff700 1 --2- 172.21.15.62:0/373888272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26881030f0 0x7f268819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268d7cd700 1 --2- 172.21.15.62:0/373888272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2688107840 0x7f2688198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268cfcc700 1 -- 172.21.15.62:0/373888272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2688107840 msgr2=0x7f2688198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:35.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268cfcc700 1 --2- 172.21.15.62:0/373888272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2688107840 0x7f2688198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268cfcc700 1 -- 172.21.15.62:0/373888272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26881030f0 msgr2=0x7f268819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:35.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268cfcc700 1 --2- 172.21.15.62:0/373888272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26881030f0 0x7f268819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268cfcc700 1 -- 172.21.15.62:0/373888272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2688199160 con 0x7f2688102740 2024-06-10T00:43:35.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f268cfcc700 1 --2- 172.21.15.62:0/373888272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688102740 0x7f268819e840 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f267c00d950 tx=0x7f267c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:35.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.509+0000 7f2685ffb700 1 -- 172.21.15.62:0/373888272 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c016070 con 0x7f2688102740 2024-06-10T00:43:35.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.510+0000 7f2685ffb700 1 -- 172.21.15.62:0/373888272 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f267c009b40 con 0x7f2688102740 2024-06-10T00:43:35.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.510+0000 7f268f230700 1 -- 172.21.15.62:0/373888272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2688199450 con 0x7f2688102740 2024-06-10T00:43:35.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.510+0000 7f2685ffb700 1 -- 172.21.15.62:0/373888272 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c00f620 con 0x7f2688102740 2024-06-10T00:43:35.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.510+0000 7f268f230700 1 -- 172.21.15.62:0/373888272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26881a5710 con 0x7f2688102740 2024-06-10T00:43:35.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.512+0000 7f2685ffb700 1 -- 172.21.15.62:0/373888272 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f267c019070 con 0x7f2688102740 2024-06-10T00:43:35.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.512+0000 7f2685ffb700 1 --2- 172.21.15.62:0/373888272 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f26780a2d90 0x7f26780a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.513+0000 7f2685ffb700 1 -- 172.21.15.62:0/373888272 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f267c0cf8a0 con 0x7f2688102740 2024-06-10T00:43:35.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.513+0000 7f2687fff700 1 --2- 172.21.15.62:0/373888272 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f26780a2d90 0x7f26780a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.513+0000 7f2687fff700 1 --2- 172.21.15.62:0/373888272 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f26780a2d90 0x7f26780a5250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f26881070c0 tx=0x7f2674005fb0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:35.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.514+0000 7f2699ab3b80 1 Processor -- start 2024-06-10T00:43:35.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.514+0000 7f2699ab3b80 1 -- start start 2024-06-10T00:43:35.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.514+0000 7f2699ab3b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eacfd0 0x5609e8ead3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.514+0000 7f2699ab3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8eb16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.514+0000 7f2699ab3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eb1c30 0x5609e8eb2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.514+0000 7f2699ab3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5609e8dfa810 con 0x5609e8eb1c30 2024-06-10T00:43:35.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.514+0000 7f2699ab3b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5609e8da2720 con 0x5609e8eacfd0 2024-06-10T00:43:35.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.514+0000 7f2699ab3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5609e8df67d0 con 0x5609e8ead8f0 2024-06-10T00:43:35.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.514+0000 7f268d7cd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eb1c30 0x5609e8eb2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.514+0000 7f268d7cd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eb1c30 0x5609e8eb2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37250/0 (socket says 172.21.15.62:37250) 2024-06-10T00:43:35.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.514+0000 7f268d7cd700 1 -- 172.21.15.62:0/724888076 learned_addr learned my addr 172.21.15.62:0/724888076 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:35.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.514+0000 7f2687fff700 1 --2- 172.21.15.62:0/724888076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8eb16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.514+0000 7f268cfcc700 1 --2- 172.21.15.62:0/724888076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eacfd0 0x5609e8ead3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.515+0000 7f268d7cd700 1 -- 172.21.15.62:0/724888076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 msgr2=0x5609e8eb16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:35.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.515+0000 7f268d7cd700 1 --2- 172.21.15.62:0/724888076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8eb16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.515+0000 7f268d7cd700 1 -- 172.21.15.62:0/724888076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eacfd0 msgr2=0x5609e8ead3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:35.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.515+0000 7f268d7cd700 1 --2- 172.21.15.62:0/724888076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eacfd0 0x5609e8ead3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.515+0000 7f268d7cd700 1 -- 172.21.15.62:0/724888076 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609e8d98f50 con 0x5609e8eb1c30 2024-06-10T00:43:35.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.515+0000 7f268cfcc700 1 --2- 172.21.15.62:0/724888076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eacfd0 0x5609e8ead3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:43:35.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.515+0000 7f268d7cd700 1 --2- 172.21.15.62:0/724888076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eb1c30 0x5609e8eb2090 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2680007f70 tx=0x7f268000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b49539be8fd69ec server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:35.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.515+0000 7f2672ffd700 1 -- 172.21.15.62:0/724888076 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268000ecc0 con 0x5609e8eb1c30 2024-06-10T00:43:35.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.515+0000 7f2672ffd700 1 -- 172.21.15.62:0/724888076 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f268000ee60 con 0x5609e8eb1c30 2024-06-10T00:43:35.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.515+0000 7f2672ffd700 1 -- 172.21.15.62:0/724888076 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680014e30 con 0x5609e8eb1c30 2024-06-10T00:43:35.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.516+0000 7f2699ab3b80 1 -- 172.21.15.62:0/724888076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eb1c30 msgr2=0x5609e8eb2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:35.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.516+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/724888076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eb1c30 0x5609e8eb2090 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2680007f70 tx=0x7f268000e440 comp rx=0 tx=0).stop 2024-06-10T00:43:35.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.516+0000 7f2699ab3b80 1 -- 172.21.15.62:0/724888076 shutdown_connections 2024-06-10T00:43:35.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.516+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/724888076 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eb1c30 0x5609e8eb2090 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.516+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/724888076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eacfd0 0x5609e8ead3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.516+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/724888076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8eb16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.516+0000 7f2699ab3b80 1 -- 172.21.15.62:0/724888076 >> 172.21.15.62:0/724888076 conn(0x5609e8dd3cc0 msgr2=0x5609e8ea3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:35.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.516+0000 7f2699ab3b80 1 -- 172.21.15.62:0/724888076 shutdown_connections 2024-06-10T00:43:35.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.516+0000 7f2699ab3b80 1 -- 172.21.15.62:0/724888076 wait complete. 2024-06-10T00:43:35.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.516+0000 7f2699ab3b80 1 -- start start 2024-06-10T00:43:35.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.516+0000 7f2699ab3b80 1 Processor -- start 2024-06-10T00:43:35.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f2699ab3b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eacfd0 0x5609e8eb9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f2699ab3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8eb6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f2699ab3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eb1c30 0x5609e8eb6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f2699ab3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5609e8eb9dd0 con 0x5609e8eb1c30 2024-06-10T00:43:35.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f2699ab3b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5609e8d9abf0 con 0x5609e8eacfd0 2024-06-10T00:43:35.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f2699ab3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5609e8eb9f50 con 0x5609e8ead8f0 2024-06-10T00:43:35.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f2699ab3b80 10 client.0 _ll_register_callbacks cb 0x5609e8d94920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:43:35.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f268d7cd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eb1c30 0x5609e8eb6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f268d7cd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eb1c30 0x5609e8eb6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37262/0 (socket says 172.21.15.62:37262) 2024-06-10T00:43:35.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f268d7cd700 1 -- 172.21.15.62:0/4149111241 learned_addr learned my addr 172.21.15.62:0/4149111241 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:35.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f2687fff700 1 --2- 172.21.15.62:0/4149111241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8eb6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f268cfcc700 1 --2- 172.21.15.62:0/4149111241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eacfd0 0x5609e8eb9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f268d7cd700 1 -- 172.21.15.62:0/4149111241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 msgr2=0x5609e8eb6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:35.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f268d7cd700 1 --2- 172.21.15.62:0/4149111241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8eb6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f268d7cd700 1 -- 172.21.15.62:0/4149111241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eacfd0 msgr2=0x5609e8eb9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:35.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f268d7cd700 1 --2- 172.21.15.62:0/4149111241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eacfd0 0x5609e8eb9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f268d7cd700 1 -- 172.21.15.62:0/4149111241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609e8d498d0 con 0x5609e8eb1c30 2024-06-10T00:43:35.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.517+0000 7f268d7cd700 1 --2- 172.21.15.62:0/4149111241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eb1c30 0x5609e8eb6a60 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f26800077a0 tx=0x7f2680015f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ca353ca7b316fa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:35.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2667fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:43:35.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2667fff700 1 -- 172.21.15.62:0/4149111241 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680022070 con 0x5609e8eb1c30 2024-06-10T00:43:35.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2699ab3b80 1 -- 172.21.15.62:0/4149111241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f268000ee60 con 0x5609e8eb1c30 2024-06-10T00:43:35.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2667fff700 1 -- 172.21.15.62:0/4149111241 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f2680019040 con 0x5609e8eb1c30 2024-06-10T00:43:35.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2667fff700 1 -- 172.21.15.62:0/4149111241 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f268002d050 con 0x5609e8eb1c30 2024-06-10T00:43:35.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2699ab3b80 10 client.17166 fetch_fsmap learned FSMap version 38 2024-06-10T00:43:35.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2699ab3b80 1 -- 172.21.15.62:0/4149111241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5609e8ec46f0 con 0x5609e8eb1c30 2024-06-10T00:43:35.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2667fff700 1 -- 172.21.15.62:0/4149111241 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f268000ee60 con 0x5609e8eb1c30 2024-06-10T00:43:35.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2667fff700 1 -- 172.21.15.62:0/4149111241 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f268001e9e0 con 0x5609e8eb1c30 2024-06-10T00:43:35.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2699ab3b80 10 client.17166 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:43:35.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2699ab3b80 10 client.17166 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:43:35.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2699ab3b80 20 client.17166 populate_metadata read hostname 'smithi062' 2024-06-10T00:43:35.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/4149111241 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5609e8ec6bc0 0x5609e8ec8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2699ab3b80 4 client.17166 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:43:35.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.518+0000 7f2699ab3b80 1 -- 172.21.15.62:0/4149111241 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609e8e07d10 con 0x5609e8ec6bc0 2024-06-10T00:43:35.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.519+0000 7f268cfcc700 1 --2- 172.21.15.62:0/4149111241 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5609e8ec6bc0 0x5609e8ec8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.519+0000 7f268cfcc700 1 --2- 172.21.15.62:0/4149111241 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5609e8ec6bc0 0x5609e8ec8fb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b6c86ac1649a18f server_cookie=a876545bbf61eef0 in_seq=0 out_seq=0 2024-06-10T00:43:35.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.519+0000 7f2667fff700 10 client.17166 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:43:35.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2667fff700 1 -- 172.21.15.62:0/4149111241 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5609e8e07d10 con 0x5609e8ec6bc0 2024-06-10T00:43:35.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2667fff700 10 client.17166 handle_command_reply: tid=0 2024-06-10T00:43:35.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2699ab3b80 1 client.17166 shutdown 2024-06-10T00:43:35.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2699ab3b80 1 -- 172.21.15.62:0/4149111241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eb1c30 msgr2=0x5609e8eb6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:35.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/4149111241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eb1c30 0x5609e8eb6a60 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f26800077a0 tx=0x7f2680015f70 comp rx=0 tx=0).stop 2024-06-10T00:43:35.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2699ab3b80 1 -- 172.21.15.62:0/4149111241 shutdown_connections 2024-06-10T00:43:35.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/4149111241 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5609e8ec6bc0 0x5609e8ec8fb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/4149111241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eb1c30 0x5609e8eb6a60 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2667fff700 0 client.17166 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:43:35.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/4149111241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eacfd0 0x5609e8eb9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/4149111241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8eb6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2699ab3b80 1 -- 172.21.15.62:0/4149111241 >> 172.21.15.62:0/4149111241 conn(0x5609e8dd3cc0 msgr2=0x5609e8ea79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:35.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2699ab3b80 1 -- 172.21.15.62:0/4149111241 shutdown_connections 2024-06-10T00:43:35.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2699ab3b80 1 -- 172.21.15.62:0/4149111241 wait complete. 2024-06-10T00:43:35.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.520+0000 7f2699ab3b80 20 client.17166 trim_cache size 0 max 16384 2024-06-10T00:43:35.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.535+0000 7f2699ab3b80 1 Processor -- start 2024-06-10T00:43:35.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.535+0000 7f2699ab3b80 1 -- start start 2024-06-10T00:43:35.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f2699ab3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eacfd0 0x5609e8eb2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f2699ab3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8ecbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f2699ab3b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eb1c30 0x5609e8ecc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f2699ab3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f26881ac530 con 0x5609e8eacfd0 2024-06-10T00:43:35.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f2699ab3b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26881ae070 con 0x5609e8eb1c30 2024-06-10T00:43:35.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f2699ab3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26881acc00 con 0x5609e8ead8f0 2024-06-10T00:43:35.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f268cfcc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eacfd0 0x5609e8eb2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f268cfcc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eacfd0 0x5609e8eb2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37274/0 (socket says 172.21.15.62:37274) 2024-06-10T00:43:35.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f268cfcc700 1 -- 172.21.15.62:0/3794991483 learned_addr learned my addr 172.21.15.62:0/3794991483 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:35.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f268d7cd700 1 --2- 172.21.15.62:0/3794991483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eb1c30 0x5609e8ecc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f268cfcc700 1 -- 172.21.15.62:0/3794991483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 msgr2=0x5609e8ecbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:35.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f2687fff700 1 --2- 172.21.15.62:0/3794991483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8ecbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f268cfcc700 1 --2- 172.21.15.62:0/3794991483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8ecbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f268cfcc700 1 -- 172.21.15.62:0/3794991483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eb1c30 msgr2=0x5609e8ecc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:35.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f268cfcc700 1 --2- 172.21.15.62:0/3794991483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eb1c30 0x5609e8ecc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f268cfcc700 1 -- 172.21.15.62:0/3794991483 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609e8eb6d70 con 0x5609e8eacfd0 2024-06-10T00:43:35.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f268d7cd700 1 --2- 172.21.15.62:0/3794991483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eb1c30 0x5609e8ecc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:43:35.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f268cfcc700 1 --2- 172.21.15.62:0/3794991483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eacfd0 0x5609e8eb2010 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f267c0cd1e0 tx=0x7f267c005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa552ae4d198eed server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:35.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f26727fc700 1 -- 172.21.15.62:0/3794991483 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c0b3070 con 0x5609e8eacfd0 2024-06-10T00:43:35.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.536+0000 7f26727fc700 1 -- 172.21.15.62:0/3794991483 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f267c099cb0 con 0x5609e8eacfd0 2024-06-10T00:43:35.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.537+0000 7f26727fc700 1 -- 172.21.15.62:0/3794991483 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c020040 con 0x5609e8eacfd0 2024-06-10T00:43:35.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.537+0000 7f2699ab3b80 1 -- 172.21.15.62:0/3794991483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eacfd0 msgr2=0x5609e8eb2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:35.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.537+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/3794991483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eacfd0 0x5609e8eb2010 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f267c0cd1e0 tx=0x7f267c005910 comp rx=0 tx=0).stop 2024-06-10T00:43:35.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.537+0000 7f2699ab3b80 1 -- 172.21.15.62:0/3794991483 shutdown_connections 2024-06-10T00:43:35.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.537+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/3794991483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eacfd0 0x5609e8eb2010 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.537+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/3794991483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eb1c30 0x5609e8ecc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.537+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/3794991483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8ecbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.537+0000 7f2699ab3b80 1 -- 172.21.15.62:0/3794991483 >> 172.21.15.62:0/3794991483 conn(0x5609e8dd3cc0 msgr2=0x5609e8ea3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:35.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.537+0000 7f2699ab3b80 1 -- 172.21.15.62:0/3794991483 shutdown_connections 2024-06-10T00:43:35.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.537+0000 7f2699ab3b80 1 -- 172.21.15.62:0/3794991483 wait complete. 2024-06-10T00:43:35.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f2699ab3b80 1 -- start start 2024-06-10T00:43:35.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f2699ab3b80 1 Processor -- start 2024-06-10T00:43:35.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f2699ab3b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eacfd0 0x5609e8f28850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f2699ab3b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8f28d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f2699ab3b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eb1c30 0x5609e8ec95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f2699ab3b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f26881ad9a0 con 0x5609e8eacfd0 2024-06-10T00:43:35.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f2699ab3b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26881ad2d0 con 0x5609e8eb1c30 2024-06-10T00:43:35.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f2699ab3b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26881aa4f0 con 0x5609e8ead8f0 2024-06-10T00:43:35.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f2699ab3b80 10 client.0 _ll_register_callbacks cb 0x5609e8d94920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:43:35.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f268cfcc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eacfd0 0x5609e8f28850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f268cfcc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eacfd0 0x5609e8f28850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37280/0 (socket says 172.21.15.62:37280) 2024-06-10T00:43:35.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f268cfcc700 1 -- 172.21.15.62:0/320639296 learned_addr learned my addr 172.21.15.62:0/320639296 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:35.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f2687fff700 1 --2- 172.21.15.62:0/320639296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8f28d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f268cfcc700 1 -- 172.21.15.62:0/320639296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 msgr2=0x5609e8f28d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:35.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f268cfcc700 1 --2- 172.21.15.62:0/320639296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8f28d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.538+0000 7f268cfcc700 1 -- 172.21.15.62:0/320639296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eb1c30 msgr2=0x5609e8ec95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:35.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f268cfcc700 1 --2- 172.21.15.62:0/320639296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eb1c30 0x5609e8ec95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f268cfcc700 1 -- 172.21.15.62:0/320639296 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609e8ec9e10 con 0x5609e8eacfd0 2024-06-10T00:43:35.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f268cfcc700 1 --2- 172.21.15.62:0/320639296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eacfd0 0x5609e8f28850 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f267c0a40f0 tx=0x7f267c099b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738136160208a315 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:35.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2667fff700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:43:35.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2667fff700 1 -- 172.21.15.62:0/320639296 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c020070 con 0x5609e8eacfd0 2024-06-10T00:43:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2667fff700 1 -- 172.21.15.62:0/320639296 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f267c09bc10 con 0x5609e8eacfd0 2024-06-10T00:43:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2699ab3b80 1 -- 172.21.15.62:0/320639296 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267c099cb0 con 0x5609e8eacfd0 2024-06-10T00:43:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2667fff700 1 -- 172.21.15.62:0/320639296 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f267c0b3050 con 0x5609e8eacfd0 2024-06-10T00:43:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2699ab3b80 10 client.17172 fetch_fsmap learned FSMap version 38 2024-06-10T00:43:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2699ab3b80 1 -- 172.21.15.62:0/320639296 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5609e8f30fc0 con 0x5609e8eacfd0 2024-06-10T00:43:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2667fff700 1 -- 172.21.15.62:0/320639296 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f267c099cb0 con 0x5609e8eacfd0 2024-06-10T00:43:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2667fff700 1 -- 172.21.15.62:0/320639296 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f267c09cce0 con 0x5609e8eacfd0 2024-06-10T00:43:35.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2699ab3b80 10 client.17172 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:43:35.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2699ab3b80 10 client.17172 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-10T00:43:35.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2699ab3b80 20 client.17172 populate_metadata read hostname 'smithi062' 2024-06-10T00:43:35.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/320639296 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5609e8ec6bc0 0x5609e8f33670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:35.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2699ab3b80 4 client.17172 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:43:35.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.539+0000 7f2699ab3b80 1 -- 172.21.15.62:0/320639296 --> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609e8e02290 con 0x5609e8ec6bc0 2024-06-10T00:43:35.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.540+0000 7f2687fff700 1 --2- 172.21.15.62:0/320639296 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5609e8ec6bc0 0x5609e8f33670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:35.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.540+0000 7f2687fff700 1 --2- 172.21.15.62:0/320639296 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5609e8ec6bc0 0x5609e8f33670 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39f9b33588e2ecbb server_cookie=6ee8a2e11aa891 in_seq=0 out_seq=0 2024-06-10T00:43:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.540+0000 7f2667fff700 10 client.17172 ms_handle_connect on v2:172.21.15.138:6838/339302206 2024-06-10T00:43:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.540+0000 7f2667fff700 1 -- 172.21.15.62:0/320639296 <== mds.1 v2:172.21.15.138:6838/339302206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609e8e02290 con 0x5609e8ec6bc0 2024-06-10T00:43:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.540+0000 7f2667fff700 10 client.17172 handle_command_reply: tid=0 2024-06-10T00:43:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.540+0000 7f2699ab3b80 1 client.17172 shutdown 2024-06-10T00:43:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f2699ab3b80 1 -- 172.21.15.62:0/320639296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eacfd0 msgr2=0x5609e8f28850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/320639296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eacfd0 0x5609e8f28850 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f267c0a40f0 tx=0x7f267c099b90 comp rx=0 tx=0).stop 2024-06-10T00:43:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f2699ab3b80 1 -- 172.21.15.62:0/320639296 shutdown_connections 2024-06-10T00:43:35.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/320639296 >> [v2:172.21.15.138:6838/339302206,v1:172.21.15.138:6839/339302206] conn(0x5609e8ec6bc0 0x5609e8f33670 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/320639296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5609e8eacfd0 0x5609e8f28850 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/320639296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609e8eb1c30 0x5609e8ec95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f2699ab3b80 1 --2- 172.21.15.62:0/320639296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609e8ead8f0 0x5609e8f28d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f2699ab3b80 1 -- 172.21.15.62:0/320639296 >> 172.21.15.62:0/320639296 conn(0x5609e8dd3cc0 msgr2=0x5609e8e9ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f2667fff700 0 client.17172 ms_handle_reset on v2:172.21.15.138:6838/339302206 2024-06-10T00:43:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f2699ab3b80 1 -- 172.21.15.62:0/320639296 shutdown_connections 2024-06-10T00:43:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f2699ab3b80 1 -- 172.21.15.62:0/320639296 wait complete. 2024-06-10T00:43:35.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f2699ab3b80 20 client.17172 trim_cache size 0 max 16384 2024-06-10T00:43:35.552 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:43:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f268f230700 1 -- 172.21.15.62:0/373888272 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f26780a2d90 msgr2=0x7f26780a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f268f230700 1 --2- 172.21.15.62:0/373888272 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f26780a2d90 0x7f26780a5250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f26881070c0 tx=0x7f2674005fb0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f268f230700 1 -- 172.21.15.62:0/373888272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688102740 msgr2=0x7f268819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.541+0000 7f268f230700 1 --2- 172.21.15.62:0/373888272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688102740 0x7f268819e840 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f267c00d950 tx=0x7f267c00dd10 comp rx=0 tx=0).stop 2024-06-10T00:43:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.542+0000 7f268f230700 1 -- 172.21.15.62:0/373888272 shutdown_connections 2024-06-10T00:43:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.542+0000 7f268f230700 1 --2- 172.21.15.62:0/373888272 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f26780a2d90 0x7f26780a5250 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.542+0000 7f268f230700 1 --2- 172.21.15.62:0/373888272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2688102740 0x7f268819e840 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.542+0000 7f268f230700 1 --2- 172.21.15.62:0/373888272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2688107840 0x7f2688198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.542+0000 7f268f230700 1 --2- 172.21.15.62:0/373888272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26881030f0 0x7f268819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:35.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.542+0000 7f268f230700 1 -- 172.21.15.62:0/373888272 >> 172.21.15.62:0/373888272 conn(0x7f26880795f0 msgr2=0x7f2688100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:35.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.542+0000 7f268f230700 1 -- 172.21.15.62:0/373888272 shutdown_connections 2024-06-10T00:43:35.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:35.542+0000 7f268f230700 1 -- 172.21.15.62:0/373888272 wait complete. 2024-06-10T00:43:35.833 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:35 smithi105 ceph-mon[36321]: pgmap v333: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:35.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:35 smithi062 ceph-mon[32390]: pgmap v333: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:35.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:35 smithi138 ceph-mon[36549]: pgmap v333: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:36.359 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T00:43:36.611 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:43:37.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:37 smithi062 ceph-mon[32390]: pgmap v334: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:43:37.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:37 smithi138 ceph-mon[36549]: pgmap v334: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:43:38.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:37 smithi105 ceph-mon[36321]: pgmap v334: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:43:38.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.349+0000 7f3e799f4700 1 -- 172.21.15.62:0/2565164931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e741084e0 msgr2=0x7f3e741088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:38.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.349+0000 7f3e799f4700 1 --2- 172.21.15.62:0/2565164931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e741084e0 0x7f3e741088c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3e640099c0 tx=0x7f3e64009cd0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.349+0000 7f3e799f4700 1 -- 172.21.15.62:0/2565164931 shutdown_connections 2024-06-10T00:43:38.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.349+0000 7f3e799f4700 1 --2- 172.21.15.62:0/2565164931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e741084e0 0x7f3e741088c0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.349+0000 7f3e799f4700 1 --2- 172.21.15.62:0/2565164931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7406ab10 0x7f3e7410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.349+0000 7f3e799f4700 1 --2- 172.21.15.62:0/2565164931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e7406a170 0x7f3e7406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.349+0000 7f3e799f4700 1 -- 172.21.15.62:0/2565164931 >> 172.21.15.62:0/2565164931 conn(0x7f3e740782c0 msgr2=0x7f3e740786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:38.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.349+0000 7f3e799f4700 1 -- 172.21.15.62:0/2565164931 shutdown_connections 2024-06-10T00:43:38.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.349+0000 7f3e799f4700 1 -- 172.21.15.62:0/2565164931 wait complete. 2024-06-10T00:43:38.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.350+0000 7f3e799f4700 1 Processor -- start 2024-06-10T00:43:38.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.350+0000 7f3e799f4700 1 -- start start 2024-06-10T00:43:38.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.350+0000 7f3e799f4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7406a170 0x7f3e7419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.350+0000 7f3e799f4700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e7406ab10 0x7f3e7419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.350+0000 7f3e799f4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e741084e0 0x7f3e74198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.350+0000 7f3e799f4700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3e74112770 con 0x7f3e7406ab10 2024-06-10T00:43:38.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.350+0000 7f3e799f4700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3e741125f0 con 0x7f3e7406a170 2024-06-10T00:43:38.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.350+0000 7f3e799f4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e741128f0 con 0x7f3e741084e0 2024-06-10T00:43:38.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.350+0000 7f3e727fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e7406ab10 0x7f3e7419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.350+0000 7f3e727fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e7406ab10 0x7f3e7419ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55172/0 (socket says 172.21.15.62:55172) 2024-06-10T00:43:38.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.350+0000 7f3e727fc700 1 -- 172.21.15.62:0/1943127421 learned_addr learned my addr 172.21.15.62:0/1943127421 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:38.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.350+0000 7f3e72ffd700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7406a170 0x7f3e7419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.350+0000 7f3e737fe700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e741084e0 0x7f3e74198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.351+0000 7f3e727fc700 1 -- 172.21.15.62:0/1943127421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e741084e0 msgr2=0x7f3e74198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:38.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.351+0000 7f3e727fc700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e741084e0 0x7f3e74198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.351+0000 7f3e727fc700 1 -- 172.21.15.62:0/1943127421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7406a170 msgr2=0x7f3e7419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:38.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.351+0000 7f3e727fc700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7406a170 0x7f3e7419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.351+0000 7f3e727fc700 1 -- 172.21.15.62:0/1943127421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e74199170 con 0x7f3e7406ab10 2024-06-10T00:43:38.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.351+0000 7f3e72ffd700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7406a170 0x7f3e7419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:43:38.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.351+0000 7f3e727fc700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e7406ab10 0x7f3e7419ed40 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800c9b0 tx=0x7f3e6800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:38.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.351+0000 7f3e789f2700 1 -- 172.21.15.62:0/1943127421 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68007b00 con 0x7f3e7406ab10 2024-06-10T00:43:38.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.351+0000 7f3e789f2700 1 -- 172.21.15.62:0/1943127421 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e6800ce50 con 0x7f3e7406ab10 2024-06-10T00:43:38.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.351+0000 7f3e799f4700 1 -- 172.21.15.62:0/1943127421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e74199460 con 0x7f3e7406ab10 2024-06-10T00:43:38.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.351+0000 7f3e789f2700 1 -- 172.21.15.62:0/1943127421 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68018660 con 0x7f3e7406ab10 2024-06-10T00:43:38.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.351+0000 7f3e799f4700 1 -- 172.21.15.62:0/1943127421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e741a56d0 con 0x7f3e7406ab10 2024-06-10T00:43:38.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.353+0000 7f3e789f2700 1 -- 172.21.15.62:0/1943127421 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3e6800f450 con 0x7f3e7406ab10 2024-06-10T00:43:38.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.354+0000 7f3e789f2700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3e5c0a2e10 0x7f3e5c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.355+0000 7f3e72ffd700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3e5c0a2e10 0x7f3e5c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.355+0000 7f3e789f2700 1 -- 172.21.15.62:0/1943127421 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f3e680cf780 con 0x7f3e7406ab10 2024-06-10T00:43:38.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.355+0000 7f3e72ffd700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3e5c0a2e10 0x7f3e5c0a52d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000c00 tx=0x7f3e64005fb0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:38.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.355+0000 7f3e84277b80 1 Processor -- start 2024-06-10T00:43:38.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.355+0000 7f3e84277b80 1 -- start start 2024-06-10T00:43:38.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e84277b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c0f850 0x5572f6c0fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e84277b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c10170 0x5572f6c14010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e84277b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c14550 0x5572f6c14990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e84277b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5572f6afdf60 con 0x5572f6c14550 2024-06-10T00:43:38.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e84277b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5572f6b5c850 con 0x5572f6c10170 2024-06-10T00:43:38.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e84277b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5572f6ac4460 con 0x5572f6c0f850 2024-06-10T00:43:38.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c14550 0x5572f6c14990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c14550 0x5572f6c14990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55180/0 (socket says 172.21.15.62:55180) 2024-06-10T00:43:38.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e737fe700 1 -- 172.21.15.62:0/1019880287 learned_addr learned my addr 172.21.15.62:0/1019880287 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:38.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e727fc700 1 --2- 172.21.15.62:0/1019880287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c10170 0x5572f6c14010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e72ffd700 1 --2- 172.21.15.62:0/1019880287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c0f850 0x5572f6c0fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e737fe700 1 -- 172.21.15.62:0/1019880287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c0f850 msgr2=0x5572f6c0fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e737fe700 1 --2- 172.21.15.62:0/1019880287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c0f850 0x5572f6c0fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e737fe700 1 -- 172.21.15.62:0/1019880287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c10170 msgr2=0x5572f6c14010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e737fe700 1 --2- 172.21.15.62:0/1019880287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c10170 0x5572f6c14010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e737fe700 1 -- 172.21.15.62:0/1019880287 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572f6aab8d0 con 0x5572f6c14550 2024-06-10T00:43:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e737fe700 1 --2- 172.21.15.62:0/1019880287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c14550 0x5572f6c14990 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c003fa0 tx=0x7f3e6c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4243e3dde6b22a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e59ffb700 1 -- 172.21.15.62:0/1019880287 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c004bb0 con 0x5572f6c14550 2024-06-10T00:43:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e59ffb700 1 -- 172.21.15.62:0/1019880287 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e6c004d50 con 0x5572f6c14550 2024-06-10T00:43:38.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.356+0000 7f3e59ffb700 1 -- 172.21.15.62:0/1019880287 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c007e40 con 0x5572f6c14550 2024-06-10T00:43:38.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.357+0000 7f3e84277b80 1 -- 172.21.15.62:0/1019880287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c14550 msgr2=0x5572f6c14990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:38.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.357+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1019880287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c14550 0x5572f6c14990 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c003fa0 tx=0x7f3e6c009ba0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.357+0000 7f3e84277b80 1 -- 172.21.15.62:0/1019880287 shutdown_connections 2024-06-10T00:43:38.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.357+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1019880287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c14550 0x5572f6c14990 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.357+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1019880287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c0f850 0x5572f6c0fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.357+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1019880287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c10170 0x5572f6c14010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.357+0000 7f3e84277b80 1 -- 172.21.15.62:0/1019880287 >> 172.21.15.62:0/1019880287 conn(0x5572f6b38080 msgr2=0x5572f6c06220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:38.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.357+0000 7f3e84277b80 1 -- 172.21.15.62:0/1019880287 shutdown_connections 2024-06-10T00:43:38.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.357+0000 7f3e84277b80 1 -- 172.21.15.62:0/1019880287 wait complete. 2024-06-10T00:43:38.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.358+0000 7f3e84277b80 1 -- start start 2024-06-10T00:43:38.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.358+0000 7f3e84277b80 1 Processor -- start 2024-06-10T00:43:38.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.358+0000 7f3e84277b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c0f850 0x5572f6c1baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.358+0000 7f3e84277b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c10170 0x5572f6c18da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.358+0000 7f3e84277b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c14550 0x5572f6c192e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.358+0000 7f3e84277b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5572f6afcfc0 con 0x5572f6c0f850 2024-06-10T00:43:38.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.358+0000 7f3e84277b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5572f6b58810 con 0x5572f6c14550 2024-06-10T00:43:38.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.358+0000 7f3e84277b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5572f6c1c680 con 0x5572f6c10170 2024-06-10T00:43:38.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.358+0000 7f3e84277b80 10 client.0 _ll_register_callbacks cb 0x5572f6af25b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:43:38.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.358+0000 7f3e72ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c0f850 0x5572f6c1baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.358+0000 7f3e72ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c0f850 0x5572f6c1baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55184/0 (socket says 172.21.15.62:55184) 2024-06-10T00:43:38.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.358+0000 7f3e72ffd700 1 -- 172.21.15.62:0/1716949387 learned_addr learned my addr 172.21.15.62:0/1716949387 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:38.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.358+0000 7f3e737fe700 1 --2- 172.21.15.62:0/1716949387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c14550 0x5572f6c192e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.358+0000 7f3e727fc700 1 --2- 172.21.15.62:0/1716949387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c10170 0x5572f6c18da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e72ffd700 1 -- 172.21.15.62:0/1716949387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c10170 msgr2=0x5572f6c18da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:38.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e72ffd700 1 --2- 172.21.15.62:0/1716949387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c10170 0x5572f6c18da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e72ffd700 1 -- 172.21.15.62:0/1716949387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c14550 msgr2=0x5572f6c192e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:38.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e72ffd700 1 --2- 172.21.15.62:0/1716949387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c14550 0x5572f6c192e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e72ffd700 1 -- 172.21.15.62:0/1716949387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572f6b22750 con 0x5572f6c0f850 2024-06-10T00:43:38.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e737fe700 1 --2- 172.21.15.62:0/1716949387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c14550 0x5572f6c192e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:43:38.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e72ffd700 1 --2- 172.21.15.62:0/1716949387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c0f850 0x5572f6c1baa0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3e64015000 tx=0x7f3e64009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c52e266f9f6277 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:38.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e52ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:43:38.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e52ffd700 1 -- 172.21.15.62:0/1716949387 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64035060 con 0x5572f6c0f850 2024-06-10T00:43:38.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e84277b80 1 -- 172.21.15.62:0/1716949387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e6c004d50 con 0x5572f6c0f850 2024-06-10T00:43:38.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e52ffd700 1 -- 172.21.15.62:0/1716949387 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f3e640248e0 con 0x5572f6c0f850 2024-06-10T00:43:38.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e52ffd700 1 -- 172.21.15.62:0/1716949387 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3e6403c050 con 0x5572f6c0f850 2024-06-10T00:43:38.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e84277b80 10 client.17184 fetch_fsmap learned FSMap version 38 2024-06-10T00:43:38.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e84277b80 1 -- 172.21.15.62:0/1716949387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5572f6c26de0 con 0x5572f6c0f850 2024-06-10T00:43:38.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.359+0000 7f3e52ffd700 1 -- 172.21.15.62:0/1716949387 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e6c004d50 con 0x5572f6c0f850 2024-06-10T00:43:38.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.360+0000 7f3e52ffd700 1 -- 172.21.15.62:0/1716949387 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f3e6402f070 con 0x5572f6c0f850 2024-06-10T00:43:38.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.360+0000 7f3e84277b80 10 client.17184 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:43:38.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.360+0000 7f3e84277b80 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:43:38.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.360+0000 7f3e84277b80 20 client.17184 populate_metadata read hostname 'smithi062' 2024-06-10T00:43:38.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.360+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1716949387 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5572f6c29420 0x5572f6c2b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.360+0000 7f3e84277b80 4 client.17184 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:43:38.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.360+0000 7f3e84277b80 1 -- 172.21.15.62:0/1716949387 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572f6b69d50 con 0x5572f6c29420 2024-06-10T00:43:38.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.360+0000 7f3e727fc700 1 --2- 172.21.15.62:0/1716949387 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5572f6c29420 0x5572f6c2b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.360+0000 7f3e727fc700 1 --2- 172.21.15.62:0/1716949387 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5572f6c29420 0x5572f6c2b810 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95c67c745d11fc5a server_cookie=aef84840a0c0a4cf in_seq=0 out_seq=0 2024-06-10T00:43:38.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.360+0000 7f3e52ffd700 10 client.17184 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:43:38.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.361+0000 7f3e52ffd700 1 -- 172.21.15.62:0/1716949387 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5572f6b69d50 con 0x5572f6c29420 2024-06-10T00:43:38.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.361+0000 7f3e52ffd700 10 client.17184 handle_command_reply: tid=0 2024-06-10T00:43:38.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.361+0000 7f3e84277b80 1 client.17184 shutdown 2024-06-10T00:43:38.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.361+0000 7f3e84277b80 1 -- 172.21.15.62:0/1716949387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c0f850 msgr2=0x5572f6c1baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:38.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.361+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1716949387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c0f850 0x5572f6c1baa0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3e64015000 tx=0x7f3e64009cd0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.362+0000 7f3e84277b80 1 -- 172.21.15.62:0/1716949387 shutdown_connections 2024-06-10T00:43:38.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.362+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1716949387 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5572f6c29420 0x5572f6c2b810 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.362+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1716949387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c0f850 0x5572f6c1baa0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.362+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1716949387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c14550 0x5572f6c192e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.362+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1716949387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c10170 0x5572f6c18da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.362+0000 7f3e52ffd700 0 client.17184 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:43:38.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.362+0000 7f3e84277b80 1 -- 172.21.15.62:0/1716949387 >> 172.21.15.62:0/1716949387 conn(0x5572f6b38080 msgr2=0x5572f6c15a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:38.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.362+0000 7f3e84277b80 1 -- 172.21.15.62:0/1716949387 shutdown_connections 2024-06-10T00:43:38.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.362+0000 7f3e84277b80 1 -- 172.21.15.62:0/1716949387 wait complete. 2024-06-10T00:43:38.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.362+0000 7f3e84277b80 20 client.17184 trim_cache size 0 max 16384 2024-06-10T00:43:38.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.377+0000 7f3e84277b80 1 Processor -- start 2024-06-10T00:43:38.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.377+0000 7f3e84277b80 1 -- start start 2024-06-10T00:43:38.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.377+0000 7f3e84277b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c0f850 0x5572f6c1ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.377+0000 7f3e84277b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c10170 0x5572f6c310e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.377+0000 7f3e84277b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c14550 0x5572f6c31620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.377+0000 7f3e84277b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3e741ac3a0 con 0x5572f6c10170 2024-06-10T00:43:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.377+0000 7f3e84277b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3e741adee0 con 0x5572f6c0f850 2024-06-10T00:43:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.377+0000 7f3e84277b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e741aca70 con 0x5572f6c14550 2024-06-10T00:43:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.377+0000 7f3e727fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c10170 0x5572f6c310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.377+0000 7f3e727fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c10170 0x5572f6c310e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55188/0 (socket says 172.21.15.62:55188) 2024-06-10T00:43:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.377+0000 7f3e727fc700 1 -- 172.21.15.62:0/430885234 learned_addr learned my addr 172.21.15.62:0/430885234 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.378+0000 7f3e72ffd700 1 --2- 172.21.15.62:0/430885234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c0f850 0x5572f6c1ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.378+0000 7f3e737fe700 1 --2- 172.21.15.62:0/430885234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c14550 0x5572f6c31620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.378+0000 7f3e727fc700 1 -- 172.21.15.62:0/430885234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c14550 msgr2=0x5572f6c31620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.378+0000 7f3e727fc700 1 --2- 172.21.15.62:0/430885234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c14550 0x5572f6c31620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.378+0000 7f3e727fc700 1 -- 172.21.15.62:0/430885234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c0f850 msgr2=0x5572f6c1ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.378+0000 7f3e727fc700 1 --2- 172.21.15.62:0/430885234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c0f850 0x5572f6c1ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.378+0000 7f3e727fc700 1 -- 172.21.15.62:0/430885234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572f6c10610 con 0x5572f6c10170 2024-06-10T00:43:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.378+0000 7f3e72ffd700 1 --2- 172.21.15.62:0/430885234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c0f850 0x5572f6c1ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:43:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.378+0000 7f3e727fc700 1 --2- 172.21.15.62:0/430885234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c10170 0x5572f6c310e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3e6809b7c0 tx=0x7f3e68086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ff2ae0e2b6e5ad server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.378+0000 7f3e597fa700 1 -- 172.21.15.62:0/430885234 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680b3070 con 0x5572f6c10170 2024-06-10T00:43:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.378+0000 7f3e597fa700 1 -- 172.21.15.62:0/430885234 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e680998d0 con 0x5572f6c10170 2024-06-10T00:43:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.378+0000 7f3e597fa700 1 -- 172.21.15.62:0/430885234 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68098030 con 0x5572f6c10170 2024-06-10T00:43:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.379+0000 7f3e84277b80 1 -- 172.21.15.62:0/430885234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c10170 msgr2=0x5572f6c310e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.379+0000 7f3e84277b80 1 --2- 172.21.15.62:0/430885234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c10170 0x5572f6c310e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3e6809b7c0 tx=0x7f3e68086040 comp rx=0 tx=0).stop 2024-06-10T00:43:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.379+0000 7f3e84277b80 1 -- 172.21.15.62:0/430885234 shutdown_connections 2024-06-10T00:43:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.379+0000 7f3e84277b80 1 --2- 172.21.15.62:0/430885234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c14550 0x5572f6c31620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.379+0000 7f3e84277b80 1 --2- 172.21.15.62:0/430885234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c0f850 0x5572f6c1ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.379+0000 7f3e84277b80 1 --2- 172.21.15.62:0/430885234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c10170 0x5572f6c310e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.379+0000 7f3e84277b80 1 -- 172.21.15.62:0/430885234 >> 172.21.15.62:0/430885234 conn(0x5572f6b38080 msgr2=0x5572f6c1a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.379+0000 7f3e84277b80 1 -- 172.21.15.62:0/430885234 shutdown_connections 2024-06-10T00:43:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.379+0000 7f3e84277b80 1 -- 172.21.15.62:0/430885234 wait complete. 2024-06-10T00:43:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.379+0000 7f3e84277b80 1 -- start start 2024-06-10T00:43:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.379+0000 7f3e84277b80 1 Processor -- start 2024-06-10T00:43:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e84277b80 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c0f850 0x5572f6c1d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e84277b80 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c10170 0x5572f6c1dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e84277b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c14550 0x5572f6c1e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e84277b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3e741ad810 con 0x5572f6c14550 2024-06-10T00:43:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e84277b80 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3e741ad140 con 0x5572f6c10170 2024-06-10T00:43:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e84277b80 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e741aa360 con 0x5572f6c0f850 2024-06-10T00:43:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e84277b80 10 client.0 _ll_register_callbacks cb 0x5572f6af25b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:43:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c14550 0x5572f6c1e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e737fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c14550 0x5572f6c1e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55196/0 (socket says 172.21.15.62:55196) 2024-06-10T00:43:38.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e737fe700 1 -- 172.21.15.62:0/1190654373 learned_addr learned my addr 172.21.15.62:0/1190654373 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:38.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e727fc700 1 --2- 172.21.15.62:0/1190654373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c10170 0x5572f6c1dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e72ffd700 1 --2- 172.21.15.62:0/1190654373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c0f850 0x5572f6c1d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e737fe700 1 -- 172.21.15.62:0/1190654373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c0f850 msgr2=0x5572f6c1d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:38.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e737fe700 1 --2- 172.21.15.62:0/1190654373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c0f850 0x5572f6c1d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e737fe700 1 -- 172.21.15.62:0/1190654373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c10170 msgr2=0x5572f6c1dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:43:38.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e737fe700 1 --2- 172.21.15.62:0/1190654373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c10170 0x5572f6c1dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e737fe700 1 -- 172.21.15.62:0/1190654373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572f6c931f0 con 0x5572f6c14550 2024-06-10T00:43:38.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e727fc700 1 --2- 172.21.15.62:0/1190654373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c10170 0x5572f6c1dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:43:38.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e737fe700 1 --2- 172.21.15.62:0/1190654373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c14550 0x5572f6c1e2b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c007a60 tx=0x7f3e6c008cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2fd9a12235ff88 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:38.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.380+0000 7f3e52ffd700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-06-10T00:43:38.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.381+0000 7f3e52ffd700 1 -- 172.21.15.62:0/1190654373 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c015690 con 0x5572f6c14550 2024-06-10T00:43:38.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.381+0000 7f3e52ffd700 1 -- 172.21.15.62:0/1190654373 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f3e6c015830 con 0x5572f6c14550 2024-06-10T00:43:38.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.381+0000 7f3e84277b80 1 -- 172.21.15.62:0/1190654373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e680998d0 con 0x5572f6c14550 2024-06-10T00:43:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.381+0000 7f3e52ffd700 1 -- 172.21.15.62:0/1190654373 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3e6c030050 con 0x5572f6c14550 2024-06-10T00:43:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.381+0000 7f3e84277b80 10 client.17190 fetch_fsmap learned FSMap version 38 2024-06-10T00:43:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.381+0000 7f3e84277b80 1 -- 172.21.15.62:0/1190654373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5572f6c94440 con 0x5572f6c14550 2024-06-10T00:43:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.381+0000 7f3e52ffd700 1 -- 172.21.15.62:0/1190654373 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e680998d0 con 0x5572f6c14550 2024-06-10T00:43:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.381+0000 7f3e52ffd700 1 -- 172.21.15.62:0/1190654373 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f3e6c004ad0 con 0x5572f6c14550 2024-06-10T00:43:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.381+0000 7f3e84277b80 10 client.17190 fetch_fsmap finished waiting for FSMap version 38 2024-06-10T00:43:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.381+0000 7f3e84277b80 10 client.17190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-10T00:43:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.381+0000 7f3e84277b80 20 client.17190 populate_metadata read hostname 'smithi062' 2024-06-10T00:43:38.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.381+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1190654373 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5572f6c94af0 0x5572f6c96f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:38.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.381+0000 7f3e84277b80 4 client.17190 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:43:38.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.381+0000 7f3e84277b80 1 -- 172.21.15.62:0/1190654373 --> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572f6b642d0 con 0x5572f6c94af0 2024-06-10T00:43:38.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e72ffd700 1 --2- 172.21.15.62:0/1190654373 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5572f6c94af0 0x5572f6c96f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:38.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e72ffd700 1 --2- 172.21.15.62:0/1190654373 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5572f6c94af0 0x5572f6c96f10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ae7e1b78fc4535 server_cookie=3e30118092dad561 in_seq=0 out_seq=0 2024-06-10T00:43:38.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e52ffd700 10 client.17190 ms_handle_connect on v2:172.21.15.105:6838/1632080154 2024-06-10T00:43:38.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e52ffd700 1 -- 172.21.15.62:0/1190654373 <== mds.0 v2:172.21.15.105:6838/1632080154 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572f6b642d0 con 0x5572f6c94af0 2024-06-10T00:43:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e52ffd700 10 client.17190 handle_command_reply: tid=0 2024-06-10T00:43:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e84277b80 1 client.17190 shutdown 2024-06-10T00:43:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e84277b80 1 -- 172.21.15.62:0/1190654373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c14550 msgr2=0x5572f6c1e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1190654373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c14550 0x5572f6c1e2b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c007a60 tx=0x7f3e6c008cd0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e84277b80 1 -- 172.21.15.62:0/1190654373 shutdown_connections 2024-06-10T00:43:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1190654373 >> [v2:172.21.15.105:6838/1632080154,v1:172.21.15.105:6839/1632080154] conn(0x5572f6c94af0 0x5572f6c96f10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e52ffd700 0 client.17190 ms_handle_reset on v2:172.21.15.105:6838/1632080154 2024-06-10T00:43:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1190654373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5572f6c14550 0x5572f6c1e2b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1190654373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572f6c0f850 0x5572f6c1d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e84277b80 1 --2- 172.21.15.62:0/1190654373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572f6c10170 0x5572f6c1dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e84277b80 1 -- 172.21.15.62:0/1190654373 >> 172.21.15.62:0/1190654373 conn(0x5572f6b38080 msgr2=0x5572f6c13bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e84277b80 1 -- 172.21.15.62:0/1190654373 shutdown_connections 2024-06-10T00:43:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.382+0000 7f3e84277b80 1 -- 172.21.15.62:0/1190654373 wait complete. 2024-06-10T00:43:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.383+0000 7f3e84277b80 20 client.17190 trim_cache size 0 max 16384 2024-06-10T00:43:38.394 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-06-10T00:43:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.383+0000 7f3e799f4700 1 -- 172.21.15.62:0/1943127421 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3e5c0a2e10 msgr2=0x7f3e5c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.383+0000 7f3e799f4700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3e5c0a2e10 0x7f3e5c0a52d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000c00 tx=0x7f3e64005fb0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.383+0000 7f3e799f4700 1 -- 172.21.15.62:0/1943127421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e7406ab10 msgr2=0x7f3e7419ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.383+0000 7f3e799f4700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e7406ab10 0x7f3e7419ed40 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800c9b0 tx=0x7f3e6800ccc0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.383+0000 7f3e799f4700 1 -- 172.21.15.62:0/1943127421 shutdown_connections 2024-06-10T00:43:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.383+0000 7f3e799f4700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f3e5c0a2e10 0x7f3e5c0a52d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.383+0000 7f3e799f4700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e741084e0 0x7f3e74198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.383+0000 7f3e799f4700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e7406a170 0x7f3e7419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.383+0000 7f3e799f4700 1 --2- 172.21.15.62:0/1943127421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e7406ab10 0x7f3e7419ed40 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:38.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.383+0000 7f3e799f4700 1 -- 172.21.15.62:0/1943127421 >> 172.21.15.62:0/1943127421 conn(0x7f3e740782c0 msgr2=0x7f3e740ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:38.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.383+0000 7f3e799f4700 1 -- 172.21.15.62:0/1943127421 shutdown_connections 2024-06-10T00:43:38.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:38.383+0000 7f3e799f4700 1 -- 172.21.15.62:0/1943127421 wait complete. 2024-06-10T00:43:39.185 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-06-10T00:43:39.192 INFO:tasks.fwd_scrub:done joining 2024-06-10T00:43:39.192 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-10T00:43:39.205 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-10T00:43:39.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.322+0000 7f8c0a39e640 1 -- 172.21.15.62:0/446627842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041008f0 msgr2=0x7f8c04100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.322+0000 7f8c0a39e640 1 --2- 172.21.15.62:0/446627842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041008f0 0x7f8c04100d70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009930 tx=0x7f8bf802a420 comp rx=0 tx=0).stop 2024-06-10T00:43:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.322+0000 7f8c0a39e640 1 -- 172.21.15.62:0/446627842 shutdown_connections 2024-06-10T00:43:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.322+0000 7f8c0a39e640 1 --2- 172.21.15.62:0/446627842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c041012b0 0x7f8c0410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.322+0000 7f8c0a39e640 1 --2- 172.21.15.62:0/446627842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041008f0 0x7f8c04100d70 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.322+0000 7f8c0a39e640 1 --2- 172.21.15.62:0/446627842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c041068b0 0x7f8c04106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.322+0000 7f8c0a39e640 1 -- 172.21.15.62:0/446627842 >> 172.21.15.62:0/446627842 conn(0x7f8c04077a00 msgr2=0x7f8c04077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:39.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.322+0000 7f8c0a39e640 1 -- 172.21.15.62:0/446627842 shutdown_connections 2024-06-10T00:43:39.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.322+0000 7f8c0a39e640 1 -- 172.21.15.62:0/446627842 wait complete. 2024-06-10T00:43:39.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.322+0000 7f8c0a39e640 1 Processor -- start 2024-06-10T00:43:39.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.322+0000 7f8c0a39e640 1 -- start start 2024-06-10T00:43:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c0a39e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041008f0 0x7f8c04199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c0a39e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c041012b0 0x7f8c04199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c0a39e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c041068b0 0x7f8c041a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c0a39e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04110cc0 con 0x7f8c041008f0 2024-06-10T00:43:39.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c0a39e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04110b40 con 0x7f8c041068b0 2024-06-10T00:43:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c0a39e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04110e40 con 0x7f8c041012b0 2024-06-10T00:43:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c03fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041008f0 0x7f8c04199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c03fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041008f0 0x7f8c04199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55224/0 (socket says 172.21.15.62:55224) 2024-06-10T00:43:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c03fff640 1 -- 172.21.15.62:0/1931938941 learned_addr learned my addr 172.21.15.62:0/1931938941 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c0893b640 1 --2- 172.21.15.62:0/1931938941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c041068b0 0x7f8c041a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c037fe640 1 --2- 172.21.15.62:0/1931938941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c041012b0 0x7f8c04199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c03fff640 1 -- 172.21.15.62:0/1931938941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c041012b0 msgr2=0x7f8c04199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:39.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c03fff640 1 --2- 172.21.15.62:0/1931938941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c041012b0 0x7f8c04199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c03fff640 1 -- 172.21.15.62:0/1931938941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c041068b0 msgr2=0x7f8c041a40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c03fff640 1 --2- 172.21.15.62:0/1931938941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c041068b0 0x7f8c041a40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c03fff640 1 -- 172.21.15.62:0/1931938941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c041a45e0 con 0x7f8c041008f0 2024-06-10T00:43:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c03fff640 1 --2- 172.21.15.62:0/1931938941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041008f0 0x7f8c04199020 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400d770 tx=0x7f8bf400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.323+0000 7f8c017fa640 1 -- 172.21.15.62:0/1931938941 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4017070 con 0x7f8c041008f0 2024-06-10T00:43:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.324+0000 7f8c017fa640 1 -- 172.21.15.62:0/1931938941 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf4005500 con 0x7f8c041008f0 2024-06-10T00:43:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.324+0000 7f8c0a39e640 1 -- 172.21.15.62:0/1931938941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c041a47b0 con 0x7f8c041008f0 2024-06-10T00:43:39.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.324+0000 7f8c017fa640 1 -- 172.21.15.62:0/1931938941 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4004e00 con 0x7f8c041008f0 2024-06-10T00:43:39.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.324+0000 7f8c0a39e640 1 -- 172.21.15.62:0/1931938941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c041a4cd0 con 0x7f8c041008f0 2024-06-10T00:43:39.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.325+0000 7f8c0a39e640 1 -- 172.21.15.62:0/1931938941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c04101fe0 con 0x7f8c041008f0 2024-06-10T00:43:39.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.326+0000 7f8c017fa640 1 -- 172.21.15.62:0/1931938941 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8bf4004fa0 con 0x7f8c041008f0 2024-06-10T00:43:39.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.327+0000 7f8c017fa640 1 --2- 172.21.15.62:0/1931938941 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8bec0a2d80 0x7f8bec0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:39.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.327+0000 7f8c037fe640 1 --2- 172.21.15.62:0/1931938941 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8bec0a2d80 0x7f8bec0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:39.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.327+0000 7f8c017fa640 1 -- 172.21.15.62:0/1931938941 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f8bf40cf4f0 con 0x7f8c041008f0 2024-06-10T00:43:39.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.327+0000 7f8c037fe640 1 --2- 172.21.15.62:0/1931938941 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8bec0a2d80 0x7f8bec0a5240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009900 tx=0x7f8bf80058d0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:39.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.330+0000 7f8c017fa640 1 -- 172.21.15.62:0/1931938941 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8bf409b7d0 con 0x7f8c041008f0 2024-06-10T00:43:39.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.516+0000 7f8c0a39e640 1 -- 172.21.15.62:0/1931938941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c04108e10 con 0x7f8c041008f0 2024-06-10T00:43:39.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.518+0000 7f8c017fa640 1 -- 172.21.15.62:0/1931938941 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+25403 (secure 0 0 0) 0x7f8bf409fad0 con 0x7f8c041008f0 2024-06-10T00:43:39.519 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:43:39.519 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":108,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:43:13.528561+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3602410963":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/1557827118":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/2350700567":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/3602410963":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/1086776708":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/2528347688":"2024-06-11T00:32:49.913741+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:43:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.520+0000 7f8c0a39e640 1 -- 172.21.15.62:0/1931938941 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8bec0a2d80 msgr2=0x7f8bec0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.520+0000 7f8c0a39e640 1 --2- 172.21.15.62:0/1931938941 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8bec0a2d80 0x7f8bec0a5240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009900 tx=0x7f8bf80058d0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.520+0000 7f8c0a39e640 1 -- 172.21.15.62:0/1931938941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041008f0 msgr2=0x7f8c04199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:39.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.520+0000 7f8c0a39e640 1 --2- 172.21.15.62:0/1931938941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041008f0 0x7f8c04199020 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400d770 tx=0x7f8bf400dc30 comp rx=0 tx=0).stop 2024-06-10T00:43:39.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.520+0000 7f8c0a39e640 1 -- 172.21.15.62:0/1931938941 shutdown_connections 2024-06-10T00:43:39.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.520+0000 7f8c0a39e640 1 --2- 172.21.15.62:0/1931938941 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8bec0a2d80 0x7f8bec0a5240 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.520+0000 7f8c0a39e640 1 --2- 172.21.15.62:0/1931938941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c041068b0 0x7f8c041a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.520+0000 7f8c0a39e640 1 --2- 172.21.15.62:0/1931938941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c041012b0 0x7f8c04199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.520+0000 7f8c0a39e640 1 --2- 172.21.15.62:0/1931938941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c041008f0 0x7f8c04199020 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.520+0000 7f8c0a39e640 1 -- 172.21.15.62:0/1931938941 >> 172.21.15.62:0/1931938941 conn(0x7f8c04077a00 msgr2=0x7f8c041033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.521+0000 7f8c0a39e640 1 -- 172.21.15.62:0/1931938941 shutdown_connections 2024-06-10T00:43:39.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.521+0000 7f8c0a39e640 1 -- 172.21.15.62:0/1931938941 wait complete. 2024-06-10T00:43:39.533 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-10T00:23:46.372672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-10T00:28:33.319774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-10T00:28:36.578569+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-10T00:28:47.112439+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-10T00:43:39.534 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-10T00:43:39.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.642+0000 7fd9de6d4640 1 -- 172.21.15.62:0/542125226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d806a9e0 msgr2=0x7fd9d810e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:39.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.642+0000 7fd9de6d4640 1 --2- 172.21.15.62:0/542125226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d806a9e0 0x7fd9d810e850 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009cd0 tx=0x7fd9d002a8e0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.642+0000 7fd9de6d4640 1 -- 172.21.15.62:0/542125226 shutdown_connections 2024-06-10T00:43:39.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.642+0000 7fd9de6d4640 1 --2- 172.21.15.62:0/542125226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d806a9e0 0x7fd9d810e850 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.642+0000 7fd9de6d4640 1 --2- 172.21.15.62:0/542125226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9d806a020 0x7fd9d806a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.642+0000 7fd9de6d4640 1 --2- 172.21.15.62:0/542125226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d8107990 0x7fd9d8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.642+0000 7fd9de6d4640 1 -- 172.21.15.62:0/542125226 >> 172.21.15.62:0/542125226 conn(0x7fd9d8077900 msgr2=0x7fd9d8077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:39.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.642+0000 7fd9de6d4640 1 -- 172.21.15.62:0/542125226 shutdown_connections 2024-06-10T00:43:39.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.642+0000 7fd9de6d4640 1 -- 172.21.15.62:0/542125226 wait complete. 2024-06-10T00:43:39.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.643+0000 7fd9de6d4640 1 Processor -- start 2024-06-10T00:43:39.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.643+0000 7fd9de6d4640 1 -- start start 2024-06-10T00:43:39.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.643+0000 7fd9de6d4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d806a020 0x7fd9d81981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:39.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.643+0000 7fd9de6d4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d806a9e0 0x7fd9d8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:39.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.643+0000 7fd9de6d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9d8107990 0x7fd9d819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:39.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.643+0000 7fd9de6d4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d8111da0 con 0x7fd9d806a020 2024-06-10T00:43:39.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.643+0000 7fd9de6d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d8111c20 con 0x7fd9d8107990 2024-06-10T00:43:39.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.643+0000 7fd9de6d4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d8111f20 con 0x7fd9d806a9e0 2024-06-10T00:43:39.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9d7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d806a020 0x7fd9d81981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:39.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9d7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d806a020 0x7fd9d81981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55236/0 (socket says 172.21.15.62:55236) 2024-06-10T00:43:39.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9d7fff640 1 -- 172.21.15.62:0/1037901821 learned_addr learned my addr 172.21.15.62:0/1037901821 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:39.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9d77fe640 1 --2- 172.21.15.62:0/1037901821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d806a9e0 0x7fd9d8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:39.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9d7fff640 1 -- 172.21.15.62:0/1037901821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d806a9e0 msgr2=0x7fd9d8198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:39.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9d7fff640 1 --2- 172.21.15.62:0/1037901821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d806a9e0 0x7fd9d8198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9dcc71640 1 --2- 172.21.15.62:0/1037901821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9d8107990 0x7fd9d819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:39.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9d7fff640 1 -- 172.21.15.62:0/1037901821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9d8107990 msgr2=0x7fd9d819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:39.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9d7fff640 1 --2- 172.21.15.62:0/1037901821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9d8107990 0x7fd9d819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9d7fff640 1 -- 172.21.15.62:0/1037901821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d819d1b0 con 0x7fd9d806a020 2024-06-10T00:43:39.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9d7fff640 1 --2- 172.21.15.62:0/1037901821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d806a020 0x7fd9d81981c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800d950 tx=0x7fd9c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:39.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9d57fa640 1 -- 172.21.15.62:0/1037901821 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8017070 con 0x7fd9d806a020 2024-06-10T00:43:39.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9d57fa640 1 -- 172.21.15.62:0/1037901821 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c8009c30 con 0x7fd9d806a020 2024-06-10T00:43:39.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9de6d4640 1 -- 172.21.15.62:0/1037901821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d819d4a0 con 0x7fd9d806a020 2024-06-10T00:43:39.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9d57fa640 1 -- 172.21.15.62:0/1037901821 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c800f6b0 con 0x7fd9d806a020 2024-06-10T00:43:39.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.644+0000 7fd9de6d4640 1 -- 172.21.15.62:0/1037901821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d81a4d30 con 0x7fd9d806a020 2024-06-10T00:43:39.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.645+0000 7fd9de6d4640 1 -- 172.21.15.62:0/1037901821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d8068c10 con 0x7fd9d806a020 2024-06-10T00:43:39.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.646+0000 7fd9d57fa640 1 -- 172.21.15.62:0/1037901821 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd9c80109d0 con 0x7fd9d806a020 2024-06-10T00:43:39.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.647+0000 7fd9d57fa640 1 --2- 172.21.15.62:0/1037901821 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd9c00a2d00 0x7fd9c00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:39.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.647+0000 7fd9d57fa640 1 -- 172.21.15.62:0/1037901821 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fd9c80cf400 con 0x7fd9d806a020 2024-06-10T00:43:39.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.647+0000 7fd9d77fe640 1 --2- 172.21.15.62:0/1037901821 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd9c00a2d00 0x7fd9c00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:39.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.648+0000 7fd9d77fe640 1 --2- 172.21.15.62:0/1037901821 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd9c00a2d00 0x7fd9c00a51c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8077e70 tx=0x7fd9cc00b410 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:39.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.652+0000 7fd9d57fa640 1 -- 172.21.15.62:0/1037901821 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd9c8009de0 con 0x7fd9d806a020 2024-06-10T00:43:39.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.837+0000 7fd9de6d4640 1 -- 172.21.15.62:0/1037901821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd9d802d2e0 con 0x7fd9d806a020 2024-06-10T00:43:39.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.837+0000 7fd9d57fa640 1 -- 172.21.15.62:0/1037901821 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) ==== 93+0+10 (secure 0 0 0) 0x7fd9c809bc10 con 0x7fd9d806a020 2024-06-10T00:43:39.838 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-06-10T00:43:39.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.839+0000 7fd9de6d4640 1 -- 172.21.15.62:0/1037901821 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd9c00a2d00 msgr2=0x7fd9c00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:39.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.839+0000 7fd9de6d4640 1 --2- 172.21.15.62:0/1037901821 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd9c00a2d00 0x7fd9c00a51c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8077e70 tx=0x7fd9cc00b410 comp rx=0 tx=0).stop 2024-06-10T00:43:39.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.840+0000 7fd9de6d4640 1 -- 172.21.15.62:0/1037901821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d806a020 msgr2=0x7fd9d81981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:39.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.840+0000 7fd9de6d4640 1 --2- 172.21.15.62:0/1037901821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d806a020 0x7fd9d81981c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800d950 tx=0x7fd9c800de10 comp rx=0 tx=0).stop 2024-06-10T00:43:39.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.840+0000 7fd9de6d4640 1 -- 172.21.15.62:0/1037901821 shutdown_connections 2024-06-10T00:43:39.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.840+0000 7fd9de6d4640 1 --2- 172.21.15.62:0/1037901821 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fd9c00a2d00 0x7fd9c00a51c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.840+0000 7fd9de6d4640 1 --2- 172.21.15.62:0/1037901821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9d8107990 0x7fd9d819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.840+0000 7fd9de6d4640 1 --2- 172.21.15.62:0/1037901821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d806a9e0 0x7fd9d8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.840+0000 7fd9de6d4640 1 --2- 172.21.15.62:0/1037901821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9d806a020 0x7fd9d81981c0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.840+0000 7fd9de6d4640 1 -- 172.21.15.62:0/1037901821 >> 172.21.15.62:0/1037901821 conn(0x7fd9d8077900 msgr2=0x7fd9d80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:39.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.840+0000 7fd9de6d4640 1 -- 172.21.15.62:0/1037901821 shutdown_connections 2024-06-10T00:43:39.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.840+0000 7fd9de6d4640 1 -- 172.21.15.62:0/1037901821 wait complete. 2024-06-10T00:43:39.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:39 smithi062 ceph-mon[32390]: pgmap v335: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:43:39.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:39 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1931938941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:43:39.851 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-10T00:43:39.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:39 smithi138 ceph-mon[36549]: pgmap v335: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:43:39.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:39 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1931938941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:43:39.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.959+0000 7f69ecb96640 1 -- 172.21.15.62:0/2328200855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69e8106890 msgr2=0x7f69e8106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:39.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.959+0000 7f69ecb96640 1 --2- 172.21.15.62:0/2328200855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69e8106890 0x7f69e8106c70 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009930 tx=0x7f69d802a420 comp rx=0 tx=0).stop 2024-06-10T00:43:39.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.960+0000 7f69ecb96640 1 -- 172.21.15.62:0/2328200855 shutdown_connections 2024-06-10T00:43:39.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.960+0000 7f69ecb96640 1 --2- 172.21.15.62:0/2328200855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e8101240 0x7f69e810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.960+0000 7f69ecb96640 1 --2- 172.21.15.62:0/2328200855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69e8100880 0x7f69e8100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.960+0000 7f69ecb96640 1 --2- 172.21.15.62:0/2328200855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69e8106890 0x7f69e8106c70 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.960+0000 7f69ecb96640 1 -- 172.21.15.62:0/2328200855 >> 172.21.15.62:0/2328200855 conn(0x7f69e8077900 msgr2=0x7f69e8077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:39.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.960+0000 7f69ecb96640 1 -- 172.21.15.62:0/2328200855 shutdown_connections 2024-06-10T00:43:39.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.960+0000 7f69ecb96640 1 -- 172.21.15.62:0/2328200855 wait complete. 2024-06-10T00:43:39.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.960+0000 7f69ecb96640 1 Processor -- start 2024-06-10T00:43:39.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.960+0000 7f69ecb96640 1 -- start start 2024-06-10T00:43:39.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69ecb96640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e8100880 0x7f69e80fd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:39.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69ecb96640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69e8101240 0x7f69e80fddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:39.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69ecb96640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69e8106890 0x7f69e80fe330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:39.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69ecb96640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8110ca0 con 0x7f69e8101240 2024-06-10T00:43:39.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69ecb96640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8110b20 con 0x7f69e8100880 2024-06-10T00:43:39.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69ecb96640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8110e20 con 0x7f69e8106890 2024-06-10T00:43:39.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69e8101240 0x7f69e80fddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:39.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69e8101240 0x7f69e80fddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55252/0 (socket says 172.21.15.62:55252) 2024-06-10T00:43:39.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69e5d9b640 1 -- 172.21.15.62:0/1333530697 learned_addr learned my addr 172.21.15.62:0/1333530697 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:39.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69e6d9d640 1 --2- 172.21.15.62:0/1333530697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69e8106890 0x7f69e80fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:39.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69e659c640 1 --2- 172.21.15.62:0/1333530697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e8100880 0x7f69e80fd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:39.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69e5d9b640 1 -- 172.21.15.62:0/1333530697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69e8106890 msgr2=0x7f69e80fe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:39.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69e5d9b640 1 --2- 172.21.15.62:0/1333530697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69e8106890 0x7f69e80fe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69e5d9b640 1 -- 172.21.15.62:0/1333530697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e8100880 msgr2=0x7f69e80fd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:39.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69e5d9b640 1 --2- 172.21.15.62:0/1333530697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e8100880 0x7f69e80fd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:39.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69e5d9b640 1 -- 172.21.15.62:0/1333530697 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e81a6850 con 0x7f69e8101240 2024-06-10T00:43:39.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69e5d9b640 1 --2- 172.21.15.62:0/1333530697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69e8101240 0x7f69e80fddf0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00c970 tx=0x7f69dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:39.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69d77fe640 1 -- 172.21.15.62:0/1333530697 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc007bb0 con 0x7f69e8101240 2024-06-10T00:43:39.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69ecb96640 1 -- 172.21.15.62:0/1333530697 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e81a6b40 con 0x7f69e8101240 2024-06-10T00:43:39.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.961+0000 7f69d77fe640 1 -- 172.21.15.62:0/1333530697 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69dc007d50 con 0x7f69e8101240 2024-06-10T00:43:39.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.962+0000 7f69d77fe640 1 -- 172.21.15.62:0/1333530697 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc00f450 con 0x7f69e8101240 2024-06-10T00:43:39.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.962+0000 7f69ecb96640 1 -- 172.21.15.62:0/1333530697 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e81a7090 con 0x7f69e8101240 2024-06-10T00:43:39.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.963+0000 7f69ecb96640 1 -- 172.21.15.62:0/1333530697 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e8068c10 con 0x7f69e8101240 2024-06-10T00:43:39.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.964+0000 7f69d77fe640 1 -- 172.21.15.62:0/1333530697 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f69dc005020 con 0x7f69e8101240 2024-06-10T00:43:39.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.964+0000 7f69d77fe640 1 --2- 172.21.15.62:0/1333530697 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f69d00a2d00 0x7f69d00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:39.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.964+0000 7f69d77fe640 1 -- 172.21.15.62:0/1333530697 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f69dc0cf700 con 0x7f69e8101240 2024-06-10T00:43:39.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.964+0000 7f69e659c640 1 --2- 172.21.15.62:0/1333530697 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f69d00a2d00 0x7f69d00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:39.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.965+0000 7f69e659c640 1 --2- 172.21.15.62:0/1333530697 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f69d00a2d00 0x7f69d00a51c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f69d8000c00 tx=0x7f69d8005a20 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:39.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:39.969+0000 7f69d77fe640 1 -- 172.21.15.62:0/1333530697 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f69dc014030 con 0x7f69e8101240 2024-06-10T00:43:40.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:39 smithi105 ceph-mon[36321]: pgmap v335: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:43:40.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:39 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1931938941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:43:40.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.158+0000 7f69ecb96640 1 -- 172.21.15.62:0/1333530697 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f69e8109e20 con 0x7f69e8101240 2024-06-10T00:43:40.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.158+0000 7f69d77fe640 1 -- 172.21.15.62:0/1333530697 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) ==== 104+0+11 (secure 0 0 0) 0x7f69dc09bf10 con 0x7f69e8101240 2024-06-10T00:43:40.159 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-06-10T00:43:40.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.161+0000 7f69ecb96640 1 -- 172.21.15.62:0/1333530697 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f69d00a2d00 msgr2=0x7f69d00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.161+0000 7f69ecb96640 1 --2- 172.21.15.62:0/1333530697 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f69d00a2d00 0x7f69d00a51c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f69d8000c00 tx=0x7f69d8005a20 comp rx=0 tx=0).stop 2024-06-10T00:43:40.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.161+0000 7f69ecb96640 1 -- 172.21.15.62:0/1333530697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69e8101240 msgr2=0x7f69e80fddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.161+0000 7f69ecb96640 1 --2- 172.21.15.62:0/1333530697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69e8101240 0x7f69e80fddf0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00c970 tx=0x7f69dc00ce30 comp rx=0 tx=0).stop 2024-06-10T00:43:40.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.161+0000 7f69ecb96640 1 -- 172.21.15.62:0/1333530697 shutdown_connections 2024-06-10T00:43:40.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.161+0000 7f69ecb96640 1 --2- 172.21.15.62:0/1333530697 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f69d00a2d00 0x7f69d00a51c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.161+0000 7f69ecb96640 1 --2- 172.21.15.62:0/1333530697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69e8106890 0x7f69e80fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.161+0000 7f69ecb96640 1 --2- 172.21.15.62:0/1333530697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69e8101240 0x7f69e80fddf0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.161+0000 7f69ecb96640 1 --2- 172.21.15.62:0/1333530697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e8100880 0x7f69e80fd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.161+0000 7f69ecb96640 1 -- 172.21.15.62:0/1333530697 >> 172.21.15.62:0/1333530697 conn(0x7f69e8077900 msgr2=0x7f69e806a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:40.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.161+0000 7f69ecb96640 1 -- 172.21.15.62:0/1333530697 shutdown_connections 2024-06-10T00:43:40.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.161+0000 7f69ecb96640 1 -- 172.21.15.62:0/1333530697 wait complete. 2024-06-10T00:43:40.172 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-10T00:43:40.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.280+0000 7f5e2dfab640 1 -- 172.21.15.62:0/1017645790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e28107990 msgr2=0x7f5e28107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.280+0000 7f5e2dfab640 1 --2- 172.21.15.62:0/1017645790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e28107990 0x7f5e28107d70 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009930 tx=0x7f5e1802a420 comp rx=0 tx=0).stop 2024-06-10T00:43:40.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.280+0000 7f5e2dfab640 1 -- 172.21.15.62:0/1017645790 shutdown_connections 2024-06-10T00:43:40.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.280+0000 7f5e2dfab640 1 --2- 172.21.15.62:0/1017645790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e2806a9e0 0x7f5e2810e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.280+0000 7f5e2dfab640 1 --2- 172.21.15.62:0/1017645790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2806a020 0x7f5e2806a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.280+0000 7f5e2dfab640 1 --2- 172.21.15.62:0/1017645790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e28107990 0x7f5e28107d70 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.280+0000 7f5e2dfab640 1 -- 172.21.15.62:0/1017645790 >> 172.21.15.62:0/1017645790 conn(0x7f5e28077900 msgr2=0x7f5e28077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:40.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.280+0000 7f5e2dfab640 1 -- 172.21.15.62:0/1017645790 shutdown_connections 2024-06-10T00:43:40.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.280+0000 7f5e2dfab640 1 -- 172.21.15.62:0/1017645790 wait complete. 2024-06-10T00:43:40.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e2dfab640 1 Processor -- start 2024-06-10T00:43:40.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e2dfab640 1 -- start start 2024-06-10T00:43:40.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e2dfab640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2806a020 0x7f5e281982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:40.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e2dfab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e2806a9e0 0x7f5e28198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:40.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e2dfab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e28107990 0x7f5e2819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:40.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e2dfab640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28111d70 con 0x7f5e2806a9e0 2024-06-10T00:43:40.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e2dfab640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28111bf0 con 0x7f5e28107990 2024-06-10T00:43:40.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e2dfab640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28111ef0 con 0x7f5e2806a020 2024-06-10T00:43:40.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e26ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e2806a9e0 0x7f5e28198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:40.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e26ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e2806a9e0 0x7f5e28198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55276/0 (socket says 172.21.15.62:55276) 2024-06-10T00:43:40.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e26ffd640 1 -- 172.21.15.62:0/293969096 learned_addr learned my addr 172.21.15.62:0/293969096 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:40.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e277fe640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2806a020 0x7f5e281982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:40.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e27fff640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e28107990 0x7f5e2819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:40.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e26ffd640 1 -- 172.21.15.62:0/293969096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2806a020 msgr2=0x7f5e281982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e26ffd640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2806a020 0x7f5e281982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e26ffd640 1 -- 172.21.15.62:0/293969096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e28107990 msgr2=0x7f5e2819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e26ffd640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e28107990 0x7f5e2819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e26ffd640 1 -- 172.21.15.62:0/293969096 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2819d2c0 con 0x7f5e2806a9e0 2024-06-10T00:43:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.281+0000 7f5e277fe640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2806a020 0x7f5e281982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:43:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.282+0000 7f5e26ffd640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e2806a9e0 0x7f5e28198810 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000c890 tx=0x7f5e1000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.282+0000 7f5e24ff9640 1 -- 172.21.15.62:0/293969096 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10004220 con 0x7f5e2806a9e0 2024-06-10T00:43:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.282+0000 7f5e24ff9640 1 -- 172.21.15.62:0/293969096 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e100043c0 con 0x7f5e2806a9e0 2024-06-10T00:43:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.282+0000 7f5e24ff9640 1 -- 172.21.15.62:0/293969096 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100049e0 con 0x7f5e2806a9e0 2024-06-10T00:43:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.282+0000 7f5e2dfab640 1 -- 172.21.15.62:0/293969096 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2819d5b0 con 0x7f5e2806a9e0 2024-06-10T00:43:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.282+0000 7f5e2dfab640 1 -- 172.21.15.62:0/293969096 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e281a4e40 con 0x7f5e2806a9e0 2024-06-10T00:43:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.283+0000 7f5e2dfab640 1 -- 172.21.15.62:0/293969096 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e28068c10 con 0x7f5e2806a9e0 2024-06-10T00:43:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.284+0000 7f5e24ff9640 1 -- 172.21.15.62:0/293969096 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5e100079b0 con 0x7f5e2806a9e0 2024-06-10T00:43:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.285+0000 7f5e24ff9640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5e140a2d00 0x7f5e140a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:40.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.285+0000 7f5e277fe640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5e140a2d00 0x7f5e140a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:40.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.285+0000 7f5e24ff9640 1 -- 172.21.15.62:0/293969096 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f5e100cf970 con 0x7f5e2806a9e0 2024-06-10T00:43:40.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.285+0000 7f5e277fe640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5e140a2d00 0x7f5e140a51c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5e18000c00 tx=0x7f5e18032c70 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:40.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.289+0000 7f5e24ff9640 1 -- 172.21.15.62:0/293969096 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5e10014030 con 0x7f5e2806a9e0 2024-06-10T00:43:40.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.475+0000 7f5e2dfab640 1 -- 172.21.15.62:0/293969096 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5e2802d2e0 con 0x7f5e2806a9e0 2024-06-10T00:43:40.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.475+0000 7f5e24ff9640 1 -- 172.21.15.62:0/293969096 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) ==== 100+0+11 (secure 0 0 0) 0x7f5e1009c180 con 0x7f5e2806a9e0 2024-06-10T00:43:40.476 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-06-10T00:43:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.477+0000 7f5e2dfab640 1 -- 172.21.15.62:0/293969096 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5e140a2d00 msgr2=0x7f5e140a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.477+0000 7f5e2dfab640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5e140a2d00 0x7f5e140a51c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5e18000c00 tx=0x7f5e18032c70 comp rx=0 tx=0).stop 2024-06-10T00:43:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.477+0000 7f5e2dfab640 1 -- 172.21.15.62:0/293969096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e2806a9e0 msgr2=0x7f5e28198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.477+0000 7f5e2dfab640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e2806a9e0 0x7f5e28198810 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000c890 tx=0x7f5e1000cd50 comp rx=0 tx=0).stop 2024-06-10T00:43:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.477+0000 7f5e2dfab640 1 -- 172.21.15.62:0/293969096 shutdown_connections 2024-06-10T00:43:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.477+0000 7f5e2dfab640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f5e140a2d00 0x7f5e140a51c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.477+0000 7f5e2dfab640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e28107990 0x7f5e2819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.477+0000 7f5e2dfab640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e2806a9e0 0x7f5e28198810 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.477+0000 7f5e2dfab640 1 --2- 172.21.15.62:0/293969096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e2806a020 0x7f5e281982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.477+0000 7f5e2dfab640 1 -- 172.21.15.62:0/293969096 >> 172.21.15.62:0/293969096 conn(0x7f5e28077900 msgr2=0x7f5e280fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.477+0000 7f5e2dfab640 1 -- 172.21.15.62:0/293969096 shutdown_connections 2024-06-10T00:43:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.477+0000 7f5e2dfab640 1 -- 172.21.15.62:0/293969096 wait complete. 2024-06-10T00:43:40.488 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-10T00:43:40.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.597+0000 7fee429ce640 1 -- 172.21.15.62:0/3940615398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee3c06a960 msgr2=0x7fee3c10e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.597+0000 7fee429ce640 1 --2- 172.21.15.62:0/3940615398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee3c06a960 0x7fee3c10e840 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fee34009ca0 tx=0x7fee3402a950 comp rx=0 tx=0).stop 2024-06-10T00:43:40.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.597+0000 7fee429ce640 1 -- 172.21.15.62:0/3940615398 shutdown_connections 2024-06-10T00:43:40.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.597+0000 7fee429ce640 1 --2- 172.21.15.62:0/3940615398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee3c06a960 0x7fee3c10e840 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.597+0000 7fee429ce640 1 --2- 172.21.15.62:0/3940615398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee3c069fa0 0x7fee3c06a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.597+0000 7fee429ce640 1 --2- 172.21.15.62:0/3940615398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee3c107ba0 0x7fee3c107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.597+0000 7fee429ce640 1 -- 172.21.15.62:0/3940615398 >> 172.21.15.62:0/3940615398 conn(0x7fee3c0766a0 msgr2=0x7fee3c076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:40.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.597+0000 7fee429ce640 1 -- 172.21.15.62:0/3940615398 shutdown_connections 2024-06-10T00:43:40.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.597+0000 7fee429ce640 1 -- 172.21.15.62:0/3940615398 wait complete. 2024-06-10T00:43:40.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.597+0000 7fee429ce640 1 Processor -- start 2024-06-10T00:43:40.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee429ce640 1 -- start start 2024-06-10T00:43:40.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee429ce640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee3c069fa0 0x7fee3c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:40.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee429ce640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee3c06a960 0x7fee3c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:40.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee429ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee3c107ba0 0x7fee3c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:40.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee429ce640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c111d50 con 0x7fee3c06a960 2024-06-10T00:43:40.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee429ce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c111bd0 con 0x7fee3c107ba0 2024-06-10T00:43:40.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee429ce640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c111ed0 con 0x7fee3c069fa0 2024-06-10T00:43:40.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee3b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee3c06a960 0x7fee3c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:40.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee3b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee3c06a960 0x7fee3c19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55294/0 (socket says 172.21.15.62:55294) 2024-06-10T00:43:40.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee3b7fe640 1 -- 172.21.15.62:0/875435705 learned_addr learned my addr 172.21.15.62:0/875435705 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:40.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee40f6b640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee3c107ba0 0x7fee3c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:40.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee3bfff640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee3c069fa0 0x7fee3c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:40.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee3b7fe640 1 -- 172.21.15.62:0/875435705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee3c069fa0 msgr2=0x7fee3c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee3b7fe640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee3c069fa0 0x7fee3c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee3b7fe640 1 -- 172.21.15.62:0/875435705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee3c107ba0 msgr2=0x7fee3c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee3b7fe640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee3c107ba0 0x7fee3c19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee3b7fe640 1 -- 172.21.15.62:0/875435705 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c19f2e0 con 0x7fee3c06a960 2024-06-10T00:43:40.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee40f6b640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee3c107ba0 0x7fee3c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:43:40.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.598+0000 7fee3b7fe640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee3c06a960 0x7fee3c19a830 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fee3000d770 tx=0x7fee3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:40.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.599+0000 7fee397fa640 1 -- 172.21.15.62:0/875435705 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30017070 con 0x7fee3c06a960 2024-06-10T00:43:40.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.599+0000 7fee397fa640 1 -- 172.21.15.62:0/875435705 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee30005500 con 0x7fee3c06a960 2024-06-10T00:43:40.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.599+0000 7fee429ce640 1 -- 172.21.15.62:0/875435705 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3c19f5d0 con 0x7fee3c06a960 2024-06-10T00:43:40.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.599+0000 7fee397fa640 1 -- 172.21.15.62:0/875435705 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30004e00 con 0x7fee3c06a960 2024-06-10T00:43:40.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.599+0000 7fee429ce640 1 -- 172.21.15.62:0/875435705 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3c1a6e60 con 0x7fee3c06a960 2024-06-10T00:43:40.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.600+0000 7fee429ce640 1 -- 172.21.15.62:0/875435705 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee3c1030d0 con 0x7fee3c06a960 2024-06-10T00:43:40.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.601+0000 7fee397fa640 1 -- 172.21.15.62:0/875435705 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fee30004fa0 con 0x7fee3c06a960 2024-06-10T00:43:40.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.601+0000 7fee397fa640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fee240a2d80 0x7fee240a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:40.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.602+0000 7fee3bfff640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fee240a2d80 0x7fee240a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:40.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.602+0000 7fee397fa640 1 -- 172.21.15.62:0/875435705 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fee300cf2d0 con 0x7fee3c06a960 2024-06-10T00:43:40.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.602+0000 7fee3bfff640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fee240a2d80 0x7fee240a5240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fee2c005e10 tx=0x7fee2c005da0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:40.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.606+0000 7fee397fa640 1 -- 172.21.15.62:0/875435705 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fee3009bae0 con 0x7fee3c06a960 2024-06-10T00:43:40.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.792+0000 7fee429ce640 1 -- 172.21.15.62:0/875435705 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fee3c19b640 con 0x7fee3c06a960 2024-06-10T00:43:40.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.792+0000 7fee397fa640 1 -- 172.21.15.62:0/875435705 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v108) ==== 103+0+11 (secure 0 0 0) 0x7fee3009b3e0 con 0x7fee3c06a960 2024-06-10T00:43:40.793 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-06-10T00:43:40.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.794+0000 7fee429ce640 1 -- 172.21.15.62:0/875435705 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fee240a2d80 msgr2=0x7fee240a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.794+0000 7fee429ce640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fee240a2d80 0x7fee240a5240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fee2c005e10 tx=0x7fee2c005da0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.794+0000 7fee429ce640 1 -- 172.21.15.62:0/875435705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee3c06a960 msgr2=0x7fee3c19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.794+0000 7fee429ce640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee3c06a960 0x7fee3c19a830 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fee3000d770 tx=0x7fee3000dc30 comp rx=0 tx=0).stop 2024-06-10T00:43:40.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.794+0000 7fee429ce640 1 -- 172.21.15.62:0/875435705 shutdown_connections 2024-06-10T00:43:40.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.794+0000 7fee429ce640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fee240a2d80 0x7fee240a5240 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.794+0000 7fee429ce640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee3c107ba0 0x7fee3c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.794+0000 7fee429ce640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee3c06a960 0x7fee3c19a830 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.794+0000 7fee429ce640 1 --2- 172.21.15.62:0/875435705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee3c069fa0 0x7fee3c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.794+0000 7fee429ce640 1 -- 172.21.15.62:0/875435705 >> 172.21.15.62:0/875435705 conn(0x7fee3c0766a0 msgr2=0x7fee3c10cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:40.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.794+0000 7fee429ce640 1 -- 172.21.15.62:0/875435705 shutdown_connections 2024-06-10T00:43:40.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.795+0000 7fee429ce640 1 -- 172.21.15.62:0/875435705 wait complete. 2024-06-10T00:43:40.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:40 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1037901821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:43:40.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:40 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1333530697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T00:43:40.805 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:40 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/293969096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T00:43:40.806 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-10T00:43:40.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:40 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1037901821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:43:40.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:40 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1333530697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T00:43:40.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:40 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/293969096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T00:43:40.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.911+0000 7f32d0f7e640 1 -- 172.21.15.62:0/3534888117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc107980 msgr2=0x7f32cc107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.911+0000 7f32d0f7e640 1 --2- 172.21.15.62:0/3534888117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc107980 0x7f32cc107d60 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009930 tx=0x7f32bc02a420 comp rx=0 tx=0).stop 2024-06-10T00:43:40.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.911+0000 7f32d0f7e640 1 -- 172.21.15.62:0/3534888117 shutdown_connections 2024-06-10T00:43:40.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.911+0000 7f32d0f7e640 1 --2- 172.21.15.62:0/3534888117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32cc102330 0x7f32cc10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.911+0000 7f32d0f7e640 1 --2- 172.21.15.62:0/3534888117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc101970 0x7f32cc101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.911+0000 7f32d0f7e640 1 --2- 172.21.15.62:0/3534888117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc107980 0x7f32cc107d60 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.911+0000 7f32d0f7e640 1 -- 172.21.15.62:0/3534888117 >> 172.21.15.62:0/3534888117 conn(0x7f32cc078d60 msgr2=0x7f32cc079170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:40.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.911+0000 7f32d0f7e640 1 -- 172.21.15.62:0/3534888117 shutdown_connections 2024-06-10T00:43:40.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.911+0000 7f32d0f7e640 1 -- 172.21.15.62:0/3534888117 wait complete. 2024-06-10T00:43:40.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32d0f7e640 1 Processor -- start 2024-06-10T00:43:40.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32d0f7e640 1 -- start start 2024-06-10T00:43:40.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32d0f7e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32cc101970 0x7f32cc1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:40.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32d0f7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc102330 0x7f32cc198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:40.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32d0f7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc107980 0x7f32cc19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:40.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32d0f7e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc111d90 con 0x7f32cc102330 2024-06-10T00:43:40.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32d0f7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc111c10 con 0x7f32cc107980 2024-06-10T00:43:40.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32d0f7e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc111f10 con 0x7f32cc101970 2024-06-10T00:43:40.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32c9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc102330 0x7f32cc198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:40.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32c9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc102330 0x7f32cc198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55304/0 (socket says 172.21.15.62:55304) 2024-06-10T00:43:40.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32c9d9b640 1 -- 172.21.15.62:0/1077855220 learned_addr learned my addr 172.21.15.62:0/1077855220 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:43:40.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32ca59c640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32cc101970 0x7f32cc1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:40.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32cad9d640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc107980 0x7f32cc19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:40.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32c9d9b640 1 -- 172.21.15.62:0/1077855220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32cc101970 msgr2=0x7f32cc1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32c9d9b640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32cc101970 0x7f32cc1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32c9d9b640 1 -- 172.21.15.62:0/1077855220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc107980 msgr2=0x7f32cc19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:40.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32c9d9b640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc107980 0x7f32cc19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:40.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32c9d9b640 1 -- 172.21.15.62:0/1077855220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc19d2c0 con 0x7f32cc102330 2024-06-10T00:43:40.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.912+0000 7f32cad9d640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc107980 0x7f32cc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:43:40.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.913+0000 7f32c9d9b640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc102330 0x7f32cc198810 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f32c000ebc0 tx=0x7f32c000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:40.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.913+0000 7f32bb7fe640 1 -- 172.21.15.62:0/1077855220 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0010630 con 0x7f32cc102330 2024-06-10T00:43:40.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.913+0000 7f32bb7fe640 1 -- 172.21.15.62:0/1077855220 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32c0010cb0 con 0x7f32cc102330 2024-06-10T00:43:40.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.913+0000 7f32d0f7e640 1 -- 172.21.15.62:0/1077855220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc19d5b0 con 0x7f32cc102330 2024-06-10T00:43:40.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.913+0000 7f32bb7fe640 1 -- 172.21.15.62:0/1077855220 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0018bd0 con 0x7f32cc102330 2024-06-10T00:43:40.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.913+0000 7f32d0f7e640 1 -- 172.21.15.62:0/1077855220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc1a4e40 con 0x7f32cc102330 2024-06-10T00:43:40.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.914+0000 7f32d0f7e640 1 -- 172.21.15.62:0/1077855220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32cc068c10 con 0x7f32cc102330 2024-06-10T00:43:40.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.915+0000 7f32bb7fe640 1 -- 172.21.15.62:0/1077855220 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f32c00107d0 con 0x7f32cc102330 2024-06-10T00:43:40.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.915+0000 7f32bb7fe640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32b40a2d80 0x7f32b40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:43:40.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.916+0000 7f32ca59c640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32b40a2d80 0x7f32b40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:43:40.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.916+0000 7f32bb7fe640 1 -- 172.21.15.62:0/1077855220 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f32c00cff60 con 0x7f32cc102330 2024-06-10T00:43:40.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.916+0000 7f32ca59c640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32b40a2d80 0x7f32b40a5240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f32bc000c00 tx=0x7f32bc005a20 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:43:40.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:40.920+0000 7f32bb7fe640 1 -- 172.21.15.62:0/1077855220 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f32c009c770 con 0x7f32cc102330 2024-06-10T00:43:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:40 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1037901821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:43:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:40 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1333530697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T00:43:41.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:40 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/293969096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T00:43:41.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:41.154+0000 7f32d0f7e640 1 -- 172.21.15.62:0/1077855220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f32cc109950 con 0x7f32cc102330 2024-06-10T00:43:41.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:41.162+0000 7f32bb7fe640 1 -- 172.21.15.62:0/1077855220 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740900 (secure 0 0 0) 0x7f32c009c070 con 0x7f32cc102330 2024-06-10T00:43:41.172 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:43:41.172 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:43:41.172 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 26, 2024-06-10T00:43:41.173 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.173 INFO:teuthology.orchestra.run.smithi062.stdout: "active_gid": 15333, 2024-06-10T00:43:41.173 INFO:teuthology.orchestra.run.smithi062.stdout: "active_name": "x", 2024-06-10T00:43:41.173 INFO:teuthology.orchestra.run.smithi062.stdout: "active_addrs": { 2024-06-10T00:43:41.173 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-06-10T00:43:41.173 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.173 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-06-10T00:43:41.173 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:6800", 2024-06-10T00:43:41.174 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 518034449 2024-06-10T00:43:41.174 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.174 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.174 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-06-10T00:43:41.174 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:6801", 2024-06-10T00:43:41.174 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 518034449 2024-06-10T00:43:41.174 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.174 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-06-10T00:43:41.175 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.175 INFO:teuthology.orchestra.run.smithi062.stdout: "active_addr": "172.21.15.62:6801/518034449", 2024-06-10T00:43:41.175 INFO:teuthology.orchestra.run.smithi062.stdout: "active_change": "2024-06-10T00:32:49.914040+0000", 2024-06-10T00:43:41.175 INFO:teuthology.orchestra.run.smithi062.stdout: "active_mgr_features": 4540701547738038271, 2024-06-10T00:43:41.175 INFO:teuthology.orchestra.run.smithi062.stdout: "available": true, 2024-06-10T00:43:41.175 INFO:teuthology.orchestra.run.smithi062.stdout: "standbys": [ 2024-06-10T00:43:41.176 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.176 INFO:teuthology.orchestra.run.smithi062.stdout: "gid": 24946, 2024-06-10T00:43:41.176 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "z", 2024-06-10T00:43:41.176 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_features": 4540701547738038271, 2024-06-10T00:43:41.176 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-06-10T00:43:41.176 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.176 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-06-10T00:43:41.177 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.177 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.177 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.177 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:41.177 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:41.177 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.177 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.177 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.178 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:41.178 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.178 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.178 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.178 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-06-10T00:43:41.178 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.178 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.179 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.179 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.179 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.179 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.179 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.179 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.179 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.179 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.179 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.180 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.180 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.180 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.180 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.180 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.180 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.180 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.180 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.181 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.181 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.181 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.181 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.181 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.181 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.181 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.181 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.182 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.182 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.182 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.182 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.182 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.182 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.182 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.182 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.183 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.183 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.183 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.183 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.183 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.183 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.183 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.183 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.184 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.184 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.184 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.184 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.184 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.184 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.184 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.184 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.185 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.185 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.185 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.185 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.185 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.185 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.186 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.186 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.186 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.186 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.186 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.187 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.187 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.187 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.187 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.187 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.187 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.187 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.187 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.188 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.188 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.188 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.188 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.188 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-06-10T00:43:41.188 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-06-10T00:43:41.188 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.188 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.189 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.189 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.189 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.189 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.189 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.189 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-06-10T00:43:41.189 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.189 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.190 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.190 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.190 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-06-10T00:43:41.190 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-06-10T00:43:41.190 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.190 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.190 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.191 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-06-10T00:43:41.191 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.191 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.191 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.191 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-06-10T00:43:41.191 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.192 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.192 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.192 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.192 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-06-10T00:43:41.192 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-06-10T00:43:41.192 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.192 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.193 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.193 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.193 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.193 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.193 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.193 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-06-10T00:43:41.193 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.193 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.194 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.194 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.194 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-06-10T00:43:41.194 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-06-10T00:43:41.194 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.194 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.194 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.194 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.195 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.195 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.195 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.195 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-06-10T00:43:41.195 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.195 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.195 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.196 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.196 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-06-10T00:43:41.196 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-06-10T00:43:41.196 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.196 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.196 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.196 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-06-10T00:43:41.196 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.197 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.197 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.197 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-06-10T00:43:41.197 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.197 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.197 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.197 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.197 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-06-10T00:43:41.198 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-06-10T00:43:41.198 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.198 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.198 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.198 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.198 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.198 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.199 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.199 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-06-10T00:43:41.199 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.199 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.199 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.199 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.199 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-06-10T00:43:41.200 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-06-10T00:43:41.200 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.200 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.200 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.200 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.200 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.200 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.200 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.201 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-10T00:43:41.201 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.201 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.201 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.201 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.201 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-06-10T00:43:41.201 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-06-10T00:43:41.202 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.202 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.202 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.202 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.202 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.202 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.202 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.202 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-06-10T00:43:41.203 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.203 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.203 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.203 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.203 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.203 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.203 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.203 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-06-10T00:43:41.204 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.204 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.204 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.204 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-06-10T00:43:41.204 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-06-10T00:43:41.204 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.204 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.204 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.205 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.205 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.205 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.205 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.205 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-06-10T00:43:41.205 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.205 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.206 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.206 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.206 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-06-10T00:43:41.206 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-06-10T00:43:41.206 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.206 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.206 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.206 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-06-10T00:43:41.207 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.207 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.207 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.207 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-06-10T00:43:41.207 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T00:43:41.207 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.207 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.207 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.208 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-06-10T00:43:41.208 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-06-10T00:43:41.208 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:41.208 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.208 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.208 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:41.208 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-06-10T00:43:41.208 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-06-10T00:43:41.209 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.209 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-10T00:43:41.209 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T00:43:41.209 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.209 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.209 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.209 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-06-10T00:43:41.210 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-06-10T00:43:41.210 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:41.210 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.210 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.210 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-06-10T00:43:41.210 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-06-10T00:43:41.210 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-06-10T00:43:41.210 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.211 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-10T00:43:41.211 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.211 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.211 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.211 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.211 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-06-10T00:43:41.211 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-06-10T00:43:41.211 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.212 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.212 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.212 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-06-10T00:43:41.212 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.212 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.212 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.212 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-10T00:43:41.212 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-10T00:43:41.213 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.213 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.213 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.213 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-06-10T00:43:41.213 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-06-10T00:43:41.213 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:41.213 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.214 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.214 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-06-10T00:43:41.214 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-06-10T00:43:41.214 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-06-10T00:43:41.214 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.214 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-06-10T00:43:41.214 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-10T00:43:41.214 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.215 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.215 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.215 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-06-10T00:43:41.215 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-06-10T00:43:41.215 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.215 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.215 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.215 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-06-10T00:43:41.216 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.216 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.216 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.216 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-06-10T00:43:41.216 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T00:43:41.216 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.216 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.217 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.217 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-06-10T00:43:41.217 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-06-10T00:43:41.217 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:41.217 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.217 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.217 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:41.217 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-06-10T00:43:41.218 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-06-10T00:43:41.218 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.218 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-10T00:43:41.218 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T00:43:41.218 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.218 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.218 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.219 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.219 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.219 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.219 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.219 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.219 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.219 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.219 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.220 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.220 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.220 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.220 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.220 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.220 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.220 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.220 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.221 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.221 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.221 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.221 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.221 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.221 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.221 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.221 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.222 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.222 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.222 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.222 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.222 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.222 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.222 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.223 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.223 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.223 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.223 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.223 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.223 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.223 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.224 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.224 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.224 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.224 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.224 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.224 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.224 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.224 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.225 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.225 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.225 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.225 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.225 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.225 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.225 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.225 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.226 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.226 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.226 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.226 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.226 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.226 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.226 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.227 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.227 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.227 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.227 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.227 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.227 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.227 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.228 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.228 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.228 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-06-10T00:43:41.228 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-06-10T00:43:41.228 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:41.228 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.228 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.228 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:41.229 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.229 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.229 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.229 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-10T00:43:41.229 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.229 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.229 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.230 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.230 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-06-10T00:43:41.230 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-06-10T00:43:41.230 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.230 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.230 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.230 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-06-10T00:43:41.230 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.231 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.231 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.231 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-06-10T00:43:41.231 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-06-10T00:43:41.231 INFO:teuthology.orchestra.run.smithi062.stdout: "read", 2024-06-10T00:43:41.231 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap", 2024-06-10T00:43:41.231 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap-read" 2024-06-10T00:43:41.231 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.232 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-06-10T00:43:41.232 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.232 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.232 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.232 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.232 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-06-10T00:43:41.232 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-06-10T00:43:41.233 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.233 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.233 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.233 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.233 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.233 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.233 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.233 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-10T00:43:41.234 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.234 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.234 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.234 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.234 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:41.234 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:41.234 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.234 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.235 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.235 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:41.235 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.235 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.235 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.235 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-10T00:43:41.235 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.236 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.236 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.236 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.236 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-06-10T00:43:41.236 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-06-10T00:43:41.236 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.236 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.236 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.237 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-06-10T00:43:41.237 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-06-10T00:43:41.237 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.237 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.237 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-10T00:43:41.237 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-10T00:43:41.237 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.237 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.238 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.238 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-06-10T00:43:41.238 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-06-10T00:43:41.238 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:41.238 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.238 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.238 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:41.239 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.239 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.239 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.239 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-10T00:43:41.239 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.239 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.239 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.239 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.240 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.240 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.240 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.240 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-06-10T00:43:41.240 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.240 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.240 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.241 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-06-10T00:43:41.241 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-06-10T00:43:41.241 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:41.241 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.241 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.241 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-06-10T00:43:41.241 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.241 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.242 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.242 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-10T00:43:41.242 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.242 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.242 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.242 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.242 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-06-10T00:43:41.243 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-06-10T00:43:41.243 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.243 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.243 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.243 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-06-10T00:43:41.243 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.243 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.243 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.244 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-10T00:43:41.244 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.244 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.244 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.244 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.244 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-06-10T00:43:41.244 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-06-10T00:43:41.244 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.245 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.245 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.245 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-06-10T00:43:41.245 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.245 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.245 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.245 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-10T00:43:41.246 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.246 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.246 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.246 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.246 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-06-10T00:43:41.246 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-06-10T00:43:41.247 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.247 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.247 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.247 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.247 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.247 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.247 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.247 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-10T00:43:41.248 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-10T00:43:41.248 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.248 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.248 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.248 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-06-10T00:43:41.248 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-06-10T00:43:41.248 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.249 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.249 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.249 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:41.249 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.249 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.249 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.249 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-10T00:43:41.249 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.250 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.250 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.250 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.250 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-06-10T00:43:41.250 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-06-10T00:43:41.250 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:41.250 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.251 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.251 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-06-10T00:43:41.251 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.251 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.251 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.251 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-10T00:43:41.251 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.251 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.252 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.252 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.252 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm_log_destination": { 2024-06-10T00:43:41.252 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm_log_destination", 2024-06-10T00:43:41.252 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.252 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.252 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.252 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.253 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.253 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.253 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.253 INFO:teuthology.orchestra.run.smithi062.stdout: "file", 2024-06-10T00:43:41.253 INFO:teuthology.orchestra.run.smithi062.stdout: "file,syslog", 2024-06-10T00:43:41.253 INFO:teuthology.orchestra.run.smithi062.stdout: "syslog" 2024-06-10T00:43:41.253 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.253 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-10T00:43:41.254 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.254 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.254 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.254 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.254 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-06-10T00:43:41.254 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-06-10T00:43:41.254 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.255 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.255 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.255 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.255 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.255 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.255 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.255 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-10T00:43:41.255 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.256 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.256 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.256 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.256 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-06-10T00:43:41.256 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-06-10T00:43:41.256 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.256 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.256 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.257 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.257 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.257 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.257 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.257 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-10T00:43:41.257 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.257 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.257 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.258 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.258 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-06-10T00:43:41.258 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-06-10T00:43:41.258 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.258 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.258 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.258 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.259 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.259 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.259 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.259 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-10T00:43:41.259 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.259 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.259 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.259 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.260 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-06-10T00:43:41.260 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-06-10T00:43:41.260 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.260 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.260 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.260 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-10T00:43:41.260 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.261 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.261 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.261 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-06-10T00:43:41.261 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.261 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.261 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.261 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.261 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-06-10T00:43:41.262 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-06-10T00:43:41.262 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.262 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.262 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.262 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-10T00:43:41.262 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.262 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.263 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.263 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-06-10T00:43:41.263 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.263 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.263 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.263 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.263 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-06-10T00:43:41.263 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-06-10T00:43:41.264 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.264 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.264 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.264 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-10T00:43:41.264 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.264 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.264 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.265 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-06-10T00:43:41.265 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.265 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.265 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.265 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.265 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-06-10T00:43:41.265 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-06-10T00:43:41.265 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.266 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.266 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.266 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-10T00:43:41.266 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.266 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.266 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.266 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-06-10T00:43:41.266 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.267 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.267 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.267 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.267 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-06-10T00:43:41.267 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-06-10T00:43:41.267 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.267 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.267 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.267 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-10T00:43:41.268 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.268 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.268 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.268 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-06-10T00:43:41.268 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.268 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.268 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.269 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.269 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-06-10T00:43:41.269 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-06-10T00:43:41.269 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.269 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.269 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.269 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-10T00:43:41.269 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.269 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.270 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.270 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-06-10T00:43:41.270 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.270 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.270 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.270 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.270 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-06-10T00:43:41.271 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-06-10T00:43:41.271 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.271 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.271 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.271 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-10T00:43:41.271 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.271 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.271 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.272 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-06-10T00:43:41.272 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.272 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.272 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.272 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.272 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-06-10T00:43:41.272 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-06-10T00:43:41.272 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.273 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.273 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.273 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-10T00:43:41.273 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.273 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.273 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.273 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-06-10T00:43:41.273 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.274 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.274 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.274 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.274 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-06-10T00:43:41.274 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-06-10T00:43:41.274 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.274 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.275 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.275 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-10T00:43:41.275 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.275 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.275 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.275 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-06-10T00:43:41.275 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.275 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.276 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.276 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.276 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-06-10T00:43:41.276 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-06-10T00:43:41.276 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.276 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.276 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.276 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-10T00:43:41.277 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.277 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.277 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.277 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-06-10T00:43:41.277 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.277 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.277 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.277 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.277 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-06-10T00:43:41.278 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-06-10T00:43:41.278 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.278 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.278 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.278 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-10T00:43:41.278 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.278 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.279 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.279 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-06-10T00:43:41.279 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.279 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.279 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.279 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.279 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-06-10T00:43:41.279 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-06-10T00:43:41.280 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.280 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.280 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.280 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-10T00:43:41.280 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.280 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.280 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.280 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-06-10T00:43:41.281 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.281 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.281 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.281 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.281 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-06-10T00:43:41.281 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-06-10T00:43:41.281 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.281 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.281 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.282 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-10T00:43:41.282 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.282 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.282 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.282 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-06-10T00:43:41.282 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.282 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.283 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.283 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.283 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-06-10T00:43:41.283 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-06-10T00:43:41.283 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.283 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.283 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.283 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-10T00:43:41.284 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.284 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.284 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.284 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-06-10T00:43:41.284 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.284 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.284 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.284 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.285 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_samba": { 2024-06-10T00:43:41.285 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_samba", 2024-06-10T00:43:41.285 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.285 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.285 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.285 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-10T00:43:41.285 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.286 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.286 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.286 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Samba/SMB container image", 2024-06-10T00:43:41.286 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.286 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.286 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.286 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.287 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-06-10T00:43:41.287 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-06-10T00:43:41.287 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.287 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.287 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.287 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-10T00:43:41.287 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.287 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.288 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.288 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-06-10T00:43:41.288 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.288 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.288 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.288 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.288 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-06-10T00:43:41.288 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-06-10T00:43:41.289 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.289 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.289 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.289 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.289 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.289 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.290 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.290 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-06-10T00:43:41.290 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.290 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.290 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.290 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.290 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-06-10T00:43:41.291 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-06-10T00:43:41.291 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.291 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.291 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.291 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:41.291 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.291 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.291 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.292 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-10T00:43:41.292 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.292 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.292 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.292 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.292 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-06-10T00:43:41.292 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-06-10T00:43:41.292 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.293 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.293 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.293 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-06-10T00:43:41.293 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.293 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.293 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.293 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-10T00:43:41.294 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.294 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.294 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.294 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.294 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-06-10T00:43:41.294 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-06-10T00:43:41.294 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.295 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.295 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.295 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-06-10T00:43:41.295 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.295 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.295 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.295 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-10T00:43:41.295 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.296 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.296 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.296 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.296 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-06-10T00:43:41.296 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-06-10T00:43:41.296 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.296 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.296 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-06-10T00:43:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-06-10T00:43:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.297 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.298 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.298 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-06-10T00:43:41.298 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-06-10T00:43:41.298 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.298 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.298 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.298 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.299 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.299 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.299 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.299 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-10T00:43:41.299 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.299 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.299 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.299 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.300 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-06-10T00:43:41.300 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-06-10T00:43:41.300 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.300 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.300 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.300 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:41.300 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.300 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.301 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.301 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-06-10T00:43:41.301 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.301 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.301 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.301 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.301 INFO:teuthology.orchestra.run.smithi062.stdout: "grafana_dashboards_path": { 2024-06-10T00:43:41.302 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "grafana_dashboards_path", 2024-06-10T00:43:41.302 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.302 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.302 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.302 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-10T00:43:41.302 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.302 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.302 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.303 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-10T00:43:41.303 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.303 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.303 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.303 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.303 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-06-10T00:43:41.303 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-06-10T00:43:41.304 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.304 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.304 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.304 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:41.304 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.304 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.304 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.304 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-06-10T00:43:41.305 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.305 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.305 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.305 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.305 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-06-10T00:43:41.305 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-06-10T00:43:41.305 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.306 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.306 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.306 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.306 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.306 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.306 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.306 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-10T00:43:41.306 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.307 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.307 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.307 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.307 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-06-10T00:43:41.307 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-06-10T00:43:41.307 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.307 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.307 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.308 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.308 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.308 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.308 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.308 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-10T00:43:41.308 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.308 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.309 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.309 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.309 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.309 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.309 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.309 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.309 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.309 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.310 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.310 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.310 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.310 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.310 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.310 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.310 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.311 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.311 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.311 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.311 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.311 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.311 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.311 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.311 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.312 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-06-10T00:43:41.312 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-06-10T00:43:41.312 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.312 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.312 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.312 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.313 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.313 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.313 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.313 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-10T00:43:41.313 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.313 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.313 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.313 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.314 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.314 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.314 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.314 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.314 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.314 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.314 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.315 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.315 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.315 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-10T00:43:41.315 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.315 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.315 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.315 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.315 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.316 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.316 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.316 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.316 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.316 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.316 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.316 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.316 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.317 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.317 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.317 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.317 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.317 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.317 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.317 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.318 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.318 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.318 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.318 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.318 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.318 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.318 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.319 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.319 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.319 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.319 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.319 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.319 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.319 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.319 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.320 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.320 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.320 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.320 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.320 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-10T00:43:41.320 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-10T00:43:41.320 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.321 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.321 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.321 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.321 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.321 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.321 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.321 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-10T00:43:41.321 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.321 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.322 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.322 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.322 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-10T00:43:41.322 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-10T00:43:41.322 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.322 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.322 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.323 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-06-10T00:43:41.323 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.323 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.323 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.323 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-10T00:43:41.323 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.323 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.323 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.324 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.324 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-06-10T00:43:41.324 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-06-10T00:43:41.324 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.324 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.324 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.324 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:41.324 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-06-10T00:43:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-06-10T00:43:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-06-10T00:43:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-10T00:43:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-06-10T00:43:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-06-10T00:43:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-06-10T00:43:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-06-10T00:43:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-06-10T00:43:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-06-10T00:43:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-06-10T00:43:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-06-10T00:43:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-06-10T00:43:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-06-10T00:43:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-06-10T00:43:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-06-10T00:43:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-10T00:43:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-06-10T00:43:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-06-10T00:43:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-10T00:43:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-10T00:43:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-06-10T00:43:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-06-10T00:43:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-10T00:43:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-06-10T00:43:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-06-10T00:43:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-10T00:43:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.339 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.339 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.339 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.339 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-06-10T00:43:41.339 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-06-10T00:43:41.339 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.339 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.339 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.340 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.340 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.340 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.340 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.340 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-10T00:43:41.340 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.340 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.340 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.341 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.341 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-06-10T00:43:41.341 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-06-10T00:43:41.341 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.341 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.341 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.341 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.341 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.342 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.342 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.342 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-10T00:43:41.342 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.342 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.342 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.342 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.343 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-06-10T00:43:41.343 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-06-10T00:43:41.343 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.343 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.343 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.343 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.343 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.343 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.344 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.344 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-10T00:43:41.344 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.344 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.344 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.344 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.344 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-06-10T00:43:41.344 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-06-10T00:43:41.345 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.345 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.345 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.345 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-06-10T00:43:41.345 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.345 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.345 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.345 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-06-10T00:43:41.346 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.346 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.346 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.346 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.346 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-06-10T00:43:41.346 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-06-10T00:43:41.346 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.346 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.347 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.347 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.347 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.347 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.347 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.347 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-10T00:43:41.347 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.347 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.348 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.348 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.348 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-06-10T00:43:41.348 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-06-10T00:43:41.348 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.348 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.348 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.349 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.349 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.349 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.349 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.349 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-10T00:43:41.349 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.349 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.349 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.350 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.350 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-06-10T00:43:41.350 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-06-10T00:43:41.350 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.350 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.350 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.350 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.351 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.351 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.351 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.351 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-10T00:43:41.351 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.351 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.351 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.351 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.352 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-06-10T00:43:41.352 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-06-10T00:43:41.352 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.352 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.352 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.352 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.352 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.353 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.353 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.353 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-06-10T00:43:41.353 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.353 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.353 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.353 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.353 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-06-10T00:43:41.354 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-06-10T00:43:41.354 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.354 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.354 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.354 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.354 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.354 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.355 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.355 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-10T00:43:41.355 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.355 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.355 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.355 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.355 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-06-10T00:43:41.355 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-06-10T00:43:41.356 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.356 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.356 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.356 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.356 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.356 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.356 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.356 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-10T00:43:41.358 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.358 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.358 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.358 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.359 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.359 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.359 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.359 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-06-10T00:43:41.359 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.359 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.359 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.360 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.360 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.360 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.360 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.360 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.360 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.360 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.360 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.361 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.361 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.361 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.361 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.361 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.361 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.361 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.362 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.362 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.362 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.362 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.362 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.362 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.362 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.362 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.363 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.363 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.363 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.363 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.363 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.363 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.363 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.363 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.364 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.364 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.364 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.364 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.364 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.364 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.364 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.364 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.365 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.365 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.365 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.365 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.365 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.365 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.365 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.365 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.365 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.366 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.366 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.366 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.366 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.366 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.366 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.366 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.367 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.367 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.367 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.367 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.367 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.367 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.367 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.367 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.368 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.368 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.368 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.368 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.368 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.368 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.368 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.368 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-06-10T00:43:41.368 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-06-10T00:43:41.369 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.369 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.369 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.369 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-06-10T00:43:41.369 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.369 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.369 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.369 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-10T00:43:41.370 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.370 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.370 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.370 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.370 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-06-10T00:43:41.370 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-06-10T00:43:41.370 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.371 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.371 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.371 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-06-10T00:43:41.371 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.371 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.371 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.371 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-10T00:43:41.371 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.372 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.372 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.372 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.372 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.372 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.372 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.372 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-06-10T00:43:41.372 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.372 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.373 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.373 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-10T00:43:41.373 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-10T00:43:41.373 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.373 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.373 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.373 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:41.373 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.374 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.374 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.374 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.374 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.374 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.374 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.374 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.374 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-06-10T00:43:41.375 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-10T00:43:41.375 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.375 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.375 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.375 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.375 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.375 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.375 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.376 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.376 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.376 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.376 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.376 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.376 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-10T00:43:41.376 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-10T00:43:41.376 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.377 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.377 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.377 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.377 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.377 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.377 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.377 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.377 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.377 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.378 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.378 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.378 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-06-10T00:43:41.378 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-06-10T00:43:41.378 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.378 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.378 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.379 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.379 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.379 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.379 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.379 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.379 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.379 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.379 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.380 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.380 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-10T00:43:41.380 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-10T00:43:41.380 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.380 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.380 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.380 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.380 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.381 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.381 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.381 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.381 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.381 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.381 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.381 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.381 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-06-10T00:43:41.382 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-10T00:43:41.382 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.382 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.382 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.382 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.382 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.382 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.383 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.383 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.383 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.383 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.383 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.383 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.383 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-10T00:43:41.384 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-10T00:43:41.384 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.384 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.384 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.384 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.384 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.384 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.384 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.385 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.385 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.385 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.385 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.385 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.385 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-10T00:43:41.385 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-10T00:43:41.385 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.385 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.386 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.386 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.386 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.386 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.386 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.386 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.386 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.387 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.387 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.387 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.387 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-10T00:43:41.387 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-10T00:43:41.387 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.387 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.387 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.388 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.388 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.388 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.388 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.388 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.388 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.388 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.388 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.389 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.389 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-10T00:43:41.389 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-10T00:43:41.389 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.389 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.389 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.389 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.389 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.389 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.390 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.390 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.390 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.390 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.390 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.390 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.390 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-10T00:43:41.390 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-10T00:43:41.391 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.391 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.391 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.391 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.391 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.391 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.391 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.391 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.392 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.392 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.392 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.392 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.392 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-10T00:43:41.392 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-10T00:43:41.392 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.392 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.393 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.393 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.393 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.393 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.393 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.393 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.393 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.393 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.394 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.394 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.394 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-10T00:43:41.394 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-10T00:43:41.394 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.394 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.394 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.394 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.395 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.395 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.395 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.395 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.395 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.395 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.395 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.395 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.396 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-10T00:43:41.396 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-10T00:43:41.396 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.396 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.396 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.396 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.396 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.396 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.396 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.397 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.397 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.397 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.397 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.397 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.397 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-06-10T00:43:41.397 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-10T00:43:41.397 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.397 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.398 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.398 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-06-10T00:43:41.398 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.398 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.398 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.398 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.398 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.398 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.399 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.399 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.399 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-10T00:43:41.399 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-10T00:43:41.399 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.399 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.399 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.399 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.400 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.400 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.400 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.400 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.400 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.400 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.400 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.400 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.401 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-06-10T00:43:41.401 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-06-10T00:43:41.401 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.401 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.401 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.401 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.401 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.402 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.402 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.402 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.402 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.402 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.402 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.402 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.402 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-06-10T00:43:41.403 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-10T00:43:41.403 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.403 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.403 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.403 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-06-10T00:43:41.403 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.403 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.403 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.404 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.404 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.404 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.404 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.404 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.404 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-10T00:43:41.404 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-10T00:43:41.405 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.405 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.405 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.405 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.405 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.405 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.405 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.405 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.405 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.406 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.406 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.406 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.406 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-10T00:43:41.406 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-10T00:43:41.406 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.406 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.407 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.407 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.407 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.407 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.407 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.407 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.407 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.407 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.407 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.408 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.408 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-10T00:43:41.408 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-10T00:43:41.408 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.408 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.408 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.408 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.408 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.408 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.409 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.409 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.409 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.409 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.409 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.409 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.409 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-10T00:43:41.409 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-10T00:43:41.410 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.410 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.410 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.410 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.410 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.410 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.410 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.410 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.411 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.411 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.411 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.411 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.411 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-06-10T00:43:41.411 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-10T00:43:41.411 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.411 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.412 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.412 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.412 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.412 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.412 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.412 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.412 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.412 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.413 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.413 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.413 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-10T00:43:41.413 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-10T00:43:41.413 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.413 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.413 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.413 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.414 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.414 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.414 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.414 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.414 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.414 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.414 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.414 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.415 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-10T00:43:41.415 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-10T00:43:41.415 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.415 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.415 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.415 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.415 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.415 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.416 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.416 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.416 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.416 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.416 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.416 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.416 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-10T00:43:41.416 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-10T00:43:41.417 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.417 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.417 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.417 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.417 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.417 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.417 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.417 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.418 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.418 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.418 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.418 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.418 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-10T00:43:41.418 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-10T00:43:41.418 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.418 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.419 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.419 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.419 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.419 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.419 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.419 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.419 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.419 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.420 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.420 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.420 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-10T00:43:41.420 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-10T00:43:41.420 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.420 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.420 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.420 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.420 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.421 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.421 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.421 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.421 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.421 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.421 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.421 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.421 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-10T00:43:41.422 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-10T00:43:41.422 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.422 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.422 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.422 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.422 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.422 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.422 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.423 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.423 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.423 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.423 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.423 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-10T00:43:41.423 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-10T00:43:41.423 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.424 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.424 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.424 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.424 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.424 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.424 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.424 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.424 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.424 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.425 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.425 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.425 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-10T00:43:41.425 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-10T00:43:41.425 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.425 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.425 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.425 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.426 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.426 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.426 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.426 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.426 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.426 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.426 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.426 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.427 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-06-10T00:43:41.427 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-10T00:43:41.427 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.427 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.427 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.427 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.427 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.427 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.428 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.428 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.428 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.428 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.428 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.428 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.428 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-10T00:43:41.429 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-10T00:43:41.429 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.429 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.429 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.429 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-10T00:43:41.429 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.429 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.430 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.430 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.430 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.430 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.430 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.430 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.430 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-10T00:43:41.430 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-10T00:43:41.431 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.431 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.431 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.431 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:41.431 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.431 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.431 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.431 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.432 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.432 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.432 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.432 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.432 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-10T00:43:41.432 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-10T00:43:41.432 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.433 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.433 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.433 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-06-10T00:43:41.433 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.433 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.433 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.433 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.433 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.434 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.434 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.434 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-10T00:43:41.434 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-10T00:43:41.434 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.434 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.434 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.435 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-06-10T00:43:41.435 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.435 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.435 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.435 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.435 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.435 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.435 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.435 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.436 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-06-10T00:43:41.436 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-10T00:43:41.436 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.436 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.436 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.436 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.436 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.436 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.437 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.437 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.437 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.437 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.437 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.437 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.437 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-10T00:43:41.437 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-10T00:43:41.438 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.438 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.438 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.438 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-06-10T00:43:41.438 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.438 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.438 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.438 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.439 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.439 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.439 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.439 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.439 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-06-10T00:43:41.439 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-10T00:43:41.439 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.439 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.439 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.440 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.440 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.440 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.440 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.440 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.440 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.440 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.440 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.441 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.441 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-06-10T00:43:41.441 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-10T00:43:41.441 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.441 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.441 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.441 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.441 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.441 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.442 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.442 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.442 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.442 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.442 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.442 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.442 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-10T00:43:41.442 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-10T00:43:41.443 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.443 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.443 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.443 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:41.443 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.443 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.443 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.443 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.444 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.444 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.444 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.444 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.444 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-10T00:43:41.444 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-10T00:43:41.444 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.444 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.445 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.445 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:41.445 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.445 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.445 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.445 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.445 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.445 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.445 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.446 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.446 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-10T00:43:41.446 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-10T00:43:41.446 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.446 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.446 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.446 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-06-10T00:43:41.447 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.447 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.447 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.447 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.447 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.447 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.447 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.447 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.448 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-06-10T00:43:41.448 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-06-10T00:43:41.448 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.448 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.448 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.448 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.448 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.448 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.448 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.449 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.449 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.449 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.449 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.449 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.449 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-06-10T00:43:41.449 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-06-10T00:43:41.449 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.450 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.450 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.450 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.450 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.450 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.450 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.450 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.450 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.451 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.451 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.451 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.451 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-06-10T00:43:41.451 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-06-10T00:43:41.451 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.451 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.452 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.452 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.452 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.452 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.452 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.452 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-06-10T00:43:41.452 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.452 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.452 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.453 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.453 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-06-10T00:43:41.453 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-06-10T00:43:41.453 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.453 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.453 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.453 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-06-10T00:43:41.453 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.454 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.454 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.454 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.454 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.454 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.454 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.454 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.454 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-06-10T00:43:41.455 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-06-10T00:43:41.455 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.455 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.455 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.455 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.455 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.455 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.455 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.456 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.456 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.456 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.456 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.456 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.456 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.456 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.456 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.457 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.457 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.457 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.457 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.457 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.457 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.457 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.457 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.458 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.458 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.458 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.458 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.458 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.458 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.458 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.458 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.459 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.459 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.459 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.459 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.459 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.459 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.459 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.459 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.460 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.460 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.460 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.460 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.460 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.460 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.460 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.461 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.461 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.461 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.461 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.461 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.461 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.461 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.461 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.462 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.462 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.462 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.462 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.462 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.462 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.462 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.462 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.463 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.463 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.463 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.463 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.463 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.463 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.463 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.463 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.464 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.464 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.464 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.464 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.464 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.464 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.464 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.464 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.465 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.465 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.465 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.465 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.465 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-06-10T00:43:41.465 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-06-10T00:43:41.465 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.465 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.466 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.466 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.466 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.466 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.466 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.466 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-06-10T00:43:41.466 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.466 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.467 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.467 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.467 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-06-10T00:43:41.467 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-06-10T00:43:41.467 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.467 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.467 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.467 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.468 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.468 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.468 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.468 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.468 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.468 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.468 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.468 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.469 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-06-10T00:43:41.469 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-06-10T00:43:41.469 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.469 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.469 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.469 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-06-10T00:43:41.469 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.469 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.470 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.470 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.470 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.470 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.470 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.470 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.470 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-06-10T00:43:41.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-06-10T00:43:41.471 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.471 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.471 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.471 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-06-10T00:43:41.471 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.471 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.471 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.471 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.472 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.472 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.472 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.472 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.472 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-06-10T00:43:41.472 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-06-10T00:43:41.472 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.472 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.473 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.473 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.473 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.473 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.473 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.473 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.473 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.473 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.473 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.474 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.474 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-06-10T00:43:41.474 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-06-10T00:43:41.474 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.474 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.474 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.474 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-06-10T00:43:41.475 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.475 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.475 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.475 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.475 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.475 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.475 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.475 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.476 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-06-10T00:43:41.476 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-06-10T00:43:41.476 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.476 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.476 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.476 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-06-10T00:43:41.476 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.477 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.477 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.477 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.477 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-06-10T00:43:41.477 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.477 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.477 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.477 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.478 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.478 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.478 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-06-10T00:43:41.478 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-06-10T00:43:41.478 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.478 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.478 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.478 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-06-10T00:43:41.479 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-06-10T00:43:41.479 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-06-10T00:43:41.479 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.479 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.479 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.479 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.479 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.479 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.480 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-06-10T00:43:41.480 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-06-10T00:43:41.480 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.480 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.480 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.480 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.480 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.480 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.481 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.481 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.481 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.481 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.481 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.481 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.481 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.481 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.481 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.482 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-06-10T00:43:41.482 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.482 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.482 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.482 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-06-10T00:43:41.482 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-06-10T00:43:41.482 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.482 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.483 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.483 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.483 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.483 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.483 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.483 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-06-10T00:43:41.483 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.483 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.483 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.484 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.484 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.484 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.484 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.484 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.484 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.484 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.484 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.484 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.485 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.485 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.485 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.485 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.485 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.485 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.485 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.486 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.486 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.486 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.486 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.486 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.486 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.486 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.487 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.487 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.487 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.487 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.487 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.487 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.487 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.487 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.488 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.488 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.488 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.488 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.488 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.488 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.488 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.488 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.488 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.489 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.489 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.489 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.489 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.489 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.489 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.489 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.489 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.489 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.490 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.490 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.490 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.490 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.490 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.490 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.490 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.490 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.490 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.491 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.491 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.491 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.491 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.491 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.491 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.491 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.491 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.491 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.492 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.492 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.492 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.492 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.492 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-06-10T00:43:41.492 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-06-10T00:43:41.492 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.492 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.492 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.493 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-06-10T00:43:41.493 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.493 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.493 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.493 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-10T00:43:41.493 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.493 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.493 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.494 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.494 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-06-10T00:43:41.494 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-06-10T00:43:41.494 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.494 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.494 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.494 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-06-10T00:43:41.494 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.494 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.495 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.495 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-10T00:43:41.495 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.495 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.495 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.495 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.495 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-06-10T00:43:41.495 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-06-10T00:43:41.495 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.496 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.496 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.496 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-06-10T00:43:41.496 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.496 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.496 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.496 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-06-10T00:43:41.496 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.497 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.497 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.497 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.497 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-06-10T00:43:41.497 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-06-10T00:43:41.497 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.497 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.497 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.497 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-06-10T00:43:41.498 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.498 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.498 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.498 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-10T00:43:41.498 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.498 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.498 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.499 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.499 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-06-10T00:43:41.499 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-06-10T00:43:41.499 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.499 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.499 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.499 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.499 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.500 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.500 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.500 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-10T00:43:41.500 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.500 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.500 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.500 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.500 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:41.500 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:41.501 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.501 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.501 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.501 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:41.501 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.501 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.501 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.501 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-06-10T00:43:41.502 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.502 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.502 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.502 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.502 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-06-10T00:43:41.502 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-06-10T00:43:41.502 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.502 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.503 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.503 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-06-10T00:43:41.503 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.503 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.503 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.503 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-10T00:43:41.503 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.503 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.503 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.504 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.504 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.504 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.504 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.504 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-06-10T00:43:41.504 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.504 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.504 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.505 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.505 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.505 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.505 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.505 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.505 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.505 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.506 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.506 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.506 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.506 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.506 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.506 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.506 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.506 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.507 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.507 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.507 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.507 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.507 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.507 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.507 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.507 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.508 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.508 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.508 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.508 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.508 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.508 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.508 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.508 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.509 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.509 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.509 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.509 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.509 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.509 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.509 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.509 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.510 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.510 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.510 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.510 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.510 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.510 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.510 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.510 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.511 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.511 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.511 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.511 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.511 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.511 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.511 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.511 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.512 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.512 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.512 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.512 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.512 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.512 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.512 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.512 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.513 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.513 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.513 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.513 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.513 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.513 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.513 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.513 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-06-10T00:43:41.514 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-06-10T00:43:41.514 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.514 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.514 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.514 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-06-10T00:43:41.514 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.514 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.514 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.515 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.515 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.515 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.515 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.515 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.515 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-06-10T00:43:41.515 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-06-10T00:43:41.515 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.516 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.516 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.516 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-06-10T00:43:41.516 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.516 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.516 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.516 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.516 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.517 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.517 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.517 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.517 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:41.517 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:41.517 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.517 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.518 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.518 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:41.518 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.518 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.518 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.518 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.518 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.518 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.518 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.519 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.519 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.519 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.519 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.519 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-06-10T00:43:41.519 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-06-10T00:43:41.519 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-06-10T00:43:41.519 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.520 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-06-10T00:43:41.520 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-06-10T00:43:41.520 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.520 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.520 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.520 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-06-10T00:43:41.520 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.520 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.521 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.521 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-10T00:43:41.521 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.521 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.521 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.521 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.521 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-06-10T00:43:41.521 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-06-10T00:43:41.522 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.522 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.522 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.522 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-06-10T00:43:41.522 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.522 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.522 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.522 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-10T00:43:41.523 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.523 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.523 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.523 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.523 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-06-10T00:43:41.523 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-06-10T00:43:41.523 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.524 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.524 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.524 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.524 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.524 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.524 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.524 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-06-10T00:43:41.524 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.525 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.525 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.525 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.525 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:41.525 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:41.525 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.525 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.525 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.525 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-06-10T00:43:41.526 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-06-10T00:43:41.526 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.526 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.526 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-10T00:43:41.526 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.526 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.526 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.527 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.527 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.527 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.527 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.527 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.527 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.527 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.527 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.527 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.528 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.528 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.528 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.528 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.528 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.528 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.528 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.528 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.529 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.529 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.529 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.529 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.529 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.529 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.529 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.529 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.530 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.530 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.530 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.530 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.530 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.530 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.530 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.530 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.531 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.531 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.531 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.531 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.531 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.531 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.531 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.531 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.532 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.532 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.532 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.532 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.532 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.532 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.532 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.532 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.533 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.533 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.533 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.533 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.533 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.533 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.533 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.534 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.534 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.534 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.534 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.534 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.534 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.534 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.535 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.535 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.535 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.535 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.535 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.535 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.535 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.535 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.536 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-06-10T00:43:41.536 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-06-10T00:43:41.536 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.536 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.536 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.536 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.536 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.536 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.537 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.537 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-06-10T00:43:41.537 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.537 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.537 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.537 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.537 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-06-10T00:43:41.538 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-06-10T00:43:41.538 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.538 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.538 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.538 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-06-10T00:43:41.538 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.538 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.538 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.539 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-06-10T00:43:41.539 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.539 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.539 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.539 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.539 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-06-10T00:43:41.539 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-06-10T00:43:41.540 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.540 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.540 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.540 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-06-10T00:43:41.540 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.540 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.540 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.540 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T00:43:41.541 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.541 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.541 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.541 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.541 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-06-10T00:43:41.541 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-06-10T00:43:41.541 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.541 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.542 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.542 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-06-10T00:43:41.542 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-06-10T00:43:41.542 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-06-10T00:43:41.542 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.542 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-10T00:43:41.542 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.542 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.543 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.543 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.543 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-06-10T00:43:41.543 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-06-10T00:43:41.543 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.543 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.543 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.543 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.543 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.544 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.544 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.544 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-06-10T00:43:41.544 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.544 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.544 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.544 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.544 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-06-10T00:43:41.545 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-06-10T00:43:41.545 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.545 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.545 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.545 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-06-10T00:43:41.545 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.545 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.545 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.546 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T00:43:41.546 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.546 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.546 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.546 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.546 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.546 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.546 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.547 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-06-10T00:43:41.547 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.547 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.547 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.547 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.547 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.547 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.547 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.548 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.548 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.548 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.548 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.548 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.548 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.548 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.548 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.548 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.549 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.549 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.549 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.549 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.549 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.549 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.549 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.549 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.549 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.550 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.550 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.550 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.550 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.550 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.550 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.550 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.550 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.551 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.551 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.551 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.551 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.551 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.551 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.551 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.551 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.552 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.552 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.552 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.552 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.552 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.552 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.552 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.552 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.553 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.553 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.553 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.553 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.553 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.553 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.553 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.553 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.553 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.554 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.554 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.554 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.554 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.554 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.554 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.554 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.554 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.555 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.555 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.555 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.555 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.555 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.555 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.555 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.556 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.556 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.556 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.556 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-06-10T00:43:41.556 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.556 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.556 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.556 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.556 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.557 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.557 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.557 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.557 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.557 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.557 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.557 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.557 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.557 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.558 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.558 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.558 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.558 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.558 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.558 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.558 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.558 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.559 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.559 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.559 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.559 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.559 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.559 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.559 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.559 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.560 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.560 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.560 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.560 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.560 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.560 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.560 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.560 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.560 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.561 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.561 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.561 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.561 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.561 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.561 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.561 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.561 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.561 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.562 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.562 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.562 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.562 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.562 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.562 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.562 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.562 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.563 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.563 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.563 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.563 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.563 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.563 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.563 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.563 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.563 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.564 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.564 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.564 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.564 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.564 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.564 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.564 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.564 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.565 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.565 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.565 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.565 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-06-10T00:43:41.565 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.565 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.565 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.565 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-06-10T00:43:41.566 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-06-10T00:43:41.566 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.566 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.566 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.566 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-06-10T00:43:41.566 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.566 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.566 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.567 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-10T00:43:41.567 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.567 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.567 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.567 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.567 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-06-10T00:43:41.567 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-06-10T00:43:41.567 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.567 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.567 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.568 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:41.568 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-06-10T00:43:41.568 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.568 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.568 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-10T00:43:41.568 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.568 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.568 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.569 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.569 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.569 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.569 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.569 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.569 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.569 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.569 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.570 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.570 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.570 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.570 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.570 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.570 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.570 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.570 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.571 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.571 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.571 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.571 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.571 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.571 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.571 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.571 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.571 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.572 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.572 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.572 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.572 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.572 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.572 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.572 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.572 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.572 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.573 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.573 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.573 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.573 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.573 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.573 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.573 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.573 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.573 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.574 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.574 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.574 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.574 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.574 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.574 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.574 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.574 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.575 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.575 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.575 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.575 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.575 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.575 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.575 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.575 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.576 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.576 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.576 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.576 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.576 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.576 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.576 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.576 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.576 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.577 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.577 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.577 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.577 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.577 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.577 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.577 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-06-10T00:43:41.577 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.577 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.578 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.578 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-06-10T00:43:41.578 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-06-10T00:43:41.578 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.578 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.578 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.578 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-06-10T00:43:41.578 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.579 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.579 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.579 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-06-10T00:43:41.579 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-10T00:43:41.579 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.579 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.579 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.580 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.580 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.580 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.580 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.580 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.580 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.580 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.580 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.580 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.581 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.581 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.581 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.581 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.581 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.581 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.581 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.581 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.581 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.582 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.582 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.582 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.582 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.582 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.582 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.582 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.582 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.583 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.583 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.583 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.583 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.583 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.583 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.583 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.583 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.583 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.584 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.584 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.584 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.584 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.584 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.584 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.584 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.584 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.585 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.585 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.585 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.585 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.585 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.585 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.585 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.585 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.586 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.586 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.586 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.586 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.586 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.586 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.586 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.586 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.587 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.587 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.587 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.587 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.587 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.587 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.587 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.587 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.588 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.588 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.588 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.588 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-06-10T00:43:41.588 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-06-10T00:43:41.588 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.588 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.588 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.588 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.589 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.589 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.589 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.589 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-06-10T00:43:41.589 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-10T00:43:41.589 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.589 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.589 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.589 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-06-10T00:43:41.590 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-06-10T00:43:41.590 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.590 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.590 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.590 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-06-10T00:43:41.590 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.590 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.591 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.591 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-06-10T00:43:41.591 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.591 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.591 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.591 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.591 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-06-10T00:43:41.591 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-06-10T00:43:41.591 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.592 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.592 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.592 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-06-10T00:43:41.592 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.592 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.592 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.592 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-06-10T00:43:41.592 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.593 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.593 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.593 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.593 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-06-10T00:43:41.593 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-06-10T00:43:41.593 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.593 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.593 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.593 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.594 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.594 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.594 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.594 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-06-10T00:43:41.594 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.594 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.594 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.594 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.595 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-06-10T00:43:41.595 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-06-10T00:43:41.595 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.595 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.595 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.595 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-06-10T00:43:41.595 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.596 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.596 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.596 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-10T00:43:41.596 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-10T00:43:41.596 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.596 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.596 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.596 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.596 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.597 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.597 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-06-10T00:43:41.597 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.597 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.597 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.597 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.597 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.597 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.598 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.598 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.598 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.598 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.598 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.598 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.598 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.598 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.599 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.599 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.599 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.599 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.599 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.599 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.599 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.599 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.600 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.600 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.600 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.600 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.600 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.600 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.600 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.600 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.601 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.601 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.601 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.601 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.601 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.601 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.601 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.601 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.602 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.602 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.602 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.602 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.602 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.602 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.602 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.602 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.602 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.603 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.603 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.603 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.603 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.603 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.603 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.603 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.603 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.604 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.604 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.604 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.604 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.604 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.604 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.604 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.604 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.604 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.605 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.605 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.605 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.605 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.605 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.605 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.605 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.605 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.606 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.606 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.606 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.606 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.606 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-06-10T00:43:41.606 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.606 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.606 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.607 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.608 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.608 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.608 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.608 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.608 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.608 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.608 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.609 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.609 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.609 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.609 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.609 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.609 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.609 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.609 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.609 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.610 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.610 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.610 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.610 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.610 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.610 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.610 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.610 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.611 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.611 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.611 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.611 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.611 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.611 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.611 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.611 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.611 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.612 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.612 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.612 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.612 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.612 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.612 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.612 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.612 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.613 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.613 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.613 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.613 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.613 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.613 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.613 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.613 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.614 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.614 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.614 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.614 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.614 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.614 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.615 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.615 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.615 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.615 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.615 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.615 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.615 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.615 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.616 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.616 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.616 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-06-10T00:43:41.616 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.616 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.616 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.616 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.617 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.617 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.617 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.617 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.617 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.617 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.617 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.617 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.618 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.618 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.618 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.618 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.618 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.618 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.619 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.619 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.619 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.619 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.619 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.619 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.619 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.619 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.619 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.620 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.620 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.620 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.620 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.620 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.620 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.620 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.621 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.621 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.621 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.621 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.621 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.621 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.621 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.621 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.621 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.622 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.622 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.622 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.622 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.622 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.622 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.622 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.622 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.623 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.623 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.623 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.623 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.623 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.623 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.623 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.623 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.624 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.624 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.624 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.624 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.624 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.624 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.624 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.624 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.624 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.625 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.625 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.625 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.625 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.625 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.625 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.625 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.625 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.626 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-06-10T00:43:41.626 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.626 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.626 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.626 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-06-10T00:43:41.626 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-06-10T00:43:41.626 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.626 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.627 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.627 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.627 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.627 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.627 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.627 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-10T00:43:41.627 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.627 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.628 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.628 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.628 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.628 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.628 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.628 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.628 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.628 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.629 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.629 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.629 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.629 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.629 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.629 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.629 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.629 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.629 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.630 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.630 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.630 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.630 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.630 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.630 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.630 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.631 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.631 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.631 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.631 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.631 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.631 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.631 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.631 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.632 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.632 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.632 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.632 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.632 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.632 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.632 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.632 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.632 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.633 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.633 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.633 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.633 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.633 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.633 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.633 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.633 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.633 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.634 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.634 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.634 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.634 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.634 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.634 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.634 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.634 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.635 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.635 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.635 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.635 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.635 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.635 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.635 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.635 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.635 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.636 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.636 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.636 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.636 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.636 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.636 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-06-10T00:43:41.636 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-06-10T00:43:41.636 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.636 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.637 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.637 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.637 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.637 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.637 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.637 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-06-10T00:43:41.637 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-06-10T00:43:41.637 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-06-10T00:43:41.638 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.638 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-06-10T00:43:41.638 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.638 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.638 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.638 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.638 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.638 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.639 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.639 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-06-10T00:43:41.639 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.639 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.639 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.639 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.639 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.639 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.639 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.640 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.640 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.640 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.640 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.640 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.640 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.640 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.640 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.640 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.641 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.641 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.641 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.641 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.641 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.641 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.641 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.641 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.641 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.642 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.642 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.642 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.642 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.642 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.642 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.642 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.642 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.643 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.643 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.643 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.643 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.643 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.643 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.643 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.643 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.643 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.644 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.644 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.644 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.644 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.644 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.644 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.644 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.644 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.644 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.645 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.645 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.645 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.645 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.645 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.645 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.645 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.645 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.645 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.646 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.646 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.646 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.646 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.646 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.646 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.646 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.646 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.647 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.647 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.647 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.647 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.647 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.647 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.647 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.647 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.647 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-06-10T00:43:41.648 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.648 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.648 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.648 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.648 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.648 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.648 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.648 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.649 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.649 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.649 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.649 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.649 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.649 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.649 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.650 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.650 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.650 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.650 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.650 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.650 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.650 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.650 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.651 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.651 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.651 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.651 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.651 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.651 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.651 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.651 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.651 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.652 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.652 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.652 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.652 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.652 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.652 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.652 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.652 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.653 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.653 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.653 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.653 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.653 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.653 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.653 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.653 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.654 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.654 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.654 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.654 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.654 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.654 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.654 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.654 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.655 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.655 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.655 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.655 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.655 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.655 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.655 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.655 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.656 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.656 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.656 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.656 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.656 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.656 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.656 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.656 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.657 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.657 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.657 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.657 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.657 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-06-10T00:43:41.657 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.657 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.657 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.658 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.658 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.658 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.658 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.658 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.658 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.658 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.658 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.659 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.660 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.661 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.662 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.662 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.662 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.662 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.662 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.662 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.662 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.662 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.663 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.663 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.663 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.663 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.663 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.663 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.663 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.663 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.664 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.664 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.664 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.664 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.664 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.664 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.664 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.664 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.665 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.665 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.665 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.665 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.665 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.665 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.665 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.665 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.666 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.666 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.666 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.666 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.666 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.666 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:41.666 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:41.667 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.667 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.667 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.667 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:41.667 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.667 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.667 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.667 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.667 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.668 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.668 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.668 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.668 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-06-10T00:43:41.668 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-06-10T00:43:41.668 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:41.668 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.668 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.668 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-06-10T00:43:41.669 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-06-10T00:43:41.669 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.669 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.669 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-06-10T00:43:41.669 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-10T00:43:41.669 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.669 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.669 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.669 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.670 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.670 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.670 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-06-10T00:43:41.670 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.670 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.670 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.670 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-06-10T00:43:41.671 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-06-10T00:43:41.671 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.671 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.671 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.671 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.671 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.671 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.671 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.671 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-10T00:43:41.672 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.672 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.672 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.672 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.672 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-06-10T00:43:41.672 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-06-10T00:43:41.672 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.672 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.672 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.673 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.673 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.673 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.673 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.673 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.673 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.673 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.673 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.673 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.674 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.674 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.674 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.674 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.674 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.674 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.674 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.675 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.675 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.675 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.675 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.675 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.675 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.675 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.675 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.676 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.676 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.676 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.676 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.676 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.676 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.676 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.676 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.677 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.677 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.677 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.677 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.677 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.677 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.677 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.677 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.677 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.678 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.678 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.678 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.678 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.678 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.678 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.678 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.678 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.679 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.679 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.679 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.679 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.679 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.679 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.679 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.679 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.679 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.680 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.680 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.680 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.680 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.680 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.680 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.680 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.680 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.680 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.681 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.681 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.681 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.681 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.681 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.681 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.681 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.681 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.681 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.681 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.682 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.682 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.682 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-06-10T00:43:41.682 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-06-10T00:43:41.682 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.682 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.682 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.682 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-06-10T00:43:41.683 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.683 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.683 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.683 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-06-10T00:43:41.683 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.683 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.683 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.683 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.683 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:41.684 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:41.684 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.684 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.684 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.684 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-06-10T00:43:41.684 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.684 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.684 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.685 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-06-10T00:43:41.685 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.685 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.685 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.685 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.685 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.685 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.685 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.686 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-06-10T00:43:41.686 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.686 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.686 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.686 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-06-10T00:43:41.686 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-06-10T00:43:41.686 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.686 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.687 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.687 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.687 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.687 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.687 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.687 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.687 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.687 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.688 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.688 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.688 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-06-10T00:43:41.688 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-06-10T00:43:41.688 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.688 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.688 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.688 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.689 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.689 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.689 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.689 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-10T00:43:41.689 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-10T00:43:41.689 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.689 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.689 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.689 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.690 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.690 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.690 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.690 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.690 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.690 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.690 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.691 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.691 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.691 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.691 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.691 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.691 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.691 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.691 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.692 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.692 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.692 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.692 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.692 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.692 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.692 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.692 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.692 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.693 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.693 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.693 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.693 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.693 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.693 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.693 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.693 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.693 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.694 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.694 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.694 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.694 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.694 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.694 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.694 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.694 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.695 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.695 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.695 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.695 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.695 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.695 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.695 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.695 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.695 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.696 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.696 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.696 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.696 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.696 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.696 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.696 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.696 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.696 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.697 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.697 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.697 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.697 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.697 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.697 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.697 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.697 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.697 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.698 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.698 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-06-10T00:43:41.698 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-06-10T00:43:41.698 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.698 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.698 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.698 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.698 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.699 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.699 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.699 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.699 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.699 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.699 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.699 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.699 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-10T00:43:41.700 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-10T00:43:41.700 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.700 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.700 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.700 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-06-10T00:43:41.700 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.700 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.700 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.700 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.701 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.701 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.701 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.701 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.701 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-06-10T00:43:41.701 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-06-10T00:43:41.701 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:41.701 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.702 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.702 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-06-10T00:43:41.702 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.702 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.702 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.702 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.702 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.702 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.703 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.703 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.703 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-06-10T00:43:41.703 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-06-10T00:43:41.703 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.703 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.703 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.703 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-06-10T00:43:41.703 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.704 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.704 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.704 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-10T00:43:41.704 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.704 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.704 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.704 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.704 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-06-10T00:43:41.704 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-06-10T00:43:41.705 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.705 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.705 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.705 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-06-10T00:43:41.705 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.705 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.705 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.705 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-10T00:43:41.705 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.706 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.706 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.706 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-06-10T00:43:41.706 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-06-10T00:43:41.706 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.706 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.706 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.707 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-06-10T00:43:41.707 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.707 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.707 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.707 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.707 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.707 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.707 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.708 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-06-10T00:43:41.708 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-06-10T00:43:41.708 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.708 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.708 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.708 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-06-10T00:43:41.708 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.708 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.708 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.709 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-06-10T00:43:41.709 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-06-10T00:43:41.709 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.709 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.709 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.709 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.709 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.709 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.709 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-06-10T00:43:41.710 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-06-10T00:43:41.710 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.710 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.710 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.710 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-06-10T00:43:41.710 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-06-10T00:43:41.710 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-06-10T00:43:41.711 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.711 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.711 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.711 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.711 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.711 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.711 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.711 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.712 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.712 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-06-10T00:43:41.712 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.712 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.712 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.712 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.712 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.712 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.712 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.713 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.713 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.713 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.713 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.713 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.713 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.713 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.713 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.713 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.714 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.714 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.714 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.714 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.714 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.714 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.714 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.714 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.715 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.715 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.715 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.715 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.715 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.715 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.715 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.715 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.715 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.716 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.716 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.716 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.716 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.716 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.716 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.716 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.716 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.717 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.717 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.717 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.717 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.717 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.717 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.717 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.717 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.717 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.718 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.718 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.718 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.718 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.718 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.718 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.718 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.718 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.718 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.719 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.719 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.719 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.719 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.719 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.719 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.719 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.719 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.720 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.720 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.720 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.720 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.720 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.720 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.720 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-06-10T00:43:41.720 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-06-10T00:43:41.721 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.721 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.721 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.721 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:41.721 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.721 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.721 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.721 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.722 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.722 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.722 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.722 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.722 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-06-10T00:43:41.722 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-06-10T00:43:41.722 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.722 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.723 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.723 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.723 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.723 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.723 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.723 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.723 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.723 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.724 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.724 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.724 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-06-10T00:43:41.724 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-06-10T00:43:41.724 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.724 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.724 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.724 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.725 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.725 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.725 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.725 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.725 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.725 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.725 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.725 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.726 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.726 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.726 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.726 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-06-10T00:43:41.726 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.726 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.726 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.727 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-06-10T00:43:41.727 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-06-10T00:43:41.727 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.727 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.727 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.727 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.727 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.727 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.728 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.728 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.728 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.728 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.728 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.728 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.728 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-06-10T00:43:41.728 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-06-10T00:43:41.729 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.729 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.729 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.729 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.729 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.729 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.729 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.729 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.730 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.730 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.730 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.730 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.730 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.731 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.731 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.731 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.731 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.731 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.731 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.731 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.731 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.732 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.732 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.732 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.733 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.734 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.734 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.734 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.734 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.734 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.734 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.734 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.734 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.735 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.735 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.735 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.735 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.735 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.735 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.735 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.735 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.736 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.736 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.736 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.736 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.736 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.736 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.736 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.737 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.737 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.737 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.737 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.737 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.737 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.737 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.737 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.738 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.738 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.738 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.738 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.738 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.738 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.738 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.739 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.739 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.739 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.739 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.739 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.739 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.739 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.739 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.739 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.740 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.740 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.740 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.740 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.740 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.740 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.740 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.740 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.741 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.741 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.741 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-06-10T00:43:41.741 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-06-10T00:43:41.741 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.741 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.741 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.741 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.742 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.742 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.742 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.742 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.742 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.742 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.742 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.742 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.743 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-06-10T00:43:41.743 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-06-10T00:43:41.743 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.743 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.743 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.743 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.743 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.743 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.744 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.744 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.744 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.744 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.744 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.744 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.744 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.744 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.744 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.745 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-06-10T00:43:41.745 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.745 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.745 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.745 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.745 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.745 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.745 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.746 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.746 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.746 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.746 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.746 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.746 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.746 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.746 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.747 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.747 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.747 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.747 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.747 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.747 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.747 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.748 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.748 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.748 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.748 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.748 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.748 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.748 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.748 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.749 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.749 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.749 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.749 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.749 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.749 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.749 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.749 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.749 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.750 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.750 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.750 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.750 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.750 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.750 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.750 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.750 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.751 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.751 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.751 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.751 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.751 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.751 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.751 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.751 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.752 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.752 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.752 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.752 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.752 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.752 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.752 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.752 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.753 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.753 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.753 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.753 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.753 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.753 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.753 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.753 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.754 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.754 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.754 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.754 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.754 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.754 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-06-10T00:43:41.754 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.754 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.755 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.755 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.755 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.755 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.755 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.755 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.755 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.755 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.755 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.756 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.756 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.756 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.756 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.756 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.756 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.756 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.757 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.757 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.757 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.757 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.757 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.757 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.757 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.757 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.758 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.758 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.758 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.758 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.758 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.758 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.758 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.758 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.759 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.759 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.759 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.759 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.759 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.759 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.759 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.759 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.760 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.760 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.760 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.760 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.760 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.760 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.760 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.760 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.761 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.761 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.761 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.761 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.761 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.761 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.761 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.761 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.762 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.762 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.762 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.762 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.762 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.762 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.762 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.762 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.763 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.763 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.763 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.763 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.763 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.763 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.763 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-06-10T00:43:41.764 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-06-10T00:43:41.764 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.764 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.764 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.764 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-06-10T00:43:41.764 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.764 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.764 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.765 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-10T00:43:41.765 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.765 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.765 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.765 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.765 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.765 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.766 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-06-10T00:43:41.766 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.766 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.766 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.766 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.766 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.766 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.766 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.767 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.767 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.767 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.767 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.767 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.767 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.767 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.767 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.768 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.768 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.768 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.768 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.768 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.768 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.768 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.768 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.769 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.769 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.769 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.769 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.769 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.769 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.769 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.769 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.770 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.770 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.770 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.770 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.770 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.770 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.770 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.770 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.771 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.771 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.771 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.771 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.771 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.771 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.771 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.772 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.772 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.772 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.772 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.772 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.772 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.772 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.773 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.773 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.773 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.773 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.773 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.773 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.773 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.773 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.774 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.774 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.774 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.774 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.774 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.774 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.774 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.774 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.775 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.775 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.775 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.775 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.775 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-06-10T00:43:41.775 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-06-10T00:43:41.775 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.775 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.775 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.776 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.776 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.776 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.776 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.776 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.776 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.776 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.776 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.777 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.777 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-06-10T00:43:41.777 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-06-10T00:43:41.777 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.777 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.777 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.777 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-06-10T00:43:41.777 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.778 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.778 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.778 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.778 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.778 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.778 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.778 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.778 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-06-10T00:43:41.779 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-06-10T00:43:41.779 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.779 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.779 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.779 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.779 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.779 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.779 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.779 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.780 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.780 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.780 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.780 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.780 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-06-10T00:43:41.780 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-06-10T00:43:41.780 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.780 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.781 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.781 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.781 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.781 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.781 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.781 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.781 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.781 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.782 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.782 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-06-10T00:43:41.782 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-06-10T00:43:41.782 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:41.782 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.782 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.783 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.783 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.783 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.783 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.783 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.783 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.783 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.783 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.784 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-06-10T00:43:41.784 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-06-10T00:43:41.784 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.784 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.784 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.784 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.784 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.785 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.785 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.785 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.785 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.785 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.785 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.785 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-06-10T00:43:41.785 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-06-10T00:43:41.786 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.786 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.786 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.786 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.786 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.786 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.786 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.786 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.787 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.787 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.787 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.787 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.787 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-06-10T00:43:41.787 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-06-10T00:43:41.787 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.787 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.788 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.788 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.788 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.788 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.788 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.788 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.788 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.788 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.789 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.789 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-06-10T00:43:41.789 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-06-10T00:43:41.789 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.789 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.789 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.790 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.790 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-06-10T00:43:41.790 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-06-10T00:43:41.790 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.790 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.790 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.790 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.790 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.791 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-06-10T00:43:41.791 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-06-10T00:43:41.791 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.791 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.791 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.791 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.791 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.791 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.792 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.792 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.792 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.792 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.792 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.792 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.792 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-06-10T00:43:41.792 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-06-10T00:43:41.793 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.793 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.793 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.793 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.793 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.793 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.793 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.793 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.794 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.794 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.794 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.794 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.794 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-06-10T00:43:41.794 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-06-10T00:43:41.794 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.795 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.795 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.795 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.795 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.795 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.795 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.795 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.795 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.796 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.796 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.796 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.796 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.796 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.796 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.796 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-06-10T00:43:41.796 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.797 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.797 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.797 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-06-10T00:43:41.797 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-06-10T00:43:41.797 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.797 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.797 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.797 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.798 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.798 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.798 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.798 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-06-10T00:43:41.798 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.798 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.798 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.799 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.799 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-06-10T00:43:41.799 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-06-10T00:43:41.799 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.799 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.799 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.799 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.799 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.799 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.800 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.800 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-06-10T00:43:41.800 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.800 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.800 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.800 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.800 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.800 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.801 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.801 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.801 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.801 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.801 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.801 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.801 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.801 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.802 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.802 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.802 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.802 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.802 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.802 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.802 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.802 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.803 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.803 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.803 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.803 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.803 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.803 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.803 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.803 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.803 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.804 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.804 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.804 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.804 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.804 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.804 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.804 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.804 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.804 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.805 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.805 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.805 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.805 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.805 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.805 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.805 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.805 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.806 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.806 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.806 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.806 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.806 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.806 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.806 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.806 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.807 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.807 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.807 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.807 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.807 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.807 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.807 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.807 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.808 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.808 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.808 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.808 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.808 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.808 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.808 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.808 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.809 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.809 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.809 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.809 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.809 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.809 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-06-10T00:43:41.809 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.809 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.810 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.810 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.810 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.810 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.810 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.810 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.810 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.810 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.811 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.811 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.811 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.811 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.811 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.811 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.811 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.811 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.812 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.812 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.812 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.812 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.812 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.812 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.812 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.812 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.813 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.813 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.813 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.813 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.813 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.813 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.813 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.813 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.814 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.814 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.814 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.814 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.814 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.814 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.814 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.814 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.815 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.815 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.815 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.815 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.815 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.815 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.815 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.815 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.816 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.816 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.816 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.816 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.816 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.816 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.816 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.816 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.817 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.817 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.817 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.817 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.817 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.817 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.817 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.817 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.818 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.818 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.818 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.818 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.818 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.818 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.818 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.818 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.819 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.819 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.819 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-06-10T00:43:41.819 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.819 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.819 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.819 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.819 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.820 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.820 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.820 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.820 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.820 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.820 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.820 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.820 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.821 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.821 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.821 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.821 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.821 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.821 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.821 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.821 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.821 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.822 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.822 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.822 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.822 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.822 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.822 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.822 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.822 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.823 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.823 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.823 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.823 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.823 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.823 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.823 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.823 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.824 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.824 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.824 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.824 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.824 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.824 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.824 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.824 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.824 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.825 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.825 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.825 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.825 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.825 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.825 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.825 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.826 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.826 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.826 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.826 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.826 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.826 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.826 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.826 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.827 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.827 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.827 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.827 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.827 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.827 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.827 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.827 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.828 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.828 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.828 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.828 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.828 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.828 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.828 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-06-10T00:43:41.828 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.829 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.829 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.829 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-06-10T00:43:41.829 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-06-10T00:43:41.829 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.829 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.829 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.829 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-10T00:43:41.830 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.830 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.830 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.830 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.830 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.830 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.830 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.831 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.831 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:41.831 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:41.831 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.831 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.831 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.831 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-06-10T00:43:41.831 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.832 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.832 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.832 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.832 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.832 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.832 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.832 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.832 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.833 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.833 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.833 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.833 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.833 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.833 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.833 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.833 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.834 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.834 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.834 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.834 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.834 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.834 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.834 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.834 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.834 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.835 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.835 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.835 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.835 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.835 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.835 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.835 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.835 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.836 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.836 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.836 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.836 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.836 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.836 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.836 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.836 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.836 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.837 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.837 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.837 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.837 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.837 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.837 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.837 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.837 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.837 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.838 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.838 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.838 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.838 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.838 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.838 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.838 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.839 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.839 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.839 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.839 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.839 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.839 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.839 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.839 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.839 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.840 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.840 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.840 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.840 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.840 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.840 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.840 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.840 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.840 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.841 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.841 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.841 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.841 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.841 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-06-10T00:43:41.841 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.841 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.841 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.841 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-06-10T00:43:41.842 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-06-10T00:43:41.842 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.842 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.842 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.842 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.842 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.842 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.842 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.843 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-10T00:43:41.843 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.843 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.843 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.843 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.843 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-06-10T00:43:41.843 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-06-10T00:43:41.843 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.843 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.844 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.844 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.844 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.844 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.844 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.844 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-10T00:43:41.844 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.844 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.844 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.845 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.845 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-06-10T00:43:41.845 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-06-10T00:43:41.845 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.845 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.845 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.845 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.845 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.845 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.846 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.846 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-10T00:43:41.846 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.846 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.846 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.846 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.846 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-06-10T00:43:41.846 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-06-10T00:43:41.847 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.847 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.847 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.847 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.847 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.847 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.847 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.848 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-10T00:43:41.848 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.848 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.848 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.848 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.848 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-06-10T00:43:41.848 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-06-10T00:43:41.848 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.849 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.849 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.849 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.849 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.849 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.849 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.849 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-10T00:43:41.849 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.850 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.850 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.850 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.850 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-06-10T00:43:41.850 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-06-10T00:43:41.850 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.850 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.850 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.851 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.851 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.851 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.851 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.851 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.851 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.851 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.851 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.852 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.852 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-06-10T00:43:41.852 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-06-10T00:43:41.852 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.852 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.852 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.852 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.853 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.853 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.853 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.853 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.853 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.853 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.853 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.854 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.854 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-06-10T00:43:41.854 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-06-10T00:43:41.854 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.854 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.854 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.854 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-10T00:43:41.854 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.855 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.855 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.855 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.855 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.855 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.855 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.855 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.855 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-06-10T00:43:41.856 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-06-10T00:43:41.856 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.856 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.856 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.856 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.856 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.856 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.857 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.857 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.857 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.857 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.857 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.857 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.857 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:41.857 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:41.858 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.858 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.858 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.858 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-06-10T00:43:41.858 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-06-10T00:43:41.858 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.858 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.858 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.859 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.859 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.859 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.859 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.859 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-06-10T00:43:41.859 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-06-10T00:43:41.859 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.859 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.860 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.860 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-06-10T00:43:41.860 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.860 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.860 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.860 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.860 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.860 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.860 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.861 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.861 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-06-10T00:43:41.861 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-06-10T00:43:41.861 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.861 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.861 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.861 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.861 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.861 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.862 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.862 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.862 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.862 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.862 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.862 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.862 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-06-10T00:43:41.863 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-06-10T00:43:41.863 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.863 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.863 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.863 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.863 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.863 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.863 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.863 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.864 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.864 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.864 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.864 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.864 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.864 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.864 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.864 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.864 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.865 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.865 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.865 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.865 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.865 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.865 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.865 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.865 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.866 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.866 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.866 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.866 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.866 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.866 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.866 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.867 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.867 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.867 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.867 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.867 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.867 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.867 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.867 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.868 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.868 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.868 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.868 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.868 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.868 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.868 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.868 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.868 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.869 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.869 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.869 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.869 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.869 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.869 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.869 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.869 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.869 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.870 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.870 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.870 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.870 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.870 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.870 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.870 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.871 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.871 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.871 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.871 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.871 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.871 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.871 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.871 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.871 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.872 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.872 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.872 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.872 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.872 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.872 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.872 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.872 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.873 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-06-10T00:43:41.873 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-06-10T00:43:41.873 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.873 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.873 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.873 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.873 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.873 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.873 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.874 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.874 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.874 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.874 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.874 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.874 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-06-10T00:43:41.874 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-06-10T00:43:41.874 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.875 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.875 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.875 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.875 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.875 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.875 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.875 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.875 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.876 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.876 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.876 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.876 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-06-10T00:43:41.876 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-06-10T00:43:41.876 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.876 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.876 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.877 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-10T00:43:41.877 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.877 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.877 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.877 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.877 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.877 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.877 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.878 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.878 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.878 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.878 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.878 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-06-10T00:43:41.878 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.878 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.878 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.879 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.879 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.879 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.879 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.879 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.879 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.879 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.879 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.880 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.880 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.880 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.880 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.880 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.880 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.880 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.880 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.881 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.881 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.881 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.881 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.881 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.881 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.881 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.881 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.882 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.882 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.882 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.882 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.882 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.882 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.882 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.882 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.883 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.883 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.883 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.883 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.883 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.883 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.883 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.883 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.884 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.884 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.884 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.884 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.884 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.884 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.884 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.884 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.885 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.885 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.885 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.885 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.885 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.885 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.885 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.885 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.886 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.886 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.886 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.886 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.886 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.886 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.886 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.886 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.887 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.887 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.887 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.887 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.887 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.887 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.887 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.887 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.888 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.888 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-06-10T00:43:41.888 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.888 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.888 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.888 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.888 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.888 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.889 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.889 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.889 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.889 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.889 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.889 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.889 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.889 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.889 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.890 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.890 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.890 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.890 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.890 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.890 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.890 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.891 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.891 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.891 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.891 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.891 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.891 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.891 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.892 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.892 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.892 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.892 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.892 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.892 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.892 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.892 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.892 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.893 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.893 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.893 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.893 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.893 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.893 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.893 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.893 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.894 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.894 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.894 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.894 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.894 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.894 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.894 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.894 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.895 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.895 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.895 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.895 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.895 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.895 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.895 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.896 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.896 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.896 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.896 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.896 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.896 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.896 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.896 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.897 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.897 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.897 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.897 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-06-10T00:43:41.897 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-06-10T00:43:41.897 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.897 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.897 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.897 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-06-10T00:43:41.898 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.898 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.898 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.898 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-10T00:43:41.898 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.898 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.898 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.899 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.899 INFO:teuthology.orchestra.run.smithi062.stdout: "periodic_async_work": { 2024-06-10T00:43:41.899 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "periodic_async_work", 2024-06-10T00:43:41.899 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.899 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.899 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.899 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.899 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.899 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.900 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.900 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Periodically check for async work", 2024-06-10T00:43:41.900 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.900 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.900 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.900 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.900 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-06-10T00:43:41.901 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-06-10T00:43:41.901 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.901 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.901 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.901 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:41.901 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.901 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.901 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.901 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-10T00:43:41.902 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.902 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.902 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.902 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.902 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-06-10T00:43:41.902 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-06-10T00:43:41.902 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.903 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.903 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.903 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.903 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.903 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.903 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.903 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-10T00:43:41.903 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.903 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.904 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.904 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.904 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.904 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.904 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.904 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-06-10T00:43:41.904 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.904 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.904 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.905 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-06-10T00:43:41.905 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-06-10T00:43:41.905 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:41.905 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.905 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.905 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-06-10T00:43:41.905 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.905 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.906 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.906 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.906 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.906 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.906 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.906 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.906 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-06-10T00:43:41.906 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-06-10T00:43:41.907 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.907 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.907 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.907 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.907 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.907 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.907 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.908 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.908 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.908 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.908 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.908 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.908 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:41.908 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:41.908 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.908 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.909 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.909 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:41.909 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.909 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.909 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.909 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.909 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.909 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.910 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.910 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.910 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.910 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.910 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.910 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.910 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.910 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.911 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.911 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.911 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.911 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.911 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.911 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.911 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.911 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.912 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.912 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.912 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.912 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.912 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.912 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.912 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.913 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.913 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.913 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.913 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.913 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.913 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.913 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.914 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.914 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.914 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.914 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.914 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.914 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.914 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.914 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.915 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.915 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.915 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.915 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.915 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.915 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.915 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.915 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.916 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.916 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.916 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.916 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.916 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.916 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.916 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.917 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.917 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.917 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.917 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.917 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.917 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.917 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.917 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.918 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.918 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.918 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.918 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.918 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.918 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.918 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.918 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.919 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.919 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.919 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.919 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-06-10T00:43:41.919 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-06-10T00:43:41.919 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.919 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.919 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.920 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.920 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.920 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.920 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.920 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.920 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.920 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.921 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.921 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.921 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-06-10T00:43:41.921 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-06-10T00:43:41.921 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.921 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.921 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.921 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-06-10T00:43:41.922 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.922 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.922 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.922 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.922 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.922 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.922 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.922 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.923 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-06-10T00:43:41.923 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-06-10T00:43:41.923 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.923 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.923 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.923 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-10T00:43:41.923 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.923 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.924 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.924 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.924 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.924 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.924 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.924 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.924 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.924 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.925 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-06-10T00:43:41.925 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.925 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.925 INFO:teuthology.orchestra.run.smithi062.stdout: "gid": 24968, 2024-06-10T00:43:41.925 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "y", 2024-06-10T00:43:41.925 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_features": 4540701547738038271, 2024-06-10T00:43:41.925 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-06-10T00:43:41.925 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.926 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-06-10T00:43:41.926 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.926 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.926 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.926 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:41.926 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:41.926 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.927 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.927 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.927 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:41.927 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.927 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.927 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.927 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-06-10T00:43:41.928 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.928 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.928 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.928 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.928 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.928 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.928 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.928 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.928 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.929 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.929 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.929 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.929 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.929 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.929 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.929 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.929 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.930 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.930 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.930 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.930 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.930 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.930 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.930 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.930 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.931 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.931 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.931 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.931 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.931 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.931 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.931 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.931 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.931 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.932 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.932 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.932 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.932 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.932 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.932 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.932 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.933 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.933 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.933 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.933 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.933 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.933 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.933 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.933 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.933 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.934 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.934 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.934 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.934 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.934 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.934 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.934 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.934 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.935 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.935 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.935 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.935 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.935 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.935 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.935 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.935 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.936 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.936 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.936 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.936 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.936 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.936 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.936 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.937 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.937 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-06-10T00:43:41.937 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-06-10T00:43:41.937 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.937 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.937 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.937 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.937 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.938 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.938 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.938 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-06-10T00:43:41.938 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.938 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.938 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.938 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.939 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-06-10T00:43:41.939 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-06-10T00:43:41.939 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.939 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.939 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.939 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-06-10T00:43:41.939 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.939 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.940 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.940 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-06-10T00:43:41.940 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.940 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.940 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.940 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.940 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-06-10T00:43:41.940 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-06-10T00:43:41.941 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.941 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.941 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.941 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.941 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.941 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.941 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.941 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-06-10T00:43:41.941 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.942 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.942 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.942 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.942 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-06-10T00:43:41.942 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-06-10T00:43:41.942 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.942 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.942 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.943 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.943 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.943 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.943 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.943 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-06-10T00:43:41.943 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.943 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.943 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.944 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.944 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-06-10T00:43:41.944 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-06-10T00:43:41.944 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.944 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.944 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.944 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-06-10T00:43:41.944 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.944 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.945 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.945 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-06-10T00:43:41.945 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.945 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.945 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.945 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.945 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-06-10T00:43:41.945 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-06-10T00:43:41.946 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.946 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.946 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.946 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.946 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.946 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.946 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.946 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-06-10T00:43:41.947 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.947 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.947 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.947 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.947 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-06-10T00:43:41.947 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-06-10T00:43:41.947 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.947 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.948 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.948 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.948 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.948 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.948 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.948 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-10T00:43:41.948 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.948 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.949 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.949 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.949 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-06-10T00:43:41.949 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-06-10T00:43:41.949 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.949 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.949 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.949 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.949 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.950 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.950 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.950 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-06-10T00:43:41.950 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.950 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.950 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.950 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.951 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.951 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.951 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.951 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-06-10T00:43:41.951 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.951 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.951 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.951 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-06-10T00:43:41.952 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-06-10T00:43:41.952 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.952 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.952 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.952 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:41.952 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.952 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.952 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.952 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-06-10T00:43:41.953 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.953 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.953 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.953 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.953 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-06-10T00:43:41.953 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-06-10T00:43:41.953 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.953 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.953 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.954 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-06-10T00:43:41.954 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.954 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.954 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.954 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-06-10T00:43:41.954 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T00:43:41.954 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.955 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.955 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.955 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-06-10T00:43:41.955 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-06-10T00:43:41.955 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:41.955 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.955 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.956 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:41.956 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-06-10T00:43:41.956 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-06-10T00:43:41.956 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.956 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-10T00:43:41.956 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T00:43:41.956 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.956 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.957 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.957 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-06-10T00:43:41.957 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-06-10T00:43:41.957 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:41.957 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.957 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.957 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-06-10T00:43:41.957 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-06-10T00:43:41.958 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-06-10T00:43:41.958 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.958 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-10T00:43:41.958 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.958 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.958 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.958 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.959 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-06-10T00:43:41.959 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-06-10T00:43:41.959 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.959 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.959 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.959 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-06-10T00:43:41.959 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.959 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.959 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.960 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-10T00:43:41.960 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-10T00:43:41.960 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.960 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.960 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.960 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-06-10T00:43:41.960 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-06-10T00:43:41.961 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:41.961 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.961 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.961 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-06-10T00:43:41.961 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-06-10T00:43:41.961 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-06-10T00:43:41.961 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.961 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-06-10T00:43:41.961 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-10T00:43:41.962 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.962 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.962 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.962 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-06-10T00:43:41.962 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-06-10T00:43:41.962 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.962 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.962 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.963 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-06-10T00:43:41.963 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.963 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.963 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.963 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-06-10T00:43:41.963 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T00:43:41.963 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.963 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.964 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.964 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-06-10T00:43:41.964 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-06-10T00:43:41.964 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:41.964 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.964 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.964 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:41.964 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-06-10T00:43:41.965 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-06-10T00:43:41.965 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.965 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-10T00:43:41.965 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T00:43:41.965 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.965 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.965 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.965 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:41.965 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:41.966 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.966 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.966 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.966 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.966 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.966 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.966 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.966 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.967 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.967 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.969 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.969 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.969 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.969 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.969 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.970 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.970 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.970 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.970 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.970 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:41.970 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:41.970 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.971 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.971 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.971 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.971 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.971 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.971 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.971 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.971 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.972 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.972 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.972 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.972 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:41.972 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:41.972 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.972 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.973 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.973 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:41.973 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.973 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.973 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.973 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:41.973 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:41.973 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:41.974 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:41.974 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:41.974 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:41.974 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.974 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.974 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.974 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.974 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.975 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.975 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:41.975 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:41.975 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.975 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.975 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.975 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.975 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.976 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.976 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.976 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:41.976 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.976 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.976 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.976 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.977 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-06-10T00:43:41.977 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-06-10T00:43:41.977 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:41.977 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.977 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.977 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:41.977 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.977 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.978 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.978 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-10T00:43:41.978 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.978 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.978 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.978 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.978 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-06-10T00:43:41.978 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-06-10T00:43:41.979 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.979 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.979 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.979 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-06-10T00:43:41.979 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.979 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.979 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:41.979 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-06-10T00:43:41.980 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-06-10T00:43:41.980 INFO:teuthology.orchestra.run.smithi062.stdout: "read", 2024-06-10T00:43:41.980 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap", 2024-06-10T00:43:41.980 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap-read" 2024-06-10T00:43:41.980 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:41.980 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-06-10T00:43:41.980 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.980 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.980 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.981 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.981 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-06-10T00:43:41.981 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-06-10T00:43:41.981 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:41.981 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.981 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.981 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:41.981 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.982 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.982 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.982 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-10T00:43:41.982 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.982 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.982 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.982 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.982 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:41.983 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:41.983 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.983 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.983 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.983 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:41.983 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.983 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.983 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.983 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-10T00:43:41.984 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.984 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.984 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.984 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.984 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-06-10T00:43:41.984 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-06-10T00:43:41.984 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.985 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.985 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.985 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-06-10T00:43:41.985 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-06-10T00:43:41.985 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.985 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.985 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-10T00:43:41.985 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-10T00:43:41.986 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.986 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.986 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.986 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-06-10T00:43:41.986 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-06-10T00:43:41.986 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:41.986 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.986 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:41.987 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:41.987 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.987 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.987 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.987 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-10T00:43:41.987 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.987 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.987 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.988 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.988 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:41.988 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.988 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:41.988 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-06-10T00:43:41.988 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:41.988 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:41.988 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:41.989 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-06-10T00:43:41.989 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-06-10T00:43:41.989 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:41.989 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.989 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.989 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-06-10T00:43:41.989 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.989 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.990 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.990 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-10T00:43:41.990 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.990 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.990 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.990 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.990 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-06-10T00:43:41.991 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-06-10T00:43:41.991 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.991 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.991 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.991 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-06-10T00:43:41.991 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.991 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.991 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.992 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-10T00:43:41.992 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.992 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.992 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.992 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.992 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-06-10T00:43:41.992 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-06-10T00:43:41.992 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:41.993 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.993 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.993 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-06-10T00:43:41.993 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.993 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.993 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.993 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-10T00:43:41.993 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.994 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.994 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.994 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.994 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-06-10T00:43:41.994 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-06-10T00:43:41.994 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:41.994 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.994 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.995 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:41.995 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.995 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.995 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.995 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-10T00:43:41.995 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-10T00:43:41.995 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.995 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.996 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.996 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-06-10T00:43:41.996 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-06-10T00:43:41.996 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:41.996 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.996 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.996 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:41.997 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.997 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.997 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.997 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-10T00:43:41.997 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.997 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.997 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.997 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.997 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-06-10T00:43:41.998 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-06-10T00:43:41.998 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:41.998 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:41.998 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:41.998 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-06-10T00:43:41.998 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:41.998 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:41.998 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:41.999 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-10T00:43:41.999 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:41.999 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:41.999 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:41.999 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:41.999 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm_log_destination": { 2024-06-10T00:43:41.999 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm_log_destination", 2024-06-10T00:43:41.999 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.000 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.000 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.000 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.000 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.000 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.000 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.000 INFO:teuthology.orchestra.run.smithi062.stdout: "file", 2024-06-10T00:43:42.000 INFO:teuthology.orchestra.run.smithi062.stdout: "file,syslog", 2024-06-10T00:43:42.001 INFO:teuthology.orchestra.run.smithi062.stdout: "syslog" 2024-06-10T00:43:42.001 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.001 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-10T00:43:42.001 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.001 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.001 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.001 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.002 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-06-10T00:43:42.002 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-06-10T00:43:42.002 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.002 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.002 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.002 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.002 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.003 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.003 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.003 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-10T00:43:42.003 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.003 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.003 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.003 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.003 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-06-10T00:43:42.003 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-06-10T00:43:42.004 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.004 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.004 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.004 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.004 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.004 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.004 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.004 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-10T00:43:42.004 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.005 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.005 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.005 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.005 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-06-10T00:43:42.005 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-06-10T00:43:42.005 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.005 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.005 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.006 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.006 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.006 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.006 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.006 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-10T00:43:42.006 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.006 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.007 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.007 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.007 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-06-10T00:43:42.007 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-06-10T00:43:42.007 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.007 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.007 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.007 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-10T00:43:42.007 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.008 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.008 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.008 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-06-10T00:43:42.008 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.008 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.008 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.008 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.008 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-06-10T00:43:42.008 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-06-10T00:43:42.009 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.009 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.009 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.009 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-10T00:43:42.009 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.009 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.009 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.009 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-06-10T00:43:42.010 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.010 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.010 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.010 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.010 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-06-10T00:43:42.010 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-06-10T00:43:42.010 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.011 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.011 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.011 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-10T00:43:42.011 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.011 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.011 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.011 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-06-10T00:43:42.011 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.011 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.012 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.012 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.012 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-06-10T00:43:42.012 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-06-10T00:43:42.012 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.012 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.012 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.013 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-10T00:43:42.013 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.013 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.013 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.013 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-06-10T00:43:42.013 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.013 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.013 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.013 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.014 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-06-10T00:43:42.014 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-06-10T00:43:42.014 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.014 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.014 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.014 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-10T00:43:42.014 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.014 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.015 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.015 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-06-10T00:43:42.015 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.015 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.015 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.015 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.015 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-06-10T00:43:42.015 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-06-10T00:43:42.015 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.016 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.016 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.016 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-10T00:43:42.016 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.016 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.016 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.016 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-06-10T00:43:42.017 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.017 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.017 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.017 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.017 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-06-10T00:43:42.017 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-06-10T00:43:42.017 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.017 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.018 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.018 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-10T00:43:42.018 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.018 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.018 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.018 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-06-10T00:43:42.018 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.018 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.019 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.019 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.019 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-06-10T00:43:42.019 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-06-10T00:43:42.019 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.019 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.019 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.019 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-10T00:43:42.019 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.020 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.020 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.020 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-06-10T00:43:42.020 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.020 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.020 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.020 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.020 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-06-10T00:43:42.021 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-06-10T00:43:42.021 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.021 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.021 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.021 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-10T00:43:42.021 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.021 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.021 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.022 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-06-10T00:43:42.022 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.022 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.022 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.022 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.022 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-06-10T00:43:42.022 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-06-10T00:43:42.022 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.023 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.023 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.023 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-10T00:43:42.023 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.023 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.023 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.023 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-06-10T00:43:42.023 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.023 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.024 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.024 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.024 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-06-10T00:43:42.024 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-06-10T00:43:42.024 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.024 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.024 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.024 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-10T00:43:42.024 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.025 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.025 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.025 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-06-10T00:43:42.025 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.025 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.025 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.025 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.025 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-06-10T00:43:42.025 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-06-10T00:43:42.025 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.026 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.026 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.026 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-10T00:43:42.026 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.026 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.026 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.026 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-06-10T00:43:42.027 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.027 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.027 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.027 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.027 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-06-10T00:43:42.027 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-06-10T00:43:42.027 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.027 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.027 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.028 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-10T00:43:42.028 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.028 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.028 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.028 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-06-10T00:43:42.028 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.028 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.028 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.028 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.029 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-06-10T00:43:42.029 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-06-10T00:43:42.029 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.029 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.029 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.029 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-10T00:43:42.029 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.029 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.029 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.030 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-06-10T00:43:42.030 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.030 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.030 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.030 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.030 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_samba": { 2024-06-10T00:43:42.030 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_samba", 2024-06-10T00:43:42.030 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.031 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.031 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.031 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-10T00:43:42.031 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.031 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.031 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.031 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Samba/SMB container image", 2024-06-10T00:43:42.031 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.031 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.032 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.032 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.032 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-06-10T00:43:42.032 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-06-10T00:43:42.032 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.032 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.032 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.032 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-10T00:43:42.033 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.033 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.033 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.033 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-06-10T00:43:42.033 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.033 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.033 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.033 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.033 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-06-10T00:43:42.034 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-06-10T00:43:42.034 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.034 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.034 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.034 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.034 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.034 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.034 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.035 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-06-10T00:43:42.035 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.035 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.035 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.035 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.035 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-06-10T00:43:42.035 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-06-10T00:43:42.035 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.036 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.036 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.036 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:42.036 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.036 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.036 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.036 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-10T00:43:42.037 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.037 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.037 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.037 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.037 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-06-10T00:43:42.037 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-06-10T00:43:42.037 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.037 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.038 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.038 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-06-10T00:43:42.038 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.038 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.038 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.038 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-10T00:43:42.038 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.039 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.039 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.039 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.039 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-06-10T00:43:42.039 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-06-10T00:43:42.039 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.039 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.039 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.039 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-06-10T00:43:42.040 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.040 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.040 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.040 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-10T00:43:42.040 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.040 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.040 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.040 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.041 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-06-10T00:43:42.041 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-06-10T00:43:42.041 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.041 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.041 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.041 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-06-10T00:43:42.041 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.041 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.041 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.042 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-06-10T00:43:42.042 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.042 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.042 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.042 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.042 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-06-10T00:43:42.042 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-06-10T00:43:42.042 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.043 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.043 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.043 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.043 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.043 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.043 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.043 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-10T00:43:42.043 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.043 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.044 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.044 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.044 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-06-10T00:43:42.044 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-06-10T00:43:42.044 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.044 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.044 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.044 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:42.044 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.045 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.045 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.045 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-06-10T00:43:42.045 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.045 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.045 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.045 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.045 INFO:teuthology.orchestra.run.smithi062.stdout: "grafana_dashboards_path": { 2024-06-10T00:43:42.046 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "grafana_dashboards_path", 2024-06-10T00:43:42.046 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.046 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.046 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.046 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-10T00:43:42.046 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.046 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.046 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.047 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-10T00:43:42.047 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.047 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.047 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.047 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.047 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-06-10T00:43:42.047 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-06-10T00:43:42.047 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.048 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.048 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.048 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:42.048 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.048 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.048 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.048 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-06-10T00:43:42.048 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.048 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.049 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.049 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.049 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-06-10T00:43:42.049 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-06-10T00:43:42.049 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.049 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.049 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.049 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.049 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.050 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.050 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.050 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-10T00:43:42.050 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.050 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.050 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.050 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-06-10T00:43:42.051 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-06-10T00:43:42.051 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.051 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.051 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.051 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.051 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.051 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.051 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.052 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-10T00:43:42.052 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.052 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.052 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.052 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.052 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.052 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.052 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.052 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.052 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.053 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.053 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.053 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.053 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.053 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.053 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.053 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.054 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.054 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.054 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.054 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.054 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.054 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.054 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.054 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.055 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.055 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-06-10T00:43:42.055 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-06-10T00:43:42.055 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.055 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.055 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.055 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.055 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.056 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.056 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.056 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-10T00:43:42.056 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.056 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.056 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.056 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.057 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-10T00:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.058 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.059 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.060 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.061 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.061 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.061 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.061 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.061 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.061 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.061 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.061 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.062 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.062 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.062 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.062 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.062 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.062 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.062 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.062 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.063 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.063 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-10T00:43:42.063 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-10T00:43:42.063 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.063 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.063 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.063 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.064 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.064 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.064 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.064 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-10T00:43:42.064 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.064 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.064 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.064 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.065 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-10T00:43:42.065 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-10T00:43:42.065 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.065 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.065 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.065 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-06-10T00:43:42.065 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.065 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.066 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.066 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-10T00:43:42.066 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.066 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.066 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.066 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.066 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-06-10T00:43:42.066 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-06-10T00:43:42.067 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.067 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.067 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.067 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:42.067 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.067 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.067 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.067 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-06-10T00:43:42.068 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.068 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.068 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.068 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.068 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-06-10T00:43:42.068 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-06-10T00:43:42.068 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.069 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.069 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.069 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:42.069 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.069 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.069 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.069 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-10T00:43:42.069 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.069 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.070 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.070 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.070 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-06-10T00:43:42.070 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-06-10T00:43:42.070 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.070 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.070 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.070 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.071 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.071 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.071 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.071 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-06-10T00:43:42.071 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.071 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.071 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.071 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.072 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-06-10T00:43:42.072 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-06-10T00:43:42.072 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.072 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.072 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.072 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-06-10T00:43:42.072 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.072 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.073 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.073 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-06-10T00:43:42.073 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-06-10T00:43:42.073 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.073 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-06-10T00:43:42.073 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.073 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.073 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.073 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.074 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-06-10T00:43:42.074 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-06-10T00:43:42.074 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.074 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.074 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.074 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-06-10T00:43:42.074 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.074 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.075 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.075 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-10T00:43:42.075 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.075 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.075 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.075 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.075 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-06-10T00:43:42.075 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-06-10T00:43:42.076 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.076 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.076 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.076 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-10T00:43:42.076 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.076 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.076 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.076 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-10T00:43:42.077 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.077 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.077 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.077 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.077 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-06-10T00:43:42.077 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-06-10T00:43:42.077 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.077 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.077 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.078 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.078 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.078 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.078 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.078 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-10T00:43:42.078 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.079 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.079 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.079 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.079 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-06-10T00:43:42.079 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-06-10T00:43:42.079 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.079 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.080 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.080 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.080 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.080 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.080 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.080 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-10T00:43:42.080 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.080 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.080 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.081 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.081 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-06-10T00:43:42.081 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-06-10T00:43:42.081 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.081 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.081 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.081 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.081 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.082 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.082 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.082 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-10T00:43:42.082 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.082 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.082 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.082 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.082 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-06-10T00:43:42.083 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-06-10T00:43:42.083 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.083 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.083 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.083 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.083 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.083 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.083 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.084 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-10T00:43:42.084 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.084 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.084 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.084 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.084 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-06-10T00:43:42.084 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-06-10T00:43:42.084 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.085 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.085 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.085 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.085 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.085 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.085 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.085 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-10T00:43:42.085 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.086 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.086 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.086 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.086 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-06-10T00:43:42.086 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-06-10T00:43:42.086 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.086 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.086 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.087 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-06-10T00:43:42.087 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.087 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.087 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.087 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-06-10T00:43:42.087 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.087 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.087 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.087 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.088 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-06-10T00:43:42.088 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-06-10T00:43:42.088 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.088 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.088 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.088 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.088 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.088 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.089 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.089 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-10T00:43:42.089 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.089 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.089 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.089 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.089 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-06-10T00:43:42.089 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-06-10T00:43:42.090 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.090 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.090 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.090 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.090 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.090 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.090 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.090 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-10T00:43:42.091 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.091 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.091 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.091 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.091 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-06-10T00:43:42.091 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-06-10T00:43:42.091 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.091 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.091 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.092 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.092 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.092 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.092 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.092 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-10T00:43:42.092 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.092 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.092 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.093 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.093 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-06-10T00:43:42.093 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-06-10T00:43:42.093 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.093 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.093 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.093 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.093 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.094 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.094 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.094 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-06-10T00:43:42.094 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.094 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.094 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.094 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.094 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-06-10T00:43:42.095 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-06-10T00:43:42.095 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.095 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.095 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.095 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.095 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.095 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.096 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.096 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-10T00:43:42.096 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.096 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.096 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.096 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.096 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-06-10T00:43:42.096 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-06-10T00:43:42.096 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.097 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.097 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.097 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.097 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.097 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.097 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.097 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-10T00:43:42.097 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.098 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.098 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.098 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.098 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.098 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.098 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.098 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-06-10T00:43:42.098 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.099 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.099 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.099 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.099 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.099 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.099 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.099 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.099 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.100 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.100 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.100 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.100 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.100 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.100 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.100 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.101 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.101 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.101 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.101 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.101 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.101 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.101 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.101 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.101 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.102 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.102 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.102 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.102 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.102 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.102 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.102 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.102 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.103 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.103 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.103 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.103 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.103 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.103 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.103 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.103 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.104 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.104 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.104 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.104 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.104 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.104 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.104 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.104 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.105 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.105 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.105 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.105 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.105 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.105 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.105 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.105 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.106 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.106 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.106 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.106 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.106 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.106 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.106 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.106 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.107 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.107 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.107 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.107 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.107 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.107 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.107 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.108 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.108 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-06-10T00:43:42.108 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-06-10T00:43:42.108 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.108 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.108 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.108 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-06-10T00:43:42.108 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.109 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.109 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.109 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-10T00:43:42.109 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.109 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.109 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.109 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.109 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-06-10T00:43:42.110 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-06-10T00:43:42.110 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.110 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.110 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.110 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-06-10T00:43:42.110 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.110 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.110 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.111 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-10T00:43:42.111 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.111 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.111 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.111 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.111 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.111 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.111 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.112 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-06-10T00:43:42.112 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.112 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.112 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.112 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-10T00:43:42.112 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-10T00:43:42.112 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.112 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.112 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.113 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:42.113 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.113 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.113 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.113 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.113 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.113 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.113 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.114 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.114 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-06-10T00:43:42.114 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-10T00:43:42.114 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.114 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.114 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.114 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.114 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.115 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.115 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.115 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.115 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.115 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.115 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.115 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.115 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-10T00:43:42.115 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-10T00:43:42.116 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.116 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.116 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.116 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.116 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.116 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.116 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.116 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.117 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.117 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.117 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.117 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.117 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-06-10T00:43:42.117 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-06-10T00:43:42.117 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.117 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.118 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.118 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.118 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.118 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.118 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.118 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.118 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.118 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.119 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.119 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.119 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-10T00:43:42.119 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-10T00:43:42.119 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.119 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.119 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.119 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.120 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.120 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.120 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.120 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.120 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.120 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.120 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-06-10T00:43:42.121 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-10T00:43:42.121 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.121 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.121 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.121 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.121 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.121 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.122 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.122 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.122 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.122 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.122 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.122 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.122 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-10T00:43:42.122 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-10T00:43:42.123 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.123 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.123 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.123 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.123 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.123 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.123 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.123 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.124 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.124 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.124 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.124 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.124 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-10T00:43:42.124 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-10T00:43:42.124 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.124 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.125 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.125 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.125 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.125 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.125 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.125 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.125 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.125 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.126 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.126 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.126 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-10T00:43:42.126 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-10T00:43:42.126 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.126 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.126 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.126 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.127 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.127 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.127 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.127 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.127 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.127 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.127 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.127 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.128 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-10T00:43:42.128 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-10T00:43:42.128 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.128 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.128 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.128 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.128 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.128 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.129 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.129 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.129 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.129 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.129 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.129 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.129 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-10T00:43:42.129 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-10T00:43:42.130 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.130 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.130 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.130 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.130 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.130 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.130 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.131 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.131 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.131 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.131 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.131 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.131 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-10T00:43:42.131 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-10T00:43:42.131 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.132 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.132 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.132 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.132 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.132 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.132 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.132 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.132 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.133 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.133 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.133 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.133 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-10T00:43:42.133 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-10T00:43:42.133 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.133 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.133 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.134 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.134 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.134 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.134 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.134 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.134 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.134 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.134 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.135 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.135 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-10T00:43:42.135 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-10T00:43:42.135 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.135 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.135 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.135 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.135 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.136 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.136 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.136 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.136 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.136 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.136 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.136 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.136 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-06-10T00:43:42.137 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-10T00:43:42.137 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.137 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.137 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.137 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-06-10T00:43:42.137 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.137 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.137 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.138 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.138 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.138 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.138 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.138 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.138 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-10T00:43:42.138 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-10T00:43:42.139 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.139 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.139 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.139 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.139 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.139 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.139 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.139 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.140 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.140 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.140 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.140 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.140 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-06-10T00:43:42.140 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-06-10T00:43:42.140 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.140 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.141 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.141 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.141 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.141 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.141 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.141 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.141 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.142 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.142 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.142 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.142 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-06-10T00:43:42.142 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-10T00:43:42.142 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.142 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.142 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.143 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-06-10T00:43:42.143 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.143 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.143 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.143 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.143 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.143 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.143 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.144 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.144 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-10T00:43:42.144 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-10T00:43:42.144 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.144 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.144 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.144 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.144 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.145 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.145 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.145 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.145 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.145 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.145 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.145 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.145 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-10T00:43:42.146 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-10T00:43:42.146 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.146 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.146 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.146 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.146 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.146 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.146 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.147 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.147 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.147 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.147 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.147 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.147 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-10T00:43:42.147 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-10T00:43:42.148 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.148 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.148 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.148 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.148 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.148 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.148 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.148 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.149 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.149 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.149 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.149 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.149 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-10T00:43:42.149 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-10T00:43:42.149 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.149 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.150 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.150 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.150 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.150 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.150 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.150 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.150 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.150 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.151 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.151 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.151 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-06-10T00:43:42.151 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-10T00:43:42.151 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.151 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.151 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.151 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.152 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.152 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.152 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.152 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.152 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.152 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.152 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.152 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.153 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-10T00:43:42.153 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-10T00:43:42.153 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.153 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.153 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.153 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.153 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.154 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.154 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.154 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.154 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.154 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.154 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.154 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.154 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-10T00:43:42.155 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-10T00:43:42.155 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.155 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.155 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.155 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.155 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.155 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.155 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.156 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.156 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.156 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.156 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.156 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.156 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-10T00:43:42.156 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-10T00:43:42.156 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.157 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.157 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.157 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.157 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.157 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.157 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.157 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.157 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.158 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.158 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.158 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.158 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-10T00:43:42.158 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-10T00:43:42.158 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.158 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.158 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.159 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.159 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.159 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.159 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.159 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.159 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.159 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.159 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.160 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.160 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-10T00:43:42.160 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-10T00:43:42.160 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.160 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.160 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.160 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.160 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.161 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.161 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.161 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.161 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.161 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.161 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.161 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.161 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-10T00:43:42.162 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-10T00:43:42.162 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.162 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.162 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.162 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.162 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.162 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.162 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.163 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.163 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.163 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.163 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.163 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.163 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-10T00:43:42.163 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-10T00:43:42.163 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.164 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.164 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.164 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.164 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.164 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.164 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.164 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.164 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.165 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.165 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.165 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.165 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-10T00:43:42.165 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-10T00:43:42.165 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.165 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.165 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.166 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.166 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.166 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.166 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.166 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.166 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.166 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.167 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.167 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.167 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-06-10T00:43:42.167 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-10T00:43:42.167 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.167 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.167 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.167 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.168 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.168 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.168 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.168 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.168 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.168 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.168 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.168 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.168 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-10T00:43:42.169 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-10T00:43:42.169 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.169 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.169 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.169 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-10T00:43:42.169 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.169 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.169 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.170 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.170 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.170 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.170 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.170 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.170 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-10T00:43:42.171 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-10T00:43:42.171 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.171 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.171 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.171 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:42.171 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.171 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.171 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.172 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.172 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.172 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.172 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.172 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.172 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-10T00:43:42.172 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-10T00:43:42.172 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.173 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.173 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.173 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-06-10T00:43:42.173 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.173 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.173 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.173 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.173 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.174 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.174 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.174 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.174 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-10T00:43:42.174 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-10T00:43:42.174 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.174 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-06-10T00:43:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-06-10T00:43:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-10T00:43:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.177 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.177 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.177 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.177 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.177 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.177 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.177 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.177 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.177 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-10T00:43:42.178 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-10T00:43:42.178 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.178 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.178 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.178 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-06-10T00:43:42.178 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.178 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.178 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.179 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.179 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.179 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.179 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.179 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.179 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-06-10T00:43:42.179 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-10T00:43:42.179 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.179 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.180 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.180 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.180 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.180 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.180 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.180 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.180 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.180 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.181 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.181 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.181 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-06-10T00:43:42.181 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-10T00:43:42.181 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.181 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.181 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.181 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.182 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.182 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.182 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.182 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.182 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.182 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.182 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.182 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.183 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-10T00:43:42.183 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-10T00:43:42.183 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.183 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.183 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.183 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:42.183 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.183 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.183 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.184 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.184 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.184 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.184 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.184 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.184 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-10T00:43:42.184 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-10T00:43:42.184 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.184 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.185 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.185 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:42.185 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.185 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.185 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.185 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.185 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.185 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.186 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.186 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.186 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-10T00:43:42.186 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-10T00:43:42.186 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.186 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.186 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.187 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-06-10T00:43:42.187 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.187 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.187 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.187 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.187 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.187 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.187 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.187 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.188 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-06-10T00:43:42.188 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-06-10T00:43:42.188 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.188 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.188 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.188 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.188 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.188 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.189 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.189 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.189 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.189 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.189 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.189 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.189 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-06-10T00:43:42.189 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-06-10T00:43:42.190 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.190 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.190 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.190 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.190 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.190 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.190 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.191 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.191 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.191 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.191 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.191 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.191 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-06-10T00:43:42.191 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-06-10T00:43:42.191 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.192 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.192 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.192 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.192 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.192 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.192 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.192 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-06-10T00:43:42.192 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.193 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.193 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.193 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.193 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-06-10T00:43:42.193 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-06-10T00:43:42.193 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.193 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.193 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.194 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-06-10T00:43:42.194 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.194 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.194 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.194 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.194 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.194 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.195 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.195 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.195 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-06-10T00:43:42.195 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-06-10T00:43:42.195 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.195 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.195 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.195 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.196 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.196 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.196 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.196 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.196 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.196 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.196 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.196 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.196 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.197 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.197 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.197 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.197 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.197 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.197 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.197 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.198 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.198 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.198 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.198 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.198 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.198 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.198 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.198 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.199 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.199 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.199 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.199 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.199 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.199 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.199 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.199 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.199 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.200 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.200 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.200 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.200 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.200 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.200 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.200 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.200 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.201 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.201 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.201 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.201 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.201 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.201 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.201 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.201 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.202 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.202 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.202 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.202 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.202 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.202 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.202 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.202 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.203 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.203 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.203 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.203 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.203 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.203 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.203 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.203 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.203 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.204 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.204 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.204 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.204 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.204 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.204 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.204 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.204 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.204 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.205 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.205 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.205 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.205 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-06-10T00:43:42.205 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-06-10T00:43:42.205 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.205 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.205 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.206 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.206 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.206 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.206 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.206 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-06-10T00:43:42.206 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.206 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.206 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.207 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.207 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-06-10T00:43:42.207 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-06-10T00:43:42.207 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.207 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.207 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.207 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.207 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.207 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.208 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.208 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.208 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.208 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.208 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.208 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.208 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-06-10T00:43:42.208 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-06-10T00:43:42.208 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.209 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.209 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.209 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-06-10T00:43:42.209 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.209 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.209 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.209 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.209 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.209 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.210 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.210 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.210 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-06-10T00:43:42.210 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-06-10T00:43:42.210 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.210 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.210 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.210 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-06-10T00:43:42.211 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.211 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.211 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.211 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.211 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.211 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.211 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.211 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.212 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-06-10T00:43:42.212 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-06-10T00:43:42.212 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.212 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.212 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.212 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.212 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.212 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.213 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.213 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.213 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.213 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.213 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.213 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.213 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-06-10T00:43:42.213 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-06-10T00:43:42.214 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.214 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.214 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.214 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-06-10T00:43:42.214 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.214 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.214 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.215 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.215 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.215 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.215 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.215 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.215 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-06-10T00:43:42.215 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-06-10T00:43:42.215 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.216 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.216 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.216 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-06-10T00:43:42.216 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.216 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.216 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.216 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.217 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-06-10T00:43:42.217 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.217 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.217 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.217 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.217 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.217 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.217 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-06-10T00:43:42.218 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-06-10T00:43:42.218 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.218 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.218 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.218 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-06-10T00:43:42.218 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-06-10T00:43:42.218 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-06-10T00:43:42.218 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.219 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.219 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.219 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.219 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.219 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.219 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-06-10T00:43:42.219 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-06-10T00:43:42.219 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.220 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.220 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.220 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.220 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.220 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.220 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.220 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.220 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.220 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.221 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.221 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.221 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.221 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.221 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.221 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-06-10T00:43:42.221 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.221 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.221 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.222 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-06-10T00:43:42.222 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-06-10T00:43:42.222 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.222 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.222 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.222 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.222 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.222 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.223 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.223 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-06-10T00:43:42.223 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.223 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.223 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.223 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.223 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.223 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.224 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.224 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.224 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.224 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.224 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.224 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.224 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.224 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.225 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.225 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.225 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.225 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.225 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.225 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.225 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.225 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.226 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.226 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.226 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.226 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.226 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.226 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.226 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.227 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.227 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.227 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.227 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.227 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.227 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.227 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.227 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.228 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.228 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.228 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.228 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.228 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.228 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.228 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.228 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.229 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.229 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.229 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.229 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.229 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.229 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.229 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.229 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.229 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.230 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.230 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.230 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.230 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.230 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.230 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.230 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.230 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.231 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.231 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.231 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.231 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.231 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.231 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.231 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.231 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.232 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.232 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.232 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.232 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.232 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-06-10T00:43:42.232 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-06-10T00:43:42.232 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.232 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.233 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.233 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-06-10T00:43:42.233 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.233 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.233 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.233 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-10T00:43:42.233 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.233 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.234 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.234 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.234 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-06-10T00:43:42.234 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-06-10T00:43:42.234 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.234 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.234 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.234 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-06-10T00:43:42.235 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.235 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.235 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.235 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-10T00:43:42.235 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.235 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.235 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.235 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.235 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-06-10T00:43:42.236 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-06-10T00:43:42.236 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.236 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.236 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.236 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-06-10T00:43:42.236 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.236 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.236 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.237 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-06-10T00:43:42.237 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.237 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.237 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.237 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.237 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-06-10T00:43:42.237 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-06-10T00:43:42.237 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.238 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.238 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.238 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-06-10T00:43:42.238 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.238 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.238 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.238 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-10T00:43:42.238 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.239 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.239 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.239 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.239 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-06-10T00:43:42.239 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-06-10T00:43:42.239 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.239 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.239 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.240 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.242 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.242 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.242 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.242 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-10T00:43:42.242 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.243 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.243 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.243 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.243 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:42.243 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:42.243 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.244 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.244 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.244 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:42.244 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.244 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.245 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.245 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-06-10T00:43:42.245 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.245 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.245 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.245 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.246 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-06-10T00:43:42.246 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-06-10T00:43:42.246 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.246 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.246 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.247 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-06-10T00:43:42.247 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.247 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.247 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.247 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-10T00:43:42.247 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.248 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.248 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.248 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.248 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.248 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.248 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.249 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-06-10T00:43:42.249 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.249 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.249 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.249 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.250 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.250 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.250 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.250 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.250 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.250 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.251 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.251 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.251 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.251 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.251 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.251 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.252 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.252 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.252 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.252 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.252 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.253 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.253 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.253 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.253 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.253 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.254 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.254 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.254 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.254 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.254 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.254 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.255 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.255 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.255 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.255 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.255 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.255 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.256 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.256 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.256 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.256 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.256 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.256 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.257 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.257 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.257 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.257 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.258 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.258 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.258 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.258 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.259 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.259 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.259 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.259 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.259 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.259 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.260 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.260 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.260 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.260 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.260 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.261 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.261 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.261 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.261 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.261 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.261 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.262 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.262 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.262 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.262 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.262 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-06-10T00:43:42.263 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-06-10T00:43:42.263 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.263 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.263 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.263 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-06-10T00:43:42.263 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.264 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.264 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.264 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.264 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.264 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.264 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.265 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.265 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-06-10T00:43:42.265 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-06-10T00:43:42.265 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.265 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.266 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.266 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-06-10T00:43:42.266 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.266 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.266 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.266 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.266 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.267 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.267 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.267 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.267 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:42.267 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:42.267 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.267 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.267 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.268 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:42.268 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.268 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.268 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.268 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.268 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.268 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.268 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.269 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.269 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.269 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.269 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.269 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-06-10T00:43:42.269 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-06-10T00:43:42.269 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-06-10T00:43:42.270 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.270 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-06-10T00:43:42.270 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-06-10T00:43:42.270 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.270 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.270 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.270 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-06-10T00:43:42.271 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.271 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.271 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.271 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-10T00:43:42.271 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.271 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.271 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.271 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.272 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-06-10T00:43:42.272 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-06-10T00:43:42.272 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.272 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.272 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.272 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-06-10T00:43:42.272 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.273 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.273 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.273 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-10T00:43:42.273 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.273 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.273 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.273 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.273 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-06-10T00:43:42.274 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-06-10T00:43:42.274 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.274 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.274 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.274 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.274 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.274 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.275 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.275 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-06-10T00:43:42.275 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.275 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.275 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.275 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.276 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:42.276 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:42.276 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.276 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.276 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.276 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-06-10T00:43:42.276 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-06-10T00:43:42.276 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.277 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.277 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-10T00:43:42.277 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.277 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.277 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.277 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.278 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.278 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.278 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.278 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.278 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.278 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.278 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.279 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.279 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.279 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.279 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.279 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.279 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.279 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.279 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.280 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.280 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.280 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.280 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.280 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.280 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.280 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.280 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.281 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.281 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.281 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.281 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.281 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.281 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.281 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.281 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.282 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.282 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.282 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.282 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.282 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.282 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.282 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.282 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.283 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.283 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.283 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.283 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.283 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.283 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.283 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.283 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.284 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.284 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.284 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.284 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.284 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.284 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.284 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.284 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.285 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.285 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.285 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.285 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.285 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.285 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.286 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.286 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.286 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.286 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.286 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.286 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.286 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.287 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.287 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.287 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-06-10T00:43:42.287 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-06-10T00:43:42.287 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.287 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.287 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.287 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.288 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.288 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.288 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.288 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-06-10T00:43:42.289 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.289 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.289 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.289 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.289 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-06-10T00:43:42.289 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-06-10T00:43:42.289 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.290 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.290 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.290 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-06-10T00:43:42.290 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.290 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.291 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.291 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-06-10T00:43:42.291 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.291 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.291 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.291 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.292 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-06-10T00:43:42.292 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-06-10T00:43:42.292 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.292 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.292 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.293 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-06-10T00:43:42.293 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.293 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.293 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.293 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T00:43:42.294 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.294 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.294 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.294 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.294 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-06-10T00:43:42.295 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-06-10T00:43:42.295 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.295 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.295 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.295 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-06-10T00:43:42.295 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-06-10T00:43:42.295 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-06-10T00:43:42.296 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.296 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-10T00:43:42.296 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.296 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.296 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.296 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.296 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-06-10T00:43:42.296 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-06-10T00:43:42.297 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.297 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.297 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.297 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.297 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.297 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.297 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.298 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-06-10T00:43:42.298 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.298 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.298 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.298 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.298 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-06-10T00:43:42.298 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-06-10T00:43:42.299 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.299 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.299 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.299 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-06-10T00:43:42.299 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.299 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.299 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.299 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T00:43:42.300 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.300 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.300 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.300 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.300 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.301 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.301 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.301 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-06-10T00:43:42.301 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.301 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.302 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.302 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.302 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.302 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.302 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.303 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.303 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.303 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.303 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.303 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.304 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.304 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.304 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.304 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.304 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.304 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.304 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.304 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.305 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.305 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.305 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.305 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.305 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.305 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.305 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.306 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.306 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.306 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.306 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.306 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.307 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.307 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.307 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.307 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.308 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.308 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.308 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.308 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.309 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.309 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.309 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.309 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.310 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.310 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.310 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.310 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.310 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.311 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.311 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.311 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.311 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.312 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.312 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.312 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.312 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.312 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.313 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.313 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.313 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.313 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.314 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.314 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.314 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.314 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.315 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.315 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.315 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.315 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.315 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.315 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.315 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.316 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.316 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.316 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.316 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-06-10T00:43:42.316 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.316 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.316 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.316 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.317 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.317 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.317 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.317 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.317 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.317 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.317 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.317 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.317 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.318 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.318 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.318 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.318 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.318 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.318 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.318 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.319 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.319 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.319 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.319 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.319 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.319 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.319 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.319 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.319 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.320 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.320 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.320 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.320 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.320 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.320 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.320 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.320 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.320 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.321 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.321 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.321 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.321 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.321 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.321 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.321 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.321 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.322 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.322 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.322 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.322 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.322 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.322 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.322 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.323 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.323 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.323 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.323 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.323 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.323 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.324 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.324 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.324 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.324 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.324 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.324 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.325 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.325 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.325 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.325 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.325 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.325 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.325 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.325 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.326 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.326 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.326 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.326 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-06-10T00:43:42.326 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.326 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.326 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.326 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-06-10T00:43:42.327 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-06-10T00:43:42.327 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.327 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.327 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.327 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-06-10T00:43:42.327 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.327 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.327 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.328 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-10T00:43:42.328 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.328 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.328 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.328 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.328 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-06-10T00:43:42.328 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-06-10T00:43:42.328 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.329 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.329 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.329 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:42.329 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-06-10T00:43:42.329 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.329 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.329 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-10T00:43:42.330 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.330 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.330 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.330 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.330 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.330 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.330 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.330 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.331 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.331 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.331 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.331 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.331 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.331 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.331 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.332 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.332 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.332 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.332 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.332 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.332 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.332 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.332 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.333 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.333 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.333 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.333 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.333 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.333 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.333 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.333 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.334 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.334 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.334 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.334 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.334 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.334 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.334 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.335 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.335 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.335 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.335 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.335 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.335 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.335 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.335 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.336 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.336 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.336 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.336 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.336 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.336 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.337 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.337 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.337 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.337 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.337 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.337 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.337 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.338 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.338 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.338 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.338 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.338 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.338 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.338 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.339 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.339 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.339 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.339 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.339 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.339 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.339 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.339 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.340 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.340 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.340 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.340 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-06-10T00:43:42.340 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.340 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.340 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.340 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-06-10T00:43:42.341 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-06-10T00:43:42.341 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.341 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.341 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.341 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-06-10T00:43:42.341 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.341 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.341 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.341 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-06-10T00:43:42.342 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-10T00:43:42.342 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.342 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.342 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.342 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.342 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.343 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.343 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.343 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.343 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.343 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.343 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.343 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.343 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.344 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.344 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.344 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.344 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.344 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.344 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.344 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.344 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.344 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.345 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.345 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.345 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.345 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.345 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.345 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.345 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.345 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.346 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.346 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.346 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.346 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.346 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.346 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.346 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.346 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.347 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.347 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.347 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.347 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.347 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.347 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.347 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.347 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.348 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.348 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.348 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.348 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.348 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.348 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.348 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.348 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.349 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.349 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.349 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.349 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.349 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.349 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.349 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.349 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.350 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.350 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.350 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.350 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.350 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.350 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.350 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.351 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.351 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.351 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.351 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.351 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-06-10T00:43:42.351 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-06-10T00:43:42.351 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.352 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.352 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.352 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.352 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.352 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.352 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.352 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-06-10T00:43:42.353 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-10T00:43:42.353 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.353 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.353 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.353 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-06-10T00:43:42.353 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-06-10T00:43:42.353 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.354 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.354 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.354 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-06-10T00:43:42.354 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.354 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.354 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.355 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-06-10T00:43:42.355 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.355 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.355 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.355 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.355 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-06-10T00:43:42.355 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-06-10T00:43:42.355 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.355 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.356 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.356 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-06-10T00:43:42.356 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.356 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.356 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.356 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-06-10T00:43:42.356 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.357 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.357 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.357 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.357 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-06-10T00:43:42.357 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-06-10T00:43:42.357 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.357 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.357 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.358 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.358 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.358 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.358 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.358 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-06-10T00:43:42.358 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.359 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.359 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.359 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.359 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-06-10T00:43:42.359 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-06-10T00:43:42.359 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.359 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.359 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.360 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-06-10T00:43:42.360 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.360 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.360 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.360 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-10T00:43:42.360 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-10T00:43:42.360 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.360 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.361 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.361 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.361 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.361 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.361 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-06-10T00:43:42.361 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.361 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.361 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.361 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.362 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.362 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.362 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.362 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.362 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.362 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.362 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.362 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.363 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.363 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.363 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.363 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.363 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.363 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.363 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.363 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.363 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.364 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.364 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.364 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.364 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.364 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.364 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.364 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.364 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.365 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.365 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.365 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.365 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.365 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.365 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.365 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.365 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.365 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.366 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.366 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.366 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.366 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.366 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.366 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.366 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.367 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.367 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.367 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.367 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.367 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.367 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.367 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.368 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.368 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.368 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.368 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.368 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.369 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.369 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.369 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.369 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.369 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.370 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.370 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.370 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.370 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.370 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.371 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.371 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.371 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.371 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.371 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.372 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.372 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.372 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.372 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.373 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-06-10T00:43:42.373 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.373 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.373 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.373 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.373 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.374 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.374 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.374 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.374 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.375 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.375 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.375 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.375 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.375 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.375 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.376 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.376 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.376 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.376 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.376 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.377 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.377 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.377 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.377 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.377 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.377 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.378 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.378 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.378 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.378 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.378 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.379 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.379 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.379 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.379 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.379 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.380 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.380 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.380 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.380 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.380 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.380 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.380 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.381 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.381 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.381 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.381 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.381 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.381 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.381 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.381 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.382 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.382 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.382 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.382 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.382 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.382 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.383 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.383 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.383 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.383 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.383 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.383 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.384 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.384 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.384 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.384 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.384 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.384 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.385 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.385 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.385 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.385 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.385 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.385 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.386 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.386 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-06-10T00:43:42.386 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.386 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.386 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.386 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.386 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.387 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.387 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.387 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.387 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.387 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.387 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.387 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.387 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.388 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.388 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.388 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.388 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.388 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.388 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.388 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.389 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.389 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.389 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.389 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.389 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.389 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.389 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.390 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.390 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.390 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.390 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.390 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.390 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.390 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.390 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.391 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.391 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.391 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.391 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.391 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.391 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.391 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.391 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.392 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.392 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.392 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.392 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.392 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.392 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.392 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.392 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.392 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.393 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.393 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.393 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.393 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.393 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.393 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.393 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.393 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.394 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.394 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.394 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.394 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.394 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.394 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.394 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.395 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.395 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.395 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.395 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.395 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.395 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.395 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.395 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.396 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.396 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-06-10T00:43:42.396 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.396 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.396 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.396 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-06-10T00:43:42.396 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-06-10T00:43:42.396 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.396 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.397 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.397 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.397 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.397 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.397 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.397 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-10T00:43:42.397 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.397 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.397 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.398 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.398 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.398 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.398 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.398 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.398 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.398 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.399 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.399 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.399 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.399 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.399 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.399 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.399 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.399 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.399 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.400 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.400 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.400 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.400 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.400 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.400 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.400 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.400 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.401 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.401 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.401 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.401 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.401 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.401 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.401 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.401 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.401 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.402 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.402 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.402 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.402 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.402 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.402 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.402 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.403 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.403 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.403 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.403 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.403 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.403 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.403 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.403 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.403 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.404 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.404 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.404 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.404 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.404 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.404 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.404 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.404 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.405 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.405 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.405 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.405 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.405 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.405 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.405 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.405 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.405 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.406 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.406 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.406 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.406 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.406 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.406 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-06-10T00:43:42.406 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-06-10T00:43:42.406 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.407 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.407 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.407 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.407 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.407 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.407 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.407 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-06-10T00:43:42.407 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-06-10T00:43:42.408 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-06-10T00:43:42.408 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.408 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-06-10T00:43:42.408 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.408 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.408 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.408 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.408 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.409 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.409 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.409 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-06-10T00:43:42.409 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.409 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.409 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.409 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.409 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.409 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.410 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.410 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.410 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.410 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.410 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.410 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.410 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.411 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.411 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.411 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.411 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.411 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.411 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.411 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.411 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.411 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.412 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.412 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.412 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.412 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.412 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.412 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.412 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.412 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.412 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.413 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.413 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.413 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.413 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.413 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.413 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.413 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.413 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.413 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.414 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.414 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.414 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.414 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.414 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.414 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.414 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.414 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.415 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.415 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.415 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.415 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.415 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.415 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.415 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.415 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.415 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.416 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.416 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.416 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.416 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.416 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.416 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.416 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.416 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.417 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.417 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.417 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.417 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.417 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.417 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.417 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.417 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.417 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.418 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.418 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.418 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-06-10T00:43:42.418 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.418 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.418 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.418 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.418 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.419 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.419 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.419 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.419 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.419 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.419 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.419 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.419 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.420 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.420 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.420 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.420 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.420 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.420 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.421 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.421 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.421 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.421 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.421 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.421 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.421 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.422 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.422 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.422 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.422 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.422 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.422 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.422 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.423 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.423 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.423 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.423 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.423 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.423 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.423 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.423 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.424 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.424 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.424 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.424 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.424 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.424 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.424 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.424 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.424 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.425 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.425 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.425 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.425 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.425 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.425 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.425 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.425 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.425 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.426 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.426 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.426 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.426 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.426 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.426 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.426 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.427 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.427 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.427 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.427 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.427 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.427 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.427 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.427 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.428 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.428 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-06-10T00:43:42.428 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.428 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.428 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.428 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.428 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.429 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.429 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.429 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.429 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.429 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.429 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.429 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.429 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.430 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.430 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.430 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.430 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.430 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.430 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.430 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.431 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.431 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.431 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.431 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.431 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.431 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.431 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.431 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.432 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.432 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.432 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.432 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.432 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.432 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.432 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.432 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.433 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.433 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.433 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.433 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.433 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.433 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.433 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.433 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.433 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.434 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.434 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.434 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.434 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.434 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.434 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.434 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.434 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.435 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.435 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.435 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.435 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.435 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.435 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.435 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.436 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.436 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.436 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.436 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.436 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.436 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.436 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.437 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.437 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.437 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.437 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.437 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.437 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.437 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:42.437 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:42.437 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.438 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.438 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.438 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:42.438 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.438 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.438 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.438 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.438 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.439 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.439 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.439 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.439 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-06-10T00:43:42.439 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-06-10T00:43:42.439 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:42.439 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.439 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.439 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-06-10T00:43:42.440 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-06-10T00:43:42.440 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.440 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.440 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-06-10T00:43:42.440 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-10T00:43:42.440 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.440 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.440 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.440 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.441 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.441 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.441 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-06-10T00:43:42.441 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.441 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.441 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.441 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-06-10T00:43:42.441 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-06-10T00:43:42.441 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.442 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.442 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.442 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.442 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.442 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.442 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.442 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-10T00:43:42.443 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.443 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.443 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.443 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.443 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-06-10T00:43:42.443 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-06-10T00:43:42.443 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.443 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.443 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.444 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.444 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.444 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.444 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.444 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.444 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.444 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.444 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.444 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.445 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.445 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.445 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.445 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.445 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.445 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.445 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.445 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.446 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.446 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.446 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.446 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.446 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.446 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.446 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.446 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.447 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.447 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.447 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.447 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.447 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.447 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.447 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.447 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.448 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.448 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.448 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.448 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.448 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.448 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.448 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.448 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.449 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.449 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.449 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.449 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.449 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.449 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.449 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.449 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.449 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.450 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.450 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.450 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.450 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.450 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.450 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.450 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.450 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.451 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.451 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.451 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.451 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.451 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.451 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.451 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.451 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.452 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.452 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.452 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.452 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.452 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.452 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.452 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.452 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.453 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.453 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.453 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.453 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.453 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.453 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-06-10T00:43:42.453 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-06-10T00:43:42.453 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.453 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.454 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.454 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-06-10T00:43:42.454 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.454 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.454 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.454 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-06-10T00:43:42.454 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.455 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.455 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.455 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.455 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:42.455 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:42.455 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.455 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.455 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.455 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-06-10T00:43:42.456 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.456 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.456 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.456 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-06-10T00:43:42.456 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.456 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.457 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.457 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.457 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.457 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.457 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.457 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-06-10T00:43:42.457 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.458 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.458 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.458 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-06-10T00:43:42.458 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-06-10T00:43:42.458 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.458 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.459 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.459 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.459 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.459 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.459 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.459 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.459 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.459 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.460 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.460 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.460 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-06-10T00:43:42.460 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-06-10T00:43:42.460 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.460 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.460 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.460 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.461 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.461 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.461 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.461 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-10T00:43:42.461 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-10T00:43:42.461 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.461 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.461 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.462 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.462 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.462 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.462 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.462 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.462 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.462 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.462 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.463 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.463 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.463 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.463 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.463 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.463 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.463 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.463 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.464 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.464 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.464 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.464 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.464 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.464 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.464 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.464 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.465 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.465 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.465 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.465 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.465 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.465 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.465 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.465 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.466 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.466 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.466 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.466 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.466 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.466 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.466 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.466 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.467 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.467 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.467 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.467 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.467 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.467 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.467 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.467 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.468 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.468 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.468 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.468 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.468 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.468 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.468 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.468 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.469 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.469 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.469 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.469 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.469 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.469 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.469 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.469 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.470 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.470 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.470 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.470 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.470 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.470 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.470 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-06-10T00:43:42.471 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-06-10T00:43:42.471 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.471 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.471 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.471 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.471 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.471 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.471 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.471 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.472 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.472 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.472 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.472 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.472 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-10T00:43:42.472 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-10T00:43:42.472 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.472 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.473 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.473 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-06-10T00:43:42.473 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.473 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.473 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.473 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.473 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.473 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.474 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.474 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.474 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-06-10T00:43:42.474 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-06-10T00:43:42.474 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:42.474 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.474 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.475 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-06-10T00:43:42.475 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.475 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.475 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.475 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.475 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.476 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.476 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.476 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.476 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-06-10T00:43:42.476 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-06-10T00:43:42.476 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.476 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.477 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.477 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-06-10T00:43:42.477 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.477 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.477 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.477 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-10T00:43:42.477 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.477 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.478 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.478 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.478 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-06-10T00:43:42.478 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-06-10T00:43:42.478 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.478 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.479 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.479 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-06-10T00:43:42.479 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.479 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.479 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.479 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-10T00:43:42.479 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.479 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.479 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.480 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.480 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-06-10T00:43:42.480 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-06-10T00:43:42.480 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.480 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.480 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.480 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-06-10T00:43:42.480 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.481 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.481 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.481 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.481 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.481 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.481 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.481 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.481 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-06-10T00:43:42.482 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-06-10T00:43:42.482 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.482 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.482 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.482 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-06-10T00:43:42.482 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.482 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.483 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.483 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-06-10T00:43:42.483 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-06-10T00:43:42.483 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.483 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.483 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.483 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.483 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.483 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.484 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-06-10T00:43:42.484 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-06-10T00:43:42.484 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.484 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.484 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.484 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-06-10T00:43:42.484 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-06-10T00:43:42.484 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-06-10T00:43:42.484 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.485 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.485 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.485 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.485 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.485 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.485 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.485 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.485 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.486 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-06-10T00:43:42.486 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.486 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.486 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.486 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.486 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.487 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.487 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.487 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.487 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.487 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.487 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.488 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.488 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.488 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.488 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.488 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.488 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.488 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.489 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.489 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.489 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.489 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.489 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.489 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.489 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.490 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.490 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.490 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.490 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.490 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.490 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.491 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.491 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.491 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.491 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.491 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.491 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.491 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.491 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.491 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.492 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.492 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.492 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.492 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.492 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.492 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.492 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.492 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.493 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.493 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.493 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.493 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.493 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.493 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.493 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.493 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.494 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.494 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.494 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.494 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.494 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.494 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.494 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.495 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.495 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.495 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.495 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.495 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.495 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.495 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.495 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.495 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.496 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.496 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-06-10T00:43:42.496 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-06-10T00:43:42.496 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.496 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.496 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.496 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:42.496 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.497 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.497 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.497 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.497 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.497 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.497 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.497 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.497 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-06-10T00:43:42.498 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-06-10T00:43:42.498 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.498 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.498 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.498 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.498 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.498 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.498 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.499 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.499 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.499 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.499 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.499 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.499 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-06-10T00:43:42.499 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-06-10T00:43:42.500 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.500 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.500 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.500 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.500 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.500 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.500 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.500 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.500 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.501 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.501 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.501 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.501 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.501 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.501 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.501 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-06-10T00:43:42.501 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.502 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.502 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.502 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-06-10T00:43:42.502 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-06-10T00:43:42.502 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.502 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.502 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.502 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.503 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.503 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.503 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.503 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.503 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.503 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.503 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.504 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.504 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-06-10T00:43:42.504 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-06-10T00:43:42.504 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.504 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.504 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.504 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.504 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.504 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.505 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.505 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.505 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.505 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.505 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.505 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.505 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.505 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.506 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.506 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.506 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.506 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.506 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.506 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.506 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.507 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.507 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.507 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.507 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.507 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.507 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.507 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.508 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.508 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.508 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.508 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.508 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.508 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.508 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.508 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.508 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.509 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.509 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.509 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.509 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.509 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.509 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.509 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.509 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.510 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.510 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.510 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.510 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.510 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.510 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.510 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.511 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.511 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.511 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.511 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.511 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.511 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.511 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.512 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.512 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.512 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.512 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.512 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.512 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.512 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.512 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.513 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.513 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.513 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.513 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.513 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.513 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.513 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.513 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.513 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.514 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.514 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.514 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.514 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.514 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.514 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.514 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-06-10T00:43:42.515 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-06-10T00:43:42.515 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.515 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.515 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.515 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.515 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.515 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.515 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.516 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.516 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.516 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.516 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.516 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.516 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-06-10T00:43:42.516 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-06-10T00:43:42.516 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.517 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.517 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.517 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.517 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.517 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.517 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.518 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.518 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.518 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.518 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.518 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.519 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.519 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.519 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.519 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-06-10T00:43:42.519 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.519 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.520 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.520 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.520 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.520 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.520 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.520 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.521 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.521 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.521 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.521 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.521 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.521 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.522 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.522 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.522 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.522 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.522 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.523 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.523 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.523 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.523 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.523 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.524 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.524 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.524 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.524 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.524 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.525 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.525 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.525 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.525 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.525 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.525 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.526 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.526 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.526 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.526 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.526 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.526 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.527 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.527 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.527 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.527 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.527 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.528 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.528 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.528 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.528 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.528 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.529 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.529 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.529 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.529 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.529 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.530 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.530 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.530 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.530 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.530 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.531 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.531 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.531 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.531 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.531 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.531 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.532 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.532 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.532 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.532 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.532 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.533 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.533 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.533 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.533 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.533 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-06-10T00:43:42.533 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.534 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.535 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.536 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.536 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.536 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.536 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.536 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.536 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.536 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.537 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.537 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.537 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.537 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.537 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.537 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.537 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.538 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.538 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.538 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.538 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.538 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.538 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.538 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.538 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.539 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.539 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.539 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.539 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.539 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.539 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.539 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.539 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.540 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.540 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.540 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.540 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.540 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.540 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.540 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.540 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.541 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.541 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.541 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.541 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.541 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.541 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.541 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.541 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.542 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.542 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.542 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.542 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.542 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.542 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.542 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.542 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.543 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.543 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.543 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.543 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-06-10T00:43:42.543 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-06-10T00:43:42.543 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.543 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.543 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.544 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-06-10T00:43:42.544 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.544 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.544 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.544 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-10T00:43:42.544 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.544 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.544 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.545 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.545 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.545 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.545 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.545 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-06-10T00:43:42.545 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.545 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.545 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.545 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.546 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.546 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.546 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.546 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.546 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.546 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.546 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.547 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.547 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.547 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.547 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.547 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.547 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.547 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.547 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.547 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.548 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.548 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.548 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.548 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.548 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.548 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.548 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.548 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.549 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.549 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.549 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.549 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.549 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.549 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.549 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.549 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.550 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.550 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.550 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.550 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.550 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.550 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.550 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.550 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.551 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.551 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.551 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.551 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.551 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.551 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.551 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.551 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.552 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.552 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.552 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.552 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.552 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.552 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.552 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.552 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.553 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.553 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.553 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.553 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.553 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.553 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.554 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.554 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.554 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.554 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.554 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.554 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.555 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.555 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-06-10T00:43:42.555 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-06-10T00:43:42.555 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.555 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.555 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.555 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.555 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.556 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.556 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.556 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.556 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.556 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.556 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.556 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.556 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-06-10T00:43:42.556 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-06-10T00:43:42.557 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.557 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.557 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.557 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-06-10T00:43:42.557 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.557 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.557 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.557 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.558 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.558 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.558 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.558 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.558 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-06-10T00:43:42.558 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-06-10T00:43:42.558 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.559 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.559 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.559 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.559 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.559 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.559 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.559 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.559 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.560 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.560 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.560 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.560 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-06-10T00:43:42.560 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-06-10T00:43:42.560 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.560 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.561 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.561 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.561 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.561 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.561 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.561 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.561 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.561 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.562 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.562 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.562 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-06-10T00:43:42.562 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-06-10T00:43:42.562 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:42.562 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.562 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.563 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.563 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.563 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.563 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.563 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.563 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.563 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.563 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.564 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.564 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-06-10T00:43:42.564 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-06-10T00:43:42.564 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.564 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.564 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.564 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.564 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.565 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.565 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.565 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.565 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.565 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.565 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.565 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.565 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-06-10T00:43:42.566 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-06-10T00:43:42.566 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.566 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.566 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.566 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.566 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.566 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.566 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.567 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.567 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.567 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.567 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.567 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.567 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-06-10T00:43:42.567 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-06-10T00:43:42.567 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.568 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.568 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.568 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.568 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.568 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.568 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.568 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.568 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.568 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.569 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.569 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.569 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-06-10T00:43:42.569 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-06-10T00:43:42.569 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.569 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.569 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.569 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.570 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-06-10T00:43:42.570 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-06-10T00:43:42.570 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.570 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.570 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.570 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.570 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.571 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-06-10T00:43:42.571 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-06-10T00:43:42.571 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.571 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.571 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.571 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.571 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.571 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.571 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.572 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.572 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.572 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.572 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.572 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.572 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-06-10T00:43:42.572 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-06-10T00:43:42.572 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.573 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.573 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.573 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.573 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.573 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.573 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.573 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.573 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.574 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.574 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.574 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.574 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-06-10T00:43:42.574 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-06-10T00:43:42.574 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.574 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.575 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.575 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.575 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.575 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.575 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.575 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.575 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.576 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.576 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.576 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.576 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.576 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.576 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.577 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-06-10T00:43:42.577 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.577 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.577 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.577 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-06-10T00:43:42.578 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-06-10T00:43:42.578 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.578 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.578 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.578 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.578 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.578 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.578 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.579 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-06-10T00:43:42.579 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.579 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.579 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.579 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.579 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-06-10T00:43:42.580 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-06-10T00:43:42.580 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.580 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.580 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.580 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.580 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.580 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.581 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.581 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-06-10T00:43:42.581 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.581 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.581 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.581 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.582 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.582 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.582 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.582 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.582 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.582 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.582 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.582 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.583 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.583 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.583 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.583 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.583 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.583 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.583 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.583 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.584 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.584 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.584 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.584 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.584 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.584 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.584 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.584 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.585 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.585 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.585 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.585 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.585 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.585 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.585 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.585 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.585 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.586 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.586 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.586 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.586 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.586 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.586 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.586 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.586 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.589 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.589 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.590 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.590 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.590 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.590 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.590 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.591 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.591 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.591 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.591 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.591 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.591 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.591 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.592 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.592 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.592 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.592 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.592 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.592 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.592 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.592 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.592 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.593 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.593 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.593 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.593 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.593 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.593 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.593 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.593 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.594 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.594 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-06-10T00:43:42.594 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.595 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.595 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.595 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.595 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.595 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.596 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.596 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.596 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.596 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.596 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.596 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.597 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.597 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.597 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.597 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.597 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.597 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.597 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.598 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.598 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.598 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.598 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.598 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.598 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.598 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.599 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.599 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.599 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.599 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.599 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.599 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.599 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.599 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.600 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.600 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.600 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.600 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.600 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.600 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.600 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.601 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.601 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.601 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.601 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.601 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.601 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.601 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.601 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.602 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.602 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.602 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.602 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.602 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.602 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.602 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.603 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.603 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.603 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.603 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.603 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.603 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.603 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.604 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.604 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.604 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.604 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.604 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.604 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.605 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.605 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.605 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.605 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.605 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.606 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.606 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.606 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-06-10T00:43:42.606 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.607 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.607 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.607 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.607 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.607 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.608 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.608 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.608 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.608 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.608 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.608 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.609 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.609 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.609 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.609 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.609 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.610 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.610 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.610 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.610 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.610 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.611 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.611 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.611 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.611 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.611 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.612 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.612 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.612 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.612 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.612 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.612 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.612 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.612 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.613 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.613 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.613 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.613 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.613 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.613 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.613 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.614 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.614 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.614 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.614 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.614 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.614 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.614 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.615 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.615 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.615 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.615 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.615 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.615 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.615 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.615 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.616 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.616 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.616 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.616 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.616 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.616 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.616 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.616 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.617 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.617 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.617 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.617 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.618 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.618 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.618 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.618 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.618 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.619 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.619 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.619 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-06-10T00:43:42.619 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.619 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.619 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.619 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-06-10T00:43:42.619 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-06-10T00:43:42.619 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.620 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.620 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.620 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-10T00:43:42.620 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.620 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.620 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.620 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.620 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.621 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.621 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.621 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.621 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:42.621 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:42.621 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.621 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.621 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.622 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-06-10T00:43:42.622 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.622 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.622 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.622 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.622 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.622 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.623 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.623 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.623 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.623 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.623 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.623 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.623 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.623 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.624 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.624 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.624 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.624 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.624 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.624 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.624 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.624 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.625 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.625 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.625 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.625 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.625 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.625 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.625 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.625 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.626 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.626 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.626 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.626 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.626 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.626 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.626 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.627 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.627 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.627 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.627 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.627 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.627 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.627 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.627 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.627 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.628 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.628 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.628 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.628 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.628 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.628 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.629 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.629 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.629 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.629 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.630 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.630 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.630 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.631 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.631 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.631 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.631 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.631 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.632 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.632 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.632 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.632 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.633 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.633 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.633 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.633 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.634 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.634 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.634 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.634 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.635 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.635 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.635 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.635 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.636 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.636 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-06-10T00:43:42.636 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.636 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.637 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.637 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-06-10T00:43:42.637 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-06-10T00:43:42.637 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.638 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.638 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.638 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.638 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.639 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.639 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.639 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-10T00:43:42.639 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.640 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.640 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.640 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-06-10T00:43:42.640 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-06-10T00:43:42.640 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.640 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.640 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.641 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.641 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.641 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.641 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.641 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-10T00:43:42.641 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.641 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.641 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.642 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.642 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-06-10T00:43:42.642 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-06-10T00:43:42.642 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.642 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.642 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.642 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.642 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.643 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.643 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.643 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-10T00:43:42.643 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.643 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.643 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.643 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.643 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-06-10T00:43:42.644 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-06-10T00:43:42.644 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.644 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.644 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.644 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.644 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.644 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.644 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.645 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-10T00:43:42.645 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.645 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.645 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.645 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.645 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-06-10T00:43:42.645 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-06-10T00:43:42.645 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.645 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.646 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.646 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.646 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.646 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.646 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.646 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-10T00:43:42.646 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.647 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.647 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.647 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.647 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-06-10T00:43:42.647 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-06-10T00:43:42.647 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.647 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.647 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.647 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.648 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.648 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.648 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.648 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.648 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.648 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.648 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.648 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.649 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-06-10T00:43:42.649 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-06-10T00:43:42.649 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.649 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.649 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.649 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.649 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.649 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.649 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.650 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.650 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.650 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.650 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.650 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.650 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-06-10T00:43:42.650 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-06-10T00:43:42.651 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.651 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.651 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.651 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-10T00:43:42.651 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.651 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.651 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.651 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.652 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.652 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.652 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.652 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.652 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-06-10T00:43:42.652 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-06-10T00:43:42.652 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.652 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.653 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.653 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.653 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.653 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.653 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.653 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.653 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.653 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.654 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.654 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.654 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:42.654 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:42.654 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.654 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.654 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.654 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-06-10T00:43:42.655 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-06-10T00:43:42.655 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.655 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.655 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.655 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.655 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.655 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.655 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.656 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-06-10T00:43:42.656 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-06-10T00:43:42.656 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.656 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.656 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.656 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-06-10T00:43:42.656 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.656 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.657 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.657 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.657 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.657 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.657 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.657 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.657 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-06-10T00:43:42.657 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-06-10T00:43:42.658 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.658 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.658 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.658 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.658 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.658 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.658 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.658 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.659 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.659 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.659 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.659 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.659 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-06-10T00:43:42.659 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-06-10T00:43:42.659 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.659 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.660 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.660 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.660 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.660 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.660 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.660 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.660 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.660 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.660 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.661 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.661 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.661 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.661 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.661 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.661 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.662 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.662 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.662 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.662 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.662 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.663 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.663 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.663 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.663 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.663 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.663 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.663 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.664 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.664 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.664 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.664 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.664 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.664 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.664 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.664 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.665 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.665 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.665 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.665 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.665 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.665 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.665 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.665 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.666 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.666 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.666 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.666 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.666 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.666 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.666 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.666 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.667 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.667 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.667 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.667 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.667 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.667 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.667 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.667 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.668 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.668 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.668 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.668 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.668 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.668 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.668 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.669 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.669 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.669 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.669 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.669 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.669 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.669 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.670 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.670 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.670 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.670 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.670 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.670 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.670 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.670 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-06-10T00:43:42.671 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-06-10T00:43:42.671 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.671 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.671 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.671 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.671 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.671 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.671 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.672 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.672 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.672 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.672 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.672 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.672 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-06-10T00:43:42.672 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-06-10T00:43:42.672 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.673 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.673 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.673 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.673 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.673 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.673 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.673 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.673 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.674 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.674 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.674 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.674 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-06-10T00:43:42.674 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-06-10T00:43:42.674 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.674 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.674 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.675 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-10T00:43:42.675 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.675 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.675 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.675 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.675 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.675 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.675 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.676 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.676 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.676 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.676 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.676 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-06-10T00:43:42.676 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.676 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.676 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.677 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.677 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.677 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.677 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.677 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.677 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.677 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.677 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.678 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.678 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.678 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.678 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.678 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.678 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.678 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.679 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.679 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.679 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.679 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.679 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.679 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.679 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.679 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.680 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.680 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.680 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.680 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.680 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.680 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.680 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.681 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.681 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.681 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.681 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.681 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.681 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.681 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.681 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.682 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.682 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.682 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.682 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.682 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.682 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.682 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.682 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.683 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.683 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.683 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.683 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.683 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.683 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.683 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.683 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.684 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.684 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.684 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.684 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.684 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.684 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.684 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.684 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.685 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.685 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.685 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.685 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.685 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.685 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.685 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.685 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.686 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.686 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.686 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.686 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-06-10T00:43:42.686 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.686 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.686 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.686 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.687 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.687 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.687 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.687 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.687 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.687 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.687 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.687 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.688 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.688 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.688 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.688 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.688 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.688 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.688 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.688 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.689 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.689 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.689 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.689 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.689 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.689 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.689 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.689 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.690 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.690 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.690 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.690 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.690 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.690 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.691 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.691 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.691 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.691 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.691 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.692 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.692 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.692 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.692 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.692 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.692 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.693 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.693 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.693 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.693 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.693 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.693 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.693 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.694 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.694 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.694 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.694 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.694 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.694 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.694 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.694 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.695 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.695 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.695 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.695 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.695 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.695 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.695 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.695 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.696 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.696 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.696 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.696 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.696 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.696 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-06-10T00:43:42.696 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-06-10T00:43:42.696 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.697 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.697 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.697 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-06-10T00:43:42.697 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.697 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.697 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.697 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-10T00:43:42.698 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.698 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.698 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.698 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.698 INFO:teuthology.orchestra.run.smithi062.stdout: "periodic_async_work": { 2024-06-10T00:43:42.698 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "periodic_async_work", 2024-06-10T00:43:42.699 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.699 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.699 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.699 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.699 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.699 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.699 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.699 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Periodically check for async work", 2024-06-10T00:43:42.700 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.700 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.700 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.700 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.700 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-06-10T00:43:42.700 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-06-10T00:43:42.700 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.700 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.701 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.701 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:42.701 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.701 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.701 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.701 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-10T00:43:42.702 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.702 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.702 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.702 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.702 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-06-10T00:43:42.702 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-06-10T00:43:42.702 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.702 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.703 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.703 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.703 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.703 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.703 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.703 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-10T00:43:42.703 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.703 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.704 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.704 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.704 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.704 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.704 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.704 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-06-10T00:43:42.705 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.705 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.705 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.705 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-06-10T00:43:42.705 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-06-10T00:43:42.705 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:42.705 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.705 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.706 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-06-10T00:43:42.706 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.706 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.706 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.706 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.706 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.706 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.707 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.707 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.707 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-06-10T00:43:42.707 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-06-10T00:43:42.707 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.707 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.707 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.707 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.708 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.708 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.708 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.708 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.708 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.708 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.708 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.708 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:42.709 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:42.709 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.709 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.709 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.709 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:42.709 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.709 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.709 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.710 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.710 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.710 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.710 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.710 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.710 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.711 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.711 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.711 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.711 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.711 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.711 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.711 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.711 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.711 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.712 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.712 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.712 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.712 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.712 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.712 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.712 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.713 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.713 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.713 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.713 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.713 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.713 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.713 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.713 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.714 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.714 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.714 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.714 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.714 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.714 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.715 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.715 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.715 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.715 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.715 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.715 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.715 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.715 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.715 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.716 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.716 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.716 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.716 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.716 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.716 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.717 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.717 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.717 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.717 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.717 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.717 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.718 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.718 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.718 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.718 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.718 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.718 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.718 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.719 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.719 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.719 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.719 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.719 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.719 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.719 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.719 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.719 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.720 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.720 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-06-10T00:43:42.720 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-06-10T00:43:42.720 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.720 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.720 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.720 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.720 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.721 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.721 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.721 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.721 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.721 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.721 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.721 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.721 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-06-10T00:43:42.721 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-06-10T00:43:42.722 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.722 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.722 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.722 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-06-10T00:43:42.722 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.722 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.722 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.722 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.723 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.723 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.723 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.723 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.723 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-06-10T00:43:42.723 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-06-10T00:43:42.723 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.723 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.724 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.724 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-10T00:43:42.724 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.724 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.724 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.724 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.724 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.724 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.724 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.725 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.725 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.725 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.725 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-06-10T00:43:42.725 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.725 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.725 INFO:teuthology.orchestra.run.smithi062.stdout: "modules": [ 2024-06-10T00:43:42.725 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-06-10T00:43:42.726 INFO:teuthology.orchestra.run.smithi062.stdout: "dashboard", 2024-06-10T00:43:42.726 INFO:teuthology.orchestra.run.smithi062.stdout: "iostat", 2024-06-10T00:43:42.726 INFO:teuthology.orchestra.run.smithi062.stdout: "nfs", 2024-06-10T00:43:42.726 INFO:teuthology.orchestra.run.smithi062.stdout: "restful", 2024-06-10T00:43:42.726 INFO:teuthology.orchestra.run.smithi062.stdout: "snap_schedule" 2024-06-10T00:43:42.726 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.726 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-06-10T00:43:42.727 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.727 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-06-10T00:43:42.727 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.727 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.727 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.727 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:42.727 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:42.727 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.728 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.728 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.728 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:42.728 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.728 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.728 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.728 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-06-10T00:43:42.728 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.729 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.729 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.729 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.729 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.729 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.729 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.729 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.729 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.730 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.730 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.730 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.730 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.731 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.731 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.731 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.731 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.732 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.732 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.732 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.732 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.732 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.732 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.733 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.733 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.733 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.733 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.733 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.733 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.733 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.733 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.734 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.734 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.734 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.734 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.734 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.734 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.734 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.734 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.735 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.735 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.735 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.735 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.735 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.735 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.735 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.735 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.736 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.736 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.736 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.736 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.736 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.736 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.737 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.737 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.737 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.737 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.738 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.738 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.738 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.738 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.739 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.739 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.739 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.739 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.740 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.740 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.740 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.740 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.741 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.741 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.741 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.741 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.742 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.742 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-06-10T00:43:42.742 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-06-10T00:43:42.742 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.743 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.743 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.743 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.743 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.744 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.744 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.744 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-06-10T00:43:42.744 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.745 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.745 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.745 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.746 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-06-10T00:43:42.746 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-06-10T00:43:42.746 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.746 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.747 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.747 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-06-10T00:43:42.747 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.748 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.748 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.748 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-06-10T00:43:42.748 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.749 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.749 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.749 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.749 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-06-10T00:43:42.749 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-06-10T00:43:42.750 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.750 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.750 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.751 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.751 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.751 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.751 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.751 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-06-10T00:43:42.752 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.752 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.752 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.752 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.753 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-06-10T00:43:42.753 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-06-10T00:43:42.753 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.754 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.754 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.754 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.754 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.755 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.755 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.755 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-06-10T00:43:42.755 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.756 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.756 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.756 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.756 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-06-10T00:43:42.756 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-06-10T00:43:42.757 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.757 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.757 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.757 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-06-10T00:43:42.757 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.757 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.757 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.757 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-06-10T00:43:42.758 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.758 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.758 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.758 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.758 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-06-10T00:43:42.758 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-06-10T00:43:42.758 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.758 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.759 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.759 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.759 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.759 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.759 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.759 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-06-10T00:43:42.759 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.759 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.760 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.760 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.760 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-06-10T00:43:42.760 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-06-10T00:43:42.760 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.760 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.760 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.760 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.761 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.761 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.761 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.761 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-10T00:43:42.761 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.761 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.761 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.761 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.761 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-06-10T00:43:42.762 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-06-10T00:43:42.762 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.762 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.762 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.762 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.762 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.762 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.762 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.763 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-06-10T00:43:42.763 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.763 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.763 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.763 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.763 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.763 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.764 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-06-10T00:43:42.764 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.764 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.764 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.764 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-06-10T00:43:42.764 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-06-10T00:43:42.764 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.764 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.765 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.765 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.765 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.765 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.765 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.765 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-06-10T00:43:42.765 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.765 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.766 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.766 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.766 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-06-10T00:43:42.766 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-06-10T00:43:42.766 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.766 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.766 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.766 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-06-10T00:43:42.767 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.767 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.767 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.767 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-06-10T00:43:42.767 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T00:43:42.767 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.767 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.767 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.768 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-06-10T00:43:42.768 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-06-10T00:43:42.768 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:42.768 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.768 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.768 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:42.768 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-06-10T00:43:42.768 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-06-10T00:43:42.769 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.769 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-10T00:43:42.769 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T00:43:42.769 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.769 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.769 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.769 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-06-10T00:43:42.769 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-06-10T00:43:42.770 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:42.770 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.770 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.770 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-06-10T00:43:42.770 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-06-10T00:43:42.770 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-06-10T00:43:42.770 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.770 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-10T00:43:42.771 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.771 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.771 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.771 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.771 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-06-10T00:43:42.771 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-06-10T00:43:42.771 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.771 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.772 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.772 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-06-10T00:43:42.772 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.772 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.772 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.772 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-10T00:43:42.772 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-10T00:43:42.772 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.773 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.773 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.773 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-06-10T00:43:42.773 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-06-10T00:43:42.773 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:42.773 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.773 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.773 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-06-10T00:43:42.774 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-06-10T00:43:42.774 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-06-10T00:43:42.774 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.774 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-06-10T00:43:42.774 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-10T00:43:42.774 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.774 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.774 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.775 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-06-10T00:43:42.775 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-06-10T00:43:42.775 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.775 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.775 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.775 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-06-10T00:43:42.775 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.775 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.776 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.776 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-06-10T00:43:42.776 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T00:43:42.776 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.776 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.776 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.776 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-06-10T00:43:42.776 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-06-10T00:43:42.777 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:42.777 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.777 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.777 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:42.777 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-06-10T00:43:42.777 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-06-10T00:43:42.777 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.777 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-10T00:43:42.778 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T00:43:42.778 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.778 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.778 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.778 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.778 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.778 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.779 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.779 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.779 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.779 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.779 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.779 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.779 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.779 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.780 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.780 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.780 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.780 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.780 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.780 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.780 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.781 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.781 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.781 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.781 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.781 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.781 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.781 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.781 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.782 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.782 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.782 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.782 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.782 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.782 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.782 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.782 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.783 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.783 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.783 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.783 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.783 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.783 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.783 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.783 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.783 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.784 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.784 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.784 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.784 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.784 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.784 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.784 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.784 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.785 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.785 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.785 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.785 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.785 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.785 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.785 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.786 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.786 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.786 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.786 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.786 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.786 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.786 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.786 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.787 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.787 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.787 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.787 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-06-10T00:43:42.787 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-06-10T00:43:42.787 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:42.787 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.787 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.788 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:42.788 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.788 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.788 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.788 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-10T00:43:42.788 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.788 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.788 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.789 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-06-10T00:43:42.789 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-06-10T00:43:42.789 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.789 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.789 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.789 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-06-10T00:43:42.789 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.790 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.790 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.790 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-06-10T00:43:42.790 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-06-10T00:43:42.790 INFO:teuthology.orchestra.run.smithi062.stdout: "read", 2024-06-10T00:43:42.790 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap", 2024-06-10T00:43:42.790 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap-read" 2024-06-10T00:43:42.790 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.791 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-06-10T00:43:42.791 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.791 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.791 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.791 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-06-10T00:43:42.791 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-06-10T00:43:42.791 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.792 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.792 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.792 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.792 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.792 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.792 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.792 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-10T00:43:42.793 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.793 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.793 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.793 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.793 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:42.793 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:42.793 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.793 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.794 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.794 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:42.794 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.794 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.794 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.797 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-10T00:43:42.797 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.798 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.798 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.798 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.798 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-06-10T00:43:42.798 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-06-10T00:43:42.798 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.798 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.799 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.799 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-06-10T00:43:42.799 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-06-10T00:43:42.799 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.799 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.799 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-10T00:43:42.799 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-10T00:43:42.799 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.799 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.800 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.800 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-06-10T00:43:42.800 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-06-10T00:43:42.800 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:42.800 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.800 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.800 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:42.800 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.801 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.801 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.801 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-10T00:43:42.801 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.801 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.801 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.801 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.801 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.802 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.802 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.802 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-06-10T00:43:42.802 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.802 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.802 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.802 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-06-10T00:43:42.802 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-06-10T00:43:42.803 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:42.803 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.803 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.803 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-06-10T00:43:42.803 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.803 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.803 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.803 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-10T00:43:42.804 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.804 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.804 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.804 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.804 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-06-10T00:43:42.804 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-06-10T00:43:42.804 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.804 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.805 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.805 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-06-10T00:43:42.805 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.805 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.805 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.805 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-10T00:43:42.805 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.806 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.806 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.806 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.806 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-06-10T00:43:42.806 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-06-10T00:43:42.806 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.806 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.806 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.807 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-06-10T00:43:42.807 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.807 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.807 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.807 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-10T00:43:42.807 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.807 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.807 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.807 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.808 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-06-10T00:43:42.808 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-06-10T00:43:42.808 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.808 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.808 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.808 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.808 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.808 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.809 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.809 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-10T00:43:42.809 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-10T00:43:42.809 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.809 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.809 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.809 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-06-10T00:43:42.809 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-06-10T00:43:42.810 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.810 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.810 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.810 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:42.810 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.810 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.810 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.810 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-10T00:43:42.811 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.811 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.811 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.811 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.811 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-06-10T00:43:42.811 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-06-10T00:43:42.811 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:42.811 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.812 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.812 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-06-10T00:43:42.812 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.812 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.812 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.812 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-10T00:43:42.812 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.812 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.813 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.813 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.813 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm_log_destination": { 2024-06-10T00:43:42.813 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm_log_destination", 2024-06-10T00:43:42.813 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.813 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.813 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.813 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.814 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.814 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.814 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.814 INFO:teuthology.orchestra.run.smithi062.stdout: "file", 2024-06-10T00:43:42.814 INFO:teuthology.orchestra.run.smithi062.stdout: "file,syslog", 2024-06-10T00:43:42.814 INFO:teuthology.orchestra.run.smithi062.stdout: "syslog" 2024-06-10T00:43:42.814 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.814 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-10T00:43:42.815 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.815 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.815 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.815 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.815 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-06-10T00:43:42.815 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-06-10T00:43:42.815 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.815 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.816 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.816 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.816 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.816 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.816 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.816 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-10T00:43:42.816 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.817 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.817 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.817 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.817 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-06-10T00:43:42.817 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-06-10T00:43:42.817 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.817 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.817 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.818 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.818 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.818 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.818 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.818 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-10T00:43:42.818 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.818 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.818 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.819 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.819 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-06-10T00:43:42.819 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-06-10T00:43:42.819 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.819 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.819 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.819 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.819 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.820 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.820 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.820 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-10T00:43:42.820 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.820 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.820 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.820 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.820 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-06-10T00:43:42.821 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-06-10T00:43:42.821 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.821 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.821 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.821 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-10T00:43:42.821 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.821 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.821 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.822 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-06-10T00:43:42.822 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.822 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.822 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.822 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.822 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-06-10T00:43:42.822 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-06-10T00:43:42.822 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.823 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.823 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.823 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-10T00:43:42.823 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.823 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.823 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.823 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-06-10T00:43:42.824 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.824 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.824 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.824 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.824 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-06-10T00:43:42.824 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-06-10T00:43:42.824 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.824 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.825 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.825 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-10T00:43:42.825 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.825 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.825 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.825 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-06-10T00:43:42.825 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.825 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.826 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.826 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.826 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-06-10T00:43:42.826 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-06-10T00:43:42.826 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.826 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.826 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.827 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-10T00:43:42.827 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.827 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.827 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.827 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-06-10T00:43:42.827 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.827 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.827 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.827 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.828 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-06-10T00:43:42.828 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-06-10T00:43:42.828 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.828 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.828 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.828 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-10T00:43:42.828 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.828 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.829 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.829 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-06-10T00:43:42.829 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.829 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.829 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.829 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.829 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-06-10T00:43:42.829 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-06-10T00:43:42.830 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.830 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.830 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.830 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-10T00:43:42.830 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.830 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.830 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.830 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-06-10T00:43:42.830 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.831 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.831 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.831 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.831 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-06-10T00:43:42.831 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-06-10T00:43:42.831 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.831 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.831 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.832 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-10T00:43:42.832 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.832 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.832 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.832 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-06-10T00:43:42.832 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.832 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.832 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.833 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.833 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-06-10T00:43:42.833 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-06-10T00:43:42.833 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.833 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.833 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.833 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-10T00:43:42.833 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.834 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.834 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.834 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-06-10T00:43:42.834 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.834 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.834 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.834 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.834 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-06-10T00:43:42.835 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-06-10T00:43:42.835 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.835 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.835 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.835 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-10T00:43:42.835 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.835 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.835 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.835 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-06-10T00:43:42.836 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.836 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.836 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.836 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.836 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-06-10T00:43:42.836 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-06-10T00:43:42.836 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.836 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.837 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.837 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-10T00:43:42.837 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.837 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.837 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.837 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-06-10T00:43:42.837 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.837 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.837 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.838 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.838 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-06-10T00:43:42.838 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-06-10T00:43:42.838 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.838 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.838 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.838 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-10T00:43:42.839 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.839 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.839 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.839 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-06-10T00:43:42.839 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.839 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.839 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.839 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.839 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-06-10T00:43:42.840 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-06-10T00:43:42.840 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.840 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.840 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.840 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-10T00:43:42.840 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.840 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.840 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.841 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-06-10T00:43:42.841 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.841 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.841 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.841 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.841 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-06-10T00:43:42.841 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-06-10T00:43:42.841 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.841 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.842 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.842 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-10T00:43:42.842 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.842 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.842 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.842 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-06-10T00:43:42.842 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.843 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.843 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.843 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.843 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-06-10T00:43:42.843 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-06-10T00:43:42.843 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.843 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.843 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.844 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-10T00:43:42.844 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.844 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.844 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.844 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-06-10T00:43:42.844 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.844 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.844 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.845 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.845 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_samba": { 2024-06-10T00:43:42.845 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_samba", 2024-06-10T00:43:42.845 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.845 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.845 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.845 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-10T00:43:42.845 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.846 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.846 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.846 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Samba/SMB container image", 2024-06-10T00:43:42.846 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.846 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.846 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.846 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.846 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-06-10T00:43:42.847 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-06-10T00:43:42.847 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.847 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.847 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.847 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-10T00:43:42.847 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.847 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.847 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.847 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-06-10T00:43:42.848 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.848 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.848 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.848 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.848 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-06-10T00:43:42.848 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-06-10T00:43:42.848 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.848 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.849 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.849 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.849 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.849 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.849 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.849 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-06-10T00:43:42.849 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.849 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.849 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.849 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.850 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-06-10T00:43:42.850 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-06-10T00:43:42.850 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.850 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.850 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.850 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:42.850 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.850 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.851 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.851 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-10T00:43:42.851 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.851 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.851 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.851 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-06-10T00:43:42.851 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-06-10T00:43:42.851 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.852 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.852 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.852 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-06-10T00:43:42.852 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.852 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.852 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.852 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-10T00:43:42.852 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.852 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.853 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.853 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.853 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-06-10T00:43:42.853 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-06-10T00:43:42.853 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.853 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.853 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.853 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-06-10T00:43:42.853 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.854 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.854 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.854 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-10T00:43:42.854 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.854 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.854 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.854 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.854 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-06-10T00:43:42.855 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-06-10T00:43:42.855 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.855 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.855 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.855 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-06-10T00:43:42.855 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.855 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.855 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.855 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-06-10T00:43:42.856 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.856 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.856 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.856 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.856 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-06-10T00:43:42.856 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-06-10T00:43:42.856 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.856 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.857 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.857 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.857 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.857 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.857 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.857 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-10T00:43:42.857 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.858 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.858 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.858 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.858 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-06-10T00:43:42.858 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-06-10T00:43:42.858 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.858 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.858 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.859 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:42.859 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.859 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.859 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.859 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-06-10T00:43:42.859 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.859 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.859 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.860 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.860 INFO:teuthology.orchestra.run.smithi062.stdout: "grafana_dashboards_path": { 2024-06-10T00:43:42.860 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "grafana_dashboards_path", 2024-06-10T00:43:42.860 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.860 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.860 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.860 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-10T00:43:42.860 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.861 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.861 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.861 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-10T00:43:42.861 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.861 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.861 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.861 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.861 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-06-10T00:43:42.862 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-06-10T00:43:42.862 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.862 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.862 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.862 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:42.862 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.862 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.863 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.863 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-06-10T00:43:42.863 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.863 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.863 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.863 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.863 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-06-10T00:43:42.863 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-06-10T00:43:42.864 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.864 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.864 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.864 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.864 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.864 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.864 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.864 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-10T00:43:42.864 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.865 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.865 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.865 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.865 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-06-10T00:43:42.865 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-06-10T00:43:42.865 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.865 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.865 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.865 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.866 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.866 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.866 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.866 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-10T00:43:42.866 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.866 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.866 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.867 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.867 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.867 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.867 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.867 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.867 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.867 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.867 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.868 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.868 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.868 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.868 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.868 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.868 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.868 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.868 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.868 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.869 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.869 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.869 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.869 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.869 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.869 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-06-10T00:43:42.869 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-06-10T00:43:42.870 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.870 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.870 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.870 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.870 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.870 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.870 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.870 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-10T00:43:42.871 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.871 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.871 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.871 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.871 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.871 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.871 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.872 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.872 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.872 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.872 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.872 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.872 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.872 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-10T00:43:42.872 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.873 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.873 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.873 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.873 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.873 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.873 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.873 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.873 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.874 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.874 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.874 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.874 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.874 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.874 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.874 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.875 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.875 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.875 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.875 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.875 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.875 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.875 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.875 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.876 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.876 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.876 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.876 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.876 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.876 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.876 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.876 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.877 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.877 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.877 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.877 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.877 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.877 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.877 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.877 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-10T00:43:42.878 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-10T00:43:42.878 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.878 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.878 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.878 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.878 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.878 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.879 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.879 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-10T00:43:42.879 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.879 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.879 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.879 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.879 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-10T00:43:42.879 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-10T00:43:42.880 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.880 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.880 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.880 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-06-10T00:43:42.880 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.880 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.880 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.880 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-10T00:43:42.881 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.881 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.881 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.881 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.881 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-06-10T00:43:42.881 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-06-10T00:43:42.881 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.881 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.881 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.882 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:42.882 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.882 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.882 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.882 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-06-10T00:43:42.882 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.882 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.882 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.883 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.883 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-06-10T00:43:42.883 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-06-10T00:43:42.883 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.883 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.883 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.883 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:42.883 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.883 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.884 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.884 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-10T00:43:42.884 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.884 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.884 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.884 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.884 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-06-10T00:43:42.884 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-06-10T00:43:42.884 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.885 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.885 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.885 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.885 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.885 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.885 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.885 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-06-10T00:43:42.885 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.885 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.886 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.886 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.886 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-06-10T00:43:42.886 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-06-10T00:43:42.886 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.886 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.886 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.886 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-06-10T00:43:42.887 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.887 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.887 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.887 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-06-10T00:43:42.887 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-06-10T00:43:42.887 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.887 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-06-10T00:43:42.887 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.887 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.888 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.888 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.888 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-06-10T00:43:42.888 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-06-10T00:43:42.888 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.888 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.888 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.888 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-06-10T00:43:42.888 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.889 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.889 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.889 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-10T00:43:42.889 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.889 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.889 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.889 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.889 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-06-10T00:43:42.890 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-06-10T00:43:42.890 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.890 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.890 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.890 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-10T00:43:42.890 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.890 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.890 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.891 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-10T00:43:42.891 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.891 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.891 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.891 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-06-10T00:43:42.891 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-06-10T00:43:42.892 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.892 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.892 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.892 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.892 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.892 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.892 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.892 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-10T00:43:42.893 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.893 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.893 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.893 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.893 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-06-10T00:43:42.893 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-06-10T00:43:42.893 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.893 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.893 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.894 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.894 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.894 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.894 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.894 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-10T00:43:42.894 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.894 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.894 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.895 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.895 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-06-10T00:43:42.895 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-06-10T00:43:42.895 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.895 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.895 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.895 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.895 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.896 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.896 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.896 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-10T00:43:42.896 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.896 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.896 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.896 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.896 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-06-10T00:43:42.896 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-06-10T00:43:42.897 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.897 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.897 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.897 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.897 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.897 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.897 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.897 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-10T00:43:42.897 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.898 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.898 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.898 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.898 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-06-10T00:43:42.898 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-06-10T00:43:42.898 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.898 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.898 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.899 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.899 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.899 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.899 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.899 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-10T00:43:42.899 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.899 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.899 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.900 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.900 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-06-10T00:43:42.900 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-06-10T00:43:42.900 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.900 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.900 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.900 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-06-10T00:43:42.900 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.901 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.901 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.901 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-06-10T00:43:42.901 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.901 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.901 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.901 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.901 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-06-10T00:43:42.901 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-06-10T00:43:42.902 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.902 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.902 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.902 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.902 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.902 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.902 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.902 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-10T00:43:42.903 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.903 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.903 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.903 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.903 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-06-10T00:43:42.903 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-06-10T00:43:42.903 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.903 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.904 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.904 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.904 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.904 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.904 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.904 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-10T00:43:42.904 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.904 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.905 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.905 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.905 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-06-10T00:43:42.905 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-06-10T00:43:42.905 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.905 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.905 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.905 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.905 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.906 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.906 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.906 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-10T00:43:42.906 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.906 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.906 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.906 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.906 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-06-10T00:43:42.907 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-06-10T00:43:42.907 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.907 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.907 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.907 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.907 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.907 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.907 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.908 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-06-10T00:43:42.908 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.908 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.908 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.908 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.908 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-06-10T00:43:42.908 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-06-10T00:43:42.908 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.909 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.909 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.909 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.909 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.909 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.909 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.909 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-10T00:43:42.910 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.910 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.910 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.910 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.910 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-06-10T00:43:42.910 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-06-10T00:43:42.910 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.911 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.911 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.911 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.911 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.911 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.911 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.911 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-10T00:43:42.912 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.912 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.912 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.912 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.912 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.912 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.912 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.912 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-06-10T00:43:42.913 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.913 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.913 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.913 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:42.913 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:42.913 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.913 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.913 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.914 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.914 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.914 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.914 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.914 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.914 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.914 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.914 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.915 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.915 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.915 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.915 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.915 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.915 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.915 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.915 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.916 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:42.916 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:42.916 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.916 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.916 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.916 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.916 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.916 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.917 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.917 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.917 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.917 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.917 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.917 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.917 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:42.917 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:42.918 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.918 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.918 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.918 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:42.918 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.918 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.918 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:42.919 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:42.919 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:42.919 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:42.919 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:42.919 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:42.919 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:42.919 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:42.919 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.920 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.920 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.920 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.920 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.920 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:42.920 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:42.920 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.921 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.921 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.921 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.921 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.921 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.921 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.921 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.921 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.922 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.922 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.922 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.922 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-06-10T00:43:42.922 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-06-10T00:43:42.922 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.922 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.923 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.923 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-06-10T00:43:42.923 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.923 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.923 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.923 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-10T00:43:42.923 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.923 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.923 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.924 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.924 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-06-10T00:43:42.924 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-06-10T00:43:42.924 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:42.924 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.924 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:42.924 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-06-10T00:43:42.924 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.925 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.925 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.925 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-10T00:43:42.925 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.925 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.925 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.925 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.925 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:42.925 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.926 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:42.926 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-06-10T00:43:42.926 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:42.926 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:42.926 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:42.926 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-10T00:43:42.926 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-10T00:43:42.926 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.927 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.927 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.927 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:42.927 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.927 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.927 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.927 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.927 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.928 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.928 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.928 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.928 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-06-10T00:43:42.928 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-10T00:43:42.928 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.928 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.928 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.929 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.929 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.929 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.929 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.929 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.929 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.929 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.929 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.929 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.930 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-10T00:43:42.930 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-10T00:43:42.930 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.930 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.930 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.930 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.930 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.930 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.931 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.931 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.931 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.931 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.931 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.931 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.931 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-06-10T00:43:42.931 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-06-10T00:43:42.932 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.932 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.932 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.932 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.932 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.932 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.932 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.932 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.933 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.933 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.933 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.933 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.933 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-10T00:43:42.933 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-10T00:43:42.933 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.933 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.933 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.934 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.934 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.934 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.934 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.934 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.934 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.934 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.934 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.935 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.935 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-06-10T00:43:42.935 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-10T00:43:42.935 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.935 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.935 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.935 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.935 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.935 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.935 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.936 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.936 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.936 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.936 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.936 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.936 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-10T00:43:42.936 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-10T00:43:42.936 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.937 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.937 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.937 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.937 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.937 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.937 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.937 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.937 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.937 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.938 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.938 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.938 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-10T00:43:42.938 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-10T00:43:42.938 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.938 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.938 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.938 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.938 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.939 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.939 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.939 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.939 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.939 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.939 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.939 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.939 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-10T00:43:42.940 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-10T00:43:42.940 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.940 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.940 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.940 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.940 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.940 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.940 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.941 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.941 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.941 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.941 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.941 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.941 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-10T00:43:42.941 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-10T00:43:42.941 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.941 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.941 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.942 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.942 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.942 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.942 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.942 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.942 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.942 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.943 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.943 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.943 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-10T00:43:42.943 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-10T00:43:42.943 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.943 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.943 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.943 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.944 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.944 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.944 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.944 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.944 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.944 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.944 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.944 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.945 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-10T00:43:42.945 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-10T00:43:42.945 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.945 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.945 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.945 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.945 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.945 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.946 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.946 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.946 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.946 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.946 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.946 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.946 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-10T00:43:42.946 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-10T00:43:42.947 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.947 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.947 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.947 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.947 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.947 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.947 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.947 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.948 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.948 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.948 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.948 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.948 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-10T00:43:42.948 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-10T00:43:42.948 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.948 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.949 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.949 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.949 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.949 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.949 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.949 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.949 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.949 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.949 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.950 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.950 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-06-10T00:43:42.950 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-10T00:43:42.950 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.950 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.950 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.950 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-06-10T00:43:42.950 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.951 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.951 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.951 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.951 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.951 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.951 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.951 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.951 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-10T00:43:42.951 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-10T00:43:42.952 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.952 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.952 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.952 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.952 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.952 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.952 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.952 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.953 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.953 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.953 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.953 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.953 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-06-10T00:43:42.953 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-06-10T00:43:42.953 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.953 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.953 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.954 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.954 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.954 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.954 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.954 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.954 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.954 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.954 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.955 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.955 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-06-10T00:43:42.955 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-10T00:43:42.955 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.955 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.955 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.955 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-06-10T00:43:42.955 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.956 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.956 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.956 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.956 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.956 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.956 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.956 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.956 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-10T00:43:42.957 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-10T00:43:42.957 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.957 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.957 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.957 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.957 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.957 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.957 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.957 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.958 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.958 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.958 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.958 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.958 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-10T00:43:42.958 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-10T00:43:42.958 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.958 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.959 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.959 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.959 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.959 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.959 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.959 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.959 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.960 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.960 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.960 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.960 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-10T00:43:42.960 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-10T00:43:42.960 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.960 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.960 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.961 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.961 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.961 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.961 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.961 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.961 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.961 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.961 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.961 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-10T00:43:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-10T00:43:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.963 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.963 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.963 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.963 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.963 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.963 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.963 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-06-10T00:43:42.963 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-10T00:43:42.964 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.964 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.964 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.964 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.964 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.964 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.964 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.964 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.964 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.965 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.965 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.965 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.965 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-10T00:43:42.965 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-10T00:43:42.965 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.965 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.965 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.965 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.966 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.966 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.966 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.966 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.966 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.966 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.966 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.966 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.967 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-10T00:43:42.967 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-10T00:43:42.967 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.967 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.967 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.967 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.967 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.967 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.968 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.968 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.968 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.968 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.968 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.968 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.968 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-10T00:43:42.968 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-10T00:43:42.969 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.969 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.969 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.969 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.969 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.969 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.969 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.969 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.969 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.969 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.970 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.970 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.970 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-10T00:43:42.970 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-10T00:43:42.970 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.970 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.970 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.970 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.971 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.971 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.971 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.971 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.971 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.971 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.971 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.971 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.972 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-10T00:43:42.972 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-10T00:43:42.972 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.972 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.972 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.972 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.972 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.972 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.973 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.973 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.973 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.973 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.973 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.973 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.973 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-10T00:43:42.973 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-10T00:43:42.973 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.974 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.974 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.974 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.974 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.974 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.974 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.974 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.974 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.975 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.975 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.975 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.975 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-10T00:43:42.975 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-10T00:43:42.975 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.975 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.976 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.976 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.976 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.976 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.976 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.976 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.976 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.976 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.977 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.977 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.977 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-10T00:43:42.977 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-10T00:43:42.977 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.977 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.977 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.977 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:42.977 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.978 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.978 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.978 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.978 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.978 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.978 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.978 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.978 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-06-10T00:43:42.979 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-10T00:43:42.979 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.979 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.979 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.979 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.979 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.979 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.979 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.980 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.980 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.980 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.980 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.980 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.980 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-10T00:43:42.980 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-10T00:43:42.980 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.981 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.981 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.981 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-10T00:43:42.981 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.981 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.981 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.981 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.981 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.982 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.982 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.982 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.982 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-10T00:43:42.982 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-10T00:43:42.982 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.982 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.982 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.983 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:42.983 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.983 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.983 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.983 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.983 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.983 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.983 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.984 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.984 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-10T00:43:42.984 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-10T00:43:42.984 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.984 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.984 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.984 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-06-10T00:43:42.984 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.985 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.985 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.985 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.985 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.985 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.985 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.985 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.985 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-10T00:43:42.986 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-10T00:43:42.986 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.986 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.986 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.986 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-06-10T00:43:42.986 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.986 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.987 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.987 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.987 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.987 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.987 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.987 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.987 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-06-10T00:43:42.987 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-10T00:43:42.988 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.988 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.988 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.988 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.988 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.988 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.988 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.988 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.988 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.989 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.989 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.989 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.989 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-10T00:43:42.989 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-10T00:43:42.989 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.989 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.989 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.990 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-06-10T00:43:42.990 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.990 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.990 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.990 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.990 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.990 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.990 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.991 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.991 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-06-10T00:43:42.991 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-10T00:43:42.991 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:42.991 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.991 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.991 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:42.992 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.992 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.992 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.992 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.992 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.992 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.992 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.992 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.993 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-06-10T00:43:42.993 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-10T00:43:42.993 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:42.993 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.993 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.993 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:42.993 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.993 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.994 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.994 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.994 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.994 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.994 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.994 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.994 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-10T00:43:42.994 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-10T00:43:42.995 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.995 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.995 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.995 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:42.995 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.995 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.995 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.995 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.996 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.996 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.996 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.996 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.996 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-10T00:43:42.996 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-10T00:43:42.996 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.996 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.996 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.997 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:42.997 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.997 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.997 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.997 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.997 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.997 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.997 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.998 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:42.998 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-10T00:43:42.998 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-10T00:43:42.998 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:42.998 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:42.998 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:42.998 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-06-10T00:43:42.998 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:42.999 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:42.999 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:42.999 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:42.999 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:42.999 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:42.999 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:42.999 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.002 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-06-10T00:43:43.002 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-06-10T00:43:43.002 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.002 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.003 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.003 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.003 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.003 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.003 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.003 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.003 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.003 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.003 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.004 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.004 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-06-10T00:43:43.004 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-06-10T00:43:43.004 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.004 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.004 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.004 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.004 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.005 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.005 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.005 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.005 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.005 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.005 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.005 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.005 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-06-10T00:43:43.006 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-06-10T00:43:43.006 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.006 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.006 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.006 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.006 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.006 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.007 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.007 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-06-10T00:43:43.007 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.007 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.007 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.007 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.007 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-06-10T00:43:43.007 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-06-10T00:43:43.008 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.008 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.008 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.008 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-06-10T00:43:43.008 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.008 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.008 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.008 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.008 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.009 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.009 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.009 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.009 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-06-10T00:43:43.009 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-06-10T00:43:43.009 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.009 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.009 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.010 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.010 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.010 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.010 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.010 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.010 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.010 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.010 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.011 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.011 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.011 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.011 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.011 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.011 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.011 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.012 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.012 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.012 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.012 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.012 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.012 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.012 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.012 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.012 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.013 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.013 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.013 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.013 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.013 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.013 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.013 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.013 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.014 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.014 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.014 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.014 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.014 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.014 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.014 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.014 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-06-10T00:43:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-06-10T00:43:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-06-10T00:43:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-06-10T00:43:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-06-10T00:43:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-06-10T00:43:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-06-10T00:43:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-06-10T00:43:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-06-10T00:43:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-06-10T00:43:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-06-10T00:43:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-06-10T00:43:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-06-10T00:43:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-06-10T00:43:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-06-10T00:43:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-06-10T00:43:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.030 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.030 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.030 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-06-10T00:43:43.030 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-06-10T00:43:43.030 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.030 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.030 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.030 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-06-10T00:43:43.031 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.031 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.031 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.031 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.031 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-06-10T00:43:43.031 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.031 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.031 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.032 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.032 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.032 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.032 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-06-10T00:43:43.032 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-06-10T00:43:43.032 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.032 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.032 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.032 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-06-10T00:43:43.033 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-06-10T00:43:43.033 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-06-10T00:43:43.033 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.033 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.033 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.033 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.033 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.033 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.033 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-06-10T00:43:43.034 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-06-10T00:43:43.034 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.034 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.034 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.034 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.034 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.034 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.034 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.035 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.035 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.035 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.035 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.035 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.035 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.035 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.035 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.036 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-06-10T00:43:43.036 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.036 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.036 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.036 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-06-10T00:43:43.036 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-06-10T00:43:43.036 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.036 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.036 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.037 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:43.037 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.037 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.037 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.037 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-06-10T00:43:43.037 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.037 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.037 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.037 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.038 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.038 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.038 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.038 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.038 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.038 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.038 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.039 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.039 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.039 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.039 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.039 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.039 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.039 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.039 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.039 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.040 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.040 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.040 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.040 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.040 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.040 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.040 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.040 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.041 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.041 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.041 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.041 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.041 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.041 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.041 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.041 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.041 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.042 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.042 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.042 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.042 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.042 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.042 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.042 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.042 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.043 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.043 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.043 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.043 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.043 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.043 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.043 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.043 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.044 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.044 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.044 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.044 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.044 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.044 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.044 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.044 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.044 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.045 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.045 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.045 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.045 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.045 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.045 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.045 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.045 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.046 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.046 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.046 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.046 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.046 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-06-10T00:43:43.046 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-06-10T00:43:43.046 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:43.047 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.047 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.047 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-06-10T00:43:43.047 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.047 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.047 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.047 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-10T00:43:43.047 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.047 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.048 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.048 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-06-10T00:43:43.048 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-06-10T00:43:43.048 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.048 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.048 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.048 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-06-10T00:43:43.049 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.049 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.049 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.049 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-10T00:43:43.049 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.049 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.049 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.049 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.050 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-06-10T00:43:43.050 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-06-10T00:43:43.050 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:43.050 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.050 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.050 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-06-10T00:43:43.050 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.050 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.051 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.051 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-06-10T00:43:43.051 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.051 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.051 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.051 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.051 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-06-10T00:43:43.052 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-06-10T00:43:43.052 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:43.052 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.052 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.052 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-06-10T00:43:43.052 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.052 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.052 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.053 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-10T00:43:43.053 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.053 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.053 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.053 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-06-10T00:43:43.053 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-06-10T00:43:43.053 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.054 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.054 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.054 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:43.054 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.054 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.054 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.054 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-10T00:43:43.054 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.055 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.055 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.055 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.055 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:43.055 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:43.055 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:43.055 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.055 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.056 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:43.056 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.056 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.056 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.056 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-06-10T00:43:43.056 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.056 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.056 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.056 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.057 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-06-10T00:43:43.057 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-06-10T00:43:43.057 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:43.057 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.057 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.057 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-06-10T00:43:43.057 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.057 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.058 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.058 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-10T00:43:43.058 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.058 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.058 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.058 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.058 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.059 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.059 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.059 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-06-10T00:43:43.059 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.059 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.059 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.059 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.059 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.059 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.060 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.060 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.060 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.060 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.060 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.060 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.060 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.060 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.061 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.061 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.061 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.061 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.061 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.061 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.061 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.061 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.062 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.062 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.062 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.062 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.062 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.062 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.062 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.062 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.063 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.063 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.063 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.063 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.063 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.063 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.063 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.063 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.064 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.064 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.064 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.064 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.064 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.064 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.064 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.064 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.064 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.065 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.065 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.065 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.065 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.065 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.065 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.065 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.065 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.066 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.066 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.066 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.066 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.066 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.066 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.066 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.066 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.067 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.067 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.067 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.067 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.067 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.067 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.067 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.068 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.068 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.068 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.068 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-06-10T00:43:43.068 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-06-10T00:43:43.068 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.068 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.068 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.069 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-06-10T00:43:43.069 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.069 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.069 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.069 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.069 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.069 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.070 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.070 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.070 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-06-10T00:43:43.070 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-06-10T00:43:43.070 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.070 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.070 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.071 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-06-10T00:43:43.071 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.071 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.071 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.071 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.071 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.071 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.071 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.071 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.072 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:43.072 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:43.072 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.072 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.072 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.072 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-06-10T00:43:43.072 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.072 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.073 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.073 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.073 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.073 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.073 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.073 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.073 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.073 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.074 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.074 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-06-10T00:43:43.074 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-06-10T00:43:43.074 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-06-10T00:43:43.074 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.074 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-06-10T00:43:43.074 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-06-10T00:43:43.075 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.075 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.075 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.075 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-06-10T00:43:43.075 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.075 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.075 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.075 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-10T00:43:43.076 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.076 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.076 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.076 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.076 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-06-10T00:43:43.076 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-06-10T00:43:43.076 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.076 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.077 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.077 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-06-10T00:43:43.077 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.077 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.077 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.077 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-10T00:43:43.077 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.077 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.078 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.078 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.078 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-06-10T00:43:43.078 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-06-10T00:43:43.078 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.078 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.078 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.078 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.079 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.079 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.079 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.079 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-06-10T00:43:43.079 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.079 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.079 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.079 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.080 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:43.080 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:43.080 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:43.080 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.080 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.080 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-06-10T00:43:43.080 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-06-10T00:43:43.080 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.081 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.081 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-10T00:43:43.081 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.081 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.081 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.081 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.081 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.081 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.082 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.082 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.082 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.082 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.082 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.082 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.082 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.082 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.083 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.083 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.083 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.083 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.083 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.083 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.083 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.083 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.084 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.084 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.084 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.084 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.084 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.084 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.084 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.084 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.085 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.085 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.085 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.085 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.085 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.085 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.085 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.085 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.086 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.086 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.086 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.086 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.086 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.086 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.086 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.087 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.087 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.087 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.087 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.087 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.087 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.087 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.087 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.088 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.088 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.088 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.088 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.088 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.088 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.088 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.088 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.088 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.089 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.089 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.089 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.089 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.089 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.089 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.089 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.089 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.090 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.090 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.090 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.090 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.090 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-06-10T00:43:43.090 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-06-10T00:43:43.090 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.090 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.091 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.091 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.091 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.091 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.091 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.091 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-06-10T00:43:43.091 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.091 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.092 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.092 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.092 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-06-10T00:43:43.092 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-06-10T00:43:43.092 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.092 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.092 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.093 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-06-10T00:43:43.093 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.093 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.093 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.093 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-06-10T00:43:43.093 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.093 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.093 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.094 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.094 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-06-10T00:43:43.094 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-06-10T00:43:43.094 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.094 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.094 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.094 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-06-10T00:43:43.094 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.095 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.095 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.095 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T00:43:43.095 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.095 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.095 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.095 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.095 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-06-10T00:43:43.096 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-06-10T00:43:43.096 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.096 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.096 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.096 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-06-10T00:43:43.096 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-06-10T00:43:43.096 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-06-10T00:43:43.096 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.096 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-10T00:43:43.097 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.097 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.097 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.097 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.097 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-06-10T00:43:43.097 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-06-10T00:43:43.097 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.097 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.098 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.098 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.098 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.098 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.098 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.098 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-06-10T00:43:43.098 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.098 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.099 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.099 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.099 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-06-10T00:43:43.099 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-06-10T00:43:43.099 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.099 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.099 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.099 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-06-10T00:43:43.100 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.100 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.100 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.100 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T00:43:43.100 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.100 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.100 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.100 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.101 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.101 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.101 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.101 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-06-10T00:43:43.101 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.101 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.101 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.101 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.102 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.102 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.102 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.102 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.102 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.102 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.102 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.102 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.103 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.103 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.103 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.103 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.103 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.103 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.103 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.103 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.104 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.104 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.104 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.104 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.104 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.104 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.104 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.104 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.104 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.105 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.105 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.105 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.105 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.105 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.105 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.105 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.105 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.106 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.106 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.106 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.106 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.106 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.106 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.106 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.106 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.107 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.107 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.107 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.107 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.107 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.107 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.107 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.107 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.107 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.108 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.108 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.108 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.108 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.108 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.108 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.108 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.108 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.109 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.109 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.109 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.109 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.109 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.109 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.109 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.109 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.110 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.110 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.110 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.110 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.110 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.110 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.110 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-06-10T00:43:43.110 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.111 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.111 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.111 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.111 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.111 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.111 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.111 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.111 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.112 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.112 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.112 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.112 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.112 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.112 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.112 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.112 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.112 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.113 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.113 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.113 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.113 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.113 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.113 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.113 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.113 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.114 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.114 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.114 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.114 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.114 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.114 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.114 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.114 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.115 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.115 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.115 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.115 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.115 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.115 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.115 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.115 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.116 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.116 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.116 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.116 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.116 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.116 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.116 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.116 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.117 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.117 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.117 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.117 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.117 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.117 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.117 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.117 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.118 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.118 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.118 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.118 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.118 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.118 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.118 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.118 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.119 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.119 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.119 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.119 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.119 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.119 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.119 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.119 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.120 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.120 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-06-10T00:43:43.120 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.120 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.120 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-06-10T00:43:43.120 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-06-10T00:43:43.121 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.121 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.121 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.121 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-06-10T00:43:43.121 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.121 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.121 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.121 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-10T00:43:43.122 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.122 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.122 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.122 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.122 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-06-10T00:43:43.122 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-06-10T00:43:43.122 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.123 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.123 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.123 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:43.123 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-06-10T00:43:43.123 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.123 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.123 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-10T00:43:43.123 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.124 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.124 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.124 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.124 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.124 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.124 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.124 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.124 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.124 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.125 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.125 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.125 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.125 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.125 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.125 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.125 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.125 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.126 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.126 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.126 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.126 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.126 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.126 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.126 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.127 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.127 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.127 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.127 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.127 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.127 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.127 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.127 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.127 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.128 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.128 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.128 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.128 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.128 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.128 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.128 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.128 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.129 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.129 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.129 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.129 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.129 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.129 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.129 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.129 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.130 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.130 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.130 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.130 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.130 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.130 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.130 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.131 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.131 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.131 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.131 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.131 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.131 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.131 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.131 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.131 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.132 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.132 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.132 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.132 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.132 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.132 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.132 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.132 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.133 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.133 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.133 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.133 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-06-10T00:43:43.133 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.133 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.133 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.133 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-06-10T00:43:43.134 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-06-10T00:43:43.134 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.134 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.134 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.134 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-06-10T00:43:43.134 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.134 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.134 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.135 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-06-10T00:43:43.135 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-10T00:43:43.135 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.135 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.135 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.135 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.135 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.135 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.135 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.136 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.136 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.136 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.136 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.136 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.136 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.136 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.136 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.137 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.137 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.137 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.137 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.137 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.137 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.137 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.137 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.138 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.138 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.138 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.138 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.138 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.138 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.138 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.138 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.139 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.139 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.139 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.139 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.139 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.139 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.139 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.139 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.140 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.140 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.140 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.140 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.140 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.140 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.140 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.140 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.141 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.141 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.141 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.141 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.141 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.141 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.141 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.141 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.141 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.142 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.142 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.142 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.142 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.142 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.142 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.142 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.142 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.143 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.143 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.143 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.143 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.143 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.143 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.143 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.143 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.144 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.144 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-06-10T00:43:43.144 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-06-10T00:43:43.144 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.144 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.144 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.144 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.145 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.145 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.145 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.145 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-06-10T00:43:43.145 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-10T00:43:43.145 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.145 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.145 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.146 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-06-10T00:43:43.146 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-06-10T00:43:43.146 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.146 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.146 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.146 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-06-10T00:43:43.146 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.146 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.147 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.147 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-06-10T00:43:43.147 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.147 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.147 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.147 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.147 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-06-10T00:43:43.147 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-06-10T00:43:43.147 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.148 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.148 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.148 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-06-10T00:43:43.148 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.148 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.148 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.148 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-06-10T00:43:43.148 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.149 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.149 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.149 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.149 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-06-10T00:43:43.149 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-06-10T00:43:43.149 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.149 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.149 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.150 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.150 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.150 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.150 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.150 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-06-10T00:43:43.150 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.150 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.151 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.151 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.151 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-06-10T00:43:43.151 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-06-10T00:43:43.151 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.151 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.151 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.151 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-06-10T00:43:43.151 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.152 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.152 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.152 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-10T00:43:43.152 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-10T00:43:43.152 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.152 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.152 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.152 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.153 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.153 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.153 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-06-10T00:43:43.153 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.153 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.153 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.153 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.153 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.154 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.154 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.154 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.154 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.154 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.154 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.154 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.154 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.155 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.155 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.155 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.155 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.155 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.155 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.155 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.155 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.156 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.156 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.156 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.156 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.156 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.156 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.156 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.156 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.157 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.157 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.157 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.157 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.157 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.157 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.157 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.157 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.158 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.158 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.158 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.158 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.158 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.158 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.158 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.158 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.159 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.159 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.159 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.159 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.159 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.159 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.159 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.159 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.159 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.160 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.160 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.160 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.160 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.160 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.160 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.160 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.160 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.161 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.161 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.161 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.161 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.161 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.161 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.161 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.161 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.162 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.162 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.162 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.162 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.162 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.162 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.162 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-06-10T00:43:43.162 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.162 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.163 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.163 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.163 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.163 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.163 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.163 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.163 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.163 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.164 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.164 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.164 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.164 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.164 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.164 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.164 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.164 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.165 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.165 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.165 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.165 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.165 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.165 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.165 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.165 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.166 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.166 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.166 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.166 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.166 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.166 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.166 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.166 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.167 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.167 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.167 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.167 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.167 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.167 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.167 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.167 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.168 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.168 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.168 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.168 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.168 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.168 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.168 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.168 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.168 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.169 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.169 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.169 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.169 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.169 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.169 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.169 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.169 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.170 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.170 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.170 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.170 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.170 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.170 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.170 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.171 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.171 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.171 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.171 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.171 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.171 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.171 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.171 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.171 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.172 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.172 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-06-10T00:43:43.172 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.172 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.172 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.172 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.172 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.172 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.173 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.173 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.173 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.173 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.173 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.173 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.173 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.173 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.173 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.174 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.174 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.174 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.174 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.174 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.174 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.174 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.174 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.175 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.175 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.175 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.175 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.175 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.175 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.175 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.175 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.176 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.176 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.176 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.176 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.176 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.176 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.176 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.176 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.177 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.177 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.177 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.177 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.177 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.177 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.177 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.177 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.178 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.178 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.178 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.178 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.178 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.178 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.178 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.178 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.179 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.179 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.179 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.179 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.179 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.179 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.179 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.179 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.180 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.180 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.180 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.180 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.180 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.180 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.180 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.180 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.180 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.181 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.181 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.181 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.181 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.181 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-06-10T00:43:43.181 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.181 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.181 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.182 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-06-10T00:43:43.182 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-06-10T00:43:43.182 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.182 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.182 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.182 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.182 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.182 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-10T00:43:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.183 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.184 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.184 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.184 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.184 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.184 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.184 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.184 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.184 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.185 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.185 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.185 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.185 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.185 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.185 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.185 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.185 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.186 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.186 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.186 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.186 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.186 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.186 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.186 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.186 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.187 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.187 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.187 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.187 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.187 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.187 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.187 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.187 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.188 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.188 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.188 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.188 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.188 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.188 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.188 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.188 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.188 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.189 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.189 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.189 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.189 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.189 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.189 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.189 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.189 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.190 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.190 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.190 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.190 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.190 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.190 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.190 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.190 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.191 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.191 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.191 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.191 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.191 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.191 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.191 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.191 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.192 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.192 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.192 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.192 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-06-10T00:43:43.192 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-06-10T00:43:43.192 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.192 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.192 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.193 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.193 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.193 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.193 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.193 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-06-10T00:43:43.193 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-06-10T00:43:43.193 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-06-10T00:43:43.193 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.193 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-06-10T00:43:43.194 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.194 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.194 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.194 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.194 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.194 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.194 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.194 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-06-10T00:43:43.195 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.195 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.195 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.195 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.195 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.195 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.195 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.196 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.196 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.196 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.196 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.196 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.196 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.196 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.196 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.196 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.197 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.197 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.197 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.197 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.197 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.197 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.197 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.197 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.198 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.198 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.198 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.198 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.198 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.198 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.198 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.198 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.199 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.199 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.199 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.199 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.199 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.199 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.199 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.199 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.199 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.200 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.200 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.200 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.200 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.200 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.200 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.200 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.200 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.201 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.201 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.201 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.201 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.201 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.201 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.201 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.201 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.202 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.202 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.202 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.202 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.202 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.202 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.202 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.202 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.203 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.203 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.203 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.203 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.205 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.205 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.205 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.205 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.206 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.206 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.206 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.206 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-06-10T00:43:43.206 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.206 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.206 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.206 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.207 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.207 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.207 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.207 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.207 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.207 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.207 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.207 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.208 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.208 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.208 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.208 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.208 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.208 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.208 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.208 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.208 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.209 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.209 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.209 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.209 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.209 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.209 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.209 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.209 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.210 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.210 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.210 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.210 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.210 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.210 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.210 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.210 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.211 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.211 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.211 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.211 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.211 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.211 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.211 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.211 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.212 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.212 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.212 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.212 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.212 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.212 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.212 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.212 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.213 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.213 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.213 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.213 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.213 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.213 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.213 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.213 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.214 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.214 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.214 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.214 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.214 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.214 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.214 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.214 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.215 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.215 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.215 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.215 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.215 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.215 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.215 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.215 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-06-10T00:43:43.215 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.216 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.216 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.216 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.216 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.216 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.216 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.216 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.216 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.217 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.217 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.217 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.217 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.217 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.217 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.217 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.217 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.218 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.219 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.219 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.219 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.219 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.219 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.219 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.219 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.219 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.220 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.220 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.220 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.220 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.220 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.220 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.220 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.220 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.221 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.221 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.221 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.221 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.221 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.221 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.221 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.221 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.222 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.222 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.222 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.222 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.222 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.222 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.222 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.222 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.223 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.223 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.223 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.223 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.223 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.223 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.223 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.223 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.224 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.224 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.224 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.224 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.224 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.224 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.224 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.224 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.225 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:43.225 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:43.225 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:43.225 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.225 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.225 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:43.225 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.225 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.226 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.226 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.226 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.226 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.226 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.226 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.226 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-06-10T00:43:43.226 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-06-10T00:43:43.227 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:43.227 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.227 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.227 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-06-10T00:43:43.227 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-06-10T00:43:43.227 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.227 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.227 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-06-10T00:43:43.228 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-10T00:43:43.228 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.228 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.228 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.228 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.228 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.228 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.228 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-06-10T00:43:43.229 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.229 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.229 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.229 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-06-10T00:43:43.229 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-06-10T00:43:43.229 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.229 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.229 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.229 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.230 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.230 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.230 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.230 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-10T00:43:43.230 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.230 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.230 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.230 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.231 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-06-10T00:43:43.231 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-06-10T00:43:43.231 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.231 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.231 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.231 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:43.231 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.231 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.232 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.232 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.232 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.232 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.232 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.232 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.232 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.232 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.233 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.233 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.233 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.233 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.233 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.233 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.233 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.233 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.234 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.234 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.234 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.234 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.234 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.234 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.234 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.234 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.235 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.235 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.235 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.235 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.235 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.235 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.235 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.235 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.236 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.236 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.236 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.236 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.236 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.236 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.236 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.236 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.237 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.237 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.237 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.237 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.237 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.237 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.237 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.238 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.238 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.238 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.238 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.238 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.238 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.238 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.238 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.239 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.239 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.239 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.239 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.239 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.239 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.239 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.239 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.240 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.240 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.240 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.240 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.240 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.240 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.240 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.240 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.241 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.241 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.241 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.241 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.241 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.241 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-06-10T00:43:43.241 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-06-10T00:43:43.241 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.242 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.242 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.242 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-06-10T00:43:43.242 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.242 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.242 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.242 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-06-10T00:43:43.242 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.243 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.243 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.243 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.243 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-06-10T00:43:43.243 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-06-10T00:43:43.243 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:43.243 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.243 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.244 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-06-10T00:43:43.244 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.244 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.244 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.244 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-06-10T00:43:43.244 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.244 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.244 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.245 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.245 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.245 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.245 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.245 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-06-10T00:43:43.245 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.245 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.245 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.245 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-06-10T00:43:43.246 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-06-10T00:43:43.246 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.246 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.246 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.246 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:43.246 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.246 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.247 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.247 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.247 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.247 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.247 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.247 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.247 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-06-10T00:43:43.247 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-06-10T00:43:43.248 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.248 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.248 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.248 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:43.248 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.248 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.248 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.248 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-10T00:43:43.249 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-10T00:43:43.249 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.249 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.249 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.249 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.249 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.249 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.250 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.250 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.250 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.250 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.250 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.250 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.250 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.250 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.251 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.251 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.251 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.251 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.251 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.251 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.251 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.251 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.252 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.252 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.252 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.252 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.252 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.252 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.252 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.253 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.253 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.253 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.253 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.253 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.253 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.253 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.253 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.254 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.254 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.254 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.254 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.254 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.254 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.254 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.254 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.255 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.255 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.255 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.255 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.255 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.255 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.255 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.255 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.256 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.256 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.256 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.256 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.256 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.256 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.256 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.256 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.257 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.257 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.257 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.257 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.257 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.257 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.257 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.257 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.258 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.258 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.258 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.258 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.258 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-06-10T00:43:43.258 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-06-10T00:43:43.258 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.258 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.259 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.259 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.259 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.259 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.259 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.259 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.259 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.259 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.260 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.260 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.260 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-10T00:43:43.260 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-10T00:43:43.260 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.260 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.260 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.260 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-06-10T00:43:43.261 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.261 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.261 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.261 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.261 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.261 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.261 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.261 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.262 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-06-10T00:43:43.262 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-06-10T00:43:43.262 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:43.262 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.262 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.262 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-06-10T00:43:43.262 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.262 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.263 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.263 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.263 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.263 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.263 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.263 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.263 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-06-10T00:43:43.263 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-06-10T00:43:43.264 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.264 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.264 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.264 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-06-10T00:43:43.264 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.264 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.264 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.264 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-10T00:43:43.265 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.265 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.265 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.265 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.265 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-06-10T00:43:43.265 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-06-10T00:43:43.265 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.265 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.265 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.266 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-06-10T00:43:43.266 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.266 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.266 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.266 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-10T00:43:43.266 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.266 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.267 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.267 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.267 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-06-10T00:43:43.267 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-06-10T00:43:43.267 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.267 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.267 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.267 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-06-10T00:43:43.268 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.268 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.268 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.268 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.268 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.268 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.268 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.268 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.269 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-06-10T00:43:43.269 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-06-10T00:43:43.269 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.269 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.269 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.269 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-06-10T00:43:43.269 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.269 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.270 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.270 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-06-10T00:43:43.270 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-06-10T00:43:43.270 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.270 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.270 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.270 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.270 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.271 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.271 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-06-10T00:43:43.271 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-06-10T00:43:43.271 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.271 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.271 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.271 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-06-10T00:43:43.271 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-06-10T00:43:43.272 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-06-10T00:43:43.272 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.272 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.272 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.272 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.272 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.272 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.272 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.273 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.273 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.273 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-06-10T00:43:43.273 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.273 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.273 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.273 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.273 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.274 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.274 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.274 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.274 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.274 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.274 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.274 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.275 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.275 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.275 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.275 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.275 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.275 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.275 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.275 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.276 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.276 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.276 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.276 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.276 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.276 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.276 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.276 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.277 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.277 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.277 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.277 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.277 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.278 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.278 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.278 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.278 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.278 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.278 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.278 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.278 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.279 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.279 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.279 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.279 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.279 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.279 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.279 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.279 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.280 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.280 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.280 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.280 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.280 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.280 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.280 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.280 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.280 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.281 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.281 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.281 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.281 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.281 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.281 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.281 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.281 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.282 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.282 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.282 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.282 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.282 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.282 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.282 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.283 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-06-10T00:43:43.283 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-06-10T00:43:43.283 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.283 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.283 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.283 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-06-10T00:43:43.283 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.283 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.283 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.284 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.284 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.284 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.284 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.284 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.284 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-06-10T00:43:43.284 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-06-10T00:43:43.284 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.285 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.285 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.285 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.285 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.285 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.285 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.285 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.285 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.286 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.286 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.286 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.286 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-06-10T00:43:43.286 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-06-10T00:43:43.286 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.286 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.286 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.287 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.287 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.287 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.287 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.287 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.287 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.287 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.287 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.288 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.288 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.288 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.288 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.288 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-06-10T00:43:43.288 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.288 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.288 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.288 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-06-10T00:43:43.289 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-06-10T00:43:43.289 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.289 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.289 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.289 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:43.289 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.289 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.289 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.290 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.290 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.290 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.290 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.290 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.290 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-06-10T00:43:43.290 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-06-10T00:43:43.290 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.291 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.291 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.291 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.291 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.291 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.291 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.291 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.291 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.292 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.292 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.292 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.292 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.292 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.292 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.292 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.292 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.293 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.293 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.293 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.293 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.293 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.293 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.293 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.293 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.294 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.294 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.294 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.294 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.294 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.294 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.294 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.294 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.295 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.295 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.295 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.295 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.295 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.295 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.295 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.295 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.296 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.296 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.296 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.296 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.296 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.296 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.296 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.296 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.297 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.297 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.297 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.297 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.297 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.297 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.297 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.297 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.298 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.298 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.298 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.298 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.298 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.298 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.298 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.298 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.299 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.299 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.299 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.299 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.299 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.299 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.299 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.299 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.299 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.300 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.300 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.300 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.300 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.300 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.300 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.300 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.300 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.300 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-06-10T00:43:43.301 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-06-10T00:43:43.301 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.301 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.301 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.301 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.301 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.301 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.301 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.302 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.302 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.302 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.302 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.302 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.302 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-06-10T00:43:43.302 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-06-10T00:43:43.302 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.303 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.303 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.303 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.303 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.303 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.303 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.303 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.303 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.303 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.304 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.304 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.304 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.304 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.304 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.304 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-06-10T00:43:43.304 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.304 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.305 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.305 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.305 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.305 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.305 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.305 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.305 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.305 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.305 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.306 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.306 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.306 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.306 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.306 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.306 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.306 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.307 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.307 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.307 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.307 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.307 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.307 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.307 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.307 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.308 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.308 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.308 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.308 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.308 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.308 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.308 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.308 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.308 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.309 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.309 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.309 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.309 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.309 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.309 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.309 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.309 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.309 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.310 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.310 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.310 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.310 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.310 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.310 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.310 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.311 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.311 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.311 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.311 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.311 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.311 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.311 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.311 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.312 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.312 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.312 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.312 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.312 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.312 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.312 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.312 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.312 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.313 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.313 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.313 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.313 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.313 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.313 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.313 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.313 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.313 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-06-10T00:43:43.314 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.314 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.314 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.314 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.314 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.314 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.314 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.314 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.315 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.315 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.315 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.315 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.315 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.315 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.315 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.315 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.316 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.316 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.316 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.316 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.316 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.316 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.316 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.316 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.317 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.317 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.317 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.317 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.317 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.317 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.317 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.317 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.317 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.318 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.318 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.318 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.318 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.318 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.318 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.318 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.319 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.319 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.319 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.319 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.319 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.319 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.319 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.319 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.320 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.320 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.320 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.320 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.320 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.320 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.320 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.320 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.321 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.321 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.321 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.321 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.321 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.321 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.321 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.321 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.322 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.322 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.322 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.322 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.322 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.322 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.322 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.322 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.323 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.323 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-06-10T00:43:43.323 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-06-10T00:43:43.323 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.323 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.323 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.323 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-06-10T00:43:43.323 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.324 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.324 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.324 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-10T00:43:43.324 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.324 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.324 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.324 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.324 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.325 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.325 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.325 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-06-10T00:43:43.325 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.325 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.325 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.325 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.326 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.326 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.326 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.326 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.326 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.326 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.326 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.326 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.327 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.327 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.327 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.327 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.327 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.327 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.327 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.328 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.328 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.328 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.328 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.328 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.328 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.328 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.328 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.329 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.329 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.329 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.329 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.329 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.329 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.329 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.329 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.330 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.330 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.330 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.330 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.330 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.330 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.330 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.330 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.331 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.331 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.331 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.331 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.331 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.331 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.331 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.331 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.332 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.332 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.332 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.332 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.332 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.332 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.332 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.332 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.332 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.333 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.333 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.333 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.333 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.333 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.333 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.333 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.333 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.333 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.334 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.334 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.334 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.334 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.334 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-06-10T00:43:43.334 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-06-10T00:43:43.334 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.335 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.335 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.335 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.335 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.335 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.335 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.335 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.336 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.336 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.336 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.336 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.336 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-06-10T00:43:43.336 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-06-10T00:43:43.336 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.336 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.337 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.337 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-06-10T00:43:43.337 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.337 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.337 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.337 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.337 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.337 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.338 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.338 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.338 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-06-10T00:43:43.338 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-06-10T00:43:43.338 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.338 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.338 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.338 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.339 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.339 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.339 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.339 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.339 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.339 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.339 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.339 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.340 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-06-10T00:43:43.340 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-06-10T00:43:43.340 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.340 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.340 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.340 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.340 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.340 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.340 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.341 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.341 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.341 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.341 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.341 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.341 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-06-10T00:43:43.341 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-06-10T00:43:43.341 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-06-10T00:43:43.342 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.342 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.342 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.342 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.342 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.342 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.342 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.342 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.343 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.343 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.343 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.343 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-06-10T00:43:43.343 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-06-10T00:43:43.343 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.343 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.343 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.344 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.344 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.344 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.344 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.344 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.344 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.344 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.344 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.345 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.345 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-06-10T00:43:43.345 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-06-10T00:43:43.345 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.345 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.345 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.345 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.345 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.346 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.346 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.346 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.346 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.346 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.346 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.346 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.347 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-06-10T00:43:43.347 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-06-10T00:43:43.347 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.347 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.347 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.347 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:43.347 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.347 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.348 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.348 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.348 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.348 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.348 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.348 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.348 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-06-10T00:43:43.348 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-06-10T00:43:43.349 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.349 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.349 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.349 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.349 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-06-10T00:43:43.349 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-06-10T00:43:43.349 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.349 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.350 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.350 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.350 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.350 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.350 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-06-10T00:43:43.350 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-06-10T00:43:43.350 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.350 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.351 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.351 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.351 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.351 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.351 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.351 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.351 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.351 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.352 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.352 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.352 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-06-10T00:43:43.352 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-06-10T00:43:43.352 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.352 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.352 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.352 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.353 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.353 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.353 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.353 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.353 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.353 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.353 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.354 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.354 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-06-10T00:43:43.354 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-06-10T00:43:43.354 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.354 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.354 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.354 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.354 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.355 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.355 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.355 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.355 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.355 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.355 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.355 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.355 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.356 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.356 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.356 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-06-10T00:43:43.356 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.356 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.356 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.356 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-06-10T00:43:43.356 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-06-10T00:43:43.357 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.357 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.357 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.357 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.357 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.357 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.357 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.357 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-06-10T00:43:43.358 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.358 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.358 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.358 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.358 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-06-10T00:43:43.358 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-06-10T00:43:43.358 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.358 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.359 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.359 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.359 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.359 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.359 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.359 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-06-10T00:43:43.359 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.360 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.360 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.360 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.360 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.360 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.360 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.360 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.360 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.361 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.361 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.361 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.361 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.361 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.361 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.361 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.362 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.362 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.362 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.362 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.362 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.362 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.362 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.362 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.363 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.363 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.363 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.363 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.363 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.363 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.363 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.364 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.364 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.364 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.364 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.364 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.364 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.364 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.364 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.365 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.365 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.365 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.365 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.365 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.365 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.365 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.366 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.366 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.366 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.366 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.366 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.366 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.366 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.366 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.367 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.367 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.367 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.367 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.367 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.367 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.367 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.368 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.368 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.368 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.368 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.368 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.368 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.368 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.368 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.369 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.369 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.369 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.369 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.369 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.369 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.369 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.370 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.370 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-06-10T00:43:43.370 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.370 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.370 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.370 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.370 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.370 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.371 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.371 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.371 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.371 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.371 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.371 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.371 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.371 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.372 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.372 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.372 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.372 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.372 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.372 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.372 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.372 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.373 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.373 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.373 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.373 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.373 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.373 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.373 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.373 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.374 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.374 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.374 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.374 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.374 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.374 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.374 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.375 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.375 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.375 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.375 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.375 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.375 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.375 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.375 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.376 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.376 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.376 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.376 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.376 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.376 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.376 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.376 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.377 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.377 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.377 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.377 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.377 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.377 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.377 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.377 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.377 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.378 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.378 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.378 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.378 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.378 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.378 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.378 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.378 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.379 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.379 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.379 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.379 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.379 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.379 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.379 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-06-10T00:43:43.379 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.379 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.380 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.380 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.380 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.380 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.380 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.380 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.380 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.380 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.381 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.381 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.381 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.381 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.381 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.381 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.381 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.381 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.381 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.382 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.382 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.382 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.382 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.382 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.382 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.382 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.382 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.383 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.383 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.383 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.383 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.383 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.383 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.383 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.383 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.384 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.384 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.384 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.384 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.384 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.384 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.384 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.384 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.384 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.385 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.385 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.385 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.385 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.385 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.385 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.385 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.385 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.386 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.386 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.386 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.386 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.386 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.386 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.386 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.387 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.387 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.387 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.387 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.387 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.387 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.387 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.387 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.387 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.388 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.388 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.388 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.388 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.388 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.388 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.388 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.388 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.388 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-06-10T00:43:43.389 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.389 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.389 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.389 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-06-10T00:43:43.389 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-06-10T00:43:43.389 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.389 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.389 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.390 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-10T00:43:43.390 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.390 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.390 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.390 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.390 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.390 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.390 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.391 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.391 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:43.391 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:43.391 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:43.391 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.391 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.391 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-06-10T00:43:43.391 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.392 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.392 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.392 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.392 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.392 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.392 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.392 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.392 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.393 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.393 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.393 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.393 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.393 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.393 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.393 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.393 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.394 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.394 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.394 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.394 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.394 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.394 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.394 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.394 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.395 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.395 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.395 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.395 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.395 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.395 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.395 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.395 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.396 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.396 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.396 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.396 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.396 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.396 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.396 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.396 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.396 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.397 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.397 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.397 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.397 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.397 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.397 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.397 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.397 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.398 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.398 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.398 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.398 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.398 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.398 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.398 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.398 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.398 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.399 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.399 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.399 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.399 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.399 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.399 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.399 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.399 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.399 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.400 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.400 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.400 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.400 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.400 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.400 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.400 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.400 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.400 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.401 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.401 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.401 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.401 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.401 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-06-10T00:43:43.401 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.401 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.401 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.402 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-06-10T00:43:43.402 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-06-10T00:43:43.402 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.402 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.402 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.402 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:43.402 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.402 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.403 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.403 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-10T00:43:43.403 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.403 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.403 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.403 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.403 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-06-10T00:43:43.403 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-06-10T00:43:43.404 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.404 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.404 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.404 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:43.404 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.404 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.404 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.405 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-10T00:43:43.405 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.405 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.405 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.405 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.405 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-06-10T00:43:43.405 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-06-10T00:43:43.406 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.406 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.406 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.406 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:43.406 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.406 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.406 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.407 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-10T00:43:43.407 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.407 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.407 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.407 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.407 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-06-10T00:43:43.407 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-06-10T00:43:43.408 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.408 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.408 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.408 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.408 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.408 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.408 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.408 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-10T00:43:43.409 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.409 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.409 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.409 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.409 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-06-10T00:43:43.409 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-06-10T00:43:43.409 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.410 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.410 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.410 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.410 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.410 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.413 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.413 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-10T00:43:43.413 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.414 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.414 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.414 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.414 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-06-10T00:43:43.414 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-06-10T00:43:43.414 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.414 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.414 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.415 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.415 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.415 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.415 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.415 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.415 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.415 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.416 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.416 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.416 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-06-10T00:43:43.416 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-06-10T00:43:43.416 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.416 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.416 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.416 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.417 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.417 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.417 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.417 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.417 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.417 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.417 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.417 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.418 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-06-10T00:43:43.418 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-06-10T00:43:43.418 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.418 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.418 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.418 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-10T00:43:43.418 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.418 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.419 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.419 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.419 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.419 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.419 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.419 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.419 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-06-10T00:43:43.419 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-06-10T00:43:43.420 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.420 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.420 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.420 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.420 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.420 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.420 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.420 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.421 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.421 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.421 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.421 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.421 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:43.421 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:43.421 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.421 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.422 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.422 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-06-10T00:43:43.422 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-06-10T00:43:43.422 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.422 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.422 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.422 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.423 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.423 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.423 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-06-10T00:43:43.423 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-06-10T00:43:43.423 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.423 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.423 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.424 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-06-10T00:43:43.424 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.424 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.424 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.424 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.424 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.424 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.424 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.425 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.425 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-06-10T00:43:43.425 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-06-10T00:43:43.425 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.425 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.425 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.425 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.426 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.426 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.426 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.426 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.426 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.426 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.426 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.426 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.427 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-06-10T00:43:43.427 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-06-10T00:43:43.427 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.427 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.427 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.427 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.427 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.428 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.428 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.428 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.428 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.428 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.428 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.428 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.428 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.429 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.429 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.429 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.429 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.429 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.429 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.429 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.429 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.430 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.430 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.430 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.430 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.430 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.430 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.430 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.430 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.431 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.431 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.431 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.431 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.431 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.431 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.431 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.431 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.432 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.432 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.432 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.432 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.432 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.432 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.432 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.433 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.433 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.433 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.433 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.433 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.433 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.433 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.433 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.434 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.434 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.434 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.434 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.434 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.434 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.434 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.434 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.435 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.435 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.435 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.435 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.435 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.435 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.435 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.435 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.436 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.436 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.436 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.436 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.436 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.436 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.436 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.437 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.437 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.437 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.437 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.437 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.437 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.437 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.437 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-06-10T00:43:43.437 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-06-10T00:43:43.438 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.438 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.438 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.438 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.438 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.438 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.438 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.439 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.439 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.439 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.439 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.439 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.439 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-06-10T00:43:43.439 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-06-10T00:43:43.439 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.440 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.440 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.440 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.440 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.440 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.440 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.440 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.441 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.441 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.441 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.441 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.441 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-06-10T00:43:43.441 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-06-10T00:43:43.441 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.442 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.442 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.442 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-10T00:43:43.442 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.442 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.442 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.442 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.442 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.443 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.443 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.443 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.443 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.443 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.443 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.443 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-06-10T00:43:43.443 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.444 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.444 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.444 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.444 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.444 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.444 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.444 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.444 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.444 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.445 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.445 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.445 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.445 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.445 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.445 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.445 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.446 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.446 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.446 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.446 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.446 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.446 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.446 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.446 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.447 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.447 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.447 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.447 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.447 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.447 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.447 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.447 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.448 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.448 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.448 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.448 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.448 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.448 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.448 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.448 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.449 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.449 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.449 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.449 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.449 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.449 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.449 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.449 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.450 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.450 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.450 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.450 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.450 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.450 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.450 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.451 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.451 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.451 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.451 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.451 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.451 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.451 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.451 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.452 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.452 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.452 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.452 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.452 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.452 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.452 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.452 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.453 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.453 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.453 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.453 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.453 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-06-10T00:43:43.453 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.453 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.453 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.454 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.454 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.454 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.454 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.454 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.454 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.454 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.454 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.455 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.455 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.455 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.455 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.455 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.455 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.455 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.455 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.456 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.456 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.456 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.456 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.456 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.456 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.456 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.457 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.457 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.457 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.457 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.457 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.457 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.457 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.457 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.458 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.458 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.458 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.458 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.458 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.458 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.458 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.458 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.459 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.459 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.459 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.459 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.459 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.459 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.459 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.459 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.460 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.460 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.460 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.460 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.460 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.460 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.460 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.461 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.461 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.461 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.461 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.461 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.461 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.461 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.461 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.462 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.462 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.462 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.462 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.462 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.462 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.462 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.462 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.463 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-06-10T00:43:43.463 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-06-10T00:43:43.463 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.463 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.463 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.463 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-06-10T00:43:43.463 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.464 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.464 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.464 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-10T00:43:43.464 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.464 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.464 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.464 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.464 INFO:teuthology.orchestra.run.smithi062.stdout: "periodic_async_work": { 2024-06-10T00:43:43.465 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "periodic_async_work", 2024-06-10T00:43:43.465 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.465 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.465 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.465 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.465 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.465 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.465 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.466 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Periodically check for async work", 2024-06-10T00:43:43.466 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.466 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.466 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.466 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.466 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-06-10T00:43:43.466 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-06-10T00:43:43.467 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.467 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.467 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.467 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-06-10T00:43:43.467 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.467 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.467 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.467 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-10T00:43:43.468 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.468 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.468 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.468 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.468 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-06-10T00:43:43.468 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-06-10T00:43:43.468 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.468 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.469 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.469 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-06-10T00:43:43.469 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.469 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.469 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.469 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-10T00:43:43.469 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.469 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.470 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.470 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.470 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.470 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.470 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-06-10T00:43:43.470 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-06-10T00:43:43.470 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-06-10T00:43:43.471 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-06-10T00:43:43.471 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-06-10T00:43:43.471 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-06-10T00:43:43.471 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-06-10T00:43:43.471 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.471 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.471 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-06-10T00:43:43.471 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.472 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.472 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.472 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.472 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.472 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.472 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.472 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.472 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-06-10T00:43:43.473 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-06-10T00:43:43.473 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.473 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.473 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.473 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.473 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.473 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.473 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.474 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.474 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.474 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.474 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.474 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.474 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-06-10T00:43:43.474 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-06-10T00:43:43.475 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-06-10T00:43:43.475 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.475 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.475 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-06-10T00:43:43.475 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.475 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.475 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.475 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.476 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.476 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.476 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.476 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.476 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-06-10T00:43:43.476 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-06-10T00:43:43.476 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.476 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.477 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.477 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.477 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.477 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.477 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.477 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.477 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.477 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.478 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.478 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.478 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.478 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.478 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.478 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.478 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.478 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.479 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.479 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-06-10T00:43:43.479 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-06-10T00:43:43.479 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.479 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.479 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.479 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.479 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.480 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.480 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.480 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.480 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.480 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.480 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.480 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.480 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-06-10T00:43:43.481 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-06-10T00:43:43.481 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.481 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.481 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.481 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-06-10T00:43:43.481 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.481 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.481 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-06-10T00:43:43.482 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-06-10T00:43:43.482 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-06-10T00:43:43.482 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-06-10T00:43:43.482 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-06-10T00:43:43.482 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-06-10T00:43:43.482 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-06-10T00:43:43.482 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.482 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.483 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.483 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.483 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.483 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.483 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-06-10T00:43:43.483 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-06-10T00:43:43.483 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-06-10T00:43:43.483 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.484 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-06-10T00:43:43.484 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-06-10T00:43:43.484 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.484 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.484 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.484 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.484 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.484 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.485 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.485 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.485 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-06-10T00:43:43.485 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-06-10T00:43:43.485 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.485 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.485 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.485 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-06-10T00:43:43.486 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.486 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.486 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.486 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.486 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.486 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.486 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.486 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.487 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-06-10T00:43:43.487 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-06-10T00:43:43.487 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-06-10T00:43:43.487 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.487 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.487 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-06-10T00:43:43.487 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.487 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.488 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.488 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.488 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.488 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.488 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.488 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.488 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-06-10T00:43:43.488 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-06-10T00:43:43.489 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-06-10T00:43:43.489 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-06-10T00:43:43.489 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-06-10T00:43:43.489 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-10T00:43:43.489 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-06-10T00:43:43.489 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-06-10T00:43:43.489 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-06-10T00:43:43.489 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-06-10T00:43:43.489 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-06-10T00:43:43.490 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-06-10T00:43:43.490 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-06-10T00:43:43.490 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.490 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.490 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.490 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.490 INFO:teuthology.orchestra.run.smithi062.stdout: "services": { 2024-06-10T00:43:43.490 INFO:teuthology.orchestra.run.smithi062.stdout: "dashboard": "https://172.21.15.62:8443/" 2024-06-10T00:43:43.491 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.491 INFO:teuthology.orchestra.run.smithi062.stdout: "always_on_modules": { 2024-06-10T00:43:43.491 INFO:teuthology.orchestra.run.smithi062.stdout: "octopus": [ 2024-06-10T00:43:43.491 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-06-10T00:43:43.491 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-06-10T00:43:43.491 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-06-10T00:43:43.491 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-06-10T00:43:43.491 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-06-10T00:43:43.492 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-06-10T00:43:43.492 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-06-10T00:43:43.492 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-06-10T00:43:43.492 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-06-10T00:43:43.492 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-06-10T00:43:43.492 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.492 INFO:teuthology.orchestra.run.smithi062.stdout: "pacific": [ 2024-06-10T00:43:43.492 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-06-10T00:43:43.492 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-06-10T00:43:43.493 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-06-10T00:43:43.493 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-06-10T00:43:43.493 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-06-10T00:43:43.493 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-06-10T00:43:43.493 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-06-10T00:43:43.493 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-06-10T00:43:43.493 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-06-10T00:43:43.493 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-06-10T00:43:43.494 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.494 INFO:teuthology.orchestra.run.smithi062.stdout: "quincy": [ 2024-06-10T00:43:43.494 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-06-10T00:43:43.494 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-06-10T00:43:43.494 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-06-10T00:43:43.494 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-06-10T00:43:43.494 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-06-10T00:43:43.494 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-06-10T00:43:43.495 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-06-10T00:43:43.495 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-06-10T00:43:43.495 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-06-10T00:43:43.495 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-06-10T00:43:43.495 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.495 INFO:teuthology.orchestra.run.smithi062.stdout: "reef": [ 2024-06-10T00:43:43.495 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-06-10T00:43:43.495 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-06-10T00:43:43.496 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-06-10T00:43:43.496 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-06-10T00:43:43.496 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-06-10T00:43:43.496 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-06-10T00:43:43.496 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-06-10T00:43:43.496 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-06-10T00:43:43.496 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-06-10T00:43:43.496 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-06-10T00:43:43.497 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-06-10T00:43:43.497 INFO:teuthology.orchestra.run.smithi062.stdout: "squid": [ 2024-06-10T00:43:43.497 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-06-10T00:43:43.497 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-06-10T00:43:43.497 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-06-10T00:43:43.497 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-06-10T00:43:43.497 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-06-10T00:43:43.497 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-06-10T00:43:43.498 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-06-10T00:43:43.498 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-06-10T00:43:43.498 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-06-10T00:43:43.498 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-06-10T00:43:43.498 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-06-10T00:43:43.498 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.498 INFO:teuthology.orchestra.run.smithi062.stdout: "last_failure_osd_epoch": 100, 2024-06-10T00:43:43.499 INFO:teuthology.orchestra.run.smithi062.stdout: "active_clients": [ 2024-06-10T00:43:43.499 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.499 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-06-10T00:43:43.499 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-06-10T00:43:43.499 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.499 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-06-10T00:43:43.499 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-06-10T00:43:43.499 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 3424735409 2024-06-10T00:43:43.499 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.500 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-06-10T00:43:43.500 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.500 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.500 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "libcephsqlite", 2024-06-10T00:43:43.500 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-06-10T00:43:43.500 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.500 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-06-10T00:43:43.500 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-06-10T00:43:43.501 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 747677978 2024-06-10T00:43:43.501 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.501 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-06-10T00:43:43.501 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.501 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.501 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-06-10T00:43:43.501 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-06-10T00:43:43.501 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.501 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-06-10T00:43:43.502 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-06-10T00:43:43.502 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 1092467833 2024-06-10T00:43:43.502 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.502 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-06-10T00:43:43.502 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.502 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.502 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-06-10T00:43:43.502 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-06-10T00:43:43.503 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.503 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-06-10T00:43:43.503 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-06-10T00:43:43.503 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 3465394863 2024-06-10T00:43:43.503 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.503 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-06-10T00:43:43.503 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.503 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.504 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-06-10T00:43:43.504 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-06-10T00:43:43.504 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.504 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-06-10T00:43:43.504 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-06-10T00:43:43.504 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2942152293 2024-06-10T00:43:43.504 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.504 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-06-10T00:43:43.505 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:43.505 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.505 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-06-10T00:43:43.505 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-06-10T00:43:43.505 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-06-10T00:43:43.505 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-06-10T00:43:43.505 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-06-10T00:43:43.505 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2844237271 2024-06-10T00:43:43.505 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.506 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-06-10T00:43:43.506 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:43.506 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-06-10T00:43:43.506 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:43:43.507 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:41 smithi105 ceph-mon[36321]: pgmap v336: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:43.508 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:41 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/875435705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T00:43:43.508 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:41 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1077855220' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-10T00:43:43.508 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:41 smithi138 ceph-mon[36549]: pgmap v336: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:43.508 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:41 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/875435705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T00:43:43.508 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:41 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1077855220' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-10T00:43:43.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:41.168+0000 7f32d0f7e640 1 -- 172.21.15.62:0/1077855220 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32b40a2d80 msgr2=0x7f32b40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:43.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:41.168+0000 7f32d0f7e640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32b40a2d80 0x7f32b40a5240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f32bc000c00 tx=0x7f32bc005a20 comp rx=0 tx=0).stop 2024-06-10T00:43:43.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:41.168+0000 7f32d0f7e640 1 -- 172.21.15.62:0/1077855220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc102330 msgr2=0x7f32cc198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:43:43.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:41.168+0000 7f32d0f7e640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc102330 0x7f32cc198810 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f32c000ebc0 tx=0x7f32c000c6a0 comp rx=0 tx=0).stop 2024-06-10T00:43:43.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:41.168+0000 7f32d0f7e640 1 -- 172.21.15.62:0/1077855220 shutdown_connections 2024-06-10T00:43:43.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:41.168+0000 7f32d0f7e640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f32b40a2d80 0x7f32b40a5240 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:43.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:41.168+0000 7f32d0f7e640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc107980 0x7f32cc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:43.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:41.168+0000 7f32d0f7e640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32cc102330 0x7f32cc198810 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:43.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:41.168+0000 7f32d0f7e640 1 --2- 172.21.15.62:0/1077855220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32cc101970 0x7f32cc1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:43:43.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:41.168+0000 7f32d0f7e640 1 -- 172.21.15.62:0/1077855220 >> 172.21.15.62:0/1077855220 conn(0x7f32cc078d60 msgr2=0x7f32cc1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:43:43.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:41.168+0000 7f32d0f7e640 1 -- 172.21.15.62:0/1077855220 shutdown_connections 2024-06-10T00:43:43.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:43:41.168+0000 7f32d0f7e640 1 -- 172.21.15.62:0/1077855220 wait complete. 2024-06-10T00:43:43.517 DEBUG:tasks.check_counter:Getting stats from a 2024-06-10T00:43:43.517 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-10T00:43:43.518 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:41 smithi062 ceph-mon[32390]: pgmap v336: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:43.518 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:41 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/875435705' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T00:43:43.518 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:41 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1077855220' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-10T00:43:43.782 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:43:43.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:43 smithi062 ceph-mon[32390]: pgmap v337: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:43:43.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:43 smithi138 ceph-mon[36549]: pgmap v337: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:43:44.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:43 smithi105 ceph-mon[36321]: pgmap v337: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:43:44.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:44 smithi062 ceph-mon[32390]: pgmap v338: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:45.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:44 smithi105 ceph-mon[36321]: pgmap v338: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:45.123 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:44 smithi138 ceph-mon[36549]: pgmap v338: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:45.679 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:43:45.679 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T00:43:45.679 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1693, 2024-06-10T00:43:45.679 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1695, 2024-06-10T00:43:45.679 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 813701, 2024-06-10T00:43:45.679 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 538375, 2024-06-10T00:43:45.680 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-06-10T00:43:45.680 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2024-06-10T00:43:45.680 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.161302510, 2024-06-10T00:43:45.680 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.049582182, 2024-06-10T00:43:45.680 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.043254476, 2024-06-10T00:43:45.680 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.054006722, 2024-06-10T00:43:45.680 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:45.681 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1693, 2024-06-10T00:43:45.681 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.064400445, 2024-06-10T00:43:45.681 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000038039 2024-06-10T00:43:45.681 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.681 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:45.681 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.681 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.681 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.681 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.682 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T00:43:45.682 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T00:43:45.682 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.682 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T00:43:45.682 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 651, 2024-06-10T00:43:45.682 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 838, 2024-06-10T00:43:45.682 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 160626, 2024-06-10T00:43:45.682 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 682354, 2024-06-10T00:43:45.683 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 7, 2024-06-10T00:43:45.683 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 5, 2024-06-10T00:43:45.683 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.080652573, 2024-06-10T00:43:45.683 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.027245390, 2024-06-10T00:43:45.683 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.017851052, 2024-06-10T00:43:45.683 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.027629663, 2024-06-10T00:43:45.683 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:45.683 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 835, 2024-06-10T00:43:45.684 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.040836568, 2024-06-10T00:43:45.684 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000048906 2024-06-10T00:43:45.684 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.684 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:45.684 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.684 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.684 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.684 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.684 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T00:43:45.685 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 475904 2024-06-10T00:43:45.685 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.685 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T00:43:45.685 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 290, 2024-06-10T00:43:45.685 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 267, 2024-06-10T00:43:45.685 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1687772, 2024-06-10T00:43:45.685 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 156254, 2024-06-10T00:43:45.685 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 3, 2024-06-10T00:43:45.686 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2024-06-10T00:43:45.686 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.046729755, 2024-06-10T00:43:45.686 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.015887707, 2024-06-10T00:43:45.686 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.035217720, 2024-06-10T00:43:45.686 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.004702802, 2024-06-10T00:43:45.686 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:45.686 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 267, 2024-06-10T00:43:45.686 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.012252010, 2024-06-10T00:43:45.687 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000045887 2024-06-10T00:43:45.687 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.687 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:45.687 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.687 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.687 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.687 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.687 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1687504, 2024-06-10T00:43:45.688 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 155936 2024-06-10T00:43:45.688 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.688 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-06-10T00:43:45.688 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-06-10T00:43:45.688 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-06-10T00:43:45.688 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.688 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-06-10T00:43:45.688 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-06-10T00:43:45.688 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-06-10T00:43:45.689 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3879, 2024-06-10T00:43:45.689 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.113082392, 2024-06-10T00:43:45.689 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000029152 2024-06-10T00:43:45.689 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:45.689 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.689 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-06-10T00:43:45.689 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-06-10T00:43:45.689 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-06-10T00:43:45.689 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.690 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.690 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.690 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:45.690 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.690 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-06-10T00:43:45.690 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 0, 2024-06-10T00:43:45.690 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 0, 2024-06-10T00:43:45.691 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-06-10T00:43:45.691 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.691 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.691 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.691 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.691 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-06-10T00:43:45.691 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-06-10T00:43:45.691 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-06-10T00:43:45.691 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-06-10T00:43:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 0, 2024-06-10T00:43:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-06-10T00:43:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-06-10T00:43:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 77, 2024-06-10T00:43:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 0, 2024-06-10T00:43:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 77, 2024-06-10T00:43:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-06-10T00:43:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 77, 2024-06-10T00:43:45.692 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-06-10T00:43:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 0, 2024-06-10T00:43:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 0, 2024-06-10T00:43:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 4, 2024-06-10T00:43:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 0, 2024-06-10T00:43:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 0, 2024-06-10T00:43:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-06-10T00:43:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-06-10T00:43:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-06-10T00:43:45.693 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-06-10T00:43:45.694 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-06-10T00:43:45.694 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-06-10T00:43:45.694 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-06-10T00:43:45.694 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-06-10T00:43:45.694 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-06-10T00:43:45.694 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-06-10T00:43:45.695 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-06-10T00:43:45.695 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-06-10T00:43:45.695 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-06-10T00:43:45.695 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-06-10T00:43:45.695 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2024-06-10T00:43:45.695 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2024-06-10T00:43:45.695 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-06-10T00:43:45.695 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T00:43:45.695 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-06-10T00:43:45.696 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T00:43:45.696 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-06-10T00:43:45.696 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-06-10T00:43:45.696 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T00:43:45.696 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T00:43:45.696 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-06-10T00:43:45.696 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-06-10T00:43:45.696 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T00:43:45.697 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 0, 2024-06-10T00:43:45.697 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T00:43:45.697 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T00:43:45.697 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T00:43:45.697 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 0, 2024-06-10T00:43:45.697 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 0, 2024-06-10T00:43:45.698 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 0, 2024-06-10T00:43:45.698 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 0 2024-06-10T00:43:45.698 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.698 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-06-10T00:43:45.698 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-06-10T00:43:45.698 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-06-10T00:43:45.698 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-06-10T00:43:45.699 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-06-10T00:43:45.699 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-06-10T00:43:45.699 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-06-10T00:43:45.699 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-06-10T00:43:45.699 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-06-10T00:43:45.699 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2024-06-10T00:43:45.699 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2024-06-10T00:43:45.699 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-06-10T00:43:45.699 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-06-10T00:43:45.700 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-06-10T00:43:45.700 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-06-10T00:43:45.700 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-06-10T00:43:45.700 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-06-10T00:43:45.700 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-06-10T00:43:45.700 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_path": 0, 2024-06-10T00:43:45.700 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_inode": 0, 2024-06-10T00:43:45.700 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T00:43:45.701 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-06-10T00:43:45.701 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-06-10T00:43:45.701 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-06-10T00:43:45.701 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-06-10T00:43:45.701 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-06-10T00:43:45.701 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.701 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-06-10T00:43:45.701 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-06-10T00:43:45.702 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 0, 2024-06-10T00:43:45.702 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-06-10T00:43:45.702 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-06-10T00:43:45.702 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 225, 2024-06-10T00:43:45.702 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-06-10T00:43:45.702 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-06-10T00:43:45.702 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 0, 2024-06-10T00:43:45.703 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-06-10T00:43:45.703 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-06-10T00:43:45.703 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 2, 2024-06-10T00:43:45.703 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 2, 2024-06-10T00:43:45.703 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-06-10T00:43:45.703 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-06-10T00:43:45.703 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-06-10T00:43:45.703 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 4503406, 2024-06-10T00:43:45.703 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 4502832, 2024-06-10T00:43:45.704 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-06-10T00:43:45.704 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.704 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.704 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.704 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.704 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 225 2024-06-10T00:43:45.704 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.704 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-06-10T00:43:45.704 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 80, 2024-06-10T00:43:45.705 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 80, 2024-06-10T00:43:45.705 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 0, 2024-06-10T00:43:45.705 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 11, 2024-06-10T00:43:45.705 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 11, 2024-06-10T00:43:45.705 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 0, 2024-06-10T00:43:45.705 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 77, 2024-06-10T00:43:45.705 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 77, 2024-06-10T00:43:45.705 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 0, 2024-06-10T00:43:45.705 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 0, 2024-06-10T00:43:45.706 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 0, 2024-06-10T00:43:45.706 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 0, 2024-06-10T00:43:45.706 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 51728, 2024-06-10T00:43:45.706 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215420 2024-06-10T00:43:45.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.706 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-06-10T00:43:45.706 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 0, 2024-06-10T00:43:45.706 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-06-10T00:43:45.707 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 0, 2024-06-10T00:43:45.707 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 0, 2024-06-10T00:43:45.707 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-06-10T00:43:45.707 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-06-10T00:43:45.707 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.707 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.707 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.707 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.707 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-06-10T00:43:45.708 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.708 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.708 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.708 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-06-10T00:43:45.708 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.708 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.708 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.709 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-06-10T00:43:45.709 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.709 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.709 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.709 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.709 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-06-10T00:43:45.709 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.709 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.710 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.710 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-06-10T00:43:45.710 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.710 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.710 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.711 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-06-10T00:43:45.711 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.711 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.711 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.711 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.711 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-06-10T00:43:45.711 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.711 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.712 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.712 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.712 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-06-10T00:43:45.712 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.712 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.712 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.712 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.712 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-06-10T00:43:45.712 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.713 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.713 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.713 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.713 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-06-10T00:43:45.713 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.713 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.713 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.713 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.714 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-06-10T00:43:45.714 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.714 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.714 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.715 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.715 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-06-10T00:43:45.715 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.715 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.715 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.715 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.715 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-06-10T00:43:45.715 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.716 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.716 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.716 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.716 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-06-10T00:43:45.716 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.716 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.716 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.716 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.717 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-06-10T00:43:45.717 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.717 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.717 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.717 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.717 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-06-10T00:43:45.717 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.717 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.718 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.718 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.718 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-06-10T00:43:45.718 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.718 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.718 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.718 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.718 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-06-10T00:43:45.719 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.719 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.719 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.719 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.719 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-06-10T00:43:45.719 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.719 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.719 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.720 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.720 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-06-10T00:43:45.720 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.720 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.720 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.720 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.720 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-06-10T00:43:45.720 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.721 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.721 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.721 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.721 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-06-10T00:43:45.721 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.721 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.721 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.721 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.722 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-06-10T00:43:45.722 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.722 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.722 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.722 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.722 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-06-10T00:43:45.722 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.722 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.723 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.723 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.723 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-06-10T00:43:45.723 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.723 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.723 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.723 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.723 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-06-10T00:43:45.724 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.724 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.724 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.724 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.724 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-06-10T00:43:45.724 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.724 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.724 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.725 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-06-10T00:43:45.725 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.725 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.725 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.725 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-06-10T00:43:45.725 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-06-10T00:43:45.726 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-06-10T00:43:45.726 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.726 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.726 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.726 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:45.726 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.726 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-06-10T00:43:45.726 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 2, 2024-06-10T00:43:45.727 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 3, 2024-06-10T00:43:45.727 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 1, 2024-06-10T00:43:45.727 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 3, 2024-06-10T00:43:45.727 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-06-10T00:43:45.727 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 0, 2024-06-10T00:43:45.727 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 0, 2024-06-10T00:43:45.727 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 2581, 2024-06-10T00:43:45.727 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-06-10T00:43:45.728 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.728 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-06-10T00:43:45.728 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-06-10T00:43:45.728 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-06-10T00:43:45.728 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T00:43:45.728 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-06-10T00:43:45.728 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T00:43:45.728 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-06-10T00:43:45.728 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T00:43:45.729 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T00:43:45.729 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T00:43:45.729 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T00:43:45.729 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T00:43:45.729 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-06-10T00:43:45.729 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T00:43:45.729 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T00:43:45.729 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-06-10T00:43:45.730 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-06-10T00:43:45.730 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-06-10T00:43:45.730 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-06-10T00:43:45.730 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T00:43:45.730 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T00:43:45.730 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T00:43:45.730 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T00:43:45.731 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T00:43:45.731 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-06-10T00:43:45.731 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-06-10T00:43:45.731 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-06-10T00:43:45.731 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T00:43:45.731 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T00:43:45.731 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-06-10T00:43:45.731 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-06-10T00:43:45.731 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-06-10T00:43:45.732 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-06-10T00:43:45.732 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T00:43:45.732 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-06-10T00:43:45.732 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T00:43:45.732 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-06-10T00:43:45.732 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 192979, 2024-06-10T00:43:45.732 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 128, 2024-06-10T00:43:45.732 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-06-10T00:43:45.733 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-06-10T00:43:45.733 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-06-10T00:43:45.733 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-06-10T00:43:45.733 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-06-10T00:43:45.733 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-06-10T00:43:45.733 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-06-10T00:43:45.733 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-06-10T00:43:45.733 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 9704, 2024-06-10T00:43:45.733 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 186, 2024-06-10T00:43:45.734 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T00:43:45.734 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-06-10T00:43:45.734 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-06-10T00:43:45.734 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-06-10T00:43:45.734 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 253839, 2024-06-10T00:43:45.734 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2139, 2024-06-10T00:43:45.734 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-06-10T00:43:45.734 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-06-10T00:43:45.735 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-06-10T00:43:45.735 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-06-10T00:43:45.735 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.735 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-06-10T00:43:45.735 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-06-10T00:43:45.735 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-06-10T00:43:45.735 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 2343, 2024-06-10T00:43:45.735 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 21, 2024-06-10T00:43:45.736 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 3, 2024-06-10T00:43:45.736 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 2343, 2024-06-10T00:43:45.736 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-06-10T00:43:45.736 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2343, 2024-06-10T00:43:45.736 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 10.285854325, 2024-06-10T00:43:45.736 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.004390035 2024-06-10T00:43:45.736 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.736 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-06-10T00:43:45.737 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-06-10T00:43:45.737 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2343, 2024-06-10T00:43:45.737 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2344 2024-06-10T00:43:45.737 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.737 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 2343, 2024-06-10T00:43:45.737 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 2343, 2024-06-10T00:43:45.737 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 0, 2024-06-10T00:43:45.737 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-06-10T00:43:45.737 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-06-10T00:43:45.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 1477, 2024-06-10T00:43:45.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 0, 2024-06-10T00:43:45.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 864, 2024-06-10T00:43:45.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 0, 2024-06-10T00:43:45.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 0, 2024-06-10T00:43:45.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-06-10T00:43:45.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-06-10T00:43:45.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-06-10T00:43:45.739 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-06-10T00:43:45.739 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 0, 2024-06-10T00:43:45.739 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-06-10T00:43:45.739 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-06-10T00:43:45.739 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-06-10T00:43:45.739 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 1, 2024-06-10T00:43:45.739 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 0, 2024-06-10T00:43:45.739 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-06-10T00:43:45.740 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-06-10T00:43:45.740 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-06-10T00:43:45.740 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-06-10T00:43:45.740 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-06-10T00:43:45.740 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-06-10T00:43:45.740 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T00:43:45.740 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-06-10T00:43:45.740 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-06-10T00:43:45.740 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-06-10T00:43:45.741 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-06-10T00:43:45.741 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-06-10T00:43:45.741 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-06-10T00:43:45.741 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-06-10T00:43:45.741 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-06-10T00:43:45.741 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-06-10T00:43:45.741 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-06-10T00:43:45.741 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-06-10T00:43:45.742 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-06-10T00:43:45.742 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-06-10T00:43:45.742 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-06-10T00:43:45.742 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-06-10T00:43:45.742 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-06-10T00:43:45.742 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-06-10T00:43:45.742 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-06-10T00:43:45.742 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-06-10T00:43:45.743 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 108, 2024-06-10T00:43:45.743 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-06-10T00:43:45.743 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 11, 2024-06-10T00:43:45.743 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 5, 2024-06-10T00:43:45.743 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 5, 2024-06-10T00:43:45.743 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-06-10T00:43:45.743 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-06-10T00:43:45.743 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 0, 2024-06-10T00:43:45.744 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2024-06-10T00:43:45.744 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 0 2024-06-10T00:43:45.744 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.744 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-06-10T00:43:45.744 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 0, 2024-06-10T00:43:45.744 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-06-10T00:43:45.744 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-06-10T00:43:45.744 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-06-10T00:43:45.745 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.745 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-06-10T00:43:45.745 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-06-10T00:43:45.745 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.745 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-06-10T00:43:45.745 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-06-10T00:43:45.745 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T00:43:45.745 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-06-10T00:43:45.746 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2024-06-10T00:43:45.746 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 0, 2024-06-10T00:43:45.746 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2024-06-10T00:43:45.746 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-06-10T00:43:45.746 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.746 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T00:43:45.746 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-06-10T00:43:45.746 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-06-10T00:43:45.747 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-06-10T00:43:45.747 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 2630, 2024-06-10T00:43:45.747 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 2407800, 2024-06-10T00:43:45.747 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:45.747 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 2630, 2024-06-10T00:43:45.747 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-06-10T00:43:45.747 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-06-10T00:43:45.747 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2630, 2024-06-10T00:43:45.748 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 2407800, 2024-06-10T00:43:45.748 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-06-10T00:43:45.748 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.748 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.748 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.748 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:45.748 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.748 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-06-10T00:43:45.748 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-06-10T00:43:45.749 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-06-10T00:43:45.749 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-06-10T00:43:45.749 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-06-10T00:43:45.749 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-06-10T00:43:45.749 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:45.749 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-06-10T00:43:45.749 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 2343, 2024-06-10T00:43:45.749 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 309107, 2024-06-10T00:43:45.750 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2343, 2024-06-10T00:43:45.750 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 309107, 2024-06-10T00:43:45.750 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-06-10T00:43:45.750 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.750 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.750 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.750 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:45.751 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.751 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-06-10T00:43:45.751 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-06-10T00:43:45.751 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-06-10T00:43:45.751 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-06-10T00:43:45.751 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-06-10T00:43:45.751 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-06-10T00:43:45.751 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:45.751 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-06-10T00:43:45.752 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 2343, 2024-06-10T00:43:45.752 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 2343, 2024-06-10T00:43:45.752 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2343, 2024-06-10T00:43:45.752 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 2343, 2024-06-10T00:43:45.752 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-06-10T00:43:45.752 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.752 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.752 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.753 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:45.753 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.753 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-06-10T00:43:45.753 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-06-10T00:43:45.753 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-06-10T00:43:45.753 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-06-10T00:43:45.753 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-06-10T00:43:45.753 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-06-10T00:43:45.754 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:45.754 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-06-10T00:43:45.754 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-06-10T00:43:45.754 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-06-10T00:43:45.754 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-06-10T00:43:45.754 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-06-10T00:43:45.754 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-06-10T00:43:45.754 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.755 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.755 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.755 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:45.755 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:45.755 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x5643ff7fe6e0": { 2024-06-10T00:43:45.755 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-06-10T00:43:45.755 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-06-10T00:43:45.755 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-06-10T00:43:45.756 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-06-10T00:43:45.756 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-06-10T00:43:45.756 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:45.756 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-06-10T00:43:45.756 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-06-10T00:43:45.756 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-06-10T00:43:45.756 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-06-10T00:43:45.756 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-06-10T00:43:45.757 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-06-10T00:43:45.757 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:45.757 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:45.757 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:45.757 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:45.757 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:45.757 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:43:46.542 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-06-10T00:43:46.542 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-06-10T00:43:46.542 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-06-10T00:43:46.543 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-06-10T00:43:46.543 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-06-10T00:43:46.543 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-06-10T00:43:46.543 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-06-10T00:43:46.543 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-06-10T00:43:46.543 DEBUG:tasks.check_counter:Getting stats from d 2024-06-10T00:43:46.544 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-10T00:43:46.801 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:43:47.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:47 smithi105 ceph-mon[36321]: pgmap v339: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:43:47.581 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:47 smithi062 ceph-mon[32390]: pgmap v339: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:43:47.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:47 smithi138 ceph-mon[36549]: pgmap v339: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:43:48.608 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:43:48.608 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T00:43:48.608 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 265, 2024-06-10T00:43:48.609 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 254, 2024-06-10T00:43:48.609 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1593168, 2024-06-10T00:43:48.609 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 156560, 2024-06-10T00:43:48.609 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-06-10T00:43:48.609 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-06-10T00:43:48.609 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.048747992, 2024-06-10T00:43:48.609 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.016813519, 2024-06-10T00:43:48.609 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.034293746, 2024-06-10T00:43:48.610 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.005009838, 2024-06-10T00:43:48.610 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:48.610 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 253, 2024-06-10T00:43:48.610 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.011893510, 2024-06-10T00:43:48.610 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000047009 2024-06-10T00:43:48.610 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:48.610 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:48.611 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:48.611 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:48.611 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:48.611 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:48.611 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1593168, 2024-06-10T00:43:48.611 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 156560 2024-06-10T00:43:48.611 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:48.611 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T00:43:48.611 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2, 2024-06-10T00:43:48.612 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 194, 2024-06-10T00:43:48.612 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 320, 2024-06-10T00:43:48.612 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 189968, 2024-06-10T00:43:48.612 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-06-10T00:43:48.612 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-06-10T00:43:48.612 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.016259763, 2024-06-10T00:43:48.612 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.011659321, 2024-06-10T00:43:48.612 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.001866072, 2024-06-10T00:43:48.613 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T00:43:48.613 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:48.613 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 191, 2024-06-10T00:43:48.613 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.011373337, 2024-06-10T00:43:48.613 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000059546 2024-06-10T00:43:48.613 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:48.613 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:48.613 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:48.614 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:48.614 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:48.614 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:48.614 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T00:43:48.614 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 189968 2024-06-10T00:43:48.614 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:48.614 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T00:43:48.614 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 4, 2024-06-10T00:43:48.615 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 4, 2024-06-10T00:43:48.615 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 3024, 2024-06-10T00:43:48.615 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 448, 2024-06-10T00:43:48.615 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-06-10T00:43:48.615 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-06-10T00:43:48.615 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.000898883, 2024-06-10T00:43:48.615 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000050852, 2024-06-10T00:43:48.615 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000825807, 2024-06-10T00:43:48.616 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T00:43:48.616 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:48.616 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3, 2024-06-10T00:43:48.616 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000900846, 2024-06-10T00:43:48.616 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000300282 2024-06-10T00:43:48.616 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:48.616 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:48.616 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:48.617 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:48.617 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:48.617 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:48.617 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-10T00:43:48.617 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 448 2024-06-10T00:43:48.617 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:48.617 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-06-10T00:43:48.617 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-06-10T00:43:48.618 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-06-10T00:43:48.618 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:48.618 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-06-10T00:43:48.618 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-06-10T00:43:48.618 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-06-10T00:43:48.618 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T00:43:48.618 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-06-10T00:43:48.618 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T00:43:48.619 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-06-10T00:43:48.619 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T00:43:48.619 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T00:43:48.619 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T00:43:48.619 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T00:43:48.619 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T00:43:48.619 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-06-10T00:43:48.619 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T00:43:48.620 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T00:43:48.620 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-06-10T00:43:48.620 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-06-10T00:43:48.620 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-06-10T00:43:48.620 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-06-10T00:43:48.620 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T00:43:48.620 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T00:43:48.620 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T00:43:48.620 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T00:43:48.621 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T00:43:48.621 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-06-10T00:43:48.621 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-06-10T00:43:48.621 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-06-10T00:43:48.621 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T00:43:48.621 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T00:43:48.621 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-06-10T00:43:48.621 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-06-10T00:43:48.622 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-06-10T00:43:48.622 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-06-10T00:43:48.622 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T00:43:48.622 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-06-10T00:43:48.622 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T00:43:48.622 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-06-10T00:43:48.622 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148714, 2024-06-10T00:43:48.622 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 30, 2024-06-10T00:43:48.623 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-06-10T00:43:48.623 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-06-10T00:43:48.623 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-06-10T00:43:48.623 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-06-10T00:43:48.623 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-06-10T00:43:48.623 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-06-10T00:43:48.623 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-06-10T00:43:48.623 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-06-10T00:43:48.624 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-06-10T00:43:48.624 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-06-10T00:43:48.624 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T00:43:48.624 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-06-10T00:43:48.624 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-06-10T00:43:48.624 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-06-10T00:43:48.625 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-06-10T00:43:48.625 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-06-10T00:43:48.625 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-06-10T00:43:48.625 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-06-10T00:43:48.625 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-06-10T00:43:48.625 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-06-10T00:43:48.625 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:48.625 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T00:43:48.626 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-06-10T00:43:48.626 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-06-10T00:43:48.626 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-06-10T00:43:48.626 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 267, 2024-06-10T00:43:48.626 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 1555790, 2024-06-10T00:43:48.626 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:48.626 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 267, 2024-06-10T00:43:48.626 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-06-10T00:43:48.627 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-06-10T00:43:48.627 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 267, 2024-06-10T00:43:48.627 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1555790, 2024-06-10T00:43:48.627 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-06-10T00:43:48.627 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:48.627 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:48.627 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:48.627 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:48.627 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:48.628 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:43:49.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:49 smithi105 ceph-mon[36321]: pgmap v340: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:43:49.570 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:49 smithi062 ceph-mon[32390]: pgmap v340: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:43:49.602 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-06-10T00:43:49.603 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-06-10T00:43:49.603 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-06-10T00:43:49.603 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-06-10T00:43:49.603 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-06-10T00:43:49.603 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-06-10T00:43:49.603 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-06-10T00:43:49.603 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-06-10T00:43:49.604 DEBUG:tasks.check_counter:Getting stats from g 2024-06-10T00:43:49.604 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-10T00:43:49.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:49 smithi138 ceph-mon[36549]: pgmap v340: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:43:49.861 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:43:51.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:51 smithi062 ceph-mon[32390]: pgmap v341: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:51.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:51 smithi105 ceph-mon[36321]: pgmap v341: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:51.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:51 smithi138 ceph-mon[36549]: pgmap v341: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:51.722 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:43:51.722 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T00:43:51.722 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1282, 2024-06-10T00:43:51.722 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1291, 2024-06-10T00:43:51.722 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 383057, 2024-06-10T00:43:51.722 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 407546, 2024-06-10T00:43:51.723 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 7, 2024-06-10T00:43:51.723 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-06-10T00:43:51.723 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.122610517, 2024-06-10T00:43:51.723 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.045055847, 2024-06-10T00:43:51.723 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.027501206, 2024-06-10T00:43:51.724 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.038897580, 2024-06-10T00:43:51.724 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:51.724 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1290, 2024-06-10T00:43:51.724 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.050768830, 2024-06-10T00:43:51.724 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000039355 2024-06-10T00:43:51.724 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.724 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:51.725 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.725 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.725 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.725 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-10T00:43:51.725 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 448 2024-06-10T00:43:51.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.726 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T00:43:51.726 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1034, 2024-06-10T00:43:51.726 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1013, 2024-06-10T00:43:51.726 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 2225466, 2024-06-10T00:43:51.726 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 394693, 2024-06-10T00:43:51.726 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-06-10T00:43:51.726 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-06-10T00:43:51.727 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.105482129, 2024-06-10T00:43:51.727 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.034116493, 2024-06-10T00:43:51.727 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.054369307, 2024-06-10T00:43:51.727 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.022174422, 2024-06-10T00:43:51.727 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:51.727 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1012, 2024-06-10T00:43:51.727 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.035200582, 2024-06-10T00:43:51.727 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000034783 2024-06-10T00:43:51.727 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.728 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:51.728 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.728 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.728 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.728 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.728 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1685296, 2024-06-10T00:43:51.728 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 157408 2024-06-10T00:43:51.728 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.728 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T00:43:51.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 5, 2024-06-10T00:43:51.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 195, 2024-06-10T00:43:51.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1668, 2024-06-10T00:43:51.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 480629, 2024-06-10T00:43:51.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 7, 2024-06-10T00:43:51.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 5, 2024-06-10T00:43:51.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.015309974, 2024-06-10T00:43:51.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.010324679, 2024-06-10T00:43:51.730 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.002611998, 2024-06-10T00:43:51.730 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000053563, 2024-06-10T00:43:51.730 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:51.730 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 192, 2024-06-10T00:43:51.730 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.017631843, 2024-06-10T00:43:51.730 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000091832 2024-06-10T00:43:51.730 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.730 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:51.731 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.731 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.731 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.731 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.731 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T00:43:51.731 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 479664 2024-06-10T00:43:51.731 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.731 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-06-10T00:43:51.731 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-06-10T00:43:51.732 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-06-10T00:43:51.732 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.732 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-06-10T00:43:51.732 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-06-10T00:43:51.732 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-06-10T00:43:51.732 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3459, 2024-06-10T00:43:51.732 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.082462678, 2024-06-10T00:43:51.732 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000023840 2024-06-10T00:43:51.732 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:51.733 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.733 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-06-10T00:43:51.733 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-06-10T00:43:51.733 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-06-10T00:43:51.733 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.733 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.733 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.733 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:51.733 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.734 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-06-10T00:43:51.734 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 0, 2024-06-10T00:43:51.734 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 0, 2024-06-10T00:43:51.734 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-06-10T00:43:51.734 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.734 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.734 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.734 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.735 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-06-10T00:43:51.735 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-06-10T00:43:51.735 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-06-10T00:43:51.735 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-06-10T00:43:51.735 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 0, 2024-06-10T00:43:51.735 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-06-10T00:43:51.735 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-06-10T00:43:51.735 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 74, 2024-06-10T00:43:51.736 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 0, 2024-06-10T00:43:51.736 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 74, 2024-06-10T00:43:51.736 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-06-10T00:43:51.736 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 74, 2024-06-10T00:43:51.736 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-06-10T00:43:51.736 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 0, 2024-06-10T00:43:51.736 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 0, 2024-06-10T00:43:51.736 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 4, 2024-06-10T00:43:51.737 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 0, 2024-06-10T00:43:51.737 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 0, 2024-06-10T00:43:51.737 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-06-10T00:43:51.737 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-06-10T00:43:51.737 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-06-10T00:43:51.737 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-06-10T00:43:51.737 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-06-10T00:43:51.737 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-06-10T00:43:51.737 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-06-10T00:43:51.738 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-06-10T00:43:51.738 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-06-10T00:43:51.738 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-06-10T00:43:51.738 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-06-10T00:43:51.738 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-06-10T00:43:51.738 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-06-10T00:43:51.738 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-06-10T00:43:51.738 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2024-06-10T00:43:51.739 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2024-06-10T00:43:51.739 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-06-10T00:43:51.739 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T00:43:51.739 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-06-10T00:43:51.739 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T00:43:51.739 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-06-10T00:43:51.739 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-06-10T00:43:51.739 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T00:43:51.739 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T00:43:51.740 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-06-10T00:43:51.740 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-06-10T00:43:51.740 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T00:43:51.740 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 0, 2024-06-10T00:43:51.740 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T00:43:51.740 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T00:43:51.740 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T00:43:51.740 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 0, 2024-06-10T00:43:51.740 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 0, 2024-06-10T00:43:51.741 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 0, 2024-06-10T00:43:51.741 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 0 2024-06-10T00:43:51.741 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.741 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-06-10T00:43:51.741 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-06-10T00:43:51.741 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-06-10T00:43:51.741 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-06-10T00:43:51.741 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-06-10T00:43:51.741 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-06-10T00:43:51.742 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-06-10T00:43:51.742 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-06-10T00:43:51.742 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-06-10T00:43:51.742 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2024-06-10T00:43:51.742 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2024-06-10T00:43:51.742 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-06-10T00:43:51.742 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-06-10T00:43:51.742 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-06-10T00:43:51.743 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-06-10T00:43:51.743 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-06-10T00:43:51.743 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-06-10T00:43:51.743 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-06-10T00:43:51.743 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_path": 0, 2024-06-10T00:43:51.743 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_inode": 0, 2024-06-10T00:43:51.743 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T00:43:51.743 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-06-10T00:43:51.744 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-06-10T00:43:51.744 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-06-10T00:43:51.744 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-06-10T00:43:51.744 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-06-10T00:43:51.744 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.744 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-06-10T00:43:51.744 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-06-10T00:43:51.745 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 0, 2024-06-10T00:43:51.745 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-06-10T00:43:51.745 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-06-10T00:43:51.745 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 257, 2024-06-10T00:43:51.745 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-06-10T00:43:51.745 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-06-10T00:43:51.746 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 0, 2024-06-10T00:43:51.746 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-06-10T00:43:51.746 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-06-10T00:43:51.746 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 2, 2024-06-10T00:43:51.746 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 2, 2024-06-10T00:43:51.746 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-06-10T00:43:51.746 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-06-10T00:43:51.746 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-06-10T00:43:51.747 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 4539230, 2024-06-10T00:43:51.747 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 4538656, 2024-06-10T00:43:51.747 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-06-10T00:43:51.747 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.747 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.747 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.747 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.747 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 257 2024-06-10T00:43:51.748 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.748 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-06-10T00:43:51.748 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 77, 2024-06-10T00:43:51.748 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 77, 2024-06-10T00:43:51.748 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 0, 2024-06-10T00:43:51.748 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 11, 2024-06-10T00:43:51.748 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 11, 2024-06-10T00:43:51.748 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 0, 2024-06-10T00:43:51.748 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 74, 2024-06-10T00:43:51.749 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 74, 2024-06-10T00:43:51.749 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 0, 2024-06-10T00:43:51.749 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 0, 2024-06-10T00:43:51.749 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 0, 2024-06-10T00:43:51.749 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 0, 2024-06-10T00:43:51.749 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 53968, 2024-06-10T00:43:51.749 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215420 2024-06-10T00:43:51.749 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.750 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-06-10T00:43:51.750 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 0, 2024-06-10T00:43:51.750 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-06-10T00:43:51.750 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 0, 2024-06-10T00:43:51.750 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 0, 2024-06-10T00:43:51.750 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-06-10T00:43:51.750 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-06-10T00:43:51.750 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.751 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.751 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.751 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.751 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-06-10T00:43:51.751 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.751 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.751 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.751 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.752 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-06-10T00:43:51.752 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.752 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.752 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.752 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.752 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-06-10T00:43:51.752 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.752 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.753 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.753 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.753 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-06-10T00:43:51.753 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.753 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.753 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.753 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.754 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-06-10T00:43:51.754 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.754 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.754 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.754 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.754 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-06-10T00:43:51.754 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.754 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.755 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.755 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.755 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-06-10T00:43:51.755 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.755 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.755 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.755 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.755 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-06-10T00:43:51.755 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.756 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.756 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.756 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.756 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-06-10T00:43:51.756 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.756 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.756 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.756 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.757 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-06-10T00:43:51.757 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.757 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.757 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.757 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.757 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-06-10T00:43:51.757 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.758 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.758 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.758 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.759 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-06-10T00:43:51.759 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.759 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.759 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.759 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.759 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-06-10T00:43:51.759 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.759 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.760 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.760 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.760 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-06-10T00:43:51.760 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.760 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.760 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.760 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.760 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-06-10T00:43:51.761 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.761 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.761 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.761 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.761 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-06-10T00:43:51.761 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.761 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.761 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.761 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.762 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-06-10T00:43:51.762 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.762 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.762 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.762 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.762 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-06-10T00:43:51.762 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.763 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.763 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.763 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-06-10T00:43:51.763 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.763 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.763 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.763 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-06-10T00:43:51.764 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.764 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.764 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.764 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.764 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-06-10T00:43:51.764 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.764 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.765 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.765 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-06-10T00:43:51.765 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.765 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.765 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.765 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-06-10T00:43:51.766 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.766 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.766 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.766 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.766 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-06-10T00:43:51.766 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.766 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.766 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.767 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.767 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-06-10T00:43:51.767 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.767 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.767 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.767 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.767 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-06-10T00:43:51.767 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.768 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.768 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.768 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.768 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-06-10T00:43:51.768 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.768 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.768 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.768 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.769 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-06-10T00:43:51.769 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.769 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.769 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.769 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.769 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-06-10T00:43:51.769 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-06-10T00:43:51.769 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-06-10T00:43:51.770 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.770 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.770 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.770 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:51.770 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.770 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-06-10T00:43:51.770 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 2, 2024-06-10T00:43:51.770 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 4, 2024-06-10T00:43:51.771 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 2, 2024-06-10T00:43:51.771 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 4, 2024-06-10T00:43:51.771 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-06-10T00:43:51.771 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 0, 2024-06-10T00:43:51.771 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 0, 2024-06-10T00:43:51.771 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 3230, 2024-06-10T00:43:51.771 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-06-10T00:43:51.771 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.772 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-06-10T00:43:51.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-06-10T00:43:51.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-06-10T00:43:51.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T00:43:51.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-06-10T00:43:51.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T00:43:51.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-06-10T00:43:51.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T00:43:51.772 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T00:43:51.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T00:43:51.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T00:43:51.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T00:43:51.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-06-10T00:43:51.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T00:43:51.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T00:43:51.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-06-10T00:43:51.773 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-06-10T00:43:51.774 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-06-10T00:43:51.774 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-06-10T00:43:51.774 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T00:43:51.774 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T00:43:51.774 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T00:43:51.774 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T00:43:51.775 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T00:43:51.775 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-06-10T00:43:51.775 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-06-10T00:43:51.775 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-06-10T00:43:51.775 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T00:43:51.776 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T00:43:51.776 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-06-10T00:43:51.776 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-06-10T00:43:51.776 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-06-10T00:43:51.776 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-06-10T00:43:51.776 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T00:43:51.776 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-06-10T00:43:51.776 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T00:43:51.777 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-06-10T00:43:51.777 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 208964, 2024-06-10T00:43:51.777 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 132, 2024-06-10T00:43:51.777 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-06-10T00:43:51.777 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-06-10T00:43:51.777 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-06-10T00:43:51.777 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-06-10T00:43:51.777 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-06-10T00:43:51.778 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-06-10T00:43:51.778 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-06-10T00:43:51.778 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-06-10T00:43:51.778 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 9704, 2024-06-10T00:43:51.778 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 186, 2024-06-10T00:43:51.778 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T00:43:51.778 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-06-10T00:43:51.779 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-06-10T00:43:51.779 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-06-10T00:43:51.779 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 245076, 2024-06-10T00:43:51.779 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2060, 2024-06-10T00:43:51.779 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-06-10T00:43:51.779 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-06-10T00:43:51.779 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-06-10T00:43:51.779 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-06-10T00:43:51.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.779 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-06-10T00:43:51.780 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-06-10T00:43:51.780 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-06-10T00:43:51.780 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 2027, 2024-06-10T00:43:51.780 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 21, 2024-06-10T00:43:51.780 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 3, 2024-06-10T00:43:51.780 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 2027, 2024-06-10T00:43:51.780 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-06-10T00:43:51.780 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2027, 2024-06-10T00:43:51.780 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 117.713334520, 2024-06-10T00:43:51.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.058072685 2024-06-10T00:43:51.781 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.781 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-06-10T00:43:51.781 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-06-10T00:43:51.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2027, 2024-06-10T00:43:51.781 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2028 2024-06-10T00:43:51.781 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.781 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 2027, 2024-06-10T00:43:51.781 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 2027, 2024-06-10T00:43:51.782 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 0, 2024-06-10T00:43:51.782 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-06-10T00:43:51.782 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-06-10T00:43:51.782 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 1277, 2024-06-10T00:43:51.782 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 0, 2024-06-10T00:43:51.782 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 748, 2024-06-10T00:43:51.782 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 0, 2024-06-10T00:43:51.782 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 0, 2024-06-10T00:43:51.783 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-06-10T00:43:51.783 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-06-10T00:43:51.783 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-06-10T00:43:51.783 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-06-10T00:43:51.783 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 0, 2024-06-10T00:43:51.783 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-06-10T00:43:51.783 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-06-10T00:43:51.783 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-06-10T00:43:51.783 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 1, 2024-06-10T00:43:51.784 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 0, 2024-06-10T00:43:51.784 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-06-10T00:43:51.784 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-06-10T00:43:51.784 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-06-10T00:43:51.784 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-06-10T00:43:51.784 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-06-10T00:43:51.784 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-06-10T00:43:51.784 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T00:43:51.784 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-06-10T00:43:51.785 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-06-10T00:43:51.785 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-06-10T00:43:51.785 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-06-10T00:43:51.785 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-06-10T00:43:51.785 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-06-10T00:43:51.785 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-06-10T00:43:51.785 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-06-10T00:43:51.785 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-06-10T00:43:51.786 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-06-10T00:43:51.786 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-06-10T00:43:51.786 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-06-10T00:43:51.786 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-06-10T00:43:51.786 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-06-10T00:43:51.786 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-06-10T00:43:51.786 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-06-10T00:43:51.786 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-06-10T00:43:51.787 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-06-10T00:43:51.787 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-06-10T00:43:51.787 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 108, 2024-06-10T00:43:51.787 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-06-10T00:43:51.787 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 11, 2024-06-10T00:43:51.787 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 7, 2024-06-10T00:43:51.787 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 7, 2024-06-10T00:43:51.787 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-06-10T00:43:51.788 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-06-10T00:43:51.788 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 0, 2024-06-10T00:43:51.788 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2024-06-10T00:43:51.788 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 0 2024-06-10T00:43:51.788 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.788 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-06-10T00:43:51.788 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 0, 2024-06-10T00:43:51.788 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-06-10T00:43:51.789 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-06-10T00:43:51.789 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-06-10T00:43:51.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.789 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-06-10T00:43:51.789 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-06-10T00:43:51.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.789 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-06-10T00:43:51.789 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-06-10T00:43:51.789 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T00:43:51.790 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-06-10T00:43:51.790 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2024-06-10T00:43:51.790 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 0, 2024-06-10T00:43:51.790 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2024-06-10T00:43:51.790 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-06-10T00:43:51.790 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.790 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T00:43:51.791 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-06-10T00:43:51.791 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-06-10T00:43:51.791 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-06-10T00:43:51.791 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 2317, 2024-06-10T00:43:51.791 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 2383906, 2024-06-10T00:43:51.791 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:51.791 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 2317, 2024-06-10T00:43:51.791 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-06-10T00:43:51.791 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-06-10T00:43:51.792 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2317, 2024-06-10T00:43:51.792 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 2383906, 2024-06-10T00:43:51.792 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-06-10T00:43:51.792 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.792 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.792 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.792 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:51.792 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.792 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-06-10T00:43:51.793 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-06-10T00:43:51.793 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-06-10T00:43:51.793 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-06-10T00:43:51.793 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-06-10T00:43:51.793 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-06-10T00:43:51.793 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:51.793 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-06-10T00:43:51.793 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 2027, 2024-06-10T00:43:51.793 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 344931, 2024-06-10T00:43:51.794 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2027, 2024-06-10T00:43:51.794 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 344931, 2024-06-10T00:43:51.794 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-06-10T00:43:51.794 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.794 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.794 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.794 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:51.794 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.795 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-06-10T00:43:51.795 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-06-10T00:43:51.795 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-06-10T00:43:51.795 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-06-10T00:43:51.795 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-06-10T00:43:51.795 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-06-10T00:43:51.795 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:51.796 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-06-10T00:43:51.796 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 2027, 2024-06-10T00:43:51.796 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 2027, 2024-06-10T00:43:51.796 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2027, 2024-06-10T00:43:51.796 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 2027, 2024-06-10T00:43:51.796 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-06-10T00:43:51.796 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.796 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.796 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.796 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:51.797 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.797 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-06-10T00:43:51.797 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-06-10T00:43:51.797 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-06-10T00:43:51.797 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-06-10T00:43:51.797 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-06-10T00:43:51.797 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-06-10T00:43:51.797 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:51.798 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-06-10T00:43:51.798 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-06-10T00:43:51.798 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-06-10T00:43:51.798 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-06-10T00:43:51.798 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-06-10T00:43:51.798 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-06-10T00:43:51.798 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.798 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.799 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.799 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:51.799 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:51.799 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x5630a256c6e0": { 2024-06-10T00:43:51.799 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-06-10T00:43:51.799 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-06-10T00:43:51.799 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-06-10T00:43:51.799 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-06-10T00:43:51.799 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-06-10T00:43:51.800 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:51.800 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-06-10T00:43:51.800 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-06-10T00:43:51.800 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-06-10T00:43:51.800 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-06-10T00:43:51.800 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-06-10T00:43:51.800 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-06-10T00:43:51.800 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:51.800 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:51.801 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:51.801 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:51.801 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:51.801 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:43:52.631 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-06-10T00:43:52.632 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-06-10T00:43:52.632 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-06-10T00:43:52.632 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-06-10T00:43:52.632 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-06-10T00:43:52.632 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-06-10T00:43:52.632 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-06-10T00:43:52.633 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-06-10T00:43:52.633 DEBUG:tasks.check_counter:Getting stats from j 2024-06-10T00:43:52.633 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-10T00:43:52.896 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:43:53.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:53 smithi105 ceph-mon[36321]: pgmap v342: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:43:53.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:53 smithi062 ceph-mon[32390]: pgmap v342: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:43:53.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:53 smithi138 ceph-mon[36549]: pgmap v342: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-06-10T00:43:54.723 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-06-10T00:43:54.723 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T00:43:54.723 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 268, 2024-06-10T00:43:54.723 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 257, 2024-06-10T00:43:54.723 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1593984, 2024-06-10T00:43:54.723 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 158480, 2024-06-10T00:43:54.723 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-06-10T00:43:54.724 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-06-10T00:43:54.724 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.053290920, 2024-06-10T00:43:54.724 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.017960384, 2024-06-10T00:43:54.724 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.037306346, 2024-06-10T00:43:54.724 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.004989091, 2024-06-10T00:43:54.724 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:54.724 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 256, 2024-06-10T00:43:54.725 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.012074314, 2024-06-10T00:43:54.725 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000047165 2024-06-10T00:43:54.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:54.725 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:54.725 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:54.725 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:54.725 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:54.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:54.726 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1593984, 2024-06-10T00:43:54.726 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 158480 2024-06-10T00:43:54.726 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:54.726 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T00:43:54.726 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 6, 2024-06-10T00:43:54.726 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 198, 2024-06-10T00:43:54.726 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 3344, 2024-06-10T00:43:54.726 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 190768, 2024-06-10T00:43:54.727 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-06-10T00:43:54.727 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-06-10T00:43:54.727 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.016364274, 2024-06-10T00:43:54.727 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.011234297, 2024-06-10T00:43:54.727 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.002259047, 2024-06-10T00:43:54.727 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T00:43:54.727 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:54.727 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 196, 2024-06-10T00:43:54.727 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.018310359, 2024-06-10T00:43:54.728 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000093420 2024-06-10T00:43:54.728 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:54.728 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:54.728 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:54.728 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:54.728 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:54.728 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:54.728 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-06-10T00:43:54.728 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 190768 2024-06-10T00:43:54.729 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:54.729 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T00:43:54.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 0, 2024-06-10T00:43:54.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2, 2024-06-10T00:43:54.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 0, 2024-06-10T00:43:54.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 0, 2024-06-10T00:43:54.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-06-10T00:43:54.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-06-10T00:43:54.729 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.000307517, 2024-06-10T00:43:54.730 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000000000, 2024-06-10T00:43:54.730 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000240209, 2024-06-10T00:43:54.730 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T00:43:54.730 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:54.730 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:54.730 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:54.730 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:54.730 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:54.731 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:54.731 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:54.731 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:54.731 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:54.731 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:54.731 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T00:43:54.731 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T00:43:54.731 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:54.732 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-06-10T00:43:54.732 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-06-10T00:43:54.732 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-06-10T00:43:54.732 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:54.732 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-06-10T00:43:54.732 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-06-10T00:43:54.732 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-06-10T00:43:54.732 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T00:43:54.733 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-06-10T00:43:54.733 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T00:43:54.733 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-06-10T00:43:54.733 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T00:43:54.733 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T00:43:54.733 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T00:43:54.733 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T00:43:54.733 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T00:43:54.733 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-06-10T00:43:54.734 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T00:43:54.734 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T00:43:54.734 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-06-10T00:43:54.734 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-06-10T00:43:54.734 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-06-10T00:43:54.734 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-06-10T00:43:54.734 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T00:43:54.735 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T00:43:54.735 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T00:43:54.735 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T00:43:54.735 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T00:43:54.735 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-06-10T00:43:54.735 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-06-10T00:43:54.735 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-06-10T00:43:54.735 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T00:43:54.735 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T00:43:54.736 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-06-10T00:43:54.736 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-06-10T00:43:54.736 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-06-10T00:43:54.736 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-06-10T00:43:54.736 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T00:43:54.736 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-06-10T00:43:54.736 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T00:43:54.736 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-06-10T00:43:54.737 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148714, 2024-06-10T00:43:54.737 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 30, 2024-06-10T00:43:54.737 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-06-10T00:43:54.737 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-06-10T00:43:54.737 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-06-10T00:43:54.737 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-06-10T00:43:54.737 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-06-10T00:43:54.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-06-10T00:43:54.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-06-10T00:43:54.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-06-10T00:43:54.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-06-10T00:43:54.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-06-10T00:43:54.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T00:43:54.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-06-10T00:43:54.739 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-06-10T00:43:54.739 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-06-10T00:43:54.739 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-06-10T00:43:54.739 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-06-10T00:43:54.739 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-06-10T00:43:54.740 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-06-10T00:43:54.740 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-06-10T00:43:54.740 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-06-10T00:43:54.740 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-06-10T00:43:54.740 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T00:43:54.740 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-06-10T00:43:54.741 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-06-10T00:43:54.741 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-06-10T00:43:54.741 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 270, 2024-06-10T00:43:54.741 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 1556180, 2024-06-10T00:43:54.741 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:54.741 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 270, 2024-06-10T00:43:54.741 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-06-10T00:43:54.741 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-06-10T00:43:54.741 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 270, 2024-06-10T00:43:54.742 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1556180, 2024-06-10T00:43:54.742 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-06-10T00:43:54.742 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-06-10T00:43:54.742 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-06-10T00:43:54.742 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-06-10T00:43:54.742 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:54.742 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-06-10T00:43:54.742 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-06-10T00:43:55.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:55 smithi062 ceph-mon[32390]: pgmap v343: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:55.454 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-06-10T00:43:55.455 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-06-10T00:43:55.455 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-06-10T00:43:55.455 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-06-10T00:43:55.455 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-06-10T00:43:55.455 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-06-10T00:43:55.455 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-06-10T00:43:55.455 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-06-10T00:43:55.455 DEBUG:tasks.check_counter:Getting stats from b 2024-06-10T00:43:55.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-10T00:43:55.483 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:55 smithi105 ceph-mon[36321]: pgmap v343: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:55.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:55 smithi138 ceph-mon[36549]: pgmap v343: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:43:55.704 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:43:57.417 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:57 smithi105 ceph-mon[36321]: pgmap v344: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:43:57.546 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-06-10T00:43:57.547 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T00:43:57.547 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 3574, 2024-06-10T00:43:57.547 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 4155, 2024-06-10T00:43:57.547 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 2743892, 2024-06-10T00:43:57.547 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 5953070, 2024-06-10T00:43:57.547 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 18, 2024-06-10T00:43:57.547 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2024-06-10T00:43:57.548 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.357856056, 2024-06-10T00:43:57.548 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.132910464, 2024-06-10T00:43:57.548 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.144414423, 2024-06-10T00:43:57.548 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.075708161, 2024-06-10T00:43:57.548 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:57.548 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4153, 2024-06-10T00:43:57.548 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.464308238, 2024-06-10T00:43:57.548 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000111800 2024-06-10T00:43:57.549 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.549 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:57.549 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3616, 2024-06-10T00:43:57.549 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000732589, 2024-06-10T00:43:57.549 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000202 2024-06-10T00:43:57.549 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.549 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 1832512, 2024-06-10T00:43:57.549 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 540048 2024-06-10T00:43:57.550 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.550 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T00:43:57.550 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 13117, 2024-06-10T00:43:57.550 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 17146, 2024-06-10T00:43:57.550 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 4501382, 2024-06-10T00:43:57.550 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 10997807, 2024-06-10T00:43:57.551 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 20, 2024-06-10T00:43:57.551 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 7, 2024-06-10T00:43:57.551 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.922853504, 2024-06-10T00:43:57.551 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.325275873, 2024-06-10T00:43:57.551 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.685044654, 2024-06-10T00:43:57.551 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.155652766, 2024-06-10T00:43:57.551 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:57.551 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 17145, 2024-06-10T00:43:57.551 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.831089785, 2024-06-10T00:43:57.552 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000048474 2024-06-10T00:43:57.552 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.552 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:57.552 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 24142, 2024-06-10T00:43:57.552 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.004086162, 2024-06-10T00:43:57.552 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000169 2024-06-10T00:43:57.552 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.552 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 316336, 2024-06-10T00:43:57.552 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 261984 2024-06-10T00:43:57.553 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.553 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T00:43:57.553 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 4680, 2024-06-10T00:43:57.553 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 5744, 2024-06-10T00:43:57.553 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1795472, 2024-06-10T00:43:57.553 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 13893902, 2024-06-10T00:43:57.553 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 91, 2024-06-10T00:43:57.553 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 9, 2024-06-10T00:43:57.554 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.413949984, 2024-06-10T00:43:57.554 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.144925944, 2024-06-10T00:43:57.554 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.166228773, 2024-06-10T00:43:57.554 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.091539105, 2024-06-10T00:43:57.554 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:43:57.554 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 5742, 2024-06-10T00:43:57.554 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.552705522, 2024-06-10T00:43:57.555 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000096256 2024-06-10T00:43:57.555 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.555 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:43:57.555 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 6757, 2024-06-10T00:43:57.555 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001248095, 2024-06-10T00:43:57.555 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000184 2024-06-10T00:43:57.555 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.555 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 42224, 2024-06-10T00:43:57.555 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 976 2024-06-10T00:43:57.556 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.556 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-06-10T00:43:57.556 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-06-10T00:43:57.556 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-06-10T00:43:57.556 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.556 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-06-10T00:43:57.556 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-06-10T00:43:57.556 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-06-10T00:43:57.556 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4297, 2024-06-10T00:43:57.557 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 3.543155369, 2024-06-10T00:43:57.557 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000824564 2024-06-10T00:43:57.557 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:43:57.557 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.557 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-06-10T00:43:57.557 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-06-10T00:43:57.557 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-06-10T00:43:57.557 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 901, 2024-06-10T00:43:57.557 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.064794026, 2024-06-10T00:43:57.558 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000071913 2024-06-10T00:43:57.558 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:43:57.558 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.558 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-06-10T00:43:57.558 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 2180, 2024-06-10T00:43:57.558 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 2172, 2024-06-10T00:43:57.558 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-06-10T00:43:57.558 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2172, 2024-06-10T00:43:57.559 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 631.993668381, 2024-06-10T00:43:57.559 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.290973143 2024-06-10T00:43:57.559 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.559 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-06-10T00:43:57.559 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 8, 2024-06-10T00:43:57.559 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-06-10T00:43:57.559 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-06-10T00:43:57.559 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 21, 2024-06-10T00:43:57.560 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 1, 2024-06-10T00:43:57.560 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-06-10T00:43:57.560 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 507, 2024-06-10T00:43:57.560 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 56, 2024-06-10T00:43:57.560 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 362, 2024-06-10T00:43:57.560 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 89, 2024-06-10T00:43:57.560 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 427, 2024-06-10T00:43:57.560 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 596, 2024-06-10T00:43:57.561 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 15, 2024-06-10T00:43:57.561 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 15, 2024-06-10T00:43:57.561 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 14, 2024-06-10T00:43:57.561 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 2953, 2024-06-10T00:43:57.561 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 2695, 2024-06-10T00:43:57.561 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 8, 2024-06-10T00:43:57.561 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-06-10T00:43:57.561 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-06-10T00:43:57.561 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-06-10T00:43:57.562 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 45, 2024-06-10T00:43:57.562 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-06-10T00:43:57.562 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-06-10T00:43:57.562 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 9, 2024-06-10T00:43:57.562 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 622, 2024-06-10T00:43:57.562 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 4, 2024-06-10T00:43:57.562 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 387, 2024-06-10T00:43:57.563 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-06-10T00:43:57.563 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 1748, 2024-06-10T00:43:57.563 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-06-10T00:43:57.563 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 325, 2024-06-10T00:43:57.563 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 13424230509, 2024-06-10T00:43:57.563 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 6, 2024-06-10T00:43:57.563 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 1748, 2024-06-10T00:43:57.563 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-06-10T00:43:57.564 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T00:43:57.564 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-06-10T00:43:57.564 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 640, 2024-06-10T00:43:57.564 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 10, 2024-06-10T00:43:57.564 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 630, 2024-06-10T00:43:57.564 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 1108, 2024-06-10T00:43:57.564 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-06-10T00:43:57.564 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 535, 2024-06-10T00:43:57.564 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 3167, 2024-06-10T00:43:57.564 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 6, 2024-06-10T00:43:57.565 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T00:43:57.565 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T00:43:57.565 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 6218, 2024-06-10T00:43:57.565 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 5364, 2024-06-10T00:43:57.565 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 182, 2024-06-10T00:43:57.565 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 920 2024-06-10T00:43:57.565 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.565 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-06-10T00:43:57.565 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 358, 2024-06-10T00:43:57.566 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-06-10T00:43:57.566 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-06-10T00:43:57.566 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-06-10T00:43:57.566 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 673, 2024-06-10T00:43:57.566 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-06-10T00:43:57.566 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-06-10T00:43:57.566 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-06-10T00:43:57.567 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 530, 2024-06-10T00:43:57.567 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 469, 2024-06-10T00:43:57.567 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 61, 2024-06-10T00:43:57.567 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-06-10T00:43:57.567 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-06-10T00:43:57.567 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-06-10T00:43:57.567 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-06-10T00:43:57.567 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-06-10T00:43:57.567 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-06-10T00:43:57.568 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 3, 2024-06-10T00:43:57.568 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 537, 2024-06-10T00:43:57.568 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 10, 2024-06-10T00:43:57.568 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 17, 2024-06-10T00:43:57.568 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-06-10T00:43:57.568 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 1, 2024-06-10T00:43:57.568 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-06-10T00:43:57.568 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-06-10T00:43:57.568 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.569 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-06-10T00:43:57.569 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-06-10T00:43:57.569 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 7949, 2024-06-10T00:43:57.569 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-06-10T00:43:57.569 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-06-10T00:43:57.569 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 7949, 2024-06-10T00:43:57.569 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-06-10T00:43:57.569 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-06-10T00:43:57.570 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 10, 2024-06-10T00:43:57.570 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-06-10T00:43:57.570 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-06-10T00:43:57.570 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 10, 2024-06-10T00:43:57.570 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-06-10T00:43:57.570 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-06-10T00:43:57.570 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-06-10T00:43:57.570 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-06-10T00:43:57.570 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 23278743, 2024-06-10T00:43:57.571 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2024-06-10T00:43:57.571 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-06-10T00:43:57.571 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1926, 2024-06-10T00:43:57.571 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 181.416552199, 2024-06-10T00:43:57.571 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.094193433 2024-06-10T00:43:57.571 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.571 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2024-06-10T00:43:57.571 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.572 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-06-10T00:43:57.572 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 324, 2024-06-10T00:43:57.572 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 1227, 2024-06-10T00:43:57.572 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 903, 2024-06-10T00:43:57.572 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 113, 2024-06-10T00:43:57.572 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 208, 2024-06-10T00:43:57.572 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 95, 2024-06-10T00:43:57.572 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 507, 2024-06-10T00:43:57.573 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 1656, 2024-06-10T00:43:57.573 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 1149, 2024-06-10T00:43:57.573 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 15, 2024-06-10T00:43:57.573 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 925, 2024-06-10T00:43:57.573 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 910, 2024-06-10T00:43:57.573 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 71868, 2024-06-10T00:43:57.573 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 231804 2024-06-10T00:43:57.573 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.574 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-06-10T00:43:57.574 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 2660, 2024-06-10T00:43:57.574 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 2, 2024-06-10T00:43:57.574 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 2180, 2024-06-10T00:43:57.574 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 44, 2024-06-10T00:43:57.574 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 6, 2024-06-10T00:43:57.574 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-06-10T00:43:57.574 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 765, 2024-06-10T00:43:57.574 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 29.993233408, 2024-06-10T00:43:57.575 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.039206841 2024-06-10T00:43:57.575 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.575 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-06-10T00:43:57.575 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 171, 2024-06-10T00:43:57.575 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.033497930, 2024-06-10T00:43:57.575 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000195894 2024-06-10T00:43:57.575 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.575 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-06-10T00:43:57.576 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.576 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.576 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.576 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.576 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-06-10T00:43:57.576 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 61, 2024-06-10T00:43:57.576 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.066954597, 2024-06-10T00:43:57.576 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001097616 2024-06-10T00:43:57.576 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.577 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-06-10T00:43:57.577 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 188, 2024-06-10T00:43:57.577 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 568.859712580, 2024-06-10T00:43:57.577 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 3.025849535 2024-06-10T00:43:57.577 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.577 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-06-10T00:43:57.577 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.577 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.577 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.578 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.578 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-06-10T00:43:57.578 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.578 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.578 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.578 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.578 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-06-10T00:43:57.578 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.578 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.579 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.579 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.579 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-06-10T00:43:57.579 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.579 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.579 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.579 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.579 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-06-10T00:43:57.580 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.580 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.580 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.580 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.580 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-06-10T00:43:57.581 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 10, 2024-06-10T00:43:57.581 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001952701, 2024-06-10T00:43:57.581 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000195270 2024-06-10T00:43:57.581 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.581 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-06-10T00:43:57.581 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 105, 2024-06-10T00:43:57.581 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 15.109051870, 2024-06-10T00:43:57.581 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.143895732 2024-06-10T00:43:57.582 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.582 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-06-10T00:43:57.582 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.582 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.582 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.582 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.582 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-06-10T00:43:57.583 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 10, 2024-06-10T00:43:57.583 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 4.389632428, 2024-06-10T00:43:57.583 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.438963242 2024-06-10T00:43:57.583 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.583 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-06-10T00:43:57.583 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 54, 2024-06-10T00:43:57.583 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.029920508, 2024-06-10T00:43:57.583 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000554083 2024-06-10T00:43:57.583 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.584 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-06-10T00:43:57.584 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 161, 2024-06-10T00:43:57.584 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.010319632, 2024-06-10T00:43:57.584 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.006275277 2024-06-10T00:43:57.584 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.584 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-06-10T00:43:57.584 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 113, 2024-06-10T00:43:57.584 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 11.465784620, 2024-06-10T00:43:57.584 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.101467120 2024-06-10T00:43:57.585 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.585 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-06-10T00:43:57.585 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.585 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.585 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.585 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.585 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-06-10T00:43:57.585 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.586 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.586 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.586 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.586 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-06-10T00:43:57.586 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 16, 2024-06-10T00:43:57.586 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.164369494, 2024-06-10T00:43:57.586 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.010273093 2024-06-10T00:43:57.586 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.587 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-06-10T00:43:57.587 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4, 2024-06-10T00:43:57.587 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.018370657, 2024-06-10T00:43:57.587 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.004592664 2024-06-10T00:43:57.587 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.587 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-06-10T00:43:57.587 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.587 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.587 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.588 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.588 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-06-10T00:43:57.588 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 8, 2024-06-10T00:43:57.588 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.036058481, 2024-06-10T00:43:57.588 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.004507310 2024-06-10T00:43:57.588 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.588 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-06-10T00:43:57.588 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.588 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.589 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.589 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.589 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-06-10T00:43:57.589 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4, 2024-06-10T00:43:57.589 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001388335, 2024-06-10T00:43:57.589 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000347083 2024-06-10T00:43:57.589 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.589 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-06-10T00:43:57.589 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.590 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.590 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.590 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.590 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-06-10T00:43:57.590 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 8, 2024-06-10T00:43:57.590 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.022249701, 2024-06-10T00:43:57.590 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002781212 2024-06-10T00:43:57.590 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.591 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-06-10T00:43:57.591 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 13, 2024-06-10T00:43:57.591 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.008760206, 2024-06-10T00:43:57.591 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000673862 2024-06-10T00:43:57.591 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.591 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-06-10T00:43:57.591 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 481, 2024-06-10T00:43:57.591 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.782411233, 2024-06-10T00:43:57.591 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001626634 2024-06-10T00:43:57.592 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.592 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-06-10T00:43:57.592 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-06-10T00:43:57.592 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-06-10T00:43:57.592 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.592 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.592 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.592 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:43:57.593 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.593 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-06-10T00:43:57.593 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 2, 2024-06-10T00:43:57.593 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 6, 2024-06-10T00:43:57.593 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 4, 2024-06-10T00:43:57.593 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 2, 2024-06-10T00:43:57.593 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-06-10T00:43:57.593 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 34, 2024-06-10T00:43:57.594 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 17, 2024-06-10T00:43:57.594 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 633, 2024-06-10T00:43:57.594 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-06-10T00:43:57.594 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.594 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-06-10T00:43:57.594 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-06-10T00:43:57.594 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-06-10T00:43:57.594 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T00:43:57.594 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-06-10T00:43:57.595 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T00:43:57.595 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-06-10T00:43:57.595 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T00:43:57.595 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T00:43:57.595 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T00:43:57.595 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T00:43:57.595 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T00:43:57.595 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-06-10T00:43:57.596 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T00:43:57.596 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T00:43:57.596 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-06-10T00:43:57.596 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-06-10T00:43:57.596 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-06-10T00:43:57.596 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-06-10T00:43:57.596 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T00:43:57.596 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T00:43:57.596 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T00:43:57.597 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T00:43:57.597 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T00:43:57.597 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-06-10T00:43:57.597 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-06-10T00:43:57.597 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-06-10T00:43:57.597 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T00:43:57.597 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T00:43:57.597 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-06-10T00:43:57.598 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-06-10T00:43:57.598 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-06-10T00:43:57.598 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-06-10T00:43:57.598 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T00:43:57.598 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-06-10T00:43:57.598 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T00:43:57.598 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-06-10T00:43:57.599 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 343416, 2024-06-10T00:43:57.599 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 448, 2024-06-10T00:43:57.599 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 88, 2024-06-10T00:43:57.599 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 1, 2024-06-10T00:43:57.599 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-06-10T00:43:57.599 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-06-10T00:43:57.599 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-06-10T00:43:57.599 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-06-10T00:43:57.599 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-06-10T00:43:57.600 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-06-10T00:43:57.600 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 9704, 2024-06-10T00:43:57.600 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 186, 2024-06-10T00:43:57.600 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T00:43:57.600 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-06-10T00:43:57.600 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-06-10T00:43:57.600 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-06-10T00:43:57.600 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 1373033, 2024-06-10T00:43:57.601 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 23665, 2024-06-10T00:43:57.601 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-06-10T00:43:57.601 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-06-10T00:43:57.601 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-06-10T00:43:57.601 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-06-10T00:43:57.601 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.601 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-06-10T00:43:57.601 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-06-10T00:43:57.602 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-06-10T00:43:57.602 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 5342, 2024-06-10T00:43:57.602 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 19414167, 2024-06-10T00:43:57.602 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 37, 2024-06-10T00:43:57.602 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 5342, 2024-06-10T00:43:57.602 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-06-10T00:43:57.602 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 5342, 2024-06-10T00:43:57.602 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 242.578169168, 2024-06-10T00:43:57.603 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.045409616 2024-06-10T00:43:57.603 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.603 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-06-10T00:43:57.603 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-06-10T00:43:57.603 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 5342, 2024-06-10T00:43:57.603 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 5493 2024-06-10T00:43:57.603 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.603 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 5342, 2024-06-10T00:43:57.604 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 1788, 2024-06-10T00:43:57.604 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 3554, 2024-06-10T00:43:57.604 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-06-10T00:43:57.604 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-06-10T00:43:57.604 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2024-06-10T00:43:57.604 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 11, 2024-06-10T00:43:57.604 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 30, 2024-06-10T00:43:57.604 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 1946, 2024-06-10T00:43:57.605 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 44, 2024-06-10T00:43:57.605 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-06-10T00:43:57.605 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-06-10T00:43:57.605 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-06-10T00:43:57.605 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 1, 2024-06-10T00:43:57.605 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 1425, 2024-06-10T00:43:57.605 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-06-10T00:43:57.605 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-06-10T00:43:57.606 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-06-10T00:43:57.606 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 1758, 2024-06-10T00:43:57.606 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 22, 2024-06-10T00:43:57.606 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-06-10T00:43:57.606 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-06-10T00:43:57.606 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-06-10T00:43:57.606 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-06-10T00:43:57.606 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-06-10T00:43:57.607 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-06-10T00:43:57.607 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T00:43:57.607 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-06-10T00:43:57.607 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-06-10T00:43:57.607 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 6, 2024-06-10T00:43:57.607 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-06-10T00:43:57.607 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-06-10T00:43:57.607 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-06-10T00:43:57.607 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-06-10T00:43:57.608 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-06-10T00:43:57.608 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-06-10T00:43:57.608 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-06-10T00:43:57.608 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-06-10T00:43:57.608 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-06-10T00:43:57.608 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-06-10T00:43:57.608 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-06-10T00:43:57.608 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-06-10T00:43:57.609 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-06-10T00:43:57.609 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-06-10T00:43:57.609 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-06-10T00:43:57.609 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-06-10T00:43:57.609 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 108, 2024-06-10T00:43:57.609 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-06-10T00:43:57.609 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 20, 2024-06-10T00:43:57.609 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 12, 2024-06-10T00:43:57.609 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 12, 2024-06-10T00:43:57.610 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-06-10T00:43:57.610 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-06-10T00:43:57.610 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 188, 2024-06-10T00:43:57.610 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2024-06-10T00:43:57.610 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 62 2024-06-10T00:43:57.610 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.610 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-06-10T00:43:57.610 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2024-06-10T00:43:57.611 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-06-10T00:43:57.611 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-06-10T00:43:57.611 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-06-10T00:43:57.611 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.611 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-06-10T00:43:57.611 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 316 2024-06-10T00:43:57.611 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.611 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-06-10T00:43:57.612 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-06-10T00:43:57.612 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 10, 2024-06-10T00:43:57.612 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-06-10T00:43:57.612 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 9, 2024-06-10T00:43:57.612 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 909, 2024-06-10T00:43:57.612 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 469, 2024-06-10T00:43:57.612 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-06-10T00:43:57.612 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.612 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T00:43:57.613 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:43:57.613 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-06-10T00:43:57.613 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:43:57.613 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 21367, 2024-06-10T00:43:57.613 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 7048661, 2024-06-10T00:43:57.613 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:57.613 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 21367, 2024-06-10T00:43:57.613 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-06-10T00:43:57.614 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-06-10T00:43:57.614 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 21367, 2024-06-10T00:43:57.614 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 7048661, 2024-06-10T00:43:57.614 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:43:57.614 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.614 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.614 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.614 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:43:57.615 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.615 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-06-10T00:43:57.615 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:43:57.615 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-06-10T00:43:57.615 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:43:57.615 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-06-10T00:43:57.615 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-06-10T00:43:57.615 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:57.616 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-06-10T00:43:57.616 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 5342, 2024-06-10T00:43:57.616 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 19465248, 2024-06-10T00:43:57.616 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 5342, 2024-06-10T00:43:57.616 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 19465248, 2024-06-10T00:43:57.616 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:43:57.616 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.616 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.617 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.617 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:43:57.617 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.617 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-06-10T00:43:57.617 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:43:57.617 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-06-10T00:43:57.617 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:43:57.618 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-06-10T00:43:57.618 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-06-10T00:43:57.618 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:57.618 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-06-10T00:43:57.618 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 5342, 2024-06-10T00:43:57.619 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 5342, 2024-06-10T00:43:57.619 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 5342, 2024-06-10T00:43:57.619 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 5342, 2024-06-10T00:43:57.619 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:43:57.619 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.619 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.619 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.619 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:43:57.619 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.620 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-06-10T00:43:57.620 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:43:57.620 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-06-10T00:43:57.620 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:43:57.620 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 469, 2024-06-10T00:43:57.621 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 51081, 2024-06-10T00:43:57.621 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:57.621 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 469, 2024-06-10T00:43:57.621 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-06-10T00:43:57.621 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-06-10T00:43:57.621 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 20, 2024-06-10T00:43:57.621 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 51081, 2024-06-10T00:43:57.621 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:43:57.621 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.622 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.622 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.622 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:43:57.622 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:43:57.622 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x55c443a54500": { 2024-06-10T00:43:57.622 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:43:57.622 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-06-10T00:43:57.622 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:43:57.623 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 7949, 2024-06-10T00:43:57.623 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 19084439, 2024-06-10T00:43:57.623 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:43:57.623 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 7949, 2024-06-10T00:43:57.623 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-06-10T00:43:57.623 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-06-10T00:43:57.623 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1926, 2024-06-10T00:43:57.623 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 19084439, 2024-06-10T00:43:57.624 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:43:57.624 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:43:57.624 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:43:57.624 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:43:57.624 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:43:57.624 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:43:57.624 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-06-10T00:43:57.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:57 smithi138 ceph-mon[36549]: pgmap v344: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:43:57.625 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:57 smithi062 ceph-mon[32390]: pgmap v344: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-10T00:43:58.337 INFO:tasks.check_counter:Daemon mds.b mds.exported=9 2024-06-10T00:43:58.338 INFO:tasks.check_counter:Daemon mds.b mds.imported=4 2024-06-10T00:43:58.338 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=358 2024-06-10T00:43:58.338 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-06-10T00:43:58.338 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-06-10T00:43:58.339 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-06-10T00:43:58.339 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-06-10T00:43:58.339 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=1 2024-06-10T00:43:58.339 DEBUG:tasks.check_counter:Getting stats from e 2024-06-10T00:43:58.339 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-10T00:43:58.621 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:43:59.466 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:43:59 smithi105 ceph-mon[36321]: pgmap v345: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:43:59.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:43:59 smithi062 ceph-mon[32390]: pgmap v345: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:43:59.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:43:59 smithi138 ceph-mon[36549]: pgmap v345: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-10T00:44:00.358 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-06-10T00:44:00.358 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T00:44:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1898, 2024-06-10T00:44:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 1877, 2024-06-10T00:44:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 8666151, 2024-06-10T00:44:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 671166, 2024-06-10T00:44:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 5, 2024-06-10T00:44:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 1, 2024-06-10T00:44:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.199952351, 2024-06-10T00:44:00.359 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.073131937, 2024-06-10T00:44:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.076462533, 2024-06-10T00:44:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.047824420, 2024-06-10T00:44:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1876, 2024-06-10T00:44:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.071133147, 2024-06-10T00:44:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000037917 2024-06-10T00:44:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.360 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 1687088, 2024-06-10T00:44:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 161408 2024-06-10T00:44:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.361 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T00:44:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 76, 2024-06-10T00:44:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 277, 2024-06-10T00:44:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 8408044, 2024-06-10T00:44:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 506121, 2024-06-10T00:44:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 7, 2024-06-10T00:44:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 5, 2024-06-10T00:44:00.362 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.028942213, 2024-06-10T00:44:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.009211864, 2024-06-10T00:44:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.011034036, 2024-06-10T00:44:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.002577098, 2024-06-10T00:44:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 274, 2024-06-10T00:44:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.019918271, 2024-06-10T00:44:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000072694 2024-06-10T00:44:00.363 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T00:44:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 481984 2024-06-10T00:44:00.364 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T00:44:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 800, 2024-06-10T00:44:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 800, 2024-06-10T00:44:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 4393938, 2024-06-10T00:44:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 253583, 2024-06-10T00:44:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 5, 2024-06-10T00:44:00.365 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-06-10T00:44:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.072956462, 2024-06-10T00:44:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.019167563, 2024-06-10T00:44:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.020622669, 2024-06-10T00:44:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.026777729, 2024-06-10T00:44:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 799, 2024-06-10T00:44:00.366 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.032358536, 2024-06-10T00:44:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000040498 2024-06-10T00:44:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.367 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-10T00:44:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 448 2024-06-10T00:44:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-06-10T00:44:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-06-10T00:44:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-06-10T00:44:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.368 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-06-10T00:44:00.369 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-06-10T00:44:00.369 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-06-10T00:44:00.369 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4119, 2024-06-10T00:44:00.369 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.101742037, 2024-06-10T00:44:00.369 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000024700 2024-06-10T00:44:00.369 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:00.369 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.369 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-06-10T00:44:00.370 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-06-10T00:44:00.370 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-06-10T00:44:00.370 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.370 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.370 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.370 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:00.370 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.371 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-06-10T00:44:00.371 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2024-06-10T00:44:00.371 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2024-06-10T00:44:00.371 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-06-10T00:44:00.371 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.371 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.371 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.371 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.371 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-06-10T00:44:00.372 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-06-10T00:44:00.372 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-06-10T00:44:00.372 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-06-10T00:44:00.372 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 0, 2024-06-10T00:44:00.372 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-06-10T00:44:00.372 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-06-10T00:44:00.372 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 939, 2024-06-10T00:44:00.373 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 35, 2024-06-10T00:44:00.373 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 904, 2024-06-10T00:44:00.373 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-06-10T00:44:00.373 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 890, 2024-06-10T00:44:00.373 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-06-10T00:44:00.373 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-06-10T00:44:00.373 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-06-10T00:44:00.374 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 14, 2024-06-10T00:44:00.374 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 0, 2024-06-10T00:44:00.374 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2024-06-10T00:44:00.374 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-06-10T00:44:00.374 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-06-10T00:44:00.374 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-06-10T00:44:00.374 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-06-10T00:44:00.374 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-06-10T00:44:00.375 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-06-10T00:44:00.375 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-06-10T00:44:00.375 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-06-10T00:44:00.375 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-06-10T00:44:00.375 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-06-10T00:44:00.375 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-06-10T00:44:00.375 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-06-10T00:44:00.375 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-06-10T00:44:00.376 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-06-10T00:44:00.376 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 325, 2024-06-10T00:44:00.376 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 13424230509, 2024-06-10T00:44:00.376 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 6, 2024-06-10T00:44:00.376 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T00:44:00.376 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-06-10T00:44:00.376 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T00:44:00.376 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-06-10T00:44:00.376 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-06-10T00:44:00.377 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T00:44:00.377 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T00:44:00.377 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-06-10T00:44:00.377 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-06-10T00:44:00.377 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T00:44:00.377 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-06-10T00:44:00.377 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T00:44:00.377 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T00:44:00.378 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T00:44:00.378 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-06-10T00:44:00.378 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-06-10T00:44:00.378 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-06-10T00:44:00.378 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2024-06-10T00:44:00.378 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.378 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-06-10T00:44:00.378 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-06-10T00:44:00.379 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-06-10T00:44:00.379 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-06-10T00:44:00.379 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-06-10T00:44:00.379 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-06-10T00:44:00.379 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-06-10T00:44:00.379 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-06-10T00:44:00.379 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-06-10T00:44:00.379 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-06-10T00:44:00.380 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-06-10T00:44:00.380 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-06-10T00:44:00.380 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-06-10T00:44:00.380 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-06-10T00:44:00.380 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-06-10T00:44:00.380 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-06-10T00:44:00.380 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-06-10T00:44:00.380 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-06-10T00:44:00.381 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-06-10T00:44:00.381 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-06-10T00:44:00.381 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T00:44:00.381 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-06-10T00:44:00.381 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-06-10T00:44:00.381 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-06-10T00:44:00.381 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-06-10T00:44:00.381 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-06-10T00:44:00.382 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.382 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-06-10T00:44:00.382 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-06-10T00:44:00.382 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 0, 2024-06-10T00:44:00.382 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-06-10T00:44:00.382 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-06-10T00:44:00.382 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 7949, 2024-06-10T00:44:00.382 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-06-10T00:44:00.383 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-06-10T00:44:00.383 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 0, 2024-06-10T00:44:00.383 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-06-10T00:44:00.385 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-06-10T00:44:00.386 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 10, 2024-06-10T00:44:00.386 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-06-10T00:44:00.386 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-06-10T00:44:00.386 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-06-10T00:44:00.386 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-06-10T00:44:00.386 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 23278743, 2024-06-10T00:44:00.386 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 23278607, 2024-06-10T00:44:00.387 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-06-10T00:44:00.387 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.387 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.387 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.387 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.387 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 7949 2024-06-10T00:44:00.388 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.388 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-06-10T00:44:00.388 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 287, 2024-06-10T00:44:00.388 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 1211, 2024-06-10T00:44:00.388 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 924, 2024-06-10T00:44:00.388 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 107, 2024-06-10T00:44:00.388 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 202, 2024-06-10T00:44:00.389 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 95, 2024-06-10T00:44:00.389 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 939, 2024-06-10T00:44:00.389 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 1577, 2024-06-10T00:44:00.389 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 638, 2024-06-10T00:44:00.389 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-06-10T00:44:00.389 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 0, 2024-06-10T00:44:00.389 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2024-06-10T00:44:00.390 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 55664, 2024-06-10T00:44:00.390 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215420 2024-06-10T00:44:00.390 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.390 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-06-10T00:44:00.390 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2024-06-10T00:44:00.390 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-06-10T00:44:00.390 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2024-06-10T00:44:00.391 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 0, 2024-06-10T00:44:00.391 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-06-10T00:44:00.391 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-06-10T00:44:00.391 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.391 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.391 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.392 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.392 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-06-10T00:44:00.392 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.392 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.392 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.392 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.392 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-06-10T00:44:00.392 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.393 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.393 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.393 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.393 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-06-10T00:44:00.393 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.393 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.394 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.394 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.394 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-06-10T00:44:00.394 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.394 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.394 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.394 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.395 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-06-10T00:44:00.395 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.395 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.395 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.395 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.395 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-06-10T00:44:00.395 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.396 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.396 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.396 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.396 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-06-10T00:44:00.396 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.396 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.396 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.397 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.397 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-06-10T00:44:00.397 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.397 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.397 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.397 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.397 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-06-10T00:44:00.398 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.398 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.398 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.398 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.398 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-06-10T00:44:00.398 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.398 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.399 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.399 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.400 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-06-10T00:44:00.400 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.400 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.400 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.400 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.400 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-06-10T00:44:00.401 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.401 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.401 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.401 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.401 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-06-10T00:44:00.401 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.401 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.402 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.402 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.402 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-06-10T00:44:00.402 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.402 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.402 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.402 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.403 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-06-10T00:44:00.403 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.403 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.403 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.403 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.403 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-06-10T00:44:00.403 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.404 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.404 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.404 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.404 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-06-10T00:44:00.404 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.404 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.404 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.405 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.405 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-06-10T00:44:00.405 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.405 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.405 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.405 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.405 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-06-10T00:44:00.406 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.406 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.406 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.406 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.406 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-06-10T00:44:00.406 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.407 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.407 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.407 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.407 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-06-10T00:44:00.407 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.407 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.407 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.408 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.408 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-06-10T00:44:00.408 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.408 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.408 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.408 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.408 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-06-10T00:44:00.409 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.409 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.409 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.409 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.409 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-06-10T00:44:00.409 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.410 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.410 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.410 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.410 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-06-10T00:44:00.410 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.410 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.410 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.411 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.411 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-06-10T00:44:00.411 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.411 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.411 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.411 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.411 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-06-10T00:44:00.412 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.412 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.412 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.412 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.412 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-06-10T00:44:00.412 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.412 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.413 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.413 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.413 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-06-10T00:44:00.413 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-06-10T00:44:00.413 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-06-10T00:44:00.413 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.413 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.414 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.414 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:00.414 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.414 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-06-10T00:44:00.414 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 2, 2024-06-10T00:44:00.414 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 6, 2024-06-10T00:44:00.414 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 4, 2024-06-10T00:44:00.415 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 4, 2024-06-10T00:44:00.415 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-06-10T00:44:00.415 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2024-06-10T00:44:00.415 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2024-06-10T00:44:00.415 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 3673, 2024-06-10T00:44:00.415 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-06-10T00:44:00.415 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.416 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-06-10T00:44:00.416 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-06-10T00:44:00.416 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-06-10T00:44:00.416 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T00:44:00.416 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-06-10T00:44:00.416 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T00:44:00.416 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-06-10T00:44:00.416 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T00:44:00.417 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T00:44:00.417 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T00:44:00.417 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T00:44:00.417 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T00:44:00.417 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-06-10T00:44:00.417 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T00:44:00.417 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T00:44:00.417 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-06-10T00:44:00.418 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-06-10T00:44:00.418 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-06-10T00:44:00.418 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-06-10T00:44:00.418 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T00:44:00.418 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T00:44:00.418 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T00:44:00.419 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T00:44:00.419 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T00:44:00.419 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-06-10T00:44:00.419 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-06-10T00:44:00.419 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-06-10T00:44:00.419 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T00:44:00.419 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T00:44:00.419 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-06-10T00:44:00.420 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-06-10T00:44:00.420 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-06-10T00:44:00.420 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-06-10T00:44:00.420 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T00:44:00.420 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-06-10T00:44:00.420 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T00:44:00.420 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-06-10T00:44:00.421 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 209703, 2024-06-10T00:44:00.421 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 347, 2024-06-10T00:44:00.421 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-06-10T00:44:00.421 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-06-10T00:44:00.421 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-06-10T00:44:00.421 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-06-10T00:44:00.421 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-06-10T00:44:00.421 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-06-10T00:44:00.422 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-06-10T00:44:00.422 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-06-10T00:44:00.422 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 9704, 2024-06-10T00:44:00.422 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 186, 2024-06-10T00:44:00.422 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T00:44:00.422 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-06-10T00:44:00.422 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-06-10T00:44:00.423 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-06-10T00:44:00.423 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 1514026, 2024-06-10T00:44:00.423 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 20075, 2024-06-10T00:44:00.423 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-06-10T00:44:00.423 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-06-10T00:44:00.423 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-06-10T00:44:00.423 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-06-10T00:44:00.423 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.424 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-06-10T00:44:00.424 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-06-10T00:44:00.424 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-06-10T00:44:00.424 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 2473, 2024-06-10T00:44:00.424 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 26, 2024-06-10T00:44:00.424 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 4, 2024-06-10T00:44:00.424 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 2473, 2024-06-10T00:44:00.425 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-06-10T00:44:00.425 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2473, 2024-06-10T00:44:00.425 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 68.260640064, 2024-06-10T00:44:00.425 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.027602361 2024-06-10T00:44:00.425 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.425 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-06-10T00:44:00.425 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-06-10T00:44:00.425 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2473, 2024-06-10T00:44:00.426 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2474 2024-06-10T00:44:00.426 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.426 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 2473, 2024-06-10T00:44:00.426 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 2473, 2024-06-10T00:44:00.426 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 0, 2024-06-10T00:44:00.426 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-06-10T00:44:00.426 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-06-10T00:44:00.427 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 1573, 2024-06-10T00:44:00.427 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 0, 2024-06-10T00:44:00.427 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 897, 2024-06-10T00:44:00.427 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 0, 2024-06-10T00:44:00.427 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 0, 2024-06-10T00:44:00.427 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-06-10T00:44:00.427 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-06-10T00:44:00.427 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-06-10T00:44:00.428 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-06-10T00:44:00.428 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 0, 2024-06-10T00:44:00.428 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-06-10T00:44:00.428 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-06-10T00:44:00.428 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-06-10T00:44:00.428 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 2, 2024-06-10T00:44:00.428 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 0, 2024-06-10T00:44:00.428 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-06-10T00:44:00.429 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-06-10T00:44:00.429 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-06-10T00:44:00.429 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-06-10T00:44:00.429 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-06-10T00:44:00.429 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-06-10T00:44:00.429 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T00:44:00.429 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-06-10T00:44:00.430 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-06-10T00:44:00.430 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-06-10T00:44:00.430 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-06-10T00:44:00.430 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-06-10T00:44:00.430 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-06-10T00:44:00.430 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-06-10T00:44:00.431 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-06-10T00:44:00.431 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-06-10T00:44:00.431 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-06-10T00:44:00.431 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-06-10T00:44:00.431 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-06-10T00:44:00.431 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-06-10T00:44:00.431 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-06-10T00:44:00.432 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-06-10T00:44:00.432 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-06-10T00:44:00.432 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-06-10T00:44:00.432 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-06-10T00:44:00.432 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-06-10T00:44:00.432 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 108, 2024-06-10T00:44:00.432 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-06-10T00:44:00.432 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 11, 2024-06-10T00:44:00.433 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 6, 2024-06-10T00:44:00.433 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 6, 2024-06-10T00:44:00.433 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-06-10T00:44:00.433 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-06-10T00:44:00.433 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 0, 2024-06-10T00:44:00.433 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 2, 2024-06-10T00:44:00.434 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 0 2024-06-10T00:44:00.434 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.434 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-06-10T00:44:00.434 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 0, 2024-06-10T00:44:00.434 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-06-10T00:44:00.434 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-06-10T00:44:00.434 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-06-10T00:44:00.435 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.435 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-06-10T00:44:00.435 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-06-10T00:44:00.435 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.435 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-06-10T00:44:00.435 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-06-10T00:44:00.436 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T00:44:00.436 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-06-10T00:44:00.436 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-06-10T00:44:00.436 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-06-10T00:44:00.436 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-06-10T00:44:00.436 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-06-10T00:44:00.437 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.437 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T00:44:00.437 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:00.437 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-06-10T00:44:00.437 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:00.437 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 2770, 2024-06-10T00:44:00.437 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 21200720, 2024-06-10T00:44:00.437 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:00.438 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 2770, 2024-06-10T00:44:00.438 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-06-10T00:44:00.438 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-06-10T00:44:00.438 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 2770, 2024-06-10T00:44:00.438 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 21200720, 2024-06-10T00:44:00.438 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:00.438 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.439 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.439 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.439 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:00.439 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.439 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-06-10T00:44:00.439 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:00.439 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-06-10T00:44:00.439 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:00.440 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-06-10T00:44:00.440 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-06-10T00:44:00.440 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:00.440 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:00.440 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 2473, 2024-06-10T00:44:00.440 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 19084449, 2024-06-10T00:44:00.440 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 2473, 2024-06-10T00:44:00.441 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 19084449, 2024-06-10T00:44:00.441 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:00.441 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.441 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.441 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.441 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:00.441 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.441 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-06-10T00:44:00.442 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:00.442 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-06-10T00:44:00.442 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:00.442 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-06-10T00:44:00.442 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-06-10T00:44:00.442 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:00.442 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:00.443 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 2473, 2024-06-10T00:44:00.443 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 2473, 2024-06-10T00:44:00.443 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 2473, 2024-06-10T00:44:00.443 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 2473, 2024-06-10T00:44:00.443 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:00.443 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.443 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.443 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.444 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:00.444 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.444 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-06-10T00:44:00.444 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:00.444 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-06-10T00:44:00.444 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:00.444 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-06-10T00:44:00.444 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-06-10T00:44:00.445 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:00.445 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:00.445 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-06-10T00:44:00.445 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-06-10T00:44:00.445 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-06-10T00:44:00.445 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-06-10T00:44:00.445 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:00.446 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.446 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.446 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.446 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:00.446 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:00.446 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x55600cce46e0": { 2024-06-10T00:44:00.446 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:00.447 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-06-10T00:44:00.447 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:00.447 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-06-10T00:44:00.447 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-06-10T00:44:00.447 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:00.447 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:00.447 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-06-10T00:44:00.448 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-06-10T00:44:00.448 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-06-10T00:44:00.448 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-06-10T00:44:00.448 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:00.448 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:00.448 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:00.448 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:00.448 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:00.449 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:00.449 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-06-10T00:44:01.145 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-06-10T00:44:01.145 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-06-10T00:44:01.145 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-06-10T00:44:01.145 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-06-10T00:44:01.146 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-06-10T00:44:01.146 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-06-10T00:44:01.146 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-06-10T00:44:01.146 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-06-10T00:44:01.146 DEBUG:tasks.check_counter:Getting stats from h 2024-06-10T00:44:01.146 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-10T00:44:01.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:01 smithi105 ceph-mon[36321]: pgmap v346: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:44:01.385 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:44:01.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:01 smithi062 ceph-mon[32390]: pgmap v346: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:44:01.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:01 smithi138 ceph-mon[36549]: pgmap v346: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-06-10T00:44:03.108 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-06-10T00:44:03.108 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T00:44:03.108 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 2182, 2024-06-10T00:44:03.108 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 1540, 2024-06-10T00:44:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 827956, 2024-06-10T00:44:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 892784, 2024-06-10T00:44:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 23, 2024-06-10T00:44:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2024-06-10T00:44:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.152977509, 2024-06-10T00:44:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.065767264, 2024-06-10T00:44:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.091148832, 2024-06-10T00:44:03.109 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.008336357, 2024-06-10T00:44:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1541, 2024-06-10T00:44:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.066369932, 2024-06-10T00:44:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000043069 2024-06-10T00:44:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:03.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3445, 2024-06-10T00:44:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000680525, 2024-06-10T00:44:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000197 2024-06-10T00:44:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-10T00:44:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 448 2024-06-10T00:44:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T00:44:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 367, 2024-06-10T00:44:03.111 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 393, 2024-06-10T00:44:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 163666, 2024-06-10T00:44:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 395355, 2024-06-10T00:44:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 19, 2024-06-10T00:44:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 7, 2024-06-10T00:44:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.035655938, 2024-06-10T00:44:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.012355971, 2024-06-10T00:44:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.023634421, 2024-06-10T00:44:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.002152358, 2024-06-10T00:44:03.112 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 390, 2024-06-10T00:44:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.084181685, 2024-06-10T00:44:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000215850 2024-06-10T00:44:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 467, 2024-06-10T00:44:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000094818, 2024-06-10T00:44:03.113 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000203 2024-06-10T00:44:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-06-10T00:44:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 156464 2024-06-10T00:44:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T00:44:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 448, 2024-06-10T00:44:03.114 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 590, 2024-06-10T00:44:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1786272, 2024-06-10T00:44:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 682380, 2024-06-10T00:44:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 19, 2024-06-10T00:44:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 7, 2024-06-10T00:44:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.074455849, 2024-06-10T00:44:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.026655397, 2024-06-10T00:44:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.049323478, 2024-06-10T00:44:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.007769240, 2024-06-10T00:44:03.115 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 587, 2024-06-10T00:44:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.040356662, 2024-06-10T00:44:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000068750 2024-06-10T00:44:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 175, 2024-06-10T00:44:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000037414, 2024-06-10T00:44:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000213 2024-06-10T00:44:03.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 1716832, 2024-06-10T00:44:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 513088 2024-06-10T00:44:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-06-10T00:44:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-06-10T00:44:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-06-10T00:44:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-06-10T00:44:03.117 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-06-10T00:44:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-06-10T00:44:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 209, 2024-06-10T00:44:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.032025951, 2024-06-10T00:44:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000153234 2024-06-10T00:44:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-06-10T00:44:03.118 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-06-10T00:44:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-06-10T00:44:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-06-10T00:44:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000008140, 2024-06-10T00:44:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000004070 2024-06-10T00:44:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-06-10T00:44:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 32, 2024-06-10T00:44:03.119 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 13, 2024-06-10T00:44:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-06-10T00:44:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 13, 2024-06-10T00:44:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.017591987, 2024-06-10T00:44:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001353229 2024-06-10T00:44:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-06-10T00:44:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 19, 2024-06-10T00:44:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-06-10T00:44:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-06-10T00:44:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 11, 2024-06-10T00:44:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-06-10T00:44:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-06-10T00:44:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 135, 2024-06-10T00:44:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 50, 2024-06-10T00:44:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 85, 2024-06-10T00:44:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-06-10T00:44:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 64, 2024-06-10T00:44:03.121 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 16, 2024-06-10T00:44:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 1, 2024-06-10T00:44:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 2, 2024-06-10T00:44:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 6, 2024-06-10T00:44:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 225, 2024-06-10T00:44:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 111, 2024-06-10T00:44:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 17, 2024-06-10T00:44:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 96, 2024-06-10T00:44:03.122 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-06-10T00:44:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-06-10T00:44:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-06-10T00:44:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-06-10T00:44:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-06-10T00:44:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-06-10T00:44:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-06-10T00:44:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 1, 2024-06-10T00:44:03.123 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 27, 2024-06-10T00:44:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-06-10T00:44:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 108, 2024-06-10T00:44:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-06-10T00:44:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 325, 2024-06-10T00:44:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 13424230509, 2024-06-10T00:44:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 6, 2024-06-10T00:44:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 108, 2024-06-10T00:44:03.124 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-06-10T00:44:03.125 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T00:44:03.125 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-06-10T00:44:03.125 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-06-10T00:44:03.125 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T00:44:03.125 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T00:44:03.125 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 108, 2024-06-10T00:44:03.125 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-06-10T00:44:03.125 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 6, 2024-06-10T00:44:03.126 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 18, 2024-06-10T00:44:03.126 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T00:44:03.126 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T00:44:03.126 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T00:44:03.126 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 84, 2024-06-10T00:44:03.126 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 65, 2024-06-10T00:44:03.126 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 12, 2024-06-10T00:44:03.127 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2024-06-10T00:44:03.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.127 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-06-10T00:44:03.127 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-06-10T00:44:03.127 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 272, 2024-06-10T00:44:03.127 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 183, 2024-06-10T00:44:03.127 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 168, 2024-06-10T00:44:03.127 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-06-10T00:44:03.128 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-06-10T00:44:03.128 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-06-10T00:44:03.128 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-06-10T00:44:03.128 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-06-10T00:44:03.128 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-06-10T00:44:03.128 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-06-10T00:44:03.128 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-06-10T00:44:03.128 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-06-10T00:44:03.129 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-06-10T00:44:03.129 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-06-10T00:44:03.129 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-06-10T00:44:03.129 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-06-10T00:44:03.129 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 3, 2024-06-10T00:44:03.129 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 187, 2024-06-10T00:44:03.129 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T00:44:03.129 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-06-10T00:44:03.130 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-06-10T00:44:03.130 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-06-10T00:44:03.130 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-06-10T00:44:03.130 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-06-10T00:44:03.130 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.130 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-06-10T00:44:03.130 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-06-10T00:44:03.131 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 99, 2024-06-10T00:44:03.131 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-06-10T00:44:03.131 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-06-10T00:44:03.131 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 99, 2024-06-10T00:44:03.131 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-06-10T00:44:03.131 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-06-10T00:44:03.131 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 2, 2024-06-10T00:44:03.131 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-06-10T00:44:03.131 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-06-10T00:44:03.132 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2024-06-10T00:44:03.132 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-06-10T00:44:03.132 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-06-10T00:44:03.132 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-06-10T00:44:03.132 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-06-10T00:44:03.132 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 4331240, 2024-06-10T00:44:03.132 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2024-06-10T00:44:03.132 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-06-10T00:44:03.132 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 29, 2024-06-10T00:44:03.133 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.397610699, 2024-06-10T00:44:03.133 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.013710713 2024-06-10T00:44:03.133 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.133 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2024-06-10T00:44:03.133 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.133 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-06-10T00:44:03.133 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 139, 2024-06-10T00:44:03.133 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 147, 2024-06-10T00:44:03.133 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 8, 2024-06-10T00:44:03.134 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 102, 2024-06-10T00:44:03.134 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 123, 2024-06-10T00:44:03.134 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 21, 2024-06-10T00:44:03.134 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 135, 2024-06-10T00:44:03.134 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 151, 2024-06-10T00:44:03.134 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 16, 2024-06-10T00:44:03.134 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 2, 2024-06-10T00:44:03.134 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 22, 2024-06-10T00:44:03.134 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 20, 2024-06-10T00:44:03.135 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 55280, 2024-06-10T00:44:03.135 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 223612 2024-06-10T00:44:03.135 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.135 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-06-10T00:44:03.135 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 32, 2024-06-10T00:44:03.135 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-06-10T00:44:03.135 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 32, 2024-06-10T00:44:03.135 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 26, 2024-06-10T00:44:03.136 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-06-10T00:44:03.136 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-06-10T00:44:03.136 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.136 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.136 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.136 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.136 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-06-10T00:44:03.136 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3, 2024-06-10T00:44:03.137 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000623181, 2024-06-10T00:44:03.137 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000207727 2024-06-10T00:44:03.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.137 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-06-10T00:44:03.137 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.137 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.137 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.137 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-06-10T00:44:03.138 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.138 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.138 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.138 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.138 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-06-10T00:44:03.138 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3, 2024-06-10T00:44:03.138 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.006018204, 2024-06-10T00:44:03.138 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002006068 2024-06-10T00:44:03.139 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.139 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-06-10T00:44:03.139 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.139 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.139 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.139 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.139 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-06-10T00:44:03.139 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2024-06-10T00:44:03.140 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000132274, 2024-06-10T00:44:03.140 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000132274 2024-06-10T00:44:03.140 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.140 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-06-10T00:44:03.140 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.140 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.140 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.140 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.140 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-06-10T00:44:03.141 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.141 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.141 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.141 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.141 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-06-10T00:44:03.141 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.141 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.141 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.141 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.142 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-06-10T00:44:03.142 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.142 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.142 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.143 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.143 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-06-10T00:44:03.143 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.143 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.144 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.144 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.144 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-06-10T00:44:03.144 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.144 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.144 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.144 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.144 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-06-10T00:44:03.145 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.145 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.145 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.145 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.145 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-06-10T00:44:03.145 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 6, 2024-06-10T00:44:03.145 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.010818328, 2024-06-10T00:44:03.145 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001803054 2024-06-10T00:44:03.145 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.146 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-06-10T00:44:03.146 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.146 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.146 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.146 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.146 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-06-10T00:44:03.146 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.146 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.147 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.147 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-06-10T00:44:03.147 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.147 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.147 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.147 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-06-10T00:44:03.148 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.148 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.148 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.148 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.148 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-06-10T00:44:03.148 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.148 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.148 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.149 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-06-10T00:44:03.149 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.149 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.149 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.149 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.149 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-06-10T00:44:03.149 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.149 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.150 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.150 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.150 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-06-10T00:44:03.150 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.150 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.150 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.150 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.150 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-06-10T00:44:03.151 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.151 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.151 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.151 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.151 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-06-10T00:44:03.151 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.151 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.151 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.152 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-06-10T00:44:03.152 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.152 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.152 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.152 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.152 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-06-10T00:44:03.152 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.152 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.153 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.153 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.153 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-06-10T00:44:03.153 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.153 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.153 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.153 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.153 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-06-10T00:44:03.154 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.154 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.154 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.154 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.154 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-06-10T00:44:03.154 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-06-10T00:44:03.154 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-06-10T00:44:03.154 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.155 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.155 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.155 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:03.155 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.155 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-06-10T00:44:03.155 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 2, 2024-06-10T00:44:03.155 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 4, 2024-06-10T00:44:03.155 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 2, 2024-06-10T00:44:03.156 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 2, 2024-06-10T00:44:03.156 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-06-10T00:44:03.156 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 9, 2024-06-10T00:44:03.156 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 4, 2024-06-10T00:44:03.156 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 615, 2024-06-10T00:44:03.156 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-06-10T00:44:03.156 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.156 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-06-10T00:44:03.157 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-06-10T00:44:03.157 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-06-10T00:44:03.157 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T00:44:03.157 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-06-10T00:44:03.157 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T00:44:03.157 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-06-10T00:44:03.157 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T00:44:03.157 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T00:44:03.158 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T00:44:03.158 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T00:44:03.158 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T00:44:03.158 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-06-10T00:44:03.158 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T00:44:03.158 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T00:44:03.158 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-06-10T00:44:03.158 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-06-10T00:44:03.159 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-06-10T00:44:03.159 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-06-10T00:44:03.159 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T00:44:03.159 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T00:44:03.159 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T00:44:03.159 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T00:44:03.159 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T00:44:03.159 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-06-10T00:44:03.159 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-06-10T00:44:03.160 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-06-10T00:44:03.160 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T00:44:03.160 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T00:44:03.160 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-06-10T00:44:03.160 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-06-10T00:44:03.160 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-06-10T00:44:03.160 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-06-10T00:44:03.160 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T00:44:03.160 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-06-10T00:44:03.161 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T00:44:03.161 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-06-10T00:44:03.161 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 308036, 2024-06-10T00:44:03.161 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 212, 2024-06-10T00:44:03.161 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-06-10T00:44:03.161 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-06-10T00:44:03.161 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-06-10T00:44:03.161 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-06-10T00:44:03.161 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-06-10T00:44:03.162 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-06-10T00:44:03.162 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-06-10T00:44:03.162 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-06-10T00:44:03.162 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 9704, 2024-06-10T00:44:03.162 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 186, 2024-06-10T00:44:03.162 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T00:44:03.162 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-06-10T00:44:03.162 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-06-10T00:44:03.163 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-06-10T00:44:03.163 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 558686, 2024-06-10T00:44:03.163 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 3896, 2024-06-10T00:44:03.163 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-06-10T00:44:03.163 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-06-10T00:44:03.163 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-06-10T00:44:03.163 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-06-10T00:44:03.163 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.164 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-06-10T00:44:03.164 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-06-10T00:44:03.164 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-06-10T00:44:03.164 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 193, 2024-06-10T00:44:03.164 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 150742, 2024-06-10T00:44:03.164 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 28, 2024-06-10T00:44:03.164 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 193, 2024-06-10T00:44:03.164 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-06-10T00:44:03.164 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 193, 2024-06-10T00:44:03.165 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 22.543550316, 2024-06-10T00:44:03.165 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.116805960 2024-06-10T00:44:03.165 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.165 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-06-10T00:44:03.165 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-06-10T00:44:03.165 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 193, 2024-06-10T00:44:03.165 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 228 2024-06-10T00:44:03.165 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.166 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 193, 2024-06-10T00:44:03.166 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 108, 2024-06-10T00:44:03.166 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 85, 2024-06-10T00:44:03.166 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-06-10T00:44:03.166 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-06-10T00:44:03.166 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2024-06-10T00:44:03.166 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 10, 2024-06-10T00:44:03.166 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 0, 2024-06-10T00:44:03.167 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 29, 2024-06-10T00:44:03.167 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 14, 2024-06-10T00:44:03.167 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-06-10T00:44:03.167 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-06-10T00:44:03.167 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-06-10T00:44:03.167 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-06-10T00:44:03.167 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 6, 2024-06-10T00:44:03.167 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-06-10T00:44:03.168 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-06-10T00:44:03.168 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-06-10T00:44:03.168 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 108, 2024-06-10T00:44:03.168 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 20, 2024-06-10T00:44:03.168 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-06-10T00:44:03.168 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-06-10T00:44:03.168 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-06-10T00:44:03.168 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-06-10T00:44:03.168 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-06-10T00:44:03.169 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-06-10T00:44:03.169 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T00:44:03.169 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-06-10T00:44:03.169 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-06-10T00:44:03.169 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-06-10T00:44:03.169 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-06-10T00:44:03.169 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-06-10T00:44:03.169 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-06-10T00:44:03.170 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-06-10T00:44:03.170 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-06-10T00:44:03.170 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-06-10T00:44:03.170 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-06-10T00:44:03.170 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-06-10T00:44:03.170 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-06-10T00:44:03.170 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-06-10T00:44:03.170 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-06-10T00:44:03.170 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-06-10T00:44:03.171 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-06-10T00:44:03.171 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-06-10T00:44:03.171 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-06-10T00:44:03.171 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-06-10T00:44:03.171 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 108, 2024-06-10T00:44:03.171 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-06-10T00:44:03.171 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 11, 2024-06-10T00:44:03.171 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 12, 2024-06-10T00:44:03.172 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 12, 2024-06-10T00:44:03.172 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-06-10T00:44:03.172 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-06-10T00:44:03.172 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 28, 2024-06-10T00:44:03.172 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2024-06-10T00:44:03.172 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 13 2024-06-10T00:44:03.172 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.172 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-06-10T00:44:03.173 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2024-06-10T00:44:03.173 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-06-10T00:44:03.173 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-06-10T00:44:03.173 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-06-10T00:44:03.173 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.173 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-06-10T00:44:03.173 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 88 2024-06-10T00:44:03.173 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.173 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-06-10T00:44:03.174 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-06-10T00:44:03.174 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T00:44:03.174 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-06-10T00:44:03.174 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-06-10T00:44:03.174 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-06-10T00:44:03.174 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-06-10T00:44:03.174 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-06-10T00:44:03.174 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.175 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T00:44:03.175 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:03.175 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-06-10T00:44:03.175 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:03.175 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 2993, 2024-06-10T00:44:03.175 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 2487526, 2024-06-10T00:44:03.175 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:03.175 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 2993, 2024-06-10T00:44:03.175 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-06-10T00:44:03.176 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-06-10T00:44:03.176 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 2993, 2024-06-10T00:44:03.176 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 2487526, 2024-06-10T00:44:03.176 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:03.176 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.176 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.176 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.176 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:03.176 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.177 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-06-10T00:44:03.177 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:03.177 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-06-10T00:44:03.177 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:03.177 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-06-10T00:44:03.177 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-06-10T00:44:03.177 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:03.177 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:03.177 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 193, 2024-06-10T00:44:03.178 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 150742, 2024-06-10T00:44:03.178 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 193, 2024-06-10T00:44:03.178 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 150742, 2024-06-10T00:44:03.178 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:03.178 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.178 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.178 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.178 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:03.179 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.179 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-06-10T00:44:03.179 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:03.179 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-06-10T00:44:03.179 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:03.179 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-06-10T00:44:03.179 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-06-10T00:44:03.179 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:03.179 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:03.179 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 193, 2024-06-10T00:44:03.180 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 193, 2024-06-10T00:44:03.180 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 193, 2024-06-10T00:44:03.180 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 193, 2024-06-10T00:44:03.180 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:03.180 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.180 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.180 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.180 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:03.181 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.181 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-06-10T00:44:03.181 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:03.181 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-06-10T00:44:03.181 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:03.181 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-06-10T00:44:03.181 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-06-10T00:44:03.182 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:03.182 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:03.182 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-06-10T00:44:03.182 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-06-10T00:44:03.182 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-06-10T00:44:03.182 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-06-10T00:44:03.182 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:03.183 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.183 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.183 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.183 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:03.183 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:03.183 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x55fbe3a4e500": { 2024-06-10T00:44:03.183 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:03.183 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-06-10T00:44:03.183 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:03.184 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 99, 2024-06-10T00:44:03.184 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 136936, 2024-06-10T00:44:03.184 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:03.184 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 99, 2024-06-10T00:44:03.184 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-06-10T00:44:03.184 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-06-10T00:44:03.184 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 29, 2024-06-10T00:44:03.184 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 136936, 2024-06-10T00:44:03.185 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:03.185 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:03.185 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:03.185 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:03.185 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:03.185 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:03.185 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-06-10T00:44:03.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:03 smithi105 ceph-mon[36321]: pgmap v347: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:44:03.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:03 smithi062 ceph-mon[32390]: pgmap v347: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:44:03.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:03 smithi138 ceph-mon[36549]: pgmap v347: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-06-10T00:44:03.924 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-06-10T00:44:03.925 INFO:tasks.check_counter:Daemon mds.h mds.imported=1 2024-06-10T00:44:03.925 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-06-10T00:44:03.925 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=272 2024-06-10T00:44:03.925 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-06-10T00:44:03.925 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-06-10T00:44:03.925 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-06-10T00:44:03.925 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-06-10T00:44:03.926 DEBUG:tasks.check_counter:Getting stats from k 2024-06-10T00:44:03.926 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-10T00:44:04.161 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:44:05.318 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:05 smithi105 ceph-mon[36321]: pgmap v348: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-06-10T00:44:05.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:05 smithi062 ceph-mon[32390]: pgmap v348: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-06-10T00:44:05.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:05 smithi138 ceph-mon[36549]: pgmap v348: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-06-10T00:44:05.929 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-06-10T00:44:05.930 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T00:44:05.930 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 877, 2024-06-10T00:44:05.931 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 879, 2024-06-10T00:44:05.931 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 366997, 2024-06-10T00:44:05.931 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 278950, 2024-06-10T00:44:05.931 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2024-06-10T00:44:05.932 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-06-10T00:44:05.932 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.080145655, 2024-06-10T00:44:05.932 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.031583064, 2024-06-10T00:44:05.932 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.018863626, 2024-06-10T00:44:05.932 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.021672833, 2024-06-10T00:44:05.933 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:05.933 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 877, 2024-06-10T00:44:05.933 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.042730172, 2024-06-10T00:44:05.933 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000048723 2024-06-10T00:44:05.933 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.934 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:05.934 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.934 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.934 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.935 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.935 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T00:44:05.935 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T00:44:05.935 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.936 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T00:44:05.936 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 760, 2024-06-10T00:44:05.936 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 957, 2024-06-10T00:44:05.936 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 186810, 2024-06-10T00:44:05.936 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 725177, 2024-06-10T00:44:05.936 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 7, 2024-06-10T00:44:05.936 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 5, 2024-06-10T00:44:05.937 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.083031441, 2024-06-10T00:44:05.937 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.028281645, 2024-06-10T00:44:05.937 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.019618735, 2024-06-10T00:44:05.937 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.026259021, 2024-06-10T00:44:05.937 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:05.937 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 954, 2024-06-10T00:44:05.937 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.045198694, 2024-06-10T00:44:05.937 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000047378 2024-06-10T00:44:05.938 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.938 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:05.938 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.938 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.938 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.938 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.938 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T00:44:05.938 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 484128 2024-06-10T00:44:05.939 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.939 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T00:44:05.939 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1154, 2024-06-10T00:44:05.939 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 1131, 2024-06-10T00:44:05.939 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1977312, 2024-06-10T00:44:05.939 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 435192, 2024-06-10T00:44:05.939 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 3, 2024-06-10T00:44:05.939 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-06-10T00:44:05.940 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.152278262, 2024-06-10T00:44:05.940 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.049225090, 2024-06-10T00:44:05.940 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.064889912, 2024-06-10T00:44:05.940 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.038746541, 2024-06-10T00:44:05.940 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:05.940 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1131, 2024-06-10T00:44:05.940 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.060885501, 2024-06-10T00:44:05.940 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000053833 2024-06-10T00:44:05.941 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.941 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:05.941 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.941 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.941 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.941 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.942 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 1691040, 2024-06-10T00:44:05.942 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 164256 2024-06-10T00:44:05.942 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.942 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-06-10T00:44:05.942 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-06-10T00:44:05.942 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-06-10T00:44:05.942 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.942 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-06-10T00:44:05.943 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-06-10T00:44:05.943 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-06-10T00:44:05.943 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4129, 2024-06-10T00:44:05.943 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.123493282, 2024-06-10T00:44:05.943 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000029908 2024-06-10T00:44:05.943 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:05.943 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.943 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-06-10T00:44:05.944 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-06-10T00:44:05.944 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-06-10T00:44:05.944 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.944 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.944 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.944 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:05.945 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.945 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-06-10T00:44:05.945 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2024-06-10T00:44:05.945 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2024-06-10T00:44:05.945 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-06-10T00:44:05.945 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.945 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.946 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.946 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.946 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-06-10T00:44:05.946 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-06-10T00:44:05.946 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-06-10T00:44:05.946 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-06-10T00:44:05.946 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 0, 2024-06-10T00:44:05.946 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-06-10T00:44:05.947 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-06-10T00:44:05.947 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 47, 2024-06-10T00:44:05.947 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2024-06-10T00:44:05.947 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 47, 2024-06-10T00:44:05.947 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-06-10T00:44:05.947 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 47, 2024-06-10T00:44:05.948 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-06-10T00:44:05.948 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-06-10T00:44:05.948 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-06-10T00:44:05.948 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 3, 2024-06-10T00:44:05.948 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 0, 2024-06-10T00:44:05.948 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2024-06-10T00:44:05.948 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-06-10T00:44:05.949 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-06-10T00:44:05.949 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-06-10T00:44:05.949 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-06-10T00:44:05.949 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-06-10T00:44:05.949 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-06-10T00:44:05.949 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-06-10T00:44:05.949 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-06-10T00:44:05.950 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-06-10T00:44:05.950 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-06-10T00:44:05.950 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-06-10T00:44:05.950 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-06-10T00:44:05.951 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-06-10T00:44:05.951 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-06-10T00:44:05.951 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 0, 2024-06-10T00:44:05.951 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 0, 2024-06-10T00:44:05.951 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-06-10T00:44:05.951 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T00:44:05.951 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-06-10T00:44:05.952 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T00:44:05.952 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-06-10T00:44:05.952 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-06-10T00:44:05.952 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T00:44:05.952 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T00:44:05.952 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-06-10T00:44:05.952 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-06-10T00:44:05.952 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T00:44:05.953 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-06-10T00:44:05.953 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T00:44:05.953 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T00:44:05.953 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T00:44:05.953 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-06-10T00:44:05.953 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-06-10T00:44:05.954 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-06-10T00:44:05.954 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2024-06-10T00:44:05.954 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.954 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-06-10T00:44:05.954 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-06-10T00:44:05.954 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-06-10T00:44:05.955 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-06-10T00:44:05.955 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-06-10T00:44:05.955 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-06-10T00:44:05.955 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-06-10T00:44:05.955 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-06-10T00:44:05.955 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-06-10T00:44:05.955 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-06-10T00:44:05.955 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-06-10T00:44:05.956 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-06-10T00:44:05.956 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-06-10T00:44:05.956 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-06-10T00:44:05.956 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-06-10T00:44:05.956 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-06-10T00:44:05.956 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-06-10T00:44:05.956 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-06-10T00:44:05.956 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-06-10T00:44:05.957 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-06-10T00:44:05.957 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T00:44:05.957 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-06-10T00:44:05.957 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-06-10T00:44:05.957 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-06-10T00:44:05.957 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-06-10T00:44:05.957 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-06-10T00:44:05.957 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.958 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-06-10T00:44:05.958 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-06-10T00:44:05.958 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 0, 2024-06-10T00:44:05.958 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-06-10T00:44:05.958 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-06-10T00:44:05.958 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 99, 2024-06-10T00:44:05.958 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-06-10T00:44:05.959 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-06-10T00:44:05.959 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 0, 2024-06-10T00:44:05.959 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-06-10T00:44:05.959 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-06-10T00:44:05.959 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2024-06-10T00:44:05.959 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-06-10T00:44:05.959 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-06-10T00:44:05.959 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-06-10T00:44:05.960 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-06-10T00:44:05.960 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 4331240, 2024-06-10T00:44:05.960 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 4330666, 2024-06-10T00:44:05.960 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-06-10T00:44:05.960 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.960 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.960 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.960 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.961 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 99 2024-06-10T00:44:05.961 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.961 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-06-10T00:44:05.961 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 50, 2024-06-10T00:44:05.961 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 50, 2024-06-10T00:44:05.961 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 0, 2024-06-10T00:44:05.961 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 10, 2024-06-10T00:44:05.961 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 10, 2024-06-10T00:44:05.962 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 0, 2024-06-10T00:44:05.962 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 47, 2024-06-10T00:44:05.962 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 47, 2024-06-10T00:44:05.962 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2024-06-10T00:44:05.962 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-06-10T00:44:05.962 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 0, 2024-06-10T00:44:05.962 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2024-06-10T00:44:05.962 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 50000, 2024-06-10T00:44:05.963 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215420 2024-06-10T00:44:05.963 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.963 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-06-10T00:44:05.963 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2024-06-10T00:44:05.963 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-06-10T00:44:05.963 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2024-06-10T00:44:05.963 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 0, 2024-06-10T00:44:05.964 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-06-10T00:44:05.964 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-06-10T00:44:05.964 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.964 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.964 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.964 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.964 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-06-10T00:44:05.964 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.965 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.965 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.965 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.965 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-06-10T00:44:05.965 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.965 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.965 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.965 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.966 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-06-10T00:44:05.966 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.966 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.966 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.966 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.966 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-06-10T00:44:05.966 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.966 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.967 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.967 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-06-10T00:44:05.967 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.967 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.967 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.967 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-06-10T00:44:05.968 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.968 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.968 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.968 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.968 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-06-10T00:44:05.968 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.968 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.968 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.969 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-06-10T00:44:05.969 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.969 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.969 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.969 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-06-10T00:44:05.970 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.970 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.970 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.970 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-06-10T00:44:05.970 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.970 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.970 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.971 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.971 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-06-10T00:44:05.971 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.972 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.972 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.972 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-06-10T00:44:05.972 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.972 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.973 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.973 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-06-10T00:44:05.973 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.973 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.973 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.973 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-06-10T00:44:05.974 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.974 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.974 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.974 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.974 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-06-10T00:44:05.974 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.974 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.975 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.975 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-06-10T00:44:05.975 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.976 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.976 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.976 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.976 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-06-10T00:44:05.976 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.976 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.976 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.977 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.977 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-06-10T00:44:05.977 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.977 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.977 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.977 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.977 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-06-10T00:44:05.977 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.978 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.978 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.978 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.978 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-06-10T00:44:05.978 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.978 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.978 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.979 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-06-10T00:44:05.979 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.979 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.979 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.979 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-06-10T00:44:05.979 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.980 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.980 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.980 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.980 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-06-10T00:44:05.980 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.980 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.980 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.980 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.981 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-06-10T00:44:05.981 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.981 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.981 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.981 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.981 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-06-10T00:44:05.981 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.982 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.982 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.982 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.982 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-06-10T00:44:05.982 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.982 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.982 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.982 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.983 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-06-10T00:44:05.983 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.983 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.983 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.983 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.983 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-06-10T00:44:05.983 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.983 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.984 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.984 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.984 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-06-10T00:44:05.984 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-06-10T00:44:05.984 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-06-10T00:44:05.984 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:05.984 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:05.984 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:05.985 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:05.985 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.985 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-06-10T00:44:05.985 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 2, 2024-06-10T00:44:05.985 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 4, 2024-06-10T00:44:05.985 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 2, 2024-06-10T00:44:05.985 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 3, 2024-06-10T00:44:05.985 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-06-10T00:44:05.986 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2024-06-10T00:44:05.986 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2024-06-10T00:44:05.986 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 2926, 2024-06-10T00:44:05.986 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-06-10T00:44:05.986 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.986 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-06-10T00:44:05.986 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-06-10T00:44:05.986 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-06-10T00:44:05.987 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T00:44:05.987 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-06-10T00:44:05.987 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T00:44:05.987 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-06-10T00:44:05.987 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T00:44:05.987 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T00:44:05.987 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T00:44:05.987 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T00:44:05.988 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T00:44:05.988 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-06-10T00:44:05.988 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T00:44:05.988 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T00:44:05.988 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-06-10T00:44:05.988 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-06-10T00:44:05.988 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-06-10T00:44:05.988 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-06-10T00:44:05.989 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T00:44:05.989 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T00:44:05.989 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T00:44:05.989 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T00:44:05.989 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T00:44:05.989 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-06-10T00:44:05.989 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-06-10T00:44:05.989 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-06-10T00:44:05.990 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T00:44:05.990 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T00:44:05.990 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-06-10T00:44:05.990 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-06-10T00:44:05.990 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-06-10T00:44:05.990 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-06-10T00:44:05.990 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T00:44:05.991 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-06-10T00:44:05.991 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T00:44:05.991 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-06-10T00:44:05.991 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 191773, 2024-06-10T00:44:05.991 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 96, 2024-06-10T00:44:05.991 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-06-10T00:44:05.991 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-06-10T00:44:05.991 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-06-10T00:44:05.992 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-06-10T00:44:05.992 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-06-10T00:44:05.992 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-06-10T00:44:05.992 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-06-10T00:44:05.992 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-06-10T00:44:05.992 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 9704, 2024-06-10T00:44:05.992 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 186, 2024-06-10T00:44:05.992 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T00:44:05.993 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-06-10T00:44:05.993 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-06-10T00:44:05.993 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-06-10T00:44:05.993 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 162265, 2024-06-10T00:44:05.993 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 1308, 2024-06-10T00:44:05.993 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-06-10T00:44:05.993 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-06-10T00:44:05.993 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-06-10T00:44:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-06-10T00:44:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-06-10T00:44:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-06-10T00:44:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-06-10T00:44:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 2487, 2024-06-10T00:44:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 21, 2024-06-10T00:44:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 3, 2024-06-10T00:44:05.995 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 2487, 2024-06-10T00:44:05.995 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-06-10T00:44:05.995 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2487, 2024-06-10T00:44:05.995 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 8.256835560, 2024-06-10T00:44:05.995 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.003319998 2024-06-10T00:44:05.995 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.995 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-06-10T00:44:05.995 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-06-10T00:44:05.996 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2487, 2024-06-10T00:44:05.996 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2488 2024-06-10T00:44:05.996 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:05.996 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 2487, 2024-06-10T00:44:05.996 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 2487, 2024-06-10T00:44:05.996 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 0, 2024-06-10T00:44:05.996 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-06-10T00:44:05.996 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-06-10T00:44:05.997 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 1608, 2024-06-10T00:44:05.997 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 0, 2024-06-10T00:44:05.997 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 877, 2024-06-10T00:44:05.997 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 0, 2024-06-10T00:44:05.997 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 0, 2024-06-10T00:44:05.997 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-06-10T00:44:05.997 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-06-10T00:44:05.997 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-06-10T00:44:05.997 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-06-10T00:44:05.998 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 0, 2024-06-10T00:44:05.998 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-06-10T00:44:05.998 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-06-10T00:44:05.998 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-06-10T00:44:05.998 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 1, 2024-06-10T00:44:05.998 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 0, 2024-06-10T00:44:05.998 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-06-10T00:44:05.999 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-06-10T00:44:05.999 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-06-10T00:44:05.999 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-06-10T00:44:05.999 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-06-10T00:44:05.999 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-06-10T00:44:05.999 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T00:44:05.999 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-06-10T00:44:05.999 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-06-10T00:44:06.000 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-06-10T00:44:06.000 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-06-10T00:44:06.000 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-06-10T00:44:06.000 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-06-10T00:44:06.000 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-06-10T00:44:06.000 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-06-10T00:44:06.000 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-06-10T00:44:06.000 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-06-10T00:44:06.001 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-06-10T00:44:06.001 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-06-10T00:44:06.001 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-06-10T00:44:06.001 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-06-10T00:44:06.001 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-06-10T00:44:06.001 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-06-10T00:44:06.001 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-06-10T00:44:06.001 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-06-10T00:44:06.002 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-06-10T00:44:06.002 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 108, 2024-06-10T00:44:06.002 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-06-10T00:44:06.002 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 11, 2024-06-10T00:44:06.002 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 5, 2024-06-10T00:44:06.002 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 5, 2024-06-10T00:44:06.002 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-06-10T00:44:06.003 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-06-10T00:44:06.003 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 0, 2024-06-10T00:44:06.003 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 2, 2024-06-10T00:44:06.003 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 0 2024-06-10T00:44:06.003 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:06.003 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-06-10T00:44:06.003 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 0, 2024-06-10T00:44:06.003 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-06-10T00:44:06.004 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-06-10T00:44:06.004 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-06-10T00:44:06.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:06.004 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-06-10T00:44:06.004 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-06-10T00:44:06.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:06.004 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-06-10T00:44:06.004 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-06-10T00:44:06.004 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T00:44:06.005 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-06-10T00:44:06.005 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-06-10T00:44:06.005 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-06-10T00:44:06.005 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-06-10T00:44:06.005 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-06-10T00:44:06.005 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:06.005 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T00:44:06.005 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:06.006 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-06-10T00:44:06.006 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:06.006 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 2787, 2024-06-10T00:44:06.006 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 2262014, 2024-06-10T00:44:06.006 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:06.007 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 2787, 2024-06-10T00:44:06.007 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-06-10T00:44:06.007 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-06-10T00:44:06.007 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 2787, 2024-06-10T00:44:06.007 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 2262014, 2024-06-10T00:44:06.007 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:06.007 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:06.008 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:06.008 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:06.008 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:06.008 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:06.008 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-06-10T00:44:06.008 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:06.009 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-06-10T00:44:06.009 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:06.009 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-06-10T00:44:06.009 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-06-10T00:44:06.009 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:06.009 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:06.009 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 2487, 2024-06-10T00:44:06.009 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 136941, 2024-06-10T00:44:06.010 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 2487, 2024-06-10T00:44:06.010 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 136941, 2024-06-10T00:44:06.010 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:06.010 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:06.010 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:06.010 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:06.010 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:06.011 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:06.011 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-06-10T00:44:06.011 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:06.011 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-06-10T00:44:06.011 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:06.011 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-06-10T00:44:06.011 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-06-10T00:44:06.012 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:06.012 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:06.012 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 2487, 2024-06-10T00:44:06.012 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 2487, 2024-06-10T00:44:06.012 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 2487, 2024-06-10T00:44:06.012 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 2487, 2024-06-10T00:44:06.012 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:06.013 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:06.013 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:06.013 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:06.013 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:06.013 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:06.013 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-06-10T00:44:06.013 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:06.013 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-06-10T00:44:06.014 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:06.014 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-06-10T00:44:06.014 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-06-10T00:44:06.014 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:06.014 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:06.014 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-06-10T00:44:06.014 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-06-10T00:44:06.014 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-06-10T00:44:06.015 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-06-10T00:44:06.015 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:06.015 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:06.015 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:06.015 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:06.015 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:06.015 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-06-10T00:44:06.015 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x555fba48c6e0": { 2024-06-10T00:44:06.016 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-06-10T00:44:06.016 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-06-10T00:44:06.016 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-06-10T00:44:06.016 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-06-10T00:44:06.016 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-06-10T00:44:06.016 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:06.016 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:06.017 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-06-10T00:44:06.017 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-06-10T00:44:06.017 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-06-10T00:44:06.017 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-06-10T00:44:06.017 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-06-10T00:44:06.017 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-06-10T00:44:06.017 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-06-10T00:44:06.017 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-06-10T00:44:06.018 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:06.018 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-06-10T00:44:06.018 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-06-10T00:44:06.771 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-06-10T00:44:06.771 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-06-10T00:44:06.771 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-06-10T00:44:06.771 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-06-10T00:44:06.772 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-06-10T00:44:06.772 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-06-10T00:44:06.772 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-06-10T00:44:06.772 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-06-10T00:44:06.772 DEBUG:tasks.check_counter:Getting stats from c 2024-06-10T00:44:06.772 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-10T00:44:07.020 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:44:07.398 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:07 smithi138 ceph-mon[36549]: pgmap v349: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-06-10T00:44:07.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:07 smithi105 ceph-mon[36321]: pgmap v349: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-06-10T00:44:07.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:07 smithi062 ceph-mon[32390]: pgmap v349: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-06-10T00:44:08.482 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:08 smithi138 ceph-mon[36549]: osdmap e109: 12 total, 12 up, 12 in 2024-06-10T00:44:08.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:08 smithi105 ceph-mon[36321]: osdmap e109: 12 total, 12 up, 12 in 2024-06-10T00:44:08.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:08 smithi062 ceph-mon[32390]: osdmap e109: 12 total, 12 up, 12 in 2024-06-10T00:44:08.904 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-06-10T00:44:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T00:44:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 2179, 2024-06-10T00:44:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 1556, 2024-06-10T00:44:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 836898, 2024-06-10T00:44:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 660060, 2024-06-10T00:44:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 15, 2024-06-10T00:44:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 2, 2024-06-10T00:44:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.163388197, 2024-06-10T00:44:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.059965653, 2024-06-10T00:44:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.107906230, 2024-06-10T00:44:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.011016853, 2024-06-10T00:44:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1559, 2024-06-10T00:44:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.083809183, 2024-06-10T00:44:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000053758 2024-06-10T00:44:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3510, 2024-06-10T00:44:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000800431, 2024-06-10T00:44:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000228 2024-06-10T00:44:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 4752, 2024-06-10T00:44:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 6320 2024-06-10T00:44:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T00:44:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 368, 2024-06-10T00:44:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 491, 2024-06-10T00:44:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 156591, 2024-06-10T00:44:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 1207494, 2024-06-10T00:44:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 35, 2024-06-10T00:44:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 11, 2024-06-10T00:44:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.050160654, 2024-06-10T00:44:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.020171559, 2024-06-10T00:44:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.022348345, 2024-06-10T00:44:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.006307279, 2024-06-10T00:44:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 486, 2024-06-10T00:44:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.099945695, 2024-06-10T00:44:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000205649 2024-06-10T00:44:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 234, 2024-06-10T00:44:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000041272, 2024-06-10T00:44:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000176 2024-06-10T00:44:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-06-10T00:44:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 491152 2024-06-10T00:44:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T00:44:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 723, 2024-06-10T00:44:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 742, 2024-06-10T00:44:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1869967, 2024-06-10T00:44:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 369664, 2024-06-10T00:44:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 8, 2024-06-10T00:44:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 7, 2024-06-10T00:44:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.089850695, 2024-06-10T00:44:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.030600308, 2024-06-10T00:44:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.065957120, 2024-06-10T00:44:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.008916058, 2024-06-10T00:44:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 739, 2024-06-10T00:44:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.111157032, 2024-06-10T00:44:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000150415 2024-06-10T00:44:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 539, 2024-06-10T00:44:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000167009, 2024-06-10T00:44:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000309 2024-06-10T00:44:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1709168, 2024-06-10T00:44:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 171696 2024-06-10T00:44:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-06-10T00:44:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-06-10T00:44:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-06-10T00:44:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-06-10T00:44:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-06-10T00:44:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-06-10T00:44:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 295, 2024-06-10T00:44:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.078830213, 2024-06-10T00:44:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000267221 2024-06-10T00:44:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-06-10T00:44:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-06-10T00:44:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-06-10T00:44:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2, 2024-06-10T00:44:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000006850, 2024-06-10T00:44:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000003425 2024-06-10T00:44:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-06-10T00:44:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 36, 2024-06-10T00:44:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 21, 2024-06-10T00:44:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-06-10T00:44:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 21, 2024-06-10T00:44:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 9.007580930, 2024-06-10T00:44:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.428932425 2024-06-10T00:44:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-06-10T00:44:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 15, 2024-06-10T00:44:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-06-10T00:44:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-06-10T00:44:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 11, 2024-06-10T00:44:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-06-10T00:44:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-06-10T00:44:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 165, 2024-06-10T00:44:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 50, 2024-06-10T00:44:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 115, 2024-06-10T00:44:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-06-10T00:44:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 94, 2024-06-10T00:44:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 16, 2024-06-10T00:44:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 1, 2024-06-10T00:44:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 1, 2024-06-10T00:44:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 7, 2024-06-10T00:44:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 229, 2024-06-10T00:44:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 117, 2024-06-10T00:44:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 15, 2024-06-10T00:44:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 97, 2024-06-10T00:44:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-06-10T00:44:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-06-10T00:44:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-06-10T00:44:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 1000, 2024-06-10T00:44:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 1, 2024-06-10T00:44:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-06-10T00:44:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-06-10T00:44:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 2, 2024-06-10T00:44:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 35, 2024-06-10T00:44:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-06-10T00:44:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 198, 2024-06-10T00:44:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-06-10T00:44:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 325, 2024-06-10T00:44:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 13424230509, 2024-06-10T00:44:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 6, 2024-06-10T00:44:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 198, 2024-06-10T00:44:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-06-10T00:44:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T00:44:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-06-10T00:44:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-06-10T00:44:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T00:44:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T00:44:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 198, 2024-06-10T00:44:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 2, 2024-06-10T00:44:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 16, 2024-06-10T00:44:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 32, 2024-06-10T00:44:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T00:44:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T00:44:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T00:44:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 201, 2024-06-10T00:44:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 151, 2024-06-10T00:44:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 21, 2024-06-10T00:44:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 0 2024-06-10T00:44:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-06-10T00:44:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 0, 2024-06-10T00:44:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 274, 2024-06-10T00:44:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 184, 2024-06-10T00:44:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 168, 2024-06-10T00:44:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 0, 2024-06-10T00:44:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-06-10T00:44:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-06-10T00:44:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-06-10T00:44:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 0, 2024-06-10T00:44:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 0, 2024-06-10T00:44:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-06-10T00:44:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-06-10T00:44:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-06-10T00:44:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-06-10T00:44:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-06-10T00:44:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-06-10T00:44:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-06-10T00:44:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 3, 2024-06-10T00:44:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 202, 2024-06-10T00:44:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T00:44:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-06-10T00:44:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-06-10T00:44:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-06-10T00:44:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-06-10T00:44:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-06-10T00:44:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-06-10T00:44:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-06-10T00:44:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 225, 2024-06-10T00:44:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-06-10T00:44:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-06-10T00:44:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 225, 2024-06-10T00:44:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-06-10T00:44:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-06-10T00:44:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 2, 2024-06-10T00:44:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-06-10T00:44:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-06-10T00:44:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 2, 2024-06-10T00:44:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 2, 2024-06-10T00:44:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-06-10T00:44:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-06-10T00:44:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-06-10T00:44:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 4503406, 2024-06-10T00:44:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-06-10T00:44:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-06-10T00:44:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 53, 2024-06-10T00:44:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 1.358986240, 2024-06-10T00:44:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.025641249 2024-06-10T00:44:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-06-10T00:44:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-06-10T00:44:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 169, 2024-06-10T00:44:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 177, 2024-06-10T00:44:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 8, 2024-06-10T00:44:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 103, 2024-06-10T00:44:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 123, 2024-06-10T00:44:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 20, 2024-06-10T00:44:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 165, 2024-06-10T00:44:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 181, 2024-06-10T00:44:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 16, 2024-06-10T00:44:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 1, 2024-06-10T00:44:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 44, 2024-06-10T00:44:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 43, 2024-06-10T00:44:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 58716, 2024-06-10T00:44:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 223612 2024-06-10T00:44:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-06-10T00:44:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 37, 2024-06-10T00:44:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 1, 2024-06-10T00:44:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 36, 2024-06-10T00:44:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 22, 2024-06-10T00:44:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 2, 2024-06-10T00:44:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-06-10T00:44:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-06-10T00:44:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3, 2024-06-10T00:44:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 8.960163984, 2024-06-10T00:44:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 2.986721328 2024-06-10T00:44:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-06-10T00:44:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-06-10T00:44:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-06-10T00:44:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2, 2024-06-10T00:44:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000664113, 2024-06-10T00:44:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000332056 2024-06-10T00:44:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-06-10T00:44:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-06-10T00:44:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1, 2024-06-10T00:44:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000158997, 2024-06-10T00:44:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000158997 2024-06-10T00:44:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-06-10T00:44:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-06-10T00:44:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-06-10T00:44:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-06-10T00:44:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-06-10T00:44:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-06-10T00:44:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-06-10T00:44:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-06-10T00:44:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 15, 2024-06-10T00:44:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.046593836, 2024-06-10T00:44:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.003106255 2024-06-10T00:44:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-06-10T00:44:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-06-10T00:44:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-06-10T00:44:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-06-10T00:44:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-06-10T00:44:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-06-10T00:44:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-06-10T00:44:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-06-10T00:44:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-06-10T00:44:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-06-10T00:44:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-06-10T00:44:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-06-10T00:44:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-06-10T00:44:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-06-10T00:44:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-06-10T00:44:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-06-10T00:44:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-06-10T00:44:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-06-10T00:44:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 2, 2024-06-10T00:44:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 3, 2024-06-10T00:44:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 1, 2024-06-10T00:44:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 2, 2024-06-10T00:44:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-06-10T00:44:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 17, 2024-06-10T00:44:08.955 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 8, 2024-06-10T00:44:08.955 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 622, 2024-06-10T00:44:08.955 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-06-10T00:44:08.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.955 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-06-10T00:44:08.955 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-06-10T00:44:08.955 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-06-10T00:44:08.956 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T00:44:08.956 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-06-10T00:44:08.956 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T00:44:08.956 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-06-10T00:44:08.956 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T00:44:08.956 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T00:44:08.956 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T00:44:08.956 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T00:44:08.957 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T00:44:08.957 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-06-10T00:44:08.957 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T00:44:08.957 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T00:44:08.957 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-06-10T00:44:08.957 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-06-10T00:44:08.957 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-06-10T00:44:08.958 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-06-10T00:44:08.958 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T00:44:08.958 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T00:44:08.958 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T00:44:08.958 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T00:44:08.958 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T00:44:08.958 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-06-10T00:44:08.958 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-06-10T00:44:08.959 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-06-10T00:44:08.959 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T00:44:08.959 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T00:44:08.959 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-06-10T00:44:08.959 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-06-10T00:44:08.959 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-06-10T00:44:08.959 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-06-10T00:44:08.959 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T00:44:08.960 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-06-10T00:44:08.960 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T00:44:08.960 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-06-10T00:44:08.960 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 284684, 2024-06-10T00:44:08.960 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 216, 2024-06-10T00:44:08.960 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-06-10T00:44:08.960 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-06-10T00:44:08.960 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-06-10T00:44:08.961 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-06-10T00:44:08.961 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-06-10T00:44:08.961 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-06-10T00:44:08.961 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-06-10T00:44:08.961 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-06-10T00:44:08.961 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 9704, 2024-06-10T00:44:08.961 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 186, 2024-06-10T00:44:08.962 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T00:44:08.962 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-06-10T00:44:08.962 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-06-10T00:44:08.962 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-06-10T00:44:08.962 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 647646, 2024-06-10T00:44:08.962 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 4583, 2024-06-10T00:44:08.962 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-06-10T00:44:08.963 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-06-10T00:44:08.963 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-06-10T00:44:08.963 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-06-10T00:44:08.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.963 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-06-10T00:44:08.963 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-06-10T00:44:08.963 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-06-10T00:44:08.963 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 324, 2024-06-10T00:44:08.964 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 326138, 2024-06-10T00:44:08.964 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 31, 2024-06-10T00:44:08.964 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 324, 2024-06-10T00:44:08.964 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-06-10T00:44:08.964 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 324, 2024-06-10T00:44:08.964 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 57.014863292, 2024-06-10T00:44:08.964 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.175971800 2024-06-10T00:44:08.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.965 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-06-10T00:44:08.965 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-06-10T00:44:08.965 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 324, 2024-06-10T00:44:08.965 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 376 2024-06-10T00:44:08.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.965 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 324, 2024-06-10T00:44:08.965 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 198, 2024-06-10T00:44:08.966 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 126, 2024-06-10T00:44:08.966 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-06-10T00:44:08.966 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-06-10T00:44:08.966 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 0, 2024-06-10T00:44:08.966 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 10, 2024-06-10T00:44:08.966 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 0, 2024-06-10T00:44:08.966 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 53, 2024-06-10T00:44:08.966 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 15, 2024-06-10T00:44:08.967 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-06-10T00:44:08.967 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-06-10T00:44:08.967 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-06-10T00:44:08.967 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-06-10T00:44:08.967 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 6, 2024-06-10T00:44:08.967 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-06-10T00:44:08.967 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-06-10T00:44:08.968 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-06-10T00:44:08.968 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 198, 2024-06-10T00:44:08.968 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 20, 2024-06-10T00:44:08.968 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-06-10T00:44:08.968 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-06-10T00:44:08.968 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-06-10T00:44:08.968 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-06-10T00:44:08.968 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-06-10T00:44:08.969 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-06-10T00:44:08.969 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T00:44:08.969 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-06-10T00:44:08.969 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-06-10T00:44:08.969 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-06-10T00:44:08.969 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-06-10T00:44:08.969 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-06-10T00:44:08.970 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-06-10T00:44:08.970 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-06-10T00:44:08.970 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-06-10T00:44:08.970 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-06-10T00:44:08.970 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-06-10T00:44:08.970 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-06-10T00:44:08.970 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-06-10T00:44:08.970 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-06-10T00:44:08.971 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-06-10T00:44:08.971 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-06-10T00:44:08.971 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-06-10T00:44:08.971 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-06-10T00:44:08.971 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-06-10T00:44:08.971 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-06-10T00:44:08.971 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 109, 2024-06-10T00:44:08.971 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-06-10T00:44:08.972 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 12, 2024-06-10T00:44:08.972 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 10, 2024-06-10T00:44:08.972 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 10, 2024-06-10T00:44:08.972 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-06-10T00:44:08.972 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-06-10T00:44:08.972 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 51, 2024-06-10T00:44:08.972 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 0, 2024-06-10T00:44:08.973 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 23 2024-06-10T00:44:08.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.973 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-06-10T00:44:08.973 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-06-10T00:44:08.973 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-06-10T00:44:08.973 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-06-10T00:44:08.973 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-06-10T00:44:08.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.974 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-06-10T00:44:08.974 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 88 2024-06-10T00:44:08.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.974 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-06-10T00:44:08.974 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-06-10T00:44:08.974 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T00:44:08.974 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-06-10T00:44:08.975 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 0, 2024-06-10T00:44:08.975 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 0, 2024-06-10T00:44:08.975 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 0, 2024-06-10T00:44:08.975 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-06-10T00:44:08.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.975 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T00:44:08.975 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:08.975 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-06-10T00:44:08.976 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:08.976 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 3266, 2024-06-10T00:44:08.976 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 2548554, 2024-06-10T00:44:08.976 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:08.976 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 3266, 2024-06-10T00:44:08.976 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-06-10T00:44:08.976 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-06-10T00:44:08.976 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 3266, 2024-06-10T00:44:08.977 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 2548554, 2024-06-10T00:44:08.977 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:08.977 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.977 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.977 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.978 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:08.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.978 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-06-10T00:44:08.978 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:08.978 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-06-10T00:44:08.978 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:08.978 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-06-10T00:44:08.978 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-06-10T00:44:08.979 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:08.979 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:08.979 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 324, 2024-06-10T00:44:08.979 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 326138, 2024-06-10T00:44:08.979 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 324, 2024-06-10T00:44:08.979 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 326138, 2024-06-10T00:44:08.979 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:08.980 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.980 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.980 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.980 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:08.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.980 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-06-10T00:44:08.980 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:08.980 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-06-10T00:44:08.981 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:08.981 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-06-10T00:44:08.981 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-06-10T00:44:08.981 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:08.981 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:08.981 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 324, 2024-06-10T00:44:08.981 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 324, 2024-06-10T00:44:08.981 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 324, 2024-06-10T00:44:08.982 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 324, 2024-06-10T00:44:08.982 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:08.982 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.982 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.982 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.982 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:08.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.982 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-06-10T00:44:08.983 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:08.983 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-06-10T00:44:08.983 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:08.983 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-06-10T00:44:08.983 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-06-10T00:44:08.983 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:08.983 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:08.983 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-06-10T00:44:08.984 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-06-10T00:44:08.984 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-06-10T00:44:08.984 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-06-10T00:44:08.984 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:08.984 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.984 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.984 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.984 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:08.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:08.985 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x555e3b944500": { 2024-06-10T00:44:08.985 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:08.985 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-06-10T00:44:08.985 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:08.985 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 225, 2024-06-10T00:44:08.985 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 309102, 2024-06-10T00:44:08.985 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:08.985 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 225, 2024-06-10T00:44:08.986 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-06-10T00:44:08.986 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-06-10T00:44:08.986 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 53, 2024-06-10T00:44:08.986 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 309102, 2024-06-10T00:44:08.986 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:08.986 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:08.986 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:08.987 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:08.987 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:08.987 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:08.987 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-06-10T00:44:09.485 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:09 smithi138 ceph-mon[36549]: pgmap v351: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:44:09.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:09 smithi105 ceph-mon[36321]: pgmap v351: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:44:09.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:09 smithi062 ceph-mon[32390]: pgmap v351: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:44:09.618 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-06-10T00:44:09.618 INFO:tasks.check_counter:Daemon mds.c mds.imported=2 2024-06-10T00:44:09.619 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-06-10T00:44:09.619 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=274 2024-06-10T00:44:09.619 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-06-10T00:44:09.619 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-06-10T00:44:09.619 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-06-10T00:44:09.619 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-06-10T00:44:09.619 DEBUG:tasks.check_counter:Getting stats from f 2024-06-10T00:44:09.620 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-10T00:44:09.860 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:44:11.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:11 smithi138 ceph-mon[36549]: pgmap v352: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:44:11.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:11 smithi105 ceph-mon[36321]: pgmap v352: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:44:11.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:11 smithi062 ceph-mon[32390]: pgmap v352: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-06-10T00:44:11.685 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-06-10T00:44:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T00:44:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 395, 2024-06-10T00:44:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 323, 2024-06-10T00:44:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 211946, 2024-06-10T00:44:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 541266, 2024-06-10T00:44:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 11, 2024-06-10T00:44:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 3, 2024-06-10T00:44:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.034239723, 2024-06-10T00:44:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.010690128, 2024-06-10T00:44:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.017076016, 2024-06-10T00:44:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.005957457, 2024-06-10T00:44:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 325, 2024-06-10T00:44:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.041600565, 2024-06-10T00:44:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000128001 2024-06-10T00:44:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 336, 2024-06-10T00:44:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000065648, 2024-06-10T00:44:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000195 2024-06-10T00:44:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 2976, 2024-06-10T00:44:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 3824 2024-06-10T00:44:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T00:44:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 492, 2024-06-10T00:44:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 610, 2024-06-10T00:44:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1787491, 2024-06-10T00:44:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 723772, 2024-06-10T00:44:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 22, 2024-06-10T00:44:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 8, 2024-06-10T00:44:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.080475578, 2024-06-10T00:44:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.027035205, 2024-06-10T00:44:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.051768643, 2024-06-10T00:44:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.008622878, 2024-06-10T00:44:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 606, 2024-06-10T00:44:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.058076897, 2024-06-10T00:44:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000095836 2024-06-10T00:44:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 118, 2024-06-10T00:44:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000027318, 2024-06-10T00:44:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000231 2024-06-10T00:44:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1704416, 2024-06-10T00:44:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 507216 2024-06-10T00:44:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T00:44:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 2362, 2024-06-10T00:44:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 1822, 2024-06-10T00:44:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 855331, 2024-06-10T00:44:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 1028369, 2024-06-10T00:44:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 27, 2024-06-10T00:44:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 7, 2024-06-10T00:44:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.177822437, 2024-06-10T00:44:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.068375312, 2024-06-10T00:44:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.125933364, 2024-06-10T00:44:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.007485160, 2024-06-10T00:44:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1819, 2024-06-10T00:44:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.222211551, 2024-06-10T00:44:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000122161 2024-06-10T00:44:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3766, 2024-06-10T00:44:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000901929, 2024-06-10T00:44:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000239 2024-06-10T00:44:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 4912, 2024-06-10T00:44:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 160368 2024-06-10T00:44:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-06-10T00:44:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-06-10T00:44:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-06-10T00:44:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-06-10T00:44:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-06-10T00:44:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-06-10T00:44:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 295, 2024-06-10T00:44:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.079006931, 2024-06-10T00:44:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000267820 2024-06-10T00:44:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-06-10T00:44:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-06-10T00:44:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-06-10T00:44:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2, 2024-06-10T00:44:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000007055, 2024-06-10T00:44:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000003527 2024-06-10T00:44:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-06-10T00:44:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 24, 2024-06-10T00:44:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 22, 2024-06-10T00:44:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-06-10T00:44:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 22, 2024-06-10T00:44:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.088371075, 2024-06-10T00:44:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.004016867 2024-06-10T00:44:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-06-10T00:44:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 2, 2024-06-10T00:44:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-06-10T00:44:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-06-10T00:44:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 11, 2024-06-10T00:44:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-06-10T00:44:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-06-10T00:44:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 162, 2024-06-10T00:44:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 50, 2024-06-10T00:44:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 112, 2024-06-10T00:44:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-06-10T00:44:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 91, 2024-06-10T00:44:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 17, 2024-06-10T00:44:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 0, 2024-06-10T00:44:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 0, 2024-06-10T00:44:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 7, 2024-06-10T00:44:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 219, 2024-06-10T00:44:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 117, 2024-06-10T00:44:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 2, 2024-06-10T00:44:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 98, 2024-06-10T00:44:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-06-10T00:44:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-06-10T00:44:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-06-10T00:44:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-06-10T00:44:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-06-10T00:44:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-06-10T00:44:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-06-10T00:44:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 2, 2024-06-10T00:44:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 33, 2024-06-10T00:44:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-06-10T00:44:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 186, 2024-06-10T00:44:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-06-10T00:44:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 325, 2024-06-10T00:44:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 13424230509, 2024-06-10T00:44:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 6, 2024-06-10T00:44:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 186, 2024-06-10T00:44:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-06-10T00:44:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T00:44:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-06-10T00:44:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-06-10T00:44:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T00:44:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T00:44:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 186, 2024-06-10T00:44:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-06-10T00:44:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 13, 2024-06-10T00:44:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 28, 2024-06-10T00:44:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T00:44:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T00:44:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T00:44:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 221, 2024-06-10T00:44:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 178, 2024-06-10T00:44:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 14, 2024-06-10T00:44:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 0 2024-06-10T00:44:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-06-10T00:44:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 0, 2024-06-10T00:44:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 274, 2024-06-10T00:44:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 184, 2024-06-10T00:44:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 169, 2024-06-10T00:44:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 0, 2024-06-10T00:44:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-06-10T00:44:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-06-10T00:44:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-06-10T00:44:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 0, 2024-06-10T00:44:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 0, 2024-06-10T00:44:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-06-10T00:44:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-06-10T00:44:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-06-10T00:44:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-06-10T00:44:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-06-10T00:44:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-06-10T00:44:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-06-10T00:44:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 3, 2024-06-10T00:44:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 200, 2024-06-10T00:44:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T00:44:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-06-10T00:44:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-06-10T00:44:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-06-10T00:44:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-06-10T00:44:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-06-10T00:44:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-06-10T00:44:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-06-10T00:44:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 257, 2024-06-10T00:44:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-06-10T00:44:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-06-10T00:44:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 257, 2024-06-10T00:44:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-06-10T00:44:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-06-10T00:44:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 2, 2024-06-10T00:44:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-06-10T00:44:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-06-10T00:44:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 2, 2024-06-10T00:44:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 2, 2024-06-10T00:44:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-06-10T00:44:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-06-10T00:44:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-06-10T00:44:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 4539230, 2024-06-10T00:44:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-06-10T00:44:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-06-10T00:44:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 58, 2024-06-10T00:44:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 1.905753762, 2024-06-10T00:44:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.032857823 2024-06-10T00:44:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-06-10T00:44:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-06-10T00:44:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 166, 2024-06-10T00:44:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 174, 2024-06-10T00:44:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 8, 2024-06-10T00:44:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 103, 2024-06-10T00:44:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 123, 2024-06-10T00:44:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 20, 2024-06-10T00:44:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 162, 2024-06-10T00:44:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 179, 2024-06-10T00:44:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 17, 2024-06-10T00:44:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 0, 2024-06-10T00:44:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 44, 2024-06-10T00:44:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 44, 2024-06-10T00:44:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 56276, 2024-06-10T00:44:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 223612 2024-06-10T00:44:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-06-10T00:44:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 28, 2024-06-10T00:44:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 0, 2024-06-10T00:44:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 24, 2024-06-10T00:44:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 26, 2024-06-10T00:44:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 1, 2024-06-10T00:44:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-06-10T00:44:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-06-10T00:44:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1, 2024-06-10T00:44:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.001580622, 2024-06-10T00:44:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001580622 2024-06-10T00:44:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-06-10T00:44:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-06-10T00:44:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-06-10T00:44:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3, 2024-06-10T00:44:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.001421984, 2024-06-10T00:44:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000473994 2024-06-10T00:44:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-06-10T00:44:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-06-10T00:44:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-06-10T00:44:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-06-10T00:44:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-06-10T00:44:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-06-10T00:44:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-06-10T00:44:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-06-10T00:44:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-06-10T00:44:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-06-10T00:44:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 18, 2024-06-10T00:44:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.085368469, 2024-06-10T00:44:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.004742692 2024-06-10T00:44:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-06-10T00:44:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-06-10T00:44:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-06-10T00:44:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-06-10T00:44:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-06-10T00:44:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-06-10T00:44:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-06-10T00:44:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-06-10T00:44:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-06-10T00:44:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-06-10T00:44:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-06-10T00:44:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-06-10T00:44:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-06-10T00:44:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-06-10T00:44:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-06-10T00:44:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-06-10T00:44:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-06-10T00:44:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-06-10T00:44:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 2, 2024-06-10T00:44:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 4, 2024-06-10T00:44:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 2, 2024-06-10T00:44:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 2, 2024-06-10T00:44:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-06-10T00:44:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 18, 2024-06-10T00:44:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 9, 2024-06-10T00:44:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 643, 2024-06-10T00:44:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-06-10T00:44:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-06-10T00:44:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-06-10T00:44:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-06-10T00:44:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T00:44:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-06-10T00:44:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T00:44:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-06-10T00:44:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T00:44:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T00:44:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T00:44:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T00:44:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T00:44:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-06-10T00:44:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T00:44:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T00:44:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-06-10T00:44:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-06-10T00:44:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-06-10T00:44:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-06-10T00:44:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T00:44:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T00:44:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T00:44:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T00:44:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T00:44:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-06-10T00:44:11.739 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-06-10T00:44:11.739 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-06-10T00:44:11.739 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T00:44:11.739 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T00:44:11.739 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-06-10T00:44:11.739 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-06-10T00:44:11.739 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-06-10T00:44:11.739 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-06-10T00:44:11.740 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T00:44:11.740 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-06-10T00:44:11.740 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T00:44:11.740 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-06-10T00:44:11.740 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 276034, 2024-06-10T00:44:11.740 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 204, 2024-06-10T00:44:11.740 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-06-10T00:44:11.740 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-06-10T00:44:11.741 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-06-10T00:44:11.741 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-06-10T00:44:11.741 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-06-10T00:44:11.741 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-06-10T00:44:11.741 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-06-10T00:44:11.741 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-06-10T00:44:11.741 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 9944, 2024-06-10T00:44:11.741 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 192, 2024-06-10T00:44:11.742 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T00:44:11.742 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-06-10T00:44:11.742 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-06-10T00:44:11.742 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-06-10T00:44:11.742 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 638764, 2024-06-10T00:44:11.742 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 4534, 2024-06-10T00:44:11.742 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-06-10T00:44:11.742 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-06-10T00:44:11.743 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-06-10T00:44:11.743 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-06-10T00:44:11.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.743 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-06-10T00:44:11.743 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-06-10T00:44:11.743 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-06-10T00:44:11.743 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 310, 2024-06-10T00:44:11.743 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 361058, 2024-06-10T00:44:11.743 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 33, 2024-06-10T00:44:11.744 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 310, 2024-06-10T00:44:11.744 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-06-10T00:44:11.744 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 310, 2024-06-10T00:44:11.744 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 132.727353732, 2024-06-10T00:44:11.744 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.428152753 2024-06-10T00:44:11.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.744 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-06-10T00:44:11.744 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-06-10T00:44:11.744 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 310, 2024-06-10T00:44:11.745 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 357 2024-06-10T00:44:11.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.745 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 310, 2024-06-10T00:44:11.745 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 186, 2024-06-10T00:44:11.745 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 124, 2024-06-10T00:44:11.745 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-06-10T00:44:11.745 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-06-10T00:44:11.745 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 0, 2024-06-10T00:44:11.746 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 10, 2024-06-10T00:44:11.746 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 0, 2024-06-10T00:44:11.746 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 58, 2024-06-10T00:44:11.746 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 13, 2024-06-10T00:44:11.746 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-06-10T00:44:11.746 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-06-10T00:44:11.746 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-06-10T00:44:11.746 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-06-10T00:44:11.747 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 6, 2024-06-10T00:44:11.747 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-06-10T00:44:11.747 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-06-10T00:44:11.747 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-06-10T00:44:11.747 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 186, 2024-06-10T00:44:11.747 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 20, 2024-06-10T00:44:11.747 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-06-10T00:44:11.747 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-06-10T00:44:11.747 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-06-10T00:44:11.748 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-06-10T00:44:11.748 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-06-10T00:44:11.748 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-06-10T00:44:11.748 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T00:44:11.748 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-06-10T00:44:11.748 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-06-10T00:44:11.748 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-06-10T00:44:11.748 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-06-10T00:44:11.748 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-06-10T00:44:11.749 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-06-10T00:44:11.749 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-06-10T00:44:11.749 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-06-10T00:44:11.749 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-06-10T00:44:11.749 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-06-10T00:44:11.749 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-06-10T00:44:11.749 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-06-10T00:44:11.749 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-06-10T00:44:11.749 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-06-10T00:44:11.750 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-06-10T00:44:11.750 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-06-10T00:44:11.750 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-06-10T00:44:11.750 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-06-10T00:44:11.750 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-06-10T00:44:11.750 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 109, 2024-06-10T00:44:11.750 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-06-10T00:44:11.750 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 12, 2024-06-10T00:44:11.751 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 11, 2024-06-10T00:44:11.751 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 11, 2024-06-10T00:44:11.751 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-06-10T00:44:11.751 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-06-10T00:44:11.751 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 42, 2024-06-10T00:44:11.751 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 0, 2024-06-10T00:44:11.751 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 22 2024-06-10T00:44:11.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.752 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-06-10T00:44:11.752 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-06-10T00:44:11.752 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-06-10T00:44:11.752 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-06-10T00:44:11.752 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-06-10T00:44:11.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.752 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-06-10T00:44:11.752 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 88 2024-06-10T00:44:11.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.753 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-06-10T00:44:11.753 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-06-10T00:44:11.753 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T00:44:11.753 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-06-10T00:44:11.753 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 0, 2024-06-10T00:44:11.753 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 0, 2024-06-10T00:44:11.753 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 0, 2024-06-10T00:44:11.753 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-06-10T00:44:11.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.754 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T00:44:11.754 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:11.754 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-06-10T00:44:11.754 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:11.754 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 3245, 2024-06-10T00:44:11.754 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 2541459, 2024-06-10T00:44:11.754 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:11.755 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 3245, 2024-06-10T00:44:11.755 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-06-10T00:44:11.755 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-06-10T00:44:11.755 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 3245, 2024-06-10T00:44:11.755 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 2541459, 2024-06-10T00:44:11.755 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:11.755 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.755 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.756 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.756 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:11.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.756 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-06-10T00:44:11.756 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:11.756 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-06-10T00:44:11.756 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:11.757 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-06-10T00:44:11.757 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-06-10T00:44:11.757 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:11.757 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:11.757 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 310, 2024-06-10T00:44:11.757 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 361058, 2024-06-10T00:44:11.757 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 310, 2024-06-10T00:44:11.757 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 361058, 2024-06-10T00:44:11.758 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:11.758 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.758 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.758 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.758 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:11.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.758 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-06-10T00:44:11.759 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:11.759 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-06-10T00:44:11.759 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:11.759 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-06-10T00:44:11.759 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-06-10T00:44:11.759 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:11.759 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:11.760 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 310, 2024-06-10T00:44:11.760 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 310, 2024-06-10T00:44:11.760 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 310, 2024-06-10T00:44:11.760 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 310, 2024-06-10T00:44:11.760 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:11.760 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.760 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.760 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:11.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.761 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-06-10T00:44:11.761 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:11.761 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-06-10T00:44:11.761 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:11.761 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-06-10T00:44:11.761 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-06-10T00:44:11.761 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:11.762 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:11.762 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-06-10T00:44:11.762 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-06-10T00:44:11.762 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-06-10T00:44:11.762 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-06-10T00:44:11.762 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:11.762 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.762 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.763 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.763 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:11.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:11.763 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x55a6dc9da500": { 2024-06-10T00:44:11.763 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:11.763 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-06-10T00:44:11.763 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:11.763 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 257, 2024-06-10T00:44:11.764 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 344926, 2024-06-10T00:44:11.764 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:11.764 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 257, 2024-06-10T00:44:11.764 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-06-10T00:44:11.764 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-06-10T00:44:11.764 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 58, 2024-06-10T00:44:11.764 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 344926, 2024-06-10T00:44:11.765 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:11.765 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:11.765 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:11.765 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:11.765 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:11.765 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:11.765 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-06-10T00:44:12.478 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-06-10T00:44:12.478 INFO:tasks.check_counter:Daemon mds.f mds.imported=2 2024-06-10T00:44:12.478 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-06-10T00:44:12.478 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=274 2024-06-10T00:44:12.479 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-06-10T00:44:12.479 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-06-10T00:44:12.479 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-06-10T00:44:12.479 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-06-10T00:44:12.479 DEBUG:tasks.check_counter:Getting stats from i 2024-06-10T00:44:12.479 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-10T00:44:12.718 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:44:13.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:13 smithi138 ceph-mon[36549]: pgmap v353: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-06-10T00:44:13.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:13 smithi105 ceph-mon[36321]: pgmap v353: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-06-10T00:44:13.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:13 smithi062 ceph-mon[32390]: pgmap v353: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-06-10T00:44:14.552 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-06-10T00:44:14.553 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T00:44:14.553 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 438, 2024-06-10T00:44:14.553 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 729, 2024-06-10T00:44:14.553 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 224175, 2024-06-10T00:44:14.553 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 671685, 2024-06-10T00:44:14.553 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 11, 2024-06-10T00:44:14.553 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 6, 2024-06-10T00:44:14.554 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.051946344, 2024-06-10T00:44:14.554 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.018135171, 2024-06-10T00:44:14.554 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.016791206, 2024-06-10T00:44:14.554 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.008087269, 2024-06-10T00:44:14.554 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:14.554 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 727, 2024-06-10T00:44:14.554 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.084157423, 2024-06-10T00:44:14.555 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000115759 2024-06-10T00:44:14.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.555 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:14.555 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 353, 2024-06-10T00:44:14.555 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000111957, 2024-06-10T00:44:14.555 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000317 2024-06-10T00:44:14.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.555 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-10T00:44:14.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 333616 2024-06-10T00:44:14.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.556 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T00:44:14.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 1051, 2024-06-10T00:44:14.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 1355, 2024-06-10T00:44:14.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1940221, 2024-06-10T00:44:14.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 912787, 2024-06-10T00:44:14.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 39, 2024-06-10T00:44:14.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 6, 2024-06-10T00:44:14.557 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.128196301, 2024-06-10T00:44:14.557 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.041551960, 2024-06-10T00:44:14.557 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.125404068, 2024-06-10T00:44:14.557 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.013936749, 2024-06-10T00:44:14.557 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:14.557 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1351, 2024-06-10T00:44:14.557 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.182144612, 2024-06-10T00:44:14.557 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000134822 2024-06-10T00:44:14.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.558 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:14.558 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 886, 2024-06-10T00:44:14.558 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000271916, 2024-06-10T00:44:14.558 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000306 2024-06-10T00:44:14.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.558 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1721264, 2024-06-10T00:44:14.558 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 174640 2024-06-10T00:44:14.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.559 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T00:44:14.559 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 2402, 2024-06-10T00:44:14.559 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 2028, 2024-06-10T00:44:14.559 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1304707, 2024-06-10T00:44:14.559 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 2011465, 2024-06-10T00:44:14.559 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 11, 2024-06-10T00:44:14.559 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 4, 2024-06-10T00:44:14.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.199472212, 2024-06-10T00:44:14.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.081042170, 2024-06-10T00:44:14.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.113231735, 2024-06-10T00:44:14.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.019951520, 2024-06-10T00:44:14.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:14.560 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2029, 2024-06-10T00:44:14.560 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.177090446, 2024-06-10T00:44:14.560 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000087279 2024-06-10T00:44:14.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.561 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:14.561 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3483, 2024-06-10T00:44:14.561 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000918492, 2024-06-10T00:44:14.561 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000263 2024-06-10T00:44:14.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.561 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 4112, 2024-06-10T00:44:14.561 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 156912 2024-06-10T00:44:14.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.562 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-06-10T00:44:14.562 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-06-10T00:44:14.562 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-06-10T00:44:14.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.562 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-06-10T00:44:14.562 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-06-10T00:44:14.562 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-06-10T00:44:14.562 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 838, 2024-06-10T00:44:14.563 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.248991408, 2024-06-10T00:44:14.563 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000297125 2024-06-10T00:44:14.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:14.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.563 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-06-10T00:44:14.563 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-06-10T00:44:14.563 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-06-10T00:44:14.564 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2, 2024-06-10T00:44:14.564 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000012946, 2024-06-10T00:44:14.564 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000006473 2024-06-10T00:44:14.564 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:14.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.564 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-06-10T00:44:14.564 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 40, 2024-06-10T00:44:14.564 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 20, 2024-06-10T00:44:14.565 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-06-10T00:44:14.565 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 20, 2024-06-10T00:44:14.565 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.057858215, 2024-06-10T00:44:14.565 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.002892910 2024-06-10T00:44:14.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.565 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-06-10T00:44:14.565 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 20, 2024-06-10T00:44:14.565 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 5, 2024-06-10T00:44:14.565 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-06-10T00:44:14.566 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 20, 2024-06-10T00:44:14.566 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 1, 2024-06-10T00:44:14.566 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-06-10T00:44:14.566 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 171, 2024-06-10T00:44:14.566 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 12, 2024-06-10T00:44:14.566 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 8, 2024-06-10T00:44:14.566 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 151, 2024-06-10T00:44:14.567 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 153, 2024-06-10T00:44:14.567 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 468, 2024-06-10T00:44:14.567 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 1, 2024-06-10T00:44:14.567 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 1, 2024-06-10T00:44:14.567 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 9, 2024-06-10T00:44:14.567 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 234, 2024-06-10T00:44:14.567 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 115, 2024-06-10T00:44:14.567 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 18, 2024-06-10T00:44:14.567 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 99, 2024-06-10T00:44:14.568 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-06-10T00:44:14.568 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-06-10T00:44:14.568 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-06-10T00:44:14.568 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 1000, 2024-06-10T00:44:14.568 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 1, 2024-06-10T00:44:14.568 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 4, 2024-06-10T00:44:14.568 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 387, 2024-06-10T00:44:14.568 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 4, 2024-06-10T00:44:14.569 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 527, 2024-06-10T00:44:14.569 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-06-10T00:44:14.569 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 776, 2024-06-10T00:44:14.569 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-06-10T00:44:14.569 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 325, 2024-06-10T00:44:14.569 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 13424230509, 2024-06-10T00:44:14.569 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 6, 2024-06-10T00:44:14.569 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 776, 2024-06-10T00:44:14.570 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-06-10T00:44:14.570 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T00:44:14.570 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-06-10T00:44:14.570 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 87, 2024-06-10T00:44:14.570 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T00:44:14.570 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 87, 2024-06-10T00:44:14.570 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 689, 2024-06-10T00:44:14.570 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 2, 2024-06-10T00:44:14.571 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 16, 2024-06-10T00:44:14.571 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 32, 2024-06-10T00:44:14.571 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T00:44:14.571 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T00:44:14.571 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T00:44:14.571 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 324, 2024-06-10T00:44:14.572 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 241, 2024-06-10T00:44:14.572 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 22, 2024-06-10T00:44:14.572 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 2 2024-06-10T00:44:14.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.572 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-06-10T00:44:14.572 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 0, 2024-06-10T00:44:14.572 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 271, 2024-06-10T00:44:14.572 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 182, 2024-06-10T00:44:14.573 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 168, 2024-06-10T00:44:14.573 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 0, 2024-06-10T00:44:14.573 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-06-10T00:44:14.573 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-06-10T00:44:14.573 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-06-10T00:44:14.573 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 0, 2024-06-10T00:44:14.573 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 0, 2024-06-10T00:44:14.573 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-06-10T00:44:14.574 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-06-10T00:44:14.574 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-06-10T00:44:14.574 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-06-10T00:44:14.574 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-06-10T00:44:14.574 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-06-10T00:44:14.574 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-06-10T00:44:14.574 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 3, 2024-06-10T00:44:14.574 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 188, 2024-06-10T00:44:14.575 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T00:44:14.575 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 4, 2024-06-10T00:44:14.575 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-06-10T00:44:14.575 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 1, 2024-06-10T00:44:14.575 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-06-10T00:44:14.575 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-06-10T00:44:14.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.575 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-06-10T00:44:14.575 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-06-10T00:44:14.576 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 402, 2024-06-10T00:44:14.576 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-06-10T00:44:14.576 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-06-10T00:44:14.576 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 402, 2024-06-10T00:44:14.576 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-06-10T00:44:14.576 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-06-10T00:44:14.576 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 2, 2024-06-10T00:44:14.576 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-06-10T00:44:14.577 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-06-10T00:44:14.577 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 2, 2024-06-10T00:44:14.577 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 2, 2024-06-10T00:44:14.577 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-06-10T00:44:14.577 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-06-10T00:44:14.577 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-06-10T00:44:14.577 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 5022249, 2024-06-10T00:44:14.577 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-06-10T00:44:14.578 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-06-10T00:44:14.578 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 73, 2024-06-10T00:44:14.578 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 1.495899542, 2024-06-10T00:44:14.578 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.020491774 2024-06-10T00:44:14.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.578 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-06-10T00:44:14.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.578 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-06-10T00:44:14.579 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 173, 2024-06-10T00:44:14.579 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 468, 2024-06-10T00:44:14.579 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 295, 2024-06-10T00:44:14.579 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 32, 2024-06-10T00:44:14.579 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 128, 2024-06-10T00:44:14.579 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 96, 2024-06-10T00:44:14.579 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 171, 2024-06-10T00:44:14.579 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 639, 2024-06-10T00:44:14.580 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 468, 2024-06-10T00:44:14.580 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 1, 2024-06-10T00:44:14.580 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 324, 2024-06-10T00:44:14.580 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 323, 2024-06-10T00:44:14.580 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 57584, 2024-06-10T00:44:14.580 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 223612 2024-06-10T00:44:14.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.580 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-06-10T00:44:14.581 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 41, 2024-06-10T00:44:14.581 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 1, 2024-06-10T00:44:14.581 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 40, 2024-06-10T00:44:14.581 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 23, 2024-06-10T00:44:14.581 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 3, 2024-06-10T00:44:14.581 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-06-10T00:44:14.581 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.581 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.582 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.582 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-06-10T00:44:14.582 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1, 2024-06-10T00:44:14.582 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000502997, 2024-06-10T00:44:14.582 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000502997 2024-06-10T00:44:14.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.582 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-06-10T00:44:14.583 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.583 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.583 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.583 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-06-10T00:44:14.583 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.583 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.583 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.584 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-06-10T00:44:14.584 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2, 2024-06-10T00:44:14.584 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.004494874, 2024-06-10T00:44:14.584 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.002247437 2024-06-10T00:44:14.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.584 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-06-10T00:44:14.584 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.585 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.585 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-06-10T00:44:14.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.585 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.586 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-06-10T00:44:14.586 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.586 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.586 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.586 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-06-10T00:44:14.586 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.586 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.587 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-06-10T00:44:14.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.587 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.587 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-06-10T00:44:14.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.588 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.588 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.588 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-06-10T00:44:14.589 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.589 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.589 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.589 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-06-10T00:44:14.589 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.589 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.589 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.590 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-06-10T00:44:14.590 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.590 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.590 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.590 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-06-10T00:44:14.590 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 17, 2024-06-10T00:44:14.591 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.052860344, 2024-06-10T00:44:14.591 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.003109432 2024-06-10T00:44:14.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.591 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-06-10T00:44:14.591 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.591 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.591 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.592 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-06-10T00:44:14.592 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.592 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.592 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.592 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-06-10T00:44:14.592 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.592 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.593 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.593 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-06-10T00:44:14.593 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.593 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.593 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.594 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-06-10T00:44:14.594 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.594 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.594 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.594 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-06-10T00:44:14.594 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.594 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.595 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.595 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-06-10T00:44:14.595 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.595 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.595 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.595 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-06-10T00:44:14.596 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.596 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.596 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.596 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-06-10T00:44:14.596 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.596 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.596 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.597 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-06-10T00:44:14.597 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.597 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.597 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.597 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-06-10T00:44:14.597 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.598 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.598 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.598 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-06-10T00:44:14.598 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.598 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.598 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.599 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-06-10T00:44:14.599 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.599 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.599 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.599 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-06-10T00:44:14.599 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.599 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.600 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.600 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-06-10T00:44:14.600 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-06-10T00:44:14.600 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-06-10T00:44:14.600 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.600 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.600 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.601 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:14.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.601 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-06-10T00:44:14.601 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 1, 2024-06-10T00:44:14.601 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 4, 2024-06-10T00:44:14.601 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 3, 2024-06-10T00:44:14.601 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 1, 2024-06-10T00:44:14.601 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-06-10T00:44:14.602 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 18, 2024-06-10T00:44:14.602 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 18, 2024-06-10T00:44:14.602 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 571, 2024-06-10T00:44:14.602 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-06-10T00:44:14.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.602 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-06-10T00:44:14.602 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-06-10T00:44:14.602 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-06-10T00:44:14.603 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T00:44:14.603 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-06-10T00:44:14.603 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T00:44:14.603 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-06-10T00:44:14.603 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T00:44:14.603 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T00:44:14.604 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T00:44:14.604 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T00:44:14.604 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T00:44:14.604 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-06-10T00:44:14.604 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T00:44:14.604 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T00:44:14.604 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-06-10T00:44:14.604 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-06-10T00:44:14.605 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-06-10T00:44:14.605 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-06-10T00:44:14.605 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T00:44:14.605 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T00:44:14.605 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T00:44:14.605 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T00:44:14.605 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T00:44:14.605 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-06-10T00:44:14.606 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-06-10T00:44:14.606 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-06-10T00:44:14.606 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T00:44:14.606 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T00:44:14.606 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-06-10T00:44:14.606 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-06-10T00:44:14.606 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-06-10T00:44:14.606 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-06-10T00:44:14.607 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T00:44:14.607 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-06-10T00:44:14.607 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T00:44:14.607 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-06-10T00:44:14.607 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 316987, 2024-06-10T00:44:14.607 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 240, 2024-06-10T00:44:14.607 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-06-10T00:44:14.607 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-06-10T00:44:14.608 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-06-10T00:44:14.608 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-06-10T00:44:14.608 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-06-10T00:44:14.608 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-06-10T00:44:14.608 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-06-10T00:44:14.608 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-06-10T00:44:14.608 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 9944, 2024-06-10T00:44:14.608 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 192, 2024-06-10T00:44:14.608 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T00:44:14.609 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-06-10T00:44:14.609 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-06-10T00:44:14.609 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-06-10T00:44:14.609 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 575734, 2024-06-10T00:44:14.609 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 5706, 2024-06-10T00:44:14.609 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-06-10T00:44:14.609 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-06-10T00:44:14.609 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-06-10T00:44:14.610 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-06-10T00:44:14.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.610 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-06-10T00:44:14.610 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-06-10T00:44:14.610 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-06-10T00:44:14.610 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 948, 2024-06-10T00:44:14.610 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 1009094, 2024-06-10T00:44:14.610 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 28, 2024-06-10T00:44:14.611 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 948, 2024-06-10T00:44:14.611 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-06-10T00:44:14.611 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 948, 2024-06-10T00:44:14.611 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 140.001208694, 2024-06-10T00:44:14.611 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.147680599 2024-06-10T00:44:14.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.611 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-06-10T00:44:14.611 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-06-10T00:44:14.611 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 948, 2024-06-10T00:44:14.612 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 1025 2024-06-10T00:44:14.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.612 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 948, 2024-06-10T00:44:14.612 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 781, 2024-06-10T00:44:14.612 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 167, 2024-06-10T00:44:14.612 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-06-10T00:44:14.612 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-06-10T00:44:14.612 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 0, 2024-06-10T00:44:14.613 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 10, 2024-06-10T00:44:14.613 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 0, 2024-06-10T00:44:14.613 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 73, 2024-06-10T00:44:14.613 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 18, 2024-06-10T00:44:14.613 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-06-10T00:44:14.613 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-06-10T00:44:14.613 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-06-10T00:44:14.613 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 1, 2024-06-10T00:44:14.614 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 6, 2024-06-10T00:44:14.614 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-06-10T00:44:14.614 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-06-10T00:44:14.614 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-06-10T00:44:14.614 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 776, 2024-06-10T00:44:14.614 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 20, 2024-06-10T00:44:14.614 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-06-10T00:44:14.614 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-06-10T00:44:14.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-06-10T00:44:14.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-06-10T00:44:14.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-06-10T00:44:14.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-06-10T00:44:14.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T00:44:14.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-06-10T00:44:14.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-06-10T00:44:14.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-06-10T00:44:14.615 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-06-10T00:44:14.616 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-06-10T00:44:14.616 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-06-10T00:44:14.616 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-06-10T00:44:14.616 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-06-10T00:44:14.616 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-06-10T00:44:14.616 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-06-10T00:44:14.616 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-06-10T00:44:14.616 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-06-10T00:44:14.617 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-06-10T00:44:14.617 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-06-10T00:44:14.617 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-06-10T00:44:14.617 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-06-10T00:44:14.617 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-06-10T00:44:14.617 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-06-10T00:44:14.617 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-06-10T00:44:14.617 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 110, 2024-06-10T00:44:14.618 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-06-10T00:44:14.618 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 13, 2024-06-10T00:44:14.618 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 12, 2024-06-10T00:44:14.618 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 12, 2024-06-10T00:44:14.618 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-06-10T00:44:14.618 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-06-10T00:44:14.618 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 78, 2024-06-10T00:44:14.618 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 10, 2024-06-10T00:44:14.619 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 33 2024-06-10T00:44:14.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.619 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-06-10T00:44:14.619 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-06-10T00:44:14.619 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-06-10T00:44:14.619 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-06-10T00:44:14.619 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-06-10T00:44:14.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.620 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-06-10T00:44:14.620 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 88 2024-06-10T00:44:14.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.620 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-06-10T00:44:14.620 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-06-10T00:44:14.620 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T00:44:14.620 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-06-10T00:44:14.620 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 0, 2024-06-10T00:44:14.621 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 0, 2024-06-10T00:44:14.621 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 0, 2024-06-10T00:44:14.621 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-06-10T00:44:14.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.621 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T00:44:14.621 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:14.621 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-06-10T00:44:14.621 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:14.621 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 3887, 2024-06-10T00:44:14.622 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 3098347, 2024-06-10T00:44:14.622 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:14.622 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 3887, 2024-06-10T00:44:14.622 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-06-10T00:44:14.622 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-06-10T00:44:14.622 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 3887, 2024-06-10T00:44:14.622 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 3098347, 2024-06-10T00:44:14.622 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:14.623 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.623 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.623 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.623 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:14.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.623 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-06-10T00:44:14.623 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:14.623 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-06-10T00:44:14.624 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:14.624 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-06-10T00:44:14.624 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-06-10T00:44:14.624 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:14.624 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:14.624 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 948, 2024-06-10T00:44:14.624 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 1009014, 2024-06-10T00:44:14.624 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 948, 2024-06-10T00:44:14.625 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1009014, 2024-06-10T00:44:14.625 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:14.625 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.625 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.625 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.625 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:14.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.625 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-06-10T00:44:14.625 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:14.626 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-06-10T00:44:14.626 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:14.626 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-06-10T00:44:14.626 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-06-10T00:44:14.626 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:14.626 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:14.626 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 948, 2024-06-10T00:44:14.626 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 948, 2024-06-10T00:44:14.627 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 948, 2024-06-10T00:44:14.627 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 948, 2024-06-10T00:44:14.627 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:14.627 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.627 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.627 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:14.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.628 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-06-10T00:44:14.628 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:14.628 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-06-10T00:44:14.628 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:14.628 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-06-10T00:44:14.629 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-06-10T00:44:14.629 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:14.629 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:14.629 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-06-10T00:44:14.629 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-06-10T00:44:14.629 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-06-10T00:44:14.629 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-06-10T00:44:14.630 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:14.630 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.630 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.630 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.630 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:14.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:14.630 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x55c720a20500": { 2024-06-10T00:44:14.630 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:14.631 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-06-10T00:44:14.631 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:14.631 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 402, 2024-06-10T00:44:14.631 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 827945, 2024-06-10T00:44:14.631 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:14.631 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 402, 2024-06-10T00:44:14.631 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-06-10T00:44:14.631 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-06-10T00:44:14.632 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 73, 2024-06-10T00:44:14.632 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 827945, 2024-06-10T00:44:14.632 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:14.632 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:14.632 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:14.632 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:14.632 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:14.633 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:14.633 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-06-10T00:44:14.633 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:14 smithi062 ceph-mon[32390]: osdmap e110: 12 total, 12 up, 12 in 2024-06-10T00:44:14.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:14 smithi138 ceph-mon[36549]: osdmap e110: 12 total, 12 up, 12 in 2024-06-10T00:44:15.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:14 smithi105 ceph-mon[36321]: osdmap e110: 12 total, 12 up, 12 in 2024-06-10T00:44:15.454 INFO:tasks.check_counter:Daemon mds.i mds.exported=4 2024-06-10T00:44:15.454 INFO:tasks.check_counter:Daemon mds.i mds.imported=4 2024-06-10T00:44:15.454 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-06-10T00:44:15.454 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=271 2024-06-10T00:44:15.454 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-06-10T00:44:15.455 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-06-10T00:44:15.455 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-06-10T00:44:15.455 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=1 2024-06-10T00:44:15.455 DEBUG:tasks.check_counter:Getting stats from l 2024-06-10T00:44:15.455 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-10T00:44:15.693 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:44:15.815 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:15 smithi138 ceph-mon[36549]: pgmap v355: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 8 op/s 2024-06-10T00:44:15.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:15 smithi062 ceph-mon[32390]: pgmap v355: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 8 op/s 2024-06-10T00:44:16.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:15 smithi105 ceph-mon[36321]: pgmap v355: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 8 op/s 2024-06-10T00:44:17.537 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-06-10T00:44:17.538 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T00:44:17.538 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 2554, 2024-06-10T00:44:17.538 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 2556, 2024-06-10T00:44:17.538 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1548966, 2024-06-10T00:44:17.538 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 812179, 2024-06-10T00:44:17.539 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 6, 2024-06-10T00:44:17.539 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 2, 2024-06-10T00:44:17.539 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.221866489, 2024-06-10T00:44:17.539 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.072017326, 2024-06-10T00:44:17.539 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.054541481, 2024-06-10T00:44:17.539 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.078701383, 2024-06-10T00:44:17.539 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:17.539 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2554, 2024-06-10T00:44:17.540 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.101684300, 2024-06-10T00:44:17.540 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000039813 2024-06-10T00:44:17.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.540 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:17.540 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.540 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.540 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.541 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T00:44:17.541 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T00:44:17.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.541 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T00:44:17.541 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 306, 2024-06-10T00:44:17.541 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 285, 2024-06-10T00:44:17.541 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1691580, 2024-06-10T00:44:17.541 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 167082, 2024-06-10T00:44:17.542 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 3, 2024-06-10T00:44:17.542 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 2, 2024-06-10T00:44:17.542 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.056121551, 2024-06-10T00:44:17.542 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.017118940, 2024-06-10T00:44:17.542 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.041782247, 2024-06-10T00:44:17.542 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.007031449, 2024-06-10T00:44:17.542 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:17.542 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 284, 2024-06-10T00:44:17.543 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.018152234, 2024-06-10T00:44:17.543 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000063916 2024-06-10T00:44:17.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.543 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:17.543 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.543 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.543 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.544 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1691024, 2024-06-10T00:44:17.544 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 166384 2024-06-10T00:44:17.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.544 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T00:44:17.544 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 7, 2024-06-10T00:44:17.544 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 206, 2024-06-10T00:44:17.544 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 3612, 2024-06-10T00:44:17.544 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 490446, 2024-06-10T00:44:17.545 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 6, 2024-06-10T00:44:17.545 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 4, 2024-06-10T00:44:17.545 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.012630867, 2024-06-10T00:44:17.545 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.007857403, 2024-06-10T00:44:17.545 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.002354102, 2024-06-10T00:44:17.545 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000032621, 2024-06-10T00:44:17.545 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T00:44:17.545 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 204, 2024-06-10T00:44:17.546 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.014473934, 2024-06-10T00:44:17.546 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000070950 2024-06-10T00:44:17.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.546 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-06-10T00:44:17.546 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.546 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.546 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.547 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-06-10T00:44:17.547 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 490128 2024-06-10T00:44:17.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.547 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-06-10T00:44:17.547 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-06-10T00:44:17.547 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-06-10T00:44:17.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.547 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-06-10T00:44:17.548 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-06-10T00:44:17.548 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-06-10T00:44:17.548 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 4324, 2024-06-10T00:44:17.548 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.119555449, 2024-06-10T00:44:17.548 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000027649 2024-06-10T00:44:17.548 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:17.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.548 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-06-10T00:44:17.549 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-06-10T00:44:17.549 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-06-10T00:44:17.549 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.549 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.549 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.549 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:17.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-06-10T00:44:17.550 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 0, 2024-06-10T00:44:17.550 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 0, 2024-06-10T00:44:17.550 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-06-10T00:44:17.550 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.550 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.550 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.551 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-06-10T00:44:17.551 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 0, 2024-06-10T00:44:17.551 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-06-10T00:44:17.551 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-06-10T00:44:17.551 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 0, 2024-06-10T00:44:17.551 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-06-10T00:44:17.551 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-06-10T00:44:17.551 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 203, 2024-06-10T00:44:17.552 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 13, 2024-06-10T00:44:17.552 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 190, 2024-06-10T00:44:17.552 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-06-10T00:44:17.552 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 185, 2024-06-10T00:44:17.552 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 0, 2024-06-10T00:44:17.552 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 0, 2024-06-10T00:44:17.552 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 0, 2024-06-10T00:44:17.552 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 9, 2024-06-10T00:44:17.553 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 0, 2024-06-10T00:44:17.553 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 0, 2024-06-10T00:44:17.553 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 0, 2024-06-10T00:44:17.553 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 0, 2024-06-10T00:44:17.553 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-06-10T00:44:17.553 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-06-10T00:44:17.553 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-06-10T00:44:17.554 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-06-10T00:44:17.554 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-06-10T00:44:17.554 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-06-10T00:44:17.554 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-06-10T00:44:17.554 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 0, 2024-06-10T00:44:17.554 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 0, 2024-06-10T00:44:17.554 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-06-10T00:44:17.555 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 0, 2024-06-10T00:44:17.555 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-06-10T00:44:17.555 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 0, 2024-06-10T00:44:17.555 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 0, 2024-06-10T00:44:17.555 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 0, 2024-06-10T00:44:17.555 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T00:44:17.555 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-06-10T00:44:17.555 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T00:44:17.556 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-06-10T00:44:17.556 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-06-10T00:44:17.556 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T00:44:17.556 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T00:44:17.556 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 0, 2024-06-10T00:44:17.556 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-06-10T00:44:17.556 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T00:44:17.556 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 0, 2024-06-10T00:44:17.557 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T00:44:17.557 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T00:44:17.557 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T00:44:17.557 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 0, 2024-06-10T00:44:17.557 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 0, 2024-06-10T00:44:17.557 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 0, 2024-06-10T00:44:17.557 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 0 2024-06-10T00:44:17.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.558 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-06-10T00:44:17.558 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 0, 2024-06-10T00:44:17.558 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 0, 2024-06-10T00:44:17.558 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 0, 2024-06-10T00:44:17.558 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 0, 2024-06-10T00:44:17.558 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 0, 2024-06-10T00:44:17.558 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-06-10T00:44:17.559 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-06-10T00:44:17.559 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-06-10T00:44:17.559 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 0, 2024-06-10T00:44:17.559 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 0, 2024-06-10T00:44:17.559 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-06-10T00:44:17.559 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-06-10T00:44:17.559 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-06-10T00:44:17.560 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-06-10T00:44:17.560 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-06-10T00:44:17.560 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-06-10T00:44:17.560 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-06-10T00:44:17.560 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 0, 2024-06-10T00:44:17.560 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 0, 2024-06-10T00:44:17.560 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T00:44:17.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-06-10T00:44:17.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-06-10T00:44:17.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-06-10T00:44:17.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-06-10T00:44:17.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-06-10T00:44:17.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.561 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-06-10T00:44:17.562 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-06-10T00:44:17.562 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 0, 2024-06-10T00:44:17.562 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-06-10T00:44:17.562 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-06-10T00:44:17.562 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 402, 2024-06-10T00:44:17.562 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-06-10T00:44:17.562 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-06-10T00:44:17.563 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 0, 2024-06-10T00:44:17.563 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-06-10T00:44:17.563 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-06-10T00:44:17.563 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 2, 2024-06-10T00:44:17.563 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 2, 2024-06-10T00:44:17.563 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-06-10T00:44:17.563 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-06-10T00:44:17.564 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-06-10T00:44:17.564 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 5022249, 2024-06-10T00:44:17.564 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 5021675, 2024-06-10T00:44:17.564 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-06-10T00:44:17.564 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.564 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.564 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.565 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 402 2024-06-10T00:44:17.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.565 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-06-10T00:44:17.565 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 162, 2024-06-10T00:44:17.565 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 426, 2024-06-10T00:44:17.565 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 264, 2024-06-10T00:44:17.565 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 18, 2024-06-10T00:44:17.565 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 98, 2024-06-10T00:44:17.566 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 80, 2024-06-10T00:44:17.566 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 203, 2024-06-10T00:44:17.566 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 590, 2024-06-10T00:44:17.566 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 387, 2024-06-10T00:44:17.566 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 0, 2024-06-10T00:44:17.566 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 0, 2024-06-10T00:44:17.566 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 0, 2024-06-10T00:44:17.566 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 54108, 2024-06-10T00:44:17.567 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 215420 2024-06-10T00:44:17.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.567 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-06-10T00:44:17.567 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 0, 2024-06-10T00:44:17.567 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 0, 2024-06-10T00:44:17.567 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 0, 2024-06-10T00:44:17.567 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 0, 2024-06-10T00:44:17.568 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 0, 2024-06-10T00:44:17.568 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-06-10T00:44:17.568 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.568 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.568 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.568 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-06-10T00:44:17.568 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.569 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.569 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.569 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-06-10T00:44:17.569 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.569 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.569 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.570 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-06-10T00:44:17.570 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.570 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.570 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.570 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-06-10T00:44:17.570 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.571 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.571 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.571 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-06-10T00:44:17.571 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.571 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.571 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.572 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-06-10T00:44:17.572 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.572 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.572 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.573 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-06-10T00:44:17.573 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.573 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.573 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.573 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-06-10T00:44:17.573 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.573 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.574 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.574 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-06-10T00:44:17.574 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.574 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.574 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.575 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-06-10T00:44:17.575 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.575 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.575 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.575 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-06-10T00:44:17.576 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.576 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.576 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.576 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-06-10T00:44:17.577 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.577 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.577 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.577 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-06-10T00:44:17.577 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.577 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.577 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.578 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-06-10T00:44:17.578 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.578 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.578 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.578 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-06-10T00:44:17.579 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.579 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.579 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.579 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-06-10T00:44:17.579 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.581 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.581 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.581 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-06-10T00:44:17.581 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.582 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.582 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.582 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-06-10T00:44:17.582 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.582 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.582 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.583 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-06-10T00:44:17.583 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.583 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.583 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.583 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-06-10T00:44:17.583 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.584 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.584 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.584 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-06-10T00:44:17.584 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.584 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.584 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.584 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-06-10T00:44:17.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.585 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.585 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-06-10T00:44:17.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.585 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.586 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-06-10T00:44:17.586 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.586 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.586 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.586 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-06-10T00:44:17.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.587 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.587 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-06-10T00:44:17.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.587 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.588 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-06-10T00:44:17.588 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.588 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.588 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.588 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-06-10T00:44:17.588 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.589 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.589 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.589 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-06-10T00:44:17.589 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-06-10T00:44:17.589 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-06-10T00:44:17.589 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.589 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.590 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.590 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:17.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.590 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-06-10T00:44:17.590 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 1, 2024-06-10T00:44:17.590 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 4, 2024-06-10T00:44:17.590 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 3, 2024-06-10T00:44:17.590 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 1, 2024-06-10T00:44:17.590 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-06-10T00:44:17.591 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 0, 2024-06-10T00:44:17.591 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 0, 2024-06-10T00:44:17.591 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 2190, 2024-06-10T00:44:17.591 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-06-10T00:44:17.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.591 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-06-10T00:44:17.591 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-06-10T00:44:17.591 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-06-10T00:44:17.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T00:44:17.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-06-10T00:44:17.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T00:44:17.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-06-10T00:44:17.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T00:44:17.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T00:44:17.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T00:44:17.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T00:44:17.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T00:44:17.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-06-10T00:44:17.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T00:44:17.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T00:44:17.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-06-10T00:44:17.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-06-10T00:44:17.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-06-10T00:44:17.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-06-10T00:44:17.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T00:44:17.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T00:44:17.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T00:44:17.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T00:44:17.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T00:44:17.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-06-10T00:44:17.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-06-10T00:44:17.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-06-10T00:44:17.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T00:44:17.595 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T00:44:17.595 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-06-10T00:44:17.595 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-06-10T00:44:17.595 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-06-10T00:44:17.595 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-06-10T00:44:17.595 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T00:44:17.595 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-06-10T00:44:17.595 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T00:44:17.596 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-06-10T00:44:17.596 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 187935, 2024-06-10T00:44:17.596 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 206, 2024-06-10T00:44:17.596 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-06-10T00:44:17.596 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-06-10T00:44:17.596 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-06-10T00:44:17.596 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-06-10T00:44:17.596 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-06-10T00:44:17.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-06-10T00:44:17.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-06-10T00:44:17.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-06-10T00:44:17.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 9704, 2024-06-10T00:44:17.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 186, 2024-06-10T00:44:17.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T00:44:17.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-06-10T00:44:17.597 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-06-10T00:44:17.597 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-06-10T00:44:17.598 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 543614, 2024-06-10T00:44:17.598 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 4742, 2024-06-10T00:44:17.598 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-06-10T00:44:17.598 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-06-10T00:44:17.598 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-06-10T00:44:17.598 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-06-10T00:44:17.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.598 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-06-10T00:44:17.599 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-06-10T00:44:17.599 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-06-10T00:44:17.599 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 2557, 2024-06-10T00:44:17.599 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 21, 2024-06-10T00:44:17.599 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 3, 2024-06-10T00:44:17.599 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 2557, 2024-06-10T00:44:17.599 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-06-10T00:44:17.600 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2557, 2024-06-10T00:44:17.600 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 3.876922843, 2024-06-10T00:44:17.600 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001516199 2024-06-10T00:44:17.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.600 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-06-10T00:44:17.600 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-06-10T00:44:17.601 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2557, 2024-06-10T00:44:17.601 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2558 2024-06-10T00:44:17.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.601 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 2557, 2024-06-10T00:44:17.601 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 2557, 2024-06-10T00:44:17.601 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 0, 2024-06-10T00:44:17.602 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-06-10T00:44:17.602 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-06-10T00:44:17.602 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 1642, 2024-06-10T00:44:17.602 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 0, 2024-06-10T00:44:17.602 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 913, 2024-06-10T00:44:17.602 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 0, 2024-06-10T00:44:17.603 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 0, 2024-06-10T00:44:17.603 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-06-10T00:44:17.603 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-06-10T00:44:17.603 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-06-10T00:44:17.603 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-06-10T00:44:17.603 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 0, 2024-06-10T00:44:17.604 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-06-10T00:44:17.604 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-06-10T00:44:17.604 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-06-10T00:44:17.604 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 1, 2024-06-10T00:44:17.604 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 0, 2024-06-10T00:44:17.604 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-06-10T00:44:17.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-06-10T00:44:17.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-06-10T00:44:17.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-06-10T00:44:17.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-06-10T00:44:17.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-06-10T00:44:17.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T00:44:17.605 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-06-10T00:44:17.606 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-06-10T00:44:17.606 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-06-10T00:44:17.606 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-06-10T00:44:17.606 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-06-10T00:44:17.606 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-06-10T00:44:17.606 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-06-10T00:44:17.607 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-06-10T00:44:17.607 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-06-10T00:44:17.607 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-06-10T00:44:17.607 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-06-10T00:44:17.607 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-06-10T00:44:17.607 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-06-10T00:44:17.608 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-06-10T00:44:17.608 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-06-10T00:44:17.608 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-06-10T00:44:17.608 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-06-10T00:44:17.608 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-06-10T00:44:17.608 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-06-10T00:44:17.608 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 110, 2024-06-10T00:44:17.608 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-06-10T00:44:17.609 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 13, 2024-06-10T00:44:17.609 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 4, 2024-06-10T00:44:17.609 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 4, 2024-06-10T00:44:17.609 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-06-10T00:44:17.609 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-06-10T00:44:17.609 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 0, 2024-06-10T00:44:17.609 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 2, 2024-06-10T00:44:17.609 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 0 2024-06-10T00:44:17.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.610 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-06-10T00:44:17.610 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 0, 2024-06-10T00:44:17.610 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-06-10T00:44:17.610 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-06-10T00:44:17.610 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-06-10T00:44:17.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.610 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-06-10T00:44:17.611 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 0 2024-06-10T00:44:17.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.611 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-06-10T00:44:17.611 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-06-10T00:44:17.611 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T00:44:17.611 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-06-10T00:44:17.611 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 0, 2024-06-10T00:44:17.612 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 0, 2024-06-10T00:44:17.612 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 0, 2024-06-10T00:44:17.612 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-06-10T00:44:17.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.612 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T00:44:17.612 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:17.612 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-06-10T00:44:17.612 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:17.613 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 2863, 2024-06-10T00:44:17.613 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 2967916, 2024-06-10T00:44:17.613 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:17.613 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 2863, 2024-06-10T00:44:17.613 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-06-10T00:44:17.613 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-06-10T00:44:17.613 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2863, 2024-06-10T00:44:17.614 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 2967916, 2024-06-10T00:44:17.614 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:17.614 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.614 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.614 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.614 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:17.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.614 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-06-10T00:44:17.615 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:17.615 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-06-10T00:44:17.615 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:17.615 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-06-10T00:44:17.615 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-06-10T00:44:17.615 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:17.615 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:17.615 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 2557, 2024-06-10T00:44:17.616 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 827950, 2024-06-10T00:44:17.616 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2557, 2024-06-10T00:44:17.616 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 827950, 2024-06-10T00:44:17.616 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:17.616 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.616 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.616 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.617 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:17.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.617 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-06-10T00:44:17.617 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:17.617 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-06-10T00:44:17.617 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:17.617 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-06-10T00:44:17.617 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-06-10T00:44:17.618 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:17.618 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:17.618 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 2557, 2024-06-10T00:44:17.618 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 2557, 2024-06-10T00:44:17.618 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2557, 2024-06-10T00:44:17.618 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 2557, 2024-06-10T00:44:17.618 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:17.618 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.619 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.619 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.619 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:17.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.619 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-06-10T00:44:17.619 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:17.619 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-06-10T00:44:17.619 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:17.620 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-06-10T00:44:17.620 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-06-10T00:44:17.620 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:17.620 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:17.620 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-06-10T00:44:17.620 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-06-10T00:44:17.620 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-06-10T00:44:17.621 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-06-10T00:44:17.621 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:17.621 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.621 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.621 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.621 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:17.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-06-10T00:44:17.621 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x55e2df3086e0": { 2024-06-10T00:44:17.622 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-06-10T00:44:17.622 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-06-10T00:44:17.622 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-06-10T00:44:17.622 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-06-10T00:44:17.622 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-06-10T00:44:17.622 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-06-10T00:44:17.622 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-06-10T00:44:17.622 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-06-10T00:44:17.623 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-06-10T00:44:17.623 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-06-10T00:44:17.623 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-06-10T00:44:17.623 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-06-10T00:44:17.623 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-06-10T00:44:17.623 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-06-10T00:44:17.623 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-06-10T00:44:17.624 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:17.624 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-06-10T00:44:17.624 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-06-10T00:44:17.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:17 smithi062 ceph-mon[32390]: pgmap v356: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 9 op/s 2024-06-10T00:44:17.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:17 smithi138 ceph-mon[36549]: pgmap v356: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 9 op/s 2024-06-10T00:44:18.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:17 smithi105 ceph-mon[36321]: pgmap v356: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 9 op/s 2024-06-10T00:44:18.425 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-06-10T00:44:18.425 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-06-10T00:44:18.425 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-06-10T00:44:18.425 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-06-10T00:44:18.425 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-06-10T00:44:18.425 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-06-10T00:44:18.426 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-06-10T00:44:18.426 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-06-10T00:44:18.426 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-06-10T00:44:18.438 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-10T00:44:18.438 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:44:18.438 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T00:44:18.468 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-06-10T00:44:18.468 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T00:44:18.533 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-10T00:44:18.832 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:44:19.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:19 smithi062 ceph-mon[32390]: pgmap v357: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:44:19.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:19 smithi138 ceph-mon[36549]: pgmap v357: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:44:20.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:19 smithi105 ceph-mon[36321]: pgmap v357: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:44:20.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.585+0000 7f6e34685700 1 -- 172.21.15.62:0/2308513199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e2c06abc0 msgr2=0x7f6e2c06b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:20.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.585+0000 7f6e34685700 1 --2- 172.21.15.62:0/2308513199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e2c06abc0 0x7f6e2c06b040 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009990 tx=0x7f6e1c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:44:20.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.585+0000 7f6e34685700 1 -- 172.21.15.62:0/2308513199 shutdown_connections 2024-06-10T00:44:20.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.585+0000 7f6e34685700 1 --2- 172.21.15.62:0/2308513199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e2c06b580 0x7f6e2c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:20.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.585+0000 7f6e34685700 1 --2- 172.21.15.62:0/2308513199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e2c108500 0x7f6e2c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:20.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.585+0000 7f6e34685700 1 --2- 172.21.15.62:0/2308513199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e2c06abc0 0x7f6e2c06b040 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:20.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.585+0000 7f6e34685700 1 -- 172.21.15.62:0/2308513199 >> 172.21.15.62:0/2308513199 conn(0x7f6e2c078340 msgr2=0x7f6e2c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:20.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.586+0000 7f6e34685700 1 -- 172.21.15.62:0/2308513199 shutdown_connections 2024-06-10T00:44:20.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.586+0000 7f6e34685700 1 -- 172.21.15.62:0/2308513199 wait complete. 2024-06-10T00:44:20.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.586+0000 7f6e34685700 1 Processor -- start 2024-06-10T00:44:20.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.586+0000 7f6e34685700 1 -- start start 2024-06-10T00:44:20.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e34685700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e2c06abc0 0x7f6e2c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:20.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e34685700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e2c06b580 0x7f6e2c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:20.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e34685700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e2c108500 0x7f6e2c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:20.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e34685700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c112760 con 0x7f6e2c06b580 2024-06-10T00:44:20.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e34685700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c1125e0 con 0x7f6e2c108500 2024-06-10T00:44:20.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e34685700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c1128e0 con 0x7f6e2c06abc0 2024-06-10T00:44:20.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e31c20700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e2c06b580 0x7f6e2c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:20.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e31c20700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e2c06b580 0x7f6e2c19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44678/0 (socket says 172.21.15.62:44678) 2024-06-10T00:44:20.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e31c20700 1 -- 172.21.15.62:0/547749835 learned_addr learned my addr 172.21.15.62:0/547749835 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:44:20.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e32c22700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e2c108500 0x7f6e2c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:20.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e32421700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e2c06abc0 0x7f6e2c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:20.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e31c20700 1 -- 172.21.15.62:0/547749835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e2c06abc0 msgr2=0x7f6e2c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:20.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e31c20700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e2c06abc0 0x7f6e2c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:20.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e31c20700 1 -- 172.21.15.62:0/547749835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e2c108500 msgr2=0x7f6e2c198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:20.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e31c20700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e2c108500 0x7f6e2c198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:20.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e31c20700 1 -- 172.21.15.62:0/547749835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c199220 con 0x7f6e2c06b580 2024-06-10T00:44:20.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e32421700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e2c06abc0 0x7f6e2c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:44:20.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e31c20700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e2c06b580 0x7f6e2c19edf0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000c00 tx=0x7f6e1c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:20.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e237fe700 1 -- 172.21.15.62:0/547749835 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c027070 con 0x7f6e2c06b580 2024-06-10T00:44:20.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e34685700 1 -- 172.21.15.62:0/547749835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2c1994b0 con 0x7f6e2c06b580 2024-06-10T00:44:20.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e237fe700 1 -- 172.21.15.62:0/547749835 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e1c003d70 con 0x7f6e2c06b580 2024-06-10T00:44:20.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e237fe700 1 -- 172.21.15.62:0/547749835 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00f6c0 con 0x7f6e2c06b580 2024-06-10T00:44:20.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.587+0000 7f6e34685700 1 -- 172.21.15.62:0/547749835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2c1a5780 con 0x7f6e2c06b580 2024-06-10T00:44:20.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.589+0000 7f6e34685700 1 -- 172.21.15.62:0/547749835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2c069180 con 0x7f6e2c06b580 2024-06-10T00:44:20.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.591+0000 7f6e237fe700 1 -- 172.21.15.62:0/547749835 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6e1c02a070 con 0x7f6e2c06b580 2024-06-10T00:44:20.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.591+0000 7f6e237fe700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6e180a2d90 0x7f6e180a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:20.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.591+0000 7f6e32421700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6e180a2d90 0x7f6e180a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:20.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.591+0000 7f6e237fe700 1 -- 172.21.15.62:0/547749835 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7f6e1c0d0d50 con 0x7f6e2c06b580 2024-06-10T00:44:20.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.592+0000 7f6e32421700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6e180a2d90 0x7f6e180a5250 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0ff9f0 tx=0x7f6e28005d80 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:20.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.595+0000 7f6e237fe700 1 -- 172.21.15.62:0/547749835 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6e1c031070 con 0x7f6e2c06b580 2024-06-10T00:44:20.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.820+0000 7f6e34685700 1 -- 172.21.15.62:0/547749835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6e2c199fa0 con 0x7f6e2c06b580 2024-06-10T00:44:20.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.820+0000 7f6e237fe700 1 -- 172.21.15.62:0/547749835 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v110) ==== 81+0+942 (secure 0 0 0) 0x7f6e1c013070 con 0x7f6e2c06b580 2024-06-10T00:44:20.821 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/3602410963 2024-06-11T00:32:49.913741+0000 2024-06-10T00:44:20.821 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1557827118 2024-06-11T00:32:49.913741+0000 2024-06-10T00:44:20.821 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/2350700567 2024-06-11T00:32:49.913741+0000 2024-06-10T00:44:20.821 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3856763121 2024-06-11T00:21:21.851532+0000 2024-06-10T00:44:20.821 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/381111141 2024-06-11T00:21:21.851532+0000 2024-06-10T00:44:20.821 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1704528283 2024-06-11T00:21:21.851532+0000 2024-06-10T00:44:20.822 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3912577307 2024-06-11T00:20:55.088039+0000 2024-06-10T00:44:20.822 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/3602410963 2024-06-11T00:32:49.913741+0000 2024-06-10T00:44:20.822 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1086776708 2024-06-11T00:32:49.913741+0000 2024-06-10T00:44:20.822 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/178507377 2024-06-11T00:20:55.088039+0000 2024-06-10T00:44:20.822 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/4137526131 2024-06-11T00:20:55.088039+0000 2024-06-10T00:44:20.822 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/2528347688 2024-06-11T00:32:49.913741+0000 2024-06-10T00:44:20.823 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/2327526412 2024-06-11T00:21:21.851532+0000 2024-06-10T00:44:20.823 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/594215923 2024-06-11T00:20:55.088039+0000 2024-06-10T00:44:20.823 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/594215923 2024-06-11T00:20:55.088039+0000 2024-06-10T00:44:20.823 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/2327526412 2024-06-11T00:21:21.851532+0000 2024-06-10T00:44:20.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.823+0000 7f6e34685700 1 -- 172.21.15.62:0/547749835 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6e180a2d90 msgr2=0x7f6e180a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:20.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.823+0000 7f6e34685700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6e180a2d90 0x7f6e180a5250 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0ff9f0 tx=0x7f6e28005d80 comp rx=0 tx=0).stop 2024-06-10T00:44:20.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.823+0000 7f6e34685700 1 -- 172.21.15.62:0/547749835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e2c06b580 msgr2=0x7f6e2c19edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:20.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.823+0000 7f6e34685700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e2c06b580 0x7f6e2c19edf0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000c00 tx=0x7f6e1c00bab0 comp rx=0 tx=0).stop 2024-06-10T00:44:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.823+0000 7f6e34685700 1 -- 172.21.15.62:0/547749835 shutdown_connections 2024-06-10T00:44:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.823+0000 7f6e34685700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f6e180a2d90 0x7f6e180a5250 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.823+0000 7f6e34685700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e2c108500 0x7f6e2c198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.823+0000 7f6e34685700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e2c06abc0 0x7f6e2c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.823+0000 7f6e34685700 1 --2- 172.21.15.62:0/547749835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e2c06b580 0x7f6e2c19edf0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.823+0000 7f6e34685700 1 -- 172.21.15.62:0/547749835 >> 172.21.15.62:0/547749835 conn(0x7f6e2c078340 msgr2=0x7f6e2c100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.823+0000 7f6e34685700 1 -- 172.21.15.62:0/547749835 shutdown_connections 2024-06-10T00:44:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:20.823+0000 7f6e34685700 1 -- 172.21.15.62:0/547749835 wait complete. 2024-06-10T00:44:20.826 INFO:teuthology.orchestra.run.smithi062.stderr:listed 16 entries 2024-06-10T00:44:21.511 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-06-10T00:44:21.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:44:21.512 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-06-10T00:44:21.568 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-06-10T00:44:21.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:44:21.568 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-10T00:44:21.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:44:21.600 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-06-10T00:44:21.657 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-10T00:44:21.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:44:21.657 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-10T00:44:21.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:44:21.724 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-06-10T00:44:21.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:21 smithi062 ceph-mon[32390]: pgmap v358: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:44:21.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:21 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/547749835' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:44:21.804 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-10T00:44:21.816 INFO:tasks.cephadm:Teardown begin 2024-06-10T00:44:21.816 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-10T00:44:21.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:44:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:21 smithi138 ceph-mon[36549]: pgmap v358: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:44:21.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:21 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/547749835' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:44:22.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:21 smithi105 ceph-mon[36321]: pgmap v358: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-06-10T00:44:22.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:21 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/547749835' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:44:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:44:23.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.762+0000 7fbefa9c0700 1 -- 172.21.15.62:0/2182097066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef4108510 msgr2=0x7fbef41088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:23.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.762+0000 7fbefa9c0700 1 --2- 172.21.15.62:0/2182097066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef4108510 0x7fbef41088f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009990 tx=0x7fbee4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:44:23.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.763+0000 7fbefa9c0700 1 -- 172.21.15.62:0/2182097066 shutdown_connections 2024-06-10T00:44:23.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.763+0000 7fbefa9c0700 1 --2- 172.21.15.62:0/2182097066 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef4108510 0x7fbef41088f0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:23.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.763+0000 7fbefa9c0700 1 --2- 172.21.15.62:0/2182097066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4102ed0 0x7fbef410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:23.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.763+0000 7fbefa9c0700 1 --2- 172.21.15.62:0/2182097066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4102510 0x7fbef4102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:23.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.763+0000 7fbefa9c0700 1 -- 172.21.15.62:0/2182097066 >> 172.21.15.62:0/2182097066 conn(0x7fbef40795b0 msgr2=0x7fbef40799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:23.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.763+0000 7fbefa9c0700 1 -- 172.21.15.62:0/2182097066 shutdown_connections 2024-06-10T00:44:23.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.763+0000 7fbefa9c0700 1 -- 172.21.15.62:0/2182097066 wait complete. 2024-06-10T00:44:23.765 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:23 smithi062 ceph-mon[32390]: pgmap v359: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-06-10T00:44:23.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.763+0000 7fbefa9c0700 1 Processor -- start 2024-06-10T00:44:23.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.763+0000 7fbefa9c0700 1 -- start start 2024-06-10T00:44:23.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbefa9c0700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef4102510 0x7fbef419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:23.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbefa9c0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4102ed0 0x7fbef419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbefa9c0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4108510 0x7fbef4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbefa9c0700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4112760 con 0x7fbef4102510 2024-06-10T00:44:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbefa9c0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41125e0 con 0x7fbef4102ed0 2024-06-10T00:44:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbefa9c0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbef41128e0 con 0x7fbef4108510 2024-06-10T00:44:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbef3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef4102510 0x7fbef419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbef3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef4102510 0x7fbef419e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44684/0 (socket says 172.21.15.62:44684) 2024-06-10T00:44:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbef3fff700 1 -- 172.21.15.62:0/604917910 learned_addr learned my addr 172.21.15.62:0/604917910 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:44:23.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbeebfff700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4102ed0 0x7fbef419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbef8f5d700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4108510 0x7fbef4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbef3fff700 1 -- 172.21.15.62:0/604917910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4108510 msgr2=0x7fbef4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbef3fff700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4108510 0x7fbef4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbef3fff700 1 -- 172.21.15.62:0/604917910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4102ed0 msgr2=0x7fbef419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbef3fff700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4102ed0 0x7fbef419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbef3fff700 1 -- 172.21.15.62:0/604917910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef4199200 con 0x7fbef4102510 2024-06-10T00:44:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.764+0000 7fbeebfff700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4102ed0 0x7fbef419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:44:23.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.765+0000 7fbef3fff700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef4102510 0x7fbef419e8e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbee4000c00 tx=0x7fbee40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.765+0000 7fbef1ffb700 1 -- 172.21.15.62:0/604917910 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4027070 con 0x7fbef4102510 2024-06-10T00:44:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.765+0000 7fbef1ffb700 1 -- 172.21.15.62:0/604917910 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbee4004050 con 0x7fbef4102510 2024-06-10T00:44:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.765+0000 7fbef1ffb700 1 -- 172.21.15.62:0/604917910 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee400f460 con 0x7fbef4102510 2024-06-10T00:44:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.765+0000 7fbefa9c0700 1 -- 172.21.15.62:0/604917910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef4199490 con 0x7fbef4102510 2024-06-10T00:44:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.765+0000 7fbefa9c0700 1 -- 172.21.15.62:0/604917910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef41a57b0 con 0x7fbef4102510 2024-06-10T00:44:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.766+0000 7fbefa9c0700 1 -- 172.21.15.62:0/604917910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef410aa10 con 0x7fbef4102510 2024-06-10T00:44:23.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.767+0000 7fbef1ffb700 1 -- 172.21.15.62:0/604917910 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbee402a070 con 0x7fbef4102510 2024-06-10T00:44:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.768+0000 7fbef1ffb700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbee00a2d90 0x7fbee00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.768+0000 7fbeebfff700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbee00a2d90 0x7fbee00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:23.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.768+0000 7fbef1ffb700 1 -- 172.21.15.62:0/604917910 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7fbee40d0820 con 0x7fbef4102510 2024-06-10T00:44:23.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.768+0000 7fbeebfff700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbee00a2d90 0x7fbee00a5250 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbef4079c60 tx=0x7fbedc00b2c0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:23.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:23.773+0000 7fbef1ffb700 1 -- 172.21.15.62:0/604917910 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbee405e850 con 0x7fbef4102510 2024-06-10T00:44:23.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:23 smithi138 ceph-mon[36549]: pgmap v359: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-06-10T00:44:23.911 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:23 smithi105 ceph-mon[36321]: pgmap v359: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-06-10T00:44:24.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:24.025+0000 7fbefa9c0700 1 -- 172.21.15.62:0/604917910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbef4199fd0 con 0x7fbef4102510 2024-06-10T00:44:24.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:24.026+0000 7fbef1ffb700 1 -- 172.21.15.62:0/604917910 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11938 (secure 0 0 0) 0x7fbee4013070 con 0x7fbef4102510 2024-06-10T00:44:24.027 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:44:24.028 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:40:10.974660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:44:24.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:24.029+0000 7fbefa9c0700 1 -- 172.21.15.62:0/604917910 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbee00a2d90 msgr2=0x7fbee00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:24.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:24.029+0000 7fbefa9c0700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbee00a2d90 0x7fbee00a5250 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbef4079c60 tx=0x7fbedc00b2c0 comp rx=0 tx=0).stop 2024-06-10T00:44:24.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:24.029+0000 7fbefa9c0700 1 -- 172.21.15.62:0/604917910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef4102510 msgr2=0x7fbef419e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:24.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:24.029+0000 7fbefa9c0700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef4102510 0x7fbef419e8e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbee4000c00 tx=0x7fbee40047b0 comp rx=0 tx=0).stop 2024-06-10T00:44:24.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:24.029+0000 7fbefa9c0700 1 -- 172.21.15.62:0/604917910 shutdown_connections 2024-06-10T00:44:24.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:24.029+0000 7fbefa9c0700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbee00a2d90 0x7fbee00a5250 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:24.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:24.029+0000 7fbefa9c0700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbef4102510 0x7fbef419e8e0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:24.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:24.029+0000 7fbefa9c0700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4108510 0x7fbef4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:24.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:24.029+0000 7fbefa9c0700 1 --2- 172.21.15.62:0/604917910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbef4102ed0 0x7fbef419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:24.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:24.029+0000 7fbefa9c0700 1 -- 172.21.15.62:0/604917910 >> 172.21.15.62:0/604917910 conn(0x7fbef40795b0 msgr2=0x7fbef4101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:24.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:24.029+0000 7fbefa9c0700 1 -- 172.21.15.62:0/604917910 shutdown_connections 2024-06-10T00:44:24.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:24.029+0000 7fbefa9c0700 1 -- 172.21.15.62:0/604917910 wait complete. 2024-06-10T00:44:24.031 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 38 2024-06-10T00:44:24.712 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:24 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/604917910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:44:24.743 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:44:24.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:24 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/604917910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:44:25.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:24 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/604917910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:44:25.038 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:44:25.857 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:25 smithi062 ceph-mon[32390]: pgmap v360: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 386 B/s wr, 10 op/s 2024-06-10T00:44:25.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:25 smithi138 ceph-mon[36549]: pgmap v360: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 386 B/s wr, 10 op/s 2024-06-10T00:44:26.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:25 smithi105 ceph-mon[36321]: pgmap v360: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 386 B/s wr, 10 op/s 2024-06-10T00:44:26.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.962+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2883194970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a4107400 msgr2=0x7f01a41077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:26.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.962+0000 7f01a9bf5700 1 --2- 172.21.15.62:0/2883194970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a4107400 0x7f01a41077e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f0194009990 tx=0x7f0194009ca0 comp rx=0 tx=0).stop 2024-06-10T00:44:26.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.962+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2883194970 shutdown_connections 2024-06-10T00:44:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.962+0000 7f01a9bf5700 1 --2- 172.21.15.62:0/2883194970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a4107400 0x7f01a41077e0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.962+0000 7f01a9bf5700 1 --2- 172.21.15.62:0/2883194970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a4101d50 0x7f01a410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.962+0000 7f01a9bf5700 1 --2- 172.21.15.62:0/2883194970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01a4101390 0x7f01a4101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.962+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2883194970 >> 172.21.15.62:0/2883194970 conn(0x7f01a4078340 msgr2=0x7f01a4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:26.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.962+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2883194970 shutdown_connections 2024-06-10T00:44:26.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.962+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2883194970 wait complete. 2024-06-10T00:44:26.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.963+0000 7f01a9bf5700 1 Processor -- start 2024-06-10T00:44:26.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.963+0000 7f01a9bf5700 1 -- start start 2024-06-10T00:44:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a9bf5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a4101390 0x7f01a419d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a9bf5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01a4101d50 0x7f01a419dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a9bf5700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a4107400 0x7f01a41979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a9bf5700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f01a4111670 con 0x7f01a4107400 2024-06-10T00:44:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a9bf5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f01a41114f0 con 0x7f01a4101390 2024-06-10T00:44:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a9bf5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f01a41117f0 con 0x7f01a4101d50 2024-06-10T00:44:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a37fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a4101390 0x7f01a419d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a3fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a4107400 0x7f01a41979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a37fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a4101390 0x7f01a419d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.62:50578/0 (socket says 172.21.15.62:50578) 2024-06-10T00:44:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a37fe700 1 -- 172.21.15.62:0/2635225521 learned_addr learned my addr 172.21.15.62:0/2635225521 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:44:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a2ffd700 1 --2- 172.21.15.62:0/2635225521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01a4101d50 0x7f01a419dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:26.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a37fe700 1 -- 172.21.15.62:0/2635225521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01a4101d50 msgr2=0x7f01a419dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:26.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a37fe700 1 --2- 172.21.15.62:0/2635225521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01a4101d50 0x7f01a419dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:26.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a37fe700 1 -- 172.21.15.62:0/2635225521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a4107400 msgr2=0x7f01a41979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:26.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a37fe700 1 --2- 172.21.15.62:0/2635225521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a4107400 0x7f01a41979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:26.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a37fe700 1 -- 172.21.15.62:0/2635225521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a4198130 con 0x7f01a4101390 2024-06-10T00:44:26.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a3fff700 1 --2- 172.21.15.62:0/2635225521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a4107400 0x7f01a41979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:44:26.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.964+0000 7f01a37fe700 1 --2- 172.21.15.62:0/2635225521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a4101390 0x7f01a419d810 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f01940038b0 tx=0x7f0194009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:26.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.965+0000 7f01a0ff9700 1 -- 172.21.15.62:0/2635225521 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194029070 con 0x7f01a4101390 2024-06-10T00:44:26.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.965+0000 7f01a0ff9700 1 -- 172.21.15.62:0/2635225521 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0194019cf0 con 0x7f01a4101390 2024-06-10T00:44:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.965+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2635225521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a41983c0 con 0x7f01a4101390 2024-06-10T00:44:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.965+0000 7f01a0ff9700 1 -- 172.21.15.62:0/2635225521 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194004110 con 0x7f01a4101390 2024-06-10T00:44:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.965+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2635225521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a406a170 con 0x7f01a4101390 2024-06-10T00:44:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.966+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2635225521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a4102ab0 con 0x7f01a4101390 2024-06-10T00:44:26.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.967+0000 7f01a0ff9700 1 -- 172.21.15.62:0/2635225521 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f019402c070 con 0x7f01a4101390 2024-06-10T00:44:26.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.968+0000 7f01a0ff9700 1 --2- 172.21.15.62:0/2635225521 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f01900a2d90 0x7f01900a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:26.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.968+0000 7f01a2ffd700 1 --2- 172.21.15.62:0/2635225521 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f01900a2d90 0x7f01900a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:26.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.968+0000 7f01a0ff9700 1 -- 172.21.15.62:0/2635225521 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7f01940d2690 con 0x7f01a4101390 2024-06-10T00:44:26.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.968+0000 7f01a2ffd700 1 --2- 172.21.15.62:0/2635225521 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f01900a2d90 0x7f01900a5250 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f01a4078a00 tx=0x7f018c00b410 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:26.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:26.973+0000 7f01a0ff9700 1 -- 172.21.15.62:0/2635225521 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0194013070 con 0x7f01a4101390 2024-06-10T00:44:27.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:27.226+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2635225521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01a4069180 con 0x7f01a4101390 2024-06-10T00:44:27.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:27.227+0000 7f01a0ff9700 1 -- 172.21.15.62:0/2635225521 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11938 (secure 0 0 0) 0x7f019409eed0 con 0x7f01a4101390 2024-06-10T00:44:27.228 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:44:27.228 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3839480602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3839480602},{"type":"v1","addr":"172.21.15.62:6835","nonce":3839480602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/147630858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":147630858},{"type":"v1","addr":"172.21.15.62:6837","nonce":147630858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:28:39.478894+0000","modified":"2024-06-10T00:40:10.974660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24464,"mds_1":24433,"mds_2":24431,"mds_3":24424,"mds_4":24418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/4040406390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":4040406390},{"type":"v1","addr":"172.21.15.62:6839","nonce":4040406390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/2460552589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":2460552589},{"type":"v1","addr":"172.21.15.62:6841","nonce":2460552589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6833/1194987741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1194987741},{"type":"v1","addr":"172.21.15.105:6833","nonce":1194987741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/2673201423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2673201423},{"type":"v1","addr":"172.21.15.138:6833","nonce":2673201423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/588077925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":588077925},{"type":"v1","addr":"172.21.15.105:6835","nonce":588077925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":4,"incarnation":26,"state":"up:active","state_seq":27,"addr":"172.21.15.138:6835/331298815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":331298815},{"type":"v1","addr":"172.21.15.138:6835","nonce":331298815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":26,"addr":"172.21.15.138:6837/3880084398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3880084398},{"type":"v1","addr":"172.21.15.138:6837","nonce":3880084398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.105:6837/4270590303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":4270590303},{"type":"v1","addr":"172.21.15.105:6837","nonce":4270590303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.138:6839/339302206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":339302206},{"type":"v1","addr":"172.21.15.138:6839","nonce":339302206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.105:6839/1632080154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":1632080154},{"type":"v1","addr":"172.21.15.105:6839","nonce":1632080154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24433,24418,24431,24424]},"id":1}]} 2024-06-10T00:44:27.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:27.229+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2635225521 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f01900a2d90 msgr2=0x7f01900a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:27.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:27.229+0000 7f01a9bf5700 1 --2- 172.21.15.62:0/2635225521 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f01900a2d90 0x7f01900a5250 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f01a4078a00 tx=0x7f018c00b410 comp rx=0 tx=0).stop 2024-06-10T00:44:27.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:27.229+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2635225521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a4101390 msgr2=0x7f01a419d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:27.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:27.229+0000 7f01a9bf5700 1 --2- 172.21.15.62:0/2635225521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a4101390 0x7f01a419d810 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f01940038b0 tx=0x7f0194009f50 comp rx=0 tx=0).stop 2024-06-10T00:44:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:27.230+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2635225521 shutdown_connections 2024-06-10T00:44:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:27.230+0000 7f01a9bf5700 1 --2- 172.21.15.62:0/2635225521 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f01900a2d90 0x7f01900a5250 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:27.230+0000 7f01a9bf5700 1 --2- 172.21.15.62:0/2635225521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01a4107400 0x7f01a41979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:27.230+0000 7f01a9bf5700 1 --2- 172.21.15.62:0/2635225521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01a4101390 0x7f01a419d810 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:27.230+0000 7f01a9bf5700 1 --2- 172.21.15.62:0/2635225521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01a4101d50 0x7f01a419dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:27.230+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2635225521 >> 172.21.15.62:0/2635225521 conn(0x7f01a4078340 msgr2=0x7f01a4100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:27.230+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2635225521 shutdown_connections 2024-06-10T00:44:27.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:27.230+0000 7f01a9bf5700 1 -- 172.21.15.62:0/2635225521 wait complete. 2024-06-10T00:44:27.232 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 38 2024-06-10T00:44:27.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:27 smithi062 ceph-mon[32390]: pgmap v361: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-06-10T00:44:27.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:27 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2635225521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:44:27.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:27 smithi138 ceph-mon[36549]: pgmap v361: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-06-10T00:44:27.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:27 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2635225521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:44:27.922 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:44:28.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:27 smithi105 ceph-mon[36321]: pgmap v361: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-06-10T00:44:28.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:27 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2635225521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:44:28.176 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:44:29.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:29 smithi062 ceph-mon[32390]: pgmap v362: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-06-10T00:44:29.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:29 smithi138 ceph-mon[36549]: pgmap v362: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-06-10T00:44:30.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.987+0000 7fbbcec47700 1 -- 172.21.15.62:0/3541474261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc8101f40 msgr2=0x7fbbc81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:30.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.987+0000 7fbbcec47700 1 --2- 172.21.15.62:0/3541474261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc8101f40 0x7fbbc81023c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0099c0 tx=0x7fbbbc009cd0 comp rx=0 tx=0).stop 2024-06-10T00:44:30.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.987+0000 7fbbcec47700 1 -- 172.21.15.62:0/3541474261 shutdown_connections 2024-06-10T00:44:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.987+0000 7fbbcec47700 1 --2- 172.21.15.62:0/3541474261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbc8102900 0x7fbbc810e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.987+0000 7fbbcec47700 1 --2- 172.21.15.62:0/3541474261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbc8107660 0x7fbbc8107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.987+0000 7fbbcec47700 1 --2- 172.21.15.62:0/3541474261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc8101f40 0x7fbbc81023c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.987+0000 7fbbcec47700 1 -- 172.21.15.62:0/3541474261 >> 172.21.15.62:0/3541474261 conn(0x7fbbc80766b0 msgr2=0x7fbbc8076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.987+0000 7fbbcec47700 1 -- 172.21.15.62:0/3541474261 shutdown_connections 2024-06-10T00:44:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.987+0000 7fbbcec47700 1 -- 172.21.15.62:0/3541474261 wait complete. 2024-06-10T00:44:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.987+0000 7fbbcec47700 1 Processor -- start 2024-06-10T00:44:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.987+0000 7fbbcec47700 1 -- start start 2024-06-10T00:44:30.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.987+0000 7fbbcec47700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbc8101f40 0x7fbbc8198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:30.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.987+0000 7fbbcec47700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc8102900 0x7fbbc8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:30.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.987+0000 7fbbcec47700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbc8107660 0x7fbbc819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:30.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbcec47700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc8111850 con 0x7fbbc8102900 2024-06-10T00:44:30.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbcec47700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81116d0 con 0x7fbbc8107660 2024-06-10T00:44:30.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbcec47700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81119d0 con 0x7fbbc8101f40 2024-06-10T00:44:30.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbc7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc8102900 0x7fbbc8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:30.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbc7fff700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc8102900 0x7fbbc8199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60924/0 (socket says 172.21.15.62:60924) 2024-06-10T00:44:30.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbc7fff700 1 -- 172.21.15.62:0/1720073911 learned_addr learned my addr 172.21.15.62:0/1720073911 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:44:30.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbcd1e4700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbc8107660 0x7fbbc819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:30.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbcc9e3700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbc8101f40 0x7fbbc8198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:30.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbc7fff700 1 -- 172.21.15.62:0/1720073911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbc8101f40 msgr2=0x7fbbc8198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:30.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbc7fff700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbc8101f40 0x7fbbc8198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:30.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbc7fff700 1 -- 172.21.15.62:0/1720073911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbc8107660 msgr2=0x7fbbc819d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:30.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbc7fff700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbc8107660 0x7fbbc819d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:30.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbc7fff700 1 -- 172.21.15.62:0/1720073911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc819ddf0 con 0x7fbbc8102900 2024-06-10T00:44:30.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbcd1e4700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbc8107660 0x7fbbc819d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:44:30.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbc7fff700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc8102900 0x7fbbc8199320 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0059d0 tx=0x7fbbbc0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:30.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbc5ffb700 1 -- 172.21.15.62:0/1720073911 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc027070 con 0x7fbbc8102900 2024-06-10T00:44:30.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbcec47700 1 -- 172.21.15.62:0/1720073911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc819e080 con 0x7fbbc8102900 2024-06-10T00:44:30.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbc5ffb700 1 -- 172.21.15.62:0/1720073911 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbbc004050 con 0x7fbbc8102900 2024-06-10T00:44:30.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.988+0000 7fbbc5ffb700 1 -- 172.21.15.62:0/1720073911 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc00f460 con 0x7fbbc8102900 2024-06-10T00:44:30.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.989+0000 7fbbcec47700 1 -- 172.21.15.62:0/1720073911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc81a5960 con 0x7fbbc8102900 2024-06-10T00:44:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.990+0000 7fbbcec47700 1 -- 172.21.15.62:0/1720073911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbc8069180 con 0x7fbbc8102900 2024-06-10T00:44:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.991+0000 7fbbc5ffb700 1 -- 172.21.15.62:0/1720073911 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbbbc02a070 con 0x7fbbc8102900 2024-06-10T00:44:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.991+0000 7fbbc5ffb700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbbb00a2d90 0x7fbbb00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.991+0000 7fbbcc9e3700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbbb00a2d90 0x7fbbb00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.991+0000 7fbbc5ffb700 1 -- 172.21.15.62:0/1720073911 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7fbbbc0d0820 con 0x7fbbc8102900 2024-06-10T00:44:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.992+0000 7fbbcc9e3700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbbb00a2d90 0x7fbbb00a5250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8005950 tx=0x7fbbb80058e0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:29.996+0000 7fbbc5ffb700 1 -- 172.21.15.62:0/1720073911 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbbbc05e850 con 0x7fbbc8102900 2024-06-10T00:44:30.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:29 smithi105 ceph-mon[36321]: pgmap v362: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-06-10T00:44:30.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:30.204+0000 7fbbcec47700 1 -- 172.21.15.62:0/1720073911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbbc802dca0 con 0x7fbbc8102900 2024-06-10T00:44:30.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:30.206+0000 7fbbc5ffb700 1 -- 172.21.15.62:0/1720073911 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+25403 (secure 0 0 0) 0x7fbbbc013070 con 0x7fbbc8102900 2024-06-10T00:44:30.207 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-06-10T00:44:30.207 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":110,"fsid":"1353b2ce-26bf-11ef-bc9b-c7b262605968","created":"2024-06-10T00:20:27.981753+0000","modified":"2024-06-10T00:44:13.541128+0000","last_up_change":"2024-06-10T00:27:33.979434+0000","last_in_change":"2024-06-10T00:27:13.887415+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:23:46.372672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:28:33.319774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:28:36.578569+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:28:47.112439+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a148b909-d252-4277-b842-365cb9f169d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6803","nonce":2865393246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6805","nonce":2865393246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6809","nonce":2865393246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":2865393246},{"type":"v1","addr":"172.21.15.62:6807","nonce":2865393246}]},"public_addr":"172.21.15.62:6803/2865393246","cluster_addr":"172.21.15.62:6805/2865393246","heartbeat_back_addr":"172.21.15.62:6809/2865393246","heartbeat_front_addr":"172.21.15.62:6807/2865393246","state":["exists","up"]},{"osd":1,"uuid":"d83d356c-ece6-401e-8f1e-156d5076c817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6801","nonce":1473053019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6803","nonce":1473053019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6807","nonce":1473053019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1473053019},{"type":"v1","addr":"172.21.15.105:6805","nonce":1473053019}]},"public_addr":"172.21.15.105:6801/1473053019","cluster_addr":"172.21.15.105:6803/1473053019","heartbeat_back_addr":"172.21.15.105:6807/1473053019","heartbeat_front_addr":"172.21.15.105:6805/1473053019","state":["exists","up"]},{"osd":2,"uuid":"863db396-689a-4e13-8b18-534f0e38a86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6801","nonce":4170554663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6803","nonce":4170554663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6807","nonce":4170554663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":4170554663},{"type":"v1","addr":"172.21.15.138:6805","nonce":4170554663}]},"public_addr":"172.21.15.138:6801/4170554663","cluster_addr":"172.21.15.138:6803/4170554663","heartbeat_back_addr":"172.21.15.138:6807/4170554663","heartbeat_front_addr":"172.21.15.138:6805/4170554663","state":["exists","up"]},{"osd":3,"uuid":"756165fb-7807-41e8-a200-d23686c300a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6811","nonce":851860835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6813","nonce":851860835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6817","nonce":851860835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":851860835},{"type":"v1","addr":"172.21.15.62:6815","nonce":851860835}]},"public_addr":"172.21.15.62:6811/851860835","cluster_addr":"172.21.15.62:6813/851860835","heartbeat_back_addr":"172.21.15.62:6817/851860835","heartbeat_front_addr":"172.21.15.62:6815/851860835","state":["exists","up"]},{"osd":4,"uuid":"73ed9b45-1a37-4078-83e4-e34607aa2835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6809","nonce":1897328738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6811","nonce":1897328738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6815","nonce":1897328738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1897328738},{"type":"v1","addr":"172.21.15.105:6813","nonce":1897328738}]},"public_addr":"172.21.15.105:6809/1897328738","cluster_addr":"172.21.15.105:6811/1897328738","heartbeat_back_addr":"172.21.15.105:6815/1897328738","heartbeat_front_addr":"172.21.15.105:6813/1897328738","state":["exists","up"]},{"osd":5,"uuid":"8c0fe663-9fd5-465e-808f-210bfd7ac201","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6809","nonce":3331484519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6811","nonce":3331484519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6815","nonce":3331484519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3331484519},{"type":"v1","addr":"172.21.15.138:6813","nonce":3331484519}]},"public_addr":"172.21.15.138:6809/3331484519","cluster_addr":"172.21.15.138:6811/3331484519","heartbeat_back_addr":"172.21.15.138:6815/3331484519","heartbeat_front_addr":"172.21.15.138:6813/3331484519","state":["exists","up"]},{"osd":6,"uuid":"2e48bb00-3d9c-4d73-aa6c-2c1cf05f896e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6819","nonce":1649680325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6821","nonce":1649680325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6825","nonce":1649680325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":1649680325},{"type":"v1","addr":"172.21.15.62:6823","nonce":1649680325}]},"public_addr":"172.21.15.62:6819/1649680325","cluster_addr":"172.21.15.62:6821/1649680325","heartbeat_back_addr":"172.21.15.62:6825/1649680325","heartbeat_front_addr":"172.21.15.62:6823/1649680325","state":["exists","up"]},{"osd":7,"uuid":"59bacdeb-ed9d-4fdb-a1e9-f49732af81df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6817","nonce":3912053087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6819","nonce":3912053087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6823","nonce":3912053087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3912053087},{"type":"v1","addr":"172.21.15.105:6821","nonce":3912053087}]},"public_addr":"172.21.15.105:6817/3912053087","cluster_addr":"172.21.15.105:6819/3912053087","heartbeat_back_addr":"172.21.15.105:6823/3912053087","heartbeat_front_addr":"172.21.15.105:6821/3912053087","state":["exists","up"]},{"osd":8,"uuid":"b5f449fd-35c6-4885-827a-ae7f5b104404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6817","nonce":3579467531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6819","nonce":3579467531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6823","nonce":3579467531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3579467531},{"type":"v1","addr":"172.21.15.138:6821","nonce":3579467531}]},"public_addr":"172.21.15.138:6817/3579467531","cluster_addr":"172.21.15.138:6819/3579467531","heartbeat_back_addr":"172.21.15.138:6823/3579467531","heartbeat_front_addr":"172.21.15.138:6821/3579467531","state":["exists","up"]},{"osd":9,"uuid":"f02b059a-ffcd-4dd2-b8a1-5782cc47a378","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6827","nonce":1109246101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6829","nonce":1109246101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6833","nonce":1109246101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":1109246101},{"type":"v1","addr":"172.21.15.62:6831","nonce":1109246101}]},"public_addr":"172.21.15.62:6827/1109246101","cluster_addr":"172.21.15.62:6829/1109246101","heartbeat_back_addr":"172.21.15.62:6833/1109246101","heartbeat_front_addr":"172.21.15.62:6831/1109246101","state":["exists","up"]},{"osd":10,"uuid":"55659c31-fcfc-4881-9533-8a1d6de76bc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6825","nonce":576155679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6827","nonce":576155679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6831","nonce":576155679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":576155679},{"type":"v1","addr":"172.21.15.105:6829","nonce":576155679}]},"public_addr":"172.21.15.105:6825/576155679","cluster_addr":"172.21.15.105:6827/576155679","heartbeat_back_addr":"172.21.15.105:6831/576155679","heartbeat_front_addr":"172.21.15.105:6829/576155679","state":["exists","up"]},{"osd":11,"uuid":"c01832bc-be75-400a-a707-5fe11d996b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6825","nonce":364141685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6827","nonce":364141685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6831","nonce":364141685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":364141685},{"type":"v1","addr":"172.21.15.138:6829","nonce":364141685}]},"public_addr":"172.21.15.138:6825/364141685","cluster_addr":"172.21.15.138:6827/364141685","heartbeat_back_addr":"172.21.15.138:6831/364141685","heartbeat_front_addr":"172.21.15.138:6829/364141685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:22:56.595732+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:20.092020+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:23:43.362238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:09.525186+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:34.760029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:24:59.481913+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:24.470913+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:25:49.952136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:14.409280+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:26:40.110074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:06.193535+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:27:31.973980+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.1a","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3602410963":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/1557827118":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/2350700567":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/3856763121":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/381111141":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/1704528283":"2024-06-11T00:21:21.851532+0000","172.21.15.62:0/3912577307":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/3602410963":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/1086776708":"2024-06-11T00:32:49.913741+0000","172.21.15.62:0/178507377":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/4137526131":"2024-06-11T00:20:55.088039+0000","172.21.15.62:0/2528347688":"2024-06-11T00:32:49.913741+0000","172.21.15.62:6801/2327526412":"2024-06-11T00:21:21.851532+0000","172.21.15.62:6800/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6801/594215923":"2024-06-11T00:20:55.088039+0000","172.21.15.62:6800/2327526412":"2024-06-11T00:21:21.851532+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:44:30.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:30.209+0000 7fbbcec47700 1 -- 172.21.15.62:0/1720073911 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbbb00a2d90 msgr2=0x7fbbb00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:30.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:30.209+0000 7fbbcec47700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbbb00a2d90 0x7fbbb00a5250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8005950 tx=0x7fbbb80058e0 comp rx=0 tx=0).stop 2024-06-10T00:44:30.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:30.209+0000 7fbbcec47700 1 -- 172.21.15.62:0/1720073911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc8102900 msgr2=0x7fbbc8199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:30.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:30.209+0000 7fbbcec47700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc8102900 0x7fbbc8199320 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0059d0 tx=0x7fbbbc0038b0 comp rx=0 tx=0).stop 2024-06-10T00:44:30.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:30.209+0000 7fbbcec47700 1 -- 172.21.15.62:0/1720073911 shutdown_connections 2024-06-10T00:44:30.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:30.209+0000 7fbbcec47700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fbbb00a2d90 0x7fbbb00a5250 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:30.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:30.209+0000 7fbbcec47700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbc8107660 0x7fbbc819d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:30.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:30.209+0000 7fbbcec47700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbc8101f40 0x7fbbc8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:30.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:30.209+0000 7fbbcec47700 1 --2- 172.21.15.62:0/1720073911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbc8102900 0x7fbbc8199320 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:30.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:30.209+0000 7fbbcec47700 1 -- 172.21.15.62:0/1720073911 >> 172.21.15.62:0/1720073911 conn(0x7fbbc80766b0 msgr2=0x7fbbc8100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:30.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:30.209+0000 7fbbcec47700 1 -- 172.21.15.62:0/1720073911 shutdown_connections 2024-06-10T00:44:30.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:30.209+0000 7fbbcec47700 1 -- 172.21.15.62:0/1720073911 wait complete. 2024-06-10T00:44:30.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:30 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1720073911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:44:30.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:30 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1720073911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:44:31.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:30 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1720073911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:44:31.042 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs fail cephfs 2024-06-10T00:44:31.294 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:44:31.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:31 smithi062 ceph-mon[32390]: pgmap v363: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-10T00:44:31.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:31 smithi138 ceph-mon[36549]: pgmap v363: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-10T00:44:32.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:31 smithi105 ceph-mon[36321]: pgmap v363: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-06-10T00:44:32.792 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:32 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:44:32.792 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:32 smithi062 ceph-mon[32390]: pgmap v364: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-10T00:44:32.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:32 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:44:32.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:32 smithi138 ceph-mon[36549]: pgmap v364: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-10T00:44:33.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:32 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:44:33.032 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:32 smithi105 ceph-mon[36321]: pgmap v364: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-06-10T00:44:33.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.039+0000 7fa5d085a700 1 -- 172.21.15.62:0/2735249676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5cc108510 msgr2=0x7fa5cc1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:33.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.039+0000 7fa5d085a700 1 --2- 172.21.15.62:0/2735249676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5cc108510 0x7fa5cc1088f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009990 tx=0x7fa5bc009ca0 comp rx=0 tx=0).stop 2024-06-10T00:44:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.039+0000 7fa5d085a700 1 -- 172.21.15.62:0/2735249676 shutdown_connections 2024-06-10T00:44:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.039+0000 7fa5d085a700 1 --2- 172.21.15.62:0/2735249676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5cc108510 0x7fa5cc1088f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.039+0000 7fa5d085a700 1 --2- 172.21.15.62:0/2735249676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5cc101440 0x7fa5cc10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.039+0000 7fa5d085a700 1 --2- 172.21.15.62:0/2735249676 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5cc100a80 0x7fa5cc100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.039+0000 7fa5d085a700 1 -- 172.21.15.62:0/2735249676 >> 172.21.15.62:0/2735249676 conn(0x7fa5cc077a00 msgr2=0x7fa5cc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.039+0000 7fa5d085a700 1 -- 172.21.15.62:0/2735249676 shutdown_connections 2024-06-10T00:44:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.039+0000 7fa5d085a700 1 -- 172.21.15.62:0/2735249676 wait complete. 2024-06-10T00:44:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.040+0000 7fa5d085a700 1 Processor -- start 2024-06-10T00:44:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.040+0000 7fa5d085a700 1 -- start start 2024-06-10T00:44:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5d085a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5cc100a80 0x7fa5cc19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5d085a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5cc101440 0x7fa5cc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5d085a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5cc108510 0x7fa5cc198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5d085a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc112790 con 0x7fa5cc108510 2024-06-10T00:44:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5d085a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc112610 con 0x7fa5cc101440 2024-06-10T00:44:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5d085a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc112910 con 0x7fa5cc100a80 2024-06-10T00:44:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5cad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5cc108510 0x7fa5cc198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5cad9d700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5cc108510 0x7fa5cc198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60938/0 (socket says 172.21.15.62:60938) 2024-06-10T00:44:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5cad9d700 1 -- 172.21.15.62:0/1186588396 learned_addr learned my addr 172.21.15.62:0/1186588396 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:44:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5c9d9b700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5cc101440 0x7fa5cc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5ca59c700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5cc100a80 0x7fa5cc19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5cad9d700 1 -- 172.21.15.62:0/1186588396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5cc100a80 msgr2=0x7fa5cc19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5cad9d700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5cc100a80 0x7fa5cc19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:33.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5cad9d700 1 -- 172.21.15.62:0/1186588396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5cc101440 msgr2=0x7fa5cc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5cad9d700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5cc101440 0x7fa5cc19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5cad9d700 1 -- 172.21.15.62:0/1186588396 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc1991b0 con 0x7fa5cc108510 2024-06-10T00:44:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.041+0000 7fa5c9d9b700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5cc101440 0x7fa5cc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:44:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.042+0000 7fa5cad9d700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5cc108510 0x7fa5cc198a50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4007ee0 tx=0x7fa5c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.042+0000 7fa5c37fe700 1 -- 172.21.15.62:0/1186588396 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4004bb0 con 0x7fa5cc108510 2024-06-10T00:44:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.042+0000 7fa5c37fe700 1 -- 172.21.15.62:0/1186588396 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c4004d50 con 0x7fa5cc108510 2024-06-10T00:44:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.042+0000 7fa5d085a700 1 -- 172.21.15.62:0/1186588396 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc1994a0 con 0x7fa5cc108510 2024-06-10T00:44:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.042+0000 7fa5c37fe700 1 -- 172.21.15.62:0/1186588396 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4005960 con 0x7fa5cc108510 2024-06-10T00:44:33.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.042+0000 7fa5d085a700 1 -- 172.21.15.62:0/1186588396 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5cc1a5710 con 0x7fa5cc108510 2024-06-10T00:44:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.043+0000 7fa5d085a700 1 -- 172.21.15.62:0/1186588396 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5cc10a9f0 con 0x7fa5cc108510 2024-06-10T00:44:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.045+0000 7fa5c37fe700 1 -- 172.21.15.62:0/1186588396 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa5c401a020 con 0x7fa5cc108510 2024-06-10T00:44:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.046+0000 7fa5c37fe700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa5b80a2d90 0x7fa5b80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:33.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.046+0000 7fa5ca59c700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa5b80a2d90 0x7fa5b80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.046+0000 7fa5c37fe700 1 -- 172.21.15.62:0/1186588396 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7fa5c40ceee0 con 0x7fa5cc108510 2024-06-10T00:44:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.046+0000 7fa5ca59c700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa5b80a2d90 0x7fa5b80a5250 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00b5c0 tx=0x7fa5bc005960 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:33.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.051+0000 7fa5c37fe700 1 -- 172.21.15.62:0/1186588396 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa5c4010070 con 0x7fa5cc108510 2024-06-10T00:44:33.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.326+0000 7fa5d085a700 1 -- 172.21.15.62:0/1186588396 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa5cc10a3d0 con 0x7fa5cc108510 2024-06-10T00:44:33.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.329+0000 7fa5c37fe700 1 -- 172.21.15.62:0/1186588396 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) ==== 161+0+0 (secure 0 0 0) 0x7fa5c4022250 con 0x7fa5cc108510 2024-06-10T00:44:33.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.333+0000 7fa5d085a700 1 -- 172.21.15.62:0/1186588396 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa5b80a2d90 msgr2=0x7fa5b80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:33.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.333+0000 7fa5d085a700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa5b80a2d90 0x7fa5b80a5250 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00b5c0 tx=0x7fa5bc005960 comp rx=0 tx=0).stop 2024-06-10T00:44:33.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.333+0000 7fa5d085a700 1 -- 172.21.15.62:0/1186588396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5cc108510 msgr2=0x7fa5cc198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:33.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.333+0000 7fa5d085a700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5cc108510 0x7fa5cc198a50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4007ee0 tx=0x7fa5c400d3b0 comp rx=0 tx=0).stop 2024-06-10T00:44:33.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.333+0000 7fa5d085a700 1 -- 172.21.15.62:0/1186588396 shutdown_connections 2024-06-10T00:44:33.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.333+0000 7fa5d085a700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa5b80a2d90 0x7fa5b80a5250 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:33.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.333+0000 7fa5d085a700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5cc108510 0x7fa5cc198a50 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:33.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.333+0000 7fa5d085a700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5cc100a80 0x7fa5cc19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:33.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.333+0000 7fa5d085a700 1 --2- 172.21.15.62:0/1186588396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5cc101440 0x7fa5cc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:33.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.333+0000 7fa5d085a700 1 -- 172.21.15.62:0/1186588396 >> 172.21.15.62:0/1186588396 conn(0x7fa5cc077a00 msgr2=0x7fa5cc0ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:33.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.333+0000 7fa5d085a700 1 -- 172.21.15.62:0/1186588396 shutdown_connections 2024-06-10T00:44:33.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:33.333+0000 7fa5d085a700 1 -- 172.21.15.62:0/1186588396 wait complete. 2024-06-10T00:44:33.335 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-10T00:44:33.601 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a[32386]: 2024-06-10T00:44:33.327+0000 7fd94415c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T00:44:33.602 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-a[74274]: ignoring --setuser ceph since I am not root 2024-06-10T00:44:33.602 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-a[74274]: ignoring --setgroup ceph since I am not root 2024-06-10T00:44:33.602 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-mds[74278]: -- 172.21.15.62:0/3384373470 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a4dedda40 con 0x558a4deb0800 2024-06-10T00:44:33.602 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-a[74274]: starting mds.a at 2024-06-10T00:44:33.602 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-g[73816]: ignoring --setuser ceph since I am not root 2024-06-10T00:44:33.602 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-g[73816]: ignoring --setgroup ceph since I am not root 2024-06-10T00:44:33.603 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-mds[73838]: -- 172.21.15.62:0/2208764259 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565100015a40 con 0x5650fffe8c00 2024-06-10T00:44:33.603 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-g[73816]: starting mds.g at 2024-06-10T00:44:33.782 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-h[66546]: ignoring --setuser ceph since I am not root 2024-06-10T00:44:33.782 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-h[66546]: ignoring --setgroup ceph since I am not root 2024-06-10T00:44:33.783 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-mds[66550]: -- 172.21.15.105:0/1445590875 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556d01a724e0 con 0x556d01a48800 2024-06-10T00:44:33.783 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-mds[66550]: -- 172.21.15.105:0/1445590875 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d01a75a40 con 0x556d01a48800 2024-06-10T00:44:33.783 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-h[66546]: starting mds.h at 2024-06-10T00:44:33.783 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-e[66232]: ignoring --setuser ceph since I am not root 2024-06-10T00:44:33.783 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-e[66232]: ignoring --setgroup ceph since I am not root 2024-06-10T00:44:33.783 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-mds[66236]: -- 172.21.15.105:0/2174965019 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e3fa929a40 con 0x55e3fa8fd000 2024-06-10T00:44:33.784 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-e[66232]: starting mds.e at 2024-06-10T00:44:33.784 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-k[65919]: ignoring --setuser ceph since I am not root 2024-06-10T00:44:33.784 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-k[65919]: ignoring --setgroup ceph since I am not root 2024-06-10T00:44:33.784 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-mds[65923]: -- 172.21.15.105:0/4034738476 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55984c00da40 con 0x55984bfe0c00 2024-06-10T00:44:33.784 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-k[65919]: starting mds.k at 2024-06-10T00:44:33.785 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:44:33.785 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:44:33.785 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:44:33.785 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1186588396' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-10T00:44:33.785 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-mon[36321]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-10T00:44:33.785 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-mon[36321]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T00:44:33.785 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1186588396' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-10T00:44:33.785 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-mon[36321]: osdmap e111: 12 total, 12 up, 12 in 2024-06-10T00:44:33.786 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-mon[36321]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-10T00:44:33.786 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-b[66851]: ignoring --setuser ceph since I am not root 2024-06-10T00:44:33.786 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-b[66851]: ignoring --setgroup ceph since I am not root 2024-06-10T00:44:33.786 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-mds[66855]: -- 172.21.15.105:0/1846085603 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5645ba313a40 con 0x5645ba2e6c00 2024-06-10T00:44:33.786 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:44:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-b[66851]: starting mds.b at 2024-06-10T00:44:33.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:44:33.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:44:33.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:44:33.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1186588396' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-10T00:44:33.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-mon[32390]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-10T00:44:33.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-mon[32390]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T00:44:33.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1186588396' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-10T00:44:33.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-mon[32390]: osdmap e111: 12 total, 12 up, 12 in 2024-06-10T00:44:33.859 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:33 smithi062 ceph-mon[32390]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-10T00:44:33.873 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-f[66287]: ignoring --setuser ceph since I am not root 2024-06-10T00:44:33.874 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-f[66287]: ignoring --setgroup ceph since I am not root 2024-06-10T00:44:33.874 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-mds[66292]: -- 172.21.15.138:0/2584400573 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ee192da40 con 0x561ee1900800 2024-06-10T00:44:33.874 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-f[66287]: starting mds.f at 2024-06-10T00:44:33.874 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-l[65983]: ignoring --setuser ceph since I am not root 2024-06-10T00:44:33.874 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-l[65983]: ignoring --setgroup ceph since I am not root 2024-06-10T00:44:33.875 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-mds[65987]: -- 172.21.15.138:0/2807759854 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55870fa65a40 con 0x55870fa39000 2024-06-10T00:44:33.875 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-l[65983]: starting mds.l at 2024-06-10T00:44:33.875 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-c[66601]: ignoring --setuser ceph since I am not root 2024-06-10T00:44:33.875 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-c[66601]: ignoring --setgroup ceph since I am not root 2024-06-10T00:44:33.875 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-mds[66605]: -- 172.21.15.138:0/1047714055 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562e65d81a40 con 0x562e65d54800 2024-06-10T00:44:33.875 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-c[66601]: starting mds.c at 2024-06-10T00:44:33.876 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:44:33.876 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:44:33.876 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' 2024-06-10T00:44:33.876 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1186588396' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-10T00:44:33.876 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-mon[36549]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-10T00:44:33.876 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-mon[36549]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T00:44:33.877 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1186588396' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-10T00:44:33.877 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-mon[36549]: osdmap e111: 12 total, 12 up, 12 in 2024-06-10T00:44:33.877 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-mon[36549]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-10T00:44:33.877 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-i[66903]: ignoring --setuser ceph since I am not root 2024-06-10T00:44:33.877 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-i[66903]: ignoring --setgroup ceph since I am not root 2024-06-10T00:44:33.877 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-mds[66907]: -- 172.21.15.138:0/60070878 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55631aac3a40 con 0x55631aa97000 2024-06-10T00:44:33.877 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:44:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-i[66903]: starting mds.i at 2024-06-10T00:44:34.084 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-10T00:44:34.335 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:44:35.584 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: pgmap v366: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 819 B/s wr, 11 op/s 2024-06-10T00:44:35.585 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.105:6833/3381479545,v1:172.21.15.105:6835/3381479545] up:boot 2024-06-10T00:44:35.585 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.105:6832/592603894,v1:172.21.15.105:6834/592603894] up:boot 2024-06-10T00:44:35.585 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.138:6833/3996753731,v1:172.21.15.138:6835/3996753731] up:boot 2024-06-10T00:44:35.585 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.138:6832/757157962,v1:172.21.15.138:6834/757157962] up:boot 2024-06-10T00:44:35.585 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.62:6838/637013556,v1:172.21.15.62:6839/637013556] up:boot 2024-06-10T00:44:35.585 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.105:6836/2814324097,v1:172.21.15.105:6838/2814324097] up:boot 2024-06-10T00:44:35.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.105:6837/3458182550,v1:172.21.15.105:6839/3458182550] up:boot 2024-06-10T00:44:35.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.138:6836/2858375783,v1:172.21.15.138:6837/2858375783] up:boot 2024-06-10T00:44:35.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.138:6838/794833557,v1:172.21.15.138:6839/794833557] up:boot 2024-06-10T00:44:35.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: mds.? [v2:172.21.15.62:6840/440773883,v1:172.21.15.62:6841/440773883] up:boot 2024-06-10T00:44:35.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-10T00:44:35.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T00:44:35.586 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T00:44:35.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T00:44:35.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T00:44:35.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T00:44:35.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T00:44:35.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T00:44:35.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T00:44:35.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T00:44:35.587 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:35 smithi062 ceph-mon[32390]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T00:44:35.588 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: pgmap v366: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 819 B/s wr, 11 op/s 2024-06-10T00:44:35.588 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.105:6833/3381479545,v1:172.21.15.105:6835/3381479545] up:boot 2024-06-10T00:44:35.588 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.105:6832/592603894,v1:172.21.15.105:6834/592603894] up:boot 2024-06-10T00:44:35.588 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.138:6833/3996753731,v1:172.21.15.138:6835/3996753731] up:boot 2024-06-10T00:44:35.588 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.138:6832/757157962,v1:172.21.15.138:6834/757157962] up:boot 2024-06-10T00:44:35.589 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.62:6838/637013556,v1:172.21.15.62:6839/637013556] up:boot 2024-06-10T00:44:35.589 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.105:6836/2814324097,v1:172.21.15.105:6838/2814324097] up:boot 2024-06-10T00:44:35.589 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.105:6837/3458182550,v1:172.21.15.105:6839/3458182550] up:boot 2024-06-10T00:44:35.589 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.138:6836/2858375783,v1:172.21.15.138:6837/2858375783] up:boot 2024-06-10T00:44:35.589 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.138:6838/794833557,v1:172.21.15.138:6839/794833557] up:boot 2024-06-10T00:44:35.589 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: mds.? [v2:172.21.15.62:6840/440773883,v1:172.21.15.62:6841/440773883] up:boot 2024-06-10T00:44:35.589 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-10T00:44:35.589 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T00:44:35.589 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T00:44:35.590 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T00:44:35.590 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T00:44:35.590 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T00:44:35.590 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T00:44:35.590 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T00:44:35.591 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T00:44:35.591 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T00:44:35.591 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:35 smithi105 ceph-mon[36321]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T00:44:35.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: pgmap v366: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 819 B/s wr, 11 op/s 2024-06-10T00:44:35.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.105:6833/3381479545,v1:172.21.15.105:6835/3381479545] up:boot 2024-06-10T00:44:35.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.105:6832/592603894,v1:172.21.15.105:6834/592603894] up:boot 2024-06-10T00:44:35.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.138:6833/3996753731,v1:172.21.15.138:6835/3996753731] up:boot 2024-06-10T00:44:35.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.138:6832/757157962,v1:172.21.15.138:6834/757157962] up:boot 2024-06-10T00:44:35.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.62:6838/637013556,v1:172.21.15.62:6839/637013556] up:boot 2024-06-10T00:44:35.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.105:6836/2814324097,v1:172.21.15.105:6838/2814324097] up:boot 2024-06-10T00:44:35.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.105:6837/3458182550,v1:172.21.15.105:6839/3458182550] up:boot 2024-06-10T00:44:35.625 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.138:6836/2858375783,v1:172.21.15.138:6837/2858375783] up:boot 2024-06-10T00:44:35.626 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.138:6838/794833557,v1:172.21.15.138:6839/794833557] up:boot 2024-06-10T00:44:35.626 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: mds.? [v2:172.21.15.62:6840/440773883,v1:172.21.15.62:6841/440773883] up:boot 2024-06-10T00:44:35.626 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-10T00:44:35.626 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T00:44:35.626 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T00:44:35.626 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T00:44:35.627 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T00:44:35.627 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T00:44:35.627 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T00:44:35.627 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T00:44:35.627 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T00:44:35.627 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T00:44:35.627 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:35 smithi138 ceph-mon[36549]: from='mgr.15333 172.21.15.62:0/846544276' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T00:44:36.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.112+0000 7fa7c23c8700 1 -- 172.21.15.62:0/2875601282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7bc107860 msgr2=0x7fa7bc10f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:36.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.112+0000 7fa7c23c8700 1 --2- 172.21.15.62:0/2875601282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7bc107860 0x7fa7bc10f3e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400b0d0 tx=0x7fa7b400b3e0 comp rx=0 tx=0).stop 2024-06-10T00:44:36.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.112+0000 7fa7c23c8700 1 -- 172.21.15.62:0/2875601282 shutdown_connections 2024-06-10T00:44:36.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.112+0000 7fa7c23c8700 1 --2- 172.21.15.62:0/2875601282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7bc107860 0x7fa7bc10f3e0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:36.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.112+0000 7fa7c23c8700 1 --2- 172.21.15.62:0/2875601282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7bc102770 0x7fa7bc102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:36.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.112+0000 7fa7c23c8700 1 --2- 172.21.15.62:0/2875601282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7bc103120 0x7fa7bc107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:36.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.112+0000 7fa7c23c8700 1 -- 172.21.15.62:0/2875601282 >> 172.21.15.62:0/2875601282 conn(0x7fa7bc079600 msgr2=0x7fa7bc079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:36.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.112+0000 7fa7c23c8700 1 -- 172.21.15.62:0/2875601282 shutdown_connections 2024-06-10T00:44:36.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.112+0000 7fa7c23c8700 1 -- 172.21.15.62:0/2875601282 wait complete. 2024-06-10T00:44:36.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.112+0000 7fa7c23c8700 1 Processor -- start 2024-06-10T00:44:36.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.112+0000 7fa7c23c8700 1 -- start start 2024-06-10T00:44:36.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c23c8700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7bc102770 0x7fa7bc19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:36.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c23c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7bc103120 0x7fa7bc19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:36.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c23c8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7bc107860 0x7fa7bc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:36.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c23c8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc112780 con 0x7fa7bc107860 2024-06-10T00:44:36.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c23c8700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc112600 con 0x7fa7bc102770 2024-06-10T00:44:36.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c23c8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc112900 con 0x7fa7bc103120 2024-06-10T00:44:36.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c0965700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7bc107860 0x7fa7bc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:36.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c0965700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7bc107860 0x7fa7bc198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32770/0 (socket says 172.21.15.62:32770) 2024-06-10T00:44:36.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c0965700 1 -- 172.21.15.62:0/1313371135 learned_addr learned my addr 172.21.15.62:0/1313371135 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:44:36.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7bb7fe700 1 --2- 172.21.15.62:0/1313371135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7bc103120 0x7fa7bc19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:36.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7bbfff700 1 --2- 172.21.15.62:0/1313371135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7bc102770 0x7fa7bc19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:36.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c0965700 1 -- 172.21.15.62:0/1313371135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7bc103120 msgr2=0x7fa7bc19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:36.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c0965700 1 --2- 172.21.15.62:0/1313371135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7bc103120 0x7fa7bc19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:36.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c0965700 1 -- 172.21.15.62:0/1313371135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7bc102770 msgr2=0x7fa7bc19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:36.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c0965700 1 --2- 172.21.15.62:0/1313371135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7bc102770 0x7fa7bc19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:36.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c0965700 1 -- 172.21.15.62:0/1313371135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7bc1990a0 con 0x7fa7bc107860 2024-06-10T00:44:36.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.113+0000 7fa7c0965700 1 --2- 172.21.15.62:0/1313371135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7bc107860 0x7fa7bc198940 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4006070 tx=0x7fa7b4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:36.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.114+0000 7fa7b97fa700 1 -- 172.21.15.62:0/1313371135 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4021070 con 0x7fa7bc107860 2024-06-10T00:44:36.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.114+0000 7fa7c23c8700 1 -- 172.21.15.62:0/1313371135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7bc199330 con 0x7fa7bc107860 2024-06-10T00:44:36.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.114+0000 7fa7b97fa700 1 -- 172.21.15.62:0/1313371135 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4007b80 con 0x7fa7bc107860 2024-06-10T00:44:36.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.114+0000 7fa7b97fa700 1 -- 172.21.15.62:0/1313371135 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4005330 con 0x7fa7bc107860 2024-06-10T00:44:36.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.114+0000 7fa7c23c8700 1 -- 172.21.15.62:0/1313371135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7bc1a5650 con 0x7fa7bc107860 2024-06-10T00:44:36.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.115+0000 7fa7c23c8700 1 -- 172.21.15.62:0/1313371135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7bc069180 con 0x7fa7bc107860 2024-06-10T00:44:36.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.116+0000 7fa7b97fa700 1 -- 172.21.15.62:0/1313371135 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa7b4024070 con 0x7fa7bc107860 2024-06-10T00:44:36.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.117+0000 7fa7b97fa700 1 --2- 172.21.15.62:0/1313371135 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa7a80a2d90 0x7fa7a80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:36.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.117+0000 7fa7bbfff700 1 --2- 172.21.15.62:0/1313371135 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa7a80a2d90 0x7fa7a80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:36.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.117+0000 7fa7b97fa700 1 -- 172.21.15.62:0/1313371135 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9896+0+0 (secure 0 0 0) 0x7fa7b400f020 con 0x7fa7bc107860 2024-06-10T00:44:36.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.117+0000 7fa7bbfff700 1 --2- 172.21.15.62:0/1313371135 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa7a80a2d90 0x7fa7a80a5250 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac005950 tx=0x7fa7ac005ed0 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:36.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.121+0000 7fa7b97fa700 1 -- 172.21.15.62:0/1313371135 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa7b402b020 con 0x7fa7bc107860 2024-06-10T00:44:36.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:36.377+0000 7fa7c23c8700 1 -- 172.21.15.62:0/1313371135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa7bc1a5880 con 0x7fa7bc107860 2024-06-10T00:44:37.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:37.339+0000 7fa7b97fa700 1 -- 172.21.15.62:0/1313371135 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) ==== 104+0+0 (secure 0 0 0) 0x7fa7b409d6e0 con 0x7fa7bc107860 2024-06-10T00:44:37.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:37.344+0000 7fa7c23c8700 1 -- 172.21.15.62:0/1313371135 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa7a80a2d90 msgr2=0x7fa7a80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:37.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:37.344+0000 7fa7c23c8700 1 --2- 172.21.15.62:0/1313371135 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa7a80a2d90 0x7fa7a80a5250 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac005950 tx=0x7fa7ac005ed0 comp rx=0 tx=0).stop 2024-06-10T00:44:37.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:37.344+0000 7fa7c23c8700 1 -- 172.21.15.62:0/1313371135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7bc107860 msgr2=0x7fa7bc198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:37.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:37.344+0000 7fa7c23c8700 1 --2- 172.21.15.62:0/1313371135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7bc107860 0x7fa7bc198940 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4006070 tx=0x7fa7b4004d00 comp rx=0 tx=0).stop 2024-06-10T00:44:37.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:37.344+0000 7fa7c23c8700 1 -- 172.21.15.62:0/1313371135 shutdown_connections 2024-06-10T00:44:37.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:37.344+0000 7fa7c23c8700 1 --2- 172.21.15.62:0/1313371135 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7fa7a80a2d90 0x7fa7a80a5250 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:37.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:37.344+0000 7fa7c23c8700 1 --2- 172.21.15.62:0/1313371135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7bc107860 0x7fa7bc198940 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:37.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:37.344+0000 7fa7c23c8700 1 --2- 172.21.15.62:0/1313371135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7bc102770 0x7fa7bc19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:37.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:37.344+0000 7fa7c23c8700 1 --2- 172.21.15.62:0/1313371135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7bc103120 0x7fa7bc19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:37.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:37.344+0000 7fa7c23c8700 1 -- 172.21.15.62:0/1313371135 >> 172.21.15.62:0/1313371135 conn(0x7fa7bc079600 msgr2=0x7fa7bc100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:37.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:37.344+0000 7fa7c23c8700 1 -- 172.21.15.62:0/1313371135 shutdown_connections 2024-06-10T00:44:37.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:37.344+0000 7fa7c23c8700 1 -- 172.21.15.62:0/1313371135 wait complete. 2024-06-10T00:44:37.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:37 smithi062 ceph-mon[32390]: pgmap v367: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 9 op/s 2024-06-10T00:44:37.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:37 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1313371135' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:44:37.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:37 smithi138 ceph-mon[36549]: pgmap v367: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 9 op/s 2024-06-10T00:44:37.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:37 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1313371135' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:44:37.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:37 smithi105 ceph-mon[36321]: pgmap v367: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 9 op/s 2024-06-10T00:44:37.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:37 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1313371135' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:44:38.155 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-10T00:44:38.414 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:44:38.536 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:38 smithi062 ceph-mon[32390]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-10T00:44:38.537 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:38 smithi062 ceph-mon[32390]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-10T00:44:38.537 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:38 smithi062 ceph-mon[32390]: Cluster is now healthy 2024-06-10T00:44:38.537 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:38 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1313371135' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-10T00:44:38.537 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:38 smithi062 ceph-mon[32390]: fsmap 12 up:standby 2024-06-10T00:44:38.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:38 smithi138 ceph-mon[36549]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-10T00:44:38.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:38 smithi138 ceph-mon[36549]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-10T00:44:38.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:38 smithi138 ceph-mon[36549]: Cluster is now healthy 2024-06-10T00:44:38.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:38 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1313371135' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-10T00:44:38.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:38 smithi138 ceph-mon[36549]: fsmap 12 up:standby 2024-06-10T00:44:38.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:38 smithi105 ceph-mon[36321]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-10T00:44:38.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:38 smithi105 ceph-mon[36321]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-10T00:44:38.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:38 smithi105 ceph-mon[36321]: Cluster is now healthy 2024-06-10T00:44:38.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:38 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1313371135' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-10T00:44:38.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:38 smithi105 ceph-mon[36321]: fsmap 12 up:standby 2024-06-10T00:44:39.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:39 smithi062 ceph-mon[32390]: pgmap v368: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-06-10T00:44:39.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:39 smithi138 ceph-mon[36549]: pgmap v368: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-06-10T00:44:39.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:39 smithi105 ceph-mon[36321]: pgmap v368: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-06-10T00:44:40.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.213+0000 7f04297ce700 1 -- 172.21.15.62:0/2076348236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f042406ab30 msgr2=0x7f042410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.213+0000 7f04297ce700 1 --2- 172.21.15.62:0/2076348236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f042406ab30 0x7f042410f320 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f041c00b0a0 tx=0x7f041c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:44:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.213+0000 7f04297ce700 1 -- 172.21.15.62:0/2076348236 shutdown_connections 2024-06-10T00:44:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.213+0000 7f04297ce700 1 --2- 172.21.15.62:0/2076348236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f042406ab30 0x7f042410f320 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.213+0000 7f04297ce700 1 --2- 172.21.15.62:0/2076348236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0424108500 0x7f04241088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.213+0000 7f04297ce700 1 --2- 172.21.15.62:0/2076348236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f042406a170 0x7f042406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.213+0000 7f04297ce700 1 -- 172.21.15.62:0/2076348236 >> 172.21.15.62:0/2076348236 conn(0x7f0424078340 msgr2=0x7f0424078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.213+0000 7f04297ce700 1 -- 172.21.15.62:0/2076348236 shutdown_connections 2024-06-10T00:44:40.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.213+0000 7f04297ce700 1 -- 172.21.15.62:0/2076348236 wait complete. 2024-06-10T00:44:40.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.214+0000 7f04297ce700 1 Processor -- start 2024-06-10T00:44:40.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.214+0000 7f04297ce700 1 -- start start 2024-06-10T00:44:40.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.214+0000 7f04297ce700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f042406a170 0x7f042419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:40.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.214+0000 7f04297ce700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f042406ab30 0x7f042419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:40.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.214+0000 7f04297ce700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0424108500 0x7f0424198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:40.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.214+0000 7f04297ce700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0424112860 con 0x7f042406a170 2024-06-10T00:44:40.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.214+0000 7f04297ce700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04241126e0 con 0x7f042406ab30 2024-06-10T00:44:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.214+0000 7f04297ce700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04241129e0 con 0x7f0424108500 2024-06-10T00:44:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.214+0000 7f0422ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f042406a170 0x7f042419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.214+0000 7f0422ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f042406a170 0x7f042419e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55538/0 (socket says 172.21.15.62:55538) 2024-06-10T00:44:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.214+0000 7f0422ffd700 1 -- 172.21.15.62:0/1256391585 learned_addr learned my addr 172.21.15.62:0/1256391585 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:44:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.215+0000 7f04227fc700 1 --2- 172.21.15.62:0/1256391585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f042406ab30 0x7f042419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.215+0000 7f04237fe700 1 --2- 172.21.15.62:0/1256391585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0424108500 0x7f0424198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.215+0000 7f0422ffd700 1 -- 172.21.15.62:0/1256391585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0424108500 msgr2=0x7f0424198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:40.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.215+0000 7f0422ffd700 1 --2- 172.21.15.62:0/1256391585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0424108500 0x7f0424198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:40.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.215+0000 7f0422ffd700 1 -- 172.21.15.62:0/1256391585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f042406ab30 msgr2=0x7f042419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:40.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.215+0000 7f0422ffd700 1 --2- 172.21.15.62:0/1256391585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f042406ab30 0x7f042419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:40.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.215+0000 7f0422ffd700 1 -- 172.21.15.62:0/1256391585 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0424199240 con 0x7f042406a170 2024-06-10T00:44:40.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.215+0000 7f0422ffd700 1 --2- 172.21.15.62:0/1256391585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f042406a170 0x7f042419e8d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f041400b530 tx=0x7f041400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:40.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.215+0000 7f040bfff700 1 -- 172.21.15.62:0/1256391585 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414015020 con 0x7f042406a170 2024-06-10T00:44:40.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.215+0000 7f040bfff700 1 -- 172.21.15.62:0/1256391585 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0414011d10 con 0x7f042406a170 2024-06-10T00:44:40.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.215+0000 7f04297ce700 1 -- 172.21.15.62:0/1256391585 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0424199530 con 0x7f042406a170 2024-06-10T00:44:40.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.215+0000 7f040bfff700 1 -- 172.21.15.62:0/1256391585 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041400f2e0 con 0x7f042406a170 2024-06-10T00:44:40.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.215+0000 7f04297ce700 1 -- 172.21.15.62:0/1256391585 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04241a57a0 con 0x7f042406a170 2024-06-10T00:44:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.216+0000 7f04297ce700 1 -- 172.21.15.62:0/1256391585 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f042410a9e0 con 0x7f042406a170 2024-06-10T00:44:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.218+0000 7f040bfff700 1 -- 172.21.15.62:0/1256391585 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0414018020 con 0x7f042406a170 2024-06-10T00:44:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.219+0000 7f040bfff700 1 --2- 172.21.15.62:0/1256391585 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f040c0a2d90 0x7f040c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.219+0000 7f04227fc700 1 --2- 172.21.15.62:0/1256391585 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f040c0a2d90 0x7f040c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.219+0000 7f040bfff700 1 -- 172.21.15.62:0/1256391585 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9896+0+0 (secure 0 0 0) 0x7f04140cee90 con 0x7f042406a170 2024-06-10T00:44:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.219+0000 7f04227fc700 1 --2- 172.21.15.62:0/1256391585 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f040c0a2d90 0x7f040c0a5250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f04240788b0 tx=0x7f0418009190 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:40.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.223+0000 7f040bfff700 1 -- 172.21.15.62:0/1256391585 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f041401f030 con 0x7f042406a170 2024-06-10T00:44:40.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:40.431+0000 7f04297ce700 1 -- 172.21.15.62:0/1256391585 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0424069180 con 0x7f042406a170 2024-06-10T00:44:41.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:41.355+0000 7f040bfff700 1 -- 172.21.15.62:0/1256391585 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) ==== 181+0+0 (secure 0 0 0) 0x7f041409b660 con 0x7f042406a170 2024-06-10T00:44:41.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:41.358+0000 7f04297ce700 1 -- 172.21.15.62:0/1256391585 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f040c0a2d90 msgr2=0x7f040c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:41.358+0000 7f04297ce700 1 --2- 172.21.15.62:0/1256391585 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f040c0a2d90 0x7f040c0a5250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f04240788b0 tx=0x7f0418009190 comp rx=0 tx=0).stop 2024-06-10T00:44:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:41.358+0000 7f04297ce700 1 -- 172.21.15.62:0/1256391585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f042406a170 msgr2=0x7f042419e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:41.358+0000 7f04297ce700 1 --2- 172.21.15.62:0/1256391585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f042406a170 0x7f042419e8d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f041400b530 tx=0x7f041400b840 comp rx=0 tx=0).stop 2024-06-10T00:44:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:41.358+0000 7f04297ce700 1 -- 172.21.15.62:0/1256391585 shutdown_connections 2024-06-10T00:44:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:41.358+0000 7f04297ce700 1 --2- 172.21.15.62:0/1256391585 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f040c0a2d90 0x7f040c0a5250 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:41.358+0000 7f04297ce700 1 --2- 172.21.15.62:0/1256391585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f042406a170 0x7f042419e8d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:41.358+0000 7f04297ce700 1 --2- 172.21.15.62:0/1256391585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0424108500 0x7f0424198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:41.358+0000 7f04297ce700 1 --2- 172.21.15.62:0/1256391585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f042406ab30 0x7f042419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:41.358+0000 7f04297ce700 1 -- 172.21.15.62:0/1256391585 >> 172.21.15.62:0/1256391585 conn(0x7f0424078340 msgr2=0x7f04240fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:41.359+0000 7f04297ce700 1 -- 172.21.15.62:0/1256391585 shutdown_connections 2024-06-10T00:44:41.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:41.359+0000 7f04297ce700 1 -- 172.21.15.62:0/1256391585 wait complete. 2024-06-10T00:44:41.362 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-06-10T00:44:41.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:41 smithi062 ceph-mon[32390]: pgmap v369: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-06-10T00:44:41.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:41 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1256391585' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T00:44:41.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:41 smithi138 ceph-mon[36549]: pgmap v369: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-06-10T00:44:41.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:41 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1256391585' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T00:44:41.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:41 smithi105 ceph-mon[36321]: pgmap v369: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-06-10T00:44:41.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:41 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1256391585' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T00:44:42.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-10T00:44:42.367 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:44:42.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:42 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/1256391585' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T00:44:42.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:42 smithi138 ceph-mon[36549]: osdmap e112: 12 total, 12 up, 12 in 2024-06-10T00:44:42.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:42 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/1256391585' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T00:44:42.660 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:42 smithi062 ceph-mon[32390]: osdmap e112: 12 total, 12 up, 12 in 2024-06-10T00:44:42.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:42 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/1256391585' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T00:44:42.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:42 smithi105 ceph-mon[36321]: osdmap e112: 12 total, 12 up, 12 in 2024-06-10T00:44:43.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:43 smithi062 ceph-mon[32390]: pgmap v371: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:44:43.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:43 smithi138 ceph-mon[36549]: pgmap v371: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:44:43.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:43 smithi105 ceph-mon[36321]: pgmap v371: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:44:44.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.207+0000 7f10019a3700 1 -- 172.21.15.62:0/815161029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ffc107570 msgr2=0x7f0ffc107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:44.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.207+0000 7f10019a3700 1 --2- 172.21.15.62:0/815161029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ffc107570 0x7f0ffc107950 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009990 tx=0x7f0fec009ca0 comp rx=0 tx=0).stop 2024-06-10T00:44:44.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.207+0000 7f10019a3700 1 -- 172.21.15.62:0/815161029 shutdown_connections 2024-06-10T00:44:44.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.207+0000 7f10019a3700 1 --2- 172.21.15.62:0/815161029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ffc107570 0x7f0ffc107950 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:44.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.207+0000 7f10019a3700 1 --2- 172.21.15.62:0/815161029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ffc101810 0x7f0ffc10f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:44.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.207+0000 7f10019a3700 1 --2- 172.21.15.62:0/815161029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ffc100e50 0x7f0ffc1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:44.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.207+0000 7f10019a3700 1 -- 172.21.15.62:0/815161029 >> 172.21.15.62:0/815161029 conn(0x7f0ffc0766b0 msgr2=0x7f0ffc076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:44.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.207+0000 7f10019a3700 1 -- 172.21.15.62:0/815161029 shutdown_connections 2024-06-10T00:44:44.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.207+0000 7f10019a3700 1 -- 172.21.15.62:0/815161029 wait complete. 2024-06-10T00:44:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.208+0000 7f10019a3700 1 Processor -- start 2024-06-10T00:44:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.208+0000 7f10019a3700 1 -- start start 2024-06-10T00:44:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.208+0000 7f10019a3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ffc100e50 0x7f0ffc102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.208+0000 7f10019a3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ffc101810 0x7f0ffc102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:44.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.208+0000 7f10019a3700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ffc107570 0x7f0ffc102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:44.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.208+0000 7f10019a3700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc112830 con 0x7f0ffc107570 2024-06-10T00:44:44.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.208+0000 7f10019a3700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1126b0 con 0x7f0ffc101810 2024-06-10T00:44:44.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.208+0000 7f10019a3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1129b0 con 0x7f0ffc100e50 2024-06-10T00:44:44.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.208+0000 7f0ffb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ffc107570 0x7f0ffc102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:44.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f0ffb7fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ffc107570 0x7f0ffc102ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55560/0 (socket says 172.21.15.62:55560) 2024-06-10T00:44:44.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f0ffb7fe700 1 -- 172.21.15.62:0/2963324097 learned_addr learned my addr 172.21.15.62:0/2963324097 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:44:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f0ffa7fc700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ffc101810 0x7f0ffc102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f0ffaffd700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ffc100e50 0x7f0ffc102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f0ffb7fe700 1 -- 172.21.15.62:0/2963324097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ffc100e50 msgr2=0x7f0ffc102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f0ffb7fe700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ffc100e50 0x7f0ffc102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f0ffb7fe700 1 -- 172.21.15.62:0/2963324097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ffc101810 msgr2=0x7f0ffc102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:44.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f0ffb7fe700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ffc101810 0x7f0ffc102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:44.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f0ffb7fe700 1 -- 172.21.15.62:0/2963324097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc1a7170 con 0x7f0ffc107570 2024-06-10T00:44:44.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f0ffaffd700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ffc100e50 0x7f0ffc102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:44:44.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f0ffb7fe700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ffc107570 0x7f0ffc102ab0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400b970 tx=0x7f0ff400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:44.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f10009a1700 1 -- 172.21.15.62:0/2963324097 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4019070 con 0x7f0ffc107570 2024-06-10T00:44:44.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f10009a1700 1 -- 172.21.15.62:0/2963324097 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff400ce10 con 0x7f0ffc107570 2024-06-10T00:44:44.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f10019a3700 1 -- 172.21.15.62:0/2963324097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc1a7460 con 0x7f0ffc107570 2024-06-10T00:44:44.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f10009a1700 1 -- 172.21.15.62:0/2963324097 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4007370 con 0x7f0ffc107570 2024-06-10T00:44:44.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.209+0000 7f10019a3700 1 -- 172.21.15.62:0/2963324097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc1a79f0 con 0x7f0ffc107570 2024-06-10T00:44:44.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.210+0000 7f10019a3700 1 -- 172.21.15.62:0/2963324097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ffc069180 con 0x7f0ffc107570 2024-06-10T00:44:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.211+0000 7f10009a1700 1 -- 172.21.15.62:0/2963324097 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0ff401c070 con 0x7f0ffc107570 2024-06-10T00:44:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.212+0000 7f10009a1700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f0fe40a2e10 0x7f0fe40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.213+0000 7f0ffaffd700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f0fe40a2e10 0x7f0fe40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.213+0000 7f10009a1700 1 -- 172.21.15.62:0/2963324097 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9313+0+0 (secure 0 0 0) 0x7f0ff40ce960 con 0x7f0ffc107570 2024-06-10T00:44:44.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.213+0000 7f0ffaffd700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f0fe40a2e10 0x7f0fe40a52d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0fec006010 tx=0x7f0fec00b540 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:44.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.217+0000 7f10009a1700 1 -- 172.21.15.62:0/2963324097 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0ff409b370 con 0x7f0ffc107570 2024-06-10T00:44:44.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:44.422+0000 7f10019a3700 1 -- 172.21.15.62:0/2963324097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0ffc1a7db0 con 0x7f0ffc107570 2024-06-10T00:44:45.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:45.370+0000 7f10009a1700 1 -- 172.21.15.62:0/2963324097 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) ==== 169+0+0 (secure 0 0 0) 0x7f0ff4085020 con 0x7f0ffc107570 2024-06-10T00:44:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:45.374+0000 7f10019a3700 1 -- 172.21.15.62:0/2963324097 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f0fe40a2e10 msgr2=0x7f0fe40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:45.374+0000 7f10019a3700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f0fe40a2e10 0x7f0fe40a52d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0fec006010 tx=0x7f0fec00b540 comp rx=0 tx=0).stop 2024-06-10T00:44:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:45.374+0000 7f10019a3700 1 -- 172.21.15.62:0/2963324097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ffc107570 msgr2=0x7f0ffc102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:45.374+0000 7f10019a3700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ffc107570 0x7f0ffc102ab0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400b970 tx=0x7f0ff400bc80 comp rx=0 tx=0).stop 2024-06-10T00:44:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:45.374+0000 7f10019a3700 1 -- 172.21.15.62:0/2963324097 shutdown_connections 2024-06-10T00:44:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:45.374+0000 7f10019a3700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f0fe40a2e10 0x7f0fe40a52d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:45.374+0000 7f10019a3700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ffc107570 0x7f0ffc102ab0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:45.374+0000 7f10019a3700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ffc100e50 0x7f0ffc102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:45.374+0000 7f10019a3700 1 --2- 172.21.15.62:0/2963324097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ffc101810 0x7f0ffc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:45.374+0000 7f10019a3700 1 -- 172.21.15.62:0/2963324097 >> 172.21.15.62:0/2963324097 conn(0x7f0ffc0766b0 msgr2=0x7f0ffc0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:45.375+0000 7f10019a3700 1 -- 172.21.15.62:0/2963324097 shutdown_connections 2024-06-10T00:44:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:45.375+0000 7f10019a3700 1 -- 172.21.15.62:0/2963324097 wait complete. 2024-06-10T00:44:45.377 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-06-10T00:44:45.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:45 smithi138 ceph-mon[36549]: pgmap v372: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:44:45.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:45 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2963324097' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T00:44:45.758 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:45 smithi062 ceph-mon[32390]: pgmap v372: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:44:45.759 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:45 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2963324097' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T00:44:45.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:45 smithi105 ceph-mon[36321]: pgmap v372: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:44:45.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:45 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2963324097' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T00:44:46.286 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-06-10T00:44:46.540 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:44:46.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:46 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2963324097' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T00:44:46.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:46 smithi062 ceph-mon[32390]: osdmap e113: 12 total, 12 up, 12 in 2024-06-10T00:44:46.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:46 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2963324097' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T00:44:46.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:46 smithi138 ceph-mon[36549]: osdmap e113: 12 total, 12 up, 12 in 2024-06-10T00:44:46.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:46 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2963324097' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T00:44:46.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:46 smithi105 ceph-mon[36321]: osdmap e113: 12 total, 12 up, 12 in 2024-06-10T00:44:47.700 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:47 smithi062 ceph-mon[32390]: pgmap v374: 33 pgs: 33 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:44:47.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:47 smithi105 ceph-mon[36321]: pgmap v374: 33 pgs: 33 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:44:47.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:47 smithi138 ceph-mon[36549]: pgmap v374: 33 pgs: 33 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:44:48.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.427+0000 7f8ec6e12700 1 -- 172.21.15.62:0/814727738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0108500 msgr2=0x7f8ec01088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:48.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.427+0000 7f8ec6e12700 1 --2- 172.21.15.62:0/814727738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0108500 0x7f8ec01088e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009990 tx=0x7f8eb4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:44:48.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.427+0000 7f8ec6e12700 1 -- 172.21.15.62:0/814727738 shutdown_connections 2024-06-10T00:44:48.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.427+0000 7f8ec6e12700 1 --2- 172.21.15.62:0/814727738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0108500 0x7f8ec01088e0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:48.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.427+0000 7f8ec6e12700 1 --2- 172.21.15.62:0/814727738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ec006b580 0x7f8ec010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:48.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.427+0000 7f8ec6e12700 1 --2- 172.21.15.62:0/814727738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ec006abc0 0x7f8ec006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:48.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.427+0000 7f8ec6e12700 1 -- 172.21.15.62:0/814727738 >> 172.21.15.62:0/814727738 conn(0x7f8ec0078340 msgr2=0x7f8ec0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:48.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.427+0000 7f8ec6e12700 1 -- 172.21.15.62:0/814727738 shutdown_connections 2024-06-10T00:44:48.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.427+0000 7f8ec6e12700 1 -- 172.21.15.62:0/814727738 wait complete. 2024-06-10T00:44:48.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.427+0000 7f8ec6e12700 1 Processor -- start 2024-06-10T00:44:48.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec6e12700 1 -- start start 2024-06-10T00:44:48.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec6e12700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ec006abc0 0x7f8ec019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:48.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec6e12700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ec006b580 0x7f8ec019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:48.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec6e12700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0108500 0x7f8ec0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:48.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec6e12700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec0112760 con 0x7f8ec0108500 2024-06-10T00:44:48.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec6e12700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec01125e0 con 0x7f8ec006b580 2024-06-10T00:44:48.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec6e12700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec01128e0 con 0x7f8ec006abc0 2024-06-10T00:44:48.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec4bae700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ec006abc0 0x7f8ec019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:48.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec4bae700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ec006abc0 0x7f8ec019e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.62:33072/0 (socket says 172.21.15.62:33072) 2024-06-10T00:44:48.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec4bae700 1 -- 172.21.15.62:0/2565585846 learned_addr learned my addr 172.21.15.62:0/2565585846 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-06-10T00:44:48.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ebffff700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ec006b580 0x7f8ec019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:48.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec53af700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0108500 0x7f8ec0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:48.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec53af700 1 -- 172.21.15.62:0/2565585846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ec006abc0 msgr2=0x7f8ec019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:48.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec53af700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ec006abc0 0x7f8ec019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:48.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec53af700 1 -- 172.21.15.62:0/2565585846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ec006b580 msgr2=0x7f8ec019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:48.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec53af700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ec006b580 0x7f8ec019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:48.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec53af700 1 -- 172.21.15.62:0/2565585846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec01990a0 con 0x7f8ec0108500 2024-06-10T00:44:48.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.428+0000 7f8ec4bae700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ec006abc0 0x7f8ec019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:44:48.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.429+0000 7f8ec53af700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0108500 0x7f8ec0198940 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800cc50 tx=0x7f8eb800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:48.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.429+0000 7f8ebdffb700 1 -- 172.21.15.62:0/2565585846 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8016020 con 0x7f8ec0108500 2024-06-10T00:44:48.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.429+0000 7f8ebdffb700 1 -- 172.21.15.62:0/2565585846 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eb8012420 con 0x7f8ec0108500 2024-06-10T00:44:48.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.429+0000 7f8ec6e12700 1 -- 172.21.15.62:0/2565585846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec0199390 con 0x7f8ec0108500 2024-06-10T00:44:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.429+0000 7f8ebdffb700 1 -- 172.21.15.62:0/2565585846 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb80115c0 con 0x7f8ec0108500 2024-06-10T00:44:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.429+0000 7f8ec6e12700 1 -- 172.21.15.62:0/2565585846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec01a5650 con 0x7f8ec0108500 2024-06-10T00:44:48.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.430+0000 7f8ec6e12700 1 -- 172.21.15.62:0/2565585846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec0069180 con 0x7f8ec0108500 2024-06-10T00:44:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.431+0000 7f8ebdffb700 1 -- 172.21.15.62:0/2565585846 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8eb8019020 con 0x7f8ec0108500 2024-06-10T00:44:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.432+0000 7f8ebdffb700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8eb00a2d90 0x7f8eb00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.432+0000 7f8ec4bae700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8eb00a2d90 0x7f8eb00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.432+0000 7f8ebdffb700 1 -- 172.21.15.62:0/2565585846 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8754+0+0 (secure 0 0 0) 0x7f8eb80cf200 con 0x7f8ec0108500 2024-06-10T00:44:48.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.432+0000 7f8ec4bae700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8eb00a2d90 0x7f8eb00a5250 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009fd0 tx=0x7f8eb4009f20 comp rx=0 tx=0).ready entity=mgr.15333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:48.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.436+0000 7f8ebdffb700 1 -- 172.21.15.62:0/2565585846 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8eb8020020 con 0x7f8ec0108500 2024-06-10T00:44:48.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:48.647+0000 7f8ec6e12700 1 -- 172.21.15.62:0/2565585846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8ec01a5880 con 0x7f8ec0108500 2024-06-10T00:44:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:49.380+0000 7f8ebdffb700 1 -- 172.21.15.62:0/2565585846 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v114) ==== 178+0+0 (secure 0 0 0) 0x7f8eb809be40 con 0x7f8ec0108500 2024-06-10T00:44:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:49.384+0000 7f8ec6e12700 1 -- 172.21.15.62:0/2565585846 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8eb00a2d90 msgr2=0x7f8eb00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:49.384+0000 7f8ec6e12700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8eb00a2d90 0x7f8eb00a5250 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009fd0 tx=0x7f8eb4009f20 comp rx=0 tx=0).stop 2024-06-10T00:44:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:49.384+0000 7f8ec6e12700 1 -- 172.21.15.62:0/2565585846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0108500 msgr2=0x7f8ec0198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:49.384+0000 7f8ec6e12700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0108500 0x7f8ec0198940 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800cc50 tx=0x7f8eb800cf60 comp rx=0 tx=0).stop 2024-06-10T00:44:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:49.385+0000 7f8ec6e12700 1 -- 172.21.15.62:0/2565585846 shutdown_connections 2024-06-10T00:44:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:49.385+0000 7f8ec6e12700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.62:6800/518034449,v1:172.21.15.62:6801/518034449] conn(0x7f8eb00a2d90 0x7f8eb00a5250 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:49.385+0000 7f8ec6e12700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ec0108500 0x7f8ec0198940 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:49.385+0000 7f8ec6e12700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ec006abc0 0x7f8ec019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:49.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:49.385+0000 7f8ec6e12700 1 --2- 172.21.15.62:0/2565585846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8ec006b580 0x7f8ec019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:49.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:49.385+0000 7f8ec6e12700 1 -- 172.21.15.62:0/2565585846 >> 172.21.15.62:0/2565585846 conn(0x7f8ec0078340 msgr2=0x7f8ec00ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:49.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:49.385+0000 7f8ec6e12700 1 -- 172.21.15.62:0/2565585846 shutdown_connections 2024-06-10T00:44:49.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-06-10T00:44:49.386+0000 7f8ec6e12700 1 -- 172.21.15.62:0/2565585846 wait complete. 2024-06-10T00:44:49.389 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data_ec' removed 2024-06-10T00:44:49.721 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:49 smithi062 ceph-mon[32390]: pgmap v375: 33 pgs: 33 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:44:49.722 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:49 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2565585846' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T00:44:49.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:49 smithi105 ceph-mon[36321]: pgmap v375: 33 pgs: 33 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:44:49.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:49 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2565585846' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T00:44:49.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:49 smithi138 ceph-mon[36549]: pgmap v375: 33 pgs: 33 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:44:49.874 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:49 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2565585846' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T00:44:50.169 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:44:50.225 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:44:50.277 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:44:50.321 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-10T00:44:50.321 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-10T00:44:50.355 DEBUG:teuthology.orchestra.run.smithi105:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-10T00:44:50.388 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-10T00:44:50.419 INFO:tasks.cephadm:Stopping all daemons... 2024-06-10T00:44:50.420 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-10T00:44:50.420 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mon.a 2024-06-10T00:44:50.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:50 smithi105 ceph-mon[36321]: from='client.? 172.21.15.62:0/2565585846' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T00:44:50.532 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:50 smithi105 ceph-mon[36321]: osdmap e114: 12 total, 12 up, 12 in 2024-06-10T00:44:50.623 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:50 smithi138 ceph-mon[36549]: from='client.? 172.21.15.62:0/2565585846' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T00:44:50.624 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:50 smithi138 ceph-mon[36549]: osdmap e114: 12 total, 12 up, 12 in 2024-06-10T00:44:50.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:50 smithi062 ceph-mon[32390]: from='client.? 172.21.15.62:0/2565585846' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T00:44:50.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:50 smithi062 ceph-mon[32390]: osdmap e114: 12 total, 12 up, 12 in 2024-06-10T00:44:50.858 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:50 smithi062 systemd[1]: Stopping Ceph mon.a for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:44:51.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:51 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a[32386]: 2024-06-10T00:44:50.999+0000 7fd949967700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:44:51.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:51 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a[32386]: 2024-06-10T00:44:50.999+0000 7fd949967700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-10T00:44:51.358 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:51 smithi062 podman[122241]: 2024-06-10 00:44:51.214321868 +0000 UTC m=+0.385627053 container died 4f3ad4cc1fad860bae7ad39415da661be69dbf560e999087ce060d8e310df1b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-10T00:44:52.323 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:51 smithi062 podman[122241]: 2024-06-10 00:44:51.954369104 +0000 UTC m=+1.125674288 container cleanup 4f3ad4cc1fad860bae7ad39415da661be69dbf560e999087ce060d8e310df1b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-10T00:44:52.323 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:51 smithi062 bash[122241]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a 2024-06-10T00:44:52.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Jun 10 00:44:52 smithi062 podman[122251]: 2024-06-10 00:44:52.323018702 +0000 UTC m=+1.102212321 container remove 4f3ad4cc1fad860bae7ad39415da661be69dbf560e999087ce060d8e310df1b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-10T00:44:52.809 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-10T00:44:52.894 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:44:52.895 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-10T00:44:52.895 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-10T00:44:52.895 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mon.b 2024-06-10T00:44:53.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:52 smithi105 systemd[1]: Stopping Ceph mon.b for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:44:53.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:53 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-b[36317]: 2024-06-10T00:44:53.486+0000 7fd4c732b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:44:53.782 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:53 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-b[36317]: 2024-06-10T00:44:53.486+0000 7fd4c732b700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-10T00:44:53.783 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:53 smithi105 podman[73227]: 2024-06-10 00:44:53.72025603 +0000 UTC m=+0.385487512 container died 00abad0a9f836135040c4dc40c8240dc92885f3c5e559714c9b124278d0b70fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528) 2024-06-10T00:44:54.416 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:54 smithi105 podman[73227]: 2024-06-10 00:44:54.1396003 +0000 UTC m=+0.804831779 container cleanup 00abad0a9f836135040c4dc40c8240dc92885f3c5e559714c9b124278d0b70fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:44:54.416 INFO:journalctl@ceph.mon.b.smithi105.stdout:Jun 10 00:44:54 smithi105 bash[73227]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-b 2024-06-10T00:44:54.622 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-10T00:44:54.700 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:44:54.701 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-10T00:44:54.701 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-10T00:44:54.701 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mon.c 2024-06-10T00:44:55.102 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:54 smithi138 systemd[1]: Stopping Ceph mon.c for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:44:55.373 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:55 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-c[36531]: 2024-06-10T00:44:55.101+0000 7fa6992a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:44:55.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:55 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-c[36531]: 2024-06-10T00:44:55.101+0000 7fa6992a3700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-10T00:44:55.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:55 smithi138 podman[73308]: 2024-06-10 00:44:55.25420602 +0000 UTC m=+0.331577561 container died f0c99954ecf68a5a60b29f82282765f6cd700f4c46a00b99da8d8bfda8e381c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-c, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-10T00:44:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:55 smithi138 podman[73308]: 2024-06-10 00:44:55.927856349 +0000 UTC m=+1.005227885 container cleanup f0c99954ecf68a5a60b29f82282765f6cd700f4c46a00b99da8d8bfda8e381c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-c, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-10T00:44:56.374 INFO:journalctl@ceph.mon.c.smithi138.stdout:Jun 10 00:44:55 smithi138 bash[73308]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mon-c 2024-06-10T00:44:57.922 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-10T00:44:58.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:44:58.032 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-10T00:44:58.032 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-10T00:44:58.033 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mgr.x 2024-06-10T00:44:58.358 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:44:58 smithi062 systemd[1]: Stopping Ceph mgr.x for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:44:59.608 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:44:59 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x[32622]: 2024-06-10T00:44:59.351+0000 7f4724fee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:44:59.608 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:44:59 smithi062 podman[122350]: 2024-06-10 00:44:59.394448572 +0000 UTC m=+1.154513978 container died e454fa3945455d94c3af2ef5312ce309bd605f756f1488a3e1303e8c0c2f7707 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:45:05.448 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:45:05 smithi062 podman[122350]: 2024-06-10 00:45:05.21821849 +0000 UTC m=+6.978283886 container cleanup e454fa3945455d94c3af2ef5312ce309bd605f756f1488a3e1303e8c0c2f7707 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-10T00:45:05.449 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Jun 10 00:45:05 smithi062 bash[122350]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-x 2024-06-10T00:45:08.463 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-10T00:45:08.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:45:08.503 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-10T00:45:08.504 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-10T00:45:08.504 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mgr.y 2024-06-10T00:45:08.782 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:45:08 smithi105 systemd[1]: Stopping Ceph mgr.y for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:45:09.052 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:45:08 smithi105 podman[73340]: 2024-06-10 00:45:08.849091149 +0000 UTC m=+0.139531503 container died 313819fe36215bef933cce694698db55433f08183a740565c647092bc992039d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:45:09.531 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:45:09 smithi105 podman[73340]: 2024-06-10 00:45:09.237757445 +0000 UTC m=+0.528197800 container cleanup 313819fe36215bef933cce694698db55433f08183a740565c647092bc992039d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-10T00:45:09.531 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Jun 10 00:45:09 smithi105 bash[73340]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-y 2024-06-10T00:45:09.671 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-10T00:45:09.707 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:45:09.707 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-10T00:45:09.707 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-10T00:45:09.708 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mgr.z 2024-06-10T00:45:10.046 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:45:09 smithi138 systemd[1]: Stopping Ceph mgr.z for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:45:10.304 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:45:10 smithi138 podman[73431]: 2024-06-10 00:45:10.046570559 +0000 UTC m=+0.141330472 container stop 2cfb57db6f25b458cf84de5df70405db286c4bdad9f6d4e33babe47ad2cd2d68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-10T00:45:10.304 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:45:10 smithi138 podman[73431]: 2024-06-10 00:45:10.073083121 +0000 UTC m=+0.167843034 container died 2cfb57db6f25b458cf84de5df70405db286c4bdad9f6d4e33babe47ad2cd2d68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-10T00:45:10.559 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:45:10 smithi138 podman[73431]: 2024-06-10 00:45:10.511131038 +0000 UTC m=+0.605890958 container cleanup 2cfb57db6f25b458cf84de5df70405db286c4bdad9f6d4e33babe47ad2cd2d68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-10T00:45:10.559 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:45:10 smithi138 bash[73431]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z 2024-06-10T00:45:10.873 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:45:10 smithi138 podman[73445]: 2024-06-10 00:45:10.704797561 +0000 UTC m=+0.648439147 container remove 2cfb57db6f25b458cf84de5df70405db286c4bdad9f6d4e33babe47ad2cd2d68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:45:10.874 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Jun 10 00:45:10 smithi138 systemd[1]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-06-10T00:45:10.948 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mgr.z.service' 2024-06-10T00:45:10.983 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:45:10.983 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-10T00:45:10.984 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-10T00:45:10.984 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.0 2024-06-10T00:45:11.358 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:45:11 smithi062 systemd[1]: Stopping Ceph osd.0 for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:45:12.108 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:45:11 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-0[47508]: 2024-06-10T00:45:11.663+0000 7f438b125700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:45:12.108 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:45:11 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-0[47508]: 2024-06-10T00:45:11.663+0000 7f438b125700 -1 osd.0 114 *** Got signal Terminated *** 2024-06-10T00:45:12.108 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:45:11 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-0[47508]: 2024-06-10T00:45:11.663+0000 7f438b125700 -1 osd.0 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T00:45:17.608 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:45:17 smithi062 podman[122485]: 2024-06-10 00:45:17.287151964 +0000 UTC m=+6.095795665 container died b9238668231162c21609f4154a52349a2711db109af60f1ff45bbf6316760c54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-10T00:45:18.108 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:45:17 smithi062 podman[122485]: 2024-06-10 00:45:17.629715396 +0000 UTC m=+6.438359075 container cleanup b9238668231162c21609f4154a52349a2711db109af60f1ff45bbf6316760c54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-10T00:45:18.108 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:45:17 smithi062 bash[122485]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-0 2024-06-10T00:45:18.108 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:45:17 smithi062 podman[122503]: 2024-06-10 00:45:17.839390851 +0000 UTC m=+0.549600723 container remove b9238668231162c21609f4154a52349a2711db109af60f1ff45bbf6316760c54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-0, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:45:18.542 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:45:18 smithi062 podman[122593]: 2024-06-10 00:45:18.157870174 +0000 UTC m=+0.017639860 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:45:18.543 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:45:18 smithi062 podman[122593]: 2024-06-10 00:45:18.273857231 +0000 UTC m=+0.133626951 container create 0899464063c8e863271617095b7143451777665d37ebde0fccbfe1f0af1a5918 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:45:18.858 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:45:18 smithi062 podman[122593]: 2024-06-10 00:45:18.543172283 +0000 UTC m=+0.402941954 container init 0899464063c8e863271617095b7143451777665d37ebde0fccbfe1f0af1a5918 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:45:18.858 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:45:18 smithi062 podman[122593]: 2024-06-10 00:45:18.547543189 +0000 UTC m=+0.407312868 container start 0899464063c8e863271617095b7143451777665d37ebde0fccbfe1f0af1a5918 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:45:18.858 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:45:18 smithi062 podman[122593]: 2024-06-10 00:45:18.626467398 +0000 UTC m=+0.486237089 container attach 0899464063c8e863271617095b7143451777665d37ebde0fccbfe1f0af1a5918 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-10T00:45:19.187 INFO:journalctl@ceph.osd.0.smithi062.stdout:Jun 10 00:45:18 smithi062 podman[122619]: 2024-06-10 00:45:18.915524484 +0000 UTC m=+0.018363181 container died 0899464063c8e863271617095b7143451777665d37ebde0fccbfe1f0af1a5918 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:45:19.503 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-10T00:45:19.546 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:45:19.546 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-10T00:45:19.547 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-10T00:45:19.547 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.1 2024-06-10T00:45:19.782 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:45:19 smithi105 systemd[1]: Stopping Ceph osd.1 for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:45:20.282 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:45:19 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-1[40899]: 2024-06-10T00:45:19.850+0000 7f85570a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:45:20.282 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:45:19 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-1[40899]: 2024-06-10T00:45:19.850+0000 7f85570a0700 -1 osd.1 114 *** Got signal Terminated *** 2024-06-10T00:45:20.282 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:45:19 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-1[40899]: 2024-06-10T00:45:19.850+0000 7f85570a0700 -1 osd.1 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T00:45:25.744 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:45:25 smithi105 podman[73474]: 2024-06-10 00:45:25.435452947 +0000 UTC m=+5.697683337 container died 5e101094640a209aa6037ca697e58f76cafc312a125e2136674b73dafde45f73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-10T00:45:26.032 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:45:25 smithi105 podman[73474]: 2024-06-10 00:45:25.843205637 +0000 UTC m=+6.105436026 container cleanup 5e101094640a209aa6037ca697e58f76cafc312a125e2136674b73dafde45f73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-1, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:45:26.032 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:45:25 smithi105 bash[73474]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-1 2024-06-10T00:45:26.456 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:45:26 smithi105 podman[73489]: 2024-06-10 00:45:26.103180687 +0000 UTC m=+0.666537999 container remove 5e101094640a209aa6037ca697e58f76cafc312a125e2136674b73dafde45f73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-1, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-10T00:45:26.756 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:45:26 smithi105 podman[73581]: 2024-06-10 00:45:26.359322794 +0000 UTC m=+0.017666144 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:45:26.756 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:45:26 smithi105 podman[73581]: 2024-06-10 00:45:26.487336735 +0000 UTC m=+0.145680080 container create bee2ae9e3ab90d2f7ae6576b0be7d30d400c6905db539a4fce41fc420d02f77d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-10T00:45:26.757 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:45:26 smithi105 podman[73581]: 2024-06-10 00:45:26.757001996 +0000 UTC m=+0.415345349 container init bee2ae9e3ab90d2f7ae6576b0be7d30d400c6905db539a4fce41fc420d02f77d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:45:27.032 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:45:26 smithi105 podman[73581]: 2024-06-10 00:45:26.760245095 +0000 UTC m=+0.418588444 container start bee2ae9e3ab90d2f7ae6576b0be7d30d400c6905db539a4fce41fc420d02f77d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-10T00:45:27.032 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:45:26 smithi105 podman[73581]: 2024-06-10 00:45:26.840563844 +0000 UTC m=+0.498907222 container attach bee2ae9e3ab90d2f7ae6576b0be7d30d400c6905db539a4fce41fc420d02f77d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:45:27.317 INFO:journalctl@ceph.osd.1.smithi105.stdout:Jun 10 00:45:27 smithi105 podman[73581]: 2024-06-10 00:45:27.045381258 +0000 UTC m=+0.703724604 container died bee2ae9e3ab90d2f7ae6576b0be7d30d400c6905db539a4fce41fc420d02f77d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-10T00:45:27.719 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-10T00:45:27.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:45:27.756 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-10T00:45:27.756 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-10T00:45:27.757 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.2 2024-06-10T00:45:28.067 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:45:27 smithi138 systemd[1]: Stopping Ceph osd.2 for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:45:28.373 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:45:28 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-2[41668]: 2024-06-10T00:45:28.066+0000 7ffa8be77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:45:28.374 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:45:28 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-2[41668]: 2024-06-10T00:45:28.066+0000 7ffa8be77700 -1 osd.2 114 *** Got signal Terminated *** 2024-06-10T00:45:28.374 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:45:28 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-2[41668]: 2024-06-10T00:45:28.066+0000 7ffa8be77700 -1 osd.2 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T00:45:33.926 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:45:33 smithi138 podman[73563]: 2024-06-10 00:45:33.67052791 +0000 UTC m=+5.721501614 container died b2438d7ff1f1417abe31b23c1cc05cc427baabdec2135e5b9c7d47ef008a2f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-10T00:45:34.211 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:45:34 smithi138 podman[73563]: 2024-06-10 00:45:34.009392347 +0000 UTC m=+6.060366042 container cleanup b2438d7ff1f1417abe31b23c1cc05cc427baabdec2135e5b9c7d47ef008a2f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-2, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:45:34.211 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:45:34 smithi138 bash[73563]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-2 2024-06-10T00:45:34.623 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:45:34 smithi138 podman[73578]: 2024-06-10 00:45:34.21133773 +0000 UTC m=+0.541621206 container remove b2438d7ff1f1417abe31b23c1cc05cc427baabdec2135e5b9c7d47ef008a2f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-10T00:45:34.915 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:45:34 smithi138 podman[73668]: 2024-06-10 00:45:34.532659939 +0000 UTC m=+0.016879802 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:45:34.916 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:45:34 smithi138 podman[73668]: 2024-06-10 00:45:34.662936907 +0000 UTC m=+0.147156764 container create 68dca1e68f308e7fdc8ceacbef99a814ff9ece1aa3c5e012b78dd55cf053e449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-10T00:45:34.916 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:45:34 smithi138 podman[73668]: 2024-06-10 00:45:34.91611325 +0000 UTC m=+0.400333107 container init 68dca1e68f308e7fdc8ceacbef99a814ff9ece1aa3c5e012b78dd55cf053e449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-2-deactivate, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-10T00:45:35.212 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:45:34 smithi138 podman[73668]: 2024-06-10 00:45:34.919683631 +0000 UTC m=+0.403903497 container start 68dca1e68f308e7fdc8ceacbef99a814ff9ece1aa3c5e012b78dd55cf053e449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528) 2024-06-10T00:45:35.212 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:45:35 smithi138 podman[73668]: 2024-06-10 00:45:35.016489761 +0000 UTC m=+0.500709624 container attach 68dca1e68f308e7fdc8ceacbef99a814ff9ece1aa3c5e012b78dd55cf053e449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-10T00:45:35.488 INFO:journalctl@ceph.osd.2.smithi138.stdout:Jun 10 00:45:35 smithi138 podman[73668]: 2024-06-10 00:45:35.212576803 +0000 UTC m=+0.696796671 container died 68dca1e68f308e7fdc8ceacbef99a814ff9ece1aa3c5e012b78dd55cf053e449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:45:35.855 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-10T00:45:35.894 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:45:35.895 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-10T00:45:35.895 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-10T00:45:35.895 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.3 2024-06-10T00:45:36.217 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:35 smithi062 systemd[1]: Stopping Ceph osd.3 for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:45:36.497 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:36 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3[53986]: 2024-06-10T00:45:36.216+0000 7f081a717700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:45:36.498 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:36 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3[53986]: 2024-06-10T00:45:36.216+0000 7f081a717700 -1 osd.3 114 *** Got signal Terminated *** 2024-06-10T00:45:36.498 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:36 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3[53986]: 2024-06-10T00:45:36.216+0000 7f081a717700 -1 osd.3 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T00:45:41.942 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:41 smithi062 podman[122710]: 2024-06-10 00:45:41.631101087 +0000 UTC m=+5.532323169 container died a57c79574a14f2e5222604057e54330b7124b67fbd38150434926635007f35e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-10T00:45:42.358 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:42 smithi062 podman[122710]: 2024-06-10 00:45:42.141585047 +0000 UTC m=+6.042807170 container cleanup a57c79574a14f2e5222604057e54330b7124b67fbd38150434926635007f35e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-10T00:45:42.358 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:42 smithi062 bash[122710]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3 2024-06-10T00:45:42.796 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:42 smithi062 podman[122740]: 2024-06-10 00:45:42.359799925 +0000 UTC m=+0.731674236 container remove a57c79574a14f2e5222604057e54330b7124b67fbd38150434926635007f35e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-10T00:45:43.055 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:42 smithi062 podman[122830]: 2024-06-10 00:45:42.699288494 +0000 UTC m=+0.017950009 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:45:43.055 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:42 smithi062 podman[122830]: 2024-06-10 00:45:42.810959153 +0000 UTC m=+0.129620678 container create ad5b04a89e04955c6e091cf9349d118d438ee7f6fc3317f5e9ebffa903e1e3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-10T00:45:43.055 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:43 smithi062 podman[122830]: 2024-06-10 00:45:43.055213602 +0000 UTC m=+0.373875118 container init ad5b04a89e04955c6e091cf9349d118d438ee7f6fc3317f5e9ebffa903e1e3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-06-10T00:45:43.358 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:43 smithi062 podman[122830]: 2024-06-10 00:45:43.059006446 +0000 UTC m=+0.377667958 container start ad5b04a89e04955c6e091cf9349d118d438ee7f6fc3317f5e9ebffa903e1e3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-06-10T00:45:43.358 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:43 smithi062 podman[122830]: 2024-06-10 00:45:43.163422742 +0000 UTC m=+0.482084251 container attach ad5b04a89e04955c6e091cf9349d118d438ee7f6fc3317f5e9ebffa903e1e3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-10T00:45:43.658 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:43 smithi062 podman[122830]: 2024-06-10 00:45:43.366016744 +0000 UTC m=+0.684678259 container died ad5b04a89e04955c6e091cf9349d118d438ee7f6fc3317f5e9ebffa903e1e3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:45:43.912 INFO:journalctl@ceph.osd.3.smithi062.stdout:Jun 10 00:45:43 smithi062 podman[122856]: 2024-06-10 00:45:43.891845228 +0000 UTC m=+0.519443465 container remove ad5b04a89e04955c6e091cf9349d118d438ee7f6fc3317f5e9ebffa903e1e3c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True) 2024-06-10T00:45:43.917 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-10T00:45:43.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:45:43.958 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-10T00:45:43.959 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-10T00:45:43.959 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.4 2024-06-10T00:45:44.273 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:45:44 smithi105 systemd[1]: Stopping Ceph osd.4 for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:45:44.532 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:45:44 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-4[47821]: 2024-06-10T00:45:44.272+0000 7fb62eda8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:45:44.532 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:45:44 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-4[47821]: 2024-06-10T00:45:44.272+0000 7fb62eda8700 -1 osd.4 114 *** Got signal Terminated *** 2024-06-10T00:45:44.532 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:45:44 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-4[47821]: 2024-06-10T00:45:44.272+0000 7fb62eda8700 -1 osd.4 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T00:45:50.032 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:45:49 smithi105 podman[73710]: 2024-06-10 00:45:49.714047995 +0000 UTC m=+5.553988730 container died c474be70c4d1b6f6b3a4a3f312b383799a5b6882eacf4ebd7a7d731836da6dca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-4, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-10T00:45:50.482 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:45:50 smithi105 podman[73710]: 2024-06-10 00:45:50.164892947 +0000 UTC m=+6.004833670 container cleanup c474be70c4d1b6f6b3a4a3f312b383799a5b6882eacf4ebd7a7d731836da6dca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-4, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-10T00:45:50.483 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:45:50 smithi105 bash[73710]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-4 2024-06-10T00:45:50.782 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:45:50 smithi105 podman[73726]: 2024-06-10 00:45:50.483108894 +0000 UTC m=+0.770800624 container remove c474be70c4d1b6f6b3a4a3f312b383799a5b6882eacf4ebd7a7d731836da6dca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5) 2024-06-10T00:45:51.112 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:45:50 smithi105 podman[73814]: 2024-06-10 00:45:50.731818017 +0000 UTC m=+0.016833135 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:45:51.112 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:45:50 smithi105 podman[73814]: 2024-06-10 00:45:50.859359037 +0000 UTC m=+0.144374129 container create 433d109af88a38d89e63a23ce3a75682139f7db5aff7bc5cde9c1521b2f9f3d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:45:51.411 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:45:51 smithi105 podman[73814]: 2024-06-10 00:45:51.112702026 +0000 UTC m=+0.397717125 container init 433d109af88a38d89e63a23ce3a75682139f7db5aff7bc5cde9c1521b2f9f3d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-10T00:45:51.411 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:45:51 smithi105 podman[73814]: 2024-06-10 00:45:51.116012101 +0000 UTC m=+0.401027208 container start 433d109af88a38d89e63a23ce3a75682139f7db5aff7bc5cde9c1521b2f9f3d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:45:51.411 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:45:51 smithi105 podman[73814]: 2024-06-10 00:45:51.220669422 +0000 UTC m=+0.505684534 container attach 433d109af88a38d89e63a23ce3a75682139f7db5aff7bc5cde9c1521b2f9f3d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-10T00:45:51.773 INFO:journalctl@ceph.osd.4.smithi105.stdout:Jun 10 00:45:51 smithi105 podman[73814]: 2024-06-10 00:45:51.411748802 +0000 UTC m=+0.696763903 container died 433d109af88a38d89e63a23ce3a75682139f7db5aff7bc5cde9c1521b2f9f3d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-10T00:45:52.104 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-10T00:45:52.142 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:45:52.142 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-10T00:45:52.143 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-10T00:45:52.143 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.5 2024-06-10T00:45:52.472 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:45:52 smithi138 systemd[1]: Stopping Ceph osd.5 for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:45:52.873 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:45:52 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5[48299]: 2024-06-10T00:45:52.471+0000 7fd5b00b9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:45:52.874 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:45:52 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5[48299]: 2024-06-10T00:45:52.471+0000 7fd5b00b9700 -1 osd.5 114 *** Got signal Terminated *** 2024-06-10T00:45:52.874 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:45:52 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5[48299]: 2024-06-10T00:45:52.471+0000 7fd5b00b9700 -1 osd.5 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T00:45:58.363 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:45:58 smithi138 podman[73800]: 2024-06-10 00:45:58.064604373 +0000 UTC m=+5.725426255 container died e01cc1b8ed95a9e747b7c0a22153f1ca3fdd6e20f984006ee9ae69dd7ff5409f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-10T00:45:58.623 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:45:58 smithi138 podman[73800]: 2024-06-10 00:45:58.364216731 +0000 UTC m=+6.025038612 container cleanup e01cc1b8ed95a9e747b7c0a22153f1ca3fdd6e20f984006ee9ae69dd7ff5409f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-10T00:45:58.624 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:45:58 smithi138 bash[73800]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5 2024-06-10T00:45:58.624 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:45:58 smithi138 podman[73817]: 2024-06-10 00:45:58.607839877 +0000 UTC m=+0.544399198 container remove e01cc1b8ed95a9e747b7c0a22153f1ca3fdd6e20f984006ee9ae69dd7ff5409f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-10T00:45:59.261 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:45:58 smithi138 podman[73903]: 2024-06-10 00:45:58.834820954 +0000 UTC m=+0.017806898 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:45:59.262 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:45:58 smithi138 podman[73903]: 2024-06-10 00:45:58.967365797 +0000 UTC m=+0.150351736 container create 4150633e52847b2808563f5486528eb88a876c2eca93772c747a1d4f01ce2a51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-10T00:45:59.555 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:45:59 smithi138 podman[73903]: 2024-06-10 00:45:59.262434012 +0000 UTC m=+0.445419951 container init 4150633e52847b2808563f5486528eb88a876c2eca93772c747a1d4f01ce2a51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-10T00:45:59.555 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:45:59 smithi138 podman[73903]: 2024-06-10 00:45:59.265653817 +0000 UTC m=+0.448639758 container start 4150633e52847b2808563f5486528eb88a876c2eca93772c747a1d4f01ce2a51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-10T00:45:59.555 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:45:59 smithi138 podman[73903]: 2024-06-10 00:45:59.362683254 +0000 UTC m=+0.545669194 container attach 4150633e52847b2808563f5486528eb88a876c2eca93772c747a1d4f01ce2a51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5-deactivate, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-10T00:45:59.873 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:45:59 smithi138 podman[73903]: 2024-06-10 00:45:59.555293576 +0000 UTC m=+0.738279515 container died 4150633e52847b2808563f5486528eb88a876c2eca93772c747a1d4f01ce2a51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-10T00:46:00.168 INFO:journalctl@ceph.osd.5.smithi138.stdout:Jun 10 00:46:00 smithi138 podman[73929]: 2024-06-10 00:46:00.150896838 +0000 UTC m=+0.589492327 container remove 4150633e52847b2808563f5486528eb88a876c2eca93772c747a1d4f01ce2a51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-10T00:46:00.173 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-10T00:46:00.211 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:46:00.211 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-10T00:46:00.211 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-10T00:46:00.212 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.6 2024-06-10T00:46:00.511 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:46:00 smithi062 systemd[1]: Stopping Ceph osd.6 for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:46:00.858 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:46:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-6[60913]: 2024-06-10T00:46:00.510+0000 7f7d19684700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:46:00.858 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:46:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-6[60913]: 2024-06-10T00:46:00.510+0000 7f7d19684700 -1 osd.6 114 *** Got signal Terminated *** 2024-06-10T00:46:00.858 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:46:00 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-6[60913]: 2024-06-10T00:46:00.510+0000 7f7d19684700 -1 osd.6 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T00:46:06.328 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:46:06 smithi062 podman[122937]: 2024-06-10 00:46:06.03971843 +0000 UTC m=+5.627491963 container died 22fcd50050644fbcd36f812899adbc003e5bc39731fb110ffcca74eccfaeabcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-10T00:46:06.609 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:46:06 smithi062 podman[122937]: 2024-06-10 00:46:06.527355123 +0000 UTC m=+6.115128655 container cleanup 22fcd50050644fbcd36f812899adbc003e5bc39731fb110ffcca74eccfaeabcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:46:06.610 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:46:06 smithi062 bash[122937]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-6 2024-06-10T00:46:07.108 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:46:06 smithi062 podman[122953]: 2024-06-10 00:46:06.787070939 +0000 UTC m=+0.746476278 container remove 22fcd50050644fbcd36f812899adbc003e5bc39731fb110ffcca74eccfaeabcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:46:07.423 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:46:07 smithi062 podman[123042]: 2024-06-10 00:46:07.064364887 +0000 UTC m=+0.017513722 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:46:07.424 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:46:07 smithi062 podman[123042]: 2024-06-10 00:46:07.179739332 +0000 UTC m=+0.132888162 container create db269da93589ad8bdc0e8bdad3aebdc860758c307e0eb69ae1b54899e3dd0c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-10T00:46:07.424 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:46:07 smithi062 podman[123042]: 2024-06-10 00:46:07.4239906 +0000 UTC m=+0.377139422 container init db269da93589ad8bdc0e8bdad3aebdc860758c307e0eb69ae1b54899e3dd0c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5) 2024-06-10T00:46:07.728 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:46:07 smithi062 podman[123042]: 2024-06-10 00:46:07.427580577 +0000 UTC m=+0.380729416 container start db269da93589ad8bdc0e8bdad3aebdc860758c307e0eb69ae1b54899e3dd0c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-06-10T00:46:07.729 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:46:07 smithi062 podman[123042]: 2024-06-10 00:46:07.515746962 +0000 UTC m=+0.468895803 container attach db269da93589ad8bdc0e8bdad3aebdc860758c307e0eb69ae1b54899e3dd0c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-10T00:46:08.037 INFO:journalctl@ceph.osd.6.smithi062.stdout:Jun 10 00:46:07 smithi062 podman[123042]: 2024-06-10 00:46:07.729434658 +0000 UTC m=+0.682583499 container died db269da93589ad8bdc0e8bdad3aebdc860758c307e0eb69ae1b54899e3dd0c95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-10T00:46:08.357 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-10T00:46:08.398 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:46:08.399 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-10T00:46:08.399 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-10T00:46:08.399 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.7 2024-06-10T00:46:08.719 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:46:08 smithi105 systemd[1]: Stopping Ceph osd.7 for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:46:08.979 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:46:08 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-7[54492]: 2024-06-10T00:46:08.718+0000 7f7b69ff9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:46:08.980 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:46:08 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-7[54492]: 2024-06-10T00:46:08.718+0000 7f7b69ff9700 -1 osd.7 114 *** Got signal Terminated *** 2024-06-10T00:46:08.980 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:46:08 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-7[54492]: 2024-06-10T00:46:08.718+0000 7f7b69ff9700 -1 osd.7 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T00:46:14.486 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:46:14 smithi105 podman[73920]: 2024-06-10 00:46:14.196753665 +0000 UTC m=+5.602008249 container died 4581264367e07e9d93520b38b1d232804ec0a47a2cd49d05eed6cce83a015220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-10T00:46:14.782 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:46:14 smithi105 podman[73920]: 2024-06-10 00:46:14.585621527 +0000 UTC m=+5.990876113 container cleanup 4581264367e07e9d93520b38b1d232804ec0a47a2cd49d05eed6cce83a015220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-10T00:46:14.782 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:46:14 smithi105 bash[73920]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-7 2024-06-10T00:46:15.210 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:46:14 smithi105 podman[73946]: 2024-06-10 00:46:14.887149692 +0000 UTC m=+0.685829634 container remove 4581264367e07e9d93520b38b1d232804ec0a47a2cd49d05eed6cce83a015220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-7, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:46:15.499 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:46:15 smithi105 podman[74037]: 2024-06-10 00:46:15.113345152 +0000 UTC m=+0.017267293 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:46:15.499 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:46:15 smithi105 podman[74037]: 2024-06-10 00:46:15.238078599 +0000 UTC m=+0.142000757 container create c73e631ecd261cb531123ec4ea6e0672921297dfd8bbed206c0c7adf432e87c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-10T00:46:15.499 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:46:15 smithi105 podman[74037]: 2024-06-10 00:46:15.499261334 +0000 UTC m=+0.403183482 container init c73e631ecd261cb531123ec4ea6e0672921297dfd8bbed206c0c7adf432e87c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-06-10T00:46:15.782 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:46:15 smithi105 podman[74037]: 2024-06-10 00:46:15.502596559 +0000 UTC m=+0.406518698 container start c73e631ecd261cb531123ec4ea6e0672921297dfd8bbed206c0c7adf432e87c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-10T00:46:15.782 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:46:15 smithi105 podman[74037]: 2024-06-10 00:46:15.573961462 +0000 UTC m=+0.477883607 container attach c73e631ecd261cb531123ec4ea6e0672921297dfd8bbed206c0c7adf432e87c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-10T00:46:16.085 INFO:journalctl@ceph.osd.7.smithi105.stdout:Jun 10 00:46:15 smithi105 podman[74037]: 2024-06-10 00:46:15.791841572 +0000 UTC m=+0.695763708 container died c73e631ecd261cb531123ec4ea6e0672921297dfd8bbed206c0c7adf432e87c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-06-10T00:46:16.440 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-10T00:46:16.480 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:46:16.481 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-10T00:46:16.481 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-10T00:46:16.481 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.8 2024-06-10T00:46:16.778 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:46:16 smithi138 systemd[1]: Stopping Ceph osd.8 for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:46:17.039 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:46:16 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-8[54827]: 2024-06-10T00:46:16.777+0000 7ff0fa7d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:46:17.039 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:46:16 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-8[54827]: 2024-06-10T00:46:16.777+0000 7ff0fa7d8700 -1 osd.8 114 *** Got signal Terminated *** 2024-06-10T00:46:17.039 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:46:16 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-8[54827]: 2024-06-10T00:46:16.777+0000 7ff0fa7d8700 -1 osd.8 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T00:46:18.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:18 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:18.494+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:19.108 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:18 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:18.621+0000 7ff0914fd700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:16.610716+0000 front 2024-06-10T00:45:16.610685+0000 (oldest deadline 2024-06-10T00:46:18.310787+0000) 2024-06-10T00:46:19.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:19 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:19.513+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:19.858 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:19 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:19.573+0000 7ff0914fd700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:16.610716+0000 front 2024-06-10T00:45:16.610685+0000 (oldest deadline 2024-06-10T00:46:18.310787+0000) 2024-06-10T00:46:20.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:20 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:20.515+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:20.858 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:20 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:20.606+0000 7ff0914fd700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:16.610716+0000 front 2024-06-10T00:45:16.610685+0000 (oldest deadline 2024-06-10T00:46:18.310787+0000) 2024-06-10T00:46:22.032 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:21 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:21.533+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:22.108 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:21 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:21.618+0000 7ff0914fd700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:16.610716+0000 front 2024-06-10T00:45:16.610685+0000 (oldest deadline 2024-06-10T00:46:18.310787+0000) 2024-06-10T00:46:22.330 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:21 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:21.924+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:22.580 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:46:22 smithi138 podman[74007]: 2024-06-10 00:46:22.330750793 +0000 UTC m=+5.655727220 container died a50e1e94001fcca40e937cc0aa5156f6ab872ff35fbf1b1e53c3e9c7ffe524fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-8, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-10T00:46:22.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:22 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:22.524+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:22.858 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:22 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:22.604+0000 7ff0914fd700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:16.610716+0000 front 2024-06-10T00:45:16.610685+0000 (oldest deadline 2024-06-10T00:46:18.310787+0000) 2024-06-10T00:46:22.873 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:46:22 smithi138 podman[74007]: 2024-06-10 00:46:22.670558156 +0000 UTC m=+5.995534576 container cleanup a50e1e94001fcca40e937cc0aa5156f6ab872ff35fbf1b1e53c3e9c7ffe524fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:46:22.873 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:46:22 smithi138 bash[74007]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-8 2024-06-10T00:46:23.299 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:22 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:22.937+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:23.300 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:46:22 smithi138 podman[74034]: 2024-06-10 00:46:22.888935996 +0000 UTC m=+0.555521796 container remove a50e1e94001fcca40e937cc0aa5156f6ab872ff35fbf1b1e53c3e9c7ffe524fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528) 2024-06-10T00:46:23.609 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:46:23 smithi138 podman[74123]: 2024-06-10 00:46:23.20239446 +0000 UTC m=+0.017636854 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:46:23.610 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:46:23 smithi138 podman[74123]: 2024-06-10 00:46:23.332211532 +0000 UTC m=+0.147453965 container create 4c91434b568bece804297cde6f798848b3f78b4e42b8ab200a01133d7faf5ed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:46:23.610 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:46:23 smithi138 podman[74123]: 2024-06-10 00:46:23.535443645 +0000 UTC m=+0.350686052 container init 4c91434b568bece804297cde6f798848b3f78b4e42b8ab200a01133d7faf5ed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-10T00:46:23.610 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:46:23 smithi138 podman[74123]: 2024-06-10 00:46:23.538686736 +0000 UTC m=+0.353929132 container start 4c91434b568bece804297cde6f798848b3f78b4e42b8ab200a01133d7faf5ed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-8-deactivate, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-10T00:46:23.610 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:46:23 smithi138 podman[74123]: 2024-06-10 00:46:23.610196978 +0000 UTC m=+0.425439370 container attach 4c91434b568bece804297cde6f798848b3f78b4e42b8ab200a01133d7faf5ed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-06-10T00:46:23.781 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:23 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:23.489+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:23.858 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:23 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:23.594+0000 7ff0914fd700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:16.610716+0000 front 2024-06-10T00:45:16.610685+0000 (oldest deadline 2024-06-10T00:46:18.310787+0000) 2024-06-10T00:46:23.874 INFO:journalctl@ceph.osd.8.smithi138.stdout:Jun 10 00:46:23 smithi138 podman[74123]: 2024-06-10 00:46:23.83973301 +0000 UTC m=+0.654975402 container died 4c91434b568bece804297cde6f798848b3f78b4e42b8ab200a01133d7faf5ed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:46:24.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:23 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:23.935+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:24.435 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.8.service' 2024-06-10T00:46:24.475 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:46:24.475 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-10T00:46:24.475 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-10T00:46:24.475 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.9 2024-06-10T00:46:24.780 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:24 smithi062 systemd[1]: Stopping Ceph osd.9 for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:46:24.780 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:24 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:24.617+0000 7ff0914fd700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:16.610716+0000 front 2024-06-10T00:45:16.610685+0000 (oldest deadline 2024-06-10T00:46:18.310787+0000) 2024-06-10T00:46:24.780 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:24 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:24.779+0000 7ff094f09700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:46:24.780 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:24 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:24.779+0000 7ff094f09700 -1 osd.9 114 *** Got signal Terminated *** 2024-06-10T00:46:24.781 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:24 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:24.779+0000 7ff094f09700 -1 osd.9 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T00:46:24.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:24 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:24.476+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:25.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:24 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:24.958+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:25.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:25 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:25.502+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:25.858 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:25 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:25.597+0000 7ff0914fd700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:16.610716+0000 front 2024-06-10T00:45:16.610685+0000 (oldest deadline 2024-06-10T00:46:18.310787+0000) 2024-06-10T00:46:26.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:25 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:25.963+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:26.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:26 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:26.542+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:27.108 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:26 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:26.631+0000 7ff0914fd700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:16.610716+0000 front 2024-06-10T00:45:16.610685+0000 (oldest deadline 2024-06-10T00:46:18.310787+0000) 2024-06-10T00:46:27.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:26 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:26.959+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:27.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:26 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:26.959+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:27.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:27 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:27.520+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:27.858 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:27 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:27.585+0000 7ff0914fd700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:16.610716+0000 front 2024-06-10T00:45:16.610685+0000 (oldest deadline 2024-06-10T00:46:18.310787+0000) 2024-06-10T00:46:28.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:27 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:27.940+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:28.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:27 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:27.940+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:28.858 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:28 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:28.603+0000 7ff0914fd700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:16.610716+0000 front 2024-06-10T00:45:16.610685+0000 (oldest deadline 2024-06-10T00:46:18.310787+0000) 2024-06-10T00:46:28.858 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:28 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:28.603+0000 7ff0914fd700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:24.712575+0000 front 2024-06-10T00:45:24.712698+0000 (oldest deadline 2024-06-10T00:46:27.612738+0000) 2024-06-10T00:46:29.032 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:28 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:28.562+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:29.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:28 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:28.956+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:29.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:28 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:28.956+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:29.858 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:29 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:29.606+0000 7ff0914fd700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:16.610716+0000 front 2024-06-10T00:45:16.610685+0000 (oldest deadline 2024-06-10T00:46:18.310787+0000) 2024-06-10T00:46:29.858 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:29 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9[67469]: 2024-06-10T00:46:29.606+0000 7ff0914fd700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:24.712575+0000 front 2024-06-10T00:45:24.712698+0000 (oldest deadline 2024-06-10T00:46:27.612738+0000) 2024-06-10T00:46:30.032 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:29 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:29.575+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:30.032 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:29 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:29.575+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:22.904370+0000 front 2024-06-10T00:45:22.904396+0000 (oldest deadline 2024-06-10T00:46:28.804562+0000) 2024-06-10T00:46:30.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:29 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:29.993+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:30.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:29 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:29.993+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:30.608 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:30 smithi062 podman[123160]: 2024-06-10 00:46:30.324131517 +0000 UTC m=+5.645614260 container died 4959bb40150669ddff23f8d0c85afa629f7dc5552e06d7a3d5bea4f3ec2dba25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-10T00:46:30.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:30 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:30.529+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:30.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:30 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:30.529+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:22.904370+0000 front 2024-06-10T00:45:22.904396+0000 (oldest deadline 2024-06-10T00:46:28.804562+0000) 2024-06-10T00:46:31.108 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:30 smithi062 podman[123160]: 2024-06-10 00:46:30.737784482 +0000 UTC m=+6.059267241 container cleanup 4959bb40150669ddff23f8d0c85afa629f7dc5552e06d7a3d5bea4f3ec2dba25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-10T00:46:31.108 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:30 smithi062 bash[123160]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9 2024-06-10T00:46:31.108 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:30 smithi062 podman[123176]: 2024-06-10 00:46:30.913937943 +0000 UTC m=+0.591283739 container remove 4959bb40150669ddff23f8d0c85afa629f7dc5552e06d7a3d5bea4f3ec2dba25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True) 2024-06-10T00:46:31.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:31 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:31.011+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:31.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:31 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:31.011+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:31.691 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:31 smithi062 podman[123269]: 2024-06-10 00:46:31.272589823 +0000 UTC m=+0.017602063 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:46:31.691 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:31 smithi062 podman[123269]: 2024-06-10 00:46:31.37299328 +0000 UTC m=+0.118005499 container create 026ce4191498d54634ca590ac07bc2736339f3a86a9d7cf15046c277e26a3c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True) 2024-06-10T00:46:31.692 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:31 smithi062 podman[123269]: 2024-06-10 00:46:31.608848025 +0000 UTC m=+0.353860254 container init 026ce4191498d54634ca590ac07bc2736339f3a86a9d7cf15046c277e26a3c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:46:31.692 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:31 smithi062 podman[123269]: 2024-06-10 00:46:31.612446301 +0000 UTC m=+0.357458525 container start 026ce4191498d54634ca590ac07bc2736339f3a86a9d7cf15046c277e26a3c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:46:31.692 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:31 smithi062 podman[123269]: 2024-06-10 00:46:31.691734236 +0000 UTC m=+0.436746467 container attach 026ce4191498d54634ca590ac07bc2736339f3a86a9d7cf15046c277e26a3c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-10T00:46:31.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:31 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:31.504+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:31.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:31 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:31.504+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:22.904370+0000 front 2024-06-10T00:45:22.904396+0000 (oldest deadline 2024-06-10T00:46:28.804562+0000) 2024-06-10T00:46:32.108 INFO:journalctl@ceph.osd.9.smithi062.stdout:Jun 10 00:46:31 smithi062 podman[123269]: 2024-06-10 00:46:31.919363759 +0000 UTC m=+0.664375989 container died 026ce4191498d54634ca590ac07bc2736339f3a86a9d7cf15046c277e26a3c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:46:32.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:31 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:31.986+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:32.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:31 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:31.986+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:32.551 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.9.service' 2024-06-10T00:46:32.589 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:46:32.589 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-10T00:46:32.590 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-10T00:46:32.590 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.10 2024-06-10T00:46:32.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:32 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:32.465+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:32.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:32 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:32.465+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:22.904370+0000 front 2024-06-10T00:45:22.904396+0000 (oldest deadline 2024-06-10T00:46:28.804562+0000) 2024-06-10T00:46:32.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:32 smithi105 systemd[1]: Stopping Ceph osd.10 for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:46:33.282 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:32 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:32.896+0000 7f1a16a2b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:46:33.282 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:32 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:32.896+0000 7f1a16a2b700 -1 osd.10 114 *** Got signal Terminated *** 2024-06-10T00:46:33.282 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:32 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:32.896+0000 7f1a16a2b700 -1 osd.10 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T00:46:33.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:32 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:32.989+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:33.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:32 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:32.989+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:33.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:33.447+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:33.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:33 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:33.447+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:22.904370+0000 front 2024-06-10T00:45:22.904396+0000 (oldest deadline 2024-06-10T00:46:28.804562+0000) 2024-06-10T00:46:34.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:33.952+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:34.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:33 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:33.952+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:34.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:34 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:34.469+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:34.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:34 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:34.469+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:22.904370+0000 front 2024-06-10T00:45:22.904396+0000 (oldest deadline 2024-06-10T00:46:28.804562+0000) 2024-06-10T00:46:35.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:34 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:34.916+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:35.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:34 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:34.916+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:35.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:35 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:35.461+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:35.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:35 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:35.461+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:22.904370+0000 front 2024-06-10T00:45:22.904396+0000 (oldest deadline 2024-06-10T00:46:28.804562+0000) 2024-06-10T00:46:36.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:35 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:35.966+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:36.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:35 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:35.966+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:36.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:36 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:36.434+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:36.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:36 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:36.434+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:22.904370+0000 front 2024-06-10T00:45:22.904396+0000 (oldest deadline 2024-06-10T00:46:28.804562+0000) 2024-06-10T00:46:37.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:36 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:36.958+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:37.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:36 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:36.958+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:37.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:37 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:37.463+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:13.503269+0000 front 2024-06-10T00:45:13.503232+0000 (oldest deadline 2024-06-10T00:46:17.603193+0000) 2024-06-10T00:46:37.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:37 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:37.463+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:22.904370+0000 front 2024-06-10T00:45:22.904396+0000 (oldest deadline 2024-06-10T00:46:28.804562+0000) 2024-06-10T00:46:37.782 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:37 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10[61361]: 2024-06-10T00:46:37.463+0000 7f1a1301f700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-06-10T00:45:29.305618+0000 front 2024-06-10T00:45:29.305804+0000 (oldest deadline 2024-06-10T00:46:36.906102+0000) 2024-06-10T00:46:38.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:37 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:37.960+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:38.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:37 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:37.960+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:38.573 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:38 smithi105 podman[74142]: 2024-06-10 00:46:38.305377345 +0000 UTC m=+5.519170641 container died 0eae061b023885f7361b51e16758c30a9cc98085074bcc98268029dbaf27ed34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-10T00:46:38.948 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:38 smithi105 podman[74142]: 2024-06-10 00:46:38.68060293 +0000 UTC m=+5.894396177 container cleanup 0eae061b023885f7361b51e16758c30a9cc98085074bcc98268029dbaf27ed34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:46:38.948 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:38 smithi105 bash[74142]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10 2024-06-10T00:46:39.282 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:38 smithi105 podman[74160]: 2024-06-10 00:46:38.948495663 +0000 UTC m=+0.635281306 container remove 0eae061b023885f7361b51e16758c30a9cc98085074bcc98268029dbaf27ed34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-10T00:46:39.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:38 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:38.913+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:39.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:38 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:38.913+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:39.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:38 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:38.913+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-06-10T00:45:32.965441+0000 front 2024-06-10T00:45:32.965415+0000 (oldest deadline 2024-06-10T00:46:38.865614+0000) 2024-06-10T00:46:39.576 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:39 smithi105 podman[74248]: 2024-06-10 00:46:39.210931332 +0000 UTC m=+0.016782067 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:46:39.576 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:39 smithi105 podman[74248]: 2024-06-10 00:46:39.324124096 +0000 UTC m=+0.129974820 container create 96a355d07a3f926bbff4c1e0fd006bf29a117c148b2b8e179e96fc435870553f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-10T00:46:39.576 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:39 smithi105 podman[74248]: 2024-06-10 00:46:39.576473599 +0000 UTC m=+0.382324317 container init 96a355d07a3f926bbff4c1e0fd006bf29a117c148b2b8e179e96fc435870553f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-06-10T00:46:39.869 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:39 smithi105 podman[74248]: 2024-06-10 00:46:39.579974856 +0000 UTC m=+0.385825589 container start 96a355d07a3f926bbff4c1e0fd006bf29a117c148b2b8e179e96fc435870553f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-10T00:46:39.870 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:39 smithi105 podman[74248]: 2024-06-10 00:46:39.659765117 +0000 UTC m=+0.465615859 container attach 96a355d07a3f926bbff4c1e0fd006bf29a117c148b2b8e179e96fc435870553f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-10T00:46:40.162 INFO:journalctl@ceph.osd.10.smithi105.stdout:Jun 10 00:46:39 smithi105 podman[74248]: 2024-06-10 00:46:39.870711102 +0000 UTC m=+0.676561832 container died 96a355d07a3f926bbff4c1e0fd006bf29a117c148b2b8e179e96fc435870553f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-10-deactivate, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-10T00:46:40.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:39 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:39.944+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:40.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:39 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:39.944+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:40.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:39 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:39.944+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-06-10T00:45:32.965441+0000 front 2024-06-10T00:45:32.965415+0000 (oldest deadline 2024-06-10T00:46:38.865614+0000) 2024-06-10T00:46:40.439 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.10.service' 2024-06-10T00:46:40.476 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:46:40.476 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-10T00:46:40.476 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-10T00:46:40.477 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.11 2024-06-10T00:46:40.785 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:40 smithi138 systemd[1]: Stopping Ceph osd.11 for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:46:41.123 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:40 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:40.784+0000 7f758d740700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:46:41.124 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:40 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:40.784+0000 7f758d740700 -1 osd.11 114 *** Got signal Terminated *** 2024-06-10T00:46:41.124 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:40 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:40.784+0000 7f758d740700 -1 osd.11 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T00:46:41.124 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:40 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:40.954+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:41.124 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:40 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:40.954+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:41.124 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:40 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:40.954+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-06-10T00:45:32.965441+0000 front 2024-06-10T00:45:32.965415+0000 (oldest deadline 2024-06-10T00:46:38.865614+0000) 2024-06-10T00:46:42.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:41 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:41.948+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:42.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:41 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:41.948+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:42.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:41 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:41.948+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-06-10T00:45:32.965441+0000 front 2024-06-10T00:45:32.965415+0000 (oldest deadline 2024-06-10T00:46:38.865614+0000) 2024-06-10T00:46:43.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:42 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:42.975+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:43.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:42 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:42.975+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:43.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:42 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:42.975+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-06-10T00:45:32.965441+0000 front 2024-06-10T00:45:32.965415+0000 (oldest deadline 2024-06-10T00:46:38.865614+0000) 2024-06-10T00:46:44.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:44 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:44.003+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:44.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:44 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:44.003+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:44.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:44 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:44.003+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-06-10T00:45:32.965441+0000 front 2024-06-10T00:45:32.965415+0000 (oldest deadline 2024-06-10T00:46:38.865614+0000) 2024-06-10T00:46:45.373 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:44 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:44.958+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-06-10T00:45:15.661125+0000 front 2024-06-10T00:45:15.661189+0000 (oldest deadline 2024-06-10T00:46:20.961324+0000) 2024-06-10T00:46:45.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:44 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:44.958+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-06-10T00:45:23.163196+0000 front 2024-06-10T00:45:23.163008+0000 (oldest deadline 2024-06-10T00:46:26.563837+0000) 2024-06-10T00:46:45.374 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:44 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11[61432]: 2024-06-10T00:46:44.958+0000 7f7585d2c700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-06-10T00:45:32.965441+0000 front 2024-06-10T00:45:32.965415+0000 (oldest deadline 2024-06-10T00:46:38.865614+0000) 2024-06-10T00:46:46.535 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:46 smithi138 podman[74230]: 2024-06-10 00:46:46.25951051 +0000 UTC m=+5.589403169 container died b4da41759dae26e00713bb5f87831449a0d1f940c2ae86b55a0c393950f5b297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-10T00:46:46.873 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:46 smithi138 podman[74230]: 2024-06-10 00:46:46.63503284 +0000 UTC m=+5.964925492 container cleanup b4da41759dae26e00713bb5f87831449a0d1f940c2ae86b55a0c393950f5b297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-10T00:46:46.874 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:46 smithi138 bash[74230]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11 2024-06-10T00:46:47.212 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:46 smithi138 podman[74248]: 2024-06-10 00:46:46.878194521 +0000 UTC m=+0.609926789 container remove b4da41759dae26e00713bb5f87831449a0d1f940c2ae86b55a0c393950f5b297 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-10T00:46:47.506 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:47 smithi138 podman[74338]: 2024-06-10 00:46:47.115577022 +0000 UTC m=+0.017310745 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:46:47.507 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:47 smithi138 podman[74338]: 2024-06-10 00:46:47.221048979 +0000 UTC m=+0.122782728 container create 3214eb875cc50c1f1c103bbd3448d4a5a8abbcd6c9f013ec4fed4a647a6d99af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-10T00:46:47.507 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:47 smithi138 podman[74338]: 2024-06-10 00:46:47.448742429 +0000 UTC m=+0.350476133 container init 3214eb875cc50c1f1c103bbd3448d4a5a8abbcd6c9f013ec4fed4a647a6d99af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-10T00:46:47.507 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:47 smithi138 podman[74338]: 2024-06-10 00:46:47.451692186 +0000 UTC m=+0.353425893 container start 3214eb875cc50c1f1c103bbd3448d4a5a8abbcd6c9f013ec4fed4a647a6d99af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:46:47.507 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:47 smithi138 podman[74338]: 2024-06-10 00:46:47.507203723 +0000 UTC m=+0.408937432 container attach 3214eb875cc50c1f1c103bbd3448d4a5a8abbcd6c9f013ec4fed4a647a6d99af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True) 2024-06-10T00:46:47.874 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:47 smithi138 podman[74338]: 2024-06-10 00:46:47.736337634 +0000 UTC m=+0.638071339 container died 3214eb875cc50c1f1c103bbd3448d4a5a8abbcd6c9f013ec4fed4a647a6d99af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:46:48.238 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.11.service' 2024-06-10T00:46:48.261 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:48 smithi138 podman[74364]: 2024-06-10 00:46:48.219455665 +0000 UTC m=+0.477814480 container remove 3214eb875cc50c1f1c103bbd3448d4a5a8abbcd6c9f013ec4fed4a647a6d99af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-10T00:46:48.261 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:48 smithi138 systemd[1]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.11.service: Deactivated successfully. 2024-06-10T00:46:48.261 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:48 smithi138 systemd[1]: Stopped Ceph osd.11 for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:46:48.261 INFO:journalctl@ceph.osd.11.smithi138.stdout:Jun 10 00:46:48 smithi138 systemd[1]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@osd.11.service: Consumed 2min 37.371s CPU time. 2024-06-10T00:46:48.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:46:48.273 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-10T00:46:48.273 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-10T00:46:48.274 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.a 2024-06-10T00:46:48.572 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:46:48 smithi062 systemd[1]: Stopping Ceph mds.a for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:46:48.844 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:46:48 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-a[74274]: 2024-06-10T00:46:48.571+0000 7f24e800f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:46:48.844 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:46:48 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-a[74274]: 2024-06-10T00:46:48.571+0000 7f24e800f700 -1 mds.a *** got signal Terminated *** 2024-06-10T00:46:52.858 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:46:52 smithi062 podman[123375]: 2024-06-10 00:46:52.585116246 +0000 UTC m=+4.105876752 container died 9267421457af753c655b89d23a1df80c609fc2666c6f753d1193c03289a32869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:46:53.247 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:46:52 smithi062 podman[123375]: 2024-06-10 00:46:52.9465059 +0000 UTC m=+4.467266405 container cleanup 9267421457af753c655b89d23a1df80c609fc2666c6f753d1193c03289a32869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-10T00:46:53.272 INFO:journalctl@ceph.mds.a.smithi062.stdout:Jun 10 00:46:52 smithi062 bash[123375]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-a 2024-06-10T00:46:53.380 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.a.service' 2024-06-10T00:46:53.417 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:46:53.417 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-10T00:46:53.418 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-10T00:46:53.418 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.d 2024-06-10T00:46:53.608 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jun 10 00:46:53 smithi062 systemd[1]: Stopping Ceph mds.d for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:46:54.108 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jun 10 00:46:53 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-d[72991]: 2024-06-10T00:46:53.741+0000 7f54efc95700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:46:54.108 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jun 10 00:46:53 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-d[72991]: 2024-06-10T00:46:53.741+0000 7f54efc95700 -1 mds.d *** got signal Terminated *** 2024-06-10T00:46:57.108 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jun 10 00:46:56 smithi062 podman[123508]: 2024-06-10 00:46:56.843642706 +0000 UTC m=+3.201979232 container died 6b93b2f244ccd884db725276fbe824c217ab39cfa0ff1ce83e6ec2e92bd4c9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:46:57.484 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jun 10 00:46:57 smithi062 podman[123508]: 2024-06-10 00:46:57.199714396 +0000 UTC m=+3.558050919 container cleanup 6b93b2f244ccd884db725276fbe824c217ab39cfa0ff1ce83e6ec2e92bd4c9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-d, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-10T00:46:57.484 INFO:journalctl@ceph.mds.d.smithi062.stdout:Jun 10 00:46:57 smithi062 bash[123508]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-d 2024-06-10T00:46:57.642 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.d.service' 2024-06-10T00:46:57.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:46:57.678 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-10T00:46:57.678 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-10T00:46:57.679 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.g 2024-06-10T00:46:58.004 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:46:57 smithi062 systemd[1]: Stopping Ceph mds.g for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:46:58.358 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:46:58 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-g[73816]: 2024-06-10T00:46:58.003+0000 7f1d486dd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:46:58.358 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:46:58 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-g[73816]: 2024-06-10T00:46:58.003+0000 7f1d486dd700 -1 mds.g *** got signal Terminated *** 2024-06-10T00:47:01.687 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:47:01 smithi062 podman[123630]: 2024-06-10 00:47:01.400391527 +0000 UTC m=+3.499868870 container died 55f780cf7a1159b766eff059db86e38b85f874034c874817bd49ec9ca92af0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:47:02.021 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:47:01 smithi062 podman[123630]: 2024-06-10 00:47:01.778716085 +0000 UTC m=+3.878193410 container cleanup 55f780cf7a1159b766eff059db86e38b85f874034c874817bd49ec9ca92af0e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-06-10T00:47:02.021 INFO:journalctl@ceph.mds.g.smithi062.stdout:Jun 10 00:47:01 smithi062 bash[123630]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-g 2024-06-10T00:47:02.158 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.g.service' 2024-06-10T00:47:02.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:47:02.192 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-10T00:47:02.192 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-10T00:47:02.193 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.j 2024-06-10T00:47:02.532 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:47:02 smithi062 systemd[1]: Stopping Ceph mds.j for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:47:02.858 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:47:02 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-j[72571]: 2024-06-10T00:47:02.532+0000 7f7a1ba90700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:47:02.858 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:47:02 smithi062 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-j[72571]: 2024-06-10T00:47:02.532+0000 7f7a1ba90700 -1 mds.j *** got signal Terminated *** 2024-06-10T00:47:04.933 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:47:04 smithi062 podman[123751]: 2024-06-10 00:47:04.655608221 +0000 UTC m=+2.236262558 container died 06963e5804a5081e43c84e7af4e149132772990ba585ee5a5486d81c9746283d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-j, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5) 2024-06-10T00:47:05.207 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:47:04 smithi062 podman[123751]: 2024-06-10 00:47:04.998557921 +0000 UTC m=+2.579212262 container cleanup 06963e5804a5081e43c84e7af4e149132772990ba585ee5a5486d81c9746283d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-j, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-10T00:47:05.207 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:47:04 smithi062 bash[123751]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-j 2024-06-10T00:47:05.440 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.j.service' 2024-06-10T00:47:05.462 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:47:05 smithi062 podman[123763]: 2024-06-10 00:47:05.207738394 +0000 UTC m=+0.550972264 container remove 06963e5804a5081e43c84e7af4e149132772990ba585ee5a5486d81c9746283d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc) 2024-06-10T00:47:05.463 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:47:05 smithi062 systemd[1]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.j.service: Deactivated successfully. 2024-06-10T00:47:05.463 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:47:05 smithi062 systemd[1]: Stopped Ceph mds.j for 1353b2ce-26bf-11ef-bc9b-c7b262605968. 2024-06-10T00:47:05.463 INFO:journalctl@ceph.mds.j.smithi062.stdout:Jun 10 00:47:05 smithi062 systemd[1]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.j.service: Consumed 1.147s CPU time. 2024-06-10T00:47:05.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:47:05.473 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-10T00:47:05.474 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-10T00:47:05.474 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.b 2024-06-10T00:47:05.781 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:47:05 smithi105 systemd[1]: Stopping Ceph mds.b for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:47:06.032 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:47:05 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-b[66851]: 2024-06-10T00:47:05.780+0000 7fac018a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:47:06.032 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:47:05 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-b[66851]: 2024-06-10T00:47:05.780+0000 7fac018a4700 -1 mds.b *** got signal Terminated *** 2024-06-10T00:47:09.690 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:47:09 smithi105 podman[74366]: 2024-06-10 00:47:09.398357328 +0000 UTC m=+3.718973224 container died c7fd797fca0ba5e2f012820206794d56a93abe76ccea2ba06a382446f056ef6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:47:10.032 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:47:09 smithi105 podman[74366]: 2024-06-10 00:47:09.79796956 +0000 UTC m=+4.118585454 container cleanup c7fd797fca0ba5e2f012820206794d56a93abe76ccea2ba06a382446f056ef6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-10T00:47:10.032 INFO:journalctl@ceph.mds.b.smithi105.stdout:Jun 10 00:47:09 smithi105 bash[74366]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-b 2024-06-10T00:47:10.183 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.b.service' 2024-06-10T00:47:10.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:47:10.216 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-10T00:47:10.216 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-10T00:47:10.216 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.e 2024-06-10T00:47:10.782 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:47:10 smithi105 systemd[1]: Stopping Ceph mds.e for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:47:10.782 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:47:10 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-e[66232]: 2024-06-10T00:47:10.526+0000 7f8f8d7c5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:47:10.782 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:47:10 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-e[66232]: 2024-06-10T00:47:10.526+0000 7f8f8d7c5700 -1 mds.e *** got signal Terminated *** 2024-06-10T00:47:13.694 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:47:13 smithi105 podman[74488]: 2024-06-10 00:47:13.406637707 +0000 UTC m=+2.971769524 container died 209bf3ad6388f996d1de1666b3b0b1bde4e3a3932582756fab0cbaabe6af47bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528) 2024-06-10T00:47:14.032 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:47:13 smithi105 podman[74488]: 2024-06-10 00:47:13.777039402 +0000 UTC m=+3.342171199 container cleanup 209bf3ad6388f996d1de1666b3b0b1bde4e3a3932582756fab0cbaabe6af47bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:47:14.032 INFO:journalctl@ceph.mds.e.smithi105.stdout:Jun 10 00:47:13 smithi105 bash[74488]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-e 2024-06-10T00:47:14.170 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.e.service' 2024-06-10T00:47:14.202 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:47:14.202 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-10T00:47:14.202 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-10T00:47:14.202 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.h 2024-06-10T00:47:14.532 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:47:14 smithi105 systemd[1]: Stopping Ceph mds.h for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:47:15.032 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:47:14 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-h[66546]: 2024-06-10T00:47:14.537+0000 7f56acf45700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:47:15.032 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:47:14 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-h[66546]: 2024-06-10T00:47:14.537+0000 7f56acf45700 -1 mds.h *** got signal Terminated *** 2024-06-10T00:47:17.664 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:47:17 smithi105 podman[74610]: 2024-06-10 00:47:17.407376618 +0000 UTC m=+2.982712632 container died abed1b6864ba2aa29e7d5b401da20791ada0f97affa12f37addab1418c7ff5fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-10T00:47:18.032 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:47:17 smithi105 podman[74610]: 2024-06-10 00:47:17.764444482 +0000 UTC m=+3.339780488 container cleanup abed1b6864ba2aa29e7d5b401da20791ada0f97affa12f37addab1418c7ff5fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-h, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-10T00:47:18.032 INFO:journalctl@ceph.mds.h.smithi105.stdout:Jun 10 00:47:17 smithi105 bash[74610]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-h 2024-06-10T00:47:18.191 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.h.service' 2024-06-10T00:47:18.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:47:18.221 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-10T00:47:18.222 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-10T00:47:18.222 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.k 2024-06-10T00:47:18.560 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:47:18 smithi105 systemd[1]: Stopping Ceph mds.k for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:47:19.032 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:47:18 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-k[65919]: 2024-06-10T00:47:18.559+0000 7f8dc3ac3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:47:19.032 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:47:18 smithi105 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-k[65919]: 2024-06-10T00:47:18.559+0000 7f8dc3ac3700 -1 mds.k *** got signal Terminated *** 2024-06-10T00:47:21.653 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:47:21 smithi105 podman[74732]: 2024-06-10 00:47:21.397505792 +0000 UTC m=+2.961605493 container died 0ccb9711c5caa2ebec3130a017830d66cb8a3bfca3aecbf482b97977a1c53d5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-10T00:47:22.032 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:47:21 smithi105 podman[74732]: 2024-06-10 00:47:21.752111669 +0000 UTC m=+3.316211372 container cleanup 0ccb9711c5caa2ebec3130a017830d66cb8a3bfca3aecbf482b97977a1c53d5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-k, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:47:22.032 INFO:journalctl@ceph.mds.k.smithi105.stdout:Jun 10 00:47:21 smithi105 bash[74732]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-k 2024-06-10T00:47:22.223 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.k.service' 2024-06-10T00:47:22.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:47:22.253 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-10T00:47:22.253 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-10T00:47:22.253 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.c 2024-06-10T00:47:22.578 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:47:22 smithi138 systemd[1]: Stopping Ceph mds.c for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:47:22.873 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:47:22 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-c[66601]: 2024-06-10T00:47:22.577+0000 7f8c91c0d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:47:22.874 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:47:22 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-c[66601]: 2024-06-10T00:47:22.577+0000 7f8c91c0d700 -1 mds.c *** got signal Terminated *** 2024-06-10T00:47:25.727 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:47:25 smithi138 podman[74444]: 2024-06-10 00:47:25.404659689 +0000 UTC m=+2.955766629 container died c3949f77fe3fbff4bbc66bb34e201f4e15df3d6b8c5c1f03fa7f91fe4eb37036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:47:26.123 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:47:25 smithi138 podman[74444]: 2024-06-10 00:47:25.72802937 +0000 UTC m=+3.279136310 container cleanup c3949f77fe3fbff4bbc66bb34e201f4e15df3d6b8c5c1f03fa7f91fe4eb37036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-10T00:47:26.124 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:47:25 smithi138 bash[74444]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-c 2024-06-10T00:47:26.124 INFO:journalctl@ceph.mds.c.smithi138.stdout:Jun 10 00:47:25 smithi138 podman[74457]: 2024-06-10 00:47:25.929478455 +0000 UTC m=+0.526725912 container remove c3949f77fe3fbff4bbc66bb34e201f4e15df3d6b8c5c1f03fa7f91fe4eb37036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:47:26.169 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.c.service' 2024-06-10T00:47:26.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:47:26.202 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-10T00:47:26.202 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-10T00:47:26.202 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.f 2024-06-10T00:47:26.548 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:47:26 smithi138 systemd[1]: Stopping Ceph mds.f for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:47:26.873 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:47:26 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-f[66287]: 2024-06-10T00:47:26.548+0000 7f7c9f14b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:47:26.874 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:47:26 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-f[66287]: 2024-06-10T00:47:26.548+0000 7f7c9f14b700 -1 mds.f *** got signal Terminated *** 2024-06-10T00:47:29.683 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:47:29 smithi138 podman[74566]: 2024-06-10 00:47:29.407476367 +0000 UTC m=+2.986152949 container died 41fcb2b3c9be39ecf2ab4f486a0d0ba2e3430b8cb056e5d8cb6c9a6e56789686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:47:30.033 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:47:29 smithi138 podman[74566]: 2024-06-10 00:47:29.757736661 +0000 UTC m=+3.336413246 container cleanup 41fcb2b3c9be39ecf2ab4f486a0d0ba2e3430b8cb056e5d8cb6c9a6e56789686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-06-10T00:47:30.034 INFO:journalctl@ceph.mds.f.smithi138.stdout:Jun 10 00:47:29 smithi138 bash[74566]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-f 2024-06-10T00:47:30.159 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.f.service' 2024-06-10T00:47:30.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:47:30.193 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-10T00:47:30.193 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-10T00:47:30.193 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.i 2024-06-10T00:47:30.373 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:47:30 smithi138 systemd[1]: Stopping Ceph mds.i for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:47:30.873 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:47:30 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-i[66903]: 2024-06-10T00:47:30.502+0000 7fa3fa60b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:47:30.874 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:47:30 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-i[66903]: 2024-06-10T00:47:30.502+0000 7fa3fa60b700 -1 mds.i *** got signal Terminated *** 2024-06-10T00:47:33.684 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:47:33 smithi138 podman[74686]: 2024-06-10 00:47:33.413416796 +0000 UTC m=+3.006759644 container died 13a4245f0167e1ef0ce5ab69fab8d64d6ab33954364b0ea0a59360fdc0300855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True) 2024-06-10T00:47:33.955 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:47:33 smithi138 podman[74686]: 2024-06-10 00:47:33.737218398 +0000 UTC m=+3.330561253 container cleanup 13a4245f0167e1ef0ce5ab69fab8d64d6ab33954364b0ea0a59360fdc0300855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-i, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-10T00:47:33.955 INFO:journalctl@ceph.mds.i.smithi138.stdout:Jun 10 00:47:33 smithi138 bash[74686]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-i 2024-06-10T00:47:34.169 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.i.service' 2024-06-10T00:47:34.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:47:34.221 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-10T00:47:34.221 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-10T00:47:34.222 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.l 2024-06-10T00:47:34.525 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:47:34 smithi138 systemd[1]: Stopping Ceph mds.l for 1353b2ce-26bf-11ef-bc9b-c7b262605968... 2024-06-10T00:47:34.873 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:47:34 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-l[65983]: 2024-06-10T00:47:34.524+0000 7f83197aa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:47:34.874 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:47:34 smithi138 ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-l[65983]: 2024-06-10T00:47:34.524+0000 7f83197aa700 -1 mds.l *** got signal Terminated *** 2024-06-10T00:47:37.684 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:47:37 smithi138 podman[74806]: 2024-06-10 00:47:37.405435607 +0000 UTC m=+2.991625926 container died f5decf1d16c409b22fd658e40889be176651fb43af3e5720657a5c0da4d93b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-l, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-10T00:47:38.026 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:47:37 smithi138 podman[74806]: 2024-06-10 00:47:37.758687871 +0000 UTC m=+3.344878236 container cleanup f5decf1d16c409b22fd658e40889be176651fb43af3e5720657a5c0da4d93b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-l, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-10T00:47:38.026 INFO:journalctl@ceph.mds.l.smithi138.stdout:Jun 10 00:47:37 smithi138 bash[74806]: ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968-mds-l 2024-06-10T00:47:38.187 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1353b2ce-26bf-11ef-bc9b-c7b262605968@mds.l.service' 2024-06-10T00:47:38.217 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T00:47:38.218 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-10T00:47:38.218 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-10T00:47:38.423 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:47:40.923 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-10T00:47:41.111 INFO:teuthology.orchestra.run.smithi105.stdout:Deleting cluster with fsid: 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:47:43.553 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-10T00:47:43.739 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:47:46.060 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:47:46.096 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:47:46.127 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:47:46.153 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-10T00:47:46.156 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323/remote/smithi062/crash 2024-06-10T00:47:46.157 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/crash -- . 2024-06-10T00:47:46.194 INFO:teuthology.orchestra.run.smithi062.stderr:tar: /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-10T00:47:46.195 INFO:teuthology.orchestra.run.smithi062.stderr:tar: Error is not recoverable: exiting now 2024-06-10T00:47:46.197 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323/remote/smithi105/crash 2024-06-10T00:47:46.198 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/crash -- . 2024-06-10T00:47:46.281 INFO:teuthology.orchestra.run.smithi105.stderr:tar: /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-10T00:47:46.282 INFO:teuthology.orchestra.run.smithi105.stderr:tar: Error is not recoverable: exiting now 2024-06-10T00:47:46.283 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323/remote/smithi138/crash 2024-06-10T00:47:46.284 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/crash -- . 2024-06-10T00:47:46.318 INFO:teuthology.orchestra.run.smithi138.stderr:tar: /var/lib/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-10T00:47:46.318 INFO:teuthology.orchestra.run.smithi138.stderr:tar: Error is not recoverable: exiting now 2024-06-10T00:47:46.319 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-10T00:47:46.319 DEBUG:teuthology.orchestra.run.smithi062:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-10T00:47:46.416 INFO:tasks.cephadm:Compressing logs... 2024-06-10T00:47:46.416 DEBUG:teuthology.orchestra.run.smithi062:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-10T00:47:46.419 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-10T00:47:46.422 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-10T00:47:46.449 INFO:teuthology.orchestra.run.smithi062.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-06-10T00:47:46.450 INFO:teuthology.orchestra.run.smithi062.stderr: /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.log 2024-06-10T00:47:46.450 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-10T00:47:46.450 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-06-10T00:47:46.450 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-06-10T00:47:46.450 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-10T00:47:46.450 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-10T00:47:46.451 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-10T00:47:46.451 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-10T00:47:46.451 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.0.log: /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-10T00:47:46.451 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-10T00:47:46.451 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.6.log: 87.4% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-10T00:47:46.451 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-06-10T00:47:46.452 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-10T00:47:46.452 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-06-10T00:47:46.452 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-06-10T00:47:46.452 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-10T00:47:46.452 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-06-10T00:47:46.452 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/cephadm.log: 94.8% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-06-10T00:47:46.454 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-06-10T00:47:46.454 INFO:teuthology.orchestra.run.smithi062.stderr: 94.6% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-06-10T00:47:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-06-10T00:47:46.465 INFO:teuthology.orchestra.run.smithi105.stderr: /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.log 2024-06-10T00:47:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-10T00:47:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-10T00:47:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-06-10T00:47:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-10T00:47:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-10T00:47:46.466 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-10T00:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-06-10T00:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-06-10T00:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-10T00:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.h.log: 90.4% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-10T00:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-10T00:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-06-10T00:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-10T00:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-10T00:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.e.log: 92.2% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-10T00:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-10T00:47:46.471 INFO:teuthology.orchestra.run.smithi138.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.log 2024-06-10T00:47:46.472 INFO:teuthology.orchestra.run.smithi138.stderr:: No such file or directory 2024-06-10T00:47:46.472 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-10T00:47:46.472 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-06-10T00:47:46.472 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-10T00:47:46.472 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-10T00:47:46.472 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-06-10T00:47:46.472 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.audit.log: /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-10T00:47:46.473 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-06-10T00:47:46.473 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-06-10T00:47:46.473 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mgr.z.log: 87.4% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-10T00:47:46.473 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-10T00:47:46.473 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-10T00:47:46.473 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-06-10T00:47:46.473 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-10T00:47:46.473 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-06-10T00:47:46.474 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.11.log: 92.2% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-06-10T00:47:46.474 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-10T00:47:46.474 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-06-10T00:47:46.475 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-10T00:47:46.476 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-06-10T00:47:46.479 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-10T00:47:46.485 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-10T00:47:46.500 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-10T00:47:46.501 INFO:teuthology.orchestra.run.smithi062.stderr: 94.4% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-06-10T00:47:46.501 INFO:teuthology.orchestra.run.smithi138.stderr: 91.1% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-10T00:47:46.505 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-10T00:47:46.510 INFO:teuthology.orchestra.run.smithi062.stderr: 84.1% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-10T00:47:46.614 INFO:teuthology.orchestra.run.smithi062.stderr: 94.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-10T00:47:46.651 INFO:teuthology.orchestra.run.smithi105.stderr: 94.4% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-06-10T00:47:46.675 INFO:teuthology.orchestra.run.smithi105.stderr: 93.8% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-06-10T00:47:46.679 INFO:teuthology.orchestra.run.smithi062.stderr: 94.3% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-06-10T00:47:46.683 INFO:teuthology.orchestra.run.smithi062.stderr: 91.1% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-10T00:47:46.683 INFO:teuthology.orchestra.run.smithi062.stderr: 90.2% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-10T00:47:46.707 INFO:teuthology.orchestra.run.smithi138.stderr: 94.3% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-06-10T00:47:46.715 INFO:teuthology.orchestra.run.smithi138.stderr: 94.0% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-06-10T00:47:46.718 INFO:teuthology.orchestra.run.smithi138.stderr: 94.1% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-06-10T00:47:46.843 INFO:teuthology.orchestra.run.smithi138.stderr: 91.7% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-10T00:47:46.846 INFO:teuthology.orchestra.run.smithi138.stderr: 94.5% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-06-10T00:47:46.867 INFO:teuthology.orchestra.run.smithi062.stderr: 90.2% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-10T00:47:46.928 INFO:teuthology.orchestra.run.smithi105.stderr: 91.8% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-10T00:47:47.032 INFO:teuthology.orchestra.run.smithi105.stderr: 95.7% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-06-10T00:47:51.135 INFO:teuthology.orchestra.run.smithi105.stderr: 94.4% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-06-10T00:47:52.612 INFO:teuthology.orchestra.run.smithi062.stderr: 90.4% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-10T00:48:16.225 INFO:teuthology.orchestra.run.smithi105.stderr: 93.1% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-06-10T00:48:21.203 INFO:teuthology.orchestra.run.smithi138.stderr: 92.7% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-06-10T00:48:25.718 INFO:teuthology.orchestra.run.smithi105.stderr: 92.7% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-10T00:48:34.808 INFO:teuthology.orchestra.run.smithi138.stderr: 92.5% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-06-10T00:48:36.401 INFO:teuthology.orchestra.run.smithi138.stderr: 92.3% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-10T00:48:40.269 INFO:teuthology.orchestra.run.smithi105.stderr: 92.7% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-10T00:48:49.411 INFO:teuthology.orchestra.run.smithi062.stderr: 92.0% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-10T00:48:50.337 INFO:teuthology.orchestra.run.smithi105.stderr: 92.0% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-10T00:48:50.340 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-06-10T00:48:50.340 INFO:teuthology.orchestra.run.smithi105.stderr:real 1m3.903s 2024-06-10T00:48:50.340 INFO:teuthology.orchestra.run.smithi105.stderr:user 3m6.662s 2024-06-10T00:48:50.340 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m5.580s 2024-06-10T00:48:57.223 INFO:teuthology.orchestra.run.smithi062.stderr: 92.5% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-10T00:48:57.801 INFO:teuthology.orchestra.run.smithi138.stderr: 92.1% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-10T00:48:57.803 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-06-10T00:48:57.803 INFO:teuthology.orchestra.run.smithi138.stderr:real 1m11.367s 2024-06-10T00:48:57.803 INFO:teuthology.orchestra.run.smithi138.stderr:user 3m19.050s 2024-06-10T00:48:57.804 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m6.109s 2024-06-10T00:49:04.897 INFO:teuthology.orchestra.run.smithi062.stderr: 92.4% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-10T00:49:22.936 INFO:teuthology.orchestra.run.smithi062.stderr: 91.6% -- replaced with /var/log/ceph/1353b2ce-26bf-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-06-10T00:49:22.938 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-06-10T00:49:22.938 INFO:teuthology.orchestra.run.smithi062.stderr:real 1m36.505s 2024-06-10T00:49:22.938 INFO:teuthology.orchestra.run.smithi062.stderr:user 4m4.268s 2024-06-10T00:49:22.938 INFO:teuthology.orchestra.run.smithi062.stderr:sys 0m6.829s 2024-06-10T00:49:22.939 INFO:tasks.cephadm:Archiving logs... 2024-06-10T00:49:22.940 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323/remote/smithi062/log 2024-06-10T00:49:22.941 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-10T00:50:00.588 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323/remote/smithi105/log 2024-06-10T00:50:00.589 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-10T00:50:28.708 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323/remote/smithi138/log 2024-06-10T00:50:28.709 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-10T00:50:59.488 INFO:tasks.cephadm:Removing cluster... 2024-06-10T00:50:59.489 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 --force 2024-06-10T00:50:59.696 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:51:00.233 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 --force 2024-06-10T00:51:00.423 INFO:teuthology.orchestra.run.smithi105.stdout:Deleting cluster with fsid: 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:51:00.911 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1353b2ce-26bf-11ef-bc9b-c7b262605968 --force 2024-06-10T00:51:01.103 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: 1353b2ce-26bf-11ef-bc9b-c7b262605968 2024-06-10T00:51:01.622 INFO:tasks.cephadm:Removing cephadm ... 2024-06-10T00:51:01.623 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-10T00:51:01.643 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-10T00:51:01.661 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-10T00:51:01.677 INFO:tasks.cephadm:Teardown complete 2024-06-10T00:51:01.677 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-10T00:51:01.692 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-10T00:51:01.692 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-10T00:51:01.695 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-10T00:51:01.704 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-10T00:51:01.812 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-10T00:51:01.822 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-10T00:51:01.822 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T00:51:01.826 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T00:51:01.828 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T00:51:01.841 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-06-10T00:51:01.842 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-06-10T00:51:01.842 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-06-10T00:51:01.846 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T00:51:01.846 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-06-10T00:51:01.846 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 -168us[ -167us] +/- 52ms 2024-06-10T00:51:01.846 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -1015us[-1013us] +/- 75ms 2024-06-10T00:51:01.846 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 +1165us[+1165us] +/- 71ms 2024-06-10T00:51:01.846 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:51:01.848 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T00:51:01.848 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-06-10T00:51:01.848 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 -197us[ -195us] +/- 52ms 2024-06-10T00:51:01.848 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -1051us[-1051us] +/- 75ms 2024-06-10T00:51:01.848 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +1154us[+1154us] +/- 71ms 2024-06-10T00:51:01.848 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:51:01.866 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T00:51:01.866 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-06-10T00:51:01.866 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 -175us[ -170us] +/- 52ms 2024-06-10T00:51:01.866 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 -1011us[-1006us] +/- 75ms 2024-06-10T00:51:01.866 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 +1163us[+1168us] +/- 71ms 2024-06-10T00:51:01.866 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:51:01.867 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-10T00:51:01.877 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-10T00:51:01.878 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-10T00:51:01.888 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-10T00:51:02.043 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-10T00:51:02.111 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-10T00:51:02.176 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T00:51:02.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:51:02.209 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T00:51:02.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:51:02.237 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T00:51:02.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:51:02.264 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-10T00:51:02.275 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-10T00:51:02.285 INFO:teuthology.task.internal:Duration was 2312.915852 seconds 2024-06-10T00:51:02.286 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-10T00:51:02.297 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-10T00:51:02.297 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-10T00:51:02.300 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-10T00:51:02.303 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-10T00:51:02.338 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T00:51:02.340 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T00:51:02.369 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T00:51:02.827 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-10T00:51:02.828 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2024-06-10T00:51:02.828 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-10T00:51:02.854 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-06-10T00:51:02.854 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-10T00:51:02.892 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-06-10T00:51:02.892 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-10T00:51:02.921 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-10T00:51:02.921 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-10T00:51:02.924 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-10T00:51:02.935 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-10T00:51:02.989 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-10T00:51:02.990 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-10T00:51:02.997 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-10T00:51:03.004 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-10T00:51:04.137 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-10T00:51:04.150 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-10T00:51:04.150 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-10T00:51:04.183 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-10T00:51:04.211 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-10T00:51:04.238 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-10T00:51:04.251 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-10T00:51:04.253 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-10T00:51:04.256 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-10T00:51:04.275 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2024-06-10T00:51:04.278 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-06-10T00:51:04.303 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-06-10T00:51:04.630 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-10T00:51:04.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:51:04.647 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-10T00:51:04.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:51:04.664 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-10T00:51:04.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:51:04.686 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-10T00:51:04.699 INFO:teuthology.task.internal:Transferring archived files... 2024-06-10T00:51:04.700 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323/remote/smithi062 2024-06-10T00:51:04.701 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-10T00:51:04.764 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323/remote/smithi105 2024-06-10T00:51:04.764 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-10T00:51:04.803 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747323/remote/smithi138 2024-06-10T00:51:04.803 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-10T00:51:04.845 INFO:teuthology.task.internal:Removing archive directory... 2024-06-10T00:51:04.846 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-10T00:51:04.849 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-10T00:51:04.852 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-10T00:51:04.897 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-10T00:51:04.911 INFO:teuthology.task.internal:Not uploading archives. 2024-06-10T00:51:04.911 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-10T00:51:04.939 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-10T00:51:04.940 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-10T00:51:04.942 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-10T00:51:04.945 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-10T00:51:04.955 INFO:teuthology.orchestra.run.smithi062.stdout: 265304 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 10 00:51 /home/ubuntu/cephtest 2024-06-10T00:51:04.959 INFO:teuthology.orchestra.run.smithi105.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 10 00:51 /home/ubuntu/cephtest 2024-06-10T00:51:04.961 INFO:teuthology.orchestra.run.smithi138.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 10 00:51 /home/ubuntu/cephtest 2024-06-10T00:51:04.962 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-10T00:51:04.974 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-10T00:51:05.005 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} duration: 2312.915851831436 failure_reason: 'Command failed (workunit test suites/ffsb.sh) on smithi062 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=5c9934f8908f4f08b3227e1b77e29b59 status: fail success: false 2024-06-10T00:51:05.005 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-10T00:51:05.097 INFO:teuthology.run:FAIL